Evaluating Interfaces with Users Why Bother? - GroupLab
study relations by manipulating one or more independent variables. - experimenter controls all environmental factors. ⢠observe effect on one or more dependent ...
Why evaluation is crucial to interface design General approaches and tradeoffs in evaluation The role of ethics
Saul Greenberg
design
Why Bother? Tied to the usability engineering lifecycle
evaluation
implementation
• Pre-design - investing in new expensive system requires proof of viability
• Initial design stages - develop and evaluate initial design ideas with the user
• Iterative design - does system behaviour match the user’s task requirements? - are there specific problems with the design? - can users provide feedback to modify design?
• Acceptance testing - verify that human/computer system meets expected performance criteria ease of learning, usability, user’s attitude, performance criteria e.g., a first time user will take 1-3 minutes to learn how to withdraw $50. from the automatic teller Saul Greenberg
Evaluation Introduction
1
Approaches: Naturalistic Naturalistic: • describes an ongoing process as it evolves over time • observation occurs in realistic setting - ecologically valid
• “real life” External validity • degree to which research results applies to real situations
Saul Greenberg
Approaches: Experimental Experimental • study relations by manipulating one or more independent variables - experimenter controls all environmental factors
• observe effect on one or more dependent variables Internal validity • confidence that we have in our explanation of experimental results
Trade-off: Natural vs Experimental • precision and direct control over experimental design versus • desire for maximum generalizability in real life situations
Saul Greenberg
Evaluation Introduction
2
Approaches: Reliability Concerns Would the same results be achieved if the test were repeated? Problem: individual differences: • best user 10x faster than slowest • best 25% of users ~2x faster than slowest 25% Partial Solution • reasonable number and range of users tested • statistics provide confidence intervals of test results - 95% confident that mean time to perform task X is 4.5+/-0.2 minutes means 95% chance true mean is between 4.3 and 4.7, 5% chance its outside that
Saul Greenberg
Approaches: Validity Concerns Does the test measure something of relevance to usability of real products in real use outside of lab? • Some typical reliability problems of testing vs real use - non-typical users tested - tasks are not typical tasks - physical environment different quiet lab vs very noisy open offices vs interruptions - social influences different motivation towards experimenter vs motivation towards boss
Partial Solution • use real users • tasks from task-centered system design • environment similar to real situation
Saul Greenberg
Evaluation Introduction
3
Ethics ...and to think that
you want me to test it!!!
Saul Greenberg
Ethics Testing can be a distressing experience • pressure to perform, errors inevitable • feelings of inadequacy • competition with other subjects
Golden rule • subjects should always be treated with respect
Saul Greenberg
Evaluation Introduction
4
Managing subjects in an ethical manner Before the test • don’t waste the user’s time - use pilot tests to debug experiments, questionnaires etc - have everything ready before the user shows up
• make users feel comfortable - emphasize that it is the system that is being tested, not the user - acknowledge that the software may have problems - let users know they can stop at any time
• maintain privacy - tell user that individual test results will be kept completely confidential
• inform the user - explain any monitoring that is being used - answer all user’s questions (but avoid bias)
• only use volunteers - user must sign an informed consent form Saul Greenberg
Managing subjects in an ethical manner During the test • don’t waste the user’s time - never have the user perform unnecessary tasks
• make users comfortable -
try to give user an early success experience keep a relaxed atmosphere in the room coffee, breaks, etc hand out test tasks one at a time never indicate displeasure with the user’s performance avoid disruptions stop the test if it becomes too unpleasant
• maintain privacy - do not allow the user’s management to observe the test
Saul Greenberg
Evaluation Introduction
5
Managing subjects in an ethical manner After the test • make the users feel comfortable - state that the user has helped you find areas of improvement
• inform the user - answer particular questions about the experiment that could have biased the results before
• maintain privacy - never report results in a way that individual users can be identified - only show videotapes outside the research group with the user’s permission
Saul Greenberg
You know now Evaluation is crucial for designing, debugging, and verifying interfaces There is a tradeoff in naturalistic vs experimental approaches • internal and external validity • reliability • precision • generalizability Subjects must be treated with respect • ethical rules of behaviour
2).1 Instead of Marx's theory of surplus value, 'Sraffians explain profits as ... belief that capitalist profits derive only from the amount of labour they hire, when in ...
replicated in the case of EEG simulations ~Lachaux et al., 1997! .... Statistical Physics, 65, 579â616. ... most of the time from embedding with self-intersections.
This paper briefly reviews social psychological research and theory related to ..... harming others (Milgram, 1974) are not driven by ...... In Obedience to authority.
robustness and speed. 5l tt. / i t. ~ 5 letters /minute. Current improvements : - Adaptive classification to optimize p p bit-rate. - Word selection instead of spelling l tt.
Oct 10, 2014 - d. this perspective is in line with minimalist and biolinguistic tenets: grammar-internal ... are simply ignored since they cannot be parsed. d. ==> every ..... tional instructions cannot be its output. By contrast ..... c. post-phonol
where their complex properties continue to puzzle the researchers' minds. A major challenge in foam science is posed by the need to protect the tightly packed ...
the field of e-commerce or security) and multi-agent systems. ... Let us first recall some elements about probability intervals, before studying how .... of f, EP (f) and EP (f) as ... Therefore, Equations (5) and (6) reduce to sums of lower and uppe
shallow clouds whereas deeper and long-lasting systems are much better ... They also point out model weaknesses that may affect regional climate ... diation via water vapor transports and cloud cover as well ... FIG. 3. Time series of observed and si
N2 fluxes from 11 different ocean carbon cycle models (Table 1), all of which .... CaCO3 pump acts as an APO sink in the southern high latitudes. (it decreases ...
Social psychology, 59, 705â718. Dehaene, S. (1997). In The number sense. Oxford .... Sloman, S. A. (1996). The empirical case for two systems of chology, 49 ...
Complete list of supported vehicles/systems in FiatECUScan. 3.2 .... You can scroll down to see the complete wiring diagrams of ...... GRANDE PUNTO VAN.
It has been placed in the background layer of the area, so you cannot directly click on the ..... curves, 4D Draw converts them to polygon line segments. (This is ...
Global sea-air CO2 flux based on climatological surface ocean pCO2 and seasonal biological and temperature effects. Deep-Sea Research II 49, 1601â1622.
2 A power transformer. I feed the output to a bridge ... and provide some isolation. U2 is a 5-V powered ... teristic makes it easy to fully isolate the radio and the ...
sion Test,â December 2018). Ideally this is performed on a warm engine, but even on a cold one it can reveal some problems. What you as a buyer are look-.
Im(log(1 â eit)). = arg(1 â eit). =t â Ï. 2 . By the fundamental theorem of calculus we have ... Proof 8: We use the calculus of residues. Let f(z) = Ïzâ2 cotÏz. Then f.
Evaluating ζ(2). Robin Chapman. Department of Mathematics. University of Exeter, Exeter, EX4 4QE, UK [email protected]. 30 April 1999 (corrected 7 July ...
It is easy to calculate and provide a quick answer to the question: How many ..... Solutions. 1. Payback. Evaluating the project with unequal cash flow using ...... this is a not unexpected situation given its historical popularity, its simplicity ..
differences between r and p, or values close to zero. The f-measure ...... the second LPG, and LPG.eq is a fragment of LPG with as many REs as VA (cf. Table 5).
Jun 19, 2005 - . I work for EADS Corporate Research Center, IT Security dpt. If you happen to need a A380 or Tigre helicopter, see me after my talk :) ..... station despites AP restrictions. 9http://www.python.org/.
