Christophe Jacquet

Human-Computer Interaction Interaction Homme-Machine

EFREI, 2009-2010

Table of Contents

Introduction to Human-Computer Interaction ...............................................1-1 Graphical interfaces........................................................................................2-1 Architecture models for HCI..........................................................................3-1 Human factors ................................................................................................4-1 Web interfaces................................................................................................5-1 Struts: MVC for the web ................................................................................6-1 Design & evaluation of user interfaces ..........................................................7-1 Major interface design concepts.....................................................................8-1 Post-WIMP interfaces ....................................................................................9-1

Glossary.........................................................................................................A-1

C. Jacquet — Human-Computer Interaction

EFREI SI 2009-2010

Introduction to HCI

Human-Computer Interaction

CHAPTER

1

Introduction to Human-Computer Interaction

Course overview 1. 2. 3. 4. 5. 6. 7. 8. 9.

Introduction to Human-Computer Interaction Graphical Interfaces Architecture Models for HCI Human factors Web interfaces Struts: MVC for the web Design & evaluation of user interfaces Major interface design concepts Post-WIMP interfaces

2

Chapter outline Introduction to Human-Computer Interaction 1.1 What is HCI 1.2 A Short History of HCI 1.3 Classical Input Devices 1.4 Classical Output Devices 1.5 Advanced I/O Devices

1-1

C. Jacquet — Human-Computer Interaction

Introduction to HCI

1.1 What is HCI? HCI – Human-Computer Interaction IHM – Interaction Homme-Machine

4

Interactive system • Interactive system – System whose behavior depends on an external environment that it does not control

→ Open systems ≠ Closed system (autonomous, embedded systems)

5

HCI: human-computer inter… • Interaction: all physical and cognitive phenomena that occur when someone is performing tasks with a computer • Interface: a setup of hardware and software components that enable a computer to be used to perform some tasks • The interface does not necessarily make a usable system! 6

1-2

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Usability • Is the system useful to perform a task? • Account for: – The interface – The sequence of actions necessary to perform the task – Consistency with context of use

7

Why study HCI?

IBM 7030, 1961

iMac, 2004

Interfaces have made headway… 8

… but still need improvement

9

1-3

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Significance of HCI • Bad interface → unusable software – Even if functionally wonderful…

• HCI in software development – 1/3 project meetings – 48% (software) to 80% (web) of the code

• Financial point of view – Call center: new HCI → gained 600ms/call, saved $3 million / year – Solved 20 out of 75 usability problems in a database → turnover +66% 10

HCI topics • Design methodology – User-centered – Participative design

Social sciences

• Know the human being – Perception psychology – Social sciences

Life sciences

• Tools – Toolkits, languages – Interaction engineering

Engineering

• Design – Art, typography

Design

11

1.2 A Short History of HCI

1-4

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Early agenda • 1945 : Vannevar Bush, As We May Think • Memex (Memory Extender) • Early idea of a hypertext system: – Library with easy access (mechanical device) – Cross-references (links) – Trails

13

Memex • Imagined interface: Memex Desk – Head-mounted camera – Typewriters – Voice recognition, – Speech synthesis – Touch screens

14

First interfaces

Terminal = Console

ENIAC First computer ever (1946) Teletype (TTY) Model 33 (1961)

DEC VT-100 (1978)

15

1-5

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Dialogic interfaces • Unix (1969), VMS (1975), CP/M (1977), PC-DOS (1980) • Still in use today • Teletypes and then consoles (=terminals) (screen+keyboard) • Commands (verb+complement) • Answers

login: jacquet Password: Compaq Computer Corporation Tru64 UNIX jacquet@gonzales: ~ % pwd /urisc/si/jacquet jacquet@gonzales: ~ % cd /etc jacquet@gonzales: /etc % ls *.conf syslog.conf imapd.conf resolv.conf inetd.conf jacquet@gonzales: /etc %

16

Early personal computers Apple II (1977)

IBM PC-AT (1984)

French Minitel (1982) MS-DOS 3.3 (1987) 17

NLS: oN-Line System • Stanford, Douglas Engelbart, 1968 • Modern features: – Mouse – Raster-scan monitors – Hypertext – Groupware

• But difficult to learn

18

1-6

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Xerox Palo Alto Research Center • Xerox: copier business • “PARC”, 1970 • Funding • Significant researchers: Alan Kay, Butler Lampson, Mark Weiser • Many inventions • Somewhat failed to market them 19

PARC Outputs • • • •

Laser printer Graphical user interface Ethernet networking WYSIWYG: What You See Is What You Get

• Object-oriented programming (Smalltalk)

20

Xerox Alto • Bitmapped display • Mouse

1973

• Ancestor of personal computers and workstations • Internal use at Xerox + a few universities

21

1-7

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Xerox Alto GUI • Desinged for blackand-white bitmapped screens • Had most components of today’s GUIs: – Buttons – Lists of items

• Still crude though 22

Xerox Star 8010 • Commercial, 1981 • Very innovative: – – – – – – –

Mouse Bitmapped display GUI Icons, folders Ethernet File/print servers E-mail

23

Xerox Star interface

24

1-8

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Apple Lisa (1983) • Based on PARC’s concepts (Star) • Pull-down menus • Commercial failure

25

Apple Macintosh (1984) • Same concepts as Lisa • Cheaper ($2495 vs $9995) • Fewer bugs • Self-contained • Hi-res B&W display

26

Macintosh Interface

27

1-9

C. Jacquet — Human-Computer Interaction

Introduction to HCI

1985: Windows 1.0

Tiled windows (no overlap) 28

1990: Windows 3

Since then, no revolution in UI concepts on desktops

29

1.3 Classical Input Devices

1-10

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Input devices • Many devices available • Combination of several devices: complementarity – Example: mouse + keyboard

• Any input must produce a feedback – Physical keyboard, virtual keyboard – Mouse cursor

Laser keyboard for Palm… « complete with realistic tapping sounds »

Input devices: typology • Input type: – Discrete: keyboard – Continuous: mouse, optical pen, joystick

• Localization: – Direct: optical pen, touch screen – Indirect: mouse, trackball, joystick, graphics tablet – Absolute: touch screen, graphics tablet – Relative: mouse, trackball, joystick

Keyboards: 1- typewriter-style • Types of keys: – Rubber keyboards • Trendy at the beginning of the 1980’s • No longer used on desktop computers

– Mechanical keyboards

1-11

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Keyboards: 2- chorded keyboards • One-hand → several keys at a time • Transcribe talks (e.g. parliaments…) • Users with disabilities (e.g. blind people)

Speech recognition • Appealing: hands-free, natural • But error-prone: – Differences of accents across people – Natural language is ambiguous

• Uses: – Type text, enter commands – Not: pointing

• Application domains: – Telephones – Specific domains: ticket booking, medical reports

Pointing devices • Continuous devices – A discrete (software) grid can ease their use

• Localization: – Direct: optical pen, touch screen – Indirect: mouse, trackball, joystick, graphics tablet – Absolute: touch screen, graphics tablet – Relative: mouse, trackball, joystick

1-12

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Mouse • Invented by Douglas Engelbart (1963) • Indirect & relative • Buttons: 1, 2 & more • Scroll wheel(s)

Mouse: motion capture • Wheels (Engelbart) • Ball and wheels • Optical detection – CCD sensor – Specialized processor – Surface lit by red LEDs – Recently: laser light

Mechanism of a ball mouse

Wheels Ball Optical

1960

1970

1980

1990

2000

Mouse equivalents Indirect & relative • Trackball: – Inverted ball mouse – Used in laptops in the 1990s – Some standalone models

• Touchpad: – Capacity detection – Used in today’s laptops

1-13

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Graphics tablet • Indirect/absolute • Stylus • Good precision – Line art

• Detects: – – – –

Position Pressure Inclination Two stylus

• Drawback – Distant feedback

Touch screen • Direct & absolute • Pointing with: – Fingers – Stylus

• Applications: – Public kiosks e.g.: SNCF vending machines – PDAs – Tablet PCs

Light pen • Direct & absolute • Detection of the beam of cathodic ray tube (CRT) screens • Hype at the beginning of the 1980’s: – Thomson MO5, TO7 – BBC Micro

• Fell out of use

1-14

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Direct pointing devices touch screens, light pens • Benefits: – Very direct interaction – Very easy to learn

• Drawbacks: – Generally imprecise – Tiresome (hands and arms) – One hides the screen while pointing

Bimanual Interaction • 2 hands: primary → precise task secondary → simpler task • Example (Engelbart): Mouse (continuous) Chorded keyboard (discrete)

• In use today (Windows, MacOS…) – Chorded keyboard → special keys on the main keyboard (shift, alt, control)

Handwriting recognition • Touch screen + stylus: – Pointing – Writing

• Used on PDAs and Tablets

• Natural…?

1-15

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Handwriting recognition • Many errors → troublesome • Special alphabets + less error-prone − less natural; one has to learn an alphabet!

Xerox Unistrokes (1995)

Palm Graffiti 1 (1997)

Handwriting recognition • Newer systems: more natural

Palm Graffiti 2 (2003) ≈ Microsoft Block Recognizer

1.4 Classical Output Devices

1-16

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Graphics… Vector

Raster (bitmap)



Shape outlines

Colors of dots (=pixels)

Vector graphics • Shape outlines + stroke + fill • Suited for: – fonts – technical/geometric drawings – line art

• Lightweight • Arbitrary size • Postscript, SVG

Example (Postscript)

4 setlinewidth newpath 270.4799 156.37714 moveto 178.16158 406.71477 lineto 363.13514 406.71477 lineto 270.4799 156.37714 lineto 232.07009 89.328441 moveto 309.22664 89.328441 lineto 500.93876 592.36218 lineto 430.18385 592.36218 lineto 384.36162 463.3187 lineto 157.60897 463.3187 lineto 111.78674 592.36218 lineto 40.021039 592.36218 lineto 232.07009 89.328441 lineto stroke

Raster graphics • Color of each dot • Suited for:

Example

– Photographs – Any graphics at a given resolution

• Not scalable • JPEG, PNG, etc.

R 52%, G 47%, B 29% R 51%, G 44%, B 27% R 50%, G 42%, B 26% R 49%, G 41%, B 25% R 52%, G 47%, B 29% R 46%, G 36%, B 22% R 42%, G 30%, B 18% R 37%, G 26%, B 14% … R 53%, G 62%, B 56% … R 20%, G 29%, B 27% R 26%, G 35%, B 33%

1-17

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Screen • Vector-based monitors = X-Y displays (1950-1980) – Oscilloscope-like Cathodic Ray Tube (CRT) – Radars – Arcade games

• Bitmapped displays = raster displays – CRT: 1960-now – Liquid Crystal Displays (LCD): 2000-…

Raster display: typology • Resolution: pixels per inch [1 inch = 2.54 cm] – Physical size [inches] – Pixel size [pixels]

• Aspect ratio: width/height (4/3, 16/9…) • Color depth: bits per pixel – Number of colors: 2color depth

• Memory size: – (number of pixels) × (color depth)

Color on raster displays • Eye: red, green, blue cones to detect all colors • Same idea to synthesize colors: red, green, blue light-emitting dots →

→

→

→

→

→

→

→

R/G/B: primary colors

C/M/Y: secondary colors

1-18

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Color coding • Monochrome: 1 bit per pixel • True color: – Each component (R/G/B) coded by n bits – Example: 24bpp → 8 bits R, 8 bits G, 8 bits B

• Indexed

… Color: m bpp

Memory n bpp

… Color table: 2n entries

Display technologies • Cathode Ray Tube (CRT) – used the deflection of an electron beam that hit a surface covered with phosphorus dots – now phased-out 1 pixel

• “Flat screens”: – Liquid Crystal Display (LCD) – Organic Light-Emitting Diodes (OLED)

1.5 Advanced I/O Devices

1-19

C. Jacquet — Human-Computer Interaction

Introduction to HCI

3D gloves • Grasp objects in virtual reality • Still limited to research experiments

Head-mounted displays • LCD, CRT • Stereoscopic • Head movement tracker → look around • Virtual reality • Augmented reality

CAVE • Cave Automatic Virtual Environment • Setup – People in a large cubic room – Projectors → 4 to 6 faces – Stereoscopic shutter glasses – Fast computers to drive all this

• Applications – CAD – DNA exploration (Orsay)

1-20

C. Jacquet — Human-Computer Interaction

Introduction to HCI

Devices for handicapped people • • • •

Eye tracker Head movement tracker Braille terminal: keyboard + tactile display Mouse with tactile/Braille feedback

Virtouch VTPlayer

Braille terminal

1-21

C. Jacquet — Human-Computer Interaction

EFREI SI 2009-2010

Graphical interfaces

Human-Computer Interaction

CHAPTER

2

Graphical Interfaces

1

Chapter Outline Graphical Interfaces 2.1 WIMP 2.2 Windowing systems 2.3 Widgets and associated tasks 2.4 Linking widgets to applications

2

WIMP Concepts

3

2-1

C. Jacquet — Human-Computer Interaction

Graphical interfaces

WIMP • • • •

Windows Icons Menus Pointers

• Benefits – Soft learning curve – Standard environments – Simple to use and to implement 4

Windows, icons, menus

• Icons may stand for “reduced” (iconified) windows • All these graphical elements are called widgets 5

Pointers • Pointer: logical position • Pointing device: change the pointer’s position • Pointer → reflects the current state of the application or system: – Ready – Busy – Moving…

6

2-2

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Windowing systems

7

Window managers • Window manager : – draw "decorations" : title bar, borders – programmers need not worry about the actual configuration of the workstation nor other applications: notion of abstract terminal – users may manage windows : resize, move – example : X Window's window managers

• Compositing window manager – each window drawn independently to a buffer – windows "composited" on the desktop afterwards, generally using 3D acceleration – example : Windows Vista, MacOS X, Compiz 8

Windowing models • Tiled windows / overlapping windows

• Possibly hierarchical

9

2-3

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Window hierarchy • SDI – Single Document Interface A

– No hierarchy, one-level

• MDI – Multiple Document Interface

A

– Sub-windows

• TDI – Tabbed Document Interface

B

A

– Newer, ex: web browsers

B

B

C

B

10

Window manipulation • Classical operations – – – –

Place, move Resize Iconify, maximize, restore initial size Close

• Decorations – Border, title bar – Menu, window icon, keyboard shortcuts

• Special operations – Reduce to title bar – Always on top

11

Re-display hidden zones • Zones of windows get hidden (overlapped, off-screen) • Re-displayed when on-screen again • By the windowing system – Keep window content in memory

• By the applications – Have to be notified when necessary

12

2-4

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Looking back • What hasn’t changed since the Star – WIMP concepts

• What has changed – More applications – Screens have gotten bigger – Many windows per app → designed to run full-screen

• Consequence: new trend – Use the GPU instead of the CPU for graphics (MacOS X, Windows Vista) 13

Widgets and associated tasks

14

Widgets • Graphical interactive object → perform a task • 3 facets: – Presentation: how it looks – Behavior: response to user’s actions – Software interface: API

• Benefits: – Standard interaction styles – Code reuse 15

2-5

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Evolution of widgets • On the Macintosh (1984): – Button – Slider – Pull-down menu – Checkbox – Radio button – Text field – File open/save dialog

• Later: – Multi-level menus, lists, combo-boxes, tabs… 16

Actual radio buttons...

17

Elementary interaction tasks • • • • • •

Data input Selection Action firing Scrolling Arguments and properties specification …

→ Widgets for each task 18

2-6

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Input tasks • Text input – Text box

• Quantity input – Slider, spinner

• Position input – Pointing

• Path input – sampling

19

Selection tasks • Choosing one or several elements in a set – Variable or fixed count – Small or large count

• Fixed count – Menu, radio buttons, checkboxes

• Variable count – List, name input, combination

• Multiple selection: range, add, remove

20

Selection • Menus – Drop-down – Pop-up – Palettes, toolbars

• Organization – Hierarchical – Pie menus = Radial menus (circular pop-up)

21

2-7

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Selection: new techniques [Ahlberg & Shneiderman, 1994]

• Range slider – Easy selection of a range

• Alpha slider North by Northwest ←

→ ABC

DEF

GHIJKL

MN

OPQ

RST

UVWXYZ

22

Firing actions • Buttons and menus • Drag-and-drop – The action depends on the source and target

• Gestures – Specify both command and object Delete

Move 23

Scrolling • Scrollbars – Direction? – Spatial decoupling

• Direct scrolling

• Automatic scrolling

24

2-8

C. Jacquet — Human-Computer Interaction

Graphical interfaces

Specifying arguments and properties • Dialog boxes – Spatial and temporal decoupling between the specification of the command, the parameters, and the execution – Modal, non-modal

• Property boxes – Immediate effect on the selected objects 25

Linking widgets to applications

26

Linking widgets to applications • (1) Interface: set of widgets – Trigger actions – Display output & feedback

• (2) Functional core: business logic interactive system user

interface

LINK

functional core 27

2-9

C. Jacquet — Human-Computer Interaction

Graphical interfaces

1- Notifications Callback functions, messages... 1) Associated to widgets upon creation doPrint(…) { … }

2) Called upon activation doPrint(…) { … } 28

2- Read-evaluation loop The application actively waits for input Read input

Process input

Quit?

29

3- Active values • Bi-directional link between – the state of a widget – a variable i=12

i=26

i=15 30

2-10

C. Jacquet — Human-Computer Interaction

EFREI SI 2009-2010

Architecture Models for HCI

Human-Computer Interaction

CHAPTER

3

Architecture Models for HCI

1

Chapter outline Architecture Models for HCI 3.1 Why architecture models? 3.2 Classical architecture models 3.3 Swing, an example of MVC toolkit

2

3.1 Why Architecture Models?

3

3-1

C. Jacquet — Human-Computer Interaction

Architecture Models for HCI

Crucial question • When designing an application How to link the functional core to the user interface? interactive system/application (desktop, web, mobile...) user

interface

?

functional core

4

Example: typical PHP code ...