Defining "usability"
Usability in common usage
Usability is generally considered to mean ease of use
A broad definition (Jakob Nielsen)
Usability is the measure of the quality of the user experience when interacting with something.
A more specific definition (ISO 9241)
Usability is a measure of the effectiveness, efficiency and satisfaction with which specified users can achieve specified goals in a particular environment.
Key terms in the ISO 9241 definition
- Effectiveness, efficiency, satisfaction
- Specified users, specified goals, particular environment
The five Es of usability
Effective
- Completeness - was the task fully completed? Were the user's goals met?
- Accuracy - was the task completed successfully? Did the user get the right or correct result? How well was the work done?
Efficient
- Speed - was the user able to complete the task quickly?
- Effort - was the user able to complete the task without undue cognitive effort?
Engaging
- Pleasant - did the user have a pleasant experience when working on the task?
- Satisfying - was the user satisfied by the way in which the application supported her work?
Error tolerant
- Error prevention - did the user interface help users avoid making errors? Were errors minor rather than major?
- Error recovery - if the user made an error, did the interface assist them in making a successful recovery?
Easy to learn
- Predictability - was the user able to work with some certainty because the user interface built on her previous knowledge?
- Consistency - was the interface consistent, so that once a user learnt how to use part of the application, they were able to easily learn how to use another part?
Balancing these attributes
- On any one project, one or more of these attributes might be considered
more important than others
- e.g. a finance application may prioritise the requirements for effectiveness over engagement, since tasks must be completed fully and with accuracy
- However, a balance of all attributes will be required to achieve usability
References
- Definition of usability
- Usability basics
- What does usability mean: looking beyond 'ease of use'
- User-centred design standards
The term "usability" is used in various ways
- Usability as an outcome: web sites and applications that are usable
- Usability as a process: a methodology or approach (usually called "user-centred design")
- Usability as a set of techniques: usability testing, contextual enquiry, heuristic evaluation - there are many techniques whose aim is to improve usability
- Usability as a philosophy: where improved usability is a value that motivates the way in which products are developed
Things that are not "usability"
- High or increasing usage does not mean that a web site or application
is usable
- increased usage may in fact mean that people cannot find things and are trying many options or going the long way around to do things
- Checking for broken links is not "doing usability"
- it's highly desirable that your site has no broken links, but it may still be difficult to use even if all links are working
- "Has Dey seen this?"is not an effective usability
process
- often I'm asked to look at something at the end of the development process - it's usually too late then
- a user-centred design process, adopted from the start of the project, is more likely to produce a good outcome
Why is usability important for the web?
The web has changed the way we do business
- Many face-to-face transactions are now done online
- Course enrolment, tutorial allocations, student support, teaching and other university functions
- Personnel-related functions including payroll queries, form downloads, policy and procedure banks
- Commercial transactions including purchasing of good and services, ebanking and payment of bills
The context of online transactions is radically different to the face-to-face business context
- There is two-way communication channel to ensure you understand what the client wants to achieve
- You can easily sift through all the business information/resources to find an exact match to the client's request - because you know exactly what you've got and where it is
- There is two-way communication channel to ensure that the client has understood your response and that your response has satisfied the client's request
Usability is the key to doing business online successfully
- We can't explain face-to-face how our business works
- We can't find the information for our clients
- We can't complete their tasks for them
- Our web user interface must be designed carefully so that it can play the role of our customer service representative
Design challenges
Making sense of our information
- We have to take a collection of information (or tasks)
- And give the information a logical structure
- Label the structures in a meaningful way
- Design a user interface that users understand
Designing for different types of users
- Not all users have the same goals
- So good design must allow different users to access the same content in different ways
Designing for special situations
- Disabled users need to be considered
- And we need to design for a global audience
Designing around organisational problems
- We need to avoid using jargon
- And watch out for organisational politics
User-centred design
- The key to improving usability is to adopt a methodology commonly referred to as user-centred design (or UCD)
- The methodology involves the use of a set of UCD techniques
- All must be supported by a commitment to the philosophy of UCD
Key concepts in user-centred design
- Early and ongoing focus on users and their goals
- Empirical measurement of how the system is used
- Iterative design cycle (of research, design, evaluation activities)
UCD versus traditional development approaches
Putting user-centred design into practice
- UCD methods must be introduced into development lifecycles
- In requirements analysis phase, you could add:
- techniques aimed to identify and profile of target user groups
- techniques to help understand how the project will affect work
- In design phase you could add
:
- techniques to understand how users work and think
- techniques to test out your designs before they get coded
- In evaluation phase you could add:
- techniques to test your designs as parts of the site/application are coded
Identify and profile the users who will be affected by the project. Find out how the project will affect their work
- Meet with your client
- Run a brainstorming session with project team, key client contacts (managers, supervisors)
- Do a web server log analysis
- we were able to identify that many of our 'external' pages were heavily used by internal audiences by checking the IP address for each page request
- we could identify heavily used resources that weren't prominently linked
- Run some user interviews (interview sample groups or knowledeable supervisors/managers)
- Use any existing data (HR, marketing may have demographic data; IT may have identified issues relating to IT skills)
Understand how users work and think
- Run some user interviews or workshops (use sample groups or knowledeable supervisors/managers)
- Do some "contextual enquiry" (observing and learning about users' work)
- Look at how work is performed, how individuals work as a group and with other groups
- Identify tasks that are critical, frequently performed
- Get a feel for the work culture and environment
- See if there are any problems with the way the work is currently performed
- Together with users, identify opportunities for streamlining work, reducing inefficiencies
- Run a card sorting exercise to come up with some content categories for information-dense sites or applications
Test out your designs before they get coded
- Run some usability testing of your content categories and labels
- Run some usability testing using paper versions of pages or screens within your site/application
Test your designs as parts of the site or application are coded
- Run some usability testing using working prototypes of parts of the site or application
References
Usability and return on investment
The cost of doing usability
- Extends project timelines - this is debatable (see Cost-Justifying Usability by Bias and Mayhew 1994)
- Requires additional resources
- Recent research showed 8-13% was being spent on usability [Nielsen Norman Group 2003]
- However, there are 'hidden' costs of not doing usability and these are rarely factored into project costings
The cost of not doing usability - some examples
Software
- The average software program has 40 design flaws that impair employees' ability to use it. The cost in lost productivity is up to 720% [Landauer 1995]
- In order to meet its customer support call needs, WordPerfect had to employ over 900 customer response specialists [1992]
- $500 billion is spent annually in the US on computers, networks and IT, with a resulting DECREASE in productivity [Roach & Morgan 1992]
Web sites
- 1 in 3 users find it 'somewhat or extremely difficult' to locate specific products online. 62 percent gave up looking on at least one occasion in previous 2 months [Zona Research Inc, 1998 survey of 239 long time Internet users]
- "Major sites have only 51% compliance with simple web usability principles", e.g. is the site organised by user goals; does a search list retrievals in order of relevance [1998 Forrester Research study of 20 commercial sites]
- Users can only find what they want 42% of the time - even when taken to the correct home page [Jared Spool, 1998 study of 18 large commercial sites]
- Dot.com crash analysis reported poor usability as a major problem: "too many options, unclear navigation, badly laid out information, and purchase or checkout process that was just too difficult to use" [cited in Australian Net Guide, April 2001]
Return on investment - some examples
Software
- Design changes due to usability work at IBM resulted in an average reduction of 9.6 mins per task, saving $6.8 million in 1991 alone [Karat 1990]
- Design changes due to usability work o one project at IDS/American Express resulted in savings of $45 million [Chalupnik & Rinehart 1992]
- Design changes from a usability study at Ford reduced the number of helpdesk calls with resulting savings of $100,000 [Kitsuse 1991]
Web sites
- Sales/conversion rates doubled due to usability efforts [Nielsen Norman Group 2003]
- User performance/productivity improved by 160% due to usability efforts [Nielsen Norman Group 2003]
Overall value of implementing usability
- Rule of thumb in many usability-aware organisations is that the cost-benefit ratio for usability is $1:$10-$100
- Once a system is in development, correcting the problem can cost 10 times more than fixing it during design
- Once a system has been released, it can cost 100 times more than fixing it during design
Summary of value propositions
- Save development costs
- Save development time
- Reduce maintenance costs
- Increase revenue
- Increase transactions/purchases
- Retain customers
- Attract more customers
- Increase market share
- Increase success rate and reduce user error
- Increase efficiency/productivity
- Increase user satisfaction
- Increase job satisfaction/decrease job turnover
- Increase ease of use
- Increase ease of learning
- Increase trust in systems
- Decrease support costs
- Reduce training/documentation cost
References
- The Proof for Usability's ROI: Statistics and Examples
- Usability and ROI: A Collection of Links
- Usability Return on Investment
UCD case study: redesign of the Monash University web site
Overview
- When decision was made to do a redesign, a UCD method was suggested
- UCD seemed the obvious approach given the results of the previous redesign
- Poor user acceptance
- Slow uptake by business units (some sites still don't use the official design - 3 years after its launch)
- Perceptions of poor usability (later confirmed through research and testing)
- Taking this approach presented a real risk
- Not used before on Monash IT projects
- Needs high level management support
- Raised expectations within user community, especially in light of poor acceptance of existing design
- Two major project phases
- Research
- Design and evaluation - this phase still in progress
Research phase
- Based on three sets of activities
- Information from and about our primary audience groups
- An evaluation of our competitors' web sites
- Understand and articulate the business objectives and strategies for the site
Primary audience groups
- Main questions for research were:
- How is the site being used?
- What information and resources were being sought and by whom?
- How successful were users in locating resources?
- How did users feel about using the site?
- What aspects of the site did they like or dislike?
- What issues should the redesign focus on?
- A number of user-centred design methods were employed:
- An analysis of email feedback from users of the site. Two separate sets of email were examined: feedback solicited by University Marketing shortly after the launch of the last redesign in early 2000, and18 months worth of email sent to the Monash webmaster address. The analysis focused on identifying patterns in user feedback that might be suggestive of usability problems that warranted further consideration.
- An analysis of web server log and search log files from mid-1999 to mid-2001 and for several key periods in 2002. This analysis aimed to identify key user groups for particular resources and browsers used by these groups. We wanted to see which resources were highly utilised or searched for and improve ease of access to these. Click-through rates were also analysed, especially on pages where information design was considered to be poor.
- A survey of secondary school careers teachers focusing on eliciting their information needs and those of year 11 and 12 students.
- An online survey of all user groups aimed at gauging user experiences of the site and identifying user information needs though the collection of quantitative data.
- A series of workshops with primary audience groups aimed at understanding user experiences of the site, identifying user information needs and prioritising issues for the redesign of the site.
- Usability testing of the existing site to identify and measure usability problems and benchmark usability in order to facilitate comparison during a set of iterative design changes over time.
Our competitors' web sites
- Main questions for research were:
- What resources do they offer?
- What resources were giving prominence?
- How is navigation designed?
- What terminology is used (e.g. "prospective" vs "future" students, "distance education" vs "off-campus learning")
- How do other sites perform (download times, etc.)
- Methods adopted in this stage were:
- Examination of key pages on competitors' web sites
- Usability testing with prospective students to measure the performance of the Monash site against competitor sites
Business objectives and strategies
- Main questions for research were:
- Identification of key target audiences by business units
- Articulation of the aims or goals of business units' web sites
- Identificiation of the main resources offered by these sites and plans for future development
- ifficulties conforming to existing design requirements
- Issues to be addressed in the redesign
- Method used:
- A series of meetings with business units was conducted
- Interestingly, many of the participants raised usability concerns with the existing design, based on their own use of the site
Design and evaluation phase
- Based on three sets of activities
- Navigation and user interface design
- Content review
- Visual design
Navigation and user interface design
- Phone and face-to-face interviews leading to the development of "personas" for use in other design activities
- A series of participatory design workshops to consider the design of global navigation, the home page, section home pages, and content pages. A set of wireframes were produced as a result
- Usability testing of wireframes to ensure that users understood key navigation labels
Content review
- Series of meetings with key content stakeholders. Research results and activities of competitor sites were discussed, along with strategies for redeveloping and maintaining content on the corporate web site
- At the time of writing, this phase has just concluded, a report is being written, and wireframes for section home pages and content pages are being refined
Visual design
- Visual design brief has been drafted. The brief includes:
- Usability and accessibility goals for the redevelopment
- Business goals and strategies
- Tested and refined wireframes
- Design concepts will be invited from within the Monash community
- Concepts will then be evaluated using this process:
- Heuristic inspection (expert evaluation) to ensure that visual design concepts comply with usability guidelines
- Usability testing (of the design concepts that pass the heuristic evaluation) to ensure that users are able to recognise structural elements such as global navigation, local navigation, search and so on. Testing will be done using full colour paper mockups of the design
- The concept selected by the project sponsor/steering commitee with then be further developed and usability testing of a working prototype will be conducted, with design modifications made as required
Implementation and post-implementation review
- We are hoping to publicise the redesign so that primary user groups will not be surprised when the new site goes live
- After launch, we intend to monitor user feedback and make changes to pages as required
- After rollout of the design to sub-sites within Monash, we hope to be able to do another round of usability testing to see how effective the process has been
Support for user-centred design at Monash
UCD methods are now being used on a range of projects
- Redesign of the structure and navigation of the Staff Development Unit
web site
- Card sorting used to create structures/navigation labels
- Redesign of the Monash International web site
- Card sorting used to create structures/navigation labels
- Usability testing of categories and labels
- Monash University Timetable System (Syllabus Plus)
- Walkthrough and workshop with user group
- Usability testing of the user interface
Support from the Web Resources and Development
- Usability and user-centred design topics will be covered in this Web Workshop series
- Over time, we will add more courses and workshops to those offered through
the Staff Development Unit
- currently offer "Writing for the Web"
- may soon be able to offer a series of short, connected web accessibility workshops
- Will soon extend our ability to offer consulting and support for UCD practices
at Monash
- hope to employ an additional person
- will be able to offer consulting services on a wider range of projects
- will start documenting UCD processes and methods and provide tools and templates so you can start experimenting with UCD on your own projects