Usability evaluation

From WikID

This article is part of the Bachelor design guide; the original version can be found at page 204.

What is product usability evaluation?

A product usability evaluation is a type of evaluation aimed at gaining an understanding of how the user interacts with the prod­uct concept. Product usability evaluation, or usability testing is a means for measuring how well people can use product ideas and concepts for its intended purpose. In product usability evaluation, usage is conceived as an ongoing sequence of user activities related to the functioning of a product in context. Usability is defined as the effectiveness, efficiency and satisfaction with which specific users achieve specific goals in a specific context of use. Product usability evaluation is primary done by using observation techniques. Users are invited to complete tasks, while talking out loud, or discussing their motivation with the researcher. Rather than showing users a rough draft and asking, “Do you understand this?”, usability testing involves watching people trying to use something for its intended purpose.

Contents 1 What is product usability evaluation? 2 When can you use product usability evaluation? 3 How to use product usability evaluation? 3.1 Starting points 3.2 Expected outcome 3.3 Possible procedure 4 Tips and concerns 5 Literature

Setting up a usability test involves carefully creating a scenario of use tasks, or realistic situation, wherein the person performs a list of tasks using the product being tested while observers watch and take notes. Several other test instruments such as scripted instructions, paper prototypes, and pre- and post-test questionnaires are also used to gather feedback on the product being tested. The aim is to observe how people function in a realistic manner, so that designers can see problem areas, and what people like. However, it is important to set up usability evaluations systematically, and to approach the evaluations as formal research projects.

In the design process, product usability evaluations can be conducted at several moments. The character of the usability evalua­tions depends on the moment in the design process:

1. Evaluation of the use of existing products, which typically takes place in the beginning of the design process to analyse existing, analogues products.
2. Evaluation of simulated use of concepts, which typically takes place with the use of sketches of ideas and concepts, and with scenarios or storyboards.
3. Evaluation of use of final designs, which typically takes place with three dimensional models that have a limited functionality. These types of evaluations take place during the design process.
4. Evaluation of use of prototypes, which typically takes places at the end of the design process. These types of evaluations make use of almost fully functioning prototypes.

During usability testing, the aim is to observe people using the product in a situation as realistic as possible, to discover errors and areas of improvement. Designers commonly focus excessively on creating designs that look “cool”, often compromising us­ability and functionality. This is often caused by pressure from the people in charge, forcing designers to develop products based on management expectations instead of people’s needs. An important aspect of product usability evaluations is verifying presump­tions regarding the use of a product. Presumptions that are investigated in product usability evaluations are the product charac­teristics (materials and shapes) that provide the users with “hints” as to how to use the products. These product characteristics are also called use cues. Use cues are meaningful product characteristic that are given to products to show user what function­alities a prroducts has and how to use these functionalities. Some use cues are deliberately designed, and some use cues are discovered in the usability evaluation. One of the goals of product usability evaluations is to test designed use cues, and discover the unobvious use cues. The elements of a product usability evaluation are the product, the respondent, the test setting and the type of evaluation. The product is often a prototype, either with limited functionality or almost full functionality. The choice of respondents is depending on how they represent the user group. Because product usability evaluations are very time consuming, often a limited number of respondents is chosen. The test setting can be either a controlled environment such as a laboratory, or with users in their natural environment. The type of evaluation could be self reports, where users are asked to report themselves on their usage of the product (for example through thinking aloud, or retrospective interviewing). Evaluation can also take the form of asking questions to users while performing tasks, or even measuring human characteristics (for example eye tracking).

When can you use product usability evaluation?

Product usability evaluations can be used throughout the design process.

How to use product usability evaluation?

Starting points

The starting point of a product usability testing is the need to investigate the usage of existing products or verify (test) the usage and ease of use (usability) of new product ideas and concepts.

Expected outcome

The outcome of product usability evaluations with existing products is often a list with requirements with which the new prod­uct must comply. Product usability evaluations with new products results in a list with useful aspects and issues about the use of the new product and improvements that could resolve those issues.

Possible procedure

1. Determine the research objective.
2. Describe the presumptions, in other words: in what way do the usecues designed by the designer help the user in using the product? Describe the presumptions very explicitly. Presumptions are not predictions though!
3. Formulate research questions.
4. Design your research. Think about: what type of models are you using (scenarios, stroyboards, prototypes), the research envi­ronment, make instructions, determine the type of evaluation.
5. Do the observations. Record the usability evaluations.
6. Analyse the results. You can choose to do a qualitative analysis or a quantitative analysis.
7. Redesign the product based on the results. Often improvements are suggested in the evaluation.
8. Communicate the results.

Tips and concerns

• You may include a limited number of qualitative questions that will help inform future design research, but don’t let these ques­tions sidetrack the users from their primary tasks.
• Employ guerilla testing techniques if money and time are limited. You don’t always need formal recruiting or testing facilities. Use your personal network to find unbiased people to test. Use a conference room as a testing lab. Any testing you can do is better than no testing at all.

Literature

• N. Roozenburg and E. Eekels (1995) Product Design: Fundamentals and Methods, Lemma, Utrecht.
• W.F. van Raaij et.al. (1999) Product en Consument, Lemma, Utrecht.
• J. Schoormans en C. de Bont (1995) Consumentenonderzoek in de productontwikkeling, Lemma, Utrecht.
• D. van Eijk (2004) Dictaat Ontwerpergonomie, Industrieel Ontwerpen, Delft.
• H. Kanis (2005) Observation as a design tool, In: Dictaat Design Observation, Industrieel Ontwerpen, Delft.
• H. Kanis et.al. (1993) Usecues in the Delft Design Course, In: Dictaat Design Observation, Industrieel Ontwerpen, Delft.