Concept evaluation

From WikID

This article is part of the Bachelor design guide; the original version can be found at page 198.

What is product concept evaluation?

A product concept evaluation is a type of evaluation in which the product concept developed so far is reviewed by the user group. Generally, product concept evaluations are aimed at selecting or optimizing product concepts based on the preferences of the user group and other stakeholders. The product concepts that are evaluated can have different forms (descriptions, drawings or prototypes). Typically, a product concept evaluation takes place in a controlled environment, where a panel of people judge product concepts based on a list with predetermined issues. Product concept evaluations serve different purposes: concept screening, concept optimization, and go/no-go-decision^[1].

Contents 1 What is product concept evaluation? 2 When can you use product concept evaluation? 3 How to use product concept evaluation? 3.1 Starting points 3.2 Expected outcome 3.3 Possible procedure 4 Tips and concerns 5 Literature 6 References

Concept screening is aimed at selecting worthwhile product concepts. Concept screening is necessary when a large quantity of product ideas or product concept has been generated. From these product ideas and concepts a selection have to be made for further development. Often experts (managers, engineers, marketers) are invited to do a concept screening rather than repre­sentatives from the user group. Concept screening often involves evaluating product ideas and concept in light of the formulated requirements. Concept optimization is aimed at determining which aspects of product ideas and concepts need further improve­ment. These tests are not directed at judging the total concept, but rather parts or elements of product ideas and concepts. The assumption is that preferred aspects or elements of the individual product concepts can be connected with each other, yielding a concept that is regarded as optimal. Product concept evaluations for go/no-go-decisions are aimed at validating important de­sign decisions. These decisions often involve the choice between two or three product concepts. Designers can make decisions based on the programme of requirements, but sometimes it is necessary to validate these decisions by the user group.

Generally, in product concept evaluations respondents are confronted with product concepts, and are invited to evaluate these product concepts. The common elements in all product concept evaluations are the concepts, the respondents, and the type of evaluation. In product concept evaluation the concepts are simple representations of the product ideas at a certain stage in the design process. This means that in a product concept evaluation the basic functions of the product idea cannot be used.

The type of concepts that can be used for product concept evaluations are the following:

1. Textual concepts: descriptions of the product idea, which generally consist of a description of what one can do with the prod­uct idea. There are roughly two forms of textual descriptions: a scenario of how a person can use the product, or summing up the aspects of the product idea.
2. 

   

   Figure 1: Example of product concept evaluation with the use of design sketch and a scenar­io/storyboard (source: Nomad/Zilver Productontwikkeling, 2001).
   Pictographical concepts: visual representations of the product ideas. Depending on the design process these visual representations are highly detailed visualisations or simple representations. Since sketching and visualisation is so important during the design process, pictographical concepts evaluations are most common. In recent years, computer graphics has en­hanced pictographical concept evaluation though easy manipulation of the perspective of the visualisation.
3. Animations: moving visual representations of the product idea. Using computer graphic software it has become quite easy to make a simple animation of how the product can be used in a particular context.
4. 

   

   Figure 2: Example of a threedimensional model used in a panel evaluation (from student report).
   Mock-ups (dummies): three dimensional, tangible representations of the product idea. Mock-ups are a type of prototype that only shows the external (form) characteristics of a product idea.

The selection of respondents is an important aspect of product concept evaluations. Repondents that are invited for product concept evaluations belong to one or more of the pre-formulated user groups. One can make a selection based on the socio-cul­tural characteristics or based on demographical characteristics. Important issues to take into account are the level of knowledge of the product category that the respondents have. To assess this level of knowledge, one could simply ask the respondent about their experiences with similar products. Another important issue to take into account when selecting respondents is related to psychological aspects such as tolerance and innovativeness. Questions that are important are: how tolerant are the respondents towards new products and new situations? How innovative, or conservative, are the respondents? Such psychological aspects have a big influence on the results of the product concept evaluations.

Different types of evaluations can be used for product concept evaluations. One of the most common methods used in product concept evaluation is the personal (individual) interview. Another form can be focus groups, or discussion groups. In focus groups, a product concept evaluation takes place with a small group of people, and has the form of a group discussion. Product concept evaluation are structured according to pre-formulated lists with questions. In the evaluation of the prduct concept the respon­dents are asked about their judgments. Respondents can give their judgment using rating scales, or with ranking scales. When rating product concepts, respondents attribute a score to several aspects of the concepts. When ranking, respondents are asked to order the concepts according to their preferences.

Product concept evaluations often take place in a controlled environment such as a laboratory. The reason for this is to provide as little distraction as possible. The evaluations are recorded using video and audio. Often questionnaires are used to capture the evaluations.

When can you use product concept evaluation?

Product concept evaluations take place throughout the design process, based on the purpose of the product concept evaluation. Concept screenings usually involve large amount of product ideas and concepts, and therefore are more frequent in the beginning of the design process. Concept optimization takes place near the end of the design process, when aspects of the concept need to improved and optimized.

How to use product concept evaluation?

Starting points

The starting point of a product concept evaluation are a number of concepts to be judged (minimum of two), and a reason for conducting the product concept evaluation. The reasons determine the type and purpose of the product concept evaluation.

Expected outcome

The expected outcome is a validated choice between a number of concepts in case of a concept screening or a go/no-go-deci­sion, or a better understanding of what aspects need improvement/optimization.

Possible procedure

1. Describe the goal of the product concept evaluation.
2. Determine what type of product concept evaluation you want to conduct.
3. Gather or create the appropriate concepts for the evaluation.
4. Create a plan for the product concept evaluation. This plan should include: the goals and type of evaluation, a description of the respondents, questions you want to ask the respondents, aspects of the product concept that need to be evaluated, a description of the test environment, means of recording the evaluation, plan of how you are going to analyse the results.
5. Search for and invite respondents to the evaluation.
6. Set up the test environment, including recording equipment.
7. Conduct the concept evaluation.
8. Analyse the results, and present the results concise using either a report or a poster.

Tips and concerns

• Make sure that you search for a valid representation of the user group when inviting respondents (don’t forget to provide them with a compensation).
• Make sure to structure the evaluation systematically with the questions you want to ask the respondents.

Literature

• N. Roozenburg and E. Eekels (1995) Product Design: Fundamentals and Methods, Lemma, Utrecht.
• W.F. van Raaij et.al. (1999) Product en Consument, Lemma, Utrecht.

References

1. ↑ J. Schoormans en C. de Bont (1995) Consumentenonderzoek in de productontwikkeling, Lemma, Utrecht.