An Introduction to Planning and Design of Product Tests with an Eye on Controlling Order Effects

Product tests are all experiments of a sort. To plan and execute them well, consideration needs be given to:

• Treatment Structure: the nature of the products to be tested
• Design Structure: who sees what and when
• Analysis of the data collected

Often, these tests ask respondents to evaluate more than one product, creating order effects: the influence of the context within which a product is evaluated. These effects can be “balanced out” with an appropriate design… but the design itself does not nullify the effects. Only with an appropriate design can these effects be measured, controlled, and used to better evaluate product performance.