Integrating the Consumer into New Product Development

The food industry relies heavily on new products to rejuvenate and maintain its business. New product development (NPD) is the lifeblood of any industry (Nuese, 1995), especially the food industry (Best, 1991; Hoban, 1998). It now commonly accepted that NPD is the only path that the food company can follow for long-term survival in the ever-growing competitive market.

This may explain why in the United States alone, approximately 20,000 food products are introduced yearly (Adams, 1997), although the number of truly “new” products is not clear and may be a function of how they are defined (Hoban, 1998; Mathews, 1997). One classification scheme categorizes new products into four main groups: classically innovative products, equity transfer products, line extensions, and clones or competitive entries, (Hoban, 1998). Based on Prime Consulting Group’s categories (product sales trend over an extended period of time, adjusted for the introductory period), data for all new products show that that only 1,100–1,200 products can be considered as truly “new.” Of those, only about 33% were deemed to be successful, 42% were still in distribution but with declining sales, and 25% failed (Adams, 1997).

The cost of NPD is strictly confidential and considered proprietary information by most companies. Nevertheless, the overall cost can be estimated or calculated with some basic assumptions. For instance, as of 7–10 years ago, the total cost of NPD for a large or small company was estimated (not considering the inflation rate) to be $9–12 million (Graf and Saguy, 1991). Given these numbers, the cost of new product failure per year is estimated to be $3–10 billion. This highlights the staggering cost of failure in NPD.

There are other losses as well, perhaps less shocking, but still costly. These include lost opportunities, lost customers, wasted shelf space, and numerous other measurable costs. In the big picture of product development and marketing, NPD failure continues to incur a huge price tag that has not declined, in spite of the enormous efforts and investments in both R&D and consumer research. Indeed, Kurt Salmon Associates, Inc., identified effective product introduction as one of the four efficient consumer response strategies that could save the industry 0.9% of its current total supply-chain operating costs (Mathews, 1997).

The formula for success can be simply stated: deliver tangible, affordable benefits to consumers in a manner that addresses their concerns about the products or processes used in their food supply (Connor and Schiek, 1997). The paradox is thus: Why, when the objective is so simple and clear, when the appropriate resources are used, when the proper steps are followed, do companies again and again experience marketplace failure? Moreover, how we can explain this low success rate in spite of the availability and use of sophisticated computer-aided methods, early-stage and late-stage consumer research, market studies, management dedication, and a wide spectrum of other activities and technologies?

To address this issue, researchers have devoted considerable efforts trying to help practitioners discover what particular tools, techniques, and methods offer a competitive edge. These efforts, starting more than 30 years ago, focused on understanding the NPD process and identifying those deemed to be “best practices” (Griffin, 1997). During the past five years, pursuit of this goal produced numerous privately available reports and two research efforts sponsored by the Product Development and Management Association, PDMA (Rosenau et al., 1996). A recent survey indicated that NPD processes continue to evolve and become more sophisticated. NPD changes continually on multiple fronts, and firms that fail to keep their NPD practices up to date will suffer an increasingly marked competitive disadvantage (Griffin, 1997).

Proven tools exist to gather, disseminate, and use market information. Yet most firms continue to fail, despite widespread recognition of the important role that market and consumer knowledge plays in NPD. This article analyzes some of the practices implemented in NPD, focusing on possible explanations for the high failure rates in the marketplace. The natural corollary to this analysis is an understanding of what “ingredients” are required today to improve the odds for launching successful new products. The article then presents a paradigm involving the extended involvement of consumers and sensory researchers in the development of product concepts and optimal products.

--- PAGE BREAK ---

Assessing New Product Success
The stable high rate of new product failures suggests quite clearly that the main reason for NPD’s lack of success in the food industry may result from basic and fundamental problems. The evolution of food products is occasionally recognized as a nonlinear process, whose path can be affected by both major events that drastically alter the environment and very minor events that may be unnoticeable at the time of their occurrence (Peleg, 1994). This approach, using the metaphor of Darwinian evolutionary principles, seems superficially appropriate when we consider the high rate of new product failures and the survival of a few products that may be “lucky,” in the “right niche,” or “possess the right properties for the current environment.”

It is difficult for the product developer to accept Darwinian theory, especially the survival of the very few products (rather than the larger number of failed products that have been well researched). Darwinian principles for NPD in an unforgiving environment is unacceptable, for the principles suggest failures as inevitable, despite tremendous efforts invested in research, methodology, consumer understanding, marketing theory, sales scenarios and promotions, advertising, etc. Darwinian principles also imply the uncomfortable, paradoxical possibility that often-ignored minor and possibly insignificant and irrelevant events may play a significant role. Hence, the recommended paradigm shift should concentrate on a new area of opportunity, distancing itself from the failure-prone dualities of the current NPD “processes’” and Darwinian principles.

The need to quantify NPD success in the marketplace is clear. Yet there are no generally accepted rules and methods that are widely accepted or utilized as standards. Measuring new product performance is valuable for at least three reasons: (1) It facilitates organizational and process learning; (2) it leads to quantifiable benefits such as improved cycle times, enhanced new product success rates, and improved ability to assess new product strategy and processes for future NPD; and (3) it furnishes crucial information for marketing, sales, and management, and helps them to adjust their advertising and marketing strategy.

A recent study (Hoban, 1998) highlights the fact that the lack of well-established and accepted criteria for assessing NPD performance supports the widely held “myth” that of the approximately 20,000 new food products introduced yearly, more than 90% fail (Connor and Schiek, 1997; Hollingsworth, 1996; Mathews, 1997). Similar failure rate figures have been reported—e.g., 80% in the past decade, reaching a peak of 91% in 1989 (Hollingsworth, 1994). It is worth noting that the definition of a new product is quite nebulous, leaving a wide void for various classifications. For instance, in Hoban’s study, only recently new products were categorized into four categories: classically innovative products, equity transfer products, line extensions, and clones or competitive entry.

The success of NPD in the food industry is even bleaker when we realize that for personal and “political” reasons the respondents “soften their answers,” painting a rosier picture of the process than is actually the case. To highlight this point, consider the data from an annual product development survey. An Executive Advisory Panel of 500 readers representing a cross-section of the food industry were asked to assess their industry’s innovation level by looking at their own company’s approach to risk, reward, and NPD. It was not surprising that about 15% of the respondents thought that their company was “excellent” in overall NPD and another 38% thought that their company was “better than most” (Glenn, 1997). Comparing these figures with the actual new product performance in the marketplace indicates that these self-reported performances bear little relation to reality.

Another surprising finding was that overall, 57% of companies in the survey used no formal system to measure product or process innovation. Some of the difficulty could perhaps be ascribed to the use of an inappropriate metric for NPD performance. Some examples of inappropriate metrics (Glenn, 1997) include revenue generated (72%); technological impact, commonly defined as the number of times other researchers cite the granted patent in developing their own product (35%); patents filed (27%), and patents granted (25%).

--- PAGE BREAK ---

New Paradigm Involves the Consumer
Whatever criterion or benchmark technique and methods are applied, and how sophisticated they may be, it is paramount to maintain a consumer perspective. It is also worth noting that most scientists tend to minimize the fact that product innovation and new technology are a necessary but not sufficient condition for success in the marketplace. It is a widely accepted fact that consumers don’t care much about the technology, and a product that pleases the senses and wallet can be made by any process and be accepted equally (Katz, 1998). Success is achieved only when the new product gains wide consumer acceptance.

Art and science aside, with new products, as with everything else in the food industry, the consumer must be the focal point of all our efforts. Hence, the optimal NPD must start and end with the consumer’s intimate involvement. This is reminiscent of Deming’s (1986) concept of Total Quality, which developed into Total Quality Management (TQM). It is essentially an organizational process that actively involves every function and every employee to satisfy customer needs, internal and external. TQM continually improves all aspects of work through structured control, improvement, and planning. All activities are carried out in concert with a guiding ideology whose top priority is quality and customer satisfaction. TQM recognizes that the customer lies at the center of every activity. It is an ongoing process that allows the manufacturer to build quality into the product by identifying and controlling resources, requirements, responsibilities, and interrelations. As part of the TQM paradigm, consumers provide continuous feedback from the starting concept through to the marketplace.

The burning issue for the food industry is how to implement this TQM paradigm with consumers, in a way that is rapid, powerful, cost effective, and market driven, so that the paradigm has a fighting chance to become an ongoing, standard, welcome, and productive framework for product development.

There is no magic bullet nor one optimal NPD method that fits all products and companies. Hence, it is not surprising that there are as many product development processes as there are food companies (Earle and Earle, 1998; Earle, 1997). Certain fundamental stages are commonly accepted, such as screening, feasibility, development, commercialization, and maintenance (Graf and Saguy, 1991). Other practitioners and theoreticians divide the process into four main stages: product strategy development, product design and process development, product commercialization, and product launch and evaluation (Earle, 1997). Still other approaches (e.g., Stinson, 1996; Fuller, 1994) treat most complex processes from a number of different angles.

To focus on the consumer involvement, we present in detail below only the stages where their contributions are cardinal, and simply list the others. It is worth noting, however, that to date, each company appears to have implemented a process that is considered (by that company) to be the “best approach” after its adaptation to the culture and its special needs and requirements. Although “comfortable” processes have evolved with time, they could be far from optimal.

Consumer data (e.g., ideation for development, sensory evaluation for prototypes, acceptance throughout the marketing cycle) can increase quality, reduce cost, and accelerate cycle time for the introduction of novel food products. Consumer data help to build the road map to guide the manufacturer. We are not talking here about the conventional sensory evaluation panel, nor about the large-scale but episodic market research studies, but rather about a systematic involvement of the consumer at all phases, with a feedback loop that is extensive, accurate, and actionable.

This paradigm shift is defined as “consumer compliance.” The paradigm is founded on four main pillars, two substantive—concept development and sensory/consumer evaluation—and two enabling—a multidisciplinary team with cross-functional integration, and intimate management involvement. The continuing involvement of consumers with the developers in an integrated fashion sustains the melding of their needs with technical capabilities. The paradigm, consisting of four “pillars,” described below, works in tandem with sociological and health/nutrition trends to take rapid advantage of newly emerging consumer needs and wants.

--- PAGE BREAK ---

Pillar 1: Consumer Concept Development
At the heart of any consumer product development paradigm is the need for concepts. Concepts today are created by fiat, perhaps from a brainstorming session or focus groups, often by marketing directors or advertising agency executives, and in general are evaluated by consumers after they have been fully shaped back in the company. There may be a better way, more empirically grounded, yet more philosophically satisfying. We return for inspiration to the philosopher Plato, who 2,500 years ago discussed the notion of “forms,” ideas residing in an imperceptible ether. This Platonic notion is quite appealing and a compelling organizing principle.

The basic idea is that innovative products exist in “ether”—i.e., the consumer does not know how to describe these products, but will know them when he sees them. Furthermore, what may be innovative today may have been unthinkable yesterday, and trite tomorrow. Thus, innovation and creativity have the aspects and constraints of time, experience, and environment. Product ideas may be universal, but innovative products possess spatial and temporal limits (Moskowitz, 1998).

Consumer-intimate concept development presents consumers with “pieces” of a product concept (features, benefits, etc.). These pieces are the raw material for the concept. Some of these features represent “close-in” ideas, available today; others represent “farther-out” ideas, available as innovations. These ideas could be just parts of the concept (e.g., statement, picture, video, audio). Computer technology utilizing multimedia software combines these “pieces” into different concepts (e.g., through a layout-dictated experimental design) and presents them to the consumer, who reacts (e.g., rates them on degree of interest and degree to which the combination fits an end use). The consumer may not be able to articulate new ideas, but can intuitively respond that one combination is better than another combination, or fits an end use better. Through statistical modeling, the developer identifies the “hot buttons” for the new ideas (i.e., relevant features), recombines them, and thus creates new and better combinations (Green and Srinivasan, 1991; Moskowitz, 1994; Moskowitz and Martin, 1993; Wittink and Cattin, 1989).

The computer acts as an enabling device. Computer presentation of concepts that embody aspects of this Platonic ideal is akin to an individual hurling buckets of paint at a passing invisible object (i.e., the idea). As the object (idea) traverses a path and as the individual hurls more and more buckets of paint, some of the paint will stick (where the invisible idea is), and some will disappear (where the invisible idea isn’t). At the end of the path and with enough buckets of paint, the idea and all its lineaments should become well outlined by the paint that was thrown and stuck.

A similar argument holds for concept development. The consumer is the integrative device. The computer hurls the experimentally designed concepts forward, and the consumer simply reacts by saying “close” or “far away,” probably more intuitively rather than with intellectual knowledge. Eventually, with enough concepts there will be some that are close to the unformed, unexpressed idea in the consumer’s mind, and some that are far away. Since the concept elements are arrayed by experimental design, it becomes straightforward through regression analysis to identify which particular elements push the concept closer to the unexpressed idea, and which move it away (Box and Draper, 1987).

Furthermore, to aid innovation, one can put the consumer into a simulated environment (e.g., a future scenario) and do the study. To the degree that the idea changes in the consumer’s mind (e.g., as a result of this simulated new environment or situation), the method reveals changes in the features of the idea. This approach—raw materials (elements) + enabling device (computer) + integrative device (consumer)—allows for sustainable innovation, is founded on consumer data, yet requires minimal resources.

It is worth noting that the above process clearly does not combine the pieces into a complex but flat and well-defined jigsaw puzzle. Rather, it combines many pieces together in a multidimensional space that ultimately yields a clear picture of the shape. This could be described as a picture generated with a kaleidoscope in n dimensions.

--- PAGE BREAK ---

Pillar 2: Sensory/Consumer Inputs
Foods serve a much wider role than just satisfying hunger needs and providing a sense of satiety. They also play a paramount role in social and cultural customs and behavior. The need to focus on the consumer has led to new ways to understand their behavior, ranging from extended product and concept tests with many stimuli to behavioral studies of how foods are selected, prepared, and served. There is a growing realization that to innovate in the food industry requires a more profound knowledge of the interaction between the consumer and food in a real-world setting, rather than a short exposure to the food and the scaling of the food on one or a few general attributes, such as overall liking.

Despite the oft-touted professionalization of the sensory analysis industry with focus on expert panels and sensory–instrumental correlations, the truth of the matter is that the consumer and only the consumer can really guide product development. All other measurements and indicators, such as expert panels and instrumental data, act only as surrogates. The issue now is how to incorporate the consumer feedback into the development system in a way that provides an actionable linkage between consumer needs/wants and product development capabilities.

Here are five recommended steps, based on knowledge of consumer abilities to evaluate products, in-house resources, and currently available statistical packages:

1. Expand the Stimulus Field. Evaluate the full range of competitor products and prototypes, using consumers, expert panelists, and instrumental measures (Munoz et al., 1996). The database should include consumer ratings of sensory characteristics, liking, images (e.g., more for adult vs more for child), appropriateness for end use, and even fit to concept. Despite the researcher’s reluctance to focus on the entire competitive frame and the easier focus on the key competitor, it is important to take a “snapshot” of all products of a similar type.

The initial exercise to understand the stimulus field attempts to better understand the consumer’s language. The consumer language comprises attributes dealing with the sensory impressions of the product, liking, appropriateness for end uses, other image attributes, sensory “directionals,” and the like. Consumer language is rich in nuances. By incorporating the consumer language as attributes in questionnaires, the developer generates a richer and potentially far more valuable database. These data provide the developer with a report card of the products on all of the key attributes relevant to the consumer. They enable representation of the product in a geometrical configuration (e.g., multidimensional scaling) easily interpreted by marketers and product developers alike.

2. Create Product Models. Develop, where possible, product models (equations) interrelating all of these variables. New techniques have been developed that allow the researcher to create these equations using standard regression methods.

3. Identify Product Opportunities from the Consumer Point of View. A product opportunity comprises a set of sensory attributes and/or formula variables that generate a product having specific properties. These properties may include optimal acceptance, novel sensory characteristics, or a combination of acceptance, image, and appropriateness for a specified end use.

--- PAGE BREAK ---

4. Create Actionable Linkages. Use the product model to link together the consumer (the attributes of the consumer language), the expert, and the instrumental measures. In this way, given the consumer’s language (consumer-rated profile of what the product should be—a subjective measure), the developer can then estimate/discover the formulation/instrumental profile/expert panel profile of the product. This information gives direct, actionable guidance to the developer, even if the consumer cannot articulate what the new product should be. It is from the pattern of consumer responses and the product model that the developer discovers what to do.

5. Implement the Product Model at the Production Level. At the production level, create an additional product model relating instrumental measures to consumer acceptance and sensory characteristics (Moskowitz, 1994). In this way, each batch emerging from production can be tested by instruments, whose profile can be immediately translated by the product model into an estimated consumer acceptance and/or profile. A corollary of this approach is that each batch can now be assessed in terms of meeting consumer acceptance overall, or meeting consumer sensory identity. The product model, acting as the intermediary, ensures this quality of production from the consumer’s point of view.

It is worth noting that the above interaction with the consumers should be initiated as early as possible in the NPD process. Furthermore, it should be repeated at numerous occasions throughout the development stage.

Pillar 3: Multidisciplinary Cross-Functional Team
To comply with and take advantage of the fast changes in the marketplace and new and expanding technologies, skills, and requirements, NPD must effectively use multidisciplinary and cross-functional teams (Gupta and Wilemon, 1996; Hughson, 1997; Szakoyni, 1999; Veganti, 1997). The team, not the individual, integrates all the facets and functions of the company that facilitate the NPD process. The team comprises empowered, capable representatives from all the functions that play a significant role in the process. Clear lines of responsibility, objectives, and deadlines should be provided for the team on a project-by-project basis.

Teams are not static. They are subject to change and optimization by methods often called “re-engineering.” The latter in its classical meaning is defined as “the fundamental rethinking and radical redesign of business processes to achieve dramatic improvements in critical contemporary measures of performance, such as cost, quality, service, and speed.” Re-engineering has been described as “starting with a clean slate,” to emphasize that it is the complete redesign of the process rather than simply the modification of existing processes.

Re-engineering restructures a process or a system to better respond to customer needs, instead of merely forcing the process into a convenient slot in the organization. Re-engineering seeks breakthroughs, not by enhancing existing processes, but by discarding them and replacing them with entirely new ones (Hammer and Champy, 1993; Hooper and Jones, 1998). The multidisciplinary and cross-functional team needs to be able to learn to disagree, to accept differences and diversity, and, more important, to promote “buy-in” and “win–win” approach. The selection of the team members must be done with an eye to providing synergism. Fostering “creativity tension” is important to avoid the “don’t rock the boat” behaviors that could be very accommodating but far from productive or innovative.

A recent study of two multinational companies indicated that re-engineering of R&D functions has often been accompanied by the emergence of groupware applications as strategic decision-making tools, while the organization of work has shifted toward various forms of “team-working.” One outcome of this research was that despite its critical role, learning was not a sufficiently valued competence. As a consequence, the innovations tended to drift, leading to unexpected outcomes, and many opportunities for learning from either mistakes or innovations were simply missed (Ciborra and Patriotta, 1998). Only in a learning organization can NPD flourish and be continuously improved (Hooper and Jones, 1998; Veganti, 1997). This teamwork learning should be an essential and fundamental part of the NPD process.

--- PAGE BREAK ---

Pillar 4: Management Involvement
Management’s active and thoughtful (rather than titular and formal) role is paramount to avoid some of the infamous R&D phases, e.g., enthusiasm, disillusion, panic, search for the guilty, punishment of the innocent, and praise and honors for nonparticipants (Graf and Saguy, 1991). Management provides leadership, crystal-clear strategy and objectives, expectations, responsibility, deadlines, evaluation criteria, climate for innovation, adequate resources, empowerment, executive reviews, and decision mechanisms. In leading companies, all too often the key strategic consideration is a focus on a defined technology and the actionable or operational link of that technology to business objectives.

Intertwined throughout the technology are people. The minimal requirements for success involve the proper selection of team leaders who can function on a multidisciplinary team. “‘Command-and-control” no longer works well, and the old structure of several layers no longer can adapt rapidly or effectively to changes in the food development arena. The new structure needs to use cross-functional business management teams, and must increase its effectiveness by outsourcing with a global perspective to buy talent that it cannot grow or profitably maintain on a fulltime basis (Hollingsworth, 1998).

Management must also truly foster and genially reward team and personal contributions. These contributions must be incorporated fairly and adequately into NPD. Rewards may range from the simple (e.g., a dinner to recognize the efforts and highlight the various contributions) to the more costly (e.g., team trips to an exotic site) or more ego-stroking (e.g., a personalized parking space, a special column in the internal paper).

Management must send continuing and clear messages that it actively supports the development teams (Grunert et al., 1995) and that it stands by its commitment to tolerate mistakes (as opposed to punishing those who dare). It is also important that management foster the true study of both successes and failures in an objective, empirical spirit. Commonly, significant successes constitute the case studies taught at numerous management schools. The more common (and realistic) failures are all too often treated as “orphans,” buried under a pile of secrecy and bureaucracy. If management is indeed committed to providing the necessary environment and tools to improve the success rate of NPD, this astigmatic focus on success alone must change. Learning from mistakes, without recrimination and blame, is difficult for an organization but extremely efficient for the NPD teams. Hence, management needs to formalize a process where success and failure are reviewed, to foster continuous improvement through learning. This recognition of the teaching value of failure will also send a clear message throughout the organization that failures do happen and are tolerated. A feedback loop allowing for failures will energize and optimize NPD.

It is important to note that most large companies try to minimize the risk they are taking. This natural and understandable process tends to hinder innovation. To allow the NPD teams to make a quantum leap toward an innovative product, management should nurture risk-taking and creativity—not simply say it does but act primarily to avert risk.

Based on a paper presented during the Annual Meeting of the Institute of Food Technologists, Chicago, Ill., July 24–28, 1999.

The authors express their gratitude to Heribert Watzke and Peter Leathwood, Nestlé R&D Center, Lausanne, Switzerland, for their important and significant inputs.

I. SAM SAGUY AND HOWARD R. MOSKOWITZ
Author Saguy is Professor, Institute of Biochemistry, Food Science and Nutrition, Faculty of Agriculture, Food and Environmental Quality Science, The Hebrew University of Jerusalem, P.O. Box 12, Rehovot 76100, Israel. Author Moskowitz is President, Moskowitz Jacobs Inc., 1025 Westchester Ave., White Plains, NY 10604. The authors are Professional Members of IFT. Send reprint requests to author Saguy.

Edited by Neil H. Mermelstein,
Senior Editor

References