New Approaches to Idea Generation and Consumer Input in the Product Development

Transcription

1 New roaches to Idea Generation and Consumer Inut in the Product Develoment Process y Olivier Toubia Ingénieur, Ecole Centrale Paris, 000 M.S. Oerations Research, Massachusetts Institute of Technology, 00 SUMITTED TO THE SLON SCHOOL OF MNGEMENT IN PRTIL FULFILL- MENT OF THE REQUIREMENT FOR THE DEGREE OF DOCTOR OF PHILOSOPHY T THE MSSCHUSETTS INSTITUTE OF TECHNOLOGY JUNE 004 Massachusetts Institute of Technology. ll rights reserved. Signature of uthor: Sloan School of Management May 0 th, 004 Certified by: John R. Hauser Kirin Professor of Marketing Thesis Suervisor cceted by: irger Wernerfelt Professor of Management Science Chair, Sloan Doctoral Program

2 New roaches to Idea Generation and Consumer Inut in the Product Develoment Process y Olivier Toubia Submitted to the Sloan School of Management on May 0 th, 004 in artial fulfillment of the requirements for the Degree of Doctor of Philosohy in Management Science STRCT This thesis consists of five related essays which exlore new aroaches to hel design successful and rofitable new roducts. The rimary focus is the front end of the rocess where the roduct develoment team is seeking imroved inut from customers and imroved ideas for develoing roducts based on that inut. Essay examines whether carefully tailored ideation incentives can imrove creative outut. The influence of incentives on idea generation is studied using a formal model of the ideation rocess. ractical, web-based, asynchronous ideation game is develoed, allowing the imlementation and test of various incentive schemes. Using this system, an exeriment is run, which demonstrates that incentives do have the caability to imrove idea generation, confirms the rediction from the theoretical analysis, and rovides additional insight on the mechanisms of ideation. Essay rooses and tests new adative question design and estimation algorithms for artial-rofile conjoint analysis. The methods are based on the identification and characterization of the set of arameters consistent with a resondent s answers. This feasible set is a olyhedron defined by equality constraints, each aired-comarison question yielding a new constraint. Polyhedral question design attemts to reduce the feasible set of arameters as raidly as ossible. nalytic Center estimation relies on the center of the feasible set. The roosed methods are evaluated relative to established benchmarks using simulations, as well as a field test with 330 resondents. Essay 3 introduces olyhedral methods for choice-based conjoint analysis, and generalizes the concet of D-efficiency to individual adatation. The erformance of the methods is evaluated using simulations, and an emirical alication to the design of executive education rograms is described. Essay 4 generalizes the existing olyhedral methods for adative choice-based conjoint analysis by taking resonse error into account in the adative design and estimation of choice-based olyhedral questionnaires. The validity of the roosed aroach is tested using simulations. Essay 5 studies the imact of Utility alance on efficiency and bias. new definition of efficiency (M-efficiency) is also introduced, which recognizes the necessity to match reference questions with the quantities used in the ultimate managerial decisions. Thesis Suervisor: John R. Hauser Title: Kirin Professor of Marketing

3 cknowledgments When I first started being a student at MIT in Setember 999, my goal was to quickly get a Master s in Oerations Research. I had just gotten engaged. Today, on May 9 th 004, I am writing the acknowledgment section of my doctoral dissertation in marketing. My son Eitan has just wished my wife lexandra her first mother s day. What haened? One hyothesis is that I have been brainwashed and maniulated. nother hyothesis is that I have had the immense rivilege of being surrounded with outstanding eole over the ast five years. Let us focus on the second hyothesis. ctually, it started well before Setember 999. My arents have always rovided me with love, suort, and advice. They have taught me imortant values and heled me develo myself, without trying to force me into any redefined ath. The well being of their children (and now, grandchildren) has always been their first riority and they have always suorted our decisions as long as you re healthy and hay and if that s what you want. I hoe to always remember their lesson that taking care of others can be more rewarding that taking care on oneself. My late maternal grandfather, Poal, has been my greatest role model. I have always admired him, his strength, his wisdom, his willower and his ersicacity. I hoe and I believe that he would have aroved the career that I have adoted, and I like to think that he might have made similar decisions in similar situations. My late maternal grandmother, Mamounie, gave me her unconditional love and affection, and transmitted to me the value of academic achievement. Their memories have been with me all along, and will continue to guide my rofessional and ersonal lives. My aternal grandarents Mammy and Daddy have been a model of hoe and otimism, have taught me how to love and cherish life, and have shown me the imortance of family. My brother Didier and my sister Valérie have excelled at the difficult job of having me as a little brother, giving me advice and leading the way for me. Then, of course, came lexandra, who suorted my decision to go study abroad and with whom I have shared all the exciting, hay and uncertain moments of this adventure. She has been my wife, my friend, my comanion and my suorter. I would have never been able to have a balanced life at MIT without her. For me she gave u her hoe for a redictable life, and we are now ready to share whatever will come next. Finally, my cousin Isabelle and her family Philie, Nicolas and Lionel have been our family away from home. It has been great to share holidays with them, and to see Nicolas and Lionel grow. t MIT, I have had the rivilege to meet and work with some incredibly brilliant rofessors and fellow students. First, of course, my advisor, John Hauser, has been a erfect academic father. John has taught me how to aly mathematical tools to the study of marketing henomena, how to balance scientific inquiry with managerial relevance, how to aly the History of science to analyze the worth of a research roject, and how to exose research. His erfectionism, his extremely high ersonal standards, and his dedication to hard work have been insiring. He has been a mentor for me at all levels, training me on research, teaching, and on how to be a good husband and father. He has also been incredibly human, reassuring me in the most troubled times. Duncan Simester has continuously amazed me with his ability to understand, analyze and critique virtually any intellectual argument. He has taught me how imortant it is to aroriately osition a research roject and to isolate the essence of an idea. Ely Dahan has been one of the most influential ersons in my decision to ursue a PhD. eyond a colleague, he has been a friend and we will never forget this 4 th of July weekend with him and his family in Hyannis. I would also like to thank Dan riely for his benevolent feedback on my research, and for his generous funding. He has been an examle of dedication to research, and his PhD seminars fed not 3

4 only my body, but also my brain. Drazen Prelec has suorted me and advised me on my job market aer. I admire his calm and the deeness of his thoughts. I have also been fortunate to interact with John Little, who confirmed my hyothesis that the most brilliant and accomlished researchers are often also the most friendly and caring human beings. Glen Urban also confirmed this hyothesis. He introduced me to teaching and gave me an oortunity to better reare for my next job. irger Wernerfelt, by exosing me to his research aradigm, has forced me to challenge my own and, I believe, to imrove the rigor and the scientific value of my research. JinGyo Kim introduced me to the world of hierarchical ayes and heled me acquire some indisensable tools. I hoe that Shane Frederick made me a slightly smarter erson, by asking all those I.Q. questions at the lunch table. Rosa lackwood gave me some advice and suort, often working extra time for me although she does not even work for me! The marketing grou would not be the same without her good humor and her great cakes. Sandra Crawford-Jenkins shared with me her music and her kindness. I will also surely miss my fellow students. My office mate Wei Wu could not have been more leasant. My former office mate Robert Zeithammer made a little room for me in his office and guided me through my first year. On mir has been like a big brother, giving me some advice and sharing him wisdom with me. Jiwoong Shin has gone through the job market with me and offered me some guidance while studying for the general exams. Leonard Lee has been a lot of fun to be around and to study with. Nina Mazar has brought her sohisticated and delicate touch to the grou. Kristina Shaman er has contributed her wit and critical thinking. Ray Weaver, with his great sense of humor, his M and his two children, has not been a tyical first year student. I hoe that he will graduate before Shane turns him into an addicted gambler. I would like to conclude by thanking a very secial erson, who in just three months has changed my life forever and introduced me to the joys of fatherhood. I am roud to give to my son Eitan Toubia his first Google hit. 4

5 Table of Contents cknowledgments... 3 Chater : Idea Generation, Creativity, and Incentives Introduction...8. Theoretical nalysis n Ideation Game Exeriment Summary and Future Research... 3 References endix : roofs of the roositions Chater : Fast Polyhedral dative Conjoint Estimation Polyhedral Methods for Conjoint nalysis Polyhedral Question Design and Estimation Monte Carlo Simulations Results of the Initial Monte Carlo Exeriments The Role of Self-Exlicated Questions Emirical lication and Test of Polyhedral Methods Results of the Field Test Product Launch Conclusions and Future Research References... 9 endix : Mathematics of Fast Polyhedral dative Conjoint Estimation endix : Internal Validity Tests for Lato Comuter ags... 0 Chater 3: Polyhedral Methods for dative Choice-ased Conjoint nalysis Introduction Existing CC Question Design and Estimation Methods Polyhedral Question Design Methods Monte Carlo Exeriments lication to the Design of Executive Education Programs Conclusions and Research Oortunities

6 Endnotes... 3 References endix: Mathematics of Polyhedral Methods for CC nalysis Chater 4: Non-Deterministic Polyhedral Methods for dative Choice-ased Conjoint nalysis Introduction asic olyhedral methods for Choice-ased Conjoint nalysis Taking resonse error into account Simulations Conclusions and Future Research References Chater 5: Proerties of Preference Questions: Utility alance, Choice alance,... Configurators, and M-Efficiency Motivation Efficiency in Question Design Utility alance for the Metric Paired-Comarison Format Choice alance for the Stated-Choice Format Configurators Generalizations of Question Focus: M-Efficiency Summary and Future Research... 8 References endix

7 Chater : Idea Generation, Creativity, and Incentives bstract Idea generation (ideation) is critical to the design and marketing of new roducts, to marketing strategy, and to the creation of effective advertising coy. However, there has been relatively little formal research on the underlying incentives with which to encourage articiants to focus their energies on relevant and novel ideas. Several roblems have been identified with traditional ideation methods. For examle, articiants often free ride on other articiants efforts because rewards are tyically based on the grou-level outut of ideation sessions. This aer examines whether carefully tailored ideation incentives can imrove creative outut. I begin by studying the influence of incentives on idea generation using a formal model of the ideation rocess. This model identifies conditions under which it is efficient to simly reward articiants based on their contributions, and other conditions under which it is not. I show that in the latter case, the grou s outut can be imroved by rewarding articiants for their imact on the other articiants contribution. I then develo a ractical, web-based, asynchronous ideation game, which allows the imlementation and test of various incentive schemes. Using this system, I run an exeriment, which demonstrates that incentives do have the caability to imrove idea generation, confirms the redictions from the theoretical analysis, and rovides additional insight on the mechanisms of ideation. 7

8 . Introduction Idea generation (ideation) is critical to the design and marketing of new roducts, to marketing strategy, and to the creation of effective advertising coy. In new roduct develoment, for examle, idea generation is a key comonent of the front end of the rocess, often called the fuzzy front end and recognized as one of the highest leverage oints for a firm (Dahan and Hauser 00). The best known idea generation methods have evolved from brainstorming, develoed by Osborn in the 950 s (Osborn 957). However, dozens of studies have demonstrated that grous generating ideas using traditional brainstorming are less effective than individuals working alone (see Diehl and Stroebe 987 or Lamm and Trommsdorff 973 for a review). One cause identified for this oor erformance is free riding (Williams et al. 98, Kerr and ruun 983, Harkins and Petty 98). In articular, articiants free ride on each other s creative effort because the outut of idea generation sessions is tyically considered at the grou level, and articiants are not rewarded for their individual contributions. This suggests that idea generation could be imroved by roviding aroriate incentives to the articiants. Note that this free riding effect is likely to be magnified as firms seek inut from customers (who tyically do not to have a strong interest in the firm s success) at an increasingly early stage of the new roduct develoment rocess (von Hiel and Katz 00). Surrisingly, agency theory aears to have aid little attention to the influence of incentives on agents creative outut. On the other hand, classic research in social sychology suggests that incentives might actually have a negative effect on ideation. For examle, the Hull- Sence theory (Sence 956) redicts an enhancing effect of rewards on erformance in simle tasks but a detrimental effect in comlex tasks. The reason is that rewards increase indiscriminately all resonse tendencies. If comlex tasks are defined as tasks in which there is a redisosition to make more errors than correct resonses, then this redisosition is amlified by rewards. Similarly, the social facilitation aradigm (Zajonc 965) suggests that, insofar as incentives are a source of arousal, they should enhance the emission of dominant, well-learned resonses, but inhibit new resonses, leading eole to erform well only at tasks with which they are familiar. McCullers (978) oints out that incentives enhance erformance when it relies on making simle, routine, unchanging resonses, (. 4) but that the role of incentives is far less 8

9 clear in situations that deend heavily on flexibility, concetual and ercetual oenness, or creativity. McGraw (978) identifies two conditions under which incentives will have a detrimental effect on erformance: first, when the task is interesting enough for subjects that the offer of incentives is a suerfluous source of motivation; second, when the solution to the task is oen-ended enough that the stes leading to a solution are not immediately obvious. (. 34) In this aer I examine whether carefully tailored ideation incentives can imrove creative outut. In Section, I study the influence of incentives on idea generation using a formal model of the ideation rocess. In Section 3, as a ste towards imlementing and testing the conclusions from Section, I resent a ractical, web-based, asynchronous ideation game, which allows the imlementation and test of various incentive schemes. Finally, in Section 4, I describe an initial exeriment conducted using this system. This exeriment demonstrates that incentives do have the caability to imrove idea generation, is consistent with the redictions from the theoretical analysis, and rovides additional insight on the mechanisms of ideation. Section 5 concludes and suggests some ossibilities for future research.. Theoretical nalysis The goal of this section is to develo a theoretical framework allowing the study of idea generation incentives. One characteristic of ideas is that they are usually not indeendent, but rather build on each other to form streams of ideas. This is analogous to academic research, in which aers build on each other, cite each other, and constitute streams of research. Let us define the contribution of a stream of ideas as the moderator s valuation for these ideas (for simlicity I shall not distinguish between the moderator and his or her client who uses and values the ideas). The exected contribution of a stream of ideas can be viewed as a function of the number of ideas in that stream. Let us assume that the exected contribution of a stream of n ideas is a non-decreasing, concave function of n, such that each new idea has a non-negative exected marginal contribution and such that there are diminishing returns to new ideas within a stream. Note that the analysis would generalize to the case of increasing, or inverted U-shaed, returns. (This might characterize situations in which the first few ideas in the stream build some foundations, allowing subsequent ideas to carry the greatest art of the contribution.) The model roosed here is based on Kuhn s notion of an essential tension between convergent thinking and divergent thinking (Kuhn 977, Simonton 988). Divergent thinking 9

10 involves changing ersectives and trying new aroaches to roblems, while convergent thinking relies on linear and logical stes and tends to be more incremental. Kuhn claims that since these two modes of thought are inevitably in conflict, it will follow that the ability to suort a tension that can occasionally become almost unbearable is one of the rime requisites for the very best of scientific research. (. 6) lthough Kuhn s focus is the history of science, his insights aly to idea generation. If several streams of ideas have been established in an idea generation session, a articiant has a choice between contributing to one of the existing streams, and starting a new stream. If he or she decides to contribute to an existing stream, he or she might have to choose between contributing to a longer, more mature stream, and a newer, emerging one. If we assume that each stream of ideas reflects a different aroach to the toic of the session, then the roblem faced by the articiant can be viewed equivalently as that of deciding which aroach to follow in his or her search for new ideas. I assume that some aroaches can be more fruitful than others, and that the articiant does not know a riori the value of each aroach but rather learns it through exerience. He or she then has to fulfill two otentially contradictory goals, reflecting Kuhn s essential tension: increasing short-term contribution by following an aroach that has already roven fruitful, versus imroving long-term contribution by investigating an aroach on which little exerience has been accumulated so far. For examle, let us consider a grou of articiants generating ideas on How to imrove the imact of the UN Security Council? (which was the toic of the exerimental sessions reorted in Section 4). Let us assume that at one oint in the session, two streams of ideas have emerged: a long stream, comosed of ten ideas, all aroaching the toic from a legal standoint, and a shorter stream, with only two ideas, both aroaching the toic from a monetary standoint. Each articiant then has to decide whether to search for new ideas following a legal, or a monetary aroach to the toic. The legal aroach has already roven fruitful, whereas the monetary aroach is more risky but otentially more romising. 0

11 This roblem is tyical in decision making and is well catured by a reresentation known as the multi-armed bandit model (ellman 96). For simlicity, let us first restrict ourselves to a two-armed bandit, reresenting a choice between two aroaches and. Let us assume that by sending one unit of time thinking about the toic using aroach (resectively ), a articiant generates a relevant idea on the toic with unknown robability (resectively ). Particiants hold some beliefs on these robabilities, which are udated after each trial. Note that starting a new stream of ideas can be viewed as a secial case in which a articiant tries an aroach that has not been used successfully yet. Let us comlement this classical multi-armed bandit model by assuming that when N trials of aroach have been made, a roortion idea found using aroach has contribution of which were successful, then the next (similarly for aroach ). This assumtion catures the fact that new ideas in a stream have diminishing marginal contribution (0 α by assumtion). In the context of our revious examle, this assumtion imlies that the third idea following the monetary aroach is likely to have a greater marginal contribution than the eleventh idea following the legal aroach, which increases the attractiveness of the riskier monetary aroach. α N Some roblems (like mathematical roblems or uzzles) have objective solutions. Once a solution has been roosed, there is usually little to build uon. Such toics would be reresented by a low value of α. On the other hand, many roblems considered in idea generation are more oen-ended (trying to imrove a roduct or service) and are such that consecutive ideas refine and imrove each other. Such situations corresond to higher values of α. The arameter α could also deend on the references of the user(s) of the ideas. For examle, the organizer of the session might value many unrelated, rough ideas (low α), or, alternatively, value ideas that lead to a deeer understanding and exloration of certain concets (high α). To analyze the tension in ideation, I first identify and characterize three stages in the idea generation rocess. Next I consider the simle incentive scheme consisting of rewarding articiants based on their individual contributions, and show that although it leads to otimal search when α is small enough, it does not when α is large. For large α, this suggests the need for incen- andit is a reference to gambling machines (slot machines) that are often called one-armed bandits. In a multi-armed bandit, the machine has multile lever (arms) and the gambler can choose to ull any one of them at a time.

12 tives that interest articiants in the future of the grou s outut. I next show that, although such incentives allow aligning the objectives of the articiants with that of the moderator of the session, they might lead to free riding. voiding free riding requires further tuning of the rewards and can be addressed by rewarding articiants more recisely for the imact of their contributions. Three stages in the idea generation rocess Let us consider a two-eriod model, in which a single articiant searches for ideas, and study the conditions under which it is otimal to follow the aroach that has the higher versus the lower rior exected robability of success. For simlicity, I shall refer to the former as the better-looking aroach and to the latter as the worse-looking aroach. I assume that the rior beliefs on and at the beginning of eriod are indeendent and such that ~eta(n S, n F ) and ~eta(n S, n F ), with n S, n F, n S, n F. The corresonding exected robabilities of success are =E()=n S /N ; =E()=n S /N, where N = n S + n F and N = n S + n F can be interreted as the total number of trials observed for and resectively rior to eriod. 3 For these assumtions the variance of the beliefs is inversely roortional to the number of trials. Two effects influence this model. First, if the uncertainty on the worse-looking aroach is large enough comared to the uncertainty on the better-looking aroach, it might be otimal to choose the worse-looking aroach in eriod although it imlies a short-term loss. In articular, if the exected robability of success of the uncertain, worse-looking aroach is close enough to that of the more certain, better-looking aroach, a success with the worse-looking aroach in eriod might lead to udated beliefs that will make this aroach much more aealing than the currently better-looking one. For examle, let us assume that the better-looking aroach at the beginning of eriod,, has a very stable exected robability of success which is around 0.6. y stable we mean that These beliefs can be interreted as corresonding to a situation in which the articiant starts with some initial rior uniform beliefs on and ( ~eta(,) and ~eta(,)) and observes, rior to eriod, n S - successes for, n F - failures for, n S - successes for, and n F - failures for. ecause beta riors are conjugates for binomial likelihoods (Gelman et al. 995), the osterior beliefs after these observations (which are used as riors at the beginning of eriod ) are ~eta(n S, n F ) and ~eta(n S, n F ). 3 The uniform rior can be can be interreted as resulting from the observation of one success and one failure for each aroach.

13 it will be similar at the beginning of eriod whether a success or failure is observed for this aroach in eriod (N is very large). Let us assume that the worse-looking aroach,, has an exected robability of success of 0.60 at the beginning of eriod. However a success with this aroach would increase the osterior robability to 0.67 while a failure would decrease this osterior robability to Then, laying in eriod would lead to a total exected contribution of =. (ignoring discounting for simlicity). Playing in eriod would lead to an exected total contribution of * *0.6=.5 (lay in eriod iff it was successful in eriod ), which is higher. Second, if the number of trials (and successes) on the better-looking aroach gets large, this aroach becomes over-exloited (due to the decreasing marginal returns) and it might become more attractive to choose the worse looking aroach: although the robability of success is lower, the contribution in case of success is higher. Let us define (we define similar quantities for ): ST() the exected contribution from eriod obtained by choosing aroach in eriod (ST stands for short-term). LT() the exected contribution from eriod, calculated at the beginning of eriod, if is layed in eriod. y definition, it is otimal to lay in eriod if and only if ST()+LT() ST()+LT(). If is such that >, then ST()>ST() unless has been over-exloited and ST() ST(). The following roositions characterize three stages in the idea generation rocess, labeled exloration, exloitation, and diversification. If has been over-exloited, diversification is otimal and should be layed in eriod. When has not been over-exloited yet, then if α is small enough, the exloitation of is always otimal. In this case, exloring would be too costly: at the same time as the articiant would learn more about, he or she would also heavily decrease the attractiveness of this aroach. On the other hand, when α is large enough, then if is uncertain enough comared to, the exloration of might be otimal in eriod. 4 These robabilities would result from N =5, n S =3, n F =, imlying that =3/5 and that after a success is udated to 4/6. 3

14 Exloration is characterized by a short-term loss incurred in order to increase long-term contribution. Proosition a: For all, N, α<, for all close enough to such that <, if has been over-exloited (ST() ST()), then diversification is otimal in eriod. Proosition b: For all,n, for all α close enough to 0, for all close enough to such that <, if has not been over-exloited (ST()>ST()), then the exloitation of is otimal in eriod. Proosition c: For all, N, for all α close enough to such that α<, for all close enough to such that <, there exists 0<N* <N ** (0<V **<V *) such that The exloitation of is otimal if N N * (Var( ) V *). The exloration of is otimal if N * N N ** (V ** Var( ) V *). Diversification is otimal if N ** N (Var( ) V **). The roofs to the roositions are in endix. Note that the above roositions focus on the more interesting case where the exected robabilities of success of the two aroaches are close. The identification and characterization of these three stages rovides a useful background for the study of idea generation incentives. In articular, short-term focus is otimal in the diversification and exloitation stages, such that articiants trying to maximize their own contributions behave as if they were maximizing the grou s outut. In the exloration stage, however, articiants have to forego some short-term contribution in order to increase the grou s future outut. In this case roblems arise if articiants are not given incentives to internalize the influence of their resent actions on the future of the grou. 4

15 Rewarding articiants for their individual contributions Consider the incentive scheme that consists of rewarding articiants based on their individual contributions. In ractice, this could be oerationalized by having an external judge rate each idea sequentially, and rewarding each articiant based on the ratings of his or her ideas. This scheme is robably among the first ones that come to mind, and addresses the free-riding issue mentioned in the introduction. There are also many situations in which individual contribution is likely to be the most sohisticated metric available. It is then useful to identify some conditions under which basing incentives on this metric leads articiants to behave in a way that maximizes the grou s total contribution, and conditions under which it does not. Let us introduce a second articiant into the model. ssume that, at each eriod, both layers simultaneously choose one of the two aroaches. t the end of eriod, the outcomes of both layers searches are observed by both layers, and layers udate their beliefs on and. 5. Each layer s ayoff is roortional to his or her individual contribution, i.e., to the sum of the contributions of his or her ideas from the two eriods. The contribution of the grou is equal to the sum of the contributions of each layer. Note that α alies equally for both layers, such that if a layer finds an idea using a certain aroach, the marginal return on this aroach is equally decreased for both articiants. The assumtion that layers indeendently choose which aroach to follow and then share the outut of their searches is robably more descritive of electronic ideation sessions (in which articiants are seated at different terminals from which they tye in their new ideas and read the other articiants ideas) than it is of traditional face-toface sessions. s will be seen later, electronic sessions have been shown to be more effective than face-to-face sessions (Gallue et al. 99, Nunamaker et al. 987). Let us consider the subgame-erfect equilibrium (SPE) of the game layed by the articiants, as well as the socially otimal strategy, i.e., the strategy that maximizes the total exected contribution of the grou. 5 Equivalently, we could assume that layers reort the outcome of their t= search only if it is successful, by submitting a new idea. In this case if a layer does not submit an idea at the end of eriod, the other layer can infer that her search was unsuccessful. Finally, we could interret this assumtion as meaning that success and failure corresond to good and fair ideas, and that both tyes of ideas are worth submitting. 5

16 Low-α case (raidly decreasing returns) When α is low, the marginal contribution of successive ideas following the same aroach diminishes quickly. In this case, exloration is never otimal, which lowers the value of grou interactions. Consequently, actions that are otimal for self-interested articiants trying to maximize their own contributions are also otimal for the grou. This is catured by the following roosition: Proosition : For all N, N, there exists α* such that 0<α<α* for all, such that < and is close enough to, if articiants are rewarded based on their own contributions, then a strategy (x,y) is layed in eriod in a SPE of the game iff it is socially otimal. High-α case (slowly decreasing returns) When α is close to, there are situations in which exloration is otimal. However, exloration results in a short-term loss suffered only by the exlorer, and a otential long-term benefit enjoyed by the whole grou (articiants are assumed to share their ideas). There are then some cases in which it is socially otimal that at least one layer exlores in eriod, but no layer will do so in any SPE of the game. This is illustrated by the following roosition when α=: Proosition 3: If α=, then for all, for all < close enough to, there exists Nlow<N high (V low <V high ) such that if N is large enough (Var( ) low enough), then N <N high (Var( )>V low ) imlies that it is not socially otimal for both layers to choose in eriod (at least one layer should exlore) N low <N <N high (V low <Var( )<V high ) imlies that both layers choose in eriod in any SPE of the game (no layer actually exlores) 6

17 Interesting articiants in the future of the grou s outut Proosition 3 demonstrates a otential misalignment roblem when building on ideas matters, that is, when the marginal returns to subsequent ideas are high. In order to overcome this roblem, articiants should be forced to internalize the effect of their resent actions on the future of the other articiants contribution. To study the dynamic nature of the incentives imlied by this observation, let us consider a more general framework, with an infinite horizon, N articiants, and a er-eriod discount factor δ. ssume that articiant i s outut from eriod t, y it, has a robability density function f( y it x t,..., x,( y jτ ) j=... N, τ =... t ), where xjτ is articiant j s action in eriod τ, and that articiant i s contribution from eriod t is C it =C ) ). N t (( y jτ j=... N, τ =... t The two-armed bandit model considered earlier is a secial case of this general model. Interesting articiants in the grou s future outut allows aligning the incentives, but it might also lead to free-riding. To illustrate this, let us consider the incentive scheme that consists of rewarding a articiant for his or her action in eriod t according to a weighted average between his or her contribution in eriod t and the other articiants contribution in eriod t+. 6 More recisely, layer i receives a ayoff for his or her action in eriod t roortional to[ γ. Cit + ( γ ). δ. C j, t+ ]. Let us call S(γ) such a scheme. We have the following roosition: j i Proosition 4: In any infinite horizon game with discount factor δ<, S(γ) aligns the objectives of the moderator and the articiants for any form of the contribution and outut functions iff S(γ) is equivalent to a roortional sharing rule in which layer i receives a ayoff for his or her actions in eriod t roortional to j C. jt Intuitively, in eriod t articiant i gets, in addition to a share of C it as a reward for his or her actions in eriod t, a share of the other articiants contribution from eriod t ( reward for his or her actions in eriod t-. However, the fact that this reward is attributed to his ) as a or her action in eriod t- is irrelevant in eriod t, since this action has already been taken. Hence j i C j, t 6 Proosition 4 would also hold with a weighted average between the articiant s contribution in eriod t and a discounted sum of the other articiants contribution in subsequent eriods. 7

18 the scheme is equivalent to giving the articiant in eriod t a reward equal to a weighted average between his or her contribution in eriod t and the other articiants contribution in the same eriod. The equivalence to a roortional sharing rule in this examle illustrates how grou incentives align the objectives at the same time as they introduce free riding (Holmstrom 98), even in a dynamic setting. In articular, although the objective function of the articiants is roortional to that of the moderator, it is ossible for a articiant to be aid without articiating in the search. This imoses a cost on the moderator. More recisely, let us assume that at each eriod, layers have an otion not to search for ideas, or to search at a er-eriod cost c, and that the first-best level of effort is for all the layers to erform a search at each eriod. Then if the reward given to articiant i at eriod t is equal to β. C jt, β has to be high enough for the incentive comatibility (IC) constraints of the ar- j ticiants to be satisfied (i.e., such that each articiant is willing to search in each eriod). In this case free riding forces the moderator to redistribute a higher share of the created value to the articiants. 7 Rewarding Particiants for their Imact In order to address free riding while still aligning the objectives, one otion is to reward each articiant more recisely for that art of the grou s future contribution that deends directly on his or her actions, i.e., for his or her imact on the grou. In articular, the following roosition considers a modification of the above scheme that rewards articiant i for his or her action in eriod t based on a weighted average between his or her contribution in eriod t, and the imact of this action on the grou s future contribution (let us denote such a scheme by S (γ)). Under this alternative scheme, the objectives of the articiants remain aligned with that of the moderator, because the only comonent removed from the articiant s objective function is not under his or her control, and free riding is reduced, because a articiant does not get rewarded for the other articiants contribution unless it is tied to his or her own actions. This reduces the share of the created value that needs to be redistributed to the articiants. (The imact 7 Note that I assume that the moderator s valuation for the ideas is high enough comared to c, such that it is ossible and otimal for him or her to induce the articiants to search in each eriod. 8

19 of articiant i s action in eriod t is defined here as: + δ τ t λ( t, τ ) τ = t+ j i [ C jτ ~ C jτ ( i, t)] where ~ C jτ ( i, t) is the exected value of C jτ obtained when y it is null and all layers maximize the t τ grou s exected outut at each eriod, and λ(t,τ)= δ.( δ ) ( δ δ ) is such that τ t= λ( t, τ ) = for all t and τ). Proosition 5: S (/) is such that the objectives of the articiants are aligned with that of the moderator, and the total exected ayoffs distributed in the session are lower than under a roortional sharing rule. Summary of this section Three stages can be defined in the idea generation rocess: exloration, exloitation and diversification. When the marginal contribution of new ideas following the same aroach decreases quickly, exloration is never otimal and rewarding articiants based on their individual contributions leads to otimal search. When the marginal contribution of new ideas following the same aroach decreases slowly, exloration can be otimal and rewarding articiants based on their individual contributions leads to subotimal search. In the latter case, the grou s outut can be imroved by rewarding each articiant based on a weighted average between his or her individual contribution and his or her imact on the grou s contribution. This last result can be translated into a testable hyothesis. First, let us note that this result, like the others summarized above, assumes that all articiants search for ideas at each eriod. Hence the different incentive schemes should be comared holding articiation constant. The following hyothesis, which will be examined in the exeriment in Section 4, is then sufficient for the result to hold: 9

20 Hyothesis: For a given level of articiation, if the marginal contribution of new ideas following the same aroach is slowly diminishing, the total contribution of the grou is greater if articiants are rewarded for their imact than it is if they are rewarded for their individual contributions. The alication and test of the insights from this theoretical analysis necessitate an idea generation system that is comatible with the incentive schemes considered in this section, i.e., which allows the measurements necessary to their imlementation. Moreover, for the theoretical results to be managerially relevant, grous generating ideas using this system should erform better when incentives are resent than when they are not, and erform better than grous or individuals generating ideas with other established methods. In articular, this idea generation system should not inhibit, and ossibly enhance articiants creative abilities. Section 3 rooses a articular system. 3. n Ideation Game I now describe an incentive-comatible ideation game that allows testing and imlementing the insights derived in Section. In this game, articiants score oints for their ideas, the scoring scheme being adjustable to reward individual contribution, imact, or a weighted average of the two. The design of the game is based on the idea generation, bibliometric, and contract theory literatures. Table rovides a summary of the requirements imosed on the system and how they were addressed. Table Ideation game requirements and corresonding solutions Requirement ddress Production locking ddress Fear of Evaluation Measure contribution and imact Prevent cheating Proosed solution synchronous nonymous Objective measures Mutual monitoring Ideas structured into trees Relational contract Mutual monitoring 0

21 ddressing fear of evaluation and roduction blocking lthough the rimary requirement imosed on this idea generation system is to allow imlementing and testing the incentive schemes treated in Section, it should also be comatible with the existing literature on idea generation. In articular, two main issues have been identified with classical (face-to-face) idea generation sessions, in addition to free riding (Diehl and Stroebe 987). The first one, roduction blocking, haens with classical idea generation sessions when articiants are unable to exress themselves simultaneously. The second one, fear of evaluation, corresonds to the fear of negative evaluation by the other articiants, the moderator, or external judges. These two issues have been shown to be reduced by electronic idea generation sessions (Gallue et al. 99, Nunamaker et al. 987), in which articiation is asynchronous (therefore reducing roduction blocking) and in which the articiants are anonymous (therefore reducing fear of evaluation). The online system roosed here is anonymous and asynchronous as well. In articular, articiants create (or are given) an anonymous login and assword, and log on to the idea generation session at their convenience (the session lasts tyically for a few days). Measuring the contribution and the imact of articiants One simle way to measure the contribution and imact of articiants would be to have them evaluated by external judges. However, this aroach has limitations. First, it is likely to trigger fear of evaluation. Second, the evaluation criteria of the moderator, the judges and the articiants might differ. Third, it increases the cost of the session. The roosed ideation game instead adots a structure that rovides objective measures of these quantities. These objective measures do not require additional evaluation, are comuted based on well-stated rules, and offer continuous feedback on the erformance of the articiants. ibliometric research suggests that the number of ideas submitted by a articiant should be a good measure of his or her contribution, and that his or her number of citations should be a good measure of his or her imact (King 987). The roosed structure adats the concet of citation count from the field of scientific ublications (which are the focus of bibliometry), to idea generation. n examle is rovided in Figure. Ideas are organized into trees, such that the son of an idea aears below it, in a different color and with a different indentation (one level further to the right). The relation between a father-idea and a son-idea is that the son-

22 idea builds on its father-idea. More recisely, when a articiant enters a new idea, he or she has the otion to lace it at the to of a new tree (by selecting enter an idea not building on any revious idea ) or to build on an existing idea (by entering the identification number of the fatheridea and clicking on build on/react to idea #... ). Note that an idea can have more than one son, but at most one father (the investigation of more comlex grah structures allowing multile fathers is left for future research). The assignment of a father-idea to a new idea is left to the author of the new idea (see below how incentives are given to make the correct assignment). Figure Examle of an ideation game The imact of an idea y can then be measured by considering the ideas that are lower than y in the same tree. In the exeriment described in the next section, imact was simly measured by counting these descendant ideas. More recisely, articiants rewarded for their imact scored one oint for each new idea that was entered in a tree below one of their ideas. For examle, if idea # built on idea #, and idea #3 on idea #, then when idea #3 was osted, one oint was given to the authors of both idea # and idea #. Particiants rewarded for their own contri-

23 butions scored one oint er idea. The exloration of alternative measures is left for future research. If a articiant decides to build on an idea, a menu of conjunctive hrases (e.g. More recisely, On the other hand, However ) is offered to him or her (a blank otion is also available), in order to facilitate the articulation of the links between ideas. Pretests suggested that these hrases were useful to the articiants. Preventing cheating lthough this structure rovides otential measures of the contribution and imact of the different articiants, if the scoring system were entirely automatic and if no subjective judgment were ever made, articiants could cheat by simly osting irrelevant ideas in order to score unbounded numbers of oints. Consequently, some level of subjective judgment seems inevitable. These judgments, however, should kee fear of evaluation to a minimum, and limit costly interventions by external judges. This is addressed by carefully defining the owers of the moderator of the session. First, the moderator does not have the right to decide which ideas deserve oints. However, he or she has the ower to exel articiants who are found to cheat reeatedly, i.e., who submit ideas that clearly do not address the toic of the session or who wrongly assign new ideas to father-ideas. This tye of relation between the articiants and the moderator is called a relational contract in agency theory. In this relationshi the source of motivation for the agent is the fear of termination of his or her relation with the rincial (aker at al. 998, Levin 003). In the resent case, if the moderator checks the session often enough, it is otimal for articiants to only submit ideas which they believe are relevant. In order to reduce the frequency with which the moderator needs to check the session, this relational contract is comlemented by a mutual monitoring mechanism (Knez and Simester 00, Varian 990). In articular, if articiant i submits an idea which is found to be fraudulent by articiant j, then j can challenge this idea. 8 This freezes the corresonding stream of ideas (no articiant can build on a challenged idea), until the moderator visits the session and determines who, between i and j, was right. 9 The articiant who was judged to be wrong then ays a fee to the other articiant. The amount of this fee was fine-tuned based on retests and was set 8 In the imlementation used so far, articiants cannot challenge an idea if it has already been built uon. 9 The idea is deleted if j was right. 3

24 to 5 oints in the exeriment. This mechanism is economical because it reduces the frequency with which the moderator needs to visit the session, without increasing its cost, since challenges result in a transfer of oints between articiants (no extra oints need to be distributed). eyond the cost factor, another argument in favor of having articiants monitor each other is the finding by Collaros and nderson (969) that fear of evaluation is greater when the evaluation is done by an exert judge rather than by fellow articiants. Note that in equilibrium, there should be no challenges, because articiants should only submit relevant ideas and challenge irrelevant ideas. Indeed, in the exeriment, there was only one challenge. Practical imlementation This ideation game was rogrammed in h, using a MySQL database. Its structure and the instructions were fine-tuned based on two retests. The retests used articiants from a rofessional marketing research/strategy firm. One retest addressed an internal roblem (imroving the worksace) and a second retest addressed an external roblem (how to re-enter a market). In the imlementation, the moderator also has the ability to enter comments, either on secific ideas or on the overall session. Particiants also have the ability to enter comments on secific ideas. This allows them, for examle, to ask for clarification or to make observations that do not directly address the toic of the session. The difference between comments and ideas is that comments are not required to contribute ositively to the session, and are not rewarded. Finally, note that the imlementation choices (e.g., amount of the fee in case of a challenge), although determined by the retests, are likely to be imroved with further exerimentation. 4. Exeriment The exeriment resented in this aer has two rimary goals. The first goal is to test whether incentives have the ower to imrove the outut of idea generation sessions. Recall that some research in social sychology suggests that incentives will have a negative influence on erformance. The second goal is to verify the hyothesis, stated in Section, that under certain conditions, the total contribution of the grou can be imroved by rewarding each articiant for the imact of his or her contribution, as oosed to his or her own contribution. This exeriment also allows studying further the influence of incentives on the dynamics of idea generation. 4