X Incentivizing Participation in Online Forums for Education


 Octavia Patrick
 1 years ago
 Views:
Transcription
1 X Incentivizing Participation in Online Forums for Education Arpita Ghosh, Cornell University Jon Kleinberg, Cornell University We present a gametheoretic model for online forums for education, where students in a class can post questions to the forum and seek responses from the instructor or other students in the class. We first show that our model predicts the anecdotally observed phenomenon that students participation in a forum is nonmonotone in the instructor s response rate to questions: if the instructor responds at too high a rate students do not respond to their peers questions, whereas an almostabsent instructor induces very little participation from the students as well. We then investigate the optimal use of a forum for two kinds of questions singleanswer questions and discussionstyle questions which lead to different levels of rewards that can be meaningfully offered. We show that for discussionstyle questions, the instructor can choose her response rate so that the expected rate at which the first student response is received increases linearly with the size of the class; for singleanswer questions, however, the optimal expected rate of arrival of the first student response remains a constant even as class size diverges. However, this slow response rate can be remedied by mixing question types as long as there is any positive probability of a discussiontype question in a forum, the instructor can choose her response rate so that the equilibrium rate of the first response from the class diverges with the number of students in the class. 1. INTRODUCTION The increasing use of online applications for education raises a spectrum of research issues ranging across the delivery of course content, the evaluation of student work, and the development of resources to stimulate discussion and address student questions. We focus on resources of this latter type online forums for education oriented around discussion and questionanswering which are increasingly found in a number of different educational contexts. In many cases, they exist as supplements to standard offline classes through sites such as Piazza a rapidly growing online platform for education forums with over 500 participating classes and already used by over 100,000 students which provide students with a separate online venue for asking questions and receiving help. In other cases, they are being explored as a key component of massive open online courses (MOOCs), creating communities of students who can discuss course content online. An important property of such forums, when they work well, is that they are not simply channels for instructors to provide information to students they also involve students providing significant help to each other, thereby tapping into the peerlearning potential of a large class which, research in education shows, can provide significant learning benefits to students [Topping and Ehly 1998]. Students arrive at the forum with questions or topics they want help with, and fellow students, together with the instructor, provide followup discussion and answers. These education forums have much in common with online questionanswering (Q&A) sites such as StackOverflow, MathOverflow, or Yahoo! Answers, but also differ in key ways which play important roles in their design: Online forums for education have an instructor who can if she chooses play a crucial and visible role in the operation of the forum. In addition to providing answers of her own, the instructor can act as a global monitor of the site, and reward students for participation on the forum, both via explicit incentives such as class grades, as well as via socialpsychological rewards from instructor approval or endorsement. For these reasons, an instructor in an online forum for educa Authors s: Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies show this notice on the first page or initial screen of a display along with the full citation. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any component of this work in other works requires prior specific permission and/or a fee. Permissions may be requested from Publications Dept., ACM, Inc., Penn Plaza, Suite 701, New York, NY USA, fax +1 (1) , or c 013 ACM /013/0ARTX $15.00 DOI:http://dx.doi.org/ /
2 X: tion has access to incentives that operate particularly powerfully relative to moderators on general Q&A sites, who tend to wield a qualitatively lower level of influence than the instructor in an education forum. The presence of an instructor who can monitor and potentially penalize poorquality contributions, as well as the presence of social norms similar to those in a classroom, mean that contribution quality, while still an issue, is arguably much less heterogeneous than in general online Q&A forums on the Web. Indeed, low effort and junk contributions appear to be significantly less common in Piazza and other education forums in comparison with generalpurpose forums like Y! Answers. From the growing use of online forums for education, we see that they can operate well with specific questions getting resolved quickly, and openended discussion questions receiving a strong flow of contributions. But they can also operate poorly, with little student participation and questions that go unanswered for long periods of time. An instructor would like to make use of an online forum so as to promote better outcomes. How can an instructor use her own behavior to incentivize higher student participation? For example, if the instructor actively appears on the site, does this tend to increase student participation (because they are hoping to get the instructor s approval) or decrease it (because they assume that the instructor will answer the questions herself)? Is there an optimal way to trade off these two forces, and how does the best outcome the instructor can elicit through her behavior depend on the size of the class as the class size becomes large, can the instructor step back and let the class do most of the work on the forum? While such questions appear at the heart of discussions about best practices in these forums [Andresen 009; Burge et al. 000; Gray 004; Hanover Research Council 009; Mazzolini and Maddison 003], there has so far not been a formal way of reasoning about them Overview of results We develop and analyze a gametheoretic model of student participation in online forums for education, and use this model to address questions regarding the optimal use of such forums. While a range of factors, including psychology and user interface design, will clearly play a role in participation on these forums, we argue using our model that a gametheoretic analysis of incentives can provide a useful framework for understanding many of the tradeoffs that have emerged in discussions of education forums thus far. In particular, our model requires very few ingredients to define formally yet exhibits rich behavior, including some of the subtle tradeoffs in class size, instructor activity, and instructor effort level that have been observed qualitatively in early empirical studies of these forums. There is a class of n students, and an instructor, each of whom chooses rates at which to check the class s online forum for new content; a rate of λ corresponds to repeatedly checking the forum according to a Poisson process with rate λ, truncated to a finite time interval ( ). Checking the forum more frequently, i.e., at a higher rate, incurs a higher cost, but potentially also leads to higher benefits, as follows. A student who answers a question is rewarded by the instructor for (usefully, and correctly) answering a question on the forum this reward could either be points awarded by the instructor that count towards the student s grade in the class, or a socialpsychological reward deriving from approval, or endorsement, by the instructor. An instructor rewards students only for answering questions in time, i.e., for answering new questions that have arrived since the last time she visited the forum, as explained in. (Of course, the instructor could choose to reward students for delayed answers as well, but as our analysis will show, this does not create the right incentives for participation). Given a rate µ chosen by the instructor, each student therefore chooses the rate λ at which he checks the forum to trade off cost and benefit: choosing a high value of λ comes at a large cost in effort (the student needs to be frequently checking the forum), but increases his expected benefit since there is a higher chance that he will see and answer questions in time to receive rewards from the instructor. In addition, if the nature of the question is such that multiple answers are redundant, a higher rate also increases the student s chance of providing a useful, nonredundant answer, since
3 only useful answers are rewarded by the instructor. This gives us a game where the instructor moves first by choosing her rate µ, and the students move second, choosing their rates λ as a best response to µ, and possibly also as a best response to each other s rates, depending on the type of the question. We explore how an instructor s choice of behavior, modeled by her rate choice µ, affect both her own utility the difference between the welfare of the student asking the question and the instructor s cost from her rate choice as well as the overall behavior of the forum, for two representative question types. We obtain an essentially complete characterization of the forum s behavior for both types of questions, which turn out to lead to very different outcomes. Openended questions. Openended or discussionstyle questions are those where a student can generate value by answering even if other students have already provided answers earlier, so any student who answers the question in time receives a reward. So here the students are not competing with each other, but only racing against the instructor. We find that the student s response rate λ for openended questions is nonmonotone in the instructor s choice of rate µ: as µ increases upward from 0, it initially causes the student to move faster (to answer the question in time), but as µ becomes too large, the reward from answering is no longer worth the cost it would require to make λ sufficiently large, and the student s choice of rate λ begins dropping back toward 0. We note that this nonmonotonic dependence of the student s rate λ on the instructor s rate µ has been motivated in earlier writings about educational forums [Andresen 009; Gray 004; Hanover Research Council 009] on qualitative grounds: when students see a highly active instructor on the forum, they conclude that it s either hopeless or not useful to keep up, and they start to drop out. 1 What is striking here is that this qualitative reasoning is captured formally within our model by the optimization performed by strategic agents. Singleanswer questions. Singleanswer questions are those with a short welldefined correct answer, where only the first student provides a nonredundant answer and therefore earns a reward. So here, students compete amongst themselves to be the first one to provide an answer, in addition to racing to arrive before the instructor in order to receive their reward. Thus for any announced rate µ by the instructor, the students are playing a simultaneousmove game in which their strategies are their choices of rate. We show that for every µ, this game has a unique symmetric Nash equilibrium in which all students choose the same rate λ, and we study this rate as a prediction for how the forum will operate and how the instructor should choose µ. In contrast to the case of openended questions where the rate of arrival of the first answer scales linearly with class size n, we find that in equilibrium, the first answer to such singleanswer questions arrives at a rate that is almost independent of the class size n in equilibrium each student in a large class slows down his own rate λ to an extent great enough that it offsets the gains from having many students trying to answer the question. Moreover, this slow race among the students is very hard for the instructor to influence through her choice of µ: for n, the equilibrium value λ is maximized at µ = 0. This principle that a large class slows itself down suggests interesting potential limitations on the power of large classes to provide crowdsourced responses in the singleanswer case, and arises from an underlying mechanism that is intuitive: in a large class, each student reasons that his chance of being the first to answer a (singleanswer) question is so low that it is not worth checking the class forum very often if it only contains singleanswer questions. We then investigate the instructor s utility U I (µ), which is the the utility derived by the question asker minus the instructor s effort cost from checking the forum at rate µ. We discover that as a function of µ, the instructor s utility U I (µ) can in fact be bimodal, with two local maxima: the optimal utility is achieved either by choosing µ 0 and letting the race among the students do its 1 Quoting from [Hanover Research Council 009], During the discussion process, it is important that instructors continuously manage students ideas and further facilitate interactions. However, if the online discussion is going well without instructor feedback, it is often best for teachers to wait to jump into the discussion until the students responses are waning.... Or in the words of an online forum moderator in Gray [Gray 004], I was conscious of Burge, Laroque, and Boak s (000) caution that moderators could not afford to get trapped into the Atlas syndrome of holding up the discussion world and consciously waited for other voices to be heard rather than making postings every day. X:3
4 X:4 work, or by choosing a particular µ > 0 in which the students are essentially driven out of the forum and the instructor answers the questions herself. Which of these two local maxima is higher depends on how the instructor values the cost of her own effort. In between, there is an intriguing local minimum in the utility where, essentially, the instructor is using the forum the wrong way in which the instructor s rate µ suppresses the students effort in a manner that doesn t increase the overall rate at which the question gets answered. The fact that the model produces a local minimum between two more efficient modes of operation is intriguing in the way it aligns with empirical evidence from online forums for education. While there are some differences in setting, Andresen discusses the basic tradeoff between a mode of operation in which students interact with each other, and a mode in which the instructor principally answers the questions on her own [Andresen 009]: In a study that asks the question of what role an instructor should undertake in an asynchronous discussion forum (sage, guide, or ghost), Mazzolini and Maddison (003) found that it depends on what the instructor wishes to accomplish. Learner ratings of a course will show that an instructor is more enthusiastic and expert if s/he increases his/her postings.... However, an instructor that contributes significantly to a discussion tends to decrease the length of discussions (this does not necessarily decrease the quality of the discussion, however) as well as their frequency. What appears to be occurring in this situation is that the instructor can decrease learnerlearner interaction because the learners begin to rely on the instructor to answer questions, becoming the expert or sage to settle debates. Analyzing Mixtures. Finally, we address the question of how an instructor might elicit higher rates from students for singleanswer questions. Recall that each student s rate parameter λ, is a global rate that governs how frequently he checks the forum for the class; by definition it is chosen before any particular question is seen. We show that one way to achieve a speedup with class size for singleanswer questions is to make sure that the forum contains a mixture of singleanswer and openended questions as long as there is any positive probability p that a question in the forum will be an openended question (which might be discussion questions posed by the instructor herself), the instructor can choose µ so as to produce a rate λ such that the rate of arrival of the first response scales linearly with the size of the class, thus obtaining a speedup with class size which is of the same order as in the case of purely openended questions. 1.. Interpreting the Model With any stylized model such as the one presented here, it is useful to consider what qualitative conclusions and recommendations it suggests. Our present model is very simple to formulate it just reflects the balance between the cost of checking the forum frequently and the benefits of getting rewarded for answering questions and hence the phenomena it captures seem to be arising intrinsically from this tension, rather than from any more complicated modeling artifacts. Moreover, these phenomena, while fairly subtle, align well with evidence from empirical studies of the kind noted earlier. Our analysis leads to qualitative recommendations (within the stylized setting of our model) for how to optimize the performance of the forum by carefully choosing the instructor s behavior. A first conclusion is that for motivating individual student effort, it is important to find an optimal operating point between motivating students to move quickly and not pushing them to move so quickly that the effort cost leads them to drop out of the system. A second conclusion is that when only a single reward is (meaningfully) available (as in a singleanswer question), having many students compete for it may not actually be so productive, because students will scale back the effort they expend in the face of a low probability of receiving the reward. Given this phenomenon, the instructor needs to avoid strategies (such as the local minimum that arises in our singleanswer model) where she puts in effort that simply has the effect of suppressing student activity without improving the performance of the forum overall. A final qualitative recommendation from the model is that if the instructor can create reasonable mechanisms by which more than one student is rewarded for the
5 same question, the behavior of the system can transition at least partially toward the faster operation it experiences with openended questions; our analysis of mixedquestion forums in 5 shows one way to achieve this speedup Related work As noted above, there has been research into the use of online forums for education focusing on empirical and qualitative evaluation based on experience with such forums [Andresen 009; Burge et al. 000; Gray 004; Hanover Research Council 009; Mazzolini and Maddison 003]. There is also a large and growing body of work on generalpurpose online Q&A forums such as Y! Answers and StackOverflow starting with Adamic et al. [008]; most of this literature is empirical, documenting and analyzing usage patterns and user behavior from data or qualitative studies on these forums. There is much less prior work on mathematical models and the design of incentives for online Q&A forums; we discuss these below. Chen et al. [009] study Q&A forums such as Y! Answers in a gametheoretic model in which each user has a unique piece of information to contribute to a question, and a user who submits an answer at some time can aggregate all previous answers into her answer. Their work analyzes the equilibria elicited by a bestanswer mechanism where the asker awards a point to the best answer received for complements and substitutes information types in terms of the round in which agents supply their answers. They show that a bestanswer rule is effective for substitutes information but not for complements information, where an approvalvoting scoring rule and a proportionalshare scoring rule can enable the most efficient equilibrium. In their model, agents do not have any cost to answering a question, and their strategic choice of when to answer is driven only by the desire to obtain the reward handed out to the best (or best k or valueproportional, depending on the mechanism) answer. Our model differs in two significant aspects, since agents in our model (i) have a cost to the rate at which they check the forum, which is what drives any delays in answering, and (ii) cannot incorporate (or are not rewarded for incorporating) previous contributions into their answer to beat previous answers to obtain a reward. These two differences lead to an entirely different model of strategic behavior and outcomes. Ghosh and Hummel [01] focus on the bestanswer type of mechanisms used by Q&A sites such as Y! Answers, and investigate implementability, asking whether this family of conteststyle mechanisms is adequate to elicit optimal outcomes when potential answerers are strategic agents, and how payoffs must be structured to incentivize this desired optimal behavior in terms of the quantity and quality of answers elicited in a simultaneous contribution model. Kumar et al. [010] study online Q&A sites in the context of network effects in twosided markets and the rate of arrival of new questionaskers (and answerers) as a function of the number of current answerers and questionaskers in the system, both empirically and theoretically. They show that the two sides of the market can display radically different patterns, and also investigate competing platforms. Finally, we note that there is some very recent work on various research challenges arising in the context of online education via massive online open courses (MOOCs). Our work studies models for online forums for education, whether for standard courses taught in offline classrooms or online ones, and intersects with the research on MOOCs to the extent that most MOOCs use Webbased forums as a platform for communication with and between students in the class.. MODEL In this section, we present a gametheoretic model for online forums for education, the central component of which is the strategic choice of rates by agents the presence of an instructor who can monitor answer quality and reward only goodenough contributions allows us to focus on the rate of participation as the strategic choice variable in a model without qualities (or where all answers have equivalent qualities), a simplification which is less justifiable in general online Q&A forums. Students and the instructor choose rates with which to check the online forum for the class, and students are rewarded when the instructor arrives and finds that a student has (usefully) answered a question on the forum. A higher rate incurs a higher cost, but can also lead to higher X:5
6 X:6 benefit to a student by increasing the probability of arriving before the instructor and thereby earning a reward. This leads to a sequential game where the instructor moves first by choosing her rate, and the students move second, choosing their rate as a best response to the rate chosen by the instructor, and possibly the rate choices of remaining students, as described formally next. There is a class of n students and an instructor. Questions arrive over a time interval [0, T ], and the students and the instructor periodically check the forum and provide answers to these questions, via a process defined below. We refer to the individuals providing the questions as the questionaskers or simply the askers, and the individuals providing the answers as the students..1. Rates Each student and the instructor choose a rate at which to check the class s online forum for new content arriving in the interval [0, T ]. Student j chooses a rate λ j, and repeatedly checks the forum according to a truncated Poisson process with rate λ j that is, (i) if j has just checked the forum at time t, then he will next check it at time t + X, where X is an exponential random variable with rate λ (i.e., with probability density function f(x) = λe λx for x 0); and (ii) j runs this process until the first time that he checks the forum after time T, at which point the process ends. Intuitively, the student s random variable X acts like an alarm clock when the alarm rings at some time t, the student checks the forum and then resets the alarm, so that it next rings at time t + X. We will allow a student to choose λ j = 0: while one cannot define an exponential random variable of rate 0, we use λ j = 0 to simply mean that the student never checks the forum. We suppose that if a student checks the forum and finds a question that still needs an answer, he can provide one. While obviously not always true, we note that this is not a limiting assumption our model can be easily extended to a include a probability s j that a student will have an answer to supply for a random question (the current model corresponds to the special case where s j = 1). Extending the model to include this probability complicates the analysis but not does not change the qualitative conclusions, so we omit it here for clarity given the limited space. The instructor also chooses her own rate µ at which to check the forum, also according to a truncated Poisson process as described above. When the instructor arrives according to her random process of rate µ, she closes the (new) questions she finds there, supplying an answer (if necessary) to these questions and handing out all appropriate rewards (as described later). Once a question is closed, no further rewards can be obtained by answering the question this could be either because the instructor explicitly disables any further answers to the question, or because no further points will be handed out even if answers can be supplied, or perhaps because no one will read any answers beyond those supplied by the instructor in the case of socialpsychological rewards from answering questions. An instructor is also allowed to choose a rate µ = 0, although with a different interpretation. A zero rate for the instructor means that questions are never closed so that a (useful, nonredundant) answer is always assigned a reward no matter how late it arrives Exponential rates. Exponential random variables have a number of nice properties that makes their analysis particularly tractable. The first is the socalled memoryless property : FACT.1. Let X be an exponential random variable with rate λ. Conditioned on the information X t 0, the random variable X t 0 is also an exponential random variable of rate λ. That is, if we know that the alarm has not rung until time t 0, then starting at time t 0 it behaves as though the alarm had just been freshly reset at t 0. Later in this section, we will use this property For simplicity, we will consider the askers and the students here as two disjoint populations, but one can also model the case in which the asker on each individual question is drawn from the population of students.
7 to show that we can reduce the problem of analyzing participants behavior over the entire time interval to the analysis of their behavior on a perquestion basis. We also note two other standard facts about exponential random variables that will be very useful in parts of the analysis. FACT.. If X 1 and X are exponential random variables with rates λ 1 and λ, then Pr [X 1 < X ] = λ 1 λ 1 + λ. FACT.3. If X 1,..., X n are exponential random variables with rates λ 1,..., λ n, then min(x 1,..., X n ) is an exponential random variable with rate λ λ n... Utilities The utility is the difference between an agent s (student s or instructor s) benefit and her cost, which we describe next. Benefits. Rewards are allocated to students by the instructor on a perquestion basis. Students are rewarded when the instructor arrives and ascertains that a student has (correctly and usefully) answered a new question on the forum this reward could either be in the forms of points awarded by the instructor that count towards the student s grade in the class, or because the student derives a socialpsychological reward from the instructor s approval or endorsement 3, or from providing a useful answer to her fellow students. Once the instructor arrives and closes a question, no further rewards can be obtained from answering that question as discussed before, this could be either because the student derives value only when the instructor sees that she has usefully answered a question (either because points are only awarded for such answers, or because the student s reward is the psychological value of the instructor s approval), or because the student derives value from having her fellow students read her answer and no answers beyond the instructor s are read by other students. We note that our model is agnostic on the question of what causes a student to feel rewarded from answering a question, and applies to a spectrum of possible underlying reasons for rewards. We use v to denote the value of each individual reward to a student. Let the random variable R jq be equal to 1 if student j receives a reward for answering question q, and be equal to 0 otherwise. Whether an answer generates value, and therefore merits a reward, is different for the two types of questions discussed in 1. In our definition of openended or discussionstyle questions, an answer adds value even if other answers appear earlier, so that any student who answers before the instructor closes the question can receive a reward (recall our assumption, discussed earlier, that students have correct answers to questions). On the other hand, for singleanswer questions, only the first student to see the question will answer, because subsequent answers would be redundant (which are not rewarded, so that we can assume that students don t bother to supply redundant answers). We derive the specific forms of the reward functions for these two types of questions in 3 and 4 respectively. We assume that the student incurs a fixed cost ξ for writing an answer to a question; because a student will only write an answer when the reward received for it is at least ξ, it makes sense to think not about the pure reward v but the net reward v = v ξ. Costs. We assume that for each student, checking the forum for new content incurs an effort cost of α each time the forum is checked, whether or not there is new content to react to. (If the student does react to new content, this incurs the effort cost of ξ noted above, which is folded into the net reward v = v ξ.) Similarly, the instructor incurs a cost of β each time she checks the forum. Arrival of Questions and Overall Utility. Questions arrive over the time interval [0, T ] according to a Poisson process with rate 1 that is, we imagine questions being generated by a rate1 Poisson X:7 3 Piazza, for instance, explicitly offers an instructor endorsement feature on its forums.
8 X:8 process defined over the positive real numbers, and then we only keep the subset of questions that arrive in [0, T ]. (The choice of 1 here provides the units of the time scale in which all other rates are measured.) Let Q denote this set of questions, and let C j denote the set of times at which student j checks the forum. Then student j s overall utility (recall that v is the net reward) is the expectation of vr jq q Q h C j α, reflecting the net rewards from each question and the cost for each checking of the forum..3. Reduction to the Case of a Single Question Because of the memoryless property of the exponential distribution, we can effectively consider each question that appears on the forum over the duration of the course in isolation, as follows. Recall that each student (and the instructor) runs his process from 0 until the first time that his exponentially distributed alarm clock rings after time T. Thus, since each question q appears before time T, all participants alarm clocks are still running when q arrives, and by the memoryless property we can thus imagine that these alarm clocks are all started at this point of arrival. It follows that for a student j and any two questions q and q, the random variables R jq and R jq have the same distribution; let R j denote a random variable with this shared distribution. Hence, by Wald s Theorem, we can write student j s overall utility as E vr jq = ve [ Q ] E [R j ] αe [ C j ]. q Q h C j α Since the questions are arriving at rate 1 over the interval [0, T ], we have E [ Q ] = T. For E [ C j ], we observe that the times at which student j checks the forum consist of draws from a rateλ j Poisson process over [0, T ], followed deterministically by one additional check of the forum (the first one that the process generates after T ). Hence we have E [ C j ] = λ j T + 1, and so j s utility is ve [R j ] T α(λ j T + 1) = (ve [R j ] αλ j )T α. T and α are independent of student j s choice of λ j, but E [R j ] depends on λ j, since a faster rate of checking the forum leads to a greater chance of receiving a reward. We can thus think of j as strategically choosing λ j to optimize the objective function ve [R j ] αλ j, and this is the basic form of the problem we consider. We make one further simplication we will focus in this exposition on the case in which the net reward v = 1, so the objective function representing student j s utility maximization problem is E [R j ] αλ j. This is purely for simplicity, and arbitrary values of v are handled completely analogously. The Instructor s Utility. For the instructor s utility, we can use a direct analogue of the reduction above and focus on her utility for a single question. The instructor s utility on a single question is the difference between her timediscounted value from the answers contributed to the question and her effort cost βµ from checking the forum at a rate µ; as with the students, the form of this utility depends on on whether the question is openended or a singleanswer question. The instructor derives benefit from the asker of the question getting an answer (for singleanswer questions) or answers (in discussion questions) as quickly as possible. We model this by saying that the value of a (useful, nonredundant) answer is discounted exponentially, so that if it arrives at time t after the question was asked, it has value e δt for a parameter δ > 0. Thus, if a question receives k answers potentially including the instructor s own answer at times t 1, t,..., t k (note that k = 1 for a
9 X:9 singleanswer question 4 ), then the instructor s utility in this realization of the process is k e δti βµ. i=1 In both 3 and 4, we will be considering the expected value of this expression, when the times t i arise from the random process of checking the forum according to the rates λ chosen (in equilibrium) by the students, and the rate µ chosen by the instructor..4. Summary of Parameters The reduction above allows us to consider the problem of equilibrium rates in the context of a single question that arrives at time 0; all strategies and utilities will be defined in the context of this single question, keeping in mind that this in fact captures the behavior of the system for the full set of questions arriving over [0, T ], Beyond the number of students n, the model thus has three basic parameters: α, β, and δ, each with a simple interpretation. The parameters α and β represent how the students and the instructor value the effort they have to put into the forum (for example, a large α means students view checking the forum as a costly activity, which affects how they set λ). The parameter δ models the impatience of the asker: a large value of δ means that answers that take longer to come in create significantly less value than answers that come in right away. 3. OPENENDED QUESTIONS We begin by considering openended questions, in which the questionasker benefits from all answers that come in, and hence students will continue to contribute answers until the instructor closes the question. As a result, the students are not competing against each other to provide answers quickly; each student is instead optimizing his choice of rate λ based solely on the instructor s choice of rate µ. Our first result shows that the student s response rate λ for openended questions is nonmonotone in the instructor s choice of rate µ, and is maximized at a specific intermediate value of µ: as µ increases upward from 0, it initially causes the student to want to move faster (to answer the question before it closes), but as µ becomes too large, the reward from answering is no longer worth the cost it would require to make λ sufficiently large, and the student s choice of rate λ begins dropping back toward 0. THEOREM 3.1. A student s optimal choice of rate λ, as a function of the instructor s rate µ, is given by µ λ(µ) = max{0, α µ}. λ(µ) increases on µ [0, 1 4α ] and decreases beyond, attaining its unique maximum at µ = 1 4α. PROOF. Consider a student with rate x and an instructor with rate µ. The student receives a reward if she arrives and answers the question before the instructor arrives; by Fact., this event has probability x/(x + µ). So the student s payoff from choosing rate x is U S (x) = x x + µ αx. 4 As discussed previously, students do not have an incentive to contribute redundant answers, nor do redundant answers generate value to the asker.
10 X:10 Note that U is concave in x, so to maximize U we set the derivative U S (x) = µ (x+µ) α to zero and solve for x, which gives µ x = α µ. This is the best response for a student when x 0. If x < 0, then the best response is to choose rate 0, since U S (0) = 0 and U S (0) < 0 in this case (U S is concave in x so U S is decreasing, and U S (x ) = 0 for x < 0) this corresponds to the instructor choosing too high a rate µ, specifically µ > α 1 at which point the student simply drops out. Therefore, the student s best response rate when the instructor chooses rate µ is µ λ(µ) = max{0, µ}. (1) α Observe that λ(µ) is nonmonotone in µ: the derivative of λ(µ) with respect to µ, for µ 1 α, is λ (µ) = αµ 1, which is positive for µ 4α so that λ is increasing for 0 µ 4α, and negative for µ > 1 1 4α, so that λ(µ) decreases with as the instructor increases her rate beyond 4α. In particular, we can ask what value of µ causes the student to answer as quickly as possible; this maximum is precisely where the derivative is 0 (note that λ is concave in µ), which is µ = 1 4α. Note that λ(µ ) = µ, so that when the instructor chooses a rate to maximize the rate chosen as a best response by the students, the instructor and students all have the same rate in equilibrium. Note also that at a given µ [0, α 1 ], the probability the student provides an answer before the instructor closes the question is equal to λ(µ) λ(µ) + µ = µα 1 µ µα 1 = 1 αµ, which is decreasing in µ. So for µ close to 0, the student s probability of providing an answer before the question closes at his optimal rate λ(µ) is close to 1; this probability decreases to exactly 1/ at the optimal µ = 1/(4α), and then continues decreasing to 0 as µ reaches 1/α. Overall utility. We now consider how the instructor should choose a rate µ to maximize her utility recall from Section that this utility consists of a benefit equal to the exponentially discounted arrival times of answers, minus a cost βµ due to the instructor s choice of µ. To begin with, let s consider the instructor s utility in the case where she is the only individual providing an answer to the question. If she chooses a rate of x, then the expected benefit of her answer is the expected value of its discounted arrival time, which is 0 e δt xe xt dt = x x + δ. () Equation () has a very clean interpretation the expected value of the discounted arrival time is equivalent to imagining that the questionasker will give up after an amount of time drawn from an exponential distribution with rate δ (the discount parameter), and the benefit of the answer is the probability that it arrives before the asker gives up. Recall that the discount rate δ models the impatience of the asker a large δ corresponds to a fast rate of decay in value on the lefthand of Equation (), and it corresponds to the asker giving up quickly on the righthand side of (). Now let s consider the instructor s full utility function. There is a benefit from each answer provided by a student, plus the benefit of the instructor s own answer, minus the instructor s effort cost. The benefit of a student s answer (provided at rate y) is equal to the probability that it arrives both before the asker gives up (at rate δ) and before the instructor closes the question (at rate µ), since
11 the student will not answer a closed question: 0 e δt e µt ye yt dt = y y + δ + µ. X:11 When the instructor chooses a rate µ, the students will each choose their best response rate λ(µ) (as in (1)), and the instructor s goal is to maximize her utility function U I (µ) = n λ(µ) λ(µ) + µ + δ + µ βµ. (3) µ + δ We now consider the maximum utility achievable in the case when n is large, so that the first term, which is linear in n, is the dominant one in the expression for U I in (3): THEOREM 3.. When µ = α( δ + δα 1 δ), we have U I (µ ) = b n + o(n) where b = 1 α( δ + δα 1 δ); also, there is no µ and b > b for which U I (µ) = bn + o(n). Thus, the rate µ maximizes the instructor s utility to within a term that is sublinear in n. Before presenting the proof, it is worth understanding how the quantities µ and b behave, particularly as we vary the impatience δ of the questionasker. Note that there is a tradeoff between speed and the number of answers received in the utility U I (µ) when µ is very small, students arrive at a much slower rate but more students are likely to find the question open, so that we obtain a larger number of answers, although at later times. As µ increases, students also increase their response rates λ(µ) and arrive faster, but fewer answers are provided since the question closes earlier. When the asker s impatience δ is close to 0, the optimal value µ is close to 0 and b is close to 1: that is, with a very patient asker, the instructor can afford to move slowly to ensure that almost every student contributes close to the full benefit of 1 via their answer. As δ goes to infinity, it follows from Taylor s Theorem that δ + δα 1 δ converges to 1/(α) from below. Hence µ converges to 1/(4α) and b converges to 1 (α)(1/(α)) = 0 from below. This makes sense: as δ increases, so that the asker becomes more impatient, the instructor needs to trade off quantity against speed to induce the students to move more quickly, so she increases her rate up to the µ that, by Theorem 3.1, maximizes the rate of each student. However, each student can only be induced to move at a maximum rate of 1/(4α), which is independent of δ, and so as δ goes to infinity the benefit contributed by each student is converging to 0. We now proceed with the proof of Theorem 3.. PROOF. In the expression for U I in (3), the second and third terms are o(n), so it suffices to find a choice of µ that maximizes the coefficient on n in the first term. Such a choice of µ will lie in the interval [0, α 1 ], since it needs to cause the students to choose a nonzero rate. By Theorem 3.1, when the instructor chooses a rate µ [0, α 1 ], each student chooses the rate λ(µ) = µα 1 µ. Applying this choice of λ in Equation (3), we get U I (n) = B(µ) n + o(n) where µα 1 µ B(µ) = µα 1 + δ. We thus wish to maximize B(µ) over the interval [0, α 1 ]. The function B(µ) is concave in µ over this interval, with a unique maximum where B (µ) = 0. Setting this derivative to 0 yields a quadratic equation in µ: and hence α 1/ µ + δ µ δα 1/ = 0 µ = α( δ + δα 1 δ).
12 X:1 At this value of µ, we have B(µ) = δ + δα 1 α( δ + δα 1 δ) δ + δα 1, which simplifies to B(µ) = 1 α( δ + δα 1 δ). 4. SINGLEANSWER QUESTIONS Singleanswer questions, as the name suggests, are questions with one welldefined correct answer, such as In the homework due tomorrow, were we allowed to assume that x is positive?, or What is the most abundant isotope of carbon?. In singleanswer questions, only the first student to see the question will answer, because subsequent answers would be redundant 5. For singleanswer questions, therefore, students are implicitly competing to be the first to provide an answer so for any particular rate µ chosen by the instructor, students play a simultaneousmove game amongst themselves in which their strategies are their choices of rate. We show that this game has a unique symmetric Nash equilibrium in which all students choose the same rate λ (µ), which we can study as a prediction for how the forum will operate. Note that even when µ = 0 (so that the instructor doesn t close the question, but only grades the outcome of the race once the class is over), the equilibrium rate λ is nontrivial for singleanswer questions, due to the competition among the students. We find that at this equilibrium, the first answer to the question arrives at a speed that is almost independent of the class size n. (There is an additive dependence on n that grows proportionally to 1/n.) Moreover, this slow race among the students is very hard for the instructor to influence through her choice of µ: for n, we will find that the equilibrium value λ (µ) is maximized at µ = 0. Thus, with a singleanswer question, each student in a large class slows down his personal rate, to an extent great enough that it offsets the gains from having many students trying to answer the question. This is a sharp contrast from the case of openended questions, where the students are choosing rates without regard to the behavior of the other students, and hence the rate of the fastest answer in that case scales linearly in n. This principle that a large class slows itself down suggests interesting potential limitations on the power of large classes to provide crowdsourced responses in the singleanswer case, and it arises from an underlying mechanism that is intuitive: in a large class, each student reasons that his chance of being the first to answer a (singleanswer) question is so low that it is not worth checking the site very often. THEOREM 4.1. For any choice of µ by the instructor, there is a unique symmetric equilibrium λ(µ) for the students. In this equilibrium: The students don t participate when µ α 1 ; in this case the instructor answers the question on her own. When µ < α 1, we have λ(µ) = n 1 αn µ n + (n 1) 4α n + µ 4 αn, so that the students do participate 3 along with the instructor, but each at a rate of Θ(1/n) as a function of the class size n. For n, the maximum student rate λ(µ) in equilibrium is achieved when the instructor does not participate (µ = 0). PROOF. Consider a student i with rate x: i receives a reward when she arrives before all the remaining n 1 students and the instructor. The payoff of i, when the other n 1 students all choose rate λ and the instructor chooses rate µ, is (using Fact ): x U(x, λ) = x + (n 1)λ + µ αx. 5 Recall that we assumed in our model that a student who sees a question can supply the correct answer to it, and discussed generalizing this assumption in. Recall also that students are rewarded only for valuable answers, so we assumed that students do not bother to provide redundant or useless answers.
13 X:13 For x to be a best response, U(x, λ, µ) must be maximized at x. Note that U is concave in x, so setting the derivative to zero is a necessary and sufficient condition for an interior maximum. (n 1)λ + µ Taking the derivative and equating to zero, we have = α, which gives us (x + (n 1)λ + µ) (n 1)λ+µ x = α ((n 1)λ + µ). Since we seek a symmetric equilibrium, we want x = λ. This gives us (n 1)λ + µ = nλ + µ, α yielding the following quadratic equation in the variable λ: The solutions to this quadratic are, respectively, αn λ + λ(nµα (n 1)) + (αµ µ) = 0. (λ 1 (µ), λ (µ)) = n 1 αn µ n ± (n 1) 4α n 4 + µ αn 3. (n 1) Since 4α n + µ 4 αn n 1 3 αn, it follows that λ (µ) 0 for any µ. This cannot be the best response since λ is the rate of an exponential and hence must be positive. So the best response of the students when the instructor chooses a rate µ is either λ 1 (µ) if λ 1 (µ) 0, or else it is 0: λ(µ) = max{0, λ 1 (µ)} = max{0, n 1 αn µ (n 1) n + 4α n 4 + µ αn 3 }. Now we investigate the function λ 1 (µ). At µ = 0, λ 1 (0) = n 1 αn. The derivative of λ 1 with respect to µ tells us how the best response rate chosen by the students changes as the instructor increases her rate up from 0, if the best response is to participate: d dµ λ 1(µ) = 1 n αn 3 = (n 1) 4α n + µ 4 αn 3 1 n 1 1. αn (n 1) 4α n + µ 4 αn 3 d Note that dµ λ 1(µ) is decreasing in µ. At µ = 0, the derivative evaluates to d dµ λ 1(µ) = 1 [ ] 1 0 n n for all n. Since the derivative is decreasing in µ, and nonpositive at µ = 0, this means that λ 1 (µ) is maximized at µ = 0, as long as n. Now, the value of λ 1 at µ = 0 is λ(0) = n 1 αn > 0, so the value of µ that maximizes λ(µ) = max{0, λ 1 (µ)} is µ = 0, as long as n. That is, if there are at least two students in the class (and we need at least two students for there to be a race between students to first supply the correct answer), the instructor can maximize the rate of response of the students by choosing the lowest possible rate for herself, i.e., by setting µ = 0. Note that for any n, λ 1 (µ) becomes 0 at µ 0 = 1 α. (Since the derivative of λ 1(µ) is decreasing in µ and nonpositive at µ = 0, and λ 1 (0) > 0, λ 1 (µ) crosses zero exactly once for µ 0). Therefore, the students best response rate λ(µ) behaves as follows: λ(µ) = n 1 αn µ (n 1) n + 4α n 4 + µ αn 3, for µ [0, 1 α ] 0, for µ 1 α.
14 X:14 Finally, we consider what the instructor should do to maximize her utility, and here too we find a complex but fairly intuitive phenomenon at work. Recall that the instructor s utility U I is based on the speed at which the question is answered (either by her or by one of the students), minus the effort cost created by checking the forum at rate µ. When we study this utility U I (µ) as a function of µ, we discover that for a small enough effort coefficient β on the part of the instructor, U I (µ) is in fact bimodal, with two local maxima: the optimal utility is achieved either by setting µ = 0 and letting the race among the students do its work, or by choosing a particular µ > 0 in which the students are essentially driven out of the forum and the instructor answers the questions herself. Which of these two local maxima is higher depends on how the instructor values the cost of her own effort. In between, there is an intriguing local minimum in the utility essentially, when the instructor is using the forum the wrong way in which the instructor s choice of rate µ suppresses the students effort in a manner that doesn t increase the overall rate (i.e., including both the instructor and the students) at which the question gets answered. To begin the analysis of this phenomenon, we write down the utilities involved. For singleanswer questions, the asker derives utility from the first (and only) answer to her question, which is discounted as before by the time at which this first answer arrives. Again, if the asker discounts time by an exponential factor δ, the expected utility from an answer arriving at time t is e δt. The time of arrival of the first answer, when the instructor chooses a rate of µ (and answers the question if she arrives first), and the students choose a rate λ(µ), is exponentially distributed with rate nλ(µ) + µ. So the expected utility to the asker is 0 e δt (nλ(µ) + µ)e (nλ(µ)+µ)t dt = nλ(µ) + µ nλ(µ) + µ + δ. The instructor would like to maximize her own utility which is the difference between the asker s utility, and her own cost to choosing a rate µ, i.e., to choose a rate µ that maximizes U I (µ) = nλ(µ) + µ nλ(µ) + µ + δ βµ. We now formalize the bimodal properties of U I in this setting. THEOREM 4.. For n sufficiently large in terms of α, β, and δ, the following holds. δ If β < (δ + α 1 ), the instructor s utility function U I(µ) has two local maxima: one at µ = 0, and the other at a point µ > α 1. In between, U I has a local minimum at µ = α 1. There is a threshold cost β such that the second local maximum has greater utility than the first if and only β < β. δ If β > (δ + α 1 ), the instructor s utility function U I(µ) decreases monotonically in µ, with a single maximum at µ = 0. PROOF. First we consider what happens if the instructor chooses a rate µ < α 1. Recall from Theorem 4.1 that the students will participate for such µ, and so the combined rate will be nλ(µ)+µ. We use φ(µ) to denote this combined rate; so by the definition of the instructor s utility we have U I (µ) = φ(µ) φ(µ)+δ βµ, and from Theorem 4.1, we have φ(µ) = n 1 αn + (n 1) 4α n + µ αn. Differentiating the instructor s utility, we get U I (µ) = δφ (µ) (φ(µ)+δ) β. Now, φ (µ) = 1 αn 1, (n 1) 4α n + µ αn
15 X:15 which goes to 0 as n diverges, so that the first term in U I (µ) also goes to 0 in n. Thus we see that for sufficiently large n, U I (µ) < 0 for µ < α 1. It follows that µ = 0 is a local maximum of the instructor s utility when n is large enough. Next we consider what happens if the instructor chooses a rate µ α 1. In this case the students will not participate so that the combined rate φ(µ) = µ, and so the instructor s utility is given by U I (µ) = µ βµ. If we consider this as a function over all µ 0, it is concave, with a unique µ+δ local maximum at µ = max { 0, } δ β δ. But this is the correct expression for the instructor s utility only if µ α 1, i.e., once the students have droppedout, so we need to ask when this local maximum µ exceeds α 1. The condition for µ > α 1 δ is β δ > 1 δ and hence β < α (δ + α 1. We thus have two cases (again keeping in ) mind that n is sufficiently large): δ When β < (δ + α 1 ), the instructor s utility has a second local maximum µ (in addition to µ = 0) at a value of µ beyond α 1 this local maximum corresponds to the instructor answering the question alone with no help from the class, at a rate which optimally trades off her cost to checking the forum and her benefit from the asker receiving an answer quickly. δ On the other hand, when β > (δ + α 1 ), then U I(µ) is monotonically decreasing as soon as µ α 1 this is because the concave function µ µ+δ βµ is decreasing beyond its maximum, i.e., for µ µ, and µ < α 1. Hence µ = 0 is the only local maximum for this range of β. δ Finally, in the case when β < (δ + α 1, we can ask which of the two local maxima 0 or µ ) achieves a higher utility. The utility at the local maximum µ = 0 is U I (0) = φ(µ) φ(µ) + δ = The utility at the other maximum µ is given by n 1 αn n 1 αn + δ αδ. U I (µ ) = (1 βδ). Now we observe that as β goes to 0, so that the instructor s effort becomes less and less costly, the utility U I (µ ) converges to 1 while U I (0) remains bounded below 1. Thus there is a β such that when β < β, the second local maximum, when the instructor answers the question on her own, has the higher utility. In the limit as n, the utility U I (0) 1 is converging to 1 + αδ, and so the crossover point where U I(0) = U I (µ ) is converging to the solution of (1 βδ) 1 = 1 + αδ, which is β = 1 δ ( δα ).
16 X:16 5. MIXED FORUMS In this section, we address the question of how an instructor might elicit higher rates from students for singleanswer questions, and show that one way to offset some of the slow behavior arising from singleanswer questions is by ensuring that the forum contains a mixture of singleanswer and openended questions. We imagine that each question is openended with probability p > 0 and singleanswer with probability 1 p. Each student s λ parameter, however, is a global rate that governs how frequently he checks the site; by definition it must be chosen before any particular question is seen. When the student checks the site (according to rate λ), he may sometimes find an openended question, and sometimes a singleanswer question. As before, we find that there is a unique symmetric equilibrium λ for any choice of µ. Our main result here is that the instructor can choose µ so as to elicit a rate λ that is roughly p times what it is in the case of purely openended questions, and this is tight for large n. In this way, the rate at which the 1 p fraction of singleanswer questions get answered benefits significantly from the presence of even a small positive fraction p > 0 of openended questions they are essentially sped up by a linear amount (in n) when p is any constant. THEOREM 5.1. Let p be the probability that a question is openended. The instructor can choose a rate µ such that the equilibrium student rate λ satisfies λ p. Moreover this is essentially 4α tight in the limit, in the following sense: For any ε > 0, there exists a large enough n so that with at least n students, there is no µ the instructor can choose so as to achieve λ (1 + ε)p. 4α PROOF. Suppose the instructor chooses rate µ; we want to determine the best response rate λ chosen by the students. Using the payoffs derived in 3 and 4, the payoff to a student who chooses rate x when the instructor has rate µ and the remaining n 1 students choose rate λ is x x U(x) = p + (1 p) x + µ x + (n 1)λ + µ αx. As usual, for x to be a best response, a necessary condition is for U (x) to be zero, which gives us µ p (x + µ) + (1 p) (n 1)λ + µ (x + (n 1)λ + µ) α = 0. Since we are looking for a symmetric equilibrium, we set x = λ, which gives us the following equation in λ: µ 1)λ + µ p + (1 p)(n (λ + µ) (nλ + µ) = α. (4) Note that (4) can be interpreted as a bivariate equation in the two variables λ and µ; for a given value of µ, if there is a solution λ(µ) 0 to this equation, then λ(µ) is a best response to µ. p We first identify a choice of µ for which the equilibrium value λ(µ) is at least. It turns out 4α that this can be achieved using a µ for which λ(µ) = µ, as follows. In order to have an equilibrium in which λ(µ) = µ, we simply need a pair (µ, λ) satisfying Equation (4) for which µ = λ. Setting µ = λ in (4), we get leading to p λ nλ + (1 p) 4λ (n + 1) λ = α, λ = p + 4 n+1 (1 p) p 4α 4α, (5)
17 X:17 as desired. This gives us a lower bound on the fastest possible equilibrium λ, by demonstrating the value of λ achieved at a particular carefullychosen equilibrium. Next we will show that as n becomes large, this lower bound is essentially the best possible. To do this, fix any constant ε > 0, choose σ small enough that (1 σ)(1+ε) > 1, and then choose n large enough in terms of ε, p, and σ so that 4 < σ. Now suppose by way of contradiction that pn for some choice of µ, there is a pair (µ, λ) that solves the bivariate equation (4) with λ Then we have µ 1)λ + µ α = p + (1 p)(n (λ + µ) (nλ + µ) µ nλ + µ p + (1 p) (λ + µ) (nλ + µ) µ = p (λ + µ) + (1 p) 1 (nλ + µ) µ p (λ + µ) + 1 nλ µ p (λ + µ) + 4α pn µ < p (λ + µ) + σα, (1 + ε)p. 4α and hence µ p > (1 σ)α. (λ + µ) p Writing r = (1 σ)α, this implies an upper bound on λ via the inequality λ < rµ µ. Now, over all µ 0, the righthand side of this inequality is maximized at µ = r/4, where it takes the value r/4, and hence we have λ < r/4 = p 4(1 σ)α (1 + ε)p <. 4α (1 + ε)p This last inequality contradicts our assumption that λ. 4α Thus, as long as there is any positive probability p that a question in the forum will be an openended question (which might be discussion questions posed by the instructor herself), the instructor can choose µ to produce a rate λ such that the rate of arrival of the first response to any question on the forum scales linearly with the size of the class, obtaining a speedup with class size which is of the same order as in the case of purely openended questions. 6. DISCUSSION In this paper, we introduced a gametheoretic model for online forums for education, and investigated how an instructor can influence student participation and utility in these forums. For singleanswer questions, we find that the instructor has very little influence over the rate at which students participate in the forum, especially as the class size grows larger and larger here, the rate of response is driven almost entirely by the students competition amongst each other to arrive first to supply the answer. However, when the instructor is optimizing for the speed at which questions are answered, taking into account her own ability to spend effort on producing answers, a
18 X:18 more complex effect emerges in which the most efficient behaviors of the forum are clustered into two modes one in which the instructor steps back and lets the students compete to answer, and one in which she effectively goes it alone. The instructor s participation plays a very different role in openended questions. First, the instructor must choose a rate µ > 0 if she would like the students to at all contribute the forum if µ = 0, the students choose a rate λ(µ) = 0 as well. Second, the students response rate λ varies nonmonotonically in the instructor s rate; as noted earlier, this provides an interesting strategic basis for the observation that students will increase their effort in the presence of an active instructor up to a point, but then start to give up if the instructor is too active [Gray 004; Hanover Research Council 009]. It is also interesting to note the way in which the student s rates range very widely as the instructor s rate ranges over values in the interval µ [0, 1 α ], where α is the students cost parameter: each student s rate increases from 0 at µ = 0 until the optimal value µ = 1 4α that elicits the fastest responses from the students, and then falls back to 0 at µ = 1 α. Thus, the overall response rate of the class, nλ, ranges all the way from 0 to n 4α and back to 0 over this interval µ [0, 1 α ]. When α is small, so that students consider effort to be inexpensive, this can be a reasonably gradual effect in µ. But for large α, the effect happens over a narrow interval; this means that for a class where students have a high cost to frequently checking the forum, the behavior of the forum is very delicate and changes dramatically over a very small interval of the instructor s rate in [0, 1 α ]. Such sensitivity to the underlying parameters connects in intriguing ways to types of delicate behavior observed in real online forums, where small shifts in activity level can lead to wide changes in participation rate. Among the natural questions for further work, a fundamental one that connects to this point about sensitivity is to ask how an instructor might learn the optimal rate to use for her forum. While we do not attempt to formally address this question here, note that the response rate for singleanswer questions is essentially insensitive to the instructor s choice of rate, but does depend on the students cost parameter α in our stylized model. This suggests that an instructor could initially use singleanswer questions to learn the students cost parameters from their response rates to these questions, and use this to infer the optimal rate µ that she should use for openended questions. More broadly, developing a model for learning parameter values to optimize student participation, as well as modeling learned behavior and evolution of response rates in such online education forums is an interesting open direction for further work. REFERENCES ADAMIC, L., ZHANG, J., BAKSHY, E., AND ACKERMAN, M Knowledge sharing and Yahoo! Answers: Everyone knows something. Proc. 17th International World Wide Web Conference. ANDRESEN, M. A Asynchronous discussion forums: success factors, outcomes, assessments, and limitations. Educational Technology & Society, 1 (1). BURGE, E., LAROQUE, D., AND BOAK, C Baring professional souls: Reflections on Web life. Journal of Distance Education, 15(1). CHEN, Y., JAIN, S., AND PARKES, D Designing Incentives for Online Question and Answer Forums, Proc. ACM EC. GHOSH, A. AND HUMMEL, P. 01. Implementing Optimal Outcomes in Social Computing: A Gametheoretic Approach, Proc. 1st International World Wide Web Conference. GRAY, B Informal Learning in an Online Community of Practice. J. Distance Educ.. 19(1). HANOVER RESEARCH COUNCIL Best Practices in Online Teaching Strategies. KUMAR, R., LIFSHITS, Y., AND TOMKINS, A Evolution of twosided markets, Proc. 3rd ACM International conference on Web Search and Data Mining (WSDM 10). MAZZOLINI, M. AND MADDISON, S Sage, guide or ghost? The effect of instructor intervention on student participation in online discussion forums. Computers & Education, 40(3). TOPPING, K. AND EHLY, S PeerAssisted Learning, Lawrence Erlbaum.
Discovering Value from Community Activity on Focused Question Answering Sites: A Case Study of Stack Overflow
Discovering Value from Community Activity on Focused Question Answering Sites: A Case Study of Stack Overflow Ashton Anderson Daniel Huttenlocher Jon Kleinberg Jure Leskovec Stanford University Cornell
More informationSteering User Behavior with Badges
Steering User Behavior with Badges Ashton Anderson Daniel Huttenlocher Jon Kleinberg Jure Leskovec Stanford University Cornell University Cornell University Stanford University ashton@cs.stanford.edu {dph,
More informationChapter 17. Network Effects
From the book Networks, Crowds, and Markets: Reasoning about a Highly Connected World. By David Easley and Jon Kleinberg. Cambridge University Press, 2010. Complete preprint online at http://www.cs.cornell.edu/home/kleinber/networksbook/
More informationMaximizing the Spread of Influence through a Social Network
Maximizing the Spread of Influence through a Social Network David Kempe Dept. of Computer Science Cornell University, Ithaca NY kempe@cs.cornell.edu Jon Kleinberg Dept. of Computer Science Cornell University,
More informationIntellectual Need and ProblemFree Activity in the Mathematics Classroom
Intellectual Need 1 Intellectual Need and ProblemFree Activity in the Mathematics Classroom Evan Fuller, Jeffrey M. Rabin, Guershon Harel University of California, San Diego Correspondence concerning
More informationChapter 19. Cascading Behavior in Networks. 19.1 Diffusion in Networks
From the book Networks, Crowds, and Markets: Reasoning about a Highly Connected World. By David Easley and Jon Kleinberg. Cambridge University Press, 010. Complete preprint online at http://www.cs.cornell.edu/home/kleinber/networksbook/
More informationChapter 18. Power Laws and RichGetRicher Phenomena. 18.1 Popularity as a Network Phenomenon
From the book Networks, Crowds, and Markets: Reasoning about a Highly Connected World. By David Easley and Jon Kleinberg. Cambridge University Press, 2010. Complete preprint online at http://www.cs.cornell.edu/home/kleinber/networksbook/
More informationThe Predecessor Attack: An Analysis of a Threat to Anonymous Communications Systems
The Predecessor Attack: An Analysis of a Threat to Anonymous Communications Systems MATTHEW K. WRIGHT, MICAH ADLER, and BRIAN NEIL LEVINE University of Massachusetts Amherst and CLAY SHIELDS Georgetown
More informationFrom Anticipations to Present Bias: A Theory of ForwardLooking Preferences
From Anticipations to Present Bias: A Theory of ForwardLooking Preferences Simone Galperti UC, San Diego Bruno Strulovici Northwestern University October 31, 2014 Abstract How do future wellbeing and
More informationWas This Review Helpful to You? It Depends! Context and Voting Patterns in Online Content
Was This Review Helpful to You? It Depends! Context and Voting Patterns in Online Content Ruben Sipos Dept. of Computer Science Cornell University Ithaca, NY rs@cs.cornell.edu Arpita Ghosh Dept. of Information
More informationChord: A Scalable Peertopeer Lookup Service for Internet Applications
Chord: A Scalable Peertopeer Lookup Service for Internet Applications Ion Stoica, Robert Morris, David Karger, M. Frans Kaashoek, Hari Balakrishnan MIT Laboratory for Computer Science chord@lcs.mit.edu
More informationGraphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations
Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations Jure Leskovec Carnegie Mellon University jure@cs.cmu.edu Jon Kleinberg Cornell University kleinber@cs.cornell.edu Christos
More informationTruthful Mechanisms for OneParameter Agents
Truthful Mechanisms for OneParameter Agents Aaron Archer Éva Tardos y Abstract In this paper, we show how to design truthful (dominant strategy) mechanisms for several combinatorial problems where each
More informationThe Relationship Between Intellectual Property Law and Competition Law: An Economic Approach
The Relationship Between Intellectual Property Law and Competition Law: An Economic Approach Pierre Régibeau and Katharine Rockett University of Essex and CEPR Revised, June 2004 Abstract This paper presents
More informationOPRE 6201 : 2. Simplex Method
OPRE 6201 : 2. Simplex Method 1 The Graphical Method: An Example Consider the following linear program: Max 4x 1 +3x 2 Subject to: 2x 1 +3x 2 6 (1) 3x 1 +2x 2 3 (2) 2x 2 5 (3) 2x 1 +x 2 4 (4) x 1, x 2
More informationApproximation in Economic Design
Approximation in Economic Design Jason D. Hartline 1 Draft: April 15, 2012 1 Northwestern University, Evanston, IL 60208. Email: hartline@eecs.northwestern.edu. 2 Contents 1 Approximation and Mechanism
More informationIn Google We Trust? Roberto Burguet Ramon Caminal Matthew Ellman This version: February 2014 (September 2013)
In Google We Trust? Roberto Burguet Ramon Caminal Matthew Ellman This version: February 2014 (September 2013) Barcelona GSE Working Paper Series Working Paper nº 717 In Google we trust? Roberto Burguet,
More informationDistributed Problem Solving and Planning
Lecture Notes in Computer Science 1 Distributed Problem Solving and Planning Edmund H. Durfee 1 Artificial Intelligence Laboratory EECS Department University of Michigan Ann Arbor, MI 48104, USA durfee@umich.edu
More informationSo Who Won? Dynamic Max Discovery with the Crowd
So Who Won? Dynamic Max Discovery with the Crowd Stephen Guo Stanford University Stanford, CA, USA sdguo@cs.stanford.edu Aditya Parameswaran Stanford University Stanford, CA, USA adityagp@cs.stanford.edu
More informationOtherRegarding Preferences: A Selective Survey of Experimental Results* David J. Cooper Florida State University. John H. Kagel Ohio State University
OtherRegarding Preferences: A Selective Survey of Experimental Results* David J. Cooper Florida State University John H. Kagel Ohio State University 2/12/2013 To appear in the Handbook of Experimental
More informationTO QUEUE OR NOT TO QUEUE: EQUILIBRIUM BEHAVIOR IN QUEUEING SYSTEMS
TO QUEUE OR NOT TO QUEUE: EQUILIBRIUM BEHAVIOR IN QUEUEING SYSTEMS REFAEL HASSIN Department of Statistics and Operations Research Tel Aviv University Tel Aviv 69978, Israel hassin@post.tau.ac.il MOSHE
More informationNot Only What But also When: A Theory of Dynamic Voluntary Disclosure
Not Only What But also When: A Theory of Dynamic Voluntary Disclosure Ilan Guttman, Ilan Kremer, and Andrzej Skrzypacz Stanford Graduate School of Business September 2012 Abstract The extant theoretical
More informationAlgorithms for the multiarmed bandit problem
Journal of Machine Learning Research 1 (2000) 148 Submitted 4/00; Published 10/00 Algorithms for the multiarmed bandit problem Volodymyr Kuleshov Doina Precup School of Computer Science McGill University
More informationHow to Use Expert Advice
NICOLÒ CESABIANCHI Università di Milano, Milan, Italy YOAV FREUND AT&T Labs, Florham Park, New Jersey DAVID HAUSSLER AND DAVID P. HELMBOLD University of California, Santa Cruz, Santa Cruz, California
More informationDesigning MultiPerson Tournaments with Asymmetric Contestants: An Experimental Study. Hua Chen, Sung H. Ham and Noah Lim +
Designing MultiPerson Tournaments with Asymmetric Contestants: An Experimental Study Hua Chen, Sung H. Ham and Noah Lim + + All authors contributed equally. Chen: Bauer College of Business, University
More informationLike What You Like or Like What Others Like? Conformity and Peer Effects on Facebook. Johan Egebark and Mathias Ekström
IFN Working Paper No. 886, 2011 Like What You Like or Like What Others Like? Conformity and Peer Effects on Facebook Johan Egebark and Mathias Ekström Research Institute of Industrial Economics P.O. Box
More informationOtherRegarding Behavior: Theories and Evidence
OtherRegarding Behavior: Theories and Evidence PRELIMINARY AND INCOMPLETE, NOT FOR CITATION Nicola Persico University of Pennsylvania Dan Silverman University of Michigan April 2006 Abstract We use fielddataontraffic
More informationReference point adaptation: Tests in the domain of security trading q
Available online at www.sciencedirect.com Organizational Behavior and Human Decision Processes 105 (2008) 67 81 www.elsevier.com/locate/obhdp Reference point adaptation: Tests in the domain of security
More informationDesign and Evaluation of a WideArea Event Notification Service
Design and Evaluation of a WideArea Event Notification Service ANTONIO CARZANIGA University of Colorado at Boulder DAVID S. ROSENBLUM University of California, Irvine and ALEXANDER L. WOLF University
More informationChapter 3. Strong and Weak Ties
From the book Networks, Crowds, and Markets: Reasoning about a Highly Connected orld. By David Easley and Jon Kleinberg. Cambridge University Press, 2010. Complete preprint online at http://www.cs.cornell.edu/home/kleinber/networksbook/
More information