Are badges useful in education?: it depends upon the type of badge and expertise of learner

Size: px
Start display at page:

Download "Are badges useful in education?: it depends upon the type of badge and expertise of learner"

Transcription

1 Education Tech Research Dev DOI /s DEVELOPMENT ARTICLE Are badges useful in education?: it depends upon the type of badge and expertise of learner Samuel Abramovich Christian Schunn Ross Mitsuo Higashi Ó Association for Educational Communications and Technology 2013 Abstract Educational Badges are touted as an alternative assessment that can increase learner motivation. We considered two distinct models for educational badges; merit badges and videogame achievements. To begin unpacking the relationship between badges and motivation, we conducted a study using badges within an intelligent-tutor system for teaching applied mathematics to middle-school students. Our findings indicate that badge earning could be driven by learner motivations and that systems with badges could have a positive effect on critical learner motivations. However, badge acquisition patterns were different across learners with different levels of prior knowledge. Different badge types also affected different learners motivation. Additionally, we believe that our findings are compatible with the research finding that extrinsic motivators have a negative influence on learning. The implication for educational badge designers is that they must consider the ability and motivations of learners when choosing what badges to include in their curricula. We believe our findings exist as one piece of the large research base needed to understand educational badges. keywords Badges Alternative assessment Motivation Intelligent tutors Introduction For designers, modern educational assessment is an example of the proverbial doubleedged sword. Assessments can contribute to learning and proper understanding (Shepard S. Abramovich (&) C. Schunn Learning Research and Development Center, University of Pittsburgh, Suite O Hara St, Pittsburgh, PA 15260, USA C. Schunn R. M. Higashi Robotics Academy, The National Robotics Engineering Center, Carnegie Mellon University, Ten 40th St., Pittsburgh, PA 15201, USA

2 S. Abramovich et al. 2000) through providing necessary formative feedback as well as additional ways to gauge student learning. But over-assessment and inaccurate assessment have clear negative consequences on motivation (Stiggins 2002). Students who are over-assessed can become motivated to seek mastery of exams rather than genuine learning. Inaccurate assessment is dangerous in that it can cause a drop in motivation to learn for a student who thought they were learning but is provided feedback that contradicts their self-assessment. Because motivation is known to be key to learning (Clark et al. 2006), the potential benefit of an assessment is determined by its ability to both maintain learning motivation and accurately communicate a student s learning. Educational reformers have looked to alternative assessments as a means to maximize the benefits of assessments while minimizing negative effects (Stiggins 2005). Moving away from the culture of standardized testing allows more options for instructional designers in the construction of assessments that permit a wider variety of feedback and data to students and teachers. One alternative assessment that has begun to gain traction among reformers and instructional designers is educational badges (Alberts 2010). Badges, much like their counterparts in scouting and videogames, are seen as a way to assess learning outside of formal schooling. The issuers of educational badges an educator or educational organization can give a symbolic award for any type of skill, knowledge, or achievement similar to how they currently provide degrees or certificates. The symbol, in the form of a badge, can then be displayed by the learner to let others know of their mastery or knowledge. Therefore, instructional designers can use educational badges to influence engagement and learning. For example, badges can provide focused goals, challenging tasks, clear standards, affirmation of performance, novelty, choice, and authenticity (Dickey 2005). The implementation of educational badges is already underway by many organizations such as Mozilla (Peer 2 Peer University and Mozilla Foundation 2011) and the Khan Academy. However, there is little research that examines how badges interact with student motivation. While badges might provide the type of formative feedback valued in alternative assessment, badges could also be a negative influence through decreasing a student s motivation to learn. For example, earlier research about other kinds of rewards has regularly found that external rewards are bad motivators for learning (Deci et al. 1999). If learners interpret badges as external rewards, then they could possibly lower a student s motivation to learn or cause the student to focus on earning badges to the exclusion of the learning goals. In order to begin unpacking the possible bi-directional relationships between educational badges and learning motivation, we conducted a study using badges within an intelligent-tutor system for teaching applied mathematics to middle-school students. We investigated how learner motivation changed with exposure to badges and how learner motivations shaped badge acquisition. Theoretical background Alternative models of badges Badge advocates claim that badges can be offered as an alternative assessment that will increase learner motivation while maintaining high-quality feedback (Davidson 2011). To understand where these claims originate, let us consider the two distinct models for educational badges: merit badges and videogame achievements.

3 Are badges useful in education? Merit-badges, a feature of the United States Boy and Girl Scout organizations, offers participants the chance to earn certification (i.e. a Badge) of specific knowledge or skills not usually addressed by formal educational systems. Children select which badges they want to earn, under the theory that the goal of earning a badge will trigger an increase in motivation for those learners who want formalized recognition. Additionally, a scout s display of their earned badges represents a type of curriculum vitae of their learning and allows others to learn both what a scout knows and what the scout values. The second model of badges is based on meta-gaming features common to videogames. Many best-selling videogame systems allow players to earn recognition of their in-game achievements outside of the game itself. For example, Xbox players have a virtual profile displaying their different game accomplishments for others to see. These achievements (i.e. badges) are awarded to players while playing a game but have no direct effect on play other than being designed to encourage more play. Within the Xbox ecosystem, a player can compare their achievements with those earned by their friends and peers. Player can choose to earn specific badges but also can earn a badge incidentally through normal game play. Educational badges share many of the same features of the merit-badge and video-game models. Like merit badges, educational badges are commonly offered for learning that occurs outside of traditional educational institutions (e.g. community building, online literacy). Educational badges are commonly viewable on a learner s online profile by a learner s peers, similar to how videogame badges are viewable to other players and how scouts display badges on sashes. Similar to videogame achievements, badges can be awarded for incidental activity in addition to mastery of skills or demonstration of knowledge. An example of an educational badge system is Carnegie Mellon University s Computer Science Student Network (CS2N; CS2N is an online learning system in which participants can earn badges while developing computer-science skills and knowledge. Although computer science courses can be found in some public schools, CS2N cover aspects of computer science that are often skipped (e.g. animation, robotics). Each CS2N user has a profile page on which they can see the badges they have earned as well as potential badges they might wish to earn. The variety of educational applications available in CS2N (e.g. intelligent tutors, virtual robots, learning management systems) on a variety of topics (e.g. algebra, programming, movie making) provides students a breadth of badges to pursue. Within each application, badges are awarded for progress toward mastery as well as continued participation. Psychological theories of learner motivation Because motivation and assessment are intertwined, to understand the educational potential of badges requires understanding of how badges affect learning motivation. In the past 15 years, a particularly successful theory for studying motivation and learning is achievement goal theory (Elliot 1999). Achievement goal theory frames motivational learning goals into four different types organized into a matrix of mastery and performance crossed with approach and avoidance. Mastery approach goals reflect a desire to achieve mastery based on one s own interest. Performance approach goals reflect a desire to perform demonstrably better. Performance avoidance goals reflect the desire to avoid the appearance of underperforming. Mastery avoidance goals, while existing theoretically, do not manifest in most real-world contexts. Research typically finds a positive correlation between mastery learning and academic performance outcomes and a negative

4 S. Abramovich et al. correlation between performance avoidance and academic outcomes (Elliot et al. 2006). Performance approach goals have had mixed outcomes, depending on other variables (Elliott et al. 2005). Overall, achievement goal orientation has proven to be a good predictor of academic performance in various academic subjects (Pajares et al. 2000). For example, mastery goals were found to positively correlate with student use of cognitive strategies while performance approach goals positively correlated with teacher-assigned grades (Wolters 2004). Because badges, at first glance, have performance and mastery elements, achievement goal theory may be a useful way of unpacking the effects of educational badges on learning-relevant motivations. It is important to note that achievement goal theory is only one well-established theory of learning motivation. Another established theoretical framing of motivation is expectancy-value theory (Wigfield and Eccles 2000). How a student expects to perform in a subject is known to be an independent predictor of achievement. For example, students with low confidence in their math ability are more likely to guess an answer to a math question rather than work to solve it (Beal et al. 2008). Additionally, the level of interest of a student has in a subject (intrinsic value) can also independently predict future learning. For example, students who are interested in math will continue to enroll in math related courses, increasing their opportunity to learn relative to students less interested in math (Pintrich 2003). Educational badges, in addition to potentially changing a learner s achievement goals, might also change how much the learner values a subject (by having them choose goals to pursue) or what their expectations are for success (by regularly flagging success or lack thereof). Interactions between learner prior knowledge and badges The ways in which learners react to badges may depend upon a range of prior experiences with the domain being badged. Prior knowledge and its interaction with other student characteristics can have a major influence on assessment performance (Dochy et al. 1999). In addition to having built up particular achievement goals, interests, and expectations for success, a learner will have prior knowledge levels that will shape how quickly and easily badges are earned, which then may also shape how much the learner values these badges. Therefore we examine whether prior ability levels moderate the relationships of badge earning with learner motivations. Methods Participants 36 seventh graders and 15 eighth graders at a charter school serving a low-income suburb of a large east coast city in North America participated in our study. Measures To measure achievement goal orientation, we used a version of the patterns of adaptive learning scales (PALS) survey instrument (Midgley et al. 2000) modified to measure motivation related to the content area focus of the study, math. The measures of Mastery goals (e.g. One of my goals this year is to master a lot of new skills in Math. ),

5 Are badges useful in education? Performance Approach goals (e.g. It s important to me that I look smart compared to others in Math. ), and Performance Avoidance goals (e.g. One of my goals in Math is to avoid looking like I have trouble doing the work. ) contained four questions each with answers in a Likert Scale from 1 5. Pre Cronbach alphas for mastery goals, performance approach, and performance avoidance were 0.82, 0.81, and 0.68 and post alphas were 0.81, 0.84, and To measure the key constructs from expected value theory, we used five questions with answers in a Likert Scale related to expectancy (e.g. If you had to take an important math test today, would you do well? ), and two questions related to interest (intrinsic value, e.g. In general, I find working on math assignments very interesting. ) that were adapted from a variety of studies (Beal et al. 2008; Wigfield 1994; Wigfield and Eccles 2000). Motivation and interest alphas, respectively, were 0.81 and 0.83 for pre measures and 0.78 and 0.82 for post measures. To determine if badge interaction with learning motivation was different for students with different levels of skills, we used a pretest of proportional reasoning that has been validated for this age group and focuses on the exact skills taught in the tutor (Weaver and Junker 2004). We divided the students in the study into two groups based on a mean split of the number of pre-assessment math problems correct. The low-performing students had a range of correct problems from 3 to 10. High-performing students had a range of problems correct. Students opinions on badges were measured using questions previously established in an earlier study (Abramovich et al. 2011) which in turn were created through several pilot studies. Because of the relative newness of educational badges, we are unaware of any other badge specific opinion questions used in prior research. Procedure We measured pre-ability and pre and post motivational levels around the use of the CS2N intelligent tutoring system. Students earned various forms of badges in the CS2N intelligent tutoring system and were notified of all awarded badges through a dialogue box within the tutor interface (Fig. 1). Students were initially unaware of the available badges but Fig. 1 Examples of participation (left) and skill (right) badges in the CS2N intelligent tutor

6 S. Abramovich et al. would compare and converse with each other about the different type of badges they had earned. The intelligent tutoring system builds upon a Cognitive Tutor approach developed at Carnegie Mellon University and deployed in thousands of schools (Anderson et al. 1995). The tutor could provide hints when students were stuck and kept track of skills being mastered based on successfully solving particular solution steps associated with each skill, but doing so without error or hint. The content area focus of the particular unit called robots in motion was proportional reasoning, understood to be a foundation for mathematics, science, and engineering education (Silk et al. 2010). Students spent on average 20 min each school day with the tutor over *1 month period. They learned how to use various mathematical reasoning strategies to program a robot to move certain distances and turn certain angles in the context of trying to solve a larger problem involving controlling a robot on a distant asteroid, eliminating the guessand-check methodology commonly employed by novice roboticists (see Fig. 2 for an example task page). In the particular version of CS2N and the intelligent tutor that was studied, badges were granted based on metrics related to progress within the tutor and mastery of measured skills. Accordingly, we classified the badges awarded by the tutor into two distinct categories (Table 1): those indicating mastery of skills and those reflecting participation in the system. Prior testing had indicated the potential for badges to re-motivate students whose frustration level was increasing while using the tutor (Abramovich et al. 2011). Fig. 2 Example tutor page. On this page, students respond via pull-down menus, whereas on some other pages, students type in number responses or free text explanations. When students click on the hint button, hints are shown in the hints window

7 Are badges useful in education? Consequently, we theorized that badges awarded for participation, regardless of the quality of the user s performance, would provide increased motivation to all users. Badges that were awarded based on mastery of skills within the tutor were aligned with the theory that badges can act as an alternative assessment. Because of their different theoretical functions with respect to motivation and learning, we separately examined correlations with number of participatory badges and number of skill badges. In addition, exploratory factor analyses on badge earning data suggested there are two underlying dimensions of skill and participation (see Table 1 for the list of each). Our research questions were: 1. Do badges have a different motivation relationship for different ability learners? 2. Do different types of badges have different motivation relationships with learners? Table 1 Badges available in the CS2N Intelligent tutoring system and number earned overall and by level of learner Description Category Number earned HP LP Correct a mistake in less than 2 min Skill Complete a problem with no errors in less than 5 min Skill Complete a problem with no errors in under 1 min Skill Find distance using scale factor Skill Find distance using unit rate Skill Find power using scale factor Skill Find power using unit rate Skill Count half of a turn Skill Count a whole turn Skill Measure with non-zero aligned start Skill Measure with zero aligned start Skill Measure remainder Skill min of cumulative cognitive tutor time Participatory min of cumulative cognitive tutor time Participatory min of cumulative cognitive tutor time Participatory min of cumulative cognitive tutor time Participatory min of cumulative cognitive tutor time Participatory Answer 25 questions correctly Participatory Answer 50 questions correctly Participatory Answer 75 questions correctly Participatory Answer 100 questions correctly Participatory Answer 200 questions correctly Participatory Answer 5 questions correctly Participatory Correct a mistake in under 5 min Participatory Complete a page with a mistake in at least 10 min Participatory Complete a page with an error in at least 15 min Participatory Getting a hint Participatory HP badges earned by high math performers, LP badges earned by low math performers

8 S. Abramovich et al. Table 2 Mean pre, post, and change in motivation measures (with S.D.) Measure Pre Post Change Low High All Low High All Low High All Mastery 4.5 (0.5) 4.6 (0.7) 4.6 (0.7) 4.5 (0.8) 4.6 (0.7) 4.5 (0.8) 0.0 (0.7) 0.0 (0.4) 0.0 (0.5) Performance approach 3.4 (1.0) 3.2 (1.3) 3.3 (1.2) 3.4 (1.0) 2.8 (1.3) 3.1 (1.2) 0.0 (0.9) -0.4 (1.0) -0.2 (1.0) Performance avoidance 3.5 (1.0) 2.8 (1.2) 3.1 (1.2) 3.0 (0.9) 2.7 (1.2) 2.9 (1.1) -0.5** (0.7) -0.1 (0.9) -0.2* (0.8) Expectancy 3.8 (0.8) 4.6 (0.4) 4.2 (0.7) 3.9 (0.7) 4.6 (0.3) 4.3 (0.7) 0.1 (0.5) 0.0 (0.4) 0.1 (0.4) Interest 3.4 (1.2) 4.2 (1.0) 3.8 (1.2) 3.8 (1.2) 4.3 (0.9) 4.1 (1.1) 0.4* (0.9) 0.1 (0.7) 0.3* (0.8) * p \ 0.05; ** p \ 0.01

9 Are badges useful in education? Fig. 3 Mean pre- and postscores (with SE bars) on performance avoidance and interest measures for low and high math pre-test groups.**p \ 0.01, *p \ 0.05 Findings Aggregate analyses across all learners Analysis of all students (n = 51) revealed a few small overall changes in motivations. Paired sample t-tests of pre to post changes revealed a 0.25 decrease (on the 1 to 5 scale) in performance avoidance motivation (t = 2.06, p \ 0.05, Cohen s d = 0.29) and a 0.25 increase in interest (t = 2.34, p \ 0.05, Cohen s d = 0.33). Overall, students who used the cognitive tutor became less concerned about poor performance relative to other students and became more interested in math. To discover if badges interacted with learning motivation (i.e. were predicted by pre motivation levels or were associated with changes in motivation levels), we conducted Pearson correlations between the number of participatory badges earned and the different measures of pre motivation, prior ability, and pre-post changes in motivation. Pre measures of motivation and number of math problems correct did not correlate with the total number of badges earned by students. However, the total number of badges earned did correlate with an increase in performance avoidance motivation (r = 0.30, p \ 0.05). The more badges a student earned indicated less of an overall decrease in the same student s concern about their performance. Based on these limited findings, we might have concluded that while the cognitive tutor might have had a positive effect on some learning motivation, the inclusion of badges possibly has a negative effect on learning. However, we further unpacked our findings, looking for whether the effects are actually specific to particular student subgroups and/or badges types. Do lower performing students and higher performing students see different motivation changes? As shown in Table 2, for the low-performing students (n = 23), we discovered that performance avoidance motivation (e.g. One of my goals in Math is to avoid looking like I have trouble doing the work.) decreased by 0.48 (t = 3.49, p \ 0.01, Cohen s d = 0.73)

10 S. Abramovich et al. Table 3 Pearson correlations (and p values) between motivation pre measures and skill and participatory badges earning for all users and separately by math performance levels Pre measures Skill badges Participation badges All users HP LP All users HP LP Mastery 0.19 (0.18) 0.13 (0.50) 0.24 (0.26) 0.19 (0.21) 0.16 (0.43) 0.19 (0.39) Performance (0.68) (0.53) 0.23 (0.30) 0.06 (0.68) (0.38) 0.48* (0.02) approach Performance -0.34* (0.01) -0.39* (0.04) 0.09 (0.67) (0.12) (0.10) 0.10 (0.65) avoidance Expectancy 0.14 (0.34) (0.06) 0.07 (0.74) 0.18 (0.21) (0.72) 0.08 (0.73) Interest 0.27 (0.06) 0.27 (0.16) (0.96) 0.06 (0.70) 0.09 (0.64) (0.47) HP high math performers, LP low math performers * p \ 0.05 Table 4 Correlations between changes in Motivation measures and Badges Badges Increase in performance avoidance Increase in expectancy All users HP LP All users HP LP Skill 0.40** (0.00) 0.37 (0.05) 0.22 (0.31) 0.20 (0.17) 0.49** (0.01) (0.72) Participation 0.30* (0.03) 0.13 (0.50) 0.46* (0.03) 0.04 (0.76) 0.21 (0.28) (0.84) HP high math performers, LP low math performers Pearson correlations, * p \ 0.05; ** p \ 0.01 and interest in math increased by 0.43 (t = 2.40, p \ 0.05, Cohen s d = 0.50) over the course of using the cognitive tutor. There were no statistically significant changes in motivation for the high-performing students (n = 28). That is, the changes in motivation that we previously uncovered for all students are, in actuality, only for low-performing students (see Figure 3). Is badge earning predicted by different motivational factors in high and low math performers? For the low-performing students, the pre measure of performance approach motivation (e.g. It s important to me that I look smart compared to others in Math.) correlated with the total badges earned (r = 0.47, p \ 0.05) and total badges earned correlated with an increase in performance avoidance (r = 0.42, p \ 0.05). Based on this data, we hypothesize that among low-performing students, those with a higher desire to outperform other students earned more badges. However, the more badges earned by low-performing students also indicated less of an overall decrease in concern about their performance relative to other students. For the high-performing students, there was a negative correlation between pre performance avoidance motivation and total badges earned (r =-0.39, p \ 0.05). This negative correlation indicates that, among the high performing students, those who were less concerned about having poor performance relative to other students were the ones who earned more badges.

11 Are badges useful in education? Fig. 4 For low-performing students, change in performance avoidance goals from pre to post as a function of # of participation badges earned Fig. 5 For high-performing students, change in expectancies for success from pre to post measures as a function of number of skill badges earned Having discovered that different ability students motivations have different relations to badges, we looked to discover whether different types of badges have different relationships with student motivation. Specifically, we performed correlations between the

12 S. Abramovich et al. different motivation measures and the two classifications of badges: participation and skill (see Table 3). For low-performing students, pre performance approach motivation (e.g. It s important to me that I look smart compared to others in Math. ) correlated with earning participation badges (r = 0.48, p \ 0.05) but not skill badges. That is, the prior correlations we detected for low-performing students can be further limited to only participation badges. For the high-performing students, pre performance avoidance (e.g. t s important to me that I don t look stupid in Math. ) had a negative correlation with skill badges (r =-0.39, p \ 0.05) but no correlation with participation badges. In fact, earning participation badges did not correlate with any pre measure for the high-performing students. Just as we can limit the correlations to low-performing student to only participatory badges, we can as well limit the correlations detected for high-performing students to just skill badges. Does badge earning predict motivational changes? As shown in Table 4 and Fig. 4, in low-performing students, earning participatory badges correlated with an increase in performance avoidance motivation (r = 0.46, p \ 0.05) while earning skill badges had no correlation with any measure. Note that relatively few students had an absolute growth in performance avoidance goals. Rather, the Table 5 Mean ratings (and SD) for each question in the badge opinion survey for all students and separately for high and low performing students Questions All students HP LP I understand why I earned all of my badges/achievements 4.3 (1.0) 4.4 (1.1) 4.2 (0.8) The badges/achievements were more important to me than other parts 2.6 (1.4) 2.6 (1.5) 2.7 (1.4) of the cognitive Tutor I think that the badges/achievements are a good addition to the 3.6 (1.4) 3.6 (1.4) 3.6 (1.4) cognitive tutor I knew what badges/achievements were before I stated working 2.8 (1.6) 2.9 (1.7) 2.6 (1.4) on the cognitive tutor I wanted to earn more robots in motion tutor badges/achievements 3.3 (1.5) 3.4 (1.5) 3.3 (1.4) I don t care about the robots in motion tutor badges/achievements 2.6 (1.4) 2.4 (1.5) 2.8 (1.3) I like earning badges/achievements but not the ones in the robots in 2.8 (1.4) 2.5 (1.4) 3.3 (1.4) motion tutor I wish the robots in motion tutor badges/achievements were harder 2.7 (1.5) 2.9 (1.6) 2.4 (1.2) to earn I wish the robots in motion tutor badges/achievements were easier 2.9 (1.5) 2.8 (1.7) 3.0 (1.4) to earn I want to look at all of the badges/achievements I earned in the robots 3.5 (1.5) 3.6 (1.7) 3.3 (1.4) in motion tutor I told others about my robots in motion tutor badges/achievements 2.4 (1.4) 2.3 (1.4) 2.6 (1.3) The robots in motion tutor badges/achievements made me want to keep working 3.5 (1.4) 3.6 (1.4) 3.3 (1.4) HP high math performers, LP low math performers

13 Are badges useful in education? low-performing students that earned the most participatory badges had a mixture of increases and decreases in performance avoidance goals. The low-performing students that earned few such badges consistently had decreases in performance avoidance goals. By contrast, as show in Table 4, for high-performing students earning more participation badges had no relationship to changing motivations. Instead it was the earning of skill badges that was associated with motivational changes and, in particular, the number of skill badges earned was highly correlated with an increase in expectancy to do well at math (r = 0.49, p \ 0.01; see Fig. 5). Here we see that students earning few skill badges tended to experience a drop in expectancies for success, whereas those who earned many skill badges tended to experience a growth in expectancies for success. Do lower performing students and higher performing students have different opinions about badges? Because educational badges are often a new experience for students, we also surveyed the students about their experience with the intelligent tutoring system badges. Overall, the students responses were favorable, with means towards the positive end of each scale for all questions (i.e. above 3 for positive statements, below 3 for negative statements; see Table 5). Further, this positive view was held by both subgroups, and there were also no statistically significant differences (p \ 0.05) in the mean ratings between high and lower performing students on any of the statements except for I like earning badges/achievements but not the ones in the robots in motion tutor (t = 2.05, p \ 0.05, Cohen s d = 0.58). Low performing students indicated more agreement with this statement, possibly related to changes in their performance avoidance motivation. To try and unpack some of the differences we had detected between low-performing and high-performing students on motivational changes and relationships to badges, we looked for correlations between these survey opinions and their scores on the motivation measures. For low-performing students, an increase in performance avoidance correlated with I wish tutor badges were harder (r = 0.42, p \ 0.05) and I knew about badges before I started the tutor (r = 0.48, p \ 0.05). For high-performing students, an increase in expectancy to do well at math correlated with I wish tutor badges were harder (r = 0.46, p \ 0.05) but also I understood why I earned all of my badges (r = 0.42, p \ 0.05). All other correlations with opinion questions were non-significant. Discussion Overall, we found evidence of improvements in interest and decrease in counter-productive motivational goal from a system using educational badges. Further, we find evidence that earning various badges can be associated in increases in expectations for success but also increases in counter-productive educational goals. Thus, in contrast to what might be expected from conceptualizing badges as only being extrinsic rewards (and therefore only bad for learning), we find evidence suggesting both positive and negative effects. However, it is also very salient that effects of educational badges vary with different ability learners: badge acquisition patterns were quite different across learners with different levels of prior knowledge. Only for the low-performing students in our study did a higher desire to outperform other students, the performance approach goal, correlate with earning more badges. But the more badges earned by low-performing students also indicated less of an overall decrease in concern about their performance relative to other

14 S. Abramovich et al. students, the performance avoidance goal. Because we did not find these same patterns for high-performing students, we conclude that educational badges adhere to the research finding that prior knowledge can have a major influence on assessment performance (Dochy et al. 1999). Different badge types also affected different learners motivation. The motivation relationship between badges and motivation for low performing students was limited to participatory badges. Skill badges earned by the low-performing students did not correlate with the change in performance avoidance goals. Not only do we conclude that different types of badges will have different effects on student motivation to learn but we also conclude that different types of badges will also affect learning performance since motivation can predict future learning performance (Pajares et al. 2000). Additionally, we believe that our findings are compatible with the research finding that extrinsic motivators can have a negative influence on learning (Deci et al. 2001). Participation badges, earned eventually by all learners, had minimal explicit connection to individual measure of skill. Consequently, learners might not be able to make a connection between internal motivation and participatory badges, perceiving of them as only external motivators. Skill badges, awarded based on direct student performance, perhaps could be more easily connected to internal motivation, and thus considered by learners as an intrinsic motivator. As an alternative assessment technique, we conclude that the design specifics of educational badges in addition to the targeted students will be the main predictors of badge influence on learning motivation. The implication for instructional designers of badges is that they must consider the ability and motivations of learners when choosing what badges to include in their curricula. If badges are offered for learners who might not excel in the content area of the badges, then there is the potential, depending on the design of the badges, for a negative motivational effect. Lowering or removing the number of participatory badges or increasing the number of skill related badges might mitigate some of the negative motivational effects. Based on our opinion survey, we also conject that providing detail to learners about how to earn the badges and what actions by resulted in earning a badge will also mitigate some negative motivational effects while preserving the assessment goal of badges. We believe that our findings represent a significant first step in empirically establishing how educational badges affect learning motivation. There is little existing empirical work on the effects of badges, despite much theoretical work and considerable system design work in many contexts. However, more research is necessary to understand the broader motivational impacts of badges under different circumstances, across other kinds of badges, at other ages, and for different kinds of learning environments. Advocates of educational badges need to further understand the interplay between different type of learners and different type of badges. Future research should replicate our findings and examine additional differentiation between badges and learners as well as the impact of choosing which badges to earn. Additionally, our findings do not allow us to determine the temporal effects of badges on motivation. Consequently, we believe our findings exist as one piece of the large research base needed to understand educational badges. Acknowledgments Work on this project was funded through grants from the Digital Media and Learning competition and the Defense Advanced Research Projects Agency.

15 Are badges useful in education? References Abramovich, S., Higashi, R., Hunkele, T., Schunn, C., & Shoop, R. (2011). An Achievement System to Increase Achievement Motivation. Paper presented at the games learning society 7.0, Madison, WI. Alberts, B. (2010). An education that inspires. Science, 330(6003), 427. Anderson, J. R., Corbett, A. T., Koedinger, K. R., & Pelletier, R. (1995). Cognitive tutors: Lessons learned. The Journal of the Learning Sciences, 4(2), Beal, C., Qu, L., & Lee, H. (2008). Mathematics motivation and achievement as predictors of high school students guessing and help-seeking with instructional software. Journal of Computer Assisted Learning, 24(6), Clark, R. E., Howard, K., & Early, S. (2006). Motivational challenges experienced in highly complex learning environments. In J. Elen & R. E. Clark (Eds.), Handling complexity in learning environments: Theory and research (pp ). Oxford: Elsevier. Davidson CN (2011). Could badges for lifelong learning be our tipping point? Retrieved November 1, 2011, from Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of experiments examining the effects of extrinsic rewards on intrinsic motivation. Psychological Bulletin, 125(6), 627. Deci, E. L., Koestner, R., & Ryan, R. M. (2001). Extrinsic rewards and intrinsic motivation in education: Reconsidered once again. Review of Educational Research, 71(1), 1. Dickey, M. (2005). Engaging by design: How engagement strategies in popular computer and video games can inform instructional design. Educational Technology Research and Development, 53(2), Dochy, F., Segers, M., & Buehl, M. M. (1999). The relation between assessment practices and outcomes of studies: The case of research on prior knowledge. Review of Educational Research, 69(2), Elliot, A. J. (1999). Approach and avoidance motivation and achievement goals. Educational psychologist, 34(3), Elliot, A. J., Cury, F., Fryer, J. W., & Huguet, P. (2006). Achievement goals, self-handicapping, and performance attainment: A mediational analysis. Journal of Sport and Exercise Psychology, 28, Elliott, A. J., Shell, M. M., Henry, K. B., & Maier, M. A. (2005). Achievement goals, performance contingencies, and performance attainment: An experimental test. Journal of Educational Psychology, 97(4), 630. Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E., et al. (2000). Manual for the patterns of adaptive learning scales (PALS). Ann Arbor: University of Michigan. Pajares, F., Britner, S. L., & Valiante, G. (2000). Relation between achievement goals and self-beliefs of middle school students in writing and science. Contemporary Educational Psychology, 25(4), Peer 2 Peer University, & Mozilla Foundation. (2011). An open badge system framework. Retrieved July 14, 2011, from Pintrich, P. R. (2003). A motivational science perspective on the role of student motivation in learning and teaching contexts. Journal of Educational Psychology, 95(4), Shepard, L. A. (2000). The role of assessment in a learning culture. Educational Researcher, 29(7), Silk, E. M., Higashi, R., Shoop, R., & Schunn, C. D. (2010). Designing technology activities that teach mathematics. The Technology Teacher, 69(4), Stiggins, R. J. (2002). Assessment crisis: The absence of assessment for learning. Phi Delta Kappan, 83(10), Stiggins, R. J. (2005). From formative assessment to assessment for learning: A path to success in standardsbased schools. The Phi Delta Kappan, 87(4), Weaver R & Junker B (2004). Model specification for cognitive assessment of proportional reasoning: Department of Statistics Technical Report. Wigfield, A. (1994). Expectancy-value theory of achievement motivation: A developmental perspective. Educational Psychology Review, 6(1), Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), Wolters, C. A. (2004). Advancing achievement goal theory: Using goal structures and goal orientations to predict students motivation, cognition, and achievement. Journal of Educational Psychology, 96(2), 236.

16 S. Abramovich et al. Samuel Abramovich is a Doctoral Student at the Learning Research and Development Center at the University of Pittsburgh. Prior to that he was a researcher at the Center for Technology in Education at Johns Hopkins University, a Technology Coordinator for a Boston Independent School, and a Social Studies Teacher. His most current research projects include assessing how Badges can be constructed to motivate and assess in a variety of in-school and out-of-school contexts and how large-scale online feedback impacts teacher practice. His past research looked at the use of games and virtual-reality to teach at-risk youth and how design-based instruction is best integrated with technology education. Christian Schunn is a Senior Scientist at the Learning Research and Development Center and a Professor of Psychology, Learning Sciences and Policy, and Intelligent Systems at the University of Pittsburgh. He directs a number of research projects in science, mathematics, and engineering education. This work includes studying expert engineering and science teams, building innovative technology-supported STEM curricula, and studying factors that influence student and teacher learning. Ross Higashi is a Robotics Education Specialist at the Robotics Academy, a Carnegie Mellon educational outreach program based at the National Robotics Engineering Center in Pittsburgh, PA. He works on the design of CS2N the computer science student network (http://www.cs2n.org) and co-leads the development of its badge system and cognitive tutored curricular units. He graduated with a B.S. in Logic and Computation from CMU, where he also studied computer science and human computer interaction.

Are badges useful in education?: it depends upon the type of badge and expertise of learner

Are badges useful in education?: it depends upon the type of badge and expertise of learner Education Tech Research Dev (2013) 61:217 232 DOI 10.1007/s11423-013-9289-2 DEVELOPMENT ARTICLE Are badges useful in education?: it depends upon the type of badge and expertise of learner Samuel Abramovich

More information

The Roles of Badges in the Computer Science Student Network

The Roles of Badges in the Computer Science Student Network The Roles of Badges in the Computer Science Student Network Ross Higashi, Robotics Academy, Carnegie Mellon University Samuel Abramovich, LRDC, University of Pittsburgh Robin Shoop, Robotics Academy, Carnegie

More information

The effects of highly rated science curriculum on goal orientation engagement and learning

The effects of highly rated science curriculum on goal orientation engagement and learning APA 04 Proposal The effects of highly rated science curriculum on goal orientation engagement and learning Purpose The paper to be presented describes a study of goal orientation (Pintrich, 2000), engagement

More information

Research Basis for Catchup Math

Research Basis for Catchup Math Research Basis for Catchup Math Robert S. Ryan, Ph. D. Associate Professor of Cognitive Psychology Kutztown University Preface Kutztown University is a 4 year undergraduate university that is one of 14

More information

The relationship between goals, metacognition, and academic success

The relationship between goals, metacognition, and academic success Educate~ Vol.7, No.1, 2007, pp. 39-47 Research Paper The relationship between goals, metacognition, and academic success by Savia A. Coutinho (saviac@yahoo.com) Northern Illinois University, United States

More information

Higher Performing High Schools

Higher Performing High Schools COLLEGE READINESS A First Look at Higher Performing High Schools School Qualities that Educators Believe Contribute Most to College and Career Readiness 2012 by ACT, Inc. All rights reserved. A First Look

More information

TOOL KIT for RESIDENT EDUCATOR and MENT OR MOVES

TOOL KIT for RESIDENT EDUCATOR and MENT OR MOVES Get to Know My RE Observe Collect Evidence Mentor Moments Reflect Review Respond Tailor Support Provide Provide specific feedback specific Feedback What does my RE need? Practice Habits Of Mind Share Data

More information

Student Preferences for Learning College Algebra in a Web Enhanced Environment

Student Preferences for Learning College Algebra in a Web Enhanced Environment Abstract Student Preferences for Learning College Algebra in a Web Enhanced Environment Laura Pyzdrowski West Virginia University Anthony Pyzdrowski California University of Pennsylvania It is important

More information

A Hands-On Exercise Improves Understanding of the Standard Error. of the Mean. Robert S. Ryan. Kutztown University

A Hands-On Exercise Improves Understanding of the Standard Error. of the Mean. Robert S. Ryan. Kutztown University A Hands-On Exercise 1 Running head: UNDERSTANDING THE STANDARD ERROR A Hands-On Exercise Improves Understanding of the Standard Error of the Mean Robert S. Ryan Kutztown University A Hands-On Exercise

More information

Designing Technology Activities that Teach Mathematics

Designing Technology Activities that Teach Mathematics Designing Technology Activities that Teach Mathematics By Eli M. Silk, Ross Higashi, Robin Shoop, and Christian D. Schunn Teaching mathematics in a technology classroom requires more than simply using

More information

Evgenia Theodotou Metropolitan College, Athens, Greece. Abstract. Introduction. Motivation to learn: the theoretical framework

Evgenia Theodotou Metropolitan College, Athens, Greece. Abstract. Introduction. Motivation to learn: the theoretical framework Vol.4, No.1. April 2014 pp. 17 21. Early years education: are young students intrinsically or extrinsically motivated towards school activities? A discussion about the effects of rewards on young children

More information

July 2009 Research Brief: Factors that support academic success

July 2009 Research Brief: Factors that support academic success July 2009 Research Brief: Factors that support academic success The single most important factor determining a student s academic success is the academic preparation a student receives and the rigor of

More information

Design Principles for Video Games as Learning Engines

Design Principles for Video Games as Learning Engines 1 Design Principles for Video Games as Learning Engines James Paul Gee Mary Lou Fulton Presidential Professor of Literacy Studies Regents Professor Arizona State University james.gee@asu.edu The literature

More information

ANALYZING THE BENEFITS OF USING TABLET PC-BASED FLASH CARDS APPLICATION IN A COLLABORATIVE LEARNING ENVIRONMENT A Preliminary Study

ANALYZING THE BENEFITS OF USING TABLET PC-BASED FLASH CARDS APPLICATION IN A COLLABORATIVE LEARNING ENVIRONMENT A Preliminary Study ANALYZING THE BENEFITS OF USING TABLET PC-BASED FLASH CARDS APPLICATION IN A COLLABORATIVE LEARNING ENVIRONMENT A Preliminary Study YoungJoo Jeong Human Computer Interaction Institute, Carnegie Mellon

More information

ACT National Curriculum Survey 2012. Policy Implications on Preparing for Higher Standards. improve yourself

ACT National Curriculum Survey 2012. Policy Implications on Preparing for Higher Standards. improve yourself ACT National Curriculum Survey 2012 Policy Implications on Preparing for Higher Standards improve yourself ACT is an independent, not-for-profit organization that provides assessment, research, information,

More information

A Guide to Curriculum Development: Purposes, Practices, Procedures

A Guide to Curriculum Development: Purposes, Practices, Procedures A Guide to Curriculum Development: Purposes, Practices, Procedures The purpose of this guide is to provide some general instructions to school districts as staff begin to develop or revise their curriculum

More information

Abstract Title Page. Authors and Affiliations: Maria Mendiburo The Carnegie Foundation

Abstract Title Page. Authors and Affiliations: Maria Mendiburo The Carnegie Foundation Abstract Title Page Title: Designing Technology to Impact Classroom Practice: How Technology Design for Learning Can Support Both Students and Teachers Authors and Affiliations: Maria Mendiburo The Carnegie

More information

Research and Digital Game- based Play: A Review of Martha Madison

Research and Digital Game- based Play: A Review of Martha Madison Research and Digital Game- based Play: A Review of Martha Madison White Paper Compiled by Anne Snyder, Ph.D. Second Avenue Learning is a certified women- owned business, specializing in the creation of

More information

Designing Socio-Technical Systems to Support Guided Discovery-Based Learning in Students: The Case of the Globaloria Game Design Initiative

Designing Socio-Technical Systems to Support Guided Discovery-Based Learning in Students: The Case of the Globaloria Game Design Initiative Designing Socio-Technical Systems to Support Guided Discovery-Based Learning in Students: The Case of the Globaloria Game Design Initiative Rebecca Reynolds 1, Sean P. Goggins 2 1 Rutgers University School

More information

Robert W. Roeser and Carol Midgley University of Michigan. Timothy C. Urdan Emory University

Robert W. Roeser and Carol Midgley University of Michigan. Timothy C. Urdan Emory University Journal of Educational Psychology 1996, Vol. 88, No. 3, 408-422 Copyright 1996 by the American Psychological Association, Inc. 0022-0663/96/S3.00 Perceptions of the School Psychological Environment and

More information

Practices Worthy of Attention Asset-Based Instruction Boston Public Schools Boston, Massachusetts

Practices Worthy of Attention Asset-Based Instruction Boston Public Schools Boston, Massachusetts Asset-Based Instruction Boston, Massachusetts Summary of the Practice. has made asset-based instruction one of its priorities in its efforts to improve the quality of teaching and learning. The premise

More information

An Individualized Web-based Algebra Tutor Based on Dynamic Deep Model Tracing

An Individualized Web-based Algebra Tutor Based on Dynamic Deep Model Tracing An Individualized Web-based Algebra Tutor Based on Dynamic Deep Model Tracing Dimitrios Sklavakis 1 and Ioannis Refanidis 1 1 University of Macedonia, Department of Applied Informatics, Egnatia 156, P.O.

More information

Elementary MEd I. The Relationship of the Program with the Unit s Conceptual Framework

Elementary MEd I. The Relationship of the Program with the Unit s Conceptual Framework Elementary MEd I. The Relationship of the Program with the Unit s Conceptual Framework Shaping Tomorrow: Ideas to Action The Early Elementary Education program for prospective elementary education candidates

More information

Text of article appearing in: Issues in Science and Technology, XIX(2), 48-52. Winter 2002-03. James Pellegrino Knowing What Students Know

Text of article appearing in: Issues in Science and Technology, XIX(2), 48-52. Winter 2002-03. James Pellegrino Knowing What Students Know Text of article appearing in: Issues in Science and Technology, XIX(2), 48-52. Winter 2002-03. James Pellegrino Knowing What Students Know Recent advances in the cognitive and measurement sciences should

More information

BEST PRACTICES IN INTERVENTION & REMEDIATION

BEST PRACTICES IN INTERVENTION & REMEDIATION BEST PRACTICES IN INTERVENTION & REMEDIATION Achieve has compiled the following examples of targeted intervention and remediation best practices with where appropriate corresponding student performance

More information

CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA. credo.stanford.edu

CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA. credo.stanford.edu CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA credo.stanford.edu April 2011 TABLE OF CONTENTS INTRODUCTION... 3 DISTRIBUTION OF CHARTER SCHOOL PERFORMANCE IN PENNSYLVANIA... 7 CHARTER SCHOOL IMPACT BY DELIVERY

More information

Theoretical perspectives: Eccles expectancy-value theory Julie Partridge, Robert Brustad and Megan Babkes Stellino

Theoretical perspectives: Eccles expectancy-value theory Julie Partridge, Robert Brustad and Megan Babkes Stellino Document name: Theoretical perspectives: Eccles expectancy-value theory Document date: 2013 Copyright information: Proprietary and used under licence OpenLearn Study Unit: OpenLearn url: Physical activity:

More information

The Bronx Achievement Pact

The Bronx Achievement Pact The Bronx Achievement Pact Every Student Counts Overview The Bronx Achievement Pact works to improve educational outcomes in the Bronx by closing the gaps in the high school-college pipeline. This comprehensive

More information

Development and Design: E- Portfolio on Project-based Learning with Assessment System

Development and Design: E- Portfolio on Project-based Learning with Assessment System DOI: 10.7763/IPEDR. 2013. V60. 34 Development and Design: E- Portfolio on Project-based Learning with Assessment System Punyapat Chanpet 1 + and Komkrit Chomsuwan 2 1 Learning Innovation in Technology,

More information

THE EFFECTIVENESS OF USING LEARNING MANAGEMENT SYSTEMS AND COLLABORATIVE TOOLS IN WEB-BASED TEACHING OF PROGRAMMING LANGUAGES

THE EFFECTIVENESS OF USING LEARNING MANAGEMENT SYSTEMS AND COLLABORATIVE TOOLS IN WEB-BASED TEACHING OF PROGRAMMING LANGUAGES THE EFFECTIVENESS OF USING LEARNING MANAGEMENT SYSTEMS AND COLLABORATIVE TOOLS IN WEB-BASED TEACHING OF PROGRAMMING LANGUAGES Nadire Cavus 1, Huseyin Uzunboylu 2, Dogan Ibrahim 3 1 Computer Information

More information

Your child, your schools, our future:

Your child, your schools, our future: A commitment from The Children s Plan Your child, your schools, our future: building a 21st century schools system SUMMARY Building a 21st century schools system Summary 1 Summary Chapter 1 Our ambition

More information

Pathways in Technology Early College High School (P-TECH) STEM Pathways to College & Careers

Pathways in Technology Early College High School (P-TECH) STEM Pathways to College & Careers Pathways in Technology Early College High School (P-TECH) STEM Pathways to College & Careers United States and China CTE Meeting Washington, DC August 8-9, 2012 The context for our work: Why P-TECH matters

More information

Pedagogy and Motivation in Introductory Accounting Courses

Pedagogy and Motivation in Introductory Accounting Courses Pedagogy and Motivation in Introductory Accounting Courses Abstract: Motivation is an integral part of the learning process.most studies in accounting that examine the effect of motivation in student learning

More information

THE FRAMEWORK FOR INSTRUCTIONAL COACH ENDORSEMENT GUIDELINES PENNSYLVANIA DEPARTMENT OF EDUCATION

THE FRAMEWORK FOR INSTRUCTIONAL COACH ENDORSEMENT GUIDELINES PENNSYLVANIA DEPARTMENT OF EDUCATION THE FRAMEWORK FOR INSTRUCTIONAL COACH ENDORSEMENT GUIDELINES PENNSYLVANIA DEPARTMENT OF EDUCATION TABLE OF CONTENTS Definition of an Endorsement Certificate... 3 Introduction to Instructional Coach Endorsement

More information

INFOBRIEF SRS. Instructional developers have been working for four SCHOOL MATHEMATICS AND SCIENCE PROGRAMS BENEFIT FROM INSTRUCTIONAL TECHNOLOGY

INFOBRIEF SRS. Instructional developers have been working for four SCHOOL MATHEMATICS AND SCIENCE PROGRAMS BENEFIT FROM INSTRUCTIONAL TECHNOLOGY INFOBRIEF SRS Science Resources Statistics National Science Foundation NSF 03-301 Directorate for Social, Behavioral, and Economic Sciences November 2002 SCHOOL MATHEMATICS AND SCIENCE PROGRAMS BENEFIT

More information

Abstract Title: Identifying and measuring factors related to student learning: the promise and pitfalls of teacher instructional logs

Abstract Title: Identifying and measuring factors related to student learning: the promise and pitfalls of teacher instructional logs Abstract Title: Identifying and measuring factors related to student learning: the promise and pitfalls of teacher instructional logs MSP Project Name: Assessing Teacher Learning About Science Teaching

More information

Albert Corbett. The ACT Programming Tutor (APT) Introduction. Abstract

Albert Corbett. The ACT Programming Tutor (APT) Introduction. Abstract From: AAAI Technical Report SS-00-01. Compilation copyright 2000, AAAI (www.aaai.org). All rights reserved. Cognitive Mastery Learning in the ACT Programming Tutor Albert Corbett Human-Computer Interaction

More information

P A L S. Manual for the Patterns of Adaptive Learning Scales

P A L S. Manual for the Patterns of Adaptive Learning Scales Manual for the Patterns of Adaptive Learning Scales P A L S Carol Midgley, Martin L. Maehr, Ludmila Z. Hruda, Eric Anderman, Lynley Anderman, Kimberley E. Freeman, Margaret Gheen, Avi Kaplan, Revathy Kumar,

More information

THE IMPACT OF TEACHING TWO COURSES (ELECTRONIC CURRICULUM DESIGN AND MULTIMEDIA) ON THE ACQUISITION OF ELECTRONIC CONTENT DESIGN SKILLS

THE IMPACT OF TEACHING TWO COURSES (ELECTRONIC CURRICULUM DESIGN AND MULTIMEDIA) ON THE ACQUISITION OF ELECTRONIC CONTENT DESIGN SKILLS THE IMPACT OF TEACHING TWO COURSES (ELECTRONIC CURRICULUM DESIGN AND MULTIMEDIA) ON THE ACQUISITION OF ELECTRONIC CONTENT DESIGN SKILLS Natheer K Gharaibeh *, Mohareb A Alsmadi ** Albalqa Applied University,

More information

Outcome: Compare and contrast different research methods used by psychologists including their respective advantages and disadvantages.

Outcome: Compare and contrast different research methods used by psychologists including their respective advantages and disadvantages. BS Psychology FY14 Closing the Loop: Previous FY Assessment Summary Annual Assessment Summary Ongoing Providing Department: BS Psychology Responsible Roles: In the text box below, please complete your

More information

High School Mathematics Pathways: Helping Schools and Districts Make an Informed Decision about High School Mathematics

High School Mathematics Pathways: Helping Schools and Districts Make an Informed Decision about High School Mathematics High School Mathematics Pathways: Helping Schools and Districts Make an Informed Decision about High School Mathematics Defining the Two Pathways For the purposes of planning for high school curriculum,

More information

A Standards-Based Technology Integration Path at Towson University

A Standards-Based Technology Integration Path at Towson University Wizer, D., & Ryan, P. (2004). A Standards-based technology integration path at Towson University. Contemporary Issues in Technology and Teacher Education, 4(1), 73-80. A Standards-Based Technology Integration

More information

GLOBAL-READY TEACHER COMPETENCY FRAMEWORK: STANDARDS AND INDICATORS

GLOBAL-READY TEACHER COMPETENCY FRAMEWORK: STANDARDS AND INDICATORS GLOBAL-READY TEACHER COMPETENCY FRAMEWORK: STANDARDS AND INDICATORS GLOBAL-READY TEACHER COMPETENCY FRAMEWORK All learners whether adult or child acquire the attitudes, skills and knowledge needed for

More information

Aurora University Master s Degree in Teacher Leadership Program for Life Science. A Summary Evaluation of Year Two. Prepared by Carolyn Kerkla

Aurora University Master s Degree in Teacher Leadership Program for Life Science. A Summary Evaluation of Year Two. Prepared by Carolyn Kerkla Aurora University Master s Degree in Teacher Leadership Program for Life Science A Summary Evaluation of Year Two Prepared by Carolyn Kerkla October, 2010 Introduction This report summarizes the evaluation

More information

Financial Education in schools: the experience of the Bank of Italy and the Ministry of Education

Financial Education in schools: the experience of the Bank of Italy and the Ministry of Education Financial Education in schools: the experience of the Bank of Italy and the Ministry of Education Maurizio Trifilidis Bank of Italy Background The individuals decision-making process could be improved

More information

LITERACY: READING LANGUAGE ARTS

LITERACY: READING LANGUAGE ARTS IMPORTANT NOTICE TO CANDIDATES: The assessment information in this document is aligned with NBPTS Literacy: Reading Language Arts Standards, Second Edition (for teachers of students ages 3 12). If you

More information

Evaluating the Impact of Engineering is Elementary Year 2 of Implementation in Minneapolis and Hopkins Public Schools

Evaluating the Impact of Engineering is Elementary Year 2 of Implementation in Minneapolis and Hopkins Public Schools Evaluating the Impact of Engineering is Elementary Year 2 of Implementation in Minneapolis and Hopkins Public Schools Executive Summary Cathy P. Lachapelle, Jonathan D. Hertel, Preeya Phadnis and Christine

More information

Self-Regulation, Goal Orientation, and Academic Achievement of Secondary Students in Online University Courses

Self-Regulation, Goal Orientation, and Academic Achievement of Secondary Students in Online University Courses Matuga, J. M. (2009). Self-Regulation, Goal Orientation, and Academic Achievement of Secondary Students in Online University Courses. Educational Technology & Society, 12 (3), 4 11. Self-Regulation, Goal

More information

Journal of Student Success and Retention Vol. 2, No. 1, October 2015 THE EFFECTS OF CONDITIONAL RELEASE OF COURSE MATERIALS ON STUDENT PERFORMANCE

Journal of Student Success and Retention Vol. 2, No. 1, October 2015 THE EFFECTS OF CONDITIONAL RELEASE OF COURSE MATERIALS ON STUDENT PERFORMANCE THE EFFECTS OF CONDITIONAL RELEASE OF COURSE MATERIALS ON STUDENT PERFORMANCE Lawanna Fisher Middle Tennessee State University lawanna.fisher@mtsu.edu Thomas M. Brinthaupt Middle Tennessee State University

More information

A Study in Learning Styles of Construction Management Students. Amit Bandyopadhyay, Ph.D., PE, F.ASCE State University of New York -FSC

A Study in Learning Styles of Construction Management Students. Amit Bandyopadhyay, Ph.D., PE, F.ASCE State University of New York -FSC A Study in Learning Styles of Construction Management Students Amit Bandyopadhyay, Ph.D., PE, F.ASCE State University of New York -FSC Abstract Students take in and process information in different ways.

More information

Improving the Progression, attainment and enjoyment of KS3 students in Mathematics using single sex classes in a co-educational, comprehensive school

Improving the Progression, attainment and enjoyment of KS3 students in Mathematics using single sex classes in a co-educational, comprehensive school TEFP1011 Susanne Hashmi-Lewis - Kings College Guildford Improving the Progression, attainment and enjoyment of KS3 students in Mathematics using single sex classes in a co-educational, comprehensive school

More information

Improving students help-seeking skills using metacognitive feedback in an intelligent tutoring system

Improving students help-seeking skills using metacognitive feedback in an intelligent tutoring system Learning and Instruction 21 (2011) 267e280 www.elsevier.com/locate/learninstruc Improving students help-seeking skills using metacognitive feedback in an intelligent tutoring system Ido Roll a, *, Vincent

More information

Research-Based Program Design Why Smart Tutor Works. Al Lockett, M.Ed. Educational Technology

Research-Based Program Design Why Smart Tutor Works. Al Lockett, M.Ed. Educational Technology Research-Based Program Design Why Smart Tutor Works Al Lockett, M.Ed. Educational Technology Introduction Smart Tutor TM, published by Learning Today, is an advanced learning system designed to accelerate

More information

Advanced Placement Environmental Science: Implications of Gender and Ethnicity

Advanced Placement Environmental Science: Implications of Gender and Ethnicity 1 Advanced Placement Environmental Science: Implications of Gender and Ethnicity Rebecca Penwell Brenau University INTRODUCTION One of the challenges of science attitude research is that many studies have

More information

WhaleWatch: An Intelligent Multimedia Math Tutor

WhaleWatch: An Intelligent Multimedia Math Tutor WhaleWatch: An Intelligent Multimedia Math Tutor David M. Hart, Ivon Arroyo, Joseph E. Beck and Beverly P. Woolf Center for Computer-Based Instructional Technology Department of Computer Science University

More information

Teacher Evaluation. Missouri s Educator Evaluation System

Teacher Evaluation. Missouri s Educator Evaluation System Teacher Evaluation Missouri s Educator Evaluation System Teacher Evaluation Protocol Introduction Missouri s Educator Evaluation System was created and refined by hundreds of educators across the state.

More information

Motivations, costs and results of AOL: Perceptions of accounting and economics faculty

Motivations, costs and results of AOL: Perceptions of accounting and economics faculty Motivations, costs and results of AOL: Perceptions of accounting and economics faculty ABSTRACT Mark J. Eschenfelder Robert Morris University Lois D. Bryan Robert Morris University Tanya M. Lee Valdosta

More information

Gamification in education: How gamification can encourage learning. Ryan Montville. The Ohio state University

Gamification in education: How gamification can encourage learning. Ryan Montville. The Ohio state University Running head: GAMIFICATION IN EDUCATION 1 Gamification in education: How gamification can encourage learning Ryan Montville The Ohio state University Running head: GAMIFICATION IN EDUCATION 2 Gamification

More information

Motivational Factors of Albanian Graduate Students Pursuing a Master Degree in Education

Motivational Factors of Albanian Graduate Students Pursuing a Master Degree in Education Ana Uka Beder University Motivational Factors of Albanian Graduate Students Pursuing a Master Degree in Education Abstract This study explained the reasons why Albanian students decide to pursue graduate

More information

GEORGIA STANDARDS FOR THE APPROVAL OF PROFESSIONAL EDUCATION UNITS AND EDUCATOR PREPARATION PROGRAMS

GEORGIA STANDARDS FOR THE APPROVAL OF PROFESSIONAL EDUCATION UNITS AND EDUCATOR PREPARATION PROGRAMS GEORGIA STANDARDS FOR THE APPROVAL OF PROFESSIONAL EDUCATION UNITS AND EDUCATOR PREPARATION PROGRAMS (Effective 9/01/08) Kelly Henson Executive Secretary Table of Contents Standard 1: Candidate Knowledge,

More information

NATIONAL COMPETENCY-BASED TEACHER STANDARDS (NCBTS) A PROFESSIONAL DEVELOPMENT GUIDE FOR FILIPINO TEACHERS

NATIONAL COMPETENCY-BASED TEACHER STANDARDS (NCBTS) A PROFESSIONAL DEVELOPMENT GUIDE FOR FILIPINO TEACHERS NATIONAL COMPETENCY-BASED TEACHER STANDARDS (NCBTS) A PROFESSIONAL DEVELOPMENT GUIDE FOR FILIPINO TEACHERS September 2006 2 NATIONAL COMPETENCY- BASED TEACHER STANDARDS CONTENTS General Introduction to

More information

Modeling learning patterns of students with a tutoring system using Hidden Markov Models

Modeling learning patterns of students with a tutoring system using Hidden Markov Models Book Title Book Editors IOS Press, 2003 1 Modeling learning patterns of students with a tutoring system using Hidden Markov Models Carole Beal a,1, Sinjini Mitra a and Paul R. Cohen a a USC Information

More information

Curriculum and Instruction

Curriculum and Instruction Curriculum and Instruction Core curriculum is the foundation of Tier 1 instruction and is the basis for building K-12 literacy in Arizona students. The curriculum at each level must be based upon the 2010

More information

AUSTRALIAN PROFESSIONAL STANDARDS FOR TEACHERS I L C O U N C

AUSTRALIAN PROFESSIONAL STANDARDS FOR TEACHERS I L C O U N C AUSTRALIAN PROFESSIONAL STANDARDS FOR TEACHERS QUALITY TEACHING I L C O U N C Contents Introduction 2 Organisation of the Australian Professional Standards for Teachers 4 Professional Knowledge 8 Professional

More information

Our research-based knowledge about good math. What Does Good Math Instruction Look Like?

Our research-based knowledge about good math. What Does Good Math Instruction Look Like? research report What Does Good Math Instruction Look Like? Nancy Protheroe It involves good teachers, an effective math environment, and a curriculum that is more than a mile wide and an inch deep. Our

More information

IPP Learning Outcomes Report. Faculty member completing template: Greg Kim Ju, Marya Endriga (Date: 1/17/12)

IPP Learning Outcomes Report. Faculty member completing template: Greg Kim Ju, Marya Endriga (Date: 1/17/12) Page 1 IPP Learning Outcomes Report Program: Department: Psychology MA (General) Psychology Number of students enrolled in the program in Fall, 2011: 48 (Appendix A) Faculty member completing template:

More information

Earn 57% of all the undergraduate degrees awarded. Much less likely than men to major in computer science, physical sciences, and engineering

Earn 57% of all the undergraduate degrees awarded. Much less likely than men to major in computer science, physical sciences, and engineering Mary Madden, Ph.D. College of Education and Human Development University of Maine Maine Girls Collaborative Project Earn 57% of all the undergraduate degrees awarded Much less likely than men to major

More information

The Massachusetts Tiered System of Support

The Massachusetts Tiered System of Support The Massachusetts Tiered System of Support Chapter 1: Massachusetts Tiered System of Support (MTSS) Overview Massachusetts has developed a blueprint outlining a single system of supports that is responsive

More information

A Framework for the Delivery of Personalized Adaptive Content

A Framework for the Delivery of Personalized Adaptive Content A Framework for the Delivery of Personalized Adaptive Content Colm Howlin CCKF Limited Dublin, Ireland colm.howlin@cckf-it.com Danny Lynch CCKF Limited Dublin, Ireland colm.howlin@cckf-it.com Abstract

More information

Learning Equity between Online and On-Site Mathematics Courses

Learning Equity between Online and On-Site Mathematics Courses Learning Equity between Online and On-Site Mathematics Courses Sherry J. Jones Professor Department of Business Glenville State College Glenville, WV 26351 USA sherry.jones@glenville.edu Vena M. Long Professor

More information

Developmental Education Pilot Report Southeastern Louisiana University Submitted January 2015

Developmental Education Pilot Report Southeastern Louisiana University Submitted January 2015 1 Developmental Education Pilot Report Southeastern Louisiana University Submitted January 2015 Developmental Education Mathematics Pilot Combined Developmental Math and College Algebra Conducted Fall

More information

Brain Points: A Growth Mindset Incentive Structure Boosts Persistence in an Educational Game

Brain Points: A Growth Mindset Incentive Structure Boosts Persistence in an Educational Game Brain Points: A Growth Mindset Incentive Structure Boosts Persistence in an Educational Game Eleanor O Rourke 1, Kyla Haimovitz 2, Christy Ballwebber 1, Carol S. Dweck 2, and Zoran Popović 1 1 Center for

More information

TEACHER PERFORMANCE APPRAISAL

TEACHER PERFORMANCE APPRAISAL 201 TEACHER PERFORMANCE APPRAISAL SYSTEM Observation and Evaluation Forms and Procedures for Instructional Practice Effective July 1, 201 A Comprehensive System for Professional Development and Annual

More information

Master of Science in Early Childhood Education Singapore, 2004 2005

Master of Science in Early Childhood Education Singapore, 2004 2005 Master of Science in Early Childhood Education Singapore, 2004 2005 Sponsored by Wheelock College s Center for International Education, Leadership, and Innovation and RTRC Asia in Singapore Background

More information

The Application of Statistics Education Research in My Classroom

The Application of Statistics Education Research in My Classroom The Application of Statistics Education Research in My Classroom Joy Jordan Lawrence University Key Words: Computer lab, Data analysis, Knowledge survey, Research application Abstract A collaborative,

More information

A LONGITUDINAL ANALYSIS OF STATE MATHEMATICS SCORES FOR OKLAHOMA SCHOOLS USING SAXON MATH

A LONGITUDINAL ANALYSIS OF STATE MATHEMATICS SCORES FOR OKLAHOMA SCHOOLS USING SAXON MATH A LONGITUDINAL ANALYSIS OF STATE MATHEMATICS SCORES FOR OKLAHOMA SCHOOLS USING SAXON MATH Report Number 363 March 2009 Advisory Board: Michael Beck, President Beck Evaluation & Testing Associates, Inc.

More information

Video Games and Academic Performance. Ronny Khadra. Cody Hackshaw. Leslie Mccollum. College of Coastal Georgia

Video Games and Academic Performance. Ronny Khadra. Cody Hackshaw. Leslie Mccollum. College of Coastal Georgia Video Games and Academic Performance Ronny Khadra Cody Hackshaw Leslie Mccollum College of Coastal Georgia Introduction and Background The development of video games goes back to the 1940s and 1950s. Computer

More information

Final Exam Performance. 50 OLI Accel Trad Control Trad All. Figure 1. Final exam performance of accelerated OLI-Statistics compared to traditional

Final Exam Performance. 50 OLI Accel Trad Control Trad All. Figure 1. Final exam performance of accelerated OLI-Statistics compared to traditional IN SEARCH OF THE PERFECT BLEND BETWEEN AN INSTRUCTOR AND AN ONLINE COURSE FOR TEACHING INTRODUCTORY STATISTICS Marsha Lovett, Oded Meyer and Candace Thille Carnegie Mellon University, United States of

More information

Course Description \ Bachelor of Primary Education Education Core

Course Description \ Bachelor of Primary Education Education Core Course Description \ Bachelor of Primary Education Education Core EDUC 310 :Foundations of Education in Qatar and School Reform : This course has been designed to acquaint the learners with the progress

More information

EDUCATIONAL LEADERSHIP PROGRAM Recognition. Standards:

EDUCATIONAL LEADERSHIP PROGRAM Recognition. Standards: EDUCATIONAL LEADERSHIP PROGRAM Recognition Standards: District Level For institutions undergoing NCATE Accreditation and ELCC Program Review For Advanced Programs at the Master, Specialist, or Doctoral

More information

CHAPTER FOUR. GIRLS AND THE MATH GAP

CHAPTER FOUR. GIRLS AND THE MATH GAP Ch Girls and the Math Gap CHAPTER FOUR. GIRLS AND THE MATH GAP STEM THE ACRONYM FOR SCIENCE, TECHNOLOGY, ENGINEERING, AND MATHEMATICS PROFESSIONS HAVE BEEN AND WILL CONTINUE TO BE ENGINES OF U.S. ECONOMIC

More information

A Matched Study of Washington State 10th Grade Assessment Scores of Students in Schools Using The Core-Plus Mathematics Program

A Matched Study of Washington State 10th Grade Assessment Scores of Students in Schools Using The Core-Plus Mathematics Program A ed Study of Washington State 10th Grade Assessment Scores of Students in Schools Using The Core-Plus Mathematics Program By Reggie Nelson Mathematics Department Chair Burlington-Edison School District

More information

Assessment of Online Learning Environments: Using the OCLES(20) with Graduate Level Online Classes

Assessment of Online Learning Environments: Using the OCLES(20) with Graduate Level Online Classes www.ncolr.org/jiol Volume 7, Number 3, Winter 2008 ISSN: 15414914 Assessment of Online Learning Environments: Using the OCLES(20) with Graduate Level Online Classes Thomas A. DeVaney Nan B. Adams Cynthia

More information

Science, Technology, Engineering, and Mathematics (STEM) Education

Science, Technology, Engineering, and Mathematics (STEM) Education Science, Technology, Engineering, and Mathematics (STEM) Education Dorinda J. Gallant, Ph.D. School of Educational Policy and Leadership College of Education and Human Ecology The Ohio State University

More information

RUNNING HEAD: TUTORING TO INCREASE STUDENT ACHIEVEMENT USING TUTORING TO INCREASE STUDENT ACHIEVEMENT ON END OF COURSE ASSESSMENTS. By KATHYRENE HAYES

RUNNING HEAD: TUTORING TO INCREASE STUDENT ACHIEVEMENT USING TUTORING TO INCREASE STUDENT ACHIEVEMENT ON END OF COURSE ASSESSMENTS. By KATHYRENE HAYES RUNNING HEAD: TUTORING TO INCREASE STUDENT ACHIEVEMENT Tutoring To Increase Student Achievement 1 USING TUTORING TO INCREASE STUDENT ACHIEVEMENT ON END OF COURSE ASSESSMENTS By KATHYRENE HAYES Submitted

More information

The IB Middle Years Programme. Education for a better world

The IB Middle Years Programme. Education for a better world The IB Middle Years Programme Education for a better world The Middle Years Programme: preparing students to be successful in school and to be active, lifelong learners What is an International Baccalaureate

More information

The Virginia Reading Assessment: A Case Study in Review

The Virginia Reading Assessment: A Case Study in Review The Virginia Reading Assessment: A Case Study in Review Thomas A. Elliott When you attend a conference organized around the theme of alignment, you begin to realize how complex this seemingly simple concept

More information

Characteristics for secondary Montessori education in the Netherlands. 1. Head, heart and hands

Characteristics for secondary Montessori education in the Netherlands. 1. Head, heart and hands Characteristics for secondary Montessori education in the Netherlands 1. Head, heart and hands In The Montessori education of the 21st century, working with head, heart and hands is translated into a wide

More information

Exploring Physics and Technology: A Study in Teaching Kinematics to Student-Athletes. Loraine Snead Science Department Chair

Exploring Physics and Technology: A Study in Teaching Kinematics to Student-Athletes. Loraine Snead Science Department Chair Exploring Physics and Technology: A Study in Teaching Kinematics to Student-Athletes Loraine Snead Science Department Chair Exploring Physics and Technology: A Study in Teaching Kinematics to Student-

More information

AN INTEGRATED APPROACH TO TEACHING SPREADSHEET SKILLS. Mindell Reiss Nitkin, Simmons College. Abstract

AN INTEGRATED APPROACH TO TEACHING SPREADSHEET SKILLS. Mindell Reiss Nitkin, Simmons College. Abstract AN INTEGRATED APPROACH TO TEACHING SPREADSHEET SKILLS Mindell Reiss Nitkin, Simmons College Abstract As teachers in management courses we face the question of how to insure that our students acquire the

More information

Quality Consistency Flexibility Affordability

Quality Consistency Flexibility Affordability Rock Valley College Developmental Mathematics Contact Information Kathleen Almy Rock Valley College, Rockford, IL Associate Professor of Mathematics Email: k.almy@rockvalleycollege.edu Blog: http://almydoesmath.blogspot.com

More information

You Have a Choice: The Power of Options in the Intrinsic Motivation of Kindergarten Students Elena Collett St. Mary s College of Maryland

You Have a Choice: The Power of Options in the Intrinsic Motivation of Kindergarten Students Elena Collett St. Mary s College of Maryland POWER OF OPTIONS ON INTRINSIC MOTIVATIOn You Have a Choice: The Power of Options in the Intrinsic Motivation of Kindergarten Students Elena Collett St. Mary s College of Maryland Abstract: Previous studies

More information

THE SCHOOL BOARD OF ST. LUCIE COUNTY, FLORIDA TEACHER PERFORMANCE APPRAISAL SYSTEM, 2014-2015 TABLE OF CONTENTS

THE SCHOOL BOARD OF ST. LUCIE COUNTY, FLORIDA TEACHER PERFORMANCE APPRAISAL SYSTEM, 2014-2015 TABLE OF CONTENTS , 2014-2015 TABLE OF CONTENTS Purpose and Key Components... 1 1. Core of Effective Practices... 1 2. Student Growth... 2 3. Evaluation Rating Criteria... 13 4. Teacher and Principal Involvement... 14 5.

More information

Evaluation Framework for Engineering Education Curriculum: a Review of Engineering

Evaluation Framework for Engineering Education Curriculum: a Review of Engineering Evaluation Framework for Engineering Education Curriculum: a Review of Engineering is Elementary Meagan Ross 11/6/2010 HOW TO CITE THIS ARTICLE Ross, Meagan. (2010) Evaluation Framework for Engineering

More information

Research into competency models in arts education

Research into competency models in arts education Research into competency models in arts education Paper presented at the BMBF Workshop International Perspectives of Research in Arts Education, Nov. 4 th and 5 th, 2013. Folkert Haanstra, Amsterdam School

More information

ASSESSMENT: Coaching Efficacy As Indicators Of Coach Education Program Needs

ASSESSMENT: Coaching Efficacy As Indicators Of Coach Education Program Needs March, 2003 Volume 5, Issue 1 ASSESSMENT: Coaching Efficacy As Indicators Of Coach Education Program Needs Lena Fung, Ph.D. Department of Physical Education Hong Kong Baptist University Hong Kong, SAR

More information

Early Literacy Assessment for Learning: Anecdotes from a School in Kosrae

Early Literacy Assessment for Learning: Anecdotes from a School in Kosrae 27 Early Literacy Assessment for Learning: Anecdotes from a School in Kosrae By Dr. Marylin Low, Assessment Specialist Winton Clarence, Program Specialist Keti William, Reading Specialist Pacific educators

More information

Testing the Revised Moving Target Tutor

Testing the Revised Moving Target Tutor COLLEGE OF INFORMATION SCIENCES AND TECHNOLOGY THE PENNSYLVANIA STATE UNIVERSITY Testing the Revised Moving Target Tutor (evaluation of D2P/MTT 11/27/2012) Kuo-Chuan (Martin) Yeh, Frank E. Ritter, and

More information