Intel Teach Elements Impact Study. September, 2012

Transcription

1 Intel Teach Elements Impact Study September, 2012

2 Introduction and Summary of Findings This report describes the results of a study conducted by SRI International to evaluate the impact of the Intel Teach Elements course on participating teachers and other education professionals. The report presents findings based on data collected from 132 survey participants and 17 interviewees in three different U.S. states during June and July months after they had taken an Intel Teach Elements course. Participants in the study responded to questions regarding the impacts they perceive the course had on their teaching knowledge and skills. The participants in this study provided very strong, positive feedback regarding the learning experience provided by the 5 Elements course(s), Assessment in 21 st Century Classrooms, Collaboration in the Digital Classroom, Project-Based Approaches, Thinking Critically with Data, and Educational Leadership in the 21 st Century. Through the survey, course participants revealed that they felt they learned from the course, built increased confidence, and made changes in their teaching practices. These impacts were substantive, no matter how many courses respondents had taken, or which course was taken. Impacts among teachers did vary, however, based upon the grade level participants teach, with middle teachers reporting greater impacts, while teachers of high school reporting less impacts. The interviews were used to explore and expand these survey trends, including gathering the motivations participants had for taking Elements courses, comparing Elements to other professional development offerings, and gathering of stories of impact on teacher understandings, confidence and practice of concepts presented in the Elements courses. While there were some minor negative comments revealing areas for future improvement, both survey and interview respondents provided very positive reviews of the Intel Teach Elements courses. This report will provide a summary of the participants who completed the online survey in Part I, and the statistical analysis of the survey data in Part II. Part III will then summarize the data gathered through telephone interviews. Finally, this report will conclude with a summary of the overall findings and insights for Intel based on these findings. 2

3 Part I: Survey Participants The 132 respondents to the SRI s online survey (see Appendix A) were drawn from teachers and other school staff who had taken Intel Teach Elements courses offered through ITAs in Alabama, Arizona and Virginia. Participants were evenly distributed across the three states (see Table 1). Table 1. Number of survey participants State N Percent Alabama 47 36% Arizona 41 32% Virginia 44 33% Overall % Survey respondents were asked to report which Elements course(s) they had taken. Because they could identify all of the Elements courses they completed, the number of courses taken by participants totals to more than 100% (see Table 2). Thirty percent (n=39) of respondents reported having taken multiple courses in the Elements series, while 70% took one course. Table 2. Elements courses taken by survey participants Alabama Arizona Virginia Assessment in 21st Century Classrooms 49% 91% 58% Collaboration in the Digital Classroom 25% 81% 48% Project-Based Approaches 61% 79% 63% Thinking Critically with Data 3%* 73% 0%* Educational Leadership in the 21st Century 0%* 68% 72% *These courses were not offered. Although not all ITAs offer all the Elements courses, they each offer courses several times a year, sometimes several courses at a time. Participants can take more than one course at a time, or take them in sequence. Participants in this study typically had years of teaching experience (44% of respondents overall). Very few were new to teaching, with less than 3 years experience (4% overall) (see Table 3). 3

4 Table 3. Years of teaching experience Years teaching experience Overall Alabama Arizona Virginia Less than 3 4% 4% 2% 5% 3 to 9 27% 23% 29% 30% 10 to 20 44% 53% 37% 41% Over 20 25% 19% 32% 25% Total 100% 100% 100% 100% Most commonly, participants in the school year reported teaching elementary or general education (24% overall) (see Table 4). Among those teaching specific subjects, English/Language Arts, Science, Math, and Special Education were most often mentioned (11-13% overall). A noteworthy portion of Elements participants reported that they do not work directly with students (15% overall). Table 4. Elements courses taken by survey participants Courses Taught Overall Alabama Arizona Virginia All/Elementary Education/General Education 24% 19% 34% 20% Art/Music 4% 2% 2% 7% English/Language Arts 12% 19% 12% 5% Computer Science/Tech Education 7% 9% 10% 2% Foreign Language 3% 2% 5% 2% Gifted Education 2% 2% 5% 0% Library/Media 4% 0% 5% 7% Math 13% 17% 7% 14% Physical Education/Health 2% 2% 0% 2% Science 11% 13% 10% 9% Social studies/history/geography 8% 13% 7% 5% Special Education 11% 19% 7% 5% Vocation/Technical Training 5% 11% 2% 2% Other 8% 0% 12% 14% Do not work directly with students 15% 9% 17% 20% * These columns total to more than 100% because participants were allowed to check all items that apply. Some reported teaching more than one of the disciplines listed. 4

5 As shown in table 5, Elements participants most often described their professional role as classroom teacher, ranging from 48-83% of respondents in the three states represented. Table 5. Professional roles of Elements participants Professional role Overall Alabama Arizona Virginia Classroom teacher 66% 83% 68% 48% Enrichment or resource teacher (such as Title I, gifted ed., reading specialist) 8% 13% 10% 2% Technology coordinator, media specialist or librarian 6% 0% 5% 14% Other professional staff (such as staff developer, instructional coach, etc. 7% 0% 8% 14% Administrator 6% 2% 3% 14% Other 6% 2% 8% 9% Total 100% 100% 100% 100% *N s vary from for states, and N=131 overall. Among those that directly teach students, participants overall were evenly spread among those that teach lower elementary, middle elementary, middle or junior high school, and high school grades (see Table 6). Table 6. Professional roles of Elements participants Grade Level Overall Alabama Arizona Virginia Lower Elementary K-3 23% 17% 20% 32% Middle Elementary % 21% 22% 25% Middle/Junior High School % 36% 29% 14% High School % 40% 32% 36% Post-secondary Pre-service Teachers 1% 2% 0% 0% Do not work directly with students 11% 2% 15% 16% *These columns total to more than 100% because participants were allowed to check all items that apply. Some reported teaching more than one of the grade ranges listed. While we were not able to randomly sample participants in this survey to statistically represent all those who take Intel Teach Elements courses, this profile of the Elements survey respondents provides a general picture of the diversity of those taking the Elements courses. The courses seem to be reaching education professionals both in and outside the classroom. The classroom teachers who take the course reflect a broad range across grade levels and 5

6 subjects. In our data, we see indication that those with more years of teaching experience are more likely to take the course than their less experienced peers. Part II: Course Impacts After consulting with Intel staff and reviewing the content of the 5 current Elements courses (Assessment in the 21 st Century, Collaboration in the Digital Classroom, Project-Based Approaches, Thinking Critically with Data, and Educational Leadership in the 21 st Century), SRI identified 6 areas of impact common to the Elements courses. These areas are: Impact area 1: Using project-based activities or strategies with students Impact area 2: Preparing and planning for student-centered instruction Impact area 3: Engaging students in self directed learning and collaboration Impact area 4: Using technology to support learning Impact area 5: Assessing student critical thinking and 21 st century skills Impact area 6: Participating in a professional network and supporting colleagues To address different types of impacts, participants were asked about the effects of their course experiences on their understanding, confidence and teaching practice in each of the 6 impact areas. Each impact area was covered in 1 to 8 items designed to capture the multiple dimensions and relevant actions associated with participant understanding, confidence, and teaching practice. Understanding When asked on the online survey about changes in their understanding for each of the impact areas, the majority of participants across the three states felt that their understanding had been affected to a large or great extent (see Table 7). In all areas, at least half of the participants felt that their understanding of the impact areas was deepened, and less than 5% felt they had learned nothing new. The impact area in which participants report the greatest change in their understanding was the assessment of student thinking and skills, with 69% agreeing that their understanding had been deepened to a large or great extent. 6

7 Table 7. To what extent did your most recent Elements course deepen your understanding of:* Not at all To a small or moderate extent To a large or great extent Impact area 1: Use of project-based activities or strategies Using project-based instructional strategies in your classroom 4.55% 35.61% 59.85% Impact area 2: Preparing and planning for student-centered instruction Preparing and planning for student-centered instructional 1.50% 34.59% 63.91% strategies Impact area 3: Engaging students in self-directed learning and collaboration Using methods of teaching that emphasize student self-directed 1.53% 38.93% 59.54% learning Using methods of teaching that emphasize collaborative student 1.52% 37.12% 61.36% learning Impact area 4: Using technology to support learning Supporting students as they use technology for learning 1.50% 35.34% 63.16% Impact area 5: Assessing student critical thinking and 21 st century skills Assessing student critical thinking, collaboration, technology use, and/or other 21 st 0.76% 30.30% 68.94% century knowledge and skills Impact area 6: Participating in a professional network and supporting colleagues Participating in a professional learning community 2.26% 36.09% 61.65% Supporting colleagues in using student-centered instructional 2.27% 40.91% 56.82% strategies * This question was asked on a 5-point scale. The small or moderate extent responses were combined for presentation in this chart, and the large or great extent responses were also combined. The data regarding changes in understanding come from all respondents, no matter which course(s) they took. Interestingly, although some impact areas associate more strongly with one Elements course than others (e.g., assessing students being a distinct focus of the Assessment course), these overall results show that participants in all 5 courses felt they had deepened their understanding in all 6 impact areas. 7

8 Confidence Understanding a teaching practice or approach can be quite different from feeling ready to use it one s own classroom. Therefore, participants were asked about changes in their confidence and feelings of effectiveness regarding the 6 Elements impact areas. Similar to results regarding participants understanding, the majority of the participants felt that their confidence had been positively impacted to a large or great extent (see Table 8). In all areas, roughly 60% of participants felt greater confidence in the 6 impact areas, while fewer than 4% felt their confidence had not been changed. As with changes in understanding, participants reported their greatest growth in feelings of effectiveness regarding the assessment of student thinking and skills. Participants across all Elements courses showed similar levels of confidence in each impact area. Table 8. To what extent did your most recent Elements help you feel more effective in:* Not at all To a small or moderate extent To a large or great extent Impact area 1: Use of project-based activities or strategies Using project-based instructional strategies in your classroom 3.82% 34.35% 61.83% Impact area 2: Preparing and planning for student-centered instruction Preparing and planning for student-centered instructional 2.27% 35.61% 62.12% strategies Impact area 3: Engaging students in self-directed learning and collaboration Using methods of teaching that emphasize student self-directed 1.55% 34.11% 64.34% learning Using methods of teaching that emphasize collaborative student 1.54% 37.69% 60.77% learning Impact area 4: Using technology to support learning Supporting students as they use technology for learning 2.27% 36.36% 61.36% Impact area 5: Assessing student critical thinking and 21 st century skills Assessing student critical thinking, collaboration, technology use, and/or other 21 st 0.77% 31.54% 67.69% century knowledge and skills Impact area 6: participating in a professional network and supporting colleagues Participating in a professional learning community 3.10% 34.88% 62.02% Supporting colleagues in using student-centered instructional strategies 3.85% 33.85% 62.31% 8

9 * This question was asked on a 5-point scale. The small or moderate extent responses were combined for presentation in this chart, and the large or great extent responses were also combined. Teaching Practice Finally, participants might understand a practice and feel confident about doing it, but they may or may not put it to use in their daily work. A third set of impact questions, then, asked participants about changes in their classroom practices as a result of completing their Elements course(s). As shown in Table 9, a relatively large number of survey respondents (roughly 10-20%) reported that these practice items are not applicable to them. These are likely the enrichment or resource teachers, technology coordinators, administrators, and other education professionals who took the Elements courses but do not teach in their own classrooms. Among survey participants who reported on classroom practices, 56% report using assessment strategies more often to capture student critical thinking and 21 st century skills, compared to their classroom in the year before they had taken the Elements course(s). In addition, 55% of respondents use the strategies of preparing and planning for student-centered instruction more often than before, 48% report their students are more often engaged in self-directed learning and collaboration, 44% participate more often in professional networks and support their colleagues, 43% have students more often use project-based activities, and 42% report their students more often use technology in their learning. Our data show that one-third to one-half of those who participate in an Elements course report their classroom practices to be different the subsequent year. Notably, the relative order of the types of changes made reflect those practices teachers can change on their own without administrative or technical support. Ongoing assessment in the classroom and planning practices are relatively easy for teachers to effect independent of any support, whereas integrating technology or changing their instructional calendar to accommodate projects are typically not changes they can make on their own. Table 9. Do students in your classes / do you do the following activities more often, about as often, or less often than students in your classes last year? Impact area 1: Students use of project-based activities or strategies Conduct investigations in which they construct a product or new knowledge More often About the same Less often N/A 48.87% 26.32% 2.26% 22.56% Conduct investigations in which they address a large, driving question 36.84% 36.09% 3.01% 24.06% Complete projects that last more than one week 43.61% 30.83% 4.51% 21.05% Impact area 2: Teachers preparing and planning for student-centered instruction Build guiding questions into tasks to help students focus on important ideas 56.06% 30.30% 0.76% 12.88% Design lessons and activities starting with what students should know and 56.39% 30.83% 1.50% 11.28% 9

10 be able to do at the end of activity Connect lessons and activities to state standards or common core 55.30% 31.82% 0.76% 12.12% Plan into lessons and activities a variety of teaching strategies to engage all learners 59.54% 28.24% 2.29% 9.92% Use online productivity tools (e.g., online calendars, lesson plan builders) to prepare lessons 50.00% 35.61% 3.03% 11.36% Impact area 3: Students engagement in self-directed learning and collaboration Communicate with others during their work (e.g., with class peers, school peers, or community members) 56.06% 22.73% 1.52% 19.70% Revise and reflect on their work 51.52% 26.52% 3.03% 18.94% Present their final work publicly (e.g., with class peers, school peers, or community members) 44.70% 32.58% 3.03% 19.70% Work independently on significant projects or activities (not only homework or worksheets) 37.88% 34.09% 6.06% 21.97% Work in teams on significant projects or activities (not only homework or worksheets) 48.87% 29.32% 1.50% 20.30% Make substantive choices regarding their learning opportunities 36.84% 39.85% 2.26% 21.05% Create original work products that represent their knowledge on a topic or question 54.14% 24.06% 2.26% 19.55% Use multiple resources (not only textbooks) to explore questions and build understanding 55.30% 23.48% 0.76% 20.45% Impact area 4: Students use of technology to support learning Use online tools to create written work products 46.97% 29.55% 0.00% 23.48% Create media work products (e.g., audio or video) 40.60% 33.08% 2.26% 24.06% Use online tools to collect, organize and share research resources 46.97% 29.55% 0.00% 23.48% Use online tools to communicate with others outside the classroom (e.g., blogging or video conferencing) 33.08% 36.09% 1.50% 29.32% Impact area 5: Teachers assessment of student critical thinking and 21 st century skills Share assessment criteria with students when starting a new lesson or activity 54.20% 27.48% 2.29% 16.03% Use multiple modes of assessment within an extended lesson or activity (e.g., checklists, rubrics) 57.58% 29.55% 0.76% 12.12% Assess student knowledge beyond information retrieval (I.e., ability to apply knowledge to problems, draw conclusions, analyze relationships) 59.40% 27.07% 0.75% 12.78% Integrate formative assessment into instruction and activities 57.14% 30.08% 0.75% 12.03% Include peer feedback and student self reflection into student assessment 51.52% 30.30% 1.52% 16.67% Impact area 6: Teachers participation in a professional network and supporting colleagues Join and visit online networks to gather ideas from other teaching professionals 51.13% 34.59% 3.01% 11.28% Post comments or resources on online networks, videoconferences, or through other technology 36.64% 36.64% 4.58% 22.14% Share ways of improving student-centered instructional strategies with grade level or departmental colleagues at your school (online or 55.64% 32.33% 3.76% 8.27% otherwise) Share ways of improving student-centered instructional strategies with colleagues within your district or state (online or otherwise) 42.11% 43.61% 5.26% 9.02% 10

11 Share ways of improving student-centered instructional strategies with other professionals across the country or internationally (online or otherwise) 38.35% 42.11% 5.26% 14.29% As an overarching question, survey participants were asked if they felt their experience with the Elements course(s) positively impacted their teaching strategies and practices (see Table 10). Forty-four percent of participants, averaged across the three states, felt that the courses had a positive impact on them to a large or great extent, while only 8% report the course influenced them not at all or to a small extent. Table 10. Overall, to what extent do you feel the Intel Teach Essentials course(s) positively impacted your teaching strategies and practices? Overall Alabama Arizona Virginia Not at all 1% 0.00% 0.00% 2.27% To a small extent 7% 6.38% 7.32% 6.82% To a moderate extent 48% 25.53% 9.76% 27.27% To a large extent 23% 46.81% 48.78% 47.73% To a great extent 21% 21.28% 34.15% 15.91% SRI employed regression analyses to explore any patterns regarding the ways in which participants found the courses most impactful in relation to their understanding, confidence and teaching practices. To do so, composite scales were created, merging the understanding, confidence and practice impacts for each survey respondent (see Table 11). These items were combined and reported on a scale from 1 to 10, where 10 is the highest impact and 1 is no impact. As shown in the table below, the means for these composite impact scales were all about 7 on the scale, with no impact area showing considerably larger or smaller impacts than others. 11

12 Table 11. Overall composite impact scales Scale Mean Alpha Number of Items Impact area 1: Use of project-based activities or strategies Impact area 2: Teachers preparing and planning for student-centered instruction Impact area 3: Students engagement in self-directed learning and collaboration Impact area 4: Students use of technology to support learning Impact area 5: Teachers assessment of student critical thinking and 21 st century skills Impact area 6: Teachers participation in a professional network and supporting colleagues Regression analyses also were used to compare these composite scores across groups of participants. Overall impacts were not different for participants in relation to which Elements course they took (or most recently took if they have had more than one Elements course), there were no differences in impact based on the number of Elements courses participants had taken, nor were there any differences in impact reported by participants based on their years of experience. The only overall impact distinction (shown in Table 12) was based on the grade level in which participants taught those who taught middle or junior high school reported significantly greater project use (area 1), preparation (area 2), collaboration (area 3), technology (area 4) and assessment (area 5) impacts that those who taught other grades. In addition, those who taught at the high school level reported significantly lower impacts regarding assessment (area 4) than those who taught at other grades. Table 12. Overall composite impact scales for Understanding, Confidence and Practice Area 1: Projects Area 2: Preparation Area 3: Collaboration Area 4: Technology Area 5: Assessment Area 6: Networks Grade Level n Means Teach Middle/Junior High School ** 8.43** 8.05** 8.44** 8.76** 7.55 All those who DO NOT Teach Middle/Junior High School 6-8 Teach High School ** 6.81 All those who DO NOT Teach High School *N = 132, All scales are 1-10, ** p <

13 Finally, the behaviors and practices of Elements participants were examined separately from the understanding and confidence items. Composite scores across the multiple behavior items in each impact area were again created, using a scale from 1 to 10. Regressions indicated very similar results, in that there were no significant differences in impacts on practice based on which Elements course participants had most recently taken, the number of Elements courses participants had taken, or participants years of experience. There were differences, however, among those who teach middle or junior high school and those who teach high school grades (see Table 13). Those who taught middle or junior high school reported significantly greater project (area 1), preparation (area 2), collaboration (area 3), and assessment (area 5) impacts than those who taught other grade levels. In addition, those who taught high school reported significantly lower impacts regarding projects (area 1) and assessment (area 4) than those who taught other grades. Table 13. Overall composite impact scales for Practice Area 1: Projects Area 2: Preparation Area 3: Collaboration Area 4: Technology Area 5: Assessment Area 6: Networks Grade Level n Means Teach Middle/Junior High School * 6.25* 8.43* * 8.04 All those who DO NOT Teach Middle/Junior High School 6-8 Teach High School * * 7.04 All those who DO NOT Teach High School *N = 132, All scales are 1-10, ** p <0.001 In sum, our survey research shows that roughly 60% of participants in Elements courses felt that their understanding in 6 key areas included in the Elements courses had been deepened, and that roughly 60% of participants felt more effective and confidence in these areas. Regarding classroom practices, 42-56% reported using strategies covered in the Elements courses more frequently than they had before taking the course. The impacts on participants were not significantly different based on the number of courses participants took, or which course participants took. The one factor that differentiated participant outcomes was the grade level they teach, with middle or junior high school teachers reporting significantly larger gains in 5 of the combined composite impact areas, and larger gains in 4 of the composite areas regarding classroom practices. Those who teach high school had lower outcomes in several impact areas (both overall and regarding classroom practices) than those teaching at other grade levels. 13

14 Part II: Participant Interviews To deepen our understanding of the survey data reported above, SRI conducted follow-up telephone interviews with 17 of the 132 respondents who complete our survey (see Table 14). Table 14. Number of interview participants State N Percent Alabama 6 35% Arizona 5 30% Virginia 6 35% Overall % The 17 interviewees represented a range of professional roles (e.g., teachers, school leaders, grade level leader, tech resource support specialist). Teachers reflected a range of content areas (e.g., business, German, English Language Arts) and grade levels (e.g., preschool, elementary, middle school, and high school). The interviews lasted about 30 minutes, and were guided by a semi-structured interview protocol (see Appendix B). These interviews covered topics regarding the reasons participants chose to take the Elements courses, the impact the courses had on their professional practices, the course resources they used, and how the Elements course compared to participants other professional development experiences. In addition, a number of interviewees provided stories or examples of their use of tools and concepts they gained from the courses. Reasons for taking the courses One of the main reasons interviewees took the Elements course(s) was the relevance and focus of the course topics. As one person put it, the courses specifically addressed what I was looking for. Another person said she took the Elements course because I was looking for training that specifically addressed [assessment]. Having gone through two Master s programs, there wasn t really coursework that addressed those specific things. I was looking for specifics. What does assessment need to look like in a classroom? And how should we be assessing? She felt that the Elements course had indeed provided the specifics she was seeking. Another interviewee described this focus, saying the thing I do like about these Intel classes is we re going to do exactly what you say in the description, and that s important to me. I don t have hours and hours and hours to waste. The classes require a lot of time, and I put the time in, but I get so much out of them, because you are truly teaching what you said you were going to teach. When asked if they were actively seeking out trainings, most interviewees said that they were typically already interested in a topic area (such as project-based learning), and when they saw the Intel course listing, they chose to act on the opportunity for professional development in that area. As one interviewee described, I think when I saw the opportunity and I saw the list of classes to take, then I was like wow, you know, I could really probably use something like 14

15 that. Interviewees were not seeking out trainings and finding the Intel offerings; instead, news of relevant Intel trainings reached them. One of the main efforts of ITAs in offering Elements courses is outreach to potential participants. The pattern described by these participants suggests that this is a critical activity, while the build it and they will come would be less successful. To those already familiar with Intel, the quality reputation is another reason to participate in Elements courses. When one teacher, who had taken past Intel trainings (including Essentials) was asked why she would take the Elements course, she said she knew Intel would help her to stay current. I know if you go through Intel you re going to really stay current. And the resources like the online resources, nobody has the time or I don t even know how to get to all those resources that just come through in the course. About the third week of every course the resource part was just phenomenal. Phenomenal enough to take it back and share with other teachers. Another interviewee commented, Intel knows what they re doing, and they know where they want you to get from the very first minute you start into a course. Once you start in the course, you can feel very comfortable that you are on your way, and you are going to get there, and you re going to build it piece by piece. When you get there you re going to have a really good idea of what you should. Other courses don t have that all the time. To others new to Intel, the quality experience they had with one Elements course was a motivation to take more Elements courses. Participants felt that the courses were well worth their time (more on this below), and that Elements met their expectations. One interviewee said, I continued to take the Intel classes because they were exactly what they said they were going to be. If they said they were addressing critical thinking skills, that s what the course addressed. That the class was on assessment, then assessment was what we were focused on. A third reason interviewees said they were motivated to take Elements course(s) was the very practical approach of the courses. Interviewees said it was important that the courses were not generic, or just theory, but an active learning experience. A lot of the things that I had gone to previously were very theory-based. That s great, but I ve been in education a long time, I know the theory. I want the hands-on, let me make the unit, let me do it. Another interviewee said, I need to do it. I need to create the unit. And that s what I loved about the classes. For an online class, they were very hands-on. And that there was a lot of collaboration, back and forth in the forums, there was a lot of, okay, we re going to show you this, now you go create it. Finally, two frequently mentioned themes regarding why participants took the Elements course(s) were the convenience of the course format and the lack of cost. The online format provided flexibility to education professionals with very busy lives. As one person put it, I really do like that I can get up at 3:00 a.m. and work on it on my own time, and if I travel I can take a laptop with me and keep up. Just as important, however, was that the courses were free to participants and their schools or districts. I have five children, so it was like, okay, I can only afford to spend so much, one interviewee explained. One school leader saw the larger picture involved in training costs: there are so many opportunities, and with budgets shrinking, when you can do an online course as opposed to traveling to a conference where you miss a 15

16 day of instruction in the classroom. Sending a teacher to the conference she s going to miss a day of instruction in the classroom. I m paying for mileage. I m paying for meals. I m paying for a hotel. I m paying for registration. So by the ability to do online training and online classes, it really brings those costs down as well as a better use of staff hours. Both of the school leaders interviewed were considering asking some of their teachers to participate in Elements courses, and while the quality of the experience with the Elements courses was critical, the expense was another important factor that made this plan possible. Course experience and impacts The participants in the Elements interviewees provided very positive feedback about the course(s) in which they participated. When asked about any impacts from the Elements courses, all interviewees felt they gained new understandings. Nearly all were able to describe changes in their leadership plans, teaching practices, or use of resources in the Elements courses. Positive feedback also was provided during the interviews when participants discussed the Elements online course design, and when they compared Elements to other professional development experiences. Understandings One type of impact most all Interviewees described was gaining new understandings of the concepts presented in the Elements courses in which they participated. For example, a participant in the Project-based Approaches described trying to dabble with projects before the course, but I felt that the Elements gave me a better understanding of what I was doing. I was always looking at more of it as problem based. Then I realized in this course that project based and problem based are two totally different things. Descriptions regarding changes in understanding were given by many of the interviewees participants often felt that they learned quite a lot, including those who have taken other Intel or Elements courses, and those with a great deal of teaching experience. Interviewees described their newly gained understandings from a new view of teaching to getting good ideas. Some examples of this range included: I think it made me kind of look at a different way of teaching, to approach things in a different manner that would be more beneficial for my kids. It was not about having an activity on Monday morning, it was much more a philosophy thing. The Assessment course made the point that students must be taught how to do peer assessment. Teachers will get frustrated with that. I m like, Well you never taught them how to do it. That has opened my eyes through these courses. Intel gave you so many good ideas, about how to integrate technology when there is only one computer available to in the classroom. Confidence These new practices coincided a new level of confidence for some participants. Though not discussed directly by interviewees, the changes in understandings also coincided with the 16

17 confidence to put practices to use in their classrooms. For sample, one interviewee described how changes in her understanding, along with tools provided in the course, also provided confidence to change her teaching practice. She said, since I started teaching so long ago, I come from this traditional background of I m the boss, and we follow a rigorous schedule, and we learn a lot. And I had to practice and try, and find out it really worked as a collaboration aspect in groups. I was always afraid to do that, but Intel gives you the tools to manage it. They re really, really good tools to manage it. You watch them and you have checklists. They (students) have rubrics, and they know what s expected. The collaboration of that was a change for me in my kind of classroom. Practice Finally, the interviews explored changes in professional practice and resources provided in their teaching and leadership activities. For example, one interviewee said the practices from her Elements course have become part of my curriculum. In fact I keep a strategy list in my lesson plan book. Just because you need that constant reminder of, okay, oh wait a minute, I m not really pushing their thinking right now. How can I ratchet up this lesson? Another example was provided by a high school business teacher: We did some digital portfolios in class. That s the first time I ve done this. And I probably should have done it before now. It was a group task group that I changed based on the course I took. In addition to changes in learning activities, interviewees often described saving and returning to the resources they received in their Elements course(s), which made a difference in their teaching practice. One interviewee said her Elements course provided checklists and rubrics and things that yes, if I had all the time in the world, could I create those? I probably could, I probably could come up with them. But having them there, that I could look at and go, oh wait, I could change this and this and that would be perfect for my class. The collaboration rubric, the critical thinking rubric, the problem-solving one, there s several different checklists that I use with my kids, that my kids go through and check off how they feel. I love the fact that they re worded in a very student-friendly way. My kids use the checklists all the time. Another interviewee described how incorporating the assessment resources has served two purposes, one was that the kids were able to learn that assessment, and thereby make themselves better at what they were doing. The second thing is the way that the Intel class works is that the peer assessment and the instruction on how to do that actually helps me as a teacher know that the child that s assessing understands the concepts. I had never done that before. It was more like teamwork. Did all your team members pull their weight? What grade would you give them? I did that in several classes where I changed that so that the assessments were designed where the kids had to know the concepts or they wouldn t be able to assess their peers, and I could tell that by their assessment what they wrote down. While most interviewees described saving resources and using new tools and practices in their classrooms, their descriptions also suggested a few points of caution. Some struggled with implementation ( I did try to let the students do [assessment] where they decided what would be assessed and what the criteria would be. That went okay. I probably need to work at that 17

18 more ), as well as time to incorporate new changes ( The biggest thing is time. That s the biggest challenge ). In addition, a few participants reported trying out new practices and tools, yet not maintaining them regularly ( I created the rubric and I tried to use it in my class and then it was one of the things that kind of fell by the wayside once you get out of the year it kind of fell through ). Comparisons to other Professional Development Interviewees uniformly praised the Elements course(s) they took. When asked to compare their Elements experience to other professional development sessions in which they have participated, the interviewees felt Elements compared favorably. Reasons for its positive review related to both the design and implementation of the course, including comments such as: On a scale of 1 to 10, 1 being low and 10 being high, probably for me it was about an 8, because it was self-regulated, I didn t have to sit after school. It was interesting to get different perspectives on things [from other participants]. This interviewee did not give the course a 10 rating because she felt it was not as applicable to her primary grade students. I ve attended conferences where the speaker has been dynamic and I ve just been enthralled by every word, but I can t listen and write at the same time. So I m either listening to the speaker or I m taking notes. Then my notes don t make any sense. So, as far as being able to organize the learning experience, this [online course] was very positive. In comparison to other PD, I enjoyed it more. I ve taken college courses, I ve done Saturday courses, I ve done a Master s Degree online, and I enjoyed all of those, but this one [had a good] time span, the periods that it covered it was six weeks, so it was short. When you started it wasn t like, oh my God, I m going to have to go through 12 weeks of this, or I m going to be doing this for a whole semester. It fit into my schedule very well. I felt like [Elements] was real high quality. Our facilitator was very knowledgeable with what was out there and that she was very good about moving us around in the program and moving us through it. I think it has definitely more impact because other online courses that we can take for recertification points and it s a book we have to read and then answer questions online. And some were very disorganized. Intel was definitely more professional, because everything was on the ball. There were times for like some of my other online ones I was still waiting for the teacher to get up the dates that we needed, or to get up the assignments that we needed to get done, and I didn t have to worry about that at all with Intel, it was very nice, you just have to click on, do what I needed to do and everything was already there. With the exception of the interviewee who felt the Elements course did not clearly apply to the very young grade level she teaches, the comments comparing Elements to other professional development activities were quite positive. That the courses were online and lasted 6 weeks gave participants access to a learning experience that they could fit into their semester 18

19 schedules. In addition, the quality of the facilitation and the clarity of the online experience (including the Moodle structure) also helped elevate Elements above other trainings. Notably, EDC s ETLO program has provided the training for facilitators to lead Elements courses online and created the online Moodle structure for the Elements courses which helped Elements stand out from other professional development experiences. Platform design & experience Finally, the feedback interviewees provided regarding the Elements course design and online experience was also quite positive. Much like the reasons interviewees provided about their ranking Elements above other professional development experiences, the Elements design and facilitation elicited very positive feedback for the courses in general. Both the facilitation and the online course design were cited as the core reasons for the positive experience. One interviewee s comment relates both of these points in a single comment saying, My initial thought of it was, what have I done to myself? But the course leader was very good, and very encouraging. I think that was one of the big things for me. Had I felt like I was so out there on an island feeling rather stupid, I might not have enjoyed the course. A little bit of stress but not much, because it was kind of new for me to get online and to go through the program. To read the questions, answer the questions, go back and look at the material again. I did like that. I liked being able to go back and review the material like if I felt I hadn t really got it, to go back to that lesson. In addition to the central points of facilitation and online access, interviewees reported having no trouble with online navigation (for example, I think the setup is wonderful, it's easy to use, easy to navigate, easy to understand ). They also valued the step-by-step design of the course building toward a clear end product, coupled with a very clear calendar of due dates (for example, I loved the fact that there were specific timeline due dates. The class was broken down. A lot of the online classes you take, it s kind of like, okay, listen to this lecture and eventually you re going to have this test ). Roughly half of the interviewees felt that the other participants in the course were an important addition to their learning experience (for example, even the people that were online too with me, I tried to touch base with them and it definitely helped me a lot. So I felt like I didn t feel like I was alone or by myself or anything, and then if by chance I had a question I could always ask a fellow teacher ) though about half did not report their fellow participants as a major factor in their course experience. Challenges regarding the courses While most comments made by the interviewees were very positive, some mentioned issues they had with the course as well. One person said that he did not use the course resources from the Leadership course because he found them to be a little bit dated. Instead he used the ideas presented to go to the web and find some things that look like that that worked for me a little bit better. He acknowledged that because the resources were regarding technology use, it is easy to be quickly out of date. He also mentioned that his school model is quite innovative, while the resources were set up for the traditional district education model, so he was not did not find them to be as useful as they could be. One course participant found the 19

20 video conversations among the teachers portrayed in the course to be annoying though another interviewee felt that the videos, activities and quizzes were very helpful and engaging. Finally, the two interviewees who taught at the preschool and first grade levels felt that the content of the course was not as useful for them as it was for others. One felt that the course was geared toward an upper elementary, middle school, high school teacher, but went on to say that she was able to pull a lot from it, a lot of ideas that I can adapt and use in the primary grades. The other felt that the ideas and practices introduced in her course were not appropriate for her students maturity or ability levels, but there were a few things she could use and felt the Elements experience was valuable. Finally, several interviewees mentioned that while they appreciated the activities in the course(s), the amount of work was challenging, particularly when encapsulated in a book that you have to put together, as one person referred to the course final project. Interviewees persisted with the effort, however, because while it was a lot of work, getting into the essential questions and all that, it gave you a lot of resources for that and a lot of guidance. Not that I would use everything from it, but there were some pieces of it that are going to be quite helpful. Stories of impact Finally, to more fully portray the impacts and experiences that the 17 interviewees described, two stories are included below. While these stories are unique to the two individuals who provided them, they reflect many of themes summarized above participants with years of experience, finding the Elements courses as a way to extend an existing interest, and leveraging the very practical, focused design of the courses to change an aspect of their work. Setting School Standards Marie 1 was a very experienced school leader when she participated in the Elements Leadership course. The instruction in her school is entirely online, using a variety of technology including laptops, ipads, Kindles, and Nooks. Marie was already focused on teaching and learning in the 21 st century and this course became available through her state department of education. She decided to take the course her first online course because she felt that in her work of observing teachers and students, she needed to better understand what is going to be happening with technology and instruction in the 21 st Century, both for the students and also for staff development. In addition, her school is moving toward offering more online professional development and supports for staff, so she wanted to have more online learning experience as well. When asked to reflect on any possible impact the Elements Leadership course had for her, her reply came quickly. Because it was a course activity, Marie looked at her state technology standards, as well as some international technology standards. She said, when I went to that [international] resource I said, Oh! And then I pulled our state standards and I said, Oh. This contrast led Marie to the conclusion that the technology standards for her state were not sufficient for her school s needs, saying they were so minimal that if you came with only those competencies you could not teach in my environment. 1 Both of the interviewees names have been changed. 20

21 As Marie told it, she worked then worked with her staff and we looked at what the [state] standards were, and then we said, Okay, let s talk about what an employee coming into or program needs, what technology competencies do they need to have? So we went back then and developed what we call our technology standards for our program, which was a direct result of the online class that I took. We broke it down into teams, and they came back with a set of standards that certainly incorporate the [state] standards, but went beyond the [state] standards saying, In our place and in our location, because we are a virtual school, employees, teachers, consultants, or art therapists coming in really need to be competent in these areas. That was one of the resources that the course provided. Since developing her school s technology standards, Marie has now partnered with her Instructional Technology Specialist to develop a self-assessment tool, and will use the results of this assessment to design professional development activities to meet staff needs. The eye-opening experience of examining technology standards during the Elements Leadership encouraged Marie to make some significant changes in her school. Out of this came individual staff assessments, and staff development activities. I think there were some things that I just really hadn t poked enough, in other words, like the standards. I just assumed standards are standards and they really don t mean that much, but I was wrong. They mean a great deal. Conducting a Survey A second story of impact came from Gwen, who was a middle school English teacher, with more than 30 years experience in the classroom. In the last year, she had taken three Elements courses, most recently the course on Thinking Critically with Data. While in the midst of the Data course, Gwen was inspired to try a new activity with her students on the topic of bullying. Her class had read several books involving bullying over the school year, since kindness and anti-bullying was a major focus at her school. Gwen created an activity involving her students in research on being a bully and what it was like to be a victim of a bully. The students created a survey about bullying relating to the literature we were reading, and went out in teams to collect replies from teachers, administrators and students in the school. Bringing this survey data back to class, students used the tools provided to Gwen in the Data course to critically reflect on the data. For her students, Gwen felt this activity was powerful. She noted that, we didn t quite know if everybody would tell the truth. But it seems like middle school kids are pretty blunt. Students learned, for example, how much it does hurt to be a victim, how much more of it must be going on than you think, and all the subtle ways it can take place. Gwen was motivated to use this activity while still in the midst of the Elements Data course, because I was taking the course at the end of this year, and it was a pretty good end of the year activity, and because we d already read the literature. She said, I did it right then. It was like, wow, this is kind of exciting stuff. I can do it. Let s try it, yeah. 21