Instructor Perceptions and Practices in the Cisco Networking Academy

Transcription

1 Cisco Networking Academy Evaluation Project Technical Report TR January 2006 Instructor Perceptions and Practices in the Cisco Networking Academy Barbara Bichelmeyer Hasan Cakir Alan Dennis Thomas Duffy JoAnne Bunnage Ali Korkmaz Semiral Oncu Kelly McNamara Hilmer Kelley Executive Partners Indiana University Kelley Executive Partners Indiana University 1275 East Tenth Street, Suite 3070 Bloomington, IN (fax) Kelley Executive Partners

2 PREFACE This Technical Report is one in a series of reports that examines the instructor teaching practices and instructor support practices in the Cisco Certified Network Associate (CCNA) Program offered through the Cisco Networking Academy. For a list of available reports, see our Web site ( This Technical Report presents a detailed view of the key findings of teaching practices and satisfaction of instructors with support activities. We presume that the reader is familiar with the Cisco Networking Academy and the CCNA program. A summary of the research discussed in this Technical Report is available on our Web site (WP and WP 05-04). This research was sponsored by the Cisco Learning Institute i

3 Executive Summary The purpose of this technical report is to describe findings from the Cisco Certified Network Associate (CCNA) Program Instructor Survey and the recommendations that arise from those findings. This paper describes the CCNA instructors in terms of demographics, it explores the impact of the CCNA program on instructors teaching practices, and it provides a picture for the value of instructor support practices in the CCNA program academies. This paper also explores instructors perceptions about whether the CCNA program approach can and should be applied to other educational settings. The conclusion of this paper offers recommendations for the CCNA program based on instructors insights. The CCNA Instructors Survey was developed by following rigorous validation procedures to ensure reliability and validity. The survey was administered to all active CCNA instructors from February through November We received responses from 2,142 instructors, of which 792 were instructors in the United States (U.S.) and Canada (which represents 20.2% of all active US and Canadian instructors). This report focuses only on the US and Canadian instructors. Instructors were grouped and analyzed based on the institutions in which they teach CCNA courses; high school, post-secondary (2-year college or 4-year college/university), and nontraditional (career or adult training centers). Most instructors have extensive teaching experience, and the majority of instructors hold a master s degree (55%), with another 29% holding bachelor s degrees. The CCNA instructors in high schools have more teaching experience and better educational credentials than typical high school teachers. Most high school CCNA instructors have no professional experience, but the majority of instructors at post-secondary institutions have significant networking experience. Analyses of teaching practices revealed CCNA instructors use a wide range of teachercentered and student-centered teaching methods. The most commonly used teaching practices in a typical class are hands-on activities, lectures, individual student work, whole class discussions, and demonstrations. Instructors report four basic practices to be most important to improving student learning: hands-on lab activities, class discussion, activity sequencing (e.g., lectures, demonstrations and then student labs) and metacognitive structuring (e.g., study strategies, structured questioning for problem solving, mnemonic devices and action plans to transfer learning to the workplace). Many instructors stated that hands-on activities help students to understand and remember content better. Furthermore, they challenge students by creating scenarios in which students must troubleshoot problems or set up lab equipment. With the exception of testing, most instructors spend more time preparing and teaching CCNA courses than other, non-cisco, courses. High school instructors spend more time than post-secondary instructors in all areas of course preparation. Most instructors believe that the standardized curriculum enables them to teach what should be taught because it helps them become more organized in their teaching and provides them with many resources for teaching. A minority of instructors feel that the standardized curriculum restricts them in teaching what they believe should be taught because of the large volume of the content and the need to teach to the test. Almost one-third of the i

4 instructors did not make any modifications. Two common changes were creating more challenging hands-on activities and adding study guides. Instructors reported that the CCNA program has prompted them to use more labs and hands-on learning activities, more technology, more online resources, more games, and more learner-centered instruction in their other courses. About 70% instructors believe that the CCNA model of instruction should be applied to other school subjects, especially the hands-on activities and online testing. Hands-on activities enhance student learning, while online testing allows the instructor s more time to focus on teaching. Instructor support is another important part in the CCNA program. A large majority of instructors reported being satisfied with the overall CCNA program. However, instructors in high schools reported significantly less satisfaction with the CCNA program. This was attributed to frequent changes in the curriculum, errors in online and printed material, the difficulty of course content, and instructors lack of networking experience. Instructors greatly value the 24/7 technical support because of its availability and the helpful responses they receive. High school instructors are likely to require more technical support and, therefore, may be more likely to be critical of the value of technical support to their teaching. Instructors also value professional development activities but have concerns about the timing of the activities and the amount of time required for them. ii

5 SECTION 1 INTRODUCTION The purpose of this report is to describe findings from the Cisco Certified Network Associate (CCNA) Program Instructor Survey and the recommendations that arise from those findings. This report identifies the classroom teaching practices and instructor support practices used for the four courses in the CCNA Program, based on an online survey completed by CCNA instructors. The survey focused on instructors teaching practices and their perceptions about the value of support they receive, along with the opportunities for improvement in classroom implementation of the CCNA program. This report summarizes results from two white papers that were provided in the fall of 2005 ( plus some additional results. In this section, we briefly describe the two principal objectives for the survey. The second section provides a summary of the research methods. The results are presented in the third section, with conclusions and recommendations in the final section. 1.1 Best Teaching Practices for Student Learning The first objective was to investigate the teaching practices used by instructors, including: a) how instructors implement CCNA curriculum in their classroom, b) what instructors believe to be the most important teaching practices to increase student achievement, c) how instructors allocate their time in class preparation and monitoring student progress, and d) how the instructors improve or modify the curriculum to make it more effective for their students, as well as ways in which Cisco and CLI could improve the curriculum to make it easier for them to teach. An instructor survey was developed to study these issues. The survey asked instructors to explain how they taught in a typical class session and the teaching approach used. We examined whether instructors used lectures, demonstrations, small group activities, handson activities, student presentations, and other methods. The instructors were also asked to note the sequence in which these activities were used. We also collected data on typical instructor/student ratios, the number and type of students assigned to help as teaching assistants, and the professional demographics of instructors, such as years of teaching experience. The survey also asked instructors to describe their most important teaching practices. This part of the survey enabled instructors to explain in some detail what they believed to have the most impact on student achievement and why. Additionally, we asked instructors to report the amount of time spent in course administration, presentation, facilitation, and feedback relative to other non-ccna courses they taught. Finally, instructors were asked to describe the most important way in which they customized, modified, or enhanced the CCNA curriculum materials to improve student learning. They also assessed the most important opportunities for improvement in regard to five key pedagogical elements (i.e., curriculum materials; assessment strategies and grading systems; teaching strategies used in classroom; teaching strategies used in labs; technology, tools, and resources available for the course). Some instructors also reported that the CCNA courses had changed the way in which they taught non-ccna courses. 1

6 1.2 Instructor Support A second objective was to understand how well the CCNA Program supports the academy instructors. We asked instructors to assess the value and quality of the major teaching support elements provided by the Cisco Networking Academy. Instructors were asked to assess each of the following components of the CCNA program based on importance for supporting the quality and effectiveness of their teaching and quality (i.e., does the program element need improvement). Open-ended areas in the survey provided space for instructors to elaborate more on their answers. Web-based learning materials Web-based quizzes and chapter tests Web-based final exams Laboratory exercises Web-based simulations and e-labs Books and other print materials Initial instructor training Annual meetings and on-going professional development workshops Periodic curriculum updates Support from RTC and/or CATC 24x7 Technical support from Cisco Quality Assessment Program (QAP) reports Student satisfaction reports Web-based community (e.g., teaching materials) Equipment discounts from Cisco Access to Cisco partners 2

7 SECTION 2 METHODS This section discusses the target population, as well as the development, administration, and analysis of the instructor survey. A copy of the Instructors Survey is included in Appendix A. Some of the sub-sections in this methods section were mentioned in August 2005 status report. Additionally, a summary of information about characteristics of instructors previously reported in the first instructor white paper (WP-05-04). 2.1 Participants of the Instructor Survey Participants were instructors actively involved with the CCNA program during the survey administration period (February through November 2005). The survey was accessible to all CCNA instructors worldwide from the CCNA program homepage. An alert appeared on the instructors homepage notifying them about the availability of the online survey. An was also sent to every CCNA instructor asking them to participate in the survey. It is common practice to send three invitation s to each potential survey participant in regular survey administrations. Following Cisco policy, only one invitation and two follow-up reminders to non-respondents were sent. We took further steps to increase the survey return rate, such as sending additional and paper mail invitations to those instructors who had given us permission to communicate with them. 2.2 The Instructors Survey The instructor survey contains five main parts. The first part of the survey includes items pertaining to instructors satisfaction with their teaching experience and the level of support from Cisco. The second part of the survey asks the instructors how they prepared for and taught a class session (e.g. the time they spent preparing and strategies they used). This factor asks the instructor to describe his or her typical approach for teaching a CCNA class session or an entire class topic. The survey s third part asks about best teaching practices and modifications instructors made in the course to increase student success. This part focuses on how the instructors in the CCNA Program modify the curriculum to improve students learning experience with the program. The fourth part includes items regarding how the CCNA Program has affected the way instructors teach and their beliefs about delivering other courses using the CCNA Program approach. The purpose of this factor is to understand instructor s insights regarding whether the CCNA Program can be a pedagogical model for teaching other courses in the educational system. The last part of the survey includes demographic questions about the instructor and the CCNA class taught. This part collects data about the instructor s educational and career background, and about his or her teaching credentials, and the demographic features of the class taught by each instructor, such as the number of students in the class and the average age of the students. The instructor survey has both closed-ended and open-ended questions. The closed-ended questions measure responses on a 5-point Likert scale (e.g., strongly agree to strongly disagree). Some closed-ended questions have an accompanying open-ended question which enables the instructor to provide additional information and opinions in a brief text comment to better explain the reasons behind the closed-end response. 3

8 2.3 Development of the Instructor Survey The development of the instructor survey was a result of an iterative process of literature review, discussions, validations, pilot tests, and revisions. The development process involved several steps, including creating factors based on the study s objective, reviewing the educational literature, conducting focus group interviews, revising the factors, creating survey items, and validating the factors and items. The purpose of the validation process for the instructor survey was to ensure that factors in the survey were consistent with the goals of evaluation for the CCNA program and that the items were clearly measuring these factors. The instructor survey also included several validation activities: a. First pilot study: The first pilot study was conducted with 12 instructors at regional CCNA meetings in New England and Texas in spring Based on the first pilot study feedback, there was a need to gather input about the changes from CATC staff who are responsible for instructor support and training. b. Focus group interviews: Two focus group interviews were conducted. At the first focus group, instructors provided their feedback about the clarity of items and their relevancy to the CCNA Program. At the second focus group, the instructors provided information about format and length of the instructor survey. c. Expert reviews: The instructor survey was reviewed several times by academics with expertise in the design of learning and teaching environments. The purpose of each review was to understand whether the questions in the survey are appropriate, communicate correctly with the instructors, and the layout of the survey was helpful. The expert reviews took place before each pilot study. d. Advisory Board review: The instructor survey was reviewed by the members of the Cisco Networking Academy Evaluation Advisory Board. The purpose of this review was to have these four experts in education, technology, and survey design review the survey and provide feedback for improvement. e. Second pilot study: The second pilot study was conducted to understand whether the revisions from the previous pilot improved the clarity of the items. We also tested the procedures to administer the survey. The survey was online at a web space for pilot administration purposes and 15 instructors had been selected to participate based on referrals from different academies. All instructors participating in the pilot study received a package in the mail containing instructions to take the survey, a hardcopy version of the survey, and a $25 check to compensate for their participation. 2.4 Administration of the Instructor Survey After developing and validating the items in the instructor survey, the IU Evaluation Team conducted usability tests to ensure smooth implementation of the survey. Because the CCNA program is taught by a wide variety of instructors, not all items in the survey would perfectly fit for every instructor. For example, an item for a high school teacher may not be appropriate for a post-secondary faculty. In order to prevent any confusion, the IU Evaluation Team developed an algorithm to change the items slightly for appropriate wording based on the instructors responses to certain items in the survey. For example, on the first page of the survey, the instructors were 4

9 asked to indicate the courses they have taught from the CCNA program and from their institutions during the past three years. Using this information, the survey algorithm selected appropriate wording for referring to the courses in certain items. In this algorithm, first CCNA2 and then CCNA3 courses were a priority. When we asked about instructor teaching satisfaction with CCNA courses, based on the information given by an instructor, the survey referred to either the CCNA2 or CCNA3 course. Similarly, in items that ask about comparing CCNA courses to other courses that instructors have taught in their institutions (such as computer technology, mathematics/statistics, science, and professional courses), the survey replaced the reference words based on the instructors responses. The algorithm prioritized other courses in order of, first Computer Technology (for post-secondary and non-traditional instructors) or Science (for high school courses) and then professional courses (for postsecondary and non-traditional instructors) or mathematics/statistics (for high school instructors). Other features of this dynamic interface include diverting the survey to different pages based on the instructor s responses and providing a save and come back function for the interface. The save and come back function was developed based on concerns expressed by some instructors that the survey was lengthy. This function allowed instructors to save their responses anytime. When an instructor wanted to stop answering questions, he or she was able to save the answers and then return later to the last item answered. This was an effective method for increasing the survey return rate and the number of completed surveys. The data collection period for the instructor survey started February 14, 2005 and ended November 1, This time period covered the instructors who taught during the spring, summer, and fall semesters. 2.5 Analysis Methods Methods for analyzing instructor data had two phases. In the first phase, we analyzed the quantitative survey data from the descriptive perspective with measures of central tendency and variation using frequency counts, means, and standards deviations. In the second phase, we analyzed the qualitative survey data to understand why the instructors thought that certain elements of the program were important or unimportant, and how the instructors conduct certain teaching practices in their classrooms. In the first phase of the analysis, instructors were placed into three groups based on where they teach CCNA courses. These groups were high school, post-secondary (2-year college or 4-year college or university), and non-traditional (career or adult education centers). Tables presented later in this report present the distribution of responses based on these three groups. The differences between these three groups for quantitative items were examined using statistical tests such as t-test or analysis of variance. Furthermore, we analyzed statistically significant differences using Cohen s d formula to understand the effect size of these differences. In the second phase of analysis, instructors open ended comments were analyzed using qualitative content analysis techniques. Content analysis is a common qualitative research technique for analyzing comments from surveys. Whole comments or parts of comments from instructors were linked to categories or codes representing similar thoughts. The codes in this analysis emerged from the comments of instructors, and then related codes were grouped into superordinate categories. Using computer analysis software, a coder read and created codes and linked the comments and codes to each other. Two coders worked 5

10 collaboratively to assure the reliability of codes and the coding process. Themes that emerged from the comments of instructors were grouped into the superordinate categories that are presented in this report. These analyses were conducted to present a picture of the CCNA instructors in terms of their teaching practices and satisfaction with the CCNA program. We plan to conduct further analysis of instructor practices in relation to student engagement and student achievement using multi-level analysis of the instructor data. Results of this analysis will be presented in subsequent reports. 2.6 Instructor Survey Participants and Participation Rates The first goal of our analysis was to understand the survey participation rates. The nature of the CCNA program allows instructors to determine the start date and length of the program based on the context of their institution. Due to differences between the actual course length and the course start and end dates instructors enter in the Academy Connection system, it is difficult to provide an absolutely precise survey participation rate because we cannot determine actual start and end dates for the courses. The method used in this report gives a very reliable participation rate for the instructor survey, but not absolutely precise numbers. We analyzed the survey return rates based on reported course start and end dates. The instructor survey was available to all active instructors between February 14, 2005 and November 1, Using the student demographic file, we identified all instructors who were teaching at least one class during the survey administration period, which gave us the target population of the instructor survey. Then, we analyzed survey responses and eliminated non-eligible and mostly incomplete cases. The ratio between these numbers gave us the participation rate. The educational systems in the U.S. and Canada are very similar to each other, therefore, we included instructor surveys from Canada in our analysis. Table 1 shows the target population and participation rates for the world, the U.S. and Canada. TABLE 1. The Instructor Survey responses based on current institution where instructors teach most CCNA courses Institution Types World Target population U.S. U.S. + Canada World Number of Completed Surveys U.S. U.S. + Canada High School (28.8%) 343 (46%) 357 (45%) Post-secondary (59%) 335 (45%) 367 (46%) Non-traditional (12.3%) 68 (9%) 68 (9%) Total (100%) 746 (100%) 792 (100%) Worldwide, 2,142 instructors completed the survey. Among the 3,928 CCNA instructors in the United States and Canada, 792 (20.2% return rate) completed the survey. As shown in Table 1, U.S. and Canadian instructors almost equally represent high schools (45%) and post-secondary schools (46%), whereas worldwide, post-secondary schools (59%) are more represented. This classification refers to the institution in which the CCNA program is offered rather than the type of degree or diploma program. 6

11 2.7 Who are the CCNA Instructors? The following section describes the characteristics of the instructors, such as highest education degree earned, networking and teaching experience, and courses taught Education At all three types of institutions offering the CCNA program, the majority of instructors hold master s degrees, with bachelor s degrees being the second most common (see Table 2). TABLE 2. Highest degree obtained by instructors Degree High School Post-Secondary Non-Traditional Ph.D. 1% 6% 3% Master s 61% 56% 37% Bachelor s 30% 25% 32% Associate 5% 10% 13% Other 3% 4% 16% Professional Networking and Teaching Experience While the majority of post-secondary instructors have considerable experience as networking professionals, more than half of the high school teachers have no professional networking experience (see Table 3). TABLE 3. Experience as a networking professional High School Post-Secondary Non-Traditional No Experience 54% 20% 39% Less than 1 year 6% 7% 2% 1-3 years 10% 18% 15% 4 years or more 30% 55% 43% Across all three types of institutions, the majority of instructors have 10 or more years of teaching experience (see Table 4). These data indicate that the instructors who teach in the CCNA program are mature professionals who have greater than average teaching experience. TABLE 4. Experience as a teaching professional High School Post-Secondary Non-Traditional Less than 1 year 2% 1% 5% 1-3 years 6% 7% 5% 4-6 years 12% 21% 25% 7-9 years 10% 13% 10% 10 years or more 70% 58% 55% 7

12 2.7.3 Subjects Taught by Instructors Instructors were asked what subjects they taught during the past three years other than CCNA courses. Instructors could select more than one type of course. Almost all CCNA instructors taught courses outside the CCNA program (see Table 5). Not surprisingly, the type of non-ccna course most frequently taught is computer technology. More than 70% of instructors in all three groups have taught at least one computer technology course other than CCNA courses during the past 3 years. TABLE 5. Percentages of instructors teaching other courses in last 3 years Type of Course High School Post-Secondary Non-Traditional Computer technology (other than courses provided by Cisco) 74.4% 86.6% 70.8% Science / Mathematics / Statistics 17.4% 12.6% 9.2% Industrial arts courses 15.3% 2.3% 1.5% Professional courses (e.g., business, engineering, nursing) 18.2% 14.5% 10.8% Other 20.4% 13.7% 18.4% Have not taught any other courses 8.5% 7.7% 16.9% Primary Teaching Focus We asked instructors about their primary teaching focus in their institutions. Instructors could select only one teaching area. Instructors in post-secondary and non-traditional institutions are more focused on computer technology courses than high school instructors (see Table 6). Almost half of the high school instructors have teaching focuses other than computer or networking technology (44%), whereas the overwhelming majority of post-secondary instructors primarily focus on Cisco or computer courses (83%). TABLE 6. Primary teaching focus of instructors Type of Course High School Post-Secondary Non-Traditional Cisco courses 19.5% 28.2% 40.0% Computer technology (other than courses provided by Cisco) 36.6% 55.2% 37.5% Science / Mathematics / Statistics 12.9% 2.8% 2.5% Industrial arts courses 9.7% 1.0% 2.5% Professional courses (e.g., business, engineering, nursing) 11.3% 5.7% 2.5% Other 10.1% 7.2% 15% 8

13 SECTION 3 RESULTS This section provides results from analysis of the instructor survey that identifies the teaching practices in a typical CCNA classroom and also the best teaching and instructor support practices. Results discussed in two previous white papers (WP and WP-06-01) are also summarized here. 3.1 What Teaching Practices Support Student Learning? Teaching Practices in CCNA Classrooms The first step towards determining best teaching practices in CCNA classrooms is to identify the full range of teaching practices that are utilized in a typical class session. The CCNA program does not prescribe a teaching method nor does it specifically suggest how to teach the CCNA curriculum. Therefore, instructors are free to choose and implement their own teaching practices in their classrooms. This creates a wide range of teaching approaches that are used in the CCNA programs at participating academies. We asked instructors to describe teaching activities in a typical CCNA classroom. They were asked to identify 6 teaching activities in the order they occur during a class session. They were provided with 12 different teaching practices to choose from. The instructors selected these activities according to the order that they occur in their class and they further explained how they were implemented. In this section, we see that the practices in a typical CCNA classroom include a variety of teaching methods. Table 7 shows the distribution of these teaching activities based on the type of institution. TABLE 7. Teaching activities used during a typical class session Teaching Activities High School Post-Secondary Non-Traditional Teacher-centered activities Lecture 70.4% 83.8% 64.0% Demonstration 52.1% 49.1% 42.0% Student-centered activities Hands-on activities 71.4% 84.9% 74.0% Individual student work 57.1% 26.2% 48.0% Online simulations/games 33.2% 15.9% 26.0% In-Class activities Small group work 26.1% 21.8% 30.0% Student presentation 8.2% 5.2% 8.0% Whole class discussion 32.5% 32.1% 22.0% Feedback activities Review homework 19.6% 15.9% 24.0% Review test results 30.4% 35.8% 36.0% Testing 54.6% 42.1% 56.0% Other 10.7% 11.8% 10.0% 9

14 Clearly the two most favored teaching methods across all three levels are hands-on activities and lectures More than 70% of the instructors reported that their students do hands-on activities at some point in a typical class session. Regardless of instructors institution level, the second most favored activity is lecture. Instructors from post-secondary institutions use lecture slightly more than instructors from other academies. However, all three level favor lectures as a typical teaching practice. After hands-on activities and lectures, other teaching practices are used relatively less frequently and they are implemented in varying degrees across the three levels. For example, the third most commonly used teaching practice for non-traditional instructors is testing. The third most commonly used teaching practice for high school instructors is individual student work. And for post-secondary instructors, the third most commonly used teaching practice is demonstrations. Table 7 lists the teaching practiced used in a typical class session and shows the diverse range of activities instructors use in their classrooms to teach CCNA courses Table 7 showed the percentages of teaching activities. Another way to look at the same data is to understand the proportion of a teaching practice relative to total teaching practices. In this way, we have an indicator of how much time instructors actually spend in a class using a particular teaching activity. Furthermore, these teaching practices can be grouped into 4 main groups for better representation. Table 8 shows the distribution of these teaching practices based on the type of institution. TABLE 8. Proportion of teaching activities relative to total classroom practices Teaching Activities High School Post-Secondary Non-Traditional Teacher-centered activities 25.7% 29.3% 23.4% Demonstration 10.9% 10.9% 9.3% Lecture 14.8% 18.4% 14.1% Student-centered activities 35.3% 29.3% 34.4% Hands-on activities 15.5% 19.8% 16.7% Individual student work 12.9% 6.0% 11.9% Online simulations and games 6.9% 3.5% 5.7% In-Class activities 14.4% 14% 13.7% Small group work 5.7% 4.8% 6.6% Student presentations 1.7% 1.2% 1.8% Whole class discussion 7.0% 8.0% 5.3% Feedback activities 22.4% 24.3% 26.4% Review homework 4.2% 3.4% 5.3% Review test results 6.2% 7.8% 7.9% Testing 12.0% 13.0% 13.2% Other 2.4% 3.2% 2.2% Table 8 provides a range of activities that are used by the teachers in CCNA classrooms. The 12 distinct teaching activities were grouped as teacher-centered activities (i.e., demonstration, lecture), student-centered activities (i.e., hands-on activities, individual 10

15 student work, online simulations and games), in-class activities (i.e., small group work, student presentations, whole class discussion), and feedback activities (i.e., review homework, review test results, testing). Table 8 presents similarities and differences among the instructors from different types of institutions. Regardless of the institution, student-centered activities are the most frequently used activities by the instructors. Instructors from high schools, colleges, and non-traditional institutions favored hands-on activities, which were the most frequently used activity. In the teacher-centered activity group, college instructors used lecture and demonstration slightly more than instructors in the other groups. Lecturing is the second most used teaching activity for college instructors. Instructors stated that they utilized teacher-centered teaching activities usually at the beginning of their classes. The third most used teaching activity group involves providing feedback. In order to inform students about their progress in the CCNA classes, teachers indicated that they mostly used testing related to chapters. These tests can be administered at the end of each class as quizzes or as midterm exams. Reviewing the test results is the second most common method of providing feedback to students. In-class teaching is the least common group of teaching activities used by CCNA instructors. The distribution of specific in-class teaching practices based on institution varied. While college instructors used whole class discussion more than others, they utilized small group student work and student presentations the least. With small differences, this statement is correct for instructors at other types of institutions as well. This is particularly striking because these teaching activities may contribute to student development in areas such as teamwork or verbalizing ideas in the classroom more than other activities. Even those these are very important skills for students, the teaching approaches that lead to development of these skills appear to be the least utilized activities in the CCNA classrooms The Most Important Teaching Strategies In addition to asking instructors to describe teaching practices in a typical CCNA class session, we asked them single most important teaching strategy that they believe helps their students best in their learning process. Teaching in the CCNA learning environment is different than teaching in the more traditional K-12 and higher education environments. This difference is due to the structural differences in the program and the nature of the program content. It is important to understand the best teaching practices in this new environment because this information can be used to better support the instructors. To learn about best teaching practices, we asked instructors to describe the most important teaching strategy they use to help students learn. Instructors described four principal teaching strategies as being most important to student learning: (1) hands-on lab activity, (2) class discussions, (3) activity sequencing, and (4) metacognitive structuring (see Figure 1). We discuss these below in order of most frequently to least frequently mentioned. 11

16 Figure 1. Most important teaching practices to help students learn Hands- on Lab 64% Activities Class Discussion 14% Activity Sequencing 11% Metacognitive Structuring 4% Other 7% Hands-on labs with small group discussion, questioning, and lab work follow-up A common strategy was to demonstrate how to do a lab task and then have students perform the same task. Additionally, some instructors place students in small groups and have students work on tasks together, and thus, teach each other. After students complete the lab tasks, some instructors follow up on student learning by providing the means for students to reflect and report on the tasks that they have just completed. Instructors also reported that they simulate troubleshooting and set up lab equipment in order to go beyond the procedural steps that are taught in lab tasks and to help students engage in higher order thinking skills Class discussions During the small group discussions, instructors provide students with a scenario in which they must solve a problem, or instructors have students discuss the topics of the week and present them to the class to share their insights with their classroom peers Activity sequencing A number of instructors stated that their best teaching strategy is one in which they follow a specific sequence of teaching activities. The sequence usually begins with lecturing to the students, then moves to demonstrating tasks, and ends with having students conduct the labs Metacognitive structuring Metacognitive structuring is a teaching strategy that aims to help learners engage in higherorder thinking, which enables them to take control of their thinking and learning processes. General examples of metacognitive strategies include teaching students to use study strategies, to engage in structured questioning for problem solving, to use mnemonic 12

17 devices for memorization, and to create action plans for transfer of learning to the workplace Time Spent on the CCNA2 Course In general, instructors reported that they spend more time preparing for CCNA classes than for their other (non-ccna) classes (see Table 9). Specifically, the time spent on preparations before the term begins and the time spent preparing for lab sessions are significantly greater than for other courses that instructors teach at their institutions. TABLE 9. Average perception of time spent on CCNA course activities relative to other course activities Time spent preparing before the term begins Time spent preparing for classroom sessions High School Post- Secondary Non- Traditional Time spent preparing for lab sessions Cohen d Effect size.53 (medium).45 (medium).38 (medium) Time spent grading student work ns Time spent providing feedback to students on their academic performance Time spent mentoring students on their academic and career plans ns ns Scales: 1 = Much less time on CCNA, 3= The same amount of time, 5 = Much more time on CCNA We also used z-scores to understand how CCNA instructors spent more or less time relative to their other courses. In figures 2 and 3, zero (0.0), shows the same amount of time on both courses. All instructors reported that for the CCNA course, they spent more time on preparations before the term begins, on preparations for classroom sessions, and on preparations for lab sessions compared to time spent on other courses (Figure 2). Figure 2. Differences in amount of time for course preparation Before the term Less time on CCNA More time on CCNA Classroom sessions Lab sessions High School Post Secondary Non-Traditional 13

18 All instructors reported that grading student work in the CCNA courses takes less time than in other courses. As figures 2 and 3 indicate, high school instructors take longer to prepare for CCNA courses than other instructors, but they spend similar amount of time as other instructors in grading, providing feedback, and mentoring students in CCNA courses. Figure 3. Differences in amount of time for student academic progress Grading student work Less time on CCNA More time on CCNA Providing feedback Mentoring students High School Post Secondary Non-Traditional In the survey, instructors were asked to explain why they spent more or less time on CCNA courses as compared to other courses. Instructors who spent more time on CCNA courses provided a wide variety of reasons. First, some instructors perceive the course content as challenging for students and therefore, spend more time preparing to teach. Second, instructors who are less familiar with the topics in CCNA courses spend more time in preparation. Third, changes and updates in the curriculum require instructors to spend time updating their knowledge and their lesson plans for teaching CCNA classes. Fourth, errors in curriculum and lab materials require that instructors spend more time preparing for classes in two ways: reviewing materials to check for errors, and correcting existing material or preparing additional material. Finally, some instructors spend more time on CCNA courses to ensure that students are prepared for CCNA certification exams. Those instructors who reported spending less time on CCNA courses than traditional courses provided three main reasons: familiarity with the content, online testing, and online curriculum. After instructors gain experience and familiarity with the course content, it takes them less time to prepare for their classes. The online testing and immediate feedback features of the curriculum saves time for instructors and enables them to focus more on teaching and mentoring students. Additionally, the teaching material and resources provided by the CCNA program decreases the amount of time required to prepare the courses Do Instructors Modify Curriculum to Improve Student Learning? Instructors play a crucial role in facilitating student learning. Most instructors customize teaching and learning materials based on their own experience and the ability level of their students in order to ensure a good fit between the materials and their students needs. We attempted to understand how instructors modify the course materials to identify possible areas for improvement in future course materials. We asked instructors if they had made any major improvements to CCNA courses that helped students to learn, and if instructors 14

19 answered yes, we asked them to describe the improvement and explain how it helped student learning. Instructors responded to this question with a wide variety of answers, which were grouped into five categories. Figure 4 presents these categories in order from the most frequent to least frequent number of responses. In the largest category, 120 instructors stated that they are happy with the curriculum, material and the content of CCNA courses and they have not made any modifications to improve student learning. Comments of instructors in this category indicated that they are either very satisfied with the curriculum and materials or that they do not have time for both teaching the material and adding new material to the curriculum. The answers in this category show a balanced distribution among instructors in high schools (51), postsecondary institutions (56), and non-traditional academies (13). The second largest category (107 instructors) focuses on improvements and adjustments related to hands-on labs. Instructors identified a number of strategies that enhance handson labs and e-labs to better facilitate student learning. Thirty-seven instructors report that they increase the amount of time students spend in labs during class or by opening the labs for long hours after class so students can work with the equipment to complete networking tasks. Some instructors stated that they have used router simulator software or had students create their own router simulator to work at remote locations from the school lab site, such as in their homes. Also, 22 instructors reported that they created new and more challenging lab tasks to develop students critical thinking about the situations they will face as professionals, or revised lab materials to make them error-free. Another 12 instructors stated that they created lab tasks that require students to build and manage a wide area network and manage it from scratch. Finally, 10 instructors stated that, because they believe lab equipment is the key to successful student learning, they work to update and enrich the lab equipment. Figure 4. How instructors modify the curriculum No changes Challenging labs Metacognitive support Adjusting content Innovative teaching methods 15

20 Sixty instructors stated that they have made improvements and adjustments to the curriculum by providing metacognitive supports such as review and guidance materials for readings, tests and study methods. Forty instructors explained that they prepare these guidance materials because they feel the curriculum materials are too difficult for some students, especially high school students, and because the curriculum does not provide adequate cues that distinguish between important concepts and nice-to-know concepts. Instructors also mentioned that they have provided students with strategies for note-taking and completing homework assignments, developed clear and organized syllabi that provide an overview of the course for students, they used analogies as much as possible to teach difficult topics, and they have created competitive environments to motivate student learning. The fourth category includes comments from 52 instructors about adjustments and improvements related to the content of CCNA courses. Thirty-one instructors stated that the content of some topics (especially subnetting and access control lists) is not sufficiently detailed so they provide extra material on these topics. Additionally, some instructors stated that the order of particular chapters does not make sense or limits the time that instructors spend, therefore, they change the order of chapters to better teach the content and give students a firm foundation for subsequent courses in the curriculum. Several instructors stated that errors in the curriculum hinder student learning and motivation so they have revised chapters and lab materials to fix errors. Some instructors stated that they enhance their courses by incorporating their own personal industry experience into their teaching. These instructors give examples, problems and solutions from their work experience to students as they see appropriate. Other instructors mentioned that they have created ongoing case studies that can be used throughout an entire course. The fifth category (also 52 instructors) focuses on the ways in which they emphasize routine teaching methods. Comments include instructors statements about bringing a new teaching method to their course or emphasizing teaching activities such as individual student work, case studies, class discussions, team work, demonstration, lecture, student presentations, field trips, study groups Does Teaching CCNA Impact on Teaching Other Courses? The curriculum and the teaching methods that are built into CCNA courses are different than those traditionally used by high school and college instructors who prepare and implement their own course materials. The CCNA program provides rich teaching and learning resources, authentic assessment methods, and online access to course content, lab activities and a testing system. We asked instructors, 1) whether their teaching way in the CCNA program is different from the way they teach other courses, and 2) whether teaching in the CCNA program has changed the way they teach in other courses. Table 10 shows the percentages of instructors who perceived differences between teaching in the CCNA program and teaching traditional courses. While most post-secondary instructors state that teaching in the CCNA program is not much different from teaching other courses, instructors from high schools think that teaching in the CCNA program is somewhat different from teaching other courses in their institutions. 16

21 TABLE 10. Difference of teaching CCNA from other courses Teaching CCNA courses is different from other courses Teaching CCNA courses is somewhat different from other courses Teaching CCNA courses is not much different from other courses High School Post- Secondary Non- Traditional 48.5% 28.2% 43.9% 26.3% 34.5% 34.1% 25.2% 37.3% 22% We asked instructors why they think teaching in the CCNA program is different from or the same as teaching non-ccna courses. We received 299 responses, and analysis of these data yielded a wide variety of answers. Among these answers, four themes emerged. Instructors who believe that teaching in the CCNA program is different from other courses state that this is due to the many hands-on labs and simulations provided by the CCNA program (73 responses), the online nature of the CCNA curriculum (51 responses), the fact that the curriculum and testing materials are pre-prepared for them to provide structured content for CCNA courses (47 responses), and the logistical aspects of online testing provide accessibility and immediate feedback (28 responses). Finally, several instructors stated the reason they teach differently in CCNA courses than other courses is because the CCNA courses have much more content and are much more complex than other courses. Figure 5. Why teaching CCNA courses is different Hands-on experience Online delivery Pre-prepared teaching material Online testing & feedback Though we expected that teaching with an online standardized curriculum might have an impact on the way instructors teach other courses, instructors reported that teaching in the CCNA program did not lead to changes in the teaching methods they use for other traditional courses (see Table 11). Instructors from all three types of institutions agreed that teaching in the CCNA program does not have much effect upon the way they teach non- CCNA courses. 17

22 TABLE 11. Impact of teaching CCNA courses teaching other courses Teaching CCNA courses changed teaching other courses Teaching CCNA courses somewhat changed teaching other courses Teaching CCNA courses did not change teaching other courses High School Post-Secondary Non-Traditional 19.2% 18.8% 19.5% 21.9% 25.2% 29.3% 58.8% 56% 51.2% We also asked instructors to explain how and why teaching in the CCNA program has changed the way they teach in non-ccna courses. Most instructors did not answer this question because they did not believe the CCNA courses have affected the way they teach other courses. The 222 instructors who provided responses stated that teaching in the CCNA program has prompted them to use more labs and hands-on learning activities in other courses (49), to utilize more technology for teaching and learning activities in the courses (26), to use more online resources (21), to conduct more practical assessments and online testing (21), to use alternative methods such as games for learning activities (18), and to engage in more learner-centered instruction in their other courses (16). Figure 6. The impact of CCNA on other courses More labs and hands-on More technology More online resources More online testing Use of alternative methods More learnercentered approach Applying the CCNA Approach to Other Courses Five components of the CCNA program make it markedly different from traditional teacherdesigned courses in high schools and post-secondary institutions. The online curriculum, online testing system, hands-on activities, professional development system and the technical-administrative support systems combine together to create a distinct CCNA approach to instruction. We measured to what extent instructors think that these five 18