Main Author: Amy Roche, Instructional Designer Contributing Authors: Tricia Clark, Instructional Technologist Mary Ann Mengel, Multimedia Specialist John Shank, Director for the Center for Learning & Teaching Rev. 2/4/2010 Page 1
Table of Contents Introduction... 3 Hybrid Course Framework... 4 Faculty Preparedness... 4 Collaborative Model Approach... 5 Course Objectives & Student Learning Outcomes... 6 Creating a Hybrid Course... 6 Initial Meeting & Project Orientation... 8 First Deliverable (Syllabus and High Level Outline)... 8 Second Deliverable (Detailed Design)... 9 Third Deliverable (Initial ANGEL Development)... 10 Fourth Deliverable (Completed Course in ANGEL)... 10 Conclusion... 10 Appendix I: Learning Activities... 11 Textbook Reading Assignments... 11 Online Instructor Lecture... 11 Web Activity... 12 Visual and Interactive Elements... 12 Discussion Forums... 13 Homework Assignments... 14 Quizzes/Exams... 14 Appendix II: Multimedia Instructional Elements... 15 Bibliography... 16 Rev. 2/4/2010 Page 2
Introduction This document summarizes the hybrid course development process at Penn State Berks as part of the Blended Learning Initiative and the Teaching & Learning Innovation Grant. Penn State Berks follows the guidelines provided by the Penn State Quality Assurance Standards (http://weblearning.psu.edu/resources) for the online component of hybrid courses. The Penn State Quality Assurance Standards provide a measure of quality assurance for online courses in order to serve the e learning needs of Penn State students. In addition, Penn State Berks also follows an adapted model of the Quality Matters program, which is a faculty centered, peer review process to certify the quality of online and hybrid courses (http://www.qualitymatters.org/). In order to understand the hybrid course development process at Penn State Berks, it is essential to understand five key terms: Face to Face Course: Also called an on campus or traditional course. A face to face course is a traditional classroom course that does not incorporate any online elements. Web enhanced Course: Web enhanced courses may be a traditional classroom course that incorporates online elements to facilitate communication and discussion and/or to provide students with enhanced content, extend communication, and provide links to other resources. The purpose is to enhance quality within an otherwise traditional environment by promoting communication and interactivity among students and faculty. (Office of the University Registrar ARUAC Schedule Course Section) Hybrid Course: Courses that combine Web and traditional classroom instruction. The percentage of online material vs. classroom sessions can vary depending on the individual course. At Penn State, these are often referred to as Blended Learning courses. Blended learning/hybrid courses are offered through resident instruction at Penn State. (Web Learning @ Penn State Glossary) Hybrid courses are specific packages of online and face to face content and processes organized to reduce or replace the number of required class sessions in order to improve effectiveness and flexibility for instructors and students and/or to achieve other efficiencies. Hybrid courses reduce by approximately 40% or more the number of required classroom sessions, although some classroom sessions are required. (Office of the University Registrar ARUAC Schedule Course Section) WEB Course: (Also called an Online Course) Penn State courses that do not meet in traditional classrooms, with the possible exception of scheduled exams. (Web Learning @ Penn State Glossary) Distance Education: The nationally recognized leader in online education, the United States Distance Learning Association, defines distance education as the organizational framework and process of providing instruction at a distance. Distance education takes place when a teacher and student(s) are physically separated, and technology (i.e., audio, video, computers, print) is used to bridge the instructional gap. (Glossary of Terms) Online courses may or may not be distance courses depending upon if students are at geographically dispersed locations. Rev. 2/4/2010 Page 3
Note: Please refer to the latest Faculty Handbook (http://www.bk.psu.edu/facultystaff/fachandbook.htm) for general Penn State Berks instructional policies and procedures. Hybrid Course Framework A recommended structure for the delivery of hybrid courses has been identified to help instructors organize materials in the learning management system, ANGEL. In hybrid courses as with face to face courses, 3 or 4 credit courses are delivered over a 15 week period during the spring or fall semesters. This is followed by up to one week of exams or comprehensive experiences. Hybrid courses are not necessarily required to have 15 class meetings in a semester. Just as an on campus course may meet once a week while others meet twice or three times a week, a hybrid course may consist of any reasonable number of virtual and face to face classes. Each virtual class (a.k.a. module) or face to face class can correspond to a chapter in the textbook, to one topic, or to the amount of material that can be covered in a specific time period. When redesigning a face to face course as a hybrid course there is a strong emphasis on integrating the face toface classes and online classes in an organized pedagogical manner. This structure allows for students to easily identify the flow of the face to face and virtual classes and how they complement one another. Furthermore, this process focuses on that fact that both components interact pedagogically with one another and are not treated as separate parts. Due to the unique flexibility of the hybrid model, the class structure of a hybrid course varies significantly from one course to another. To assist in establishing structure, instructors should consider referring to each online module as a "class" and to clearly establish due dates. Also be sure to schedule appropriate time for submission of final exams or comprehensive experiences. During the summer, 3 or 4 credit courses are delivered over a 6, 7, or 8 week period; however, instructors should cover the same content covered in a regular 15 week semester. If desired, courses can be reorganized into 6, 7, or 8 modules and face to face class sessions. Some instructors may choose to maintain the same course organization by simply conducting condensed face to face classes and changing the due dates of the online modules to fit the shorter time frame. For example, if the online assignments are due every 7 days during the regular semester, they might be due roughly every 3 days during the summer. In this way, instructors keep track of only one version of the course and reduce the possibility of conducting a course that does not contain all the updates made in the past year. Faculty Preparedness The most successful hybrid courses are created by faculty who, in addition to being subject matter experts, are comfortable with using technology, computers, and the Internet. In addition, faculty should also take steps to establish the correct mindset to teach a hybrid course. As part of the hybrid course design process, faculty are expected to take a faculty self assessment (http://weblearning.psu.edu/news/faculty self assessment). This will help them determine how prepared they are for teaching the online aspect of a hybrid course, and assist them in becoming a good course author(s). It is also recommended that faculty take the OL 2000 course offered by the Faculty Development office of the World Campus to further prepare for the online aspect of a hybrid course. OL 2000 is a faculty development course that is designed for faculty who are teaching or preparing to teach an Rev. 2/4/2010 Page 4
online course through the World Campus, but is also available to residential faculty. OL 2000 provides faculty and instructors an introduction to the online experience from a student perspective. As part of the hybrid development process, the CLT will assist in evaluating the technology readiness of faculty members and will provide instructional technology training and resources for faculty members to learn more about technologies and their pedagogical uses. Collaborative Model Approach The process of creating a hybrid course at Penn State Berks is an interdisciplinary team effort. This interdisciplinary team consists of one or more faculty members who serve as the course author(s), and members of the Center for Learning & Teaching (Director of CLT, Instructional Designer, Multimedia Specialist, and Instructional Technologist). A librarian or other technical person may be a part of this interdisciplinary team as needed based upon faculty request. This interdisciplinary team model is represented in the graph below. Course Author (Faculty Member) Librarian / Other (if needed) Instructional Technologist Collaborative Model Approach Director of CLT Instructional Designer Multimedia Specialist The team will meet on a regular basis throughout the entire development of the hybrid course. Meetings include, but are not limited to, an initial meeting and deliverable sign off meetings where the team agrees that a deliverable is met and the hybrid course development process can move to the next step. Rev. 2/4/2010 Page 5
The specific roles of each team member within this collaborative model approach can be found in the chart below: Course Author(s) (Faculty Member(s)) Director of CLT Instructional Designer Multimedia Specialist Instructional Technologist Subject Matter Expert Provides course content Schedules team meetings Sets deadlines for deliverables Ensures completion of deliverables in a timely manner Ensures completion of assessment and evaluation of deliverables Recommends pedagogical applications of instructional technologies Oversees pedagogical aspects of hybrid courses Serves as main point of contact with Course Author Designs new multimedia instructional elements (animations, videos, interactive games, online tutorials) Develops new multimedia instructional elements (animations, videos, interactive games, online tutorials) Consults with faculty to incorporate multimedia tools (podcasts, video...) Assists with the integration and support of instructional technologies Consults with faculty on issues related to using the learning management system The team, as a whole, acts as a course reviewer to ensure all guidelines within the Penn State Quality Assurance Standards and adapted model of the Quality Matters program are adhered to. In addition, student interns within the CLT (who are not included in the graph and chart above), will be a part of this process by assisting in the responsibility of entering the course content into the learning management system. Course Objectives & Student Learning Outcomes The recommended foundation of any course at Penn State Berks is the student learning outcomes that are articulated for each program and general education areas. This information is available from the Student Learning Outcomes website (http://bk.psu.edu/information/30584.htm?cn711) or request to the respective Program Coordinator. Questions about course content or course outcomes should be directed to the Program Coordinator for that major or course. The course design must align with the student learning objectives for that course. It must also outline course sequence, activities, and expectations, and create a structural framework that ensures a degree of learning equivalent to the same face to face course. Creating a Hybrid Course During the hybrid course development process, there is an option to create multimedia instructional elements. Multimedia instructional elements, such as animations, videos, interactive games, and online tutorials can be an effective part of a hybrid course. (For examples of multimedia instructional elements, please visit the Samples of our Work portion of the CLT website.) However, because of the time and effort required for design and development, multimedia elements are considered an optional (or separate) part of the hybrid course Rev. 2/4/2010 Page 6
development process. If interested in creating multimedia instructional elements during this process, please see Appendix II: Multimedia Instructional Elements. The Center for Learning & Teaching tries to allow at least 17 weeks for the design and creation of a hybrid course. The course development process can vary. The process can be completed in less time than the allotted 17 weeks if: the course author(s) can devote a significant amount of time to the project in a condensed timeframe, the course approver(s) can review the course promptly, no significant/major revisions of the course are required, and/or there are few or no multimedia instructional elements that need to be created. The process may require more than the allotted 17 weeks if there are numerous multimedia instructional elements, scheduling conflicts amongst the team members occurs, etc. The Center for Learning & Teaching estimates that hybrid course design takes at least 65 hours of effort on the part of the course author(s), including assisting with entering content into the ANGEL Learning Management System. The complete number of hours of effort for the course author(s) can be as high as 120 or more hours, depending on the learning activities integrated into the hybrid course. Prior to creating a hybrid course, an instructor must get proper approval from their program coordinator, division head, and dean. A hybrid course is created in four discrete phases. Each phase concludes with a deliverable that is agreed upon and signed off by all members of the team, as well as external members when designated. External members can include faculty members within the department or discipline who will provide feedback on course content throughout the hybrid course development process. Completion and approval of each of these phases and associated deliverables is required before the course can be completed. The four phases with approximate timelines are outlined in the following table. The completion dates for all phases are based on the assumption that the course development process is started at least 17 weeks prior to the start of the semester in which the course is to be delivered. Component Hours Effort for the Course Author(s) (Faculty Member(s)) Initial Meeting & Project Orientation 2 1 st Deliverable Syllabus and Outline 4 5 Team Members and, if applicable, External Member Review 1 3 2 nd Deliverable Detailed Design +25 +45 Team Members and, if applicable, External Member Review 3 8 ANGEL Training and Support 6 10 3rd Deliverable ANGEL Development 12 20 Instructional Design Review 4 10 4th Deliverable Completed Course 5 10 Instructional Design Review 2 Team Members and, if applicable, External Member Review 1 5 Rev. 2/4/2010 Page 7
The Center for Learning & Teaching and the course author(s) will agree on a due date for each of the major deliverables listed above. These deliverables must be completed by the agreed upon dates or it may be necessary for the class to be cancelled or delayed. While an instructor is developing a course, assistance is available from the Center for Learning & Teaching staff. This should be an iterative process between the course author(s) and all members of the Center for Learning & Teaching, especially the Instructional Designer. Frequent contact with the Instructional Designer will ensure that a course: Stays on schedule for delivery Incorporates appropriate use of technology Is manageable for both you and your students Uses effective interaction and lecture elements Integrates constructivist and web based learning activities Initial Meeting & Project Orientation Prior to the Initial Meeting, faculty will complete the New Course Questionnaire that is adapted from Penn State Harrisburg. During the Initial Meeting, the faculty responses to the questionnaire will be discussed. The questionnaire contains questions regarding how the faculty member(s) has previously taught the course, what they plan to teach if they have not taught the course before, their background and interests, the resources available to them within their department, and their working style. This will allow the Center for Learning & Teaching to better understand the course and the resources that are available to the course author(s). First Deliverable (Syllabus and High Level Outline) The first deliverable consists of basic information about the course, including the course syllabus and a highlevel outline of the learning activities for each of the online and face to face classes that make up the course. Learning activities for the high level outline may include: In class instructor lectures Online instructor lectures via audio/video recording, narrated PowerPoint, or lecture notes Rev. 2/4/2010 Page 8
Reading assignments In class activities Web activities Visual and interactive elements Discussion forums Homework assignments Quizzes/exams For a detailed description of each learning activity listed above, as well as additional learning activities, please refer to Appendix I: Learning Activities located at the end of this document. At this point in the process, the learning activities do not have to be completely designed, but there must be enough descriptive information for each element so that the instructional activity is clearly understood. A template for this deliverable will be provided in a Microsoft Word format. In addition, faculty are highly encouraged to insert the high level outline into the syllabus, as this will greatly benefit students. The high level outline and course syllabus must be agreed upon by the team members and, if necessary, external members. This will provide mutually agreed upon design standards that will be utilized through the rest of the course development process. This deliverable must be completed in two to three weeks and agreed upon by team members and, if necessary, external members. Second Deliverable (Detailed Design) This deliverable fully fleshes out the high level design created in phase one by providing detailed information about each learning activity. This means all major course interactions are designed including quizzes, exams, experiential learning activities, web based discovery assignments, etc. In order to complete the second deliverable, you will consult with the Instructional Technologist, Instructional Designer, and Multimedia Specialist in the design of learning activities. The roles of each position during this time are as follows: The Instructional Technologist will assist in the integration and support of an appropriate instructional technology for each activity. The Instructional Designer will recommend pedagogical applications of instructional technologies and oversee the pedagogical aspects of the hybrid course. The Multimedia Specialist will consult with faculty to incorporate multimedia tools (podcasts, video ). After completing this deliverable, you will have a complete course map of the online and face to face content and activities. The online content and activities, as outlined in the course map, are ready to be implemented into ANGEL. The course map will organize all of the text based information and media for each instructional element by week. This plan will be delivered as a Microsoft Word document. Rev. 2/4/2010 Page 9
This deliverable must be completed in four to six weeks and agreed upon by team members and, if necessary, external members. Depending on circumstances, including the availability of external team members (e.g. faculty members within the department or discipline during the summer), the second deliverable may include three or four sample modules instead of the entire course. If this case, the remaining modules must be completed before initial ANGEL development of the online aspects of the course. Third Deliverable (Initial ANGEL Development) The primary hybrid course delivery platform at Penn State Berks is ANGEL. The Center for Learning & Teaching will coach and collaborate with the course author(s) on migrating the online content and activities onto the ANGEL platform. This will include overseeing CLT interns who will assist in importing and/or creating the online content and activities for the course into the ANGEL learning management system. During this time, faculty will consult with the Instructional Technologist on issues related to using the learning management system. This deliverable must be completed in two weeks and agreed upon by team members and, if necessary, external members. Fourth Deliverable (Completed Course in ANGEL) At this milestone, the course is ready for final review. All of the online classes and discussion forum entries for the hybrid course are complete. The instructor should be comfortable with ANGEL and be ready to deliver the hybrid course if needed. This deliverable must be reviewed and agreed upon by the team members and, if necessary, external members. This activity is an iterative process where the instructor, with the assistance of the CLT interns, may have to make adjustments. This activity must be completed and approved in two weeks. Conclusion After the first semester of implementation, all team members will reconvene to make sure all goals and objectives were achieved, evaluate student feedback, and review student assessment. This process will allow the team to identify if any further course enhancements are indicated. Rev. 2/4/2010 Page 10
Appendix I: Learning Activities The most successful hybrid courses have the following characteristics: Contain a variety of course materials, interactive activities, and assessments all of which are clearly and directly aligned with the course outcomes. Are typically constructed as a collection of structured, recurring activities along with one or more major projects. Make efficient and effective use of the technology available to achieve the learning identified in the course outcomes. Include problem scenarios or case studies that give opportunities for student interaction, critical thinking, and exposure to several sources of electronically based and retrieved information. Include some self check or practice types of assignments to give students an opportunity to measure their own learning progress (for example: practice quizzes, model papers, peer reviews, interactive games, submission of drafts). Focus on student centered teaching, utilizing strategies that promote discovery based learning and accommodate different learning styles. The face to face class meetings can contain the typical activities such as instructor lectures, in class discussions, question and answer sessions, student presentations, group work, etc. The virtual class meetings should contain several of the learning activities described below. Textbook Reading Assignments Textbook reading assignments are just one of several tools used to create activities which enable students to master the related learning objective(s). The following should also be incorporated into all hybrid courses: Clear directions about where to locate the reading assignments within the textbook. Either the same or greater amount of reading assignments than a traditional course. Directions that include how the reading assignments are aligned with specific learning outcomes Online Instructor Lecture In an on campus course, students are not expected to simply read a chapter in a textbook, do homework, or take a quiz. Instead, they receive valuable guidance through lectures, presentations, discussions, clarifications, etc. Hybrid students deserve no less. In a hybrid course, an instructor can utilize face to face lectures or incorporate online lectures that can be delivered via text, audio, narrated PowerPoint, or even video. If using audio, narrated PowerPoint, or video, provide a transcript to accommodate different abilities and learning styles. The online lecture should be used to explain, focus, and teach. Rather than just stating, read chapter one, tell students which elements in the chapter are important. Provide clarifying information and relevant, real world examples. Remember, unlike in a physical classroom, in a virtual class it is not possible to gauge reaction and measure understanding from visual cues and feedback. Therefore, it is vital to provide substantive guidance, examples, and frequent opportunities for learning. Rev. 2/4/2010 Page 11
Web Activity Web activities are one way to create interesting, immersive experiences for students. The following should be incorporated into all hybrid courses: Links to related materials for information or research purposes. Hyperlinks to websites that increase, contextualize, or deepen learning, and stimulate discussion. Assignments that require the student to demonstrate proficiencies for not only information access, but also information management, organization, and evaluation. Look for websites that will enhance the course and incorporate these sites into the course design. Often, textbook publishers will provide companion websites that complement the textbook. In addition, assign students the task of exploring the Internet to find sites that amplify a concept that is being covered. Visual and Interactive Elements For the online aspect of hybrid courses, it is essential to use a variety of visual and interactive means to enhance learning due to the physical absence of the faculty member for a portion of the course. The Center for Learning & Teaching staff can help create PowerPoint presentations, audio clips, podcasts, video clips, animations, simulations, interactive games, other multimedia learning objects, virtual classrooms (Adobe Connect and Second Life), wikis, blogs, etc. that can actively engage students. Take advantage of ancillary materials such as CDs, PowerPoint presentations, pre existing multimedia learning objects, or course cartridges that many publishers now include with the textbook. Furthermore, there are reliable websites of online multimedia resources, such as: Merlot (http://www.merlot.org/merlot/index.htm) Wisconsin Online (http://www.wisc online.com/) The Monterey Institute (http://www.montereyinstitute.org/nroc/) The Center for Learning & Teaching staff can assist in finding online instructional resources such as multimedia repositories that contain multimedia learning objects that are specific to a discipline. When considering incorporation of a multimedia learning object from an external source, it is important to carefully review these materials before making use of them. Just because an item is provided by the publisher or a reliable online course website does not guarantee its quality. Also, all multimedia learning objects should promote and support the achievement of the stated learning objectives. When selecting visual and interactive elements, be sure that the selected media supports the learning objectives. In addition, do not use technology just for the sake of using technology. The best applications of media utilize media elements that contribute to the course outcomes and learning objectives, as well as, enhance interaction with course information in a way that would not have otherwise been possible. Rev. 2/4/2010 Page 12
Discussion Forums Discussion forums are an important component of hybrid courses. Discussion forums are an avenue to provide student to student interaction, outside of the face to face classes, that helps build a sense of community in a hybrid course. They help make up for the reduced face time that occurs naturally with students in a traditional classroom setting. The instructor can post various topics, and students reply to and converse about the topics online. This is similar to a chat room but is designed for asynchronous use, meaning students do not have to be online at the same time to have a conversation. An additional advantage of the discussion forum is that student conversations are logged and organized. When designing discussion forums, some instructors ask their students to first make one original post and then to respond to at least one other student s posting. Instructors may require students to meet this expectation in order to receive full credit for the assignment. This helps ensure that conversations are held, rather than a series of individuals responding to the original question. In addition, some instructors provide a grading rubric and examples of good discussion forum postings to ensure that students comprehend what is expected of them within a discussion forum. What types of topics are appropriate for a discussion forum? The first discussion forum in most hybrid courses consists of introductions. This gives students a nonthreatening way to become familiar with using the forum. The instructor usually begins with a selfintroduction and asks students to tell the class something about their background, why they are taking the class, whether or not this is their first online course, etc. Another popular use of discussion forums is to create an avenue to post questions and answers. For simplification/clarity: If an instructor receives a question from a student, they might consider answering the question in a class forum. This allows the entire class to benefit from the explanation/clarification, allows the answer to always be readily available as a reference, and duplicate answers to multiple students are eliminated. Many instructors set up a specific forum for questions and answers. Others ask their students to use the current week's forum to post questions. Along the same lines, some instructors have a class café discussion forum. This is a communication area where students can discuss topics that are not related to the course material, which provides a venue for outside of class chatter among students. An additional idea is to create one forum in which students are asked to find a website related to that week's class topic and to post a critique of the site. Formal discussion topics should allow students to share or exchange ideas. Usually, there is no right or wrong answer. Each student should be able to provide a slightly different perspective on the question. If a question has a definite answer, it generally does not make a good discussion topic. Everyone would post the same answer, or the first person would post the answer and everyone else would agree. Many good suggestions for discussion topics can be found in the textbook or its accompanying instructor's resource guide. Rev. 2/4/2010 Page 13
Homework Assignments Homework and practice are a vital part of any hybrid course. In addition to regular structured activities, most successful courses also contain more substantial student assignments such as research papers, collaborative group projects, and portfolio development. It is advisable to make homework mandatory and count homework as a portion of the students final grade. By requiring and grading homework, there is another opportunity to interact with students, helping to take some of the distance out of the online aspect of hybrid learning. Hybrid courses at Penn State are not self paced and each online class should have a definite start and end date. Assignments should be due throughout the semester and not all at once at the end of the semester. Address this with students via an introductory statement that explains the course calendar. This statement can be posted within the course syllabus or within an introductory or welcome area of the course. Quizzes/Exams Quizzes/Exams are an integral part of any teaching and learning process because they are one way to assess if a student successfully accomplishes the learning objectives. ANGEL has an online assessment feature for delivering and automatically grading objective type (multiple choice, true/false, matching, order, and fill inthe blank) quizzes and exams. Short answer and essay type questions can also be included, but must be manually graded. ANGEL provides many options to create exams and quizzes, including creating tests directly using ANGEL s Quiz tool, importing questions and exams/quizzes from a publisher s question bank, and importing questions from a Microsoft Word file. ANGEL can randomly select a specific number of questions from a pool of questions, which allows each student to receive a slightly different test. Not only can the order the questions are delivered be randomized, the order of answers for a specific question can be randomized. It is important to stress that online testing is not always secure. Even though there are some controls that can be used to inhibit the sharing of questions and answers, without the existence of a controlled access, monitored testing environment, online tests should not be used for high stakes assignments. Examples of assignments using online quizzes could include (from ANGEL help documentation) Practice quizzes on basic terms, concepts, and principles. Quizzes over homework problems, chapter readings, or expectations (have them respond to quiz questions with a due date a few hours before class time). Quizzes to gauge preconceptions before a topic is covered. In general, do not rely solely on online tests to evaluate student progress. Instead, utilize face to face tests and include other means of assessments such as writing assignments and portfolio development. Rev. 2/4/2010 Page 14
Appendix II: Multimedia Instructional Elements In some courses, multimedia instructional elements may be part of the hybrid course development model process. If the instructor requests that multimedia instructional elements be included, they will be designed and developed independently of the hybrid course development model timeline. In some situations, this means that the process for the creation of the multimedia instructional elements can be started prior to the hybrid course development model timeline. The multimedia instructional elements may include animations, videos, interactive games, online tutorials, etc. For examples of multimedia instructional elements, please visit the Samples of our Work portion of the CLT website. Depending upon the content and complexity of each multimedia instructional element, the design and development can take as little as a few weeks or as long as a few months. The time required is determined by the choice of development tools, the amount of content covered, and the desired degree of interaction. Rev. 2/4/2010 Page 15
Bibliography Glossary of Terms. September 2009 <http://www.usdla.org/glossary_distance.pdf>. Web Learning @ Penn State Glossary. October 2009 <http://weblearning.psu.edu/glossary>. Rev. 2/4/2010 Page 16