Intentional Design of an Online Graduate Course Using Merrill s First Principles: A Case in Progress

Transcription

1 Intentional Design of an Online Graduate Course Using Merrill s First Principles: A Case in Progress Martha M. Snyder, Ph.D. Nova Southeastern University Graduate School of Computer and Information Sciences Carl DeSantis Building, Fourth Floor 3301 College Avenue Fort Lauderdale, Florida Descriptors: first principles of instruction, design case Background There is a growing appreciation in the research literature for studies that address what instructional designers do in everyday life and how they do it. For example, Visscher-Voerman and Gustafson (2004) studied the design strategies of professional instructional designers to determine how their activities were similar and different from ADDIE, and instructional design process model. Williams, South, Yanchar, Wilson, and Allen (2011) investigated how instructional designers use evaluation to improve their instructional products. Williams et al. commend studies that have given a clearer picture of how instructional designers do their work. They suggest these studies shed light on lessons learned from practice and note how important this information is to improve the scholarly research in this field as well as how we educate and train instructional designers. However, sometimes lessons learned, principles, or heuristics resulting from such studies lack nuances of design that could be beneficial to other designers. Boling (2011) offers a different perspective to disseminating design knowledge specifically, how designers share precedent through design cases (Boling, 2011). Precedent refers to the experiential or episodic knowledge that is often not explicit but held in minds rather than recorded in an organized manner and is accumulated through practice rather than through instruction (Lawson, 2004, p. 453). Boling differentiates design cases from other forms of research including design and development research (Richey & Klein, 2007), formative research (Reigeluth & Frick, 1999) and design-based research (Design-based Research Collective, 2003). Described as a distinct form of knowledge dissemination, design cases use precedent in a proactive versus reactive synthetic versus linear concrete and situated versus theoretical and generalized and fluid versus fixed manner (Boling, p. 4). In design cases, precedent is not used as explicit guidance but rather as a contribution to that designer s store of patterns, her ability to discern and appreciate the qualities of patterns, and her acts of innovation both in the near and long term (Boling & Howard, 2011). Design cases can be valued for their precedent, utility, and rigor. While arguing there is value in design cases that have utility but do not necessarily have rigor, Smith (2010) suggests the more rigorous the design case, the more likely the case will serve as a useful case in a broader context. Smith suggests a naturalistic paradigm (Lincoln & Guba, 1985) and action research (Argyris, Putnam, & Smith, 1985) serve as good starting points for organizing, conducting, describing, and reporting a design case. Standards of rigor will develop over time through the interplay of those who develop design cases and their peers who review them. A Reflection Paper The aim of the reflection paper was to further explore the nature of design cases and how to establish rigor in such cases. The idea for the presentation was to explore these issues within the context of an instructional design project. This project is the redesign of an online doctoral course. While it is not necessary to base design cases on specific instructional design theories and models (Boling & Howard, 2011), the use of first principles of instruction (Merrill, 2002) was selected as a framework to redesign the course. The aim of the reflection presentation was not to present the design case (yet) but rather to generate discussion about design cases in general and seek feedback about this particular case. The aim of the session was to discuss: the process and value of producing and publishing design cases specifically, how design cases differ from other research designs in the field of instructional design and development, how to set up a design case, how to establish rigor through the use of existing research paradigms and methods, and ideas for presenting this particular case as a design case. 444

2 The Proposed Design Case Since submission of the conference proceedings preceded the reflection discussion, the information provided here describes the case and presents initial design decisions pertaining to each of the five principles. Discussion at the conference will further inform how the design of the course progresses and is presented as a design case. I teach a directed research course in instructional design and development as part of a doctoral computing technology in education curriculum. Our doctoral program is a limited-residency program where students come to campus twice per year for an extended weekend during coursework. Once students complete their 700-level coursework, they are not required to return to the physical campus. Therefore, all of the student-to-faculty, studentto-student, and student-to-resource interaction happens virtually via Blackboard (including , web conferencing, and discussion forums) and either phone or Skype conversations. The program s curriculum includes 700-level courses in assistive technology, human-computer interaction, instruction delivery systems, instructional design and development, knowledge management, learning theory, online learning environments, online program administration, telecommunications, and qualitative and quantitative research methods. Students are required to take 64 credit hours, of which 40 are for courses and 24 are for the dissertation. Of the 40 credit hours in coursework, students are required to take two sections of an 800-level directed research course (4 credits each) in their subject area of interest (e.g., assistive technology, human-computer interaction, instructional delivery systems, instructional design and development) prior to entering candidacy. The directed research course is intended to help students focus on a specific research problem within a specific content area that a faculty member can support. With the professor s permission, students can take both 800-level courses with the same professor; however, many students prefer to take their two 800-level courses with two different professors. Problem During the first few years of graduate programs, students are busy learning about particular concepts, principles, theories, practices, and techniques their chosen field of study (Lovitts, 2007). Boote and Beile (2005) emphasize the need to develop skills needed to analyze and synthesize research in a field of specialization is crucial to understanding educational ideas and Such preparation is prerequisite to choosing a productive dissertation topic and appropriating fruitful methods of data collection and analysis (p. 3). Unfortunately, many students enter the dissertation phase of their studies unprepared for the transition from course-taker (a consumer of knowledge) to independent researcher (a producer of knowledge) (Lovitts, 2005). In the past, the 800-level courses were primarily project courses that extended the work of the 700-level courses and did not necessarily help students become independent researchers. For example, the 700-level instructional design course might have the students design and develop an instructional product and the 800-level course (now the directed research course) would be a continuation of that design with the implementation and evaluation of the product. While the school s faculty has decided to make the 800-level courses research-focused with the goal of helping students transition from coursework to dissertation, and they have subsequently changed the nature of their 800-level courses to reflect that shift, faculty have autonomy in course design and delivery. That is, there is no standard 800-level course template. Purpose The purpose of this design case will be to describe how I redesigned the course and the decisions I made along the way as I attempted to apply first principles of instruction (Merrill, 2002a) as a guiding framework. First principles describe a cycle of instructional phases consisting of activation, demonstration, application, and integration all in the context of real world problems or tasks (Merrill, 2007, p. 6) and offer a structured and prescriptive approach toward instruction. This first phase of the redesign is to think about how first principles could be applied to facilitate instruction towards achieving the course s learning outcomes. As a result of the course redesign, it is anticipated students will be more prepared to conduct independent research; the instructor will have clearer direction about specific instructional activities that can guide students in achieving the learning outcomes; and the instructional designer will have a basis for evaluating the course design and instructor implementation of this course. It is also hoped that once the design case is completed, readers can gain insight into how they might design similar courses. 445

3 Current State Currently, the goal of my directed research course is for students to conduct a review of the literature about a topic of interest in the field of instructional design. There are three major assignments including: 1) literature review topic proposal, 2) annotated bibliography, and 3) literature review. The course is designed primarily as an independent study and there is minimal guided instruction. Detailed assignment guidelines are provided, students are expected to complete the assignments independently, and I provide copious feedback once the assignment has been submitted. Students have one chance to submit the assignment for a grade. After teaching a few iterations of the course in this manner, I have found that students need more guidance and formative feedback on the front-end of the research process particularly with the identification of a research-worthy problem and how to use that problem as the basis of a dissertation idea paper. While students study research problems, questions, and methods in their research methods courses, these elements are included as part of a larger focus on quantitative or qualitative analysis and reporting. Also, depending on when during the program the student decides to take their methods courses, these front-end research components are often studied out of the context of the students specific research interests. Students would benefit from thinking about research problems, goals, questions, and methods within the context of an authentic task that could potentially become their dissertation research. In addition, implementing structured and guided instruction with more frequent, yet smaller assignments, would allow more opportunities for practice and formative feedback throughout the 16-week online course. Kirschner, Sweller, and Clark (2010) recommend strong and direct guidance for both novice and intermediate learners and suggest a minimally-guided approach may be less effective and even produce negative results if students acquire misconceptions or incomplete disorganized knowledge (p. 84). Desired State Rather than focusing on a review of the literature as the final deliverable of the course, I want to design this course so that there is more focus on the beginning stages of a dissertation study such as identifying a research problem, goal, questions, and approach. That is, I want students work in this course to align more closely to what is expected during the initial stages of the dissertation. Reviewing the literature remains a critical component of the course; however, the focus of the instruction is less on the literature review process and more on the front-end components of an independent research study and the process of writing an idea paper. It is hoped that by aligning this course with the expectations of the first phase of the dissertation process (i.e., development of the idea paper), students will develop their research knowledge and skills in the context of an authentic task, gain a clearer understanding of the expectations of a dissertation, and potentially be able to use the draft idea paper they prepared in the course as a starting point for their dissertation study. Merrill s First Principles Reigeluth and Carr-Chellman (2009) suggest that research in instructional theory has reached the stage where it is time to develop a common knowledge base. They argue that developing consistent terminology, for example, would encourage and advance future research in this area. Specific to developing a common knowledge base related to instructional-design theories and models, Merrill (2002a) set out to identify a set of common principles about instruction. Specifically, he proposes a set of interrelated instructional design principles referred to as First Principles of Instruction and suggests that the effectiveness, efficiency, and engagement of a particular model or method of instruction is a function of the degree to which these principles are implemented (Merrill, in Reigeluth & Carr-Chellman, p. 43). To that end, he challenges researchers to verify these principles in a wide variety of settings, for a wide variety of different audiences, in other cultures, and across subject-matter domains (p. 56). In an attempt to verify these principles in the context of an online graduate research course, I am using Merrill s (2002a) first principles of instruction to guide the redesign of this course. Merrill (2002b) defines a first principle of instruction as a prescriptive design principle on which various instructional design theories and models are in essential agreement (p. 42). These principles are underlying, basic methods that support more specific instructional activities. Merrill suggests learning is promoted when: 1. Learners engage in solving real-world problems 2. Existing knowledge is activated as the basis for new knowledge 3. A demonstration of the new knowledge is given 4. The learner applies the new knowledge 5. The learner integrates knowledge into his or her world 446

4 Evaluation of Course Design and Teaching and Learning Quality Frick, Chadha, Watson, Green, and Zlatkovska (2009) address the first principles from a student evaluation perspective. They conducted an empirical study to verify these principles by asking students about their learning experiences in undergraduate, face-to-face, college courses that they had taken or were about to complete. That is, Frick et al. (2009) focused on students perceptions of teaching and learning quality (ends) rather than the design process (means). In a subsequent study, Frick, Chadha, Watson, Green, and Zlatkovska (2010), propose an instrument to measure teaching and learning quality (TALQ). A component of the instrument is a measurement of evidence of first principles. While the results of these studies provide guidance to instructors for measuring the principles in their online courses, the authors suggest further research is needed to determine whether 1) students learn better or learn more when instructors use first principles (Frick, et al., 2009) and 2) the TALQ instrument is useful in providing feedback to instructors regarding how effectively they designed and implemented their courses according to the first principles (Frick, et al., 2010). In order to determine whether students learn better or learn more, or whether courses have been effectively designed according to the first principles in a fully online learning environment, it is important to understand how to design a course that effectively incorporates these first principles in this setting. First Principles Applied to an Online Directed Research in Instructional Design and Development Course Description Students will work independently and under the guidance of the professor to produce a complete draft of an idea paper. The idea paper is the first dissertation deliverable and includes the following major components: background/introduction, problem statement, dissertation goal, research questions, relevance and significance, barriers and issues, brief review of literature, approach, milestones, resources, and references. Through direct instruction, activities, assignments, and practice, students will hone their research focus in the field of instructional design and development and gain knowledge and skills related to the idea paper process and product. Course Goal and Objectives The goal is to identify and develop the fundamental components of a dissertation idea/concept paper. Students will: Identify a research topic in instructional design and define its component issues. Identify a research-worthy problem within the selected topic area. Define the goal of the research study and formulate research questions. Describe the relevance and significance and barriers and issues relating to the proposed research. Analyze, synthesize, and evaluate the research literature and present a brief review of the literature that supports the research problem and goal. Formulate a research approach that is suitable for the proposed research. Outline milestones and resources for the proposed research. In the following sections, each of the five principles and its corollaries are briefly described along with guiding questions for course design and an initial exploration of how the principles could be used to guide the redesign of the directed research course. Principle 1 - Problem-Centered (corollaries: show task, task level, and problem progression) Merrill (2007) uses the term, problem, to represent a real-world whole task as opposed to various parts of a task that are similar to what one might encounter in the real-world. He contrasts problem-centered (aka taskcentered) instruction with topic-centered instruction where problem-centered instruction addresses the whole task and topic-centered instruction addresses components of a task and each task is taught separately. Guiding questions pertaining to course design include: What is the whole task or problem? What are the inputs, goal, and solution for the whole task and how can the whole task be represented as a problem-solving process within the course? Corollary Show Task: Merrill (2002a) suggests that instead of using objectives to gain attention and introduce the instruction, a better orientation to the instruction includes a demonstration of the whole task that is similar to the one being taught. This demonstration would preferably show something similar to what the learners would be able to do after they complete the instruction. In this instance, the goal of the course is for students to write a complete rough draft of a dissertation idea paper. At the beginning of the course, various exemplar research papers in the field of instructional design will be presented. (Since it is not customary for the school to show students examples of other students idea papers, 447

5 published research papers or dissertations will be selected instead.) Each example will highlight components of the study that are included in the idea paper (i.e., background, research problem, goal, research questions, and approach). It is important to select exemplars that represent the broad spectrum of research being done in this field (e.g., quantitative, qualitative, mixed-method, design-based, formative, and evaluative studies). Showing these various examples would emphasize the various ways to address research problems in the field of instructional design. Corollary Task Level: Merrill (2002a) identifies four levels on which the problem should be addressed including: (a) the problem, (b) the tasks required to solve the problem, (c) the operations that comprise the tasks, and (d) the actions that comprise the operations (p. 46). In this instance, the problem or whole task is producing a dissertation idea paper. The tasks include: identifying a research-worthy problem, goal, and research questions; describing the relevance, significance, barriers, and issues; conducting a brief review of the literature, and outlining an initial approach (i.e., research method). The operations and actions for completing these tasks are taught in the 700-level research methods coursework are will be reviewed as part of this course. For the first and second assignment, students will be asked to identify two recent scholarly research articles in the field of instructional design and using a rubric similar to the CASP tool used to appraise qualitative research (Public Health Resource Unit 2006, see: appraise the articles by identifying the various components of the idea paper (e.g., problem statement, goal, research questions, etc.). Specific questions in the rubric will assist the learner in identifying strengths and weaknesses of each of these components. We will focus only from the problem identification to the methodology (as opposed to evaluating the entire research article) since our aim is to help students write the idea paper, which includes from problem identification through a brief outline of the approach. Corollary Problem Progression: Merrill (2002a) advises some problems can be quite complex. For example, writing an idea paper is considered by many dissertation students one of the most difficult hurdles to overcome. Merrill suggests providing simple problems at the beginning and once those problems are mastered, moving to more complex problems. It is difficult to conceptualize an instantiation of problem progression for this course because there is no simple representation of an idea paper (i.e., the whole task). Therefore, the following modification is provided to address the idea of presenting assignments that progress from simple to more complex; however, each does not represent the whole task but rather offers a scaffold to help the student master the whole task. For example, students are presented with exemplar idea papers and are shown components of the papers that make them good. Next, students are instructed to identify two recent scholarly research articles and appraise them on the quality and rigor of the study. Following these two appraisals, students will be instructed to produce a 5-7 page abridged idea paper as the third assignment. This paper includes the following components: Title page (include your name, and working title of your proposed research) (1 page) Relevant expertise and experience (brief overview of your relevant expertise and experience in the selected research area) (1/2 page) Research problem (1-2 pages) Research questions (1/2 page) Initial concept for solving the problem (1-2 pages) References (APA format) (1 page) Students are given formative feedback on viability of their research problem and proposed solution. Giving feedback on this abridged version of the idea paper will allow the professor to provide feedback earlier in the process and provide remediation, if necessary before the student spends too much time on an idea that is not viable. Students whose research ideas are not viable continue to work on this assignment and receive feedback until a viable research idea is presented. The question remains as to whether to grade this particular aspect of the course (i.e., summative vs. formative feedback). Perhaps the student is give three chances with formative feedback to produce a viable idea before a summative grade is assigned. Another option is to assign a pass/fail grade for this course instead of grading each assignment. Principle 2 - Activation (corollaries: previous experience, new experience, and structure) Activation pertains to the idea that rather than jump right into instruction, it is important to prime the student by anchoring the instruction to previous experiences, making the instruction relevant so that it can be used to build new knowledge, and providing a schema or asking students to recall a schema for them to organize new knowledge (Merrill, 2002a). Guiding questions include: What are some ways to activate relevant previous knowledge pertaining to the whole task of writing an idea paper? 448

6 In this instance, students can employ the same rubric they used to appraise research articles to appraise their components of their idea paper. In order to help students and provide structure as students prepare their abridged version of the idea paper, a worksheet can be used to guide them through the process (Ellis, 2006, see for an example.). Principle 3 Demonstration (corollaries: demonstration consistency, learner guidance, and relevant media) Merrill (2002a) stresses the importance of not only presenting information (e.g., These are the components that need to be in an idea paper. ) but also portraying the information to be learned (e.g., Here is an example of a complete idea paper that includes the various components. ). Portraying information means the information is demonstrated as a part of a specific situation or case. Specific portrayals of information help the learner to remember and apply the information. Guiding questions include: What strategies can be employed to demonstrate effectively the whole task? What are some ways to guide students? What media can be incorporated that is relevant to the course? Again, in this case, it would be worthwhile to show examples of completed idea papers. If these examples are unavailable, an alternative might be to show exemplar articles and identify the various components that would be included in an idea paper. Corollary Demonstration Consistency - The corollary, demonstration consistency, refers to the need to ensure that the instructional strategy is consistent with the content category and learning outcome. Various content categories include facts, concepts, principles, rules, and procedures (Morrison, Ross, Kalman, & Kemp, 2011). For example, if one is teaching the concept of a problem statement, a definition is provided followed by a best example of a problem statement. To help students identify problem statements, examples and non-examples could be provided and students could differentiate between good and bad problem statements within the context of their research topic. Once they are able to identify a problem statement, they could be given the opportunity to practice writing their own problem statements. Perhaps this is an aspect of the course that could be integrated into an online discussion forum and not necessarily part of a course assignment. Corollary Learner Guidance - Merrill (2002a) advises that learner guidance should be provided early an often especially at the early stages of instruction. Explicit guidance focuses the learner s attention on the relevant information to be learned. In this course, formative feedback is provided on every assignment via Microsoft Word s revision marking mode. Comments are provided to guide the student as the write their idea paper. For example, students who miss the mark in identification of a research-worthy problem are given formative feedback to guide them in the appropriate direction. This feedback might include guiding questions, comments, and references to additional information that will help them improve their work. Providing multiple representations of the content being taught along with demonstrations of the content is important. Multiple representations of ill-defined problems (e.g., providing various types of approaches to solving research problems in the field of instructional design) will help the learner realize that there are many ways to solving research problems. These multiple representations will result in the learners having a broader perspective of how they might solve their specific research problem. Corollary Relevant Media - Merrill (2002a) advises, Learning is promoted when media play a relevant instructional role and multiple forms of media do not compete for the attention of the learner (p. 48). In this instance of an online course, the incorporation of relevant media is critical. Relevant videos, for example, can support the various components of the idea paper from writing the research problem to conducting the review of the literature. These videos will be incorporated into the course as supplemental reference material for the students. Additional media including audio and video feedback can be embedded into student assignments to provide clarification over the traditional text-based feedback. Principle 4 Application (corollaries: practice consistency, diminishing coaching, varied problems) This principle refers to the idea that learning is promoted when students are required to apply what they have learned. Guiding questions include: What types of assignments can be given to students that enable them to practice the whole task? What feedback strategies can be used (e.g., rubrics, in-text comments, audio feedback, video feedback)? The corollary, practice consistency pertains to the idea that learning is promoted when the application of the content is consistent with the stated learning objective (Merrill, 2002a). In this instance, one of the learning objectives is: identify a research topic in the field of instructional design and development. Following a presentation about research in instructional design and development, learners apply what they learned by writing a paper that includes the identification of a research topic that is relevant and timely. Two additional corollaries, diminishing coaching and varied problems are also described. Merrill (2002a) suggests learning is promoted when appropriate guidance and feedback is provided up front and gradually withdrawn. Learning is also promoted when a sequence of 449

7 varied problems is presented and learners are required to solve them. In this instance, this corollary can be represented through the appraisal of a variety of research articles in the field of instructional design. Principle 5 Integration (corollaries: watch me, reflection, and creation) Learners integrate instruction when they can show evidence that they have gained new knowledge and skills. This evidence can be demonstrated when learners demonstrate improvement of a skill or can show how their new knowledge is used in the real world (Merrill, 2002a). Guiding questions include: How can we incorporate opportunities for learners to publicly demonstrate their new knowledge and reflect on their learning? In this instance, students submit a complete first draft of an idea paper at the conclusion of the course. Students receive feedback based on a rubric that is used for evaluating dissertation idea papers. Following the course, students can be invited to present their research idea, defend their newly acquired knowledge, and receive feedback from faculty and peers at an on-site or virtual poster session. Next Steps The purpose of the reflections presentation was two-fold: to explore the nature of design cases and how to establish rigor in such cases and to describe and reflect on the design of an online graduate research course. This first phase involved thinking thorough how Merrill s (2002a) first principles of instruction might be used to guide the design of the course. The initial ideas are presented here. The next steps include: 1) completing the course redesign and development, 2) implementing the design 3) reporting the instantiation of the design including important design decisions, mistakes made throughout the design process, and vicarious experiences about the design and implementation process, and 4) evaluating the results of the course design and teaching and learning quality. Frick, et al. s (2010) TALQ instrument will be used in two ways: first, to determine whether students learn more or better as a result of the implementation of first principles and second, to determine whether the instrument is useful in providing feedback regarding how effectively the course was designed and implemented according to the first principles. References Argyris, C. Putnam, R. & Smith, D Action science: Concepts, methods and skills for research and intervention. San Francisco: Jossey-Bass. Boling, E. (2010). The need for design cases: Disseminating design knowledge. International Journal of Designs for Learning, 1, 2-8. Boling, E. & Howard, C. (Producers). (2011, October 6). Creating and dissemination design knowledge. [Webinar]. Graduate Student Assembly of AECT in partnership with the Design and Development Division. Retrieved from: Boote, D. & Beile, P. (2005). Scholars before researchers: On the centrality of the dissertation literature review in research preparation. Educational Researcher, 34(6), Design-Based Research Collective. (2003). Design-based research: An emerging paradigm for educational inquiry. Educational Researcher, 32(1), 5 8. Ellis, T. (2006). Some idea paper ideas. Retrieved from Frick, T.W., Chadha, R., Watson, C., Wang, Y., & Green, P. (2009). College student perceptions of teaching and learning quality. Educational Technology Research and Development, 57, Frick, T., Chadha, R., Watson and Zlatkovska, E. (2010). Improving course evaluations to imporive instruction and complex learning in higher education. Educational Technology Research and Development, 58(2), Kirschner, P.A., Sweller, J., & Clark, R.E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry-based teaching. Educational Psychologist, 41(2), Lawson, B. (2004). Schemata, gambits, and precedent: Some factors in design expertise. Design Studies, 25, doi: /j-destud Lincoln, Y. & Guba, E. (1985). Naturalistic inquiry. Thousand Oaks, CA: Sage Publications. Lovitts, B.E. (2007). Making the implicit explicit: Creating performance expectations for the dissertation. Sterling, VA: Stylus Publishing, LLC. ISBN:

8 Merrill, M.D. (2009). First principles of instruction. In Reigeluth, C.M., & Carr-Chellman, A.A. (Eds.), Instructional-design theories and models: Building a common knowledge base, volume III. (pp ). New York, NY: Routledge. Merrill, M.D. (2007). A task-centered instructional strategy. Journal of Research on Technology in Education, 40(1), Merrill, M.D. (2002a). First principles of instruction. Educational Technology Research and Development, 50(3), doi: /BF Merrill, M.D. (2002b). A pebble-in-the-pond model for instructional design. Performance Improvement, 41(7), Morrison, Ross, Kalman, & Kemp (2011). Designing effective instruction. (6 th ed.). Hoboken, NJ: John Wiley & Sons, Inc. Public Health Resource Unit (2006). Critical Appraisal Skills Programme (CASP). Making sense of evidence. 10 questions to help you make sense of qualitative research. Retrieved from Reigeluth, C.M. & Carr-Chellman, A.A. (Eds.). (2009). Instructional-design theories and models: Building a common knowledge base (Vol. III). New York, NY: Routledge. Reigeluth, C.M. & Frick, T.W. (1999). Formative research: A methodology for creating and improving design theories. In Reigeluth, C. (Ed.), Instructional-design theories and models: A new paradigm of instructional theory, volume II. (pp ). Mahwah, NJ: Lawrence Erlbaum Associates, Inc. ISBN Richey, R. C. & Klein, J.D., (2007). Design and development research. Mahwah, N.J.: Lawrence Erlbaum Associates, Inc. Howard, C. (2011). Writing and rewriting the instructional design case: A view from two sides. International Journal of Designs for Learning, 2, Smith, K. (2010). Producing the rigorous design case. International Journal of Designs for Learning, 1, Visscher-Voerman, I., & Gustafson (2004). Paradigms in the theory and practice of education and training design. Educational Technology Research & Development, 52(2), Williams, D. D., South, J.B., Yanchar, S. C., Wilson, B.G., & Allen, S. (2011). How do instructional designers evaluate? A qualitative study of evaluation in practice. Educational Technology Research and Development. Advance online publication. doi: /s