Teaching Instructional Design Expertise: Strategies to Support Students Problem-Finding Skills

Transcription

1 Tech., Inst., Cognition and Learning, Vol. 7, pp Reprints available directly from the publisher Photocopying permitted by license only 2009 Old City Publishing, Inc. Published by license under the OCP Science imprint, a member of the Old City Publishing Group Teaching Instructional Design Expertise: Strategies to Support Students Problem-Finding Skills Donald A. Stepich 1, * and Peggy A. Ertmer 2 1 Boise State University 1910 University Dr., ET 323 Boise, ID Fax: Phone: Purdue University 3144 Beering Hall of Liberal Arts and Education 100 N. University St. West Lafayette, IN Fax: Phone: pertmer@purdue.edu A common problem within professional education is the acquisition of conceptual knowledge that is essentially inert (Bransford, 1993, p. 174). That is, while students often graduate from educational programs with substantial conceptual knowledge, they are unable to apply that knowledge to solve the kinds of ill-structured problems they encounter in practice. Based on the premise that the purpose of professional education is to help students develop the ability to solve complex, messy problems, we have decomposed expert problem solving into identifiable components and have used that model to investigate problem finding among experienced instructional designers. In this article we describe the results of three related studies that examined problem finding among instructional designers. Based on the results of those studies, we then describe three strategies that professional educators can use to support the development of problem-finding expertise in their students. Keywords: Expertise, instructional design, instructional design education, problemsolving Education within the professions isn t always as effective as we would like. For example, Mills and Treagust (2003) noted that graduates from engineering schools have good knowledge of fundamental engineering science but they don t know how to apply that in practice (p. 3). Similarly, Dahlgren and Pramling (1985) noted a need for physicians to reorganize what they learned *Corresponding author: dstepich@boisestate.edu pp TICL_005_Stepich.indd /23/2009 3:18:19 PM

2 148 Stepich and Ertmer in medical school from a focus on content to a focus on common clinical problems. Similar difficulties have been described within the field of instructional design (ID). For example, Julian, Kinzie, and Larsen (2000) noted that students often graduate from instructional design programs with an understanding of the ID process but without the knowledge base that can help them solve instructional design problems (p. 165). The consistent theme in these reports is the inert knowledge problem (Bransford, 1993, p. 174, paraphrasing Whitehead), which refers to graduates who have acquired domain knowledge but who are not adept at applying their knowledge to the solution of common problems. Graduates might perform reasonably well in situations that are familiar and relatively simple. But their performances quickly deteriorate as situations become more complex. According to Bereiter and Scardamalia (1993), the reason for this deterioration in performance is that students (novices) lack the kind of fluid expertise (p. 36) that would enable them to apply their knowledge in unfamiliar situations. Instead, they rely on a standard recipe which they are unable to adapt to fit the requirements of a new situation. Quinn (1994) suggested a possible reason for this limitation in professional education. He described an emerging epistemology of practice (p. 71) in which conceptual knowledge within a domain is necessary but not, by itself, sufficient for competent professional practice. That is, in addition to conceptual domain knowledge, individuals must acquire practical skills in order to function as competent professionals. Unfortunately, many professional educators tend to over-emphasize conceptual knowledge at the expense of practical skills, with the result that new graduates are not fully prepared for the demands of professional practice. According to Quinn, this emphasis on conceptual knowledge exists because most professional education is based on the concept of technical rationality (Schon, 1987, pp. 3 4), which incorrectly assumes that professional activity is made up of the relatively straightforward application of conceptual knowledge to the solution of relatively well-structured problems. More realistically, professionals are likely to encounter ill-structured problems made up of incomplete data and competing (often conflicting) demands (Mills & Treagust, 2003, p. 2). Instructional design educators have long recognized this problem and have worked to integrate the development of practical skills with conceptual knowledge in a variety of ways. For example, Quinn (1994) developed an ID course in which small teams of students worked under the supervision of an instructor to design instruction for a real client. Rowland, Para, and Basnet (1995) created a pp TICL_005_Stepich.indd /23/2009 3:18:19 PM

3 Teaching Instructional Design Expertise 149 design studio (p. 231) in which students worked collaboratively to solve ID problems that gradually increased in complexity. Jonassen and Hernandez- Serrano (2002) provided students with stories that had been elicited from experienced instructional designers as a way to help them gain conditionalized knowledge (Bransford, Brown, & Cocking, 2000, p. 43). In each case, students learning was placed in context. That is, students were asked to apply their emerging knowledge of instructional design within the context of real-world situations, with their inherent messiness left intact. This allowed students to develop their practical skills, over time, by working on realistic, complex problems. Like these authors, we re interested in finding ways to improve ID education via learning in context, mostly through the use of case studies (Ertmer & Quinn, 2007; Ertmer & Russell, 1995; Stepich, Ertmer, & Lane, 2001). In 2005 we proposed a conceptual model, shown in Figure 1, with the goal of decomposing expert thinking into identifiable components (Ertmer & Stepich, 2005). The model, based on available expert-novice literature, divides expert thinking into two major components, problem finding and problem solving, and identifies several interrelated sub-components. Synthesizing Underlying Principles Problem Finding Relationships Reflective Problem Solving Implications Flexible Figure 1 Conceptual model of instructional design expertise (Ertmer & Stepich, 2005) Used with permission pp TICL_005_Stepich.indd /23/2009 3:18:27 PM

4 150 Stepich and Ertmer Problem Finding articulating a clear and concise representation of the problem(s) in a particular situation Synthesizing combining the information available about the problem situation with existing knowledge and experience to articulate a coherent representation of the situation in the form of one or two central issues. Focus on principles interpreting the available information and describing the issues in terms of abstract conceptual principles that underlie the given situation. Identifying relationships among issues developing a coherent picture of the situation in which the identified issues are linked together in terms of their interrelationships and/or operational priorities. Reflection focusing on what is known and making inferences based on given information. Problem Solving developing a clear and relevant solution plan that explicitly describes how the proposed solutions address the found problem(s). Identifying relationships among solutions developing a coherent solution plan, in which suggested solutions are linked back to the identified issues as well as woven together in terms of their interrelationships. Consideration of implications thinking through solution ideas, explicitly considering how those ideas might be implemented and/or what effects they might have. Flexible problem solving presenting solutions early in the problem-solving process, but modifying or eliminating them as additional information becomes available. Using this model as a framework of teachable skills, our overarching goal is to find ways to help our students develop their abilities to think more like experts. This is based on the assumption that the purpose of professional education is to accelerate this development and to provide students with appropriate scaffolding at different points along the developmental continuum (Hardre, Ge, & Thomas, 2005). Our previous research efforts in this area have focused primarily on the problem-finding efforts of instructional designers, as problem finding is considered by many to be the linchpin of the problem-solving process. This was, perhaps, best pp TICL_005_Stepich.indd /23/2009 3:18:27 PM

5 Teaching Instructional Design Expertise 151 captured by John Dewey (1933) who stated a question well put is half answered (p. 108). The Encyclopedia Britannica Online (2008) describes problem finding as the process of identifying problems that are worth solving in the first place. Jonassen (2004) emphasized the importance of problem finding, stating that the key to problem solving is how problem solvers represent or frame the problems to themselves (p. 59). Schon (1983, 1987) referred to this as the problem of problem setting noting that problems do not present themselves as givens (1983, p. 40). He explained: the problems of real-world practice do not present themselves to practitioners as well-formed structures. Indeed they tend not to present themselves as problems at all but as messy, indeterminate situations. Through complementary acts of naming and framing, the practitioner selects things for attention and organizes them, guided by an appreciation of the situation that gives it coherence, and sets a direction for action (1987, p. 4). For the past several years, we have focused both our teaching and research efforts on finding and implementing effective strategies to facilitate and support successful problem finding among our students. The overarching purposes of this paper, then, are 1) to review what we have learned through our previous research and 2) to describe strategies that might be used to apply those lessons to the practical task of teaching problem-finding within the context of an instructional design course. It is important to note that while our earlier work focused on developing a conceptual model of ID expertise, our research efforts have been directed toward verifying that model through investigation of the practice of experts. We begin by briefly reviewing three studies that have been foundational to this work. What Did Our Early Studies Tell Us About ID Expertise? Study 1 (Ertmer et al., PIQ, 2008) As noted earlier, our conceptualization of ID expertise outlined in Educational Technology (Ertmer & Stepich, 2005; see Figure 1) was based primarily on descriptions of expert-novice differences in a variety of domains. However, this model had not yet been validated empirically in the ID domain. Although our previous work (Stepich et al., 2001) had provided insights into novices approaches to solving ill-structured problems, we lacked comparable data from experts. Thus, our first research study was designed to fill this gap. Specifically we asked, Do ID experts synthesize the issues presented in an ill-structured case narrative (as pp TICL_005_Stepich.indd /23/2009 3:18:27 PM

6 152 Stepich and Ertmer suggested by our conceptual model)? Furthermore, in order to more fully understand the process by which experts find the problem in an ill-structured problem scenario, we asked, How do experts use their abstract knowledge and practical experiences to analyze a case problem? Seven expert instructional designers (M = 20.5 years of ID experience; ranging from 8 to 32) were purposively selected to participate in a think-aloud procedure in which they read and reflected on a written ID case study (Hooper & Doering, 2007), followed by a retrospective interview. While previous work with ID experts had typically engaged practitioners in solving a specific ID problem (e.g., evaluating instructional materials [LeMaistre, 1998]; designing instructional materials to address a given problem [Rowland, 1992]), few researchers had examined how experts analyzed an ill-structured design problem presented via a case narrative. Since we have, for some time, advocated the use of case studies to promote the development of students ID problem-solving skills, we hoped to understand how experts approached the same task. To begin the think-aloud procedure, participants were instructed to read aloud the case problem and to think aloud as you work on the problem, telling us everything you are thinking Prompts, such as What are you thinking or Please continue, were used only if necessary, after relatively long pauses. Retrospective interviewing occurred immediately after as a way to help the participants recall their thought processes during the think aloud. All sessions were videotaped and transcribed. Transcriptions were examined using a constant comparison method (Strauss & Corbin, 1998), with specific attention given to participants references to prior knowledge and experience. Initial themes were created to reflect each participant s responses; similarities among themes were used to create a set of four assertions that applied across participants. The four assertions captured how ID experts used their knowledge and experience to analyze an ill-structured ID problem: Experts narrowed the problem space, typically by highlighting key factors, based on a personal frame of reference that was used to filter the information in the case. Experts used an amalgamation of knowledge and experience to create their frames of reference. Frames of reference incorporated a mental model of the ID process. Experts came to similar conclusions about how to respond to the problem in the case. In answer to our first research question (Do ID experts synthesize the issues presented in an ill-structured case narrative), we noted that the experts in this pp TICL_005_Stepich.indd /23/2009 3:18:27 PM

7 Teaching Instructional Design Expertise 153 study did synthesize the information in the problem situation. However, synthesizing looked slightly different than we had hypothesized. Our model (Ertmer & Stepich, 2005) suggested that experts would distill the problem situation into one or two central issues. Most of the experts in this study did not identify central issues. Instead, they highlighted key factors in the problem situation that, then, served as the focal point for their problem-finding. Still, the effect was largely the same. Thus, the results of this study verified that experts were able to quickly filter through the details of a situation to narrow the problem space and determine key elements. This had the benefit of focusing the expert s problem-solving efforts (LeMaistre, 1998; Perez, Jacobson, & Emery, 1995: Schon, 1987). Furthermore, this filtering was shaped by a set of idiosyncratic rules of thumb that were based on a combination of knowledge and experience. These results suggested the need to 1) help novices accumulate a variety of ID experiences, either directly or vicariously, as a first step in developing rules of thumb to apply in practice and 2) scaffold novices efforts to conceptualize the issues in more expert-like ways. These two implications, then, led to the questions that framed Studies 2 and 3. Study 2 (Ertmer, York, & Gedik, 2009) After finding in Study 1 that our seven ID experts applied a set of idiosyncratic rules when solving an ill-structured problem, we decided to examine, more closely, the rules that experts use during complex problem solving in order to identify similarities and differences among them. Specifically we asked, What rules of thumb do experienced instructional designers use when solving ID problems? and To what extent do rules of thumb incorporate the use of ID models? By examining the heuristics and models (published or not) that guided the practice of experienced designers, we hoped to provide new insights for novices who, typically, are unsure of how to translate conceptual book knowledge into realworld practice. Sixteen experienced designers (M = 23 years of ID experience, ranging from years) agreed to participate in the study. During semi-structured interviews, lasting between 30 and 90 minutes, the participants shared stories about ID projects in which they had faced challenging or complex problems. Participants stories were recorded and transcribed and then examined for instances in which they referred to general guidelines or principles that framed their decision making. These guidelines were subsequently organized into themes that appeared across participants. Results revealed rules of thumb that, while often individual, also had universal elements. The rules of thumb fell into four themes, specific to the research questions (For more specific examples, see Ertmer et al., 2009.): pp TICL_005_Stepich.indd /23/2009 3:18:27 PM

8 154 Stepich and Ertmer Experts use ID models, though in various ways Application of the ID model does not follow textbook descriptions Experts focus on situational constraints, early and often To deal with current and potential constraints, experts make use of strong communication strategies Stories seemed to be a particularly fruitful method for providing insights into what experienced designers do when faced with complex design problems. Although designers, like other experts, may not always be able to articulate the rules of thumb they use in practice, embedding those rules within a compelling story from practice seemed to occur fairly naturally. At first glance, many of these rules may seem to be little more than common sense. But perhaps that simply speaks to the universal quality of many of them. Still, among these universal rules are a number of unique, perhaps even personal, rules that were most likely formed through specific experiences that individual designers had. Regardless, it is likely that many others (novices and experienced designers alike) can benefit from these stories and the lessons the designers took away from them. As in many other professions, storytelling may prove to be a powerful method for not only capturing expertise in situ but also for inducting novices into the culture of ID practice (Jonassen & Hernandez- Sorreno, 2002). Study 3 (Ertmer et al., PIQ, 2009) Results from Studies 1 and 2 helped us better understand how experts approach ill-structured problems, including the rules of thumb they employ. Based on these results, we turned our attention to the question posed earlier regarding ways to scaffold novices efforts to engage in effective problem finding when confronted with complex ID problems. Our earlier work (Stepich et al., 2001) suggested that instructors can coach students during problem analysis to think about things that they might not think about on their own. According to Saye and Brush (2002), these types of supports comprise a form of soft scaffold, which are dynamic and require teachers to continuously diagnose the understandings of learners and provide timely support based on student responses (p. 82). In contrast to soft scaffolds, Saye and Brush also defined hard scaffolds, which include those that act as static supports that can be anticipated and planned in advance based on typical student difficulties with a task (p. 82). Study 3 was designed to examine the effect of using hard scaffolds on students ID problem-finding efforts pp TICL_005_Stepich.indd /23/2009 3:18:27 PM

9 Teaching Instructional Design Expertise 155 Based on our knowledge about how experts synthesize issues in a problem situation by narrowing the problem space and focusing on key problem elements (Study 1), as well as our understanding that they use their knowledge of ID models in flexible and dynamic ways (Studies 1 and 2), we developed a set of analysis guidelines to serve as hard scaffolds for novices who were presented with an illstructured ID problem (see Figure 2). Guidelines were fairly general but invoked the various characteristics of problem finding outlined in our conceptual framework (2005) and confirmed by the previous studies. Write up your understanding of the case as an individual. Please do not consult with other people regarding the case, but you can use other written resources, if you wish. Your analysis should take no more than 2 hours. To guide you in this task, consider the following suggestions: Use your own words Focus on the big picture rather than surface details Make assumptions about missing information Focus on root causes rather than quick fixes Consider the core issues - those that are most central to your understanding of this situation Consider the critical issues - those that are likely to have the greatest impact on a successful resolution If you identify multiple issues, think about how those issues fit together Think about where the issues you identify fit within a traditional instructional design model (e.g., Dick & Carey, Smith & Ragan, or Morrison, Ross, & Kemp) Figure 2 Treatment guidelines for completing the case analysis. Specifically, Study 3 was designed to examine the impact these guidelines had on novices problem representations and how those representations compared to those developed by experts. Our research questions included: How do the problem representations of experts and novices compare? and What is the effect of guidance on novices problem representations? Can we scaffold novices representations to look more like experts representations? To answer these questions, we used a posttest only, control-group research design in which 24 novices and 8 experts were asked to describe their understand pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

10 156 Stepich and Ertmer ings, in approximately 500 written words, of a problem scenario presented in a given case study (Simons & Salter, 2007). Participants were divided into 3 groups: Novice treatment group (n = 13) had less than 3 years experience and were given analysis guidelines to scaffold their written responses Novice control group (n = 11) had less than 3 years experience and were not given analysis guidelines Expert group (n = 8) had 8+ years of experience and were not given analysis guidelines Participants case analyses were blind scored using a four-point scale (0 = novice; 3 = expert) on each of the four dimensions of problem finding (Ertmer & Stepich, 2005): coherent representation, underlying principles, relationships among issues, and reflective thinking. After reaching consensus on the scores for each case response, average scores for each group (expert, novice treatment, and novice control) were calculated on the four dimensions. Due to the small sample size and the ordinal nature of the scoring rubric, a one-tailed nonparametric test (Wilcoxon 2-sample tests) was performed. Table 1 presents the average scores on the four dimensions of problem finding earned by the three groups of participants. As predicted, novices in the control group had the lowest average score on each of the four dimensions, while experts had the highest. Results of the Wilcoxon nonparametric tests indicated a significant difference between experts and control novices on the total score (p =.02) and on the coherent representation (p =.02) and underlying principles (p =.02) dimensions. There were no significant differences between the experts and the treatment Table 1 Means and SDs of Four Dimensions of Problem Finding. Dimension / Group Coherent Representation Underlying Principles Relationships among Issues Reflective Thinking Total Score Expert M = 2.13 M = 2.38 M = 2.25 M = 1.75 M = 8.5 (n = 8) SD =.64 SD =.74 SD =.89 SD = 1.17 SD = 2.98 Treatment M = 2.0 M = 1.69 M = 2.15 M = 1.54 M = 7.39 (n = 13) SD =.91 SD =.95 SD =.90 SD = 1.2 SD = 3.64 Control M = 1.27 M = 1.18 M = 1.46 M = 1.09 M = 5.0 (n = 11) SD = 1.01 SD = 1.08 SD = 1.04 SD = 1.14 SD = 3.58 Note. Scale ranges from 0 (mostly novice) to 3 (mostly expert) pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

11 Teaching Instructional Design Expertise 157 novices on any dimension or the total score. Treatment and control novices showed a significant difference only on the coherent representation dimension (p =.04). In general, the results supported our hypothesis that analysis guidelines can support the problem-finding efforts of novices. However, a small sample size, as well as some additional limitations (e.g., not knowing if the treatment participants actually used the guidelines, not knowing the types of previous experiences participants had), suggest the need for caution in our interpretations. Additionally, further consideration must be given to the fact that average scores on each dimension, even for the experts, were not as high as might be expected. This suggests that the task may have unduly constrained the experts responses. The limitations of a 500-word written response might not have allowed experts to show the kind of sophisticated problem analyses that would distinguish them from novices. (Of course, we might also wonder if the experts in this study were truly experts, or perhaps, just experienced non-experts. ) While the preliminary results of this study would benefit from additional verification, they suggest that hard scaffolds can help novice designers focus on the more critical aspects of a problem situation. And because these types of scaffolds are relatively easy to implement in an instructional setting, ID educators may find them to be a worthwhile addition to the soft scaffolds they already provide as students work through complex ID problems. Summary Results from the three studies described above suggest that experts use an amalgamation of their knowledge and experience to synthesize issues in a problem situation. They tend to narrow the problem space by focusing on key problem elements and to apply their knowledge of ID models in flexible and dynamic ways. Furthermore, it appears that we can coach problem finding. That is, relative novices can be directed to perform in ways that are similar to relative experts. As ID educators, we re interested in finding ways to improve ID education. Therefore, a logical next question is: Can we apply these research results to the practical task of teaching ID? The next section attempts to answer this question by describing three specific strategies that can be used to support the development of problem-finding skills in ID students. For each strategy, we provide a description, relate the strategy to results from the three studies described above, describe possible ways to implement the strategy, and pose questions that might guide future research on the strategy pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

12 158 Stepich and Ertmer What Can We Do to Support the Development of Expertise? Help Students Accumulate Experiences A prominent characteristic of experts in any discipline is a large and well-organized store of domain-related knowledge acquired through years of experience (Ericsson, 2006; Glaser & Chi, 1988). Because of this extensive experience, experts are able to quickly recognize important information in a given situation, interpret that information, and decide on an appropriate course of action in a process that has been called recognition-primed decision-making (Klein, 1998, p. 17 ) and situation awareness (Endsley, 2006, p. 633). For example, the experts in Study 1 were able to sift through the details in an ill-structured ID problem, highlight those details that were central to understanding the problem, and determine how to respond to the identified problem(s). Therefore, the first challenge for ID educators is to help students accumulate relevant experiences. There are various ways to do this. Among the most common are the use of projects and case studies. Projects. Thomas (2000) described a project as a complex task that: is the central instructional strategy within the course requires students to struggle with the central concepts and principles of a discipline requires the transformation and construction of knowledge is student-driven rather than instructor-led centers on an authentic situation in which there is the potential to implement solutions developed by the students As an example of using projects to teach instructional design, the first author teaches an introductory course in which the central assignment is an ID project that requires students to work in small teams to create a training program that will be delivered to employees within a client organization outside of the university. In completing the project, students must carry out common ID tasks (e.g., learner analysis, formative evaluation), apply relevant principles of learning and instruction, and resolve typical design challenges (e.g., promoting transfer of learning, fitting the training program within prescribed time constraints). This requires them to continually synthesize their emerging knowledge of instructional design and figure out how to apply that knowledge to a practical problem situation. Throughout the project, the instructor provides direction, guidance, and coaching to the student teams pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

13 Teaching Instructional Design Expertise 159 Project-based learning has the advantage of high fidelity. That is, it closely represents real instructional design work in two important ways. First, it immerses students in a messy situation and requires them to find ways to resolve a variety of problems that practicing instructional designers commonly encounter. Second, it promotes the same high level of accountability that practicing instructional designers experience. In completing the project, students must collaborate with their teammates and meet the expectations of their clients. Case studies. A case study has been described as a darn good story (Wasserman, 1994, p. 44) that presents students with a partial, historical, clinical study of a situation which has confronted a practitioner (Christensen, 1987, p. 27) and asks them to analyze and solve the problems presented by the situation through reflection and discussion. As an example of using case studies to teach instructional design, the authors co-taught an advanced ID course that combined students from their respective universities (Ertmer & Stepich, 2004). The central assignment within the course was the analysis of a series of published case studies (Ertmer & Quinn, 2003). For each case, students were asked to (1) identify the problems and issues in the case and (2) make recommendations for solving the identified problems. Throughout the course, students participated in weekly online and face-to-face discussions designed to promote reflection and dialog. During the discussions, students shared their views of the case, challenged one another s views, asked questions, and worked to connect their emerging knowledge of instructional design to the problems presented in the case. Discussions were held via an asynchronous electronic bulletin board and synchronous Internet-based videoconferencing. The instructors participated in each discussion and worked primarily to keep the discussion on track, articulate connections among students comments, correct any misconceptions, and ask questions intended to promote deeper thought about the case. Case-based learning might seem less direct than project-based learning in that it does not immerse students in the process of instructional design as much as project-based learning. However, this also has a distinct advantage. Case-based learning encourages a focus on issues rather than tasks and allows students to explore, in depth, the range of problems within a given situation without the time constraints that are often associated with meeting project timelines. In addition, case-based learning makes it easy for students to experience multiple, varied problem situations. This allows them to continually reflect on their emerging knowledge as they examine ID issues within a broad range of settings (Julian et al., 2000). What else do we need to know about accumulating ID expertise? Project-based learning and case-based learning are commonly used within ID education. How pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

14 160 Stepich and Ertmer ever, the benefits of these instructional methods have not been fully examined. As a result, professional educators would benefit from the exploration of several questions: What are the comparative benefits of project-based learning vs. case-based learning? Which results in greater student motivation, learning effectiveness, or transfer of learning? Which helps students think/act more like experts? Are these methods useful with different students? For example, is projectbased learning more effective for students who have more or less experience with instructional design? What are the essential characteristics of either project-based learning or case-based learning? That is, what characteristics of the learning situation contribute most to student learning? Help Students Index Experiences Experts have accumulated large stores of domain-related experiences and, perhaps more importantly, are able to use those experiences to solve current problems. This is because the expert s experiences are organized, much like a library, in a way that allows quick and easy access. In fact, it is the organization, as much as the volume, of experiences that distinguishes an expert practitioner from someone who has conceptual knowledge alone (Schank & Cleary, 1995). The keys to the expert s ability to use his/her prior experiences are the interrelated processes of indexing and reminding. Indexing refers to the process of encoding a new experience into an existing memory structure in a way that allows later retrieval (Kolodner & Guzdial, 2000; Schank & Cleary, 1995). Indexes are dynamic rather than static. That is, with each new experience, experts clarify and expand their indexes in an ongoing effort to make sense of their collected experiences. Reminding refers to the process in which something in the current situation triggers the recall of relevant prior experiences, which then serve as the foundation for a response to the current situation (Jonassen, 2007; Kolodner, 1997; Schank & Cleary, 1995). Reminding does not require a perfect one-to-one match between the current situation and prior experience. In fact, experts are adept at partial matching (Schank, 1990, p. 225) in which they recall prior experiences that are similar to, but not the same as, the current situation. They then use those partial matches to guide their responses to the current situation. Therefore, the second challenge for ID educators is to help students develop indexes that are meaningful to them and will promote effective reminding. This pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

15 Teaching Instructional Design Expertise 161 challenge is accentuated with students who are new to instructional design because they are starting from scratch. That is, while more experienced individuals can build on existing indexes, new students have no existing indexes on which to build. As a result, their efforts to organize new experiences can seem random. The challenge with new students, then, is to help them develop starter indexes that they can use immediately and continue to build on as they accumulate more experiences. Ideally, these starter indexes would be: Easy to remember Flexible enough to allow individuals to incorporate a wide variety of new experiences Clear and coherent enough to allow easy communication with others in the field. The results of Studies 1 and 2 suggest one possibility. The experts in those studies based their problem finding on one or more rules of thumb that were, at least in part, related to a mental model of the ID process. These mental models varied, but they all included common elements such as identifying important characteristics of the learners, specifying learning objectives, and developing instructional strategies. This suggests that an ID model could serve as a starter index for new students. One simple example might be the ADDIE model that is common in instructional design (Molenda, 2003). ADDIE divides the ID process into five broad stages: Analysis, Design, Development, Implementation, and Evaluation. The ADDIE model provides an index that is: Easy to remember. Students can easily recall the acronym. Flexible. The indexes can easily be expanded as individuals gain experience. For example, individuals can easily develop an increasingly detailed design index by gradually adding elements, as shown in Figure 3. Part of the common language of instructional design. Right from the start, students can easily talk with other instructional designers about their experiences with analysis, design, and so on. As their experiences increase, they can talk with other designers about, for example, the practice component of instructional strategies. Then, as their experiences increase further, they can talk with other designers about the importance of assuring the authenticity of those practice exercises. Given these characteristics, the ADDIE model comprises one possible starter index for novice instructional designers. This is not to suggest that ADDIE is the pp TICL_005_Stepich.indd /23/2009 3:18:28 PM

16 162 Stepich and Ertmer Design Objectives Instructional strategies Instructional methods Instructional media Information presentation Practice exercises Assessment tools Authenticity Guidance Variation Figure 3 An index for the design stage of the instructional design process. optimal target index; indeed, there are many models that more accurately capture the complexity of the ID process. However, ADDIE does seem to provide a relatively simple starter index in that it is easy to remember, easy to build upon, and part of the common language of the field. It is likely that other fields have similar models that could be used as starter indexes for novices in those domains. Once individuals have developed a starter index, they should be encouraged to use and gradually expand it. This means indexing each new experience in a way that facilitates the process of reminding when faced with a new problem situation. Kolodner (1997) suggested that educators can help students accomplish this by encouraging them to: Extract the lessons learned from a new experience. Articulate those lessons learned in a variety of rich ways. Predict the circumstances in which a lesson learned might be applied in the future. ID educators can help students accomplish these tasks by asking them to engage in some reflective thinking about a new experience. An important goal of this reflective thinking is generalization, or translating particular lessons into the pp TICL_005_Stepich.indd /23/2009 3:18:29 PM

17 Teaching Instructional Design Expertise 163 broader, more inclusive rules of thumb that guided the problem-finding efforts of the ID experts in Studies 1 and 2. The questions used to guide this reflective thinking will vary from situation to situation, but might include one or more of the following: If you were in a similar situation again, what would you do the same? What would you do differently? How would you index this case for future reminding? What implications does your lesson have for other parts of the ID process? For example, what implications does a lesson about analysis have for the design of the instruction in that situation? Is a particular lesson similar to any other lessons, either from this experience or from previous experiences? Can you think of a rule of thumb that encompasses these similar lessons? What else do we need to know about indexing? Several authors have suggested that indexing and reminding are governed by abstract principles rather than specific experiences. For example, based on studies of urban fire ground commanders, wild land incident commanders, tank platoon commanders, and system design engineers, Klein and Calderwood (1988) concluded that specific incidents disappear under the weight of cumulative experiences because the specific incidents have been blended together into what may be considered a prototype or schema representation (p. 216). Our research is consistent with this view. Rather than recalling specific prior experiences, our ID experts were likely to recall a schema in the form of a rule of thumb drawn from their collected knowledge and experience. How does the individual s level of experience influence this result? That is, are more experienced individuals more likely to recall rules of thumb than less experienced individuals? As an individual gains experience, at what point does s/he begin to develop rules of thumb? What mechanisms does s/he use to develop those rules? What implications does this have for teaching ID students with different levels of experience? And specifically for beginning students, what is the relative effectiveness of giving them a starter index vs. helping them develop their own indexes? Will an ID model be an effective starter index? Will it promote recall and transfer of students lessons learned? Provide Scaffolds As noted earlier, expertise exists on a continuum and our job, as ID educators, is to help students move toward the expert end of the continuum. Although it is critical that we help students accumulate and index relevant experiences, the use pp TICL_005_Stepich.indd /23/2009 3:18:29 PM

18 164 Stepich and Ertmer of hard and soft scaffolds (as defined by Saye & Brush, 2002) offers additional strategies for supporting students developmental efforts, as demonstrated by our own work (Ertmer, Stepich et al., 2009; Stepich et al., 2001) and that of others (e.g., Ge, Chen, & Davis, 2005). For example, results from Study 3 showed that students performed more like experts on some problem-finding dimensions when they were given hard scaffolds in the form of general suggestions that guided their analyses of a written case study. According to Hogan (1977), an instructional scaffold is a tool for enculturating students into the thinking patterns of experts (p. 2). Within this enculturation process, scaffolds serve three primary purposes: 1) to reduce cognitive load, 2) to provide expert guidance, and 3) to help students acquire disciplinary ways of thinking and acting (Hmelo-Silver, Duncan, & Chinn, 2007, p. 101). We describe each of these three functions in more detail. Scaffolds can reduce cognitive load by structuring a task in ways that allow students to focus on those aspects that are relevant to the learning goals. For example, providing structured workspaces can help students decompose a task and/or organize their work (e.g., Map each potential solution back to the issues identified in the case. ). Additionally, prompts can be used to help students recognize important goals to pursue (Reiser, 2004; e.g., Focus on the big picture rather than the surface details. ). Scaffolds can also restrict the options that are available to the students at any point in time to make the task more accessible and manageable (e.g., Choose the statement, from the list of possibilities below, that best captures the core issue in the given problem situation. ). Scaffolds can embed expert guidance by providing explanations that can be accessed by the students, when needed, to understand either the process they are engaged in or the rationale underlying it. These types of scaffolds are likely to take the form of hints and/or verbal explanations (e.g., When analyzing the issues in this case, consider how communication barriers influenced the decisionmaking process. When experts begin new projects, they tend to narrow the problem space by using the ID model as a heuristic. Consider where the design issues in this case would be classified within a traditional ID model. ) Scaffolds can prompt disciplinary thinking and actions by making key aspects of expertise visible to the students. These types of scaffolds typically include various forms of modeling and coaching. For example, prompts may be given to trigger the use of particular reasoning strategies ( Focus on root causes rather than quick fixes. ); templates or task structures can guide students reasoning along pre-defined paths (e.g., filling in argument diagrams to distinguish between claims and evidence); or models of expert performance/thinking (video- or audiorecordings) can demonstrate what an expert approach looks like pp TICL_005_Stepich.indd /23/2009 3:18:29 PM

19 Teaching Instructional Design Expertise 165 Based on more recent conceptions of the scaffolding metaphor, scaffolds enable students to accomplish tasks more complex than they could do alone in such a way that they can still learn from the experience (Quintana et al., 2004, p. 340). Thus, when designing effective scaffolds for novice designers, it is important to include both those that structure the design/problem-solving tasks (i.e., reduce the complexity) and those that increase students abilities to solve the problem independently (i.e., promote deeper understanding). However, these dual purposes may, in fact, be contradictory, thus introducing a number of tensions into the problem-solving process (Simons & Ertmer, 2005). One of the biggest tensions reflects the need to constrain students efforts without creating an environment that is no longer open-ended (Reiser, 2004). That is, our scaffolds should constrain, but not inhibit, students independence [e.g., If students are asked to pick the relevant problem statement from among a list of choices, how will they learn to create a problem statement on their own?]. Another tension relates to the need to simplify components of the problem and content without making them appear artificial or superficial (Brush & Simons, 2004; Reiser, 2004). That is, our scaffolds should enable students to understand the complexity of a given domain without oversimplifying it to such an extent that it is no longer accurately represented [e.g., If we only present students with well-structured problem situations, how will they learn to analyze an ill-structured problem situation?]. Developing scaffolds that support our students without overly simplifying the problem or unduly constraining their analysis efforts poses a huge challenge to design educators and would benefit from further investigation. Finally, a third tension relates to the tendency of experts to engage in heuristic decision making. That is, our scaffolds should support effective decision making without rigidly prescribing the steps to be followed [e.g., If we present students with prescriptive algorithms for analyzing problem situations, how will they learn to adapt their analysis methods to the requirements of the particular situation?]. What else do we need to know about scaffolding ID expertise? Previous research on instructional scaffolds has been conducted within disciplines, such as science or math, which are more content-specific than instructional design (Hmelo-Silver et al., 2007). To what extent can we assume that these same types of scaffolds will be useful when solving ID problems? Are there other types of scaffolds that might be more relevant, and therefore more effective, in supporting the problem-solving efforts of ID students? If the three purposes of scaffolds described above are relevant to ID, then research can help determine whether any one of these purposes is more useful than the others, or whether scaffolds that address more than one purpose are more effective than those that address only one. Furthermore, how can we assure that students mindfully engage with these scaf pp TICL_005_Stepich.indd /23/2009 3:18:29 PM

20 166 Stepich and Ertmer folds? Can scaffolds be effective without being promoted, modeled, and supported by instructors and/or trainers? Additional research examining the efficacy of specific scaffolding strategies is needed to address these important questions. And what about the issues of transfer and fading? One of the basic ideas behind the use of scaffolds is that they serve as temporary supports that enable students to perform at higher levels than is possible without them and that, over time, independent performance will be possible. According to Sharma and Hannafin (2004), directive guidance, especially when used to support the development of higher-order thinking skills, may engender an over-dependence on the scaffolds, thus limiting students future capacity to act independently. However, this suggestion is contradictory to findings by Dufresne, Gerace, Hardiman, and Mestre (1992) who demonstrated that students who were forced to approach a physics problem in the same manner as an expert used the more expert-like approach even after the constraints were removed. Additional work is needed to sort out these apparent contradictions. How quickly and to what extent can our suggested scaffolds be faded so that students transfer relevant metacognitive skills to future tasks? Furthermore, how many trials with the scaffolds will be necessary before we can reliably remove them? What s the proper balance between the use of broadly applicable guidelines and those that are more tailored to a specific type of problem (classified by content, topic, or ID step)? Conclusion Preparing ID students for professional practice is a complex and challenging task. To be successful, instructors must prepare students to apply deep conceptual knowledge to the solution of authentic problems under conditions of uncertainty (Gardner & Shulman, 2005). By examining how ID experts approach problemfinding tasks we gain a better understanding of how to support our students development. In this paper, we described three specific strategies for facilitating students growth: 1) helping students accumulate experiences, 2) helping students index those experiences, and 3) scaffolding students efforts as they are developing their problem-solving skills. Although these strategies are based on our work with ID novices and experts, we believe they have implications beyond the ID field. Based on our growing understanding of ID expertise, we know that students need to adaptively apply both knowledge and experience when solving ill-structured problems. Accumulating more experiences, directly or vicariously, can provide students with more sophisticated rules of thumb to apply in practice. However, accumulating experience, in and of itself, is not enough to build exper pp TICL_005_Stepich.indd /23/2009 3:18:29 PM