Technology-mediated learning methods are widely used by organizations and educational institutions to

Transcription

1 Information Systems Research Vol. 19, No. 1, March 2008, pp issn eissn informs doi /isre INFORMS Using Self-Regulatory Learning to Enhance E-Learning-Based Information Technology Training Radhika Santhanam Gatton College of Business and Economics, University of Kentucky, Lexington, Kentucky 40506, Sharath Sasidharan Lewis College of Business, Marshall University, Huntington, West Virginia Jane Webster Queen s School of Business, Queen s University, Kingston, Ontario Canada K7L 3N6, [email protected] Technology-mediated learning methods are widely used by organizations and educational institutions to deliver information technology training. One form of technology-mediated learning, e-learning, in which the platform is the tutor, is quickly becoming the cost-effective solution of choice for many corporations. Unfortunately, the learning outcomes have been very disappointing. E-learning training makes an implicit assumption that learners can apply a high level of self-directed learning to assimilate the training content. In contrast, based on perspectives from social cognitive theory, we propose that instructional strategies need to persuade learners to follow self-regulated learning strategies. We test our ideas with participants who were trained through e-learning to design a website. Our findings indicate that participants who were induced to follow self-regulated learning strategies scored significantly higher on learning outcomes than those who were not persuaded to do so. We discuss our findings, and suggest that the interaction among information technology features, instructional strategies, and psychological learning processes offers a fruitful avenue for future information systems training research. Key words: e-learning; laboratory experimentation; information technology training; self-regulatory learning; social cognitive perspective; pretraining scripts; website development; self-efficacy History: Ritu Agarwal, Senior Editor; H. Raghav Rao, Associate Editor. This paper was received on August 19, 2005, and was with the authors 8 months for 3 revisions. Introduction Almost every employee today has to be skilled in using information technology (IT); therefore, organizations continue to invest significantly in IT training for their employees, and universities offer many IT training courses for students (Agarwal and Ferratt 2001, Bureau of Labor Statistics 2004, Homer and Povar 2004, Piccoli et al. 2001). Given the extensive deployment of IT training in corporations and institutions, it is not surprising that information systems (IS) researchers expend substantial efforts to identify the most effective training methods and strategies (e.g., Agarwal et al. 2000, Bostrom et al. 1990, Compeau et al. 1995, Compeau and Higgins 1995a, Johnson and Marakas 2000, Olfman and Mandviwalla 1994a, Lim et al. 1997, Santhanam and Sein 1994, Venkatesh 1999, Yi and Davis 2003). In recent years, IT training is increasingly being delivered through electronic means such as technology-mediated learning (TML) methods instead of through face-to face interactions between learners and trainers. TML as a training vehicle is growing rapidly because it is seen as a cost-effective way to deliver training at convenient times and at remote locations to large numbers of employees and students (Zhang et al. 2004). A study by the Sloan Consortium finds that educational institutions use TML extensively: 81% offer at least one fully online or blended course, and 34% offer complete technology-based degree programs (Allen and Seaman 2005). In practice, however, 26

2 Information Systems Research 19(1), pp , 2008 INFORMS 27 TML has not provided the benefits that were originally anticipated; corporations, educational institutions, and IS researchers are thus very motivated to search for ways to improve TML effectiveness (Alavi and Leidner 2001, Gupta and Bostrom 2005, Moller 2002, Olfman et al. 2006, Sasidharan and Santhanam 2006). IS researchers emphasize the exigency of conducting research on TML for several reasons. First, this research can build on and contribute to the cumulative knowledge developed by IS research on the interaction of IT and human problem solving and learning processes (Alavi and Leidner 2001, Leidner and Jarvenpaa 1995). Second, because IT training is an integral component of IS research, existing findings on how best to develop and test IT skills could contribute to our understanding of whether IT infrastructures are effective IT training platforms (Gupta and Bostrom 2005, Olfman et al. 2006). It can also help us research IT software aspects that provide new opportunities for the control and pacing of user learning. Finally, enhancing IT skills of students through TML is a research topic of growing interest to IS academics as evidenced by research articles, specialized journals, and special interest groups on IT education (Hardaway and Scamell 2005, Leidner and Jarvenpaa 1995, Markus 2005). Consequently, there is a need to study the role of TML as an effective IT training delivery mechanism (Gupta and Bostrom 2005, Olfman et al. 2006, Salas and Canon-Bowers 2001, Zhang et al. 2004). We believe that IS researchers, with their multidisciplinary focus and ability to integrate social and cognitive processes with technology affordances, are uniquely qualified to study TML. In this research, we address the role of TML as an IT training mechanism in the following manner: (1) We draw on a key learning-through-technology framework (Alavi and Leidner 2001) that has not received much empirical attention; (2) we examine one specific TML environment and the underlying instructional strategies in order to identify methods that can improve IT training outcomes; and (3) we propose and test interventions based on social cognitive (SC) theory (Bandura 1991) that can induce learners to self-regulate their learning and enhance their IT skills via TML training. Combined, this approach provides a foundation for future work in both learning through IT and learning about IT. We focus our attention on the specific TML environment, referred to as e-learning, where the learner interacts primarily with the IT platform rather than with other learners or instructors (Jonassen and Reeves 2001, Jones and Paolucci 1999, Zhang et al. 2004). As we elaborate in the next section, the instructional strategy in this type of training infrastructure is anchored on self-directed and independent learning. The implicit assumption is that learners are able to meet the demands expected of the instructional strategy, apply high levels of learner control, and self-direct their learning. Reports indicate, however, that learners are not able to apply the anticipated high levels of learner control, are not motivated to learn, and tend to use inadequate learning strategies (Bell and Kozlowski 2002, Brown 2001, Rossett and Schafer 2003). Therefore, we argue that if e-learning is to become an efficacious IT training method, instructional strategies should be modified to include interventions that persuade learners to follow self-regulated learning (SRL) strategies. In the next section, we describe how the social cognitive perspective on self-regulation can help design interventions that modify instructional strategies in e-learning-based training environments. We then report on an experiment in which we provided manipulations to encourage learners to self-regulate while learning to use Website-development software. We conclude the paper by discussing the implications of the study for practice and future research. Research Framework In this section, we first explain the different terms used in reference to TML. We then review TML research and describe the learning-through-technology framework proposed by Alavi and Leidner (2001). Finally, we propose that e-learning-based training could be improved by paying attention to self-regulation, and elaborate on how it could be applied by proposing specific hypotheses to be tested. Characteristics of TML and E-Learning TML refers to an environment in which IT is used to mediate/support teaching and course delivery and

3 28 Information Systems Research 19(1), pp , 2008 INFORMS includes a variety of learning environments: different functions and features of IT may be selectively applied, instructional packages may be bundled in various ways, and learners may be provided with different levels of control (Benbunan-Fich 2002, Bostrom 2003, Jonassen 2004). E-learning describes a TML environment in which a single user interacts with technology and attempts to self-direct and complete a training course (Zhang et al. 2004). In Figure 1, we list a few commonly used terms and corresponding descriptions related to e-learning. Virtual learning is a broad term to refer to computerbased environments with a wide range of resources made available to learners (Anohina 2005, Piccoli et al. 2001). Technology-based learning, or learning from computers, encompasses both distance learning and e- learning because technology is used in both as the primary medium to deliver course content (Gupta and Bostrom 2005, Jonassen and Reeves 2001, Jones and Paolucci 1999). In distance learning, the teacher and learner are separated in space and time, but they can communicate with one another; in many implementations, learners can communicate with other learners as well. Among TML contexts, e-learning provides maximal control to learners so that they can learn in an independent manner. However, as seen in Figure 1, in e-learning the learner may be unable to communicate with the instructor or with other learners. Figure 1 E-Learning-Related Terms and Descriptions Virtual Learning An encompassing term denoting computer based instructional environments that are relatively open systems with a wide range of resources that learners can use to interact with other learners and instructors. Some researchers refer to this as Web-based learning or online learning if educational content is transferred via the Web browser and computers connected to the intranet. Delivery of educational content via a Web browser over the public internet, a private intranet, or an extranet. Training via computers connected to the World Wide Web. Technology-Mediated Learning (TML) A term used in instructional environments where information technologies are used to support course delivery and to manage the teaching and the learning process. It can refer to a broad set of applications such as learning labs, teleconferencing systems, computer-mediated communications, collaborative learning systems, etc. Though some IS researchers refer to this as e-learning, most disciplines make a distinction between TML (learning with computers) vs. technology-based learning (learning from computers). When information technology is used primarily to support the learning process but course content is not necessarily delivered via computers, it tends to be referred to as TML. Technology Based Learning/Training (TBL/TBT) A term used when the delivery of course content is via computer technology. This is thus referred to as learning from computers. Within this are two groupings of terms: distance learning and e-learning. Distance Education/Learning Separation of teacher and learner in space and/or time Provision for two-way communication Educational institution typically provides certification on course completion May involve some classroom teaching as well May also include learners engaged in group learning E-Learning User engaged in self-paced learning with learner in control. Learning package is delivered or transacted through electronic means, sometimes referred to as computer microworlds. If e-learning is conducted through stand-alone computers, it is generally referred to as Computer-Based Training/Computer-Assisted Instruction. If e-learning is delivered through the internet, intranet (or extranet), or a Transmission Control Protocol/Internet Protocol network, it is referred to as Internet-based training/learning. If e-learning involves use of communication technologies to facilitate discussions among learners, it is referred to as collaborative e-learning. Note. Figure adapted from Anohina (2005). Descriptions obtained from Anohina (2005), Alavi and Leidner (2001), Piccoli et al. (2001), Jonassen (2004), and Keegan (1996).

4 Information Systems Research 19(1), pp , 2008 INFORMS 29 An e-learning infrastructure typically consists of a database-centric learning content-management system that lists the course catalogs (from which the learner can choose a specific course), registers the learner, and manages the interaction between user and system (Bostrom 2003). A typical IT training course, such as an introductory course on website development, might consist of several sequential learning modules, with each module providing instructional information on one topic related to designing a website. The first module may consist of general information on websites, the second on features of the specific software; the third may introduce procedures to develop a website, and so on. Advanced courses could comprise a dozen such modules. Prior Research on TML The general challenges of TML-based training have been addressed by researchers in other disciplines, but little attention has been paid to IT skill development courses (e.g., Bernard et al. 2004, Jonassen 2004, Jones and Paolucci 1999). Factors impacting distance learning effectiveness have been examined even from the days of correspondence courses, with recent studies focusing on technology-mediated environments and their implications for the new roles of teachers and students (Berge and Mrozowski 2001, Keegan 1996, Noffsinger 1926). A meta-analytic review of over 200 research studies on distance learning concludes, among other findings, that learning outcomes can be enhanced by paying attention to specific features of the instructional environment and to the proper use of instructional strategies (Bernard et al. 2004). Other reviews of TML also point to the importance of paying attention to instructional strategies and to the specific features in a TML environment. A review that summarizes research findings comparing TML contexts involving groups of learners versus individual learners concludes that TML courses are not superior overall, but that group learning with TML has more favorable effects than individual learning from TML. This suggests that feedback, the social context, instructional strategies, and interaction among learners can maximize learning outcomes (Lou et al. 2001). Unfortunately, many commercial technologies are based on an individual learning model, not on a collaborative learning model (Benbunan-Fich and Hiltz 2003, Richardson and Swan 2003). This points to the importance of paying attention to instructional strategies in specific TML environments, and to the special challenges in contexts where learners have to learn independently without opportunities to collaborate with other learners. As is evident from Figure 1, e-learning, the technology examined in this study, represents such an environment where learners do not get opportunities to interact with other learners. A review of TML research in IS indicates that, similar to other disciplines, IT skill development has not been given much attention (Alavi 1994; Alavi et al. 1995, 2002; Coppola et al. 2002; Leidner and Fuller 1997; Sasidharan and Santhanam 2006). Most of these studies test TML in collaborative learning contexts and, as in other disciplines, conclude that using TML methods does not automatically lead to superior learning outcomes: attention must be given to instructional strategies and specifics of the TML context. Hence, as articulated by Alavi and Leidner (2001), learning outcomes can be increased by paying attention to the interaction of three key factors in specific TML environments, namely, information technology, instructional strategy, and learners psychological processes. Except for a study by Piccoli et al. (2001), little research has addressed these factors and their impact on learning outcomes. Based on Alavi and Leidner s (2001) framework, we argue that, because IT features in e-learning often do not permit learners to interact with other learners or instructors, the IT platform becomes the dominant mode of communication with the learner. The accompanying instructional strategy relies on selfdirected learning with the assumption that learners can independently regulate their learning and absorb the training content. In other words, the conditions of training are established to evoke a high level of selfdirected learning: the training design is based on an objectivist learning model, which assumes that learners learn best in an isolated and intensive manner by regulating their own learning (Gagne 1977, Gagne et al. 1992, Leidner and Jarvenpaa 1995). But such a design imposes a very high burden on the learner: to be motivated and focused on learning without any guidance from human instructors or other learners. Even in e-learning environments where learners can interact with other learners and instructors, learners

5 30 Information Systems Research 19(1), pp , 2008 INFORMS complain that they find it difficult to take on the responsibility for directing their own learning (Piccoli et al. 2001). Empirical observations attest that learners situated in e-learning environments do not adequately self-direct their learning, nor do they exercise high levels of learner control, which might explain the higher drop-out rate (Allen and Seaman 2005, Bell and Kozlowski 2002, Brown 2001, Zhang et al. 2004). Therefore, as described below, we propose that the instructional strategy, or the sequence of activities that systematically exposes learners to experiences to help them acquire knowledge, must be modified to enhance the effectiveness of e-learning-based training. Self-Regulation and E-Learning The interaction between IT features and instructional strategies necessitates that the learner exhibits an e-learning strategy that includes high learner control, self-discipline, and self-motivation. We propose that learning outcomes will be enhanced if instructional strategies are modified to include interventions that instruct learners to follow self-regulatory learning strategies that include, among other things, encouraging learners to believe that they can learn through e-learning training, enhancing their motivation to learn, formulating appropriate goals for the course, and devising methods for organizing course content. In other words, instructional strategies should include interventions that instruct learners to apply self-regulation in their learning. Self-regulation refers to a general skill that keeps people focused on a task, helps them monitor their task-completion progress, and explains success in a broad range of phenomena, for example, management of chronic illnesses, training for sports, treatment of obsessive behaviors, and learning in academic settings (Bandura 1991, Boekaerts et al. 2000). Particularly in academic environments, researchers have found that students who self-regulate their learning reach higher academic achievements irrespective of their courses of study (Pintrich and DeGroot 1990, Zimmerman et al. 1992, Zimmerman and Schunk 2001). SRL involves strategies by which learners actively engage in learning and apply intentional efforts to manage and direct their learning activities. Zimmerman and Schunk (2001) review various theoretical perspectives on SRL, including the operant, Vygotskian, constructivist, volitional, phenomenological, information processing, and social cognitive. All of these perspectives tend to view SRL as something purposive that involves the use of specific strategies, but they differ on the factors they view as being relevant to a learner s use of SRL. For example, operant theorists emphasize the role of external factors, suggesting that learning responses are ultimately controlled by external reward/punishment contingencies such as verbal coaching and reinforcement. Information processing theorists emphasize the role of three types of memory (sensory, short term, and long term) in the SRL process. Learning strategies in this perspective emphasize learners ability to cluster bits of information into larger units and develop their capacity to process information. In contrast with the external factors highlighted in the operant theory, the social cognitive perspective takes an agency perspective and highlights individuals roles and their own ability to enact self-regulation. It states that learners by themselves can self-regulate through their specific use of strategies toward learning, and emphasizes that several factors, such as learners self-efficacy beliefs, their motivation to learn, and their learning goals and strategies, must be addressed in combination (Kauffman 2004, Pintrich and DeGroot 1990, Schunk 2001, Zimmerman 1989). We focus on this perspective because it takes a collective approach to addressing SRL, and the SC theory has been used frequently in IS research. The Social Cognitive Perspective on SRL The salient aspects of the SC perspective include, among other factors, learners motivations, their outcome expectancies, and their perceived self-efficacy beliefs (Schunk 2001, Zimmerman 2000). Learners perceived self-efficacy beliefs (i.e., beliefs about their capabilities to learn) are essential factors that affect all phases of self-regulation, and are formed based on prior observations or prior performance or through some form of persuasion (Schunk and Ertmer 1999, 2000; Zimmerman et al. 1992). The SC perspective also emphasizes the situation-specific nature of SRL; that is, learners may not engage in self-regulation equally in all learning environments (Schunk 2001). For example, research indicates that, although high achievers tend to use SRL strategies, they may not

6 Information Systems Research 19(1), pp , 2008 INFORMS 31 apply them in every situation (Schunk and Ertmer 2000, Zimmerman and Martinez-Pons 1990). Therefore, SC theorists emphasize the situational task goals of the learner, such as completing a specific homework assignment, and situation-specific self-efficacy beliefs, such as beliefs in their abilities to solve fraction problems in arithmetic (Schunk 2001, Zimmerman and Schunk 2001). Although some SRL strategies may generalize across settings, learners must know how to adapt to situation-specific domains and feel competent in doing so. In line with the situation specificity of SRL, forethought is described as an important period that occurs before learning and sets the stage for action. Motivations and preparations during forethought influence learners level of engagement with the learning task and impact learning outcomes (Schunk and Ertmer 2000). During learning, outcomes can be improved by providing performance-based feedback and facilitating learners metacognitive monitoring (Bell and Kozlowski 2002, Schmidt and Ford 2003). But addressing only metacognition or other isolated single components of learning may not be enough; instead, a collective (i.e., multifaceted) approach is useful because learners who discover a lack of progress through metacognitive monitoring also need some motivational regulation to continue their learning efforts (Pintrich and DeGroot 1990, Zimmerman et al. 1992). Most individuals use some level of self-regulation, but they differ on the quality and extent to which they apply it in specific contexts (Zimmerman 2000). This depends on their knowledge of SRL strategies, their decisions to use known strategies, and their ability to use these strategies skillfully (Schunk and Ertmer 1999). Thus, adopting an SC perspective considers not only learners choices of cognitive strategies, but also their fears, doubts, confidence, self-beliefs, and sense of personal agency in specific performance contexts (Zimmerman 1995, Zimmerman and Martinez-Pons 1988). Based on our adoption of the SC perspective on SRL, we believe that because learners use of SRL has to be activated in a specific situation (such as while undergoing e-learning-based training), context- Figure 2 Phases and Beliefs in Self-Regulatory Learning Performance or volitional control Forethought task analysis motivational beliefs Self-reflection and adaptation Phase Structure and Sub-Processes of Self-Regulation Cyclical self-regulatory phases Forethought Performance/volitional control Self-reflection Task analysis Self-control Self-judgment Goal setting Self-instruction Self-evaluation Strategic planning Imagery Causal attribution Seft-motivation beliefs Attention focusing Self-reaction Self-efficacy Task strategies Self-satisfaction/affect Outcome expectations Self-observation Adaptive defensive Instrinsic interest/value Self-recording Goal orientation Self-experimentation Note. Adapted from Zimmerman (2000). This article was published in the Handbook of Self-Regulation, edited by M. Boekarts, P. P intrich, and M. Zeidner, Attaining self-regulation: A social cognitive perspective, 13 39, copyright Elsevier, 2000.

7 32 Information Systems Research 19(1), pp , 2008 INFORMS specific interventions that seek to enhance learner motivation and that instruct learners to follow SRL strategies could improve learning outcomes. Development of Hypotheses We draw on a process model of SRL (shown in Figure 2) to design an instructional strategy that can help to persuade learners to apply SRL during e-learning training (Zimmerman 2000). In the model, SRL can occur through three cyclical phases: forethought, volitional control, and self-reflection. Although this is a process model suggesting how SRL could be affected, it can be observed from Figure 2 that the model includes the multifaceted aspects of SRL. In the forethought phase, the learners learning goals, motivation to learn, self-efficacy beliefs, and learning plan should be addressed. In the performance/volitional control phase, the learner focuses on the learning task, applies cognitive strategies like note taking, and organizes study materials, thereby self-directing their learning. During self-reflection, learners conduct metacognitive monitoring of their learning progress and adapt their strategies. One could use this model to design an instructional strategy such that a pretraining intervention occurs at the forethought phase, when learners get engaged in task analysis through which they understand the learning plans and goals for the training session. Hence, before the learner begins the e-learning tutorial, trainers could provide a task analysis for the training session by describing learning plans and goals for the session. During the forethought phase, learners must also develop motivational beliefs toward training and, once again, trainers could influence these motivational beliefs. This could be achieved if trainers were to provide learners with scripts that inform them about the various modules in the e-learning course, the goals and elements of each module, and a list of what they should have learned by the end of each module. The scripts could also be designed to boost learners motivational beliefs, outcome expectations, and interest in learning through e-learning. When learners start on the actual learning task, they should engage in performance/volitional control activities such as attention focusing and selfrecording, and self-observational strategies such as note taking. Similarly, scripts could also be used to influence these activities by instructing learners to apply the cognitive strategies of self-directing, recording, note taking, etc. As learning progresses, learners should engage in metacognitive activity and evaluate whether their strategies are working and, if necessary, adapt. Either of two methods can be effective in accomplishing this: (1) learners could self-evaluate and reflect of their own volition or (2) they could be provided with external evaluations and feedback (Zimmerman and Kitsantas 1997). Therefore, one way to help learners reflect on their strategies could be via external feedback emphasizing the success of the learners self-regulatory strategies. The feedback could be used to set goals for the rest of the training session and enhance learners motivation. This type of feedback and positive evaluation is an integral aspect of self-regulation that reinforces the use of SRL strategies and sustains motivation and self-efficacy beliefs (Schunk 2001, Schunk and Ertmer 1999). Therefore, we propose two hypotheses: Hypothesis 1A. In an e-learning-based IT training session, presession interventions designed to increase the learner s use of self-regulatory learning strategies and accompanying beliefs will enhance learning outcomes. Hypothesis 1B. In an e-learning-based IT training session, interventions that provide positive feedback to learners on their use of self-regulatory strategies will enhance learning outcomes. Learner Characteristics Researchers propose that certain dimensions of SRL can be influenced by stable dispositional traits: if a learner has a higher ability to set goals, to apply metacognitive strategies, and to self-regulate, etc., those skills could help the learner apply SRL and achieve higher learning outcomes (Ford et al. 1998, Schmidt and Ford 2003, Winnie 1995). In IS research, individual learner traits are noted as influencing training outcomes (Agarwal et al. 2000, Bostrom et al. 1990, Webster and Martocchio 1992). Studies on distance learning also indicate that learner characteristics may influence learning outcomes and should be considered (Bernard et al. 2004). Therefore, it is worthwhile to examine whether individual characteristics could influence e-learning-based training outcomes.

8 Information Systems Research 19(1), pp , 2008 INFORMS 33 Because the focus of this study is on SRL and the training interventions attendant on SRL, we were interested specifically in examining learner characteristics that might relate to SRL or to behaviors in a new learning environment. As shown in Figure 2, individual goal orientation could influence performance and volitional control. Beaubien and Payne (1999) describe goal orientation as an important individual motivational variable that explains how people develop competence in new learning and performance situations. Although first understood as a two-factor structure, goal orientation is now described as comprising three factors: learning orientation, performance approach, and performance avoidance orientations (VandeWalle 1997, Zweig and Webster 2004). For the purposes of this study, the learning orientation factor is relevant because e-learning training represents a new learning task. People with high learning orientations are more likely to find new learning tasks challenging and to view learning performance as indicative of mastery (Brett and VandeWalle 1999, Fisher and Ford 1998, VandeWalle 1997). Therefore, we propose the following hypothesis: Hypothesis 2A. In an e-learning-based IT training session, learning orientation will be positively associated with learning outcomes. Self-efficacy beliefs are important motivational beliefs in SRL (see Figure 2). In IS training research, self-efficacy beliefs are identified as a critical predictor of learning outcomes (Agarwal et al. 2000, Colquitt et al. 2000, Compeau and Higgins 1995b) and, as described earlier, task-specific self-efficacy beliefs are considered to be relevant (Bandura 1997). In this study, learners were trained through a computerbased program, so it can be expected that individuals self-efficacy beliefs regarding learning through computers may influence learning outcomes. In addition, individuals differ in their ability to self-regulate learning, because this ability is developed over a lifetime through social and other influences (Schunk and Zimmerman 1997). Users may understandably have differing self-efficacy beliefs regarding their ability to self-regulate learning and this could influence learning outcomes. Hence, we propose the following: Hypothesis 2B. In an e-learning-based IT training session, computer-learning self-efficacy will be positively associated with learning outcomes. Hypothesis 2C. In an e-learning-based IT training session, self-efficacy for self-regulatory learning will be positively associated with learning outcomes. Research Method To test our hypotheses, we chose a laboratory experiment as the suitable approach. We performed a pilot test before we conducted the experiment (see Appendix 1). Facilities, Target IT Application, and Participants Using an e-learning platform used in training employees and students, we conducted the experiment at a large public university. The IT training package provided a course on website design and development with Microsoft FrontPage software. In this course, learners proceeded through several modules as they underwent training in the conceptual elements and procedures used to set up websites. The training laboratory was similar to an IT training room, with 16 standalone computers in individual cubicles. Each participant, wearing a headset, completed the course independently, without interference from other learners. Training was interactive and delivered by audio and video components. The participant sequentially activated and completed the learning modules. For our experiment, we selected eight consecutive modules from the course, providing enough information content to develop a website with linked pages and internal and external links. Each screen within a course module displayed a typical FrontPage interface with a box that described the screen layout and menu options. The participant was asked to choose one of the menu options, to type text, and to execute appropriate actions. The participant viewed the results. If satisfied, the participant hit the forward button and proceeded to the next screen, but could move backward or forward within the module at any time. After a module was completed, the participant moved on to the next one. Undergraduate business students participated over four semesters and were given course credit for volunteering. Students who had web-design experience were excluded, but were given course credit through other means. We next describe our experimental procedures and then provide details on the interventions and measures used in this study.

9 34 Information Systems Research 19(1), pp , 2008 INFORMS Experimental Procedures Immediately after students volunteered for the study, they were given a background information survey that measured their individual learner traits and demographics (see Appendix 2). They were then assigned to one of the training sessions. Prior to starting the e-learning training, participants received either a treatment or control script. During the training session, they received feedback in the form of another treatment or control script. Thus, the combination of treatment or control scripts prior to and during training resulted in four different conditions: treatment-treatment (T 1 -T 2 ), treatment-control (T 1 -C 2 ), control-treatment (C 1 -T 2 ), and control-control (C 1 -C 2 ). We refer to the scripts provided prior to training as pretraining scripts and those provided during training as midpoint scripts (see Figures 3 and 4). At their scheduled training session, participants were randomly assigned to receive either the treatment (T 1 ) or the control (C 1 ) pretraining script. They were given eight minutes to read the pretraining scripts and then they completed manipulation check measures on motivation to learn and computerlearning self-efficacy beliefs, described below. Next, participants received training through four modules of the e-learning system. As a pretext to provide participants with feedback, we asked them to answer five easy-to-answer multiple-choice questions on website design. The purpose was not to determine participants learning scores, but to demonstrate that our feedback was based on their performance. Therefore, after completing four modules, participants answered these questions and received feedback from the midpoint script (treatment [T 2 ] or control [C 2 ], randomly assigned). After reading this script, they continued through the next four modules. At the end of the eight modules they were asked to answer questions testing their declarative knowledge. Next they were asked to complete a hands-on performance task (see Appendix 3): to develop a website in about 20 minutes (the time determined in the pilot study). We collected data through 16 experimental sessions. Participants spent an average of about 2 hours and 20 minutes completing the assignment: the background information survey took about 20 minutes, the learning session with the midpoint feedback took about 90 minutes, and the final testing through comprehension and development of a website took about 30 minutes. Interventions and Measures Background Information Survey. We created a survey that gathered background information (including demographics such as age, gender, and computer experience) and measured individual learner traits. Measures of learner traits included learning orientation, ( = 0 85), computer-learning self-efficacy measure ( = 0 92), both provided by Zweig and Webster (2004), and a self-efficacy for SRL measure ( = 0 87) from Zimmerman et al. (1992) (see Appendix 2). Pretraining Scripts. The pretraining treatment script was based on the self-regulation model provided by Schunk and Zimmerman (1998) (Figure 2). After pilot testing and modifications, the final pretraining treatment script (T 1 ) provided task analysis and learning goals for the session (see Figure 3a). We asked participants to take notes, pay attention, and stay focused. We also attempted to boost their motivational beliefs by influencing their self-efficacy beliefs and their outcome expectations from the learning exercise (e.g., with statements such as You are a very capable learner ). The pretraining control script (C 1 ) was about the same length to control for the amount of information presented to the participants (Webster and Martocchio 1995). It provided only general information on communication technologies (see Figure 3b). Midpoint Scripts. One group of participants received the midpoint treatment script (T 2 ), which evaluated their performance and asked them to focus on learning goals, to pay attention, and to monitor their learning progress, all elements of SRL. Consistent with other research on feedback (e.g., Martocchio and Webster 1992), all participants in this group received positive feedback, regardless of their actual performance, to control for differential effects. The other group received a midpoint control script (C 2 ) that did not provide any SRL-specific information or feedback (see Figures 4a and 4b, respectively).

10 Information Systems Research 19(1), pp , 2008 INFORMS 35 Figure 3 Pretraining Scripts (a) Pretraining Treatment Script (T 1 ) Welcome to this session where you will learn to design a website using Microsoft FrontPage. FrontPage is a user-friendly software, and you will find that designing a website is a simple and easy task. It is extremely important to learn the skills necessary to design a website because it is a critical skill that is highly valued by employers. By attending this session, you will learn the skills to set up your own personal website and show it to prospective employers. Your goal in this session is to learn methods and concepts needed to design a website, and then apply these skills to develop a simple website. You will learn through the computerized e-learning system at called. Remember that you must pay close attention and stay focused on the material presented on the screen. There will be many explanations, and the system will guide you in learning. You will find that it is easy to follow and understand. We are very confident that you will find that learning through this e-learning system is easy, and perhaps even more structured than learning from a classroom lecture. Several business students have used it before; they found it to be useful, and have given positive comments. Most important, by being part of the business college and taking business courses, you are among a select set of students in the university who have met the criteria to be admitted to attend business courses. It means you have a relatively higher GPA and are capable and smart. We therefore believe that you are a very capable learner and will be able to focus, pay attention, and understand the material conveyed to you. You have shown that you can set your goals for this session and apply yourself to the task of learning to design a website. We think that you will be successful in learning to use FrontPage. The information will be conveyed to you through eight simple, short, and consecutive modules. In the first module, you will understand what is meant by a website and learn about the different components in a website. Then, you will learn what the FrontPage screen looks like: i.e., the various menu options, the toolbars, and buttons on the screen. You will also learn how the application window is divided into different areas and the names/uses of these areas. In the subsequent modules, you will understand the notion of a template, create a webpage, and add more pages to the site. You will learn procedures to enter text on a webpage, format text, apply themes, and create hyperlinks to other webpages and websites. Each module is typically focused on one or two important aspects related to designing a website. Please pay attention to the material presented in each module and understand the main ideas. Note that it is very important for you to pay close attention and stay focused on your task. Please be patient; read and follow the instructions correctly. After every module, you should reflect and try to recall the main elements learned in the module. If necessary, visualize what you learned, recall the main operations, and think how you would execute it. For example, recall the main procedures you would follow to change the color of a page. Hence, you must reflect on what you learned and monitor your learning progress. It is critical to be very focused and understand everything that is being described. Because you are capable learners, we are confident that you will be able pay deep attention, process the information provided by the system, and monitor your learning progress. You can pace your own learning. Hence, use your own discretion and, if you need to, go back over the material. Please feel free to take notes as you would in any class and write down whatever you need. When you see a red arrow on the screen, it implies that you should point/choose the option indicated by this arrow. Because you are quite capable, we think you can manage your learning and that you will find the instructions fairly easy to follow. At the end of this session, you will have learned how to create a simple website using FrontPage. Enjoy and have a successful learning experience! (b) Pretraining Control Script (C 1 ) Welcome to this session where you will learn some fundamental issues relating to designing a webpage. The software that you will learn is Microsoft FrontPage. Today, you will learn through the computerized online learning system at the called. In this system, a computer e-learning system will teach and guide you in the procedures required to design a webpage. The is a typical technology-based training (TBT) software that is finding increased use in academic and business circles. The American Society for Training and Development (ASTD) defines technology-based training (TBT) as the delivery of content via Internet, LAN or WAN (intranet or extranet), satellite broadcast, audio- or videotape, interactive TV, or CD-ROM. TBT in turn is considered to encompass the dimensions of Computer Based-Training (CBT) and Web-Based-Training (WBT). CBT refers to the use of computers in the instruction and management of the teaching and learning process and includes both Computer Assisted Instruction (CAI) and Computer Managed Instruction (CMI). WBT refers to the delivery of educational content through a Web browser over the public Internet, a private intranet, or an extranet. Typically, they provide links to other learning resources such as references, , bulletin boards, and discussion groups. A related component of TBT is the concept of Distance Learning or Distance Education. ASTD defines Distance Learning as the educational situation in which the instructor and students are separated by time, location, or both. Courses are delivered to remote locations via synchronous or asynchronous means. The former refers to real-time, instructor-led learning with direct and simultaneous communication between participants. Associated technologies will include whiteboards, audio or videoconferencing, Internet telephony, or two-way live broadcasts. The latter has a time-lag component in the interaction between the instructor and the student. The popular self-paced courses taken via the Internet or CD-ROM, online discussion groups and are examples of the asynchronous mode. The use of technology in training spans three decades and can broadly be classified into three distinct phases. The initial phase was in the eighties where the primary driver was multimedia technology that provided rich audio, video, graphical, and animated content. The second phase in the nineties saw the development of the Internet as the primary vehicle for the electronic dissemination of training and learning. The development of efficient Web browsers, hyper-text markup languages, and media players propelled the Internet as a viable training medium. The third phase commenced at the turn of the century with the explosive increase in bandwidth access and availability.

11 36 Information Systems Research 19(1), pp , 2008 INFORMS Figure 3 (Cont d.) This, coupled with advances in Web-design technologies and high quality streaming media, enabled the development of real-time, low-cost, highly effective online training and learning environments. This is a typical software and we expect that at the end of this session, you will have understood some of the elements that go into creating a webpage using FrontPage. Please be patient, read the instructions, and follow them correctly. You can pace your own learning process. Hence, use your own discretion and go backwards if you need to. Please feel free to take notes as you would in any class and write down whatever you need to. When you see a red arrow on the screen, it implies that you should point/choose the option indicated by this arrow. Dependent Variables. Learning outcomes on IT software training are measured with a declarative knowledge test (i.e., a written test to evaluate learners conceptual understanding), and a hands-on taskperformance test that evaluates their procedural knowledge and ability to use the software (Yi and Davis 2003). In this study, the declarative knowledge test included multiple choice and fill-in-theblank questions that were pretested in the pilot study. The hands-on performance task required participants to develop a website. These tests were graded by a graduate student who was blind to the conditions and trained on this task (see Appendix 3). Manipulation Checks on SRL. Researchers have noted both the lack of standard instruments to assess whether participants follow SRL strategies and the problems with developing such instruments (e.g., Winnie and Perry 2000). In developing our instruments, we relied on Zimmerman s (1994) identification of four dimensions to SRL: motives, methods, performance outcomes, and social-environmental resources. The motives dimension deals with learners motivations to learn and complete the course. The methods dimension encompasses the application of SRL strategies. The performance outcomes dimension deals with the monitoring of learning goals and closely corresponds to metacognitive activities. The social-environmental resources dimension deals with learners access to peers and teachers who can provide guidance (this dimension was not applicable for our training context). We conducted two types of manipulation checks, one with our participants and the other with a separate group. We did not want to ask our participants directly about their self-regulatory learning strategies (such as their methods for learning or monitoring their learning goals) because doing so could Figure 4 Midpoint Scripts (a) Midpoint Treatment Script (T 2 ) Dear Student, Excellent effort and progress!!! You are making good progress toward the goal of learning to develop a website using FrontPage. This midpoint evaluation shows that you have understood the fundamental concepts of what constitutes a website, what is meant by a hyperlink, and how to present information on a webpage. Your scores are excellent compared with other students who have learned through this e-learning system. Because you are learning on your own without any human instructor, this evaluation shows that you are quite capable of paying attention to the material, understanding, learning, and monitoring your learning progress. Please continue to do so, and focus on this learning task. In the next few modules, you will continue to learn other aspects of designing a website such as learning to create internal and external links from your page and to apply themes to your website. Please continue with the good progress you have made so far, and remember to reflect on the content learned in each module. Continue to pay attention and stay focused on the task and you will achieve the goal of learning how to design a website. GOOD JOB!! (b) Midpoint Control Script (C 2 ) Dear Student, Web-based training (WBT) is an innovative approach to distance learning in which computer-based training (CBT) is transformed by the technologies and methodologies of the World Wide Web, the Internet, and intranets. Web-based training presents live content, as fresh as the moment and modified at will, in a structure allowing self-directed, self-paced instruction in any topic. WBT is media-rich training fully capable of evaluation, adaptation, and remediation, all independent of computer platform. Web-based training is an ideal vehicle for delivering training to individuals anywhere in the world at any time. Web browsers that support 3-D virtual reality, animation, interactions, chat, and conferencing, and real-time audio and video will offer many training opportunities. Instructional designers and training analysts are learning more about how to write and produce WBT. Therefore, in the future you will see a variety of WBT course offerings that will be distributed over the public Internet and private intranets. In the next few modules, you will learn more about webpage design.

12 Information Systems Research 19(1), pp , 2008 INFORMS 37 cue them to use these methods. Consequently, we measured only the motives dimension for our main study participants. Specifically, we twice measured their states of computer-learning self-efficacy (Zweig and Webster 2004) ( = 0 92) and motivation to learn (Hicks and Klimoski 1987) ( = 0 88): once at the beginning of training after the participants had read the pretraining scripts, and once during the training session (see Appendix 4A). We expected that these measures would indirectly indicate self-regulatory learning strategies because self-efficacy beliefs and motivation are described as being important components in the application of SRL strategies as per the social cognitive perspective. It is to be noted that these manipulation checks are state measures of computer learning self-efficacy and motivation to learn and may have quite different effects on learning performance than the corresponding trait measures that are considered to be relatively stable dispositional variables (George 1991). The second manipulation check was conducted on a separate group of participants. These were measured at the midpoint, before participants received the midpoint scripts. The results from this group were not included in the main study and, thus, we could assess the methods and performance dimensions without concern for cueing their behaviors. That is, these experimental sessions were conducted solely to perform manipulation checks on whether participants who received the treatment script (T 1 ) engaged in relatively more SRL methods and performance strategies than those who received the pretraining control script (C 1 ). For this separate group of participants, seven experimental sessions similar to the main study were conducted. After participants completed four modules, they were given the manipulation test questionnaire (see Appendix 4). Consistent with Schunk and Ertmer (1999), we developed this questionnaire based on Zimmerman s work. Specifically, we chose methods found to be highly significant in SRL (Zimmerman and Martinez- Pons 1986, 1988): organizing and transforming (student-initiated overt or covert rearrangements of instructional material to improve learning), and rehearsing and memorizing (student-initiated efforts to memorize material by overt or covert practice). Based on these descriptions, we adapted three items for each of these methods from Kanfer et al. (1994) and Pintrich and DeGroot (1990). To assess the performance outcomes dimension (student-initiated efforts to set specific goals and monitor progress), three items were adapted from Schunk and Ertmer (2000) and Schmidt and Ford (2003) (see Appendix 4B). Results Of the 134 participants who volunteered for the experiment, 10 were eliminated because of their prior Webdevelopment experience and six because of technical problems (their computers froze or their audio did not work). For the remaining 118 participants, the average age was 21.6, the average GPA was 3.3, and they were almost equally divided between males and females (see Table 1a). All had experience with popular software packages like Word and PowerPoint, but none had experience with Web-development software. As shown in Tables 1a and 1b, we found no significant demographic differences among the training groups. A factor analysis and validity checks of individual learner traits showed a high correlation between learning orientation and self-efficacy for self-regulation, and a lack of discriminant validity. Therefore, it was decided to drop the learning orientation measure and conduct the factor analy- Table 1a Demographic Information Demographic measures T 1 -T 2 T 1 -C 2 C 1 -T 2 C 1 -C 2 Overall Mean age (SD) 21 5 (1.7) 21 8 (2.0) 21 8 (1.7) 21 4 (1.5) 21 6 (1.7) Mean GPA (SD) 3 3 (0.4) 3 3 (0.4) 3 2 (0.4) 3 3 (0.5) 3 3 (0.4) Number of males Number of females Note. Treatment-treatment (T 1 -T 2 ), treatment-control (T 1 -C 2 ), controltreatment (C 1 -T 2 ), control-control (C 1 -C 2 ). Table 1b Tests of Demographic Differences Between Conditions Experimental conditions Demographic measures T 1 -T 2 vs. C 1 -C 2 T 1 -T 2 vs. T 1 -C 2 T 1 -T 2 vs. C 1 -T 2 p values of differences in means test Age GPA Note. Treatment-treatment (T 1 -T 2 ), treatment-control (T 1 -C 2 ), controltreatment (C 1 -T 2 ), control-control (C 1 -C 2 ).

13 38 Information Systems Research 19(1), pp , 2008 INFORMS Table 1c Factor Loadings Table 2a Manipulation Check: Motives Items Factor loadings Experimental conditions SR SR SR SR SR SR SR CSE CSE CSE CSE CSE SR SR CSE SR CSE SR Notes. Trait variables: CSE, computer-learning self-efficacy; SR, self-efficacy for self regulated learning. Those items in bold indicate items used for subsequent analysis. Table 1d Means, Standard Deviations, and Intercorrelations Cronbach s Mean SD alpha CSE SR HO DC CSE SR (0.80) HO NA NA DC NA NA Notes. Trait variables: CSE, computer-learning self-efficacy; SR, self-efficacy for self regulated learning. Dependent variables: HO, hands-on performance; DC, declarative knowledge; NA, not applicable. Diagonal values enclosed in parentheses indicate the square root of the AVE. Correlation is significant at the 0.05 level (two-tailed). Correlation is significant at the 0.01 level (two-tailed). sis again. In Table 1c, the factor analysis of individual learner traits of computer-learning self-efficacy and self-efficacy for self-regulation are provided. It is seen that seven of the 10 items in the measure of self efficacy for self-regulation loaded together whereas five of the seven items in the computerlearning self-efficacy scale loaded together and they had acceptable factor loadings. These seven and five items, respectively, were used in our final analysis. As seen in Table 1d, the measures of individual learner traits of computer-learning self-efficacy and self-efficacy for self-regulation satisfy tests of convergent and discriminant validity, with the square root of the average variance extracted (AVE) of each con- Mean score (SD) Differences Manipulation check Cronbach s Treatment Control in means measures alpha n = 61 (n = 57) (t-tests) After pretraining script Motivation to learn (0.57) 6.11 (0.81) p<0 03 Computer learning (0.59) 5.93 (0.77) p<0 11 self-efficacy After midpoint script Motivation to learn (0.66) 5.96 (0.90) p<0 03 Computer learning (0.72) 5.90 (0.79) p<0 13 self-efficacy Table 2b Manipulation Check: Methods Experimental conditions Mean score (SD) Differences Manipulation check Cronbach s Treatment Control in means measures alpha n = 16 (n = 17) (t-tests) Organizing and (0.78) 5.35 (0.83) p<0 04 transforming Rehearsing and (1.00) 4.84 (1.19) p<0 05 memorizing Note. Checks were conducted with a separate group of participants. Table 2c Manipulation Check: Performance Outcomes Experimental conditions Mean score (SD) Differences Manipulation check Cronbach s Treatment Control in means measures alpha n = 16 (n = 17) (t-tests) Performance outcomes (0.79) 5.33 (0.84) p<0 03 Note. Checks were conducted with a separate group of participants. struct being larger than the correlation between the constructs. The Cronbach alpha tests of reliability are also respectable. The results of our manipulation checks are shown in Tables 2a 2c. As shown in Table 2a, participants who received the pretraining treatment scripts scored higher on a state measure of computer-learning selfefficacy beliefs and motivation to learn, suggesting that our pretraining treatment scripts affected the motive dimension of SRL. Tables 2b and 2c illustrate that those participants who received the pretraining treatment script reported following a significantly greater amount of SRL strategies (both methods and performance outcomes) compared with those who

14 Information Systems Research 19(1), pp , 2008 INFORMS 39 received the pretraining control script. Note (as per theory) that all learners indicated following some level of SRL; however, it is the quality and extent of SRL that is important (and the participants in the treatment group indicated that they applied a relatively higher level of SRL). When we conducted tests of hypotheses, we found that the dependent measures of declarative knowledge and hands-on task performance were correlated (r = 0 45, p<0 0001). We first conducted an overall MANCOVA (in which the individual learner characteristics were entered as covariates). With a significant MANCOVA (Wilks Lambda = 0 887, p<0 05), we then conducted a Dunnet s t-test, a recommended test for simultaneous comparison of treatment condition scores (see Table 3). The mean dependent measure scores and standard deviations for each of the four conditions are shown in Table 4 and graphed in Figure 5. The scores for the treatment-treatment (T 1 -T 2 ) condition are the highest, followed by treatmentcontrol (T 1 -C 2 ), control-treatment (C 1 -T 2 ), and controlcontrol (C 1 -C 2 ) conditions. For Hypotheses 1A and 1B concerning the training conditions, the T 1 -T 2 condition differed significantly from the C 1 -C 2 condition. As shown in Tables 4 and 5a, the pretraining treatment script was effective in improving the learning outcomes. Hence, Hypothesis 1A was supported, suggesting that presession interventions would enhance learning outcomes. Furthermore, the midpoint scripts resulted in higher dependent measure scores for the treatment groups, supporting Hypothesis 1B as shown in Table 5b. It can be seen from Tables 5a and 5b that the differences in scores between the treatment and control groups are higher on the declarative knowledge test than the hands-on performance test, but both are significantly different. Table 3 Differences in Learning Outcomes for Experimental Conditions Experimental conditions T 1 -T 2 vs. T 1 -T 2 vs. T 1 -T 2 vs. Learning outcomes C 1 -C 2 T 1 -C 2 C 1 -T 2 Difference in means (p values) Declarative knowledge Hands-on performance We dropped the learning-orientation measure because of validity issues in relation to the other individual traits, so we could not test Hypothesis 2A. To test Hypotheses 2B and 2C concerning the effects of individual learner traits of computer learning selfefficacy and self-efficacy for self-regulation on learning outcomes, we examined the covariates in the MANCOVA and in individual ANCOVAs. These individual learner traits were not significantly related to the learning outcomes. 1 Thus, we did not find support for Hypotheses 2B and 2C concerning the effects of individual learner traits on learning outcomes. Discussion Based on Alavi and Leidner s (2001) framework, we examined the interaction of features of information technology, psychological learning processes, and instructional strategies embedded in an e-learningbased IT training environment. Because learners may not be exercising the high levels of self-directed learning strategies required in this training set-up, we developed instructional strategies that included interventions to induce learners to apply higher levels of self-directed learning strategies. Our tests showed that when the instructional strategy included such interventions that taught learners to self-regulate, learners applied more self-regulatory learning strategies, leading to enhanced learning outcomes. We also found that during the training session, providing positive feedback to learners on their use of SRL reinforces the use of SRL strategies, suggesting that learners must be reminded and motivated if they are to continue using SRL strategies. Results show that 1 We also conducted a hierarchical regression analysis using the T 1 -T 2 and C 1 -C 2 conditions. On each of the dependent variables (declarative knowledge and hands-on performance) we first regressed the treatment conditions, then the individual learner trait measures, and then the interaction terms between individual learner trait measures and treatment conditions. On declarative knowledge, the addition of the individual learner traits did not account for significant additional variance (R 2 change = 0 04, p<0 85), nor did the addition of the interaction terms (R 2 change = 0 05, p<0 19). Similarly, with hands-on performance, the addition of individual learner traits did not account for significant additional variance (R 2 change = 0 01, p<0 77), nor did the addition of the interaction terms (R 2 change = 0 01, p<0 80). Thus, the results do not indicate interactions between the individual learner traits and training interventions.

15 40 Information Systems Research 19(1), pp , 2008 INFORMS Table 4 Learning Outcomes by Condition Experimental conditions Treatment-treatment Treatment-control Control-treatment Control-control Learning outcomes (T 1 -T 2 ) n = 36 (T 1 -C 2 ) n = 25 (C 1 -T 2 ) n = 25 (C 1 -C 2 ) n = 32 Mean outcome score (SD) Declarative knowledge (Max = 15) 7.67 (2.50) 6.40 b (2.18) 5.76 a (2.31) 5.63 a (2.87) Hands-on performance (Max = 35) (4.14) (3.64) c (4.23) b (5.36) Note. Means with superscripts a, b, and c differ from the corresponding treatment-treatment (T 1 -T 2 ) condition at significance levels of 0.01, 0.05, and 0.1 respectively. the group instructed to follow SRL before and during training achieved the highest learning outcomes, whereas the group that received no instructions to self-regulate learning either before or during training received the lowest learning scores. Our results showed that all learners apply some level of SRL, but e-learning necessitates a higher level of SRL. Interventions like those examined in this study could induce learners to apply the higher levels of SRL needed in e-learning-based training contexts, thereby leading to higher learning outcomes. Based on past IS training research, we hypothesized that individual learner traits relating to SRL may influence learning outcomes. Our results did not support this premise. We found that the learning ori- Figure Graphs of Learning Outcomes Hands-on performance T 1 -T 2 T 1 -C 2 C 1 -T 2 C 1 -C 2 Declarative knowledge 4.0 T 1 -T 2 T 1 -C 2 C 1 -T 2 C 1 -C 2 Note. T 1 -T 2, Treatment-treatment; T 1 -C 2, treatment-control; C 1 -T 2, controltreatment; C 1 -C 2, control-control. entation and self-efficacy for self-regulation measures were correlated, perhaps because each of them taps into different forms of regulation. Although learning orientation taps into regulation relating to achievement motivation, and self-efficacy for self-regulation taps into academic motivation, they did not emerge as distinct measures and could not be tested simultaneously. We tested self-efficacy for self-regulation and computer-learning efficacy; these did not have direct or interactive effects on learning performance. One explanation can be found in the theory itself, which states that SRL is very situation specific and has to be activated in any given context, pointing to the importance of interventions. Learners who had higher self-efficacy beliefs for self-regulation and computer learning may not all have activated their self-efficacy beliefs in the new e-learning context. Another explanation for the lack of significant effects for individual learner traits can be found in the debates over the Table 5a Learning Outcomes for Pretraining Conditions Experimental conditions Differences Outcome measures for Treatment Control in means pretraining conditions n = 61 n= 57 (t-tests) Mean outcome score (SD) Declarative knowledge 7 15 (2.44) 5 68 (2.62) p<0 001 Hands-on performance (3.94) (4.87) p<0 03 Table 5b Learning Outcomes for Midpoint Conditions Experimental conditions Differences Outcome measures for Treatment Control in means midpoint conditions n = 61 n= 57 (t-tests) Mean outcome score (SD) Declarative knowledge 6 89 (2.59) 5 96 (2.60) p<0 03 Hands-on performance (4.21) (4.69) p<0 07

16 Information Systems Research 19(1), pp , 2008 INFORMS 41 role of individual traits and their effects on learning performance. Researchers have argued that there is a marked difference in the effects of the distal trait variables when compared to the proximal state variables of the same construct (George 1991). Our findings, along with other prior findings, signal conflicting findings on the relationship between trait-like individual differences and learning performance, suggesting that we must investigate more closely the influence of traitlike measures of individual differences relating to selfregulation and learning performance. Our findings that individual traits relating to SRL did not have an influence on learning outcomes, but that interventions did, point to the value of encouraging learners to follow SRL strategies when taking part in e-learning training. Results show that, unlike other IS training environments where individual learner traits play a relatively important role, in e-learningbased training environments, instructional strategies may play a more critical role than pre-existing differences in individual learner traits. Though the pretraining scripts were fairly straightforward and simple to implement, we found strong learning effects, consistent with other research manipulating pretraining interventions (e.g., Webster and Martocchio 1995). According to the social cognitive perspective, people engage in some degree of SRL, but what truly matters is the quality and extent to which they apply these strategies in a specific context. Messages such as those provided in the pretraining scripts seem to persuade the learner to apply SRL strategies in the given training situation. From an organizational standpoint, once an e-learning technology platform has been purchased and an IT course chosen as the target content, interventions in the instructional strategy such as those tested in this study may be the most suitable means to increase learning effectiveness. But it is also worthwhile to research whether persuasive messages such as those used in this study could be incorporated as part of the e-learning technology infrastructures so that learners can be encouraged to apply SRL strategies when they undergo e-learning courses. Prior IS research has concluded that the use of TMLbased methods for IT training may require the identification of special skill sets to achieve success (Piccoli et al. 2001). In this study, we identify and demonstrate that SRL is one such learner skill. This study advances the body of IT research on social cognitive theory (Compeau and Higgins 1995a, Johnson and Marakas 2000, Yi and Davis 2003) to encompass SRL. Therefore, our study helps contribute to a cumulative theoretical foundation for IT training both from a TML and a social cognitive research perspective. Implications for Research and Practice This study introduces the concept of self-regulation to IS research and shows that it is relevant to training programs. Our findings are based on a strong experimental design, with data collected over four time periods and training conditions crossed at the midpoint. As with any experimental study, however, this research has several limitations. We used selfreports on the use of SRL as a manipulation check rather than interviews, observation, or verbal protocols, because self-reports provide a method by which to compare the use of SRL among the groups. Furthermore, although research suggests that training outcomes should be evaluated using several dimensions, we focused only on learning outcomes due to our primary interest in improving the effectiveness of e-learning. Consequently, this research should be extended to other outcome variables. Another limitation is the use of student participants. Although we used college students as participants, our findings can be considered to be fairly realistic: the students were learning the topic for the first time and website design is an important skill for this group. This research, however, should also be extended to employees in the workplace. SRL may be more relevant to employee training because of its capacity to counteract work-related distractions; however, our study does not address desktopbased training, only training conducted in controlled environments. Reports indicate that many organizations, including the American federal government, use e-learning as the primary method to deliver technology training to employees (e.g., Hasson 2005) and research should be extended to this setting. We applied paper-based interventions to influence self-regulatory strategies, but we believe that these interventions could be designed as an integral part of e-learning platforms. There are several major players in the corporate IT e-learning market who offer

17 42 Information Systems Research 19(1), pp , 2008 INFORMS a variety of IT training courses covering hardware, software, programming languages, Web development, networking, and database administration, and also offer certification courses (e.g., Learning Steps 2007, Skill Soft 2007, Thomson Course Technology 2006). When we examined these training platforms, we found that they provide course content in a very procedural manner (i.e., they instruct the learner to operate commands and menu options one by one). They provide messages before the start of training that outline the course and inform the learners about the course objectives. Thus, it could be argued (as per Figure 2), that these systems provide learning goals for the training session but do not instruct the learner on any other aspect of SRL strategy. We believe that these commercial training tools could be designed such that they deliver persuasive messages to learners to improve e-learning outcomes. That is, one approach would be for vendors to add instructions such as those used in the study to the existing pretraining instructions that are provided to the learner. Several commercial tools also provide features such as text boxes for trainers to add their own instructions; another approach, therefore, would be for trainers to add messages similar to those used in this study. These training tools could also provide feedback during the training session as was done in this study. Thus, if these commercial tools could deliver, through the technology component of the e-learning platform, similar messages to those we provided via paper-based scripts, we believe it could lead to more SRL and better learning outcomes. Future research should examine the software, programming, and other technical issues in the design of e-learning tools that can provide such messages at the beginning and during the training session, and also identify how they could be personalized to each learner. One avenue for future research on SRL is to determine if users SRL skills could be developed via training programs. After users are trained to use SRL skills, users could participate in e-learning, and researchers could then determine whether the learners applied their newly developed skills. If they did not, what situational reminders might be needed? Although we specifically focused on IT training in this study, the proposed interventions could be useful in other e-learning-based training courses as well. This study indicates that learners benefit by self-regulating their learning in new situations and, therefore, an interesting corollary would be to examine whether the extent of use of self-regulatory strategies may explain some of the conflicting training outcomes observed in prior IS training research (e.g., Davis and Weidenbeck 1998, Olfman and Mandviwalla 1994b, Santhanam and Sein 1994). Another avenue for future research is to utilize Alavi and Leidner s (2001) framework in other contexts. In this study, we applied the framework to one specific TML environment e-learning but it could be applied in a similar manner to improve learning outcomes in other TML environments. For example, in collaborative TML environments, the expected learning process involves cooperative learning with other learners. But research findings indicate that not all learners use the communication tools to cooperatively learn and improve learning outcomes in these TML environments. Cooperative learning theories suggest that learners can learn effectively and improve learning outcomes only if instructional strategies include exercises such as group projects and case studies that require a high degree of interdependence and learner interaction (Cohen 1994). IS research could therefore determine the appropriate conditions, i.e., the types of IT group projects, and other modifications to instructional strategies that can induce learners to use the communication features in collaborative learning environments, not only to communicate, but also to interact share ideas, and thus develop higher learning outcomes. Our finding that individual trait differences relating to self-regulatory skills did not significantly affect learning performance opens new research avenues. Researchers could extend the examination of the relationship between traits and learning performance by using state measures (e.g., state anxiety) that might play a mediating role (Chen et al. 2000). Particularly in learning and training contexts, relationships between trait-like individual-difference constructs and learning performance seem to vary across performance episodes, with positive effects in some cases but not in others. For example, researchers have found a mixed relationship between performance orientation and the regulation of cognitive learning strategies. Some findings suggest no relationship (e.g., Ford et al. 1998,

18 Information Systems Research 19(1), pp , 2008 INFORMS 43 Middleton and Midgley 1997) and others suggest an interactive relationship (Schmidt and Ford 2003). A more detailed theorized examination of relationships among more distal traits, learning performance, and more proximal state-like constructs is necessary (Chen et al. 2000). Furthermore, as described earlier, self-regulatory competencies, such as academic writing and mathematical reasoning, may differ across domains (Schunk and Zimmerman 1997, Zimmerman and Kitsantas 1997). Therefore, another avenue for research is to determine whether self-regulatory traits for e-learning tasks differ from other academic selfregulatory skills. If so, could we develop a measure to test these traits and determine which employees can or cannot learn effectively from e-learning training programs? Our study informs IT training practice. Almost every employee needs IT skills, and when new IS are implemented, corporations increasingly conduct training via e-learning. Unfortunately, e-learning-based IT training has yielded disappointing results. Based on our findings, organizations might use simple interventions before and during training to improve learning outcomes. For those vendors designing e-learning infrastructures, our results suggest that these interventions could be imbedded in the technology platform to make learning more effective. Finally, based on social cognitive theory, our study findings alert IS researchers to an important concept that could be pivotal in explaining the learning and use of information systems: self-regulation. Acknowledgments This work was supported in part by an award to the first author from the Kentucky Science and Engineering Foundation as per Award Agreement #KSEF with the Kentucky Science and Technology Corporation. The authors thank the reviewers and the associate and senior editor for the many insightful comments that helped to improve this research study and shape this paper. They also benefited from feedback provided by Adrienne Olnick Kutzschan of Queen s University and research seminar participants from Temple University and Arizona State University. Appendix 1. Description of the Pilot Test After a few tests to check that the infrastructure was working and that typical students could interact with the e-learning system, we conducted our pilot test. For the pilot test, we developed pretraining treatment and control Table A.1 Pilot Study Results on Impact of Pretraining Interventions Training outcomes with pretraining treatment scripts Training outcomes with pretraining control scripts Mean outcome score (SD) 9 (1.84) 8.08 (1.82) Observations Differences in means (t-test) p<0 02 scripts. The pretraining treatment script encouraged participants to follow self-regulatory strategies, whereas the control script provided general information on communication technologies. We also measured participants individual differences in goal orientation and self-efficacy beliefs. We recruited participants who were randomly assigned to a group that received either the pretraining treatment script or the pretraining control script. When participants reported for training, they completed a background questionnaire that measured their traits. Participants then took the e-learning course on FrontPage, which consisted of four modules. We measured their learning outcomes with a conceptual understanding test. Table A.1 illustrates that the group that received the pretraining treatment script exhibited higher learning outcomes. Based on these pilot results and feedback from participants, we elected to change our experimental materials and procedures. First, we modified and fine-tuned the questionnaires and scripts. We determined that if we were going to provide midpoint feedback, participants must undergo training with eight course modules, not four. We modified the pretraining treatment scripts to be clearer, more specific, and to reflect the goals for these eight modules. We developed midpoint scripts. We conducted Pearson correlation tests between the individual traits and the comprehension scores. Although the correlation between computer learning self-efficacy and the training outcome score was positive (Pearson correlation = 0.110, p = 0 53), it was not significant. The correlation between self-efficacy for self-regulatory learning and learning outcomes was also not significant (Pearson correlation = 0.09, p = 0 59). We felt that we may not have seen significant correlations because our sample size was relatively small. Also, because all the participants were of similar backgrounds and experience, their scores on these measures showed low variance. We also decided that to measure individual differences in traits we must administer the background questionnaire well before the training session (i.e., when participants volunteered for the experiment). We felt that this timing would be a better measure of their trait variables. Therefore, we made these changes and planned to collect a larger set of data for the main experiment. Appendix 2. Questionnaire Items for Measuring Individual Traits (Anchored on seven-point Likert scales; anchors: strongly disagree to strongly agree.)

19 44 Information Systems Research 19(1), pp , 2008 INFORMS Learning Orientation 1. The opportunity to learn new things is important to me. 2. The opportunity to do challenging work is important to me. 3. I prefer to work on tasks that force me to learn new things. 4. If I don t succeed on a difficult task, I plan to try harder the next time. 5. In learning situations, I tend to set fairly challenging goals for myself. 6. I am always challenging myself to learn new concepts. 7. The opportunity to extend my range of abilities is important to me. Computer Learning Self-Efficacy 1. I feel confident using a computer to learn about and apply new concepts. 2. Using a computer is an efficient way for me to learn new things. 3. I could apply new concepts that I learned from a computerized training program. 4. I don t feel that I could learn new skills from a computerized training program. 5. It would be easy for me to become skillful at tasks learned from a computerized training program. 6. I would be comfortable using a computerized training program. 7. I could successfully use a computerized training program. Self-Efficacy for Self-Regulation 1. I am able to finish homework assignments by deadlines. 2. I am able to study even when there are other interesting things to do. 3. I am able to concentrate on school subjects. 4. I am able to take class notes of class instruction. 5. I am able to use the library and the internet for information for class assignments. 6. I am able to plan my schoolwork. 7. I am able to organize my schoolwork. 8. I am able to remember information presented in class and textbooks. 9. I am able to arrange a place to study at my residence/home without distractions. 10. I am able to motivate myself to do schoolwork. 11. I am able to participate in class discussions. This item was reverse coded. Appendix 3A. Declarative Knowledge Test Questions (1) The HTML button is used for: 1. converting the text material to source code 2. displaying how a page will appear in a browser 3. generating text material from the source code 4. ensuring validation of JavaScript with source code 5. none of the above (2) FrontPage creates a homepage called Index.htm and two associated folders called Images and. (3) Which of the following enables you to create a new webpage from a template? 1. page templates 2. work pane 3. template bar 4. content pane 5. task pane (4) When a hyperlink is created, the destination of the hyperlink is encoded as a. (5) Bookmark links can connect a page in a website to a location on: 1. a page in another website 2. a page in the same website 3. the same page 4. 2 and , 2, and 3 (6) In FrontPage, bookmark links are displayed using a. (7) A navigation bar is a set of text or button hyperlinks typically used to: 1. access pages in another website 2. test the links in a website 3. access pages in the same website 4. give a pictorial view of the links 5. none of the above (8) The navigation bar uses the as labels for the hyperlink. (9) To ensure that the navigation bar contains hyperlinks to only the webpages linking to that webpage, which of the following Link Bar Properties options should be selected: 1. parent level 2. global level 3. child pages under global level 4. child level 5. none of the above (10) The same formatting can be applied to other text on the same page using the option. Appendix 3B. Web Development Activities and Scores (Maximum of 35 points) Now you are ready to design your own website. Please click on Start at the bottom left corner of your screen, and select Programs, Microsoft FrontPage. This will open Microsoft FrontPage for you. Please design a simple website based on the activities listed below. You have 20 minutes to complete this exercise. Activity 1 (Maximum of 11 points) Create a personal home page named index.htm or default.htm; 2 points. Enter 3 4 lines of text related to you containing the phrases College of Business and hobbies. [e.g., My name is XYZ and I am an accounting student at the College of Business. My hobbies include reading, swimming and hiking ]. 3 points.

20 Information Systems Research 19(1), pp , 2008 INFORMS 45 Format the text in Times New Roman. 1 point. Provide the text material with the heading My Home Page formatted in bold, Arial. 2 points. Apply a theme to this webpage. 3 points. Activity 2 (Maximum of 9 points) Create a webpage named hobbies.htm. 1 point. Enter 3 4 lines of text related to your hobbies. 1 point. Format the text in Times New Roman. 1 point. Provide the text material with the heading My Hobbies formatted in bold, Arial. 2 points. At the very end of the text material, on a new line, enter the phrase To Home Page. 1 point. Apply a suitable theme to this webpage. 3 points. Activity 3 (Maximum of 15 points) From your home page, convert the phrase College of Business into an external hyperlink pointing to 5 points. From the same webpage, convert the phrase hobbies into an internal hyperlink pointing to your hobbies page. 5 points. From you hobbies webpage, convert the phrase To Home Page into an internal hyperlink pointing to your home page. 5 points. Appendix 4A. Questionnaire Items for Manipulation Checks (Anchored on 7-point Likert scales; anchors: strongly disagree to strongly agree.) Motives Measures of SRL (Used with Main Study Participants) Computer Learning Self-Efficacy 1. In this session, I feel confident in using a computer to learn about and apply new concepts. 2. Using a computer in this session is an efficient way for me to learn new things. 3. I could apply new concepts that I will learn from this computerized online training program. 4. I don t feel that I could learn new skills from this computerized training program. 5. It would be easy for me to become skillful at tasks learned from this computerized training program. 6. I am comfortable using a computerized training program in this session. 7. I could successfully use a computerized training program such as the one in this session. Motivation to Learn 1. In this session I will exert considerable effort in learning the material on FrontPage. 2. I am motivated to learn the material on FrontPage presented in this session. 3. I am trying to learn as much as I can about FrontPage from this session. This item was reverse coded. Appendix 4B Additional Manipulation Check Measures Methods Measures of SRL (Used with a Separate Group of Participants) Organizing and Transforming 1. I tried to put important ideas into my own words. 2. I used what I have learned in the past to try to succeed at learning the material. 3. I tried to connect information to things that I already know. Rehearsing and Memorizing 1. I tried to recall the key ideas from earlier in the tutorial. 2. I went over things I had learned in prior modules. 3. I worked on the Web-design procedures as they were being presented even though I did not have to. Performance Measures of SRL (Used with a Separate Group of Participants) 1. I tried to think through each topic and decide what I was supposed to learn. 2. I was aware of the progress of my learning with respect to my goals for this session. 3. I thought about what things I needed to do to learn. References Agarwal, R., T. W. Ferratt Crafting an HR strategy to meet the need for IT workers. Comm. ACM 44(7) Agarwal, R., V. Sambamurthy, R. Stair Research report: The evolving relationship between general and specific computer self-efficacy An empirical assessment. Inform. Systems Res. 11(4) Alavi, M Computer-mediated collaborative learning: An empirical evaluation. MIS Quart. 18(2) Alavi, M., D. Leidner Research commentary: Technology mediated learning A call for greater depth and breadth of research. Inform. Systems Res. 12(1) Alavi, M., B. Wheeler, J. Valacich Using IT to reengineer business education: An exploratory investigation of collaborative tele-learning. MIS Quart. 19(3) Alavi, M., G. M. Marakas, Y. Yoo A comparative study of distributed learning environments on learning outcomes. Inform. Systems Res. 13(4) Allen, I., J. Seaman Growing by Degrees: Online Education in the United States, The Sloan Consortium, Needham, MA. Anohina, A Analysis of the terminology used in the field of virtual learning. Educational Tech. Soc. 8(3) Bandura, A Social cognitive theory of self-regulation. Organ. Behav. Human-Decision Processes 50(2) Bandura, A Self-Efficacy: The Exercise of Control. W. H. Freeman, New York. Beaubien, J. M., S. C. Payne Individual goal-orientation as a predictor of job and academic performance: A meta analytic review and integration. Proc. 14th Annual Meeting of The Society for Indust. and Organ. Psych., SIOP, Bowling Green, OH. Bell, B. S., S. W. Kozlowski Adaptive guidance: Enhancing self-regulation, knowledge, and performance in technologybased training. Personnel Psych. 55(2)

21 46 Information Systems Research 19(1), pp , 2008 INFORMS Benbunan-Fich, R Improving education and training with IT. Comm. ACM 45(6) Benbunan-Fich, R., S. R. Hiltz Mediators of the effectiveness of online courses. IEEE Trans. Professional Comm. 46(4) Berge, Z. L., S. Mrozowski Review of research in distance education: 1990 to Amer. J. Distance Ed. 15(3) Bernard, R. M., P. C. Abrami, Y. Lou, E. Borokhowski, A. Wade, L. Woznoy, P. A. Wallet, M. Fiset, B. Huang How does distance education compare with classroom instruction? A meta-analysis of the empirical literature. Rev. Educational Res. 74(3) Boekaerts, M., P. R. Pintrich, M. Zeidner Handbook of Self- Regulation. Academic Press, San Diego, CA. Bostrom, R Facilitating learning through technology. Proc. 9th Americas Conf. Inform. Systems. Assoc. Inform. Systems, Atlanta. Bostrom, R., L. Olfman, M. K. Sein The importance of learning style in end-user training. MIS Quart. 14(1) Brett, J., D. VandeWalle Goal orientation and goal content as predictors of performance in a training program. J. Appl. Psych. 84(6) Brown, K Using computers to deliver training: Which employees learn and why? Personnel Psych. 54(2) Bureau of Labor Statistics BLS releases employment projections. t04.htm. Chen, G., S. M. Gully, J. Whiteman, R. N. Kilcullen Examination of relationships among trait-like individual differences, state-like individual differences, and learning performance. J. Appl. Psych. 85(6) Cohen, E. G Restructuring the classroom: Conditions for productive small groups. Rev. Educational Res. 64(1) Colquitt, J., J. LePine, R. Noe Toward an integrative theory of training motivation: A meta-analytic path analysis of 20 years of research. J. Appl. Psych. 85(5) Compeau, D., C. A. Higgins. 1995a. Application of social cognitive theory to training for computer skills. Inform. Systems Res. 6(2) Compeau, D., C. A. Higgins. 1995b. Computer self-efficacy: Development of a measure and initial test. MIS Quart. 19(2) Compeau, D., L. Olfman, M. K. Sein, J. Webster End-user training and learning. Comm. ACM 38(7) Coppola, N., S. Hiltz, N. Rotter Becoming a virtual professor: Pedagogical roles and asynchronous learning networks. J. Management Inform. Systems 18(4) Davis, S., S. Wiedenbeck The effect of interaction style and training method on end-user learning of software packages. Interacting with Comput. 11(2) Fisher, S., J. K. Ford Differential effects of learner effort and goal orientation on two learning outcomes. Personnel Psych. 51(2) Ford, J. K., E. M. Smith, D. A. Weissbein, S. M. Gully, E. Salas Relationships of goal orientation, metacognitive activity, and practice strategies with learning outcomes and transfer. J. Appl. Psych. 83(2) Gagne, R. M The Conditions of Learning. Holt, Rinehart, and Winston, New York. Gagne, R. M., L. J. Briggs, W. W. Wager Principles of Instructional Design, 4th ed. Hartcourt Brace Jovanich, Fort Worth, TX. George, J. M State or trait: Effects of positive mood on prosocial behaviors at work. J. Appl. Psych. 76(2) Gupta, S., R. Bostrom Research framework for collaborative e-learning in an end-user training context. Proc. 11th Americas Conf. Inform. Systems. Assoc. Inform. Systems, Atlanta. Hardaway, D. E., R. W. Scamell Use of a technology-mediated learning instructional approach for teaching an introduction to information technology course. J. Inform. Systems Ed. 16(2) Hasson, J E-learning: A progress report. Federal Computer Week, FCW Media Group, Falls Church, VA, August Hicks, W. D., R. J. Klimoski Entry into training programs and its effects on training outcome: A field experiment. Acad. Management J. 30(3) Homer, J., D. Povar The 2004 state of the industry report: Press release from the American Society for Training and Development. Training and Development (March). Johnson, R. D., G. M. Marakas The role of behavioral modeling in computer skills acquisition: Toward refinement of the model. Inform. Systems Res. 11(4) Jonassen, D. H., ed Handbook of Research on Educational Communications and Technology. Lawrence Erlbaum Associates, Mahwah, NJ. Jonassen, D. H., T. C. Reeves Learning with technology: Using computers as cognitive tools. D. H. Jonassen, ed. Handbook of Research for Educational Communications and Technology. Macmillan, New York, Jones, T. H., R. Paolucci Research framework and dimensions for evaluating the effectiveness of educational technology systems on learning. J. Res. Comput. Ed. 32(1) Kanfer, R. A., L. Philip, T. C. Murtha, B. Dugdale, L. Nelson Goal setting, conditions of practice, and task performance: A resource allocation perspective. J. Appl. Psych. 79(6) Kauffman, D Self-regulated learning in web-based environments: Instructional tools designed to facilitate self-regulated learning. J. Educating Comput. Res. 30(1/2) Keegan, D Foundations of Distance Education, 3rd ed. Routledge, London, UK. Learning Steps Online computer training courses and distance learning tutorials. courses.php. Leidner, D., M. Fuller Improving student processing and assimilation of conceptual information: GSS supported collaborative learning vs. individual constructive learning. Decision Support Systems 20(2) Leidner, D., S. Jarvenpaa The use of information technology to enhance management school education: A theoretical view. MIS Quart. 19(3) Lim, K. H., L. M. Ward, I. Benbasat An empirical study of computer system learning: Comparison of co-discovery and self-discovery methods. Inform. Systems Res. 8(3) Lou, Y., P. C. Abrami, S. D Apollonia Small group and individual learning with technology: A meta-analysis. Rev. Educational Res. 71(3) Markus, L. M Innovations in information systems education. Paper presented as part of the Special Interest Group on Education. Proc. 11th Americas Conf. Inform. Systems. Assoc. Inform. Systems, Atlanta. Martocchio, J. J., J. Webster Effects of feedback and cognitive playfulness on performance in microcomputer software training. Personnel Psych

22 Information Systems Research 19(1), pp , 2008 INFORMS 47 Middleton, M. J., C. Midgley Avoiding the demonstration of lack of ability: An underexplored aspect of goal theory. J. Educational Psych. 89(4) Moller, L Introduction to the special issue of online training in an online world. Quart. Rev. Distance Ed. 3(1) 7 8. Noffsinger, J. S Correspondence Schools, Lyceums, Chautauquas. Macmillan, New York. Olfman, L., M. Mandviwalla. 1994a. Conceptual vs. procedural software training for graphical user interfaces: A longitudinal field experiment. MIS Quart. 18(4) Olfman, L., M. Mandviwalla. 1994b. An experimental analysis of software training manuals. Inform. Systems J. 5(1) Olfman, L., R. P. Bostrom, M. K. Sein Developing training strategies with an HCI perspective. D. Galletta, P. S. Zhang, eds. Human-Computer Interaction and Management Information Systems: Applications. M. E. Sharpe, Armonk, NY, Piccoli, G., R. Ahmad, B. Ives Web-based virtual learning environments: A research framework and a preliminary assessment of effectiveness in basic IT skills training. MIS Quart. 25(4) Pintrich, P., E. DeGroot Motivational and self-regulated learning components of classroom academic performance. J. Educational Psych. 82(1) Richardson, J. C., K. Swan Examining social presence in online courses in relation to students perceived learning and satisfaction. J. Asynchronous Learn. Networks 7(1) Rossett, A., L. Schafer What to do about e-dropouts: What if it s not e-learning but the e-learner? Training and Development (June) Salas, E., J. A. Canon-Bowers The science of training: A decade of progress. Annual Rev. Psych. 52(10) Santhanam, R., M. Sein Improving end-user proficiency: Effects of conceptual training and task variation. Inform. Systems Res. 5(4) Sasidharan, S., R. Santhanam Technology-based training: Toward a learner centric research agenda. D. Galletta, P. Zhang, eds. Human-Computer Interaction and Management Information Systems: Applications. M. E. Sharpe, Armonk, NY, Schmidt, A. M., J. K. Ford Learning within a learner control training environment: The interactive effects of goal orientation and meta-cognitive instruction on learning outcomes. Personnel Psych. 56(2) Schunk, D. H Social cognitive theory and self-regulated learning. B. Zimmerman, D. Schunk, eds. Self-Regulated Learning and Academic Achievement: Theoretical Perspectives. Lawrence Erlbaum Associates, Hillsdale, NJ, Schunk, D. H., P. A. Ertmer Self-regulatory processes during computer skill acquisition: Goal and self-evaluative influences. J. Educational Psych. 91(2) Schunk, D. H., P. A. Ertmer Self-efficacy and academic learning: Self-efficacy enhancing interventions. M. Boekaerts, P. R. Pintrich, M. Zeidner, eds. Handbook of Self-Regulation. Academic Press, San Diego, CA, Schunk, D., B. Zimmerman Social origins of self-regulatory competence. Educational Psychologist 32(4) Schunk, D., B. Zimmerman Self-Regulation of Learning and Performance: Issues and Educational Applications. Lawrence Erlbaum Associates, Hillsdale, NJ. Skill Soft About us. asp. Thomson Course Technology Teach with technology. VandeWalle, D Development of a work domain goal orientation instrument. Educational Psych. Measurement 57(6) Venkatesh, V Creation of favorable user perceptions: Exploring the role of intrinsic motivation. MIS Quart. 23(2) Webster, J., J. J. Martocchio Microcomputer playfulness: Development of a measure with workplace implications. MIS Quart. 16(2) Webster, J., J. J. Martocchio The differential effects of software training previews on training outcomes. J. Management 21(4) Winnie, P. H Inherent details in self-regulated learning. Educational Psychologist 30(4) Winnie, P. H., N. E. Perry Measuring self-regulated learning. M. Boekarts, P. Pintrich, M. Zeidner, eds. Handbook of Self- Regulation. Academic Press, San Diego, CA, Yi, M. Y., F. D. Davis Developing and validating an observational learning model of computer software training and skill acquisition. Inform. Systems Res. 14(2) Zhang, D., J. L. Zhao, L. Zhou, J. F. Nunamaker Can e-learning replace classroom learning? Comm. ACM 47(5) Zimmerman, B. J A social cognitive perspective of self-regulated academic learning. J. Educational Psych. 81(3) Zimmerman, B. J Dimensions of academic self-regulation: A conceptual framework for education. D. Schunk, B. J. Zimmerman, eds. Self-Regulation of Learning and Performance, Issues and Educational Applications. Lawrence Erlbaum Associates, Hillsdale, NJ, Zimmerman, B. J Self-regulation involves more than metacognition: A social cognitive perspective. Educational Psychologist 30(4) Zimmerman, B Attaining self-regulation: A social cognitive perspective. M. Boekarts, P. Pintrich, M. Zeidner, eds. Handbook of Self-Regulation. Academic Press, San Diego, CA, Zimmerman, B. J., A. Kitsantas Developmental phases in self-regulation: Shifting from process goals to outcome goals. J. Educational Psych. 89(1) Zimmerman, B. J., M. Martinez-Pons Development of a structured interview for assessing student use of self-regulated learning strategies. Amer. Educational Res. J. 23(4) Zimmerman, B. J., M. Martinez-Pons Construct validation of a strategy model of student self-regulated learning. J. Educational Psych. 80(3) Zimmerman, B. J., M. Martinez-Pons Student differences in self-regulated learning: Relating grade, sex, and giftedness to self-efficacy and strategy use. J. Educational Psych. 82(1) Zimmerman, B. J., D. H. Schunk Theories of self-regulated learning and academic achievement: An overview and analysis. B. J. Zimmerman, D. H. Schunk, eds. Self-Regulated Learning and Academic Achievement: Theoretical Perspectives. Lawrence Erlbaum Associates, Hillsdale, NJ, Zimmerman, B. J., A. Bandura, M. Martinez-Pons Selfmotivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. Amer. Educational Res. J. 29(3) Zweig, D., J. Webster Validation of a multidimensional measure of goal orientation. Canadian J. Behavioral Sci. 36(3)