Development of Competency-Based Web Learning Material and Effect Evaluation of Self-Directed Learning Aptitudes on Learning Achievements

Transcription

1 Interactive Learning Environments Vol. 14, No. 3, December 2006, pp Development of Competency-Based Web Learning Material and Effect Evaluation of Self-Directed Learning Aptitudes on Learning Achievements Chi-Cheng Chang* National Taipei University of Technology, Taiwan ROC This study aims to develop and evaluate competency-based web learning material (CBWLM) for the college practicum Microprocessor Laboratory. After using the CBWLM for 8 weeks, this study investigates CBWL s learning effects and self-directed learning aptitudes (SDLAs) as well as exploring the influence of SDLA on learning effects based on the sample of 38 students. The results of this study indicate that over half of the students achieve the mastery level after using CBWLM. SDLAs of the mid-cbwl and post-cbwl do not influence learning effects. Research Background Competency-based learning (CBL), a non-linear, systematic, and self-learning process, allows learners to individually study each unit of the learning material depending on their own competency. Furthermore, it allows learners to perform repeated learning, remedial learning until they master each unit s knowledge and skill in order to achieve required competency standards. CBL possesses the nature of systematic process, mastery learning, emphasizing individual differences and selfdirected learning. Such a learning method stressing competency standards is specifically suitable for technical learning and training. Web learning is an inevitable trend in the future. However, it has a weakness there are limitations to the presentation and learning methods for technical subjects. If we apply the concept of CBL to web learning, both will complement each other and improve web technical learning effects. As indicated by Lin and Liao (2000), the concept of competencybased learning material (CBLM) can enhance the effect of web learning materials. To sum up the argument mentioned above, it s quite clear that competency-based web learning (CBWL) or competency-based web learning material (CBWLM) ought to *Institute of Technological and Vocational Education, National Taipei University of Technology, No. 1, Section 3, Chung-Hsiao E. Road, Taipei, Taiwan f10980@ntut.edu.tw ISSN (print)/issn (online)/06/ Ó 2006 Taylor & Francis DOI: /

2 266 C.-C. Chang have a certain level of demand and feasibility. As there are few studies combining the practices regarding CBL and web learning, CBWL or CBWLM becomes a topic worthy of studying. Westera and Sloep (1998) developed so-called The Virtual Company, a distributed distance learning environment built upon the notions of CBL and constructivist learning, but the system seems not to bear supporting mechanisms of CBL. Lin (2002) developed the web learning material of the competency-based Digital Logic course in the vocational senior high school by taking an ordinary web learning system as a platform where CBLM was placed. However, there is no control mechanism for CBL processes between each unit of the teaching material, which seems not to conform to the nature of competency-based individual self-learning. Ho, Hsu, and Tao (1999) referred to CBL in their web learning material, but the material does not appear to match the concept and spirit of CBL. Most studies in the world simply propose the idea of applying the competency-based concept in web learning initially; no CBWLM s substantial output is produced. Kirschner (2000), for instance, discussed what instructional designers (ID) need to do when designing electronic CBL environments. Woelk (2002) delivered a conception of a competency-based e-learning just-in-time system; Ritzen and Kosters (2002) also proposed an idea of a competency-based web-based portfolio system. Kirschner (2000) further argued that no full-fledged ID models are yet available to provide clear prescriptions for how to design electronic CBL environments. Therefore, design and development of electronic CBL environments may become highly cost-ineffective. For this reason, there are some discussible issues: What on earth should a CBWLM consist of? And, what are its learning effects? Theses study results can provide references for the design and implementation of CBWLM, and will help enhance the effects of CBWL. Elshout-Mohr, Oostdam, Snoek, and Dietze (2000) stated, CBL provides learners with opportunities to regulate and manage their own learning. Wielenga (2000) argued, competency-oriented learning should offer students an environment in which they are given the responsibility to monitor their own learning processes. These arguments imply self-directed learning is the characteristic of CBL. Hanna, Dudka, and Runlee (2000) argued that students are required to possess self-directed learning characteristics for successful web learning. This implies that web learning accentuates active and self-directed learning. Richards, Dooley, and Lindner (2004) conceived that web learning courses draw on the ability of learners to be self-directed, therefore, web courses should focus on the self-directed nature of learners. Learners of e-learning must experience self-directed learning (Ghaoui & Janvier, 2004; Ryan, Scott, Freean, & Patel, 2000). Conrad and Donaldson (2004) argued that a learner on the web must establish comfort with a higher level of self-direction than in a traditional classroom. They further asserted that the learner would quit the online course in frustration if this comfort level was not reached. Self-directed learning aptitude (SDLA), derived from Guglielmino (1977), is a learning behaviour allowing students to continue learning on their own initiative. The SDLA consists of six factor aspects: Effective Learning, Fondness for Learning, Learning Motivation, Active Learning, Independent Learning, and Creative

3 Competency-Based Web Learning Material 267 Learning. Because web learning is becoming more prevalent in colleges, an issue thus arises. What is the SDLA generated by college students through CBL on the web? Moreover, Hanna et al. (2000) argued that assignments, projects, tests, and activities might help students consider the important self-direction elements of online learning. It means that assignments and activities may be the incitements to self-directed learning behaviours. Thus, the other issue arising is that whether or not SDLA reflects on the outcomes of assignments, projects, or tests. Namely, does SDLA influence any learning effect? Are learning effects different among students with varied SDLA? They are all valuable issues for further discussion. Research Purposes Based on the referred research background and motivation, this research aims to develop a CBWLM for the practicum Microprocessor Laboratory of the Electronic Department in college as well as to evaluate CBWL s learning effects and SDLAs, and to probe the influence of SDLA on learning effects with an empirical study. Detailed research objectives associated with their corresponding research questions and statistical methods are shown in Table 1. The learning effect referred to in this study constitutes two aspects, the selfperceived learning effect and the outcome of an achievement assessment, derived Table 1. Research objectives, research questions and statistical method Research objectives Research questions Statistical method 1. To evaluate learning effects of CBWLM. 2. To evaluate SDLAs for different stages of CBWL. 3. To explore influences of SDLA on learning effects for CBWL What are the self-perceived learning effects after students have used CBWLM? 1.2. What are the outcomes of achievement assessment after students have used CBWLM? 2.1. What is the SDLA of students before CBWL? 2.2. What is the SDLA of students during CBWL? 2.3. What is the SDLA of students after CBWL? 3.1. Is there any statistically significant difference in learning effects among the students with various SDLAs during CBWL? 3.2. Is there any statistically significant difference in learning effects among the students with various SDLAs after CBWL? Means Means One-way MANOVA (dependent variables are self-perceived learning effect and outcome of achievement assessment)

4 268 C.-C. Chang from the first and second level based on Kirkpatrick s (1996) training effect evaluation (Noe, 1999). The self-perceived learning effect captures average scores of students comments through the questionnaire of a Likert 5-point rating scale; the score of achievement assessment includes the laboratory assessment of four CBWLM units (representing 50%) plus paper-pencil test (representing the other 50%). SDLA consists of various factor aspects such as Effective Learning, Fondness for Learning, Learning Motivation, and Active Learning, etc. Literature Review Competency-Based Learning Materials Competency-based learning material is based upon a competency profile derived from the competency analysis. As Westera and Sloep (1998) have stated, a CBL environment should be based on authentic profiles that cover a set of competencies. Woelk (2002) mentioned a competency gap analysis, identifying what competencies the workers lack to effectively carry out their jobs, that is, based on what the workers need to know and what they already know. In respect of knowledge, skill, and attitude, CBLM stipulates the learning objectives and organizes the units of the learning material with the substantial behaviour objectives. Each unit s learning steps and competency requirements should be described in detail, and behaviour objectives of each competency should be specified as well. These behaviour objectives not only explain the measurable learning outcomes but also represent the standards to be achieved (Weng, 1996). According to the guide of competency-based training material delivered by the Division of Employee Training, Department of Labor (2005), Taiwan, the structure of CBLM is composed of cover pages, learning guidelines, introduction, definitions, learning objectives, learning activities, and testing. According to Tseng (1999), this study prepares and induces mandatory contents and goals of CBLM as shownintable2. Furthermore, according to Woelk and Lefrere (2002) learner profile and competency ontology may be included in the CBL or performance-based learning environment. Learner profile refers to a description of the skills and of an individual. The learner profile for a worker or learner may include, for instance, preferences, experiences, and assessment of the worker s competencies. Competency ontology is a description including the required competencies for specified job tasks and relationships among these competencies (Woelk & Lefrere, 2002, pp. 92, 95). It is primarily to capture the competencies that a worker or learner must possess to solve problems in his or her working processes. Competency ontology described by Woelk and Lefrere may be regarded as so-called competency profile included in the CBWLM developed by this study. Woelk and Lefrere further indicated that the competency ontology might provide reasoning about competencies. Generally speaking, the learner profile must be able to make reference to competency ontology, learning objects, and working processes of employees.

5 Competency-Based Web Learning Material 269 Table 2. Contents of CBLM and their goals Contents Goals 1. Code and title of learning material 1. The code and topic are disclosed on the cover page of each unit of the learning material. 2. Introduction 2. Specify highlights and importance of the unit of the learning material to inspire learning interests of students. 3. Learning objectives 3. List the fundamental occupation capability that students should provide after completing the unit in a behaviour objective approach. 4. Pretest 4. Test students in their achievement level before learning; students reaching the level are allowed to discard this unit of the learning material. 5. Learning tasks 5. Based on the behaviour objectives, test results prior to learning, and depending on the capability degree of students, teacher assigns students the selective tasks such as learning activities, learning contents, and assignments, etc. 6. Instructional media 6. To improve learning effects, it s recommended to use teaching tools or educational media that are favourable for students learning. 7. Learning activities 7. The learning activity can be divided into two parts: knowledge and skill, and, occasionally, some tests. 8. References 8. Students may refer to relevant data if they find problems or intend to have an in-depth discussion or further learning contents during the learning process. 9. Posttest 9. Allow students to undertake the test after completing the unit of the learning material for evaluation of learning effects. Self-Directed Learning Aptitude Self-directed learning means learning something proactively, independently, and patiently; being responsible to learn; learning which is a challenge; a self-training ability; high curiosity; intense impetus to learning; self-assurance; enabling a fundamental learning skill; scheduling time for learning; and planning the integral learning and enjoying learning toward an objective (Teng, 1995). Self-directed learning may help adapt to changing environment and enhance creativity (Ramsey & Couch, 1994). Self-directed learning readiness (SDLR) questionnaire, developed by Guglielmino (1977), measures learner or worker readiness for self-directed learning. The SDLR questionnaire, completed several times through the Delphi approach, has been verified to possess sufficient validity and reliability. It is based on a Likert 5-point scale covering 58 items that yield a total score for identifying self-directed readiness. After Guglielmino s SDLR questionnaire was presented, there were some scholars applying it to measuring SDLR or self-directed learning aptitude (SDLA) for various students, and there were some scholars implementing studies regarding the reliability and validity of the SDLR questionnaire (Long, 1987; Reynolds, 1986).

6 270 C.-C. Chang Eight factor aspects were included in the SDLR questionnaire: openness to learning opportunities, self-concept as an effective learner, initiative and independence in learning, informed acceptance of responsibility for one s own learning, love of learning, creativity, positive orientation to the future, ability to use basic study skill and problem-solving skills (Bonham, 1989; Guglielmino, 1977; Hung, 1995). These eight factor aspects underlying the SDLR ascertain learner readiness for self-directed learning and have been identified and further applied in many later studies (McCune & Guglielmino, 1992; Siaw, 2002). Guglielmino s SDLR questionnaire has been translated into several languages around the world (Guglielmino & Guglielmino, 1991), for instance, translated into Chinese by Teng (1995) and revised by some experts in Taiwan. Consequently the number of the questions was reduced from 58 to 55 and the original eight factor aspects were slashed down to six aspects based on the consideration for cultural differences. The four factor aspects of the Chinese-based SDLR, able to measure the degree of self-directed learning aptitude are (1) Effective Learning, Fondness for Learning, (2) Learning Motivation, Active Learning, (3) Independent Learning, and (4) Creative Learning. Development of Competency-Based Web Learning Material The CBWLM contents are based on the competency profile (required competencies for a technician of microprocessor control, including 16 responsibilities), announced by the Division of Employee Training, Department of Labor (2005) in Taiwan, plus the proposal delivered by the instructor for the curriculum progress. Finally, we decided to take the Task 1 (to understand basic architecture of MCS-51 Microprocessor) and Task 4 (to familiarize with control and application of MCS-51 Instruction) in Task 6 (Microprocessor Control) of the preceding 16 responsibilities as the CBWLM competency standard. Table 3 shows the competency profile of a technician of microprocessor control (the sixth responsibility). Based on the referred two competency standards (for Task 1 and Task 2), two behaviour objectives in the CBWLM were formulated. They are, respectively, to familiarize with attribute of output ports of MCS-51 and application of LED, and to familiarize with attribute of output ports of MCS-51 and application of seven-section displayer. Four learning units of the CBWLM were designed based on the two behaviour objectives. The first behaviour covers three units of the CBWLM: Practice of Single- Lamp Right-Transferal, Practice of Advertising Lamp, and Practice of Thunderbolt Lamp. The second behaviour represents only one unit of the learning material, Seven-Section Displayer of One Order. These four units of the learning material are mainly designed to teach students the Application of MCS-51 Output Port. Contents and their descriptions of one unit of CBWLM based on the practicum Microprocessor Laboratory are as follows:. Material title: Subject of the unit.. Competency profile: A set of competency standards which can be achieved after completing the CBWLM.

7 Competency-Based Web Learning Material 271 Table 3. Competency profile of a technician of microprocessor control Competencies of each task for the 6th responsibility (microprocessor control) *Task 1. To understand basic architecture of MCS-51 Microprocessor Task 5. To be able to manipulate basic I/O of PIC Task 9. To be able to manipulate timepiece control of MCS-51 Task 13. To be able to use PIC to produce product Task 17. To be able to manipulate timepiece of EM78 Task 2. To understand basic architecture of EM-78 Microprocessor Task 6. To be able to manipulate interruption control of PIC Task 10. To be able to manipulate electrical circuit of MCS-51 Task 14. To be able to manipulate EM78 signal ship Task 18. To be able to manipulate electrical circuit of EM78 Task 3. To understand basic architecture of PIC Microprocessor Task 7. To be able to manipulate timepiece control of PIC Task 11. To be able to manipulate MCS-51 LED Task 15. To be able to manipulate basic I/O control of EM78 Task 19. To be able to use EM78 to produce product *Task 4. To familiarize with control and application of MCS-51 instruction Task 8. To be able to manipulate electrical circuit control of PIC Task 12. To be able to manipulate PIC Task 16. To be able to manipulate interruption control of EM78 Note: *Learning contents of Task 1 and Task 4 are included in the CBWLM developed by this study.. Learning objectives: Competency levels which should be reached after completing the unit.. Learning guideline: Teaching students how to use CBWLM.. Content description: Describing contents of the unit.. Pretest: Learning the content of the unit only if passing the test.. Test feedback: Listing wrongly answered questions and correct answers of the pretest.. Remedial learning (1): Students failing in the pretest need to take remedial learning (1). After having learned the content of remedial learning (1), they should attend the pretest again, and learn the laboratory knowledge only if they pass the test.. Laboratory knowledge: Laboratory knowledge associated with skill operations.. Laboratory demonstration: Teaching students skill operations.. Posttest: Entering into the next unit if passing the test.. Test feedback: Listing wrongly answered questions and correct answers of the posttest.. Remedial learning (2): Students failing in the posttest need to take remedial learning (2). After having learned the content of remedial learning (2), they

8 272 C.-C. Chang should attend the posttest again, and enter into the next unit only if they pass the test.. Learning log: Recording the students online behaviours of using CBWLM. Methods Subjects The subject of this study consists of 38 undergraduate students in the practicum Microprocessor Laboratory, a sophomore course of the Department of Electronic Engineering at a college located in northern Taiwan. This course, elective for one semester, is a practicum that requires more learning by doing. After implementation of the CBWLM for 8 weeks, we evaluate students learning effect through the questionnaire for self-perceived learning effect survey. We investigate the SDLA before, during, and after the use of the CBWLM and further explore influences of SDLA on learning effects. After completing the course, these students are supposed to achieve some technical standards. This is really correspondent to the rationale of CBL and thus becomes a content of CBWL. Since the students are majoring in electronic or computer fields they have a high level of computer use and access to the Internet. Process The study lasts 8 weeks. Before the CBWL course begins, the students complete the pretest of the SDLA questionnaire. After the CBWL course ends, the students complete the posttest of the SDLA questionnaire and the questionnaire for selfperceived learning effects. The instructor first teaches students how to use CBWLM before this study. The process of the CBWLM implemented in classroom teaching is that the instructor, after part of the theory teaching, asks the students to access the CBWLM website and practice writing in MCS-51 Microprocessor language; the students, following the required procedure of CBWL, systematically learn the CBWLM unit by unit. The procedure of CBWL followed by students is based on the procedure of CBL proposed by Kuo (2000). In the procedure of CBWL, shown in Figure 1, the students are to:. Select a unit to self-learn and read learning objectives and laboratory contents.. Take the pretest.. Take remedial learning (1) if failing to pass the test.. Learn laboratory knowledge and watch demonstration of the unit if passing the pretest.. Take the posttest after completing the learning of laboratory knowledge and demonstration.. Take remedial learning (2) until he or her succeeds if failing in the posttest.. Enter into the next learning unit.

9 Competency-Based Web Learning Material 273 Figure 1. Procedure of competency-based web learning The relationships among the research variables in correspondence to the research questions proposed above are listed as follows and shown in Figure 2.. Learning effects, a dependent variable, is composed of two aspects, namely, the self-perceived learning effect and the outcome of the achievement assessment.. Self-directed learning aptitude (SDLA), an independent variable, is divided into three stages (before, during, and after CBWL). The SDLA consists of four factor aspects: Effective Learning, Fondness for Learning, Learning Motivation, and Active Learning. Descriptive statistics is used to analyse the self-perceived learning effect of CBWL and investigate SDLAs for three stages of CBWL. One-way MANOVA is used is to determine if there is any statistically significant difference among learning effects for various SDLAs during CBWL (and also after CBWL).

10 274 C.-C. Chang Figure 2. Relationships among research variables

11 Competency-Based Web Learning Material 275 Methods of Data Gathering The study uses the questionnaire survey in the following stages to collect relevant data. Scoring of achievement assessment. According to Elshout-Mohr et al. (2000), assessment of CBL cannot only be focused on the use of standard tests and fixed criteria. There should also be an emphasis on the fairness of an assessment that allows students to demonstrate relevant competencies at the end of a longer learning period. Westera and Sloep (1998) claimed that students performances should be assessed within practical, life-like contexts. Thus, grading of CBL achievement might integrate tests, work, the learning process, and progress. In this study, the score of achievement assessment is the summation of the laboratory assessment (representing 50%) and a paper-pencil test (representing 50%); both are graded by the teacher. The laboratory assessment consists of five checking aspects in terms of accuracy of laboratory outcomes, amplitude of laboratory outcomes, creativity of laboratory outcomes, correction of laboratory procedure, and attitude toward doing the laboratory. The total score of the laboratory assessment is 100 points, that is, 20 points for each checking aspect. The paper-pencil test includes 25 questions in fill-in-the-blank format based on a scale of 100 points. The questions on the paper-pencil test cover knowledge in four CBWLM units. Reliabilities of the laboratory assessment and paper-pencil test are acceptable since they are all retrieved from assessment bases in the teaching guide for instructors. Evaluation of self-perceived learning effects. On finishing CBWL, the study uses the questionnaire for self-perceived learning effects to detect perceptive learning effects for students after using CBWLM. Key contents in the questionnaire are used to assess whether the functions of CBWLM will enhance students MCS-51 Microprocessor knowledge and practical ability. The questionnaire contains 10 questions of a Likert 5-point rating scale and one opening question. The data acquired is deemed a self-perceived learning effect. Examination of SDLAs before, during, and after CBWL. The Chinese version of the SDLA questionnaire, with some minor amendments by Teng (1995), is used in this study to measure the extent to which individuals perceive themselves as processing abilities and attitudes frequently associated with self-directed aptitudes in CBWL experiences. A week prior to CBWL, during CBWL, and a week after CBWL we implement the pretest, midtest and posttest of the SDLA questionnaire for the students, so as to evaluate students SDLAs before, during, and after using CBWLM. Validity and Reliability of Instruments Validity and reliability analysis of SDLA questionnaire. Since the SDLA questionnaire, modified by Teng (1995), is intended to be used for the measurement of web

12 276 C.-C. Chang learning students, the ulterior questions need to be moderately modified in accordance with web learning nature. We modified the presentation style of the questions in the SDLA questionnaire and sent them to a web-learning scholar, a CBL expert, and an instructor for review. Since the purpose of the initial SDLA questionnaire was more applicable to workers and is now used for college students in this study, some terms used in the questions were modified. Since the Chinese-based SDLA questionnaire, initially derived from Guglielmino (1977) and later modified by Teng, has been revised by this study through expert review to achieve so-called expert validity, it is really not necessary to redo the validity analysis by a pilot test. After removing 17 questions in two factor aspects for inadequate Cronbach s a values (see the next section for details), 37 items in the SDLA questionnaire were analysed by using a Likert 5-point rating scale that yielded a total score for selfdirected learning aptitude. As shown on the left-hand side in Table 4, five levels of the SDLR constructed by Guglielmino and Guglielmino (1991) can be identified according to their overall or mean scores. Based on the same rate of overall score or mean scores, five levels and their ranges of scores for the SDLA in this study are shown on the right-hand side in Table 4. The study uses Cronbach s a coefficient to test the reliability of the SDLA questionnaire. Table 5 listed the internal reliability of the six factor aspects, where a coefficients lie between 0.6 and 0.9. It s obvious that, besides Creative Learning and Independent Learning, a values of the remaining factor aspects are all higher than 0.7. As a result, those two factor aspects are removed. Validity and reliability analysis of learning effect questionnaire. We randomly select 30 students from the 38 students participating in the study to undertake the pilot test of the questionnaire for self-perceived learning effects. We have received 28 effective questionnaires in total. These students have properly checked the answers to the questions. We have also modified ambiguous questions. Originally this study would plan to use the method of factor analysis for building the validity. This method, however, is discarded, because a small number of samples do not lead to genuine validity (to build the validity via factor analysis usually requires over a hundred samples). Table 4. Five levels of the SDLA for students based on overall and mean scores Guglielmino s study This study Scale by overall scores Scale by mean scores Term of SDLA level Scale by overall scores Scale by mean scores Term of SDLA level low very low below average low average average above average high high very high

13 Competency-Based Web Learning Material 277 Table 5. Reliability of pilot test of SDLA questionnaire (n ¼ 28) Reliability Effective learning (11 questions) Fondness for learning (9 questions) Learning motivation (7 questions) Active learning (10 questions) Independent learning (13 questions) Creative learning (4 questions) Cronbach s a

14 278 C.-C. Chang For the pilot test, we gain a high value of Cronbach s a coefficient that is equal to It implies a sufficient reliability for the learning effect questionnaire. Results and Discussions Perceptive Learning Effects After finishing competency-based web learning, we use the questionnaire for selfperceived learning effects to implement measurement for the students and receive 38 effective questionnaires, no void questionnaires. The results are shown in Table 6. In Table 6, the overall students learning effect (3.55 points) is classified as a competent level. This indicates that the students are mostly satisfied with CBWLM. In terms of the points, each question scores over 3 points. Question 5, benefits of feedbacks of the pretest and posttest, scores the highest (3.71 points), and Question 8, online discussion is helpful to resolve curriculum problems scores the lowest (3.18 points), which indicates the highest effect of the feedback of test while the effect of online discussion requires enhancement. Table 6. Self-perceived learning effects (n ¼ 38) Self-perceived learning effects Mean SD 1. Does the mechanism, pretest and remedial learning (1), enable you to consolidate your prerequisite knowledge before learning the CBWLM? 2. Does the mechanism, posttest and remedial learning (2), enable you to master the laboratory knowledge of the learning material? 3. Does the visualized simulation of Laboratory Knowledge allow you to have clear understanding of the executive outcomes of MCS-51 Microprocessor? 4. Does the streaming media material of Laboratory Demonstration allow you to have more understanding of the practical product of MCS-51 Microprocessor? 5. Does the feedback from the pretest and posttest allow you to have clear understanding of laboratory knowledge? 6. Is the procedure of CBWL [Pretest! Remedial learning (1)! Posttest ! Remedial learning (2)] helpful to you in learning MCS-51 Microprocessor? 7. Generally, are contents of CBWLM able to enhance your knowledge and capability of MCS-51 Microprocessor? 8. Is online discussion forum with classmates helpful to you while you are encountering problems in learning MCS-51 Microprocessor? 9. Are website-linking resources helpful to you in completing the MCS-51 Microprocessor project? 10. Are online-handout resources helpful to you in completing the MCS-51 Microprocessor project? Overall Note: The scoring is based on a 5-point rating scale with a total of 50 points.

15 Competency-Based Web Learning Material 279 Figure 3. Scores of achievement assessment (n ¼ 38, M ¼ 81.1, SD ¼ 7.05) Scores of Achievement Assessment After totalling the scores of the laboratory assessment (50%) and paper-pencil test (50%), we obtain the scores of achievement assessment. In Figure 3, it is clear that all of the students scores of achievement pass 60 points. If we take 80 points (100 points based) as the mastery learning standard, over a half of the students (20) achieve certain competency standards (over 80 points), but there are less than half of the students (18) failing to reach mastery standards (below 80 points). Finally, the students achieve mastery standards (a mean of 81.1 points). Self-Directed Learning Aptitude At a time before, during (a month later), and a week after CBWL, the study implements measurements for the students by using the SDLA questionnaire and receives 38 questionnaires without any void questionnaires. Pretest of SDLA questionnaire. In Table 7, before the CBWL intervention, the overall students SDLA (mean ¼ 3.50 points) is classified as an average level. Factor aspect of Fondness for Learning, classified as an average level, scores the highest (3.75 points), and Effective Learning, classified as a low level, scores the lowest (3.27 points). As regards the deviation between each factor aspect and its mean value, Fondness for Learning has the largest deviation (standard deviation ¼ 0.75) while Effective Learning has the smallest one (standard deviation ¼ 0.52).

16 280 C.-C. Chang Table 7. Pretest of SDLA questionnaire (n ¼ 38) Factor aspects Minimum Maximum Mean SD Level of SDLA Fondness for Learning (9 questions) Average Effective Learning (11 questions) Low Learning Motivation (7 questions) Average Active Learning (10 questions) Low Overall (37 questions) Average Note: The scoring is based on a 5-point rating scale with a total of 185 points. The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1 m 3.03 (very low), 3.04 m 3.46 (low), 3.47 m 3.90 (average), 3.91 m 4.32 (high), and 4.33 m (very high). Table 8. Midtest of SDLA questionnaire (n ¼ 38) Factor aspects Minimum Maximum Mean SD Level of SDLA Fondness for Learning (9 questions) Average Effective Learning (11 questions) Low Learning Motivation (7 questions) Average Active Learning (10 questions) Average Overall (37 questions) Average Note: The scoring is based on a 5-point rating scale with a total of 185 points. The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1 m 3.03 (very low), 3.04 m 3.46 (low), 3.47 m 3.90 (average), 3.91 m 4.32 (high), and 4.33 m (very high). Midtest of SDLA questionnaire. In Table 8, during the CBWL intervention, the overall students SDLA (mean ¼ 3.53) is classified as an average level. Factor aspect of Fondness for Learning, classified as an average level, scores the highest (mean ¼ 3.86 points) while Effective Learning, classified as a low level, scores the lowest (mean ¼ 3.27 points). As regards the deviation between each factor aspect and its mean, Fondness for Learning has the largest deviation (standard deviation ¼ 0.75) while Effective Learning has the smallest one (standard deviation ¼ 0.63). Posttest of SDLA questionnaire. In Table 9, after the CBWL intervention, the overall students SDLA (mean ¼ 3.47 points) is classified as an average level. Factor aspect of Fondness for Learning, classified as an average level, scores the highest (mean ¼ 3.57 points) while Active Learning, classified as a low level, scores the lowest (mean ¼ 3.27 points). As regards the deviation between each factor aspect and its mean, Fondness for Learning has the largest deviation (standard deviation ¼ 0.65) while Active Learning has the smallest one (standard deviation ¼ 0.52). Concerning the SDLAs measured before, during, and after CBWL, the midtest of SDLA has the highest result, the pretest of SDLA follows, and the posttest of SDLA

17 Competency-Based Web Learning Material 281 Table 9. Posttest of SDLA questionnaire (n ¼ 38) Factor aspects Minimum Maximum Mean SD Level of SDLA Fondness for Learning (9 questions) Average Effective Learning (11 questions) Average Learning Motivation (7 questions) Average Active Learning (10 questions) Low Overall (37 questions) Average Note: The scoring is based on a 5-point rating scale with a total of 185 points. The qualitative interpretation of the SDLA levels is based on the mean scores (m): 1 m 3.03 (very low), 3.04 m 3.46 (low), 3.47 m 3.90 (average), 3.91 m 4.32 (high), and 4.33 m (very high). generates the lowest outcome by small differences. If comparing by level norm, there is no changing level among the three tests of SDLA. Influences of SDLAs of Mid-CBWL and Post-CBWL on Learning Effects Based on the quartile, we divide SDLA of mid-cbwl into low (first 25%), middle (50%, middle), and high group (last 25%), and perform one-way MANOVA together with learning effects (self-perceived learning effect and outcome of achievement assessment). From Table 10 it is clear that there is no significant difference in learning effects among the students of the three groups of SDLA during CBWL, showing the students SDLA of mid-cbwl will not influence their learning effects. From Table 11 it is clear that there is no significant difference in learning effects among the students of the three groups of SDLA after CBWL, showing the students SDLA of post-cbwl will not influence their learning effects. Conclusions and Implications After CBWL has been used in classroom teaching and after-class review for the past 8 weeks, each student s score of the achievement assessment is higher than 60 points (100 points based). If we take 80 points as the mastery level, over a half of the students (20 students) obtain a certain competency standard, less than half of them (18 students) fail to pass the mastery level. For this reason, CBWL is generally capable of enabling the students to achieve the mastery learning level and may further identify CBL is mastery learning (Kang, 1997; Yang, 1998). Comparison among SDLAs of pre-cbwl, mid-cbwl, and post-cbwl, SDLA of mid-cbwl scores the highest, SDLA of pre-cbwl follows, and SDLA of post-cbwl scores the lowest by slight differences. As regards factor aspects of SDLA, Fondness for Learning is the highest for the three stages of CBWL, Effective Learning is the lowest for pre-cbwl and mid-cbwl, and Active Learning is the

18 282 C.-C. Chang Table 10. One-way MANOVA on learning effects for various groups of SDLA of mid-cbwl (n ¼ 38) Sources of variation Degree Wilk s value (significant level) Learning effects Degree Sum of square (SS) Mean of SS (MS) F value Significant level Between groups (SDLA) (0.41) Self-perceived learning effect Achievement assessment Within groups (residual) 35 Self-perceived learning effect Achievement assessment Total 37 Self-perceived learning effect Achievement assessment

19 Competency-Based Web Learning Material 283 Table 11. One-way MANOVA on learning effects for various groups of SDLA of post-cbwl (n ¼ 38) Sources of variation Degree Wilk s value (significant level) Learning effects Degree Sum of square (SS) Mean of SS (MS) F value Significant level Between groups (SDLA) (0.38) Self-perceived learning effect Achievement assessment Within groups (residual) 35 Self-perceived learning effect Achievement assessment Total 37 Self-perceived learning effect Achievement assessment

20 284 C.-C. Chang lowest for post-cbwl. After using further inferential statistics, we find that CBWL will enhance the factor, Fondness for Learning, in SDLA. As indicated in the study result of Chung (2001), the students learning effects were correlative with SDLA after web learning. But we find, in this study, that there is no significant difference in learning effects among the students of various groups of SDLA during CBWL, likewise having the same result for after CBWL. These imply that SDLAs of the mid-cbwl and post-cbwl do not influence learning effects. The discrepancy between this study result and that of Chung needs a further examination. Subject to the instructor s limitations, this study is targeted at a class with few students, instead of two classes (or groups), for the contrastive experimental study. It is recommended to perform the comparative study based on experimental and controlled groups in the future, and to eliminate some variables that may affect the study potentially, for a better understanding of the difference between learning effects of CBWL and traditional CBL, CBWL and traditional classroom learning. References Bonham, L. A. (1989). Self-directed learning: Emerging theory & practice. In H. Long (Ed.), Self-directed orientation toward learning: A learning style? (pp ). Norman, OK: Oklahoma Research Center for Continuing Professional and Higher Education, University of Oklahoma. Chung, Y. (2001). Influences of self-directed learning aptitude on asynchronous distance learning. Unpublished master s thesis, National Kaoshiung Normal University, Taiwan. Conrad, R., & Donaldson, J. (2004). Engaging the online learner: Activities and resources for creative instruction. San Francisco: Jossey-Bass. Division of Employee and Training. (2005). Competency-based training. Retrieved November 30, 2005, from Elshout-Mohr, M., Oostdam, R., Snoek, M., & Dietze, A. (2000). Assessment in a competencyoriented dynamic curriculum. Retrieved November 30, 2005, from publicaties/publiexplo/english/index.html Ghaoui, C., & Janvier, W. (2004). Smart proflexlearn: An intuitive approach to virtual learning environment. In C. Ghaoui (Ed.), E-education applications: Human factors and innovative approach (pp ). Hershey, PA: Information Science Publishing. Guglielmino, L. (1977). Development of self-directed learning readiness scale. Dissertation Abstracts International, 38, 6467A. Guglielmino, L., & Guglielmino, P. (1991). Learning preference assessment facilitator guide. King of Prussia, PA: Organizational Design and Development, Inc. Hanna, D. E., Dudka, M. G., & Runlee, S. C. (2000). 147 practical tips for teaching online groups: Essentials of web-based education. Madison, WI: Atwood. Ho, C., Hsu, P., & Tao, C. (1999). Distance teaching for vocational distance and Internet training material. Employee and Training, 17(4), Hung, S. (1995). Relationships between self-directed learning aptitude and learning achievement. Unpublished master s thesis, National Taiwan Normal University, Taipei, Taiwan. Kang, T. (1997). Theory and curriculum development of competency-based vocational training. Employee and Training, 15(6), 3 9.

21 Competency-Based Web Learning Material 285 Kirkpatrick, D. (1996). Evaluation. In R. Craig (Ed.), The ASTD training and development handbook. New York: McGraw-Hill. Kirschner, P. A. (2000). Design strategies for competency based learning. Heerlen, The Netherlands: Open University of The Netherlands. Kuo, C. (2000). Procedure and management of CBT. Journal of Human Resources Development, 4, Lin, C., & Liao, M. (2000). Competency-based training of distance instruction. In Proceedings of Conference on Application of Internet (pp ). Tainan, Taiwan: Cheng-Kung University. Lin, F. (2002). Planning and construction of competency-based instructional website: A case of Digital Logic course in vocational technical high school. Unpublished master s thesis, National Chang- Hua University of Education, Taiwan. Long, H. (1987). Item analysis of Guglielmino s self-directed learning readiness scale. International Journal of Lifelong Education, 6, McCune, S., & Guglielmino, L. (1992, January). Factor analytic study of the self-directed learning readiness scale. Paper presented at the Sixth International Symposium on Adult Self-Directed Learning, West Palm Beach, FL. Noe, R. (1999). Employee training and development. New York: McGraw-Hill. Ramsey, V., & Couch, P. (1994). Beyond self-directed learning: A partnership model of teaching and learning. Journal of Management Education, 18(2), Reynolds, M. (1986). The directedness and motivational orientations of adult part-time students at a community college. Unpublished doctoral dissertation, Syracuse University, NY. Richards, L., Dooley, K., & Lindner, J. (2004). Online course design principles. In C. Howard, K. Schenk, & R. Discenza (Eds.), Distance learning and university effectiveness: Changing educational paradigms for online learning (pp ). Hershey, PA: Information Science Publishing. Ritzen, M., & Kosters, J. (2002). Combining different aims in a portfolio system: A web-based portfolio and the various ways in which it can serve the student. In C. Montgomerie & J. Viteli (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp ). Norfolk, VA: AACE. Ryan, S., Scott, B., Freean, H., & Patel, D. (2000). The virtual university: The internet and resourcebased learning. London: Kogan Page. Siaw, S. C. (2002). Fostering self-directed learning readiness by way of PBL intervention in business education. In Online Proceedings of 2nd Asia-Pacific Conference on Problem- Based Learning, Singapore. Retrieved November 30, 2005, from pblconference/ Teng, Y. (1995). Adult instruction and self-directed learning. Taipei, Taiwan: Wu-Nan Publication. Ting, T. (1996). Relationships between self-directed learning readiness, computer attitude and computer achievement for students of open university. Unpublished master s thesis, National Chung-Cheng University, Taiwan. Tseng, K. (1999). Development of instructional material. In W. Chiang (Ed.), Introduction of vocational technical education. Taipei, Taiwan: NTNU Publication. Weng, S. (1996). Development of vocational competency-based training material: A case of industrial electronic curriculum. Unpublished doctoral dissertation, National Taiwan Normal University, Taipei, Taiwan. Westera, W., & Sloep, P. (1998). The virtual company: Toward a self-directed, competence-based learning environment in distance education. Educational Technology, 38(1), Wielenga, D. (2000). Proving competency: Integrative assessment and web-based portfolio system in a dynamic curriculum. Retrieved November 30, 2005, from ass-eng.html Woelk, D. (2002). E-learning, semantic web services and competency ontologies. In P. Barker & S. Rebelsky (Eds.), Proceedings of World Conference on Educational Multimedia, Hypermedia and Telecommunications (pp ). Norfolk, VA: AACE.

22 286 C.-C. Chang Woelk, D., & Lefrere, P. (2002). Technology for performance-based lifelong learning. In R. Kinshuk et al. (Eds.), Proceedings of International Conference on Computers in Education (pp ). Los, Alamitos: IEEE Computer Society. Yang, C. (1998). Theory and practice of vocational and technical education. Taipei, Taiwan: Fu-Hsing Publication.

23