Learning Style and Instructional Methods in a Graduate Level Engineering Program Delivered by Video Teleconferencing Technology

Transcription

1 Learning Style and Instructional Methods in a Graduate Level Engineering Program Delivered by Video Teleconferencing Technology Gary Rafe and John H. Manley Manufacturing Systems Engineering Program University of Pittsburgh Pittsburgh, PA Abstract The recent availability of new telecommunication technologies to the educational community has generated a growing interest in the use of distance education methods by institutions to reach larger student bodies. This research considers the use of learning style theory to identify effective instructional methods in a video teleconferencing-based graduate-level engineering program at the University of Pittsburgh. Mismatches in student learning style and instructional strategies have been found to affect students perceptions of program quality and ultimately their completion of distance education programs. Introduction Rapidly shifting paradigms, globally competitive markets, and a changing workforce represent typical challenges for many firms and agencies involved in the manufacturing and service sectors. To help industries meet such challenges, technology-mediated educational delivery systems have been used by universities to bring graduate-level engineering degree programs to engineering professionals at or near their workplaces since the 1960s [1]. The recent availability of new telecommunication technologies (e.g., the World Wide Web and interactive video teleconferencing) have created a growing interest in the use of distance education methods by institutions to reach a larger student body, especially adult students living and working greater distances from traditional campus communities. An important consideration for educational programs delivered remotely is the issue of student persistence (i.e., program completion) [2], which is affected by multiple factors, including student satisfaction with the course or program, time constraints while working full-time, and the availability of interactive contact with instructors. Research in the distance education field suggests that student learning style is a another significant factor in persistence [3,4]. Engineering educators have begun to consider the application of learning style theory for improving instruction in their own disciplines [5,6,7]. Research related to the proposition that matching student learning style with certain broad instructional strategies can improve the effectiveness of education and training has been promising, but inconclusive [8]. Numerous studies have considered video-based distance education methods in engineering [9,10,11]. The predominant conclusion from this past research has been that there appears to be no significant difference, overall, in observed performance among graduate student groups (i.e., on-campus and remote sites). While this conclusion has been reassuring to program providers, concern has been raised that certain aspects of an engineering education which can be problematic in a distance education program (e.g., laboratory courses) may have a negative impact on student satisfaction [12]. Furthermore, a study of educators and trainers in business and industry suggests that more effective instructional methods might be applied in the typical university classroom [14]. It is not difficult to extend this view to newer technology-based instructional environments (e.g., those using distance education technologies). The research reported here investigated the use of learning style theory to identify instructional methods that are perceived to be effective by graduate students enrolled in select engineering courses within the University of Pittsburgh s Manufacturing Systems Engineering Program (MSEP). These courses have used commercially available video teleconferencing (VTC) systems to provide interactive communications between the University s Pittsburgh and Johnstown campuses. In the first five semesters since the distance education project began in the fall of 1994, some 19 courses were offered with a total graduate enrollment (both local and remote) of 137. This study focuses on the students and faculty involved in these 19 courses. Experiential Learning Model Kolb described the process of experiential learning as a four-stage cycle involving the four separate adaptive learning modes concrete experience, reflective observation, abstract conceptualization, and active experimentation [14]. In operationalizing the experiential learning model (ELM), the relative emphasis an individual places on each of the four learning process modes (i.e., concrete experience: CE; reflective observation: RO; abstract conceptualization: AC; and active experimentation: AE) is measured using a selfadministered learning style inventory (LSI) questionnaire. Two additional combination scores are computed, indicating

2 the extent an individual emphasizes concreteness over abstractness (CE AC) and similarly, reflection over action (RO AE). The results of these computations are then plotted on the axis shown in Figure 1, which Kolb describes as a learning-style map. The relative placement on this map indicates one of four basic underlying learning styles (i.e., convergent, divergent, assimilative, and accommodative) according to the ELM. Active Experimentation Accomodators Convergers Concrete Experience Divergers Assimilators Abstract Conceptualization Reflective Observation Figure 1 Dimensions of the Experiential Learning Model While Kolb s ELM has generated interest in understanding how to enhance the effectiveness of the educational process, the survey instrument used to measure the ELM s four dimensions has come under sharp criticism by researchers, particularly with regard to its forced-choice scoring format [15]. Also, evidence suggesting the existence of the experiential learning model s bipolar dimensions (i.e., concrete experience/abstract conceptualization and active experimentation/reflective observation) has not been observed by research using Kolb s LSI and its variations. Thus, the research reported here does not employ the model s learning styles themselves, but rather the four dimensions comprising the ELM. Purpose The application of effective instructional methods is an important consideration in educational and training programs. Such programs delivered to remote students by distance education technologies may prove to be especially sensitive to choices of instructional methods. A goal of this research was the identification of instructional methods that are perceived as being highly effective by both local and remote student groups, given that the target audience exhibits certain learning style dimension emphases. That is, this research posited that instructional methods may be matched with each of the four dimensions of Kolb s ELM based on their perceived effectiveness by students. A second aspect of the current study measured the relative frequency of use of the instructional methods, considered above, by faculty in the study group. Information regarding the reported application of instructional methods by graduate faculty in the remotely delivered courses, coupled with findings from the previous goal should allow development of instructional recommendations for the highly effective delivery of these courses. Method An anonymous survey procedure was used in this research to collect relevant information from the graduate student and faculty subjects comprising the study population. The questionnaire sent to the study s student group contained four sections. The first section was a learning style inventory following Kolb [16] and Geiger et al. [17] in which student subjects were asked to respond to 28 statements pertaining to the four dimensions of the ELM. A five-point Likert response scale was used in this inventory, with response categories ranging from not like me (1) to very much like me (5). The item arithmetic mean was used to compute scale scores for each of the four dimensions being measured in this section. The second and third sections of the student survey elicited attitudes regarding the perceived effectiveness of various instructional methods in general educational and the VTC-based distance education contexts, respectively. The 20 instructional methods chosen for consideration in this study came from suggestions made by Harb et al. [7] and Svinicki and Dixon [18] according to the four dimensions of the ELM (Figure 2). A five-point Likert response scale was used to measure the level of student subjects perceived effectiveness of each of the teaching methods shown in Figure 2 in the learning of new concepts and skills. The response scale categories ranged from ineffective (1) to highly effective (5); a sixth response category, not applicable (0) was provided for each of these items for cases where subjects had no personal experience with the instructional method indicated. Final scale scores in the general context section was computed from the arithmetic average of valid item response values, and could range from 1 (method considered ineffective) to 5 (method considered highly effective). The survey instrument to which instructors were asked to respond measured their relative frequency of use of the 20 instructional methods listed in Figure 2. As in the student survey, separate sections considered usage in general graduate-level courses, and courses specific to the VTCbased distance education environment described here. The five-point Likert response scale used in these two sections ranged from 1 (never) to 5 (always).

3 Concrete Experience Observations from experiments Games Film/video presentations Field trips Role playing exercises Reflective Observation Primary text reading assignments Personal journal writing Small group discussion Brainstorming session Formal lectures Abstract Conceptualization Demonstration lectures Research papers Self-paced computer-assisted instruction Individual research projects Textbook examples (illustrations) Active Experimentation Open-ended case studies Laboratories Field work Group (applied) research projects Homework problems Figure 2 Instructional Methods Considered and their Proposed Match to ELM Dimensions Results Three mailings were conducted in the fall of 1996 to 129 graduate students and 13 instructors. A total of 74 student and 10 instructor responses were received for inclusion in the analysis reported here. Response rates for the study s two groups were 57.3% and 76.9%, respectively. A majority (58%) of the former group reported having participated in courses from a local site (i.e., the site at which the instructor is usually present). Ranked teaching method items for the student instrument (in terms of perceived effectiveness in general contexts) and the instructor instrument (relative frequency of use in the distance context) are presented in Tables 1 and 2, respectively. A family level of statistical significance of.10 was used in each analysis, with the application of the appropriate Bonferroni adjustment. Thus, 99.5% confidence intervals are reported in Tables 1 and 2, while α =.005 was used in subsequent t tests. Results of significant paired t tests on teaching method items across the two contexts for the student survey data are given in Table 3. No significant differences were observed in the paired t tests of teaching method items across the two contexts for the instructor survey data. When the effect of participation site on perceived effectiveness of the instructional methods reported by students was considered Table 1 Perceived Effectiveness of Teaching Methods Reported by Students in General Contexts Method n x s 99.5% C.I. Demonstration lectures , 4.52 Homework problems , 4.25 Field work , 4.23 Laboratories , 4.17 Field trips , 4.10 Independent research projects , 4.02 Textbook examples , 3.87 Brainstorming sessions , 3.91 Observations from experiments , 3.84 Film/video presentations , 3.81 Small group discussions , 3.76 Research papers , 3.75 Games , 3.77 Open-ended case studies , 3.70 Group (applied) research , 3.63 Formal lectures , 3.46 Self-paced CAI , 3.50 Primary text readings , 3.41 Role playing exercises , 3.16 Personal journal writing , 2.92 Table 2 Frequency of Use of Teaching Methods Reported by Faculty in the VTC Context, n = 10 Method x s µ 99.5% C.I. Primary text reading , 5.26 Research papers , 5.22 Formal lectures , 4.94 Individual research projects , 5.35 Homework problems , 5.52 Group (applied) research , 5.26 Textbook examples , 4.81 Small group discussions , 5.08 Open-ended case studies , 4.85 Demonstration lectures , 4.74 Brainstorming sessions , 4.74 Film/video presentations , 4.01 Field work , 4.58 Observations from experiments , 3.86 Role playing exercises , 3.54 Personal journal writing , 3.35 Games , 2.94 Laboratories , 2.73 Field trips , 2.85 Self-paced CAI , 1.86 using pooled t tests, no significant differences in mean response were observed in the sample. Internally reliable scales were indicated for the learning mode scales comprising the student instrument, as α.80 was observed in each case (α CE =.84, α RO =.84, α AC =.87, and α AE =.84). Ranked summary statistics for the ELM scales are presented in Table 4.

4 Table 3 Perceived Effectiveness of Teaching Methods, Significant Paired Differences, α =.005 Method n x _ distance x general p Demonstration lectures Formal lectures <.001 Laboratories Primary text reading Table 4 Learning Mode Scale Statistics, n = 74 Dimension x s x min x max µ 95% C.I. AC , 4.28 AE , 4.24 RO , 3.71 CE , 2.97 None of the four proposed teaching method scales could be considered internally reliable with the current sample, as none had α.80 here (α CE =.46, α RO =.23, α AC =.35, and α AE =.61). The relatively small sample size here precluded the use of exploratory factor analysis to investigate the loading of teaching method items into relevant scales. The assertion that the instructional activities identified here form well defined sets corresponding to Kolb s ELM was not supported by this sample. Lacking evidence of valid teaching method scales corresponding to the four dimensions of the ELM, simple linear relationships between each of these dimensions and the individual teaching methods were considered. Table 5 reports the Pearson s correlation coefficients and two-tailed significance level (p) for teaching method items with significant (α =.0013) linear relationships to any one of the four ELM dimensions. Table 5 Significant Teaching Method and Learning Mode Correlations (p); α = Learning Mode Scale Method CE RO AC AE Demonstration lectures (.1660) (.9026) (.0010) (.5671) Textbook examples (.9547) (.0001) (.2591) (.0882) Games (.7725) (.4009) (.5570) (.0003) Formal lectures (.4476) (.0001) (.0710) (.0125) As indicated in Table 5, four of the 20 instructional methods studied were found to be significantly related, albeit weakly, to one of the established learning mode scales. Of these, two were predicted by the groupings suggested in Figure 2 (i.e., demonstration lectures with abstract conceptualization and formal lectures with reflective observation). The remaining significant relationships were observed between instructional activity items and adjacent learning mode dimensions (i.e., the relationship existed not with the suggested learning mode scale, but with an adjacent dimension, according to Figure 1). Discussion This study found the two experiential learning mode dimensions with the highest means, abstract conceptualization (AC) and active experimentation (AE), to be significantly greater than the remaining two, while the lowest mean was observed on the concrete experience (CE) scale. This result was not surprising, as research performed by Kolb indicated that persons working in engineeringrelated disciplines tend to emphasize the active experimentation learning mode (relative to the reflective observation learning mode) and abstract conceptualization (over concrete experience) [19]. Under the ELM, persons with such an emphases are considered to exhibit the convergent learning style. This does not indicate, however, that evidence of the bipolar scales posited by Kolb existed in the current sample, as no strong negative correlations were observed along these dimensions. Rather, the observation appeared to be consistent with Kolb s characterization of engineers in general. Having made the observations that (1) persons involved in engineering and technically-related activities dominated the present study s student sample, and (2) that concrete experience was the learning mode scale with the significantly lowest mean (thus, average perceived emphasis), it is understandable that no strong positive relationships between the teaching methods considered and that particular scale were observed. Inspection of the 99.5% confidence intervals for mean perceived effectiveness in general contexts reported by students (Table 1) shows that for the hypothesis H 0 : µ i 3 ; H 1 : µ i > 3 (i.e., item means greater than 3 indicate greater average perceived effectiveness) H 0 cannot be rejected for 7 of the 20 items. The instructional activities that could not be considered to be at least somewhat effective by this sample were games, group (applied) research, formal lectures, self-paced computer-aidedinstruction, primary text readings, role playing exercises, and personal journal writing. Similarly, the lone teaching method observed to be perceived as less than somewhat effective by this sample was personal journal writing. A similar inspection of Table 2, which reports instructors average frequency of use of the teaching methods in the VTC context, shows that for the hypothesis H 0 : µ i 3 ; H 1 : µ i > 3 (i.e., item means greater than 3 indicate greater average usage by faculty), H 0 is rejected for

5 3 of the 20 items. That is, three of the 20 methods were shown to be used more frequently, on average, than sometimes by the instructors in this sample. These more frequently used teaching methods included primary text readings, research papers, and formal lectures. Likewise, four methods were reported to be used less often than sometimes, including games, laboratories, field trips, and self-paced computer-aided-instruction. It is interesting to note that of the three teaching methods used more frequently by instructors in their VTCbased courses, only one method, research papers, was considered to be more than somewhat effective by the student group. In the case of the other two methods, formal lectures was found to follow students emphasis of the reflective observation learning mode dimension, which was not emphasized highly by this sample of graduate students, while primary text readings was among the group of teaching methods with the lowest perceived effectiveness reported by the student group. Implications Results from this study suggest that for graduate-level courses enrolling engineering professionals, a number of effective instructional activities can be identified which do not appear to be sensitive to student learning styles. An important challenge facing content experts (i.e., graduate engineering faculty) will be the effective application of an expanded repertoire of instructional activities in order to accommodate the learning style needs of their adult students. In terms of the learning cycle suggested by the ELM, access to teaching methods perceived as effective in all four learning modes by adult learners should certainly enhance the efficacy of the experiential learning cycle and educational programs employing it. Many of the more effective teaching methods considered here would appear to be well suited for the highly interactive, media-rich instructional delivery environments allowed by video teleconferencing. For example, demonstration lectures can readily be adapted to create a certain level of interactivity from students at both near- (live) and far-end (remote) sites. The use of assigned homework problems to apply topical concepts is an accepted practice in many disciplines. Discussion of solution alternatives and techniques by students at local and remote sites can reinforce these concepts for all participants. Open-ended case studies presented by content experts, perhaps brought into the session from his or her own location via a multi-point teleconference, may serve as focal points for brainstorming sessions by groups at geographically distant sites. Small group discussions, possibly from the various remote sites might generate analysis of video presentations supplied by yet another site. Other applications in these environments may well be envisioned. Conclusion Advances in telecommunication and computing technologies are expected to increase the number of delivery methods available to educational providers. This will be especially beneficial to persons who, because of where they live, have not had access to relevant educational opportunities. As delivery technologies become more economically accessible and the number of remotely located students seeking access to courses and programs of study delivered at a distance grows, it could be expected that competition among educational program providers will grow, as well. The coming challenge will be for graduate engineering faculty to expand their repertoire of teaching methods to include those which can provide the greatest benefit to both local and remote students. The effective use of technology resources such as interactive video teleconferencing will be necessary to achieve the long term outcomes desired by all parties involved in these educational efforts (i.e., the completion of advanced degree programs by professionals working and studying at remote sites). References 1. Baldwin, L. V., New Modes For Advanced Engineering Study, Engineering Education, 1983, Vol. 73, No. 5, pp Cookson, P. S., Persistence in Distance Education: A Review. In Contemporary Issues in American Distance Education, (New York: Pergamon Press, 1990). 3. Coggins, C. C., Preferred Learning Styles and Their Impact on Completion of External Degree Programs, The American Journal of Distance Education, 1988, Vol. 2, No. 1, pp Garland, M. R., Student Perceptions of the Situational, Institutional, Dispositional, and Epistemological Barriers to Persistence, Distance Education, 1993, Vol. 14, No. 2, pp Smith, T., Learning Styles: A Method For Achieving More Effective Classroom Instruction. In 1993 Annual Conference Proceedings, Volume I (Washington, DC: ASEE), pp Felder, R. M. & L. K. Silverman, Learning and Teaching Styles In Engineering Education, Engineering Education, 1988, Vol. 78, No. 7, pp Harb, J. N. et al., Use of the Kolb Learning Cycle and the 4Mat System in Engineering Education, Journal of Engineering Education, 1993, Vol. 82, No. 2, pp

6 8. Hayes, J. & C. W. Allinson, Matching Learning Style and Instructional Strategy: An Application of the Person-Environment Interaction Paradigm, Perceptual and Motor Skills, 1993, Vol. 76, No. 1, pp Wergin, J. F. et al., Televising Graduate Engineering Courses: Results of an Instructional Experiment, Engineering Education, 1986, Vol. 77, No. 2, pp Souder, W. E., The Effectiveness of Traditional vs. Satellite Delivery in Three Management of Technology Master s Degree Programs, The American Journal of Distance Education, 1993, Vol. 7, No. 1, pp Daily, B. & M. Daily, Effectiveness of a Multimedia Televised Distance Education Program for Engineering Majors, Journal of Engineering Education, 1994, Vol. 83, No. 4, pp Bulkeley, P. Z. et al., The Attractiveness of Interactive Compressed Video In Delivering Engineering Education to the Workplace (Boston: Boston University, 1992). 13. Everett, D. R. & R. A. Drapeau, Effective Instructional Methods in Business and Education, The Phi Delta Kappa Journal, 1994, Vol. 26, No. 3, pp Kolb, D. A., Experiential Learning: Experience as the Source of Learning and Development (Englewood Cliffs: Prentice Hall, 1984). 15. Kolb, D. A., LSI: Learning-Style Inventory (Boston: McBer & Co., 1985). 16. Ruble, T. L. & D. E. Stout, A Critical Assessment of Kolb s Learning Style Inventory (ERIC Document Reproduction Service, ED , 1994). 17. Geiger, M. A. et al., An Examination of Ipsative and Normative Versions of Kolb s Revised Learning Style Inventory, Educational and Psychological Measurement, 1993, Vol. 53, pp Svinicki, M. D. & N. M. Dixon, The Kolb Model Modified for Classroom Activities, College Teaching, 1987, Vol. 35, No. 4, pp Kolb, D. A., Management and Learning Processes, California Management Review, 1976, Vol. 18, No. 3, pp