The Design and Implementation of an Enculturated Web-Based Intelligent Tutoring System

Transcription

1 The Design and Ipleentation of an Enculturated Web-Based Intelligent Tutoring Syste Phaedra Mohaed Departent of Coputing and Inforation Technology The University of the West Indies ABSTRACT Accoodating for learner diversity based on cultural backgrounds has not yet been a ajor personalisation focus until recently. Increasing nubers of Internet-ready devices have propelled e-learning forward such that these deficiencies in cultural-awareness can no longer reain unattended. Despite being investigated fro an instructional design standpoint, the enculturation of digital learning environents has largely been theorized, necessitating anual enculturation by experts, instructors and even students. Consequently, enculturated learning environents are liited in practice. In this paper, a preliinary design for building a web-based Intelligent Tutoring Syste (ITS) is described together with the features and intended functionality of the various coponents. This work contributes a practical approach that was ipleented and evaluated using two concrete systes within the doain of Coputer Science education. An analysis of the findings and epirical evidence reported in the study supports the viability of the approach taken and reveals that intelligent tutoring systes benefit fro enculturation. Author Keywords Intelligent Tutoring Systes; Culture; Coputer Science Education; Software Design; Experientation INTRODUCTION Interactive educational systes deliver instructional content to learners with the intention of providing a custoised learning experience. This is achieved through personalization based on a variety of diensions: learning styles, instructional objectives, devices used for content delivery and so on. Accoodating for learner diversity based on cultural backgrounds however has not yet been a ajor personalisation focus until recently. Increasing nubers of devices capable of accessing the Internet have propelled e-learning forward such that deficiencies in cultural-awareness can no longer reain unattended. Educational content and online tools were originally considered to be ore usable if they were designed without any culture-specific features. However, the developent of culturally neutral content and tools is virtually ipossible since cultural partiality pervades every design choice. The design of user interfaces, the selection of teaching strategies, the forat and content of the educational aterial all vary depending on the cultural background of the developers [7]. Subtle cultural influences seep into the final product and this can be counter-productive to learning when stereotypes and personal interpretations clash with the practices and beliefs of the students. So, by internationalizing or localizing these systes [12], certain users ay be included and others left out thereby working against the goal of providing individualized instruction to any learner at any tie. This happens largely because the cultural background of a learner plays a significant role in shaping his/her learning habits, and cultural appropriateness can no longer be treated as an optional personalisation diension. Despite being investigated fro an instructional design standpoint, the enculturation of digital learning environents has largely been theorized, necessitating anual enculturation by experts, instructors and even students. Consequently, enculturated learning environents are liited in practice because any developers have shied away due to the coplexity in reliably representing aspects of a particular culture [11, 2]. Young [12] points out that the dearth of culturally-aware ICT systes can also be attributed to the lack of guidance regarding the integration of culture-specific eleents into present-day instructional design. Above all, the knowledge and processes for incorporating culture have not been clearly defined with autoation in ind [2]. Interactive educational systes are in essence pieces of software and therefore any enculturated approach ust be expressed in a well-defined, unabiguous anner. In this paper, a preliinary design for building an enculturated interactive educational syste naely a web-based Intelligent Tutoring Syste (ITS) is described together with the features and intended functionality of the various 26

2 coponents. Owing to the lack of coputationally-viable approaches for building enculturated systes [11], this work contributes a practical ethod that can be replicated irrespective of the instructional doain chosen. The ipleentationn details and tools used to create two concrete systes based on this architecture for the Coputer Science doain are discussed. The results of an experiental study and qualitative evaluation conducted using the systes are outlined. An analysiss of the findings and epirical evidence reported in the study is done together with a discussion of the significance of the results. The paper concludes with a suary of the research contributions ade and the future plans for iproving the prototypes and advancing the research ideas expressed. ENCULTURATED SYSTEM DESIGN AND COMPONENTS Enculturated instructional software systes ust satisfy several requireents, and design decisions were ade in order to satisfy these requireents. A odular design was chosen becausee of the coplexity involved in delivering a culturally-relevant instructional experience. Flexible alteration and iproveent of the coponent features are easily accoodated as a result. Many of the coponents featured in the design of the enculturated web-based ITS are based on the following traditional ITS coponents: a student odel, a doain odel, and an expert odel [8]. Figure 1. Enculturated web-baseapproach is taken. A cultural student odel, cultural heuristics, a content repository, and a pedagogical odule ake up the ajor architectural units of the design. Figure 1 shows how these coponents are connected in a cohesive syste and it also shows the exchange of data aongst the coponents. Web related coponents such as server software and content aggregators will not be discussed since these are standard in web-based ITS syste architecturee A client-server systes. CULTURAL HEURISTICS A culturally-relevant instructional approach requires that cultural references ade by the software syste should be applicable to the learning content, failiar to the students, authentically rendered, and integrated into the context of the instructional aterial [4, 6]. Cultural rules odify the textuall portions of instructional content such as question descriptions, scenarios, hints, and instructional feedback produced by the web-based syste. The enculturation of visual portions of these systes, naely the ultiedia relatedd to the learning activities, is also handled by these rules. The textual odifications include custoizing the language of the instructional feedback, whereas the ultiedia enculturation involves swapping in cultural assets for generalized assets such as iages. Textual outputss of the enculturation processs are sentencess expressed in a cultural dialect specifically esolect fors 1, and equivalent cultural lexical ters. The culturall target is deterined by the student s cultural background (fro the studentt odel). CULTURAL STUDENT MODEL A student odel serves the traditional purpose of recording the student s knowledge levels, learning achieveents, and learning goals. It stores logs of the pedagogical events in the web session. The student odel stores perforance- related data such as prerequisite knowledge, topics copleted, subitted answers, questions copleted, successful and failed attepts at learning activities, nuber of attepts, tie taken, suggested hints and instructional guidance given to the student. Econoides [3] and Blanchard et al. [1] recoend storing cultural learner- related data in a student odel and consequently the cultural learner odel was assiilated into the student odel since they are inherently related. The odel also tracks the player s interaction with the software, and records inforation related to how the syste is being used such as dwell tie on areas of the screen for exaple. It is essentially a snapshot of the player s educational experiences. 27 CONTENT REPOSITORY The content repository handles the organisation and distribution of all web-related, instructional, and cultural assets to the web content aggregator. Enculturated web- ITSs because of the additional diension of cultural based ITSs rely on reusable conten ore than non-cultural personalisation; this was the basis for having a separate asset repository - reusability. The content repository priarily hosts all of the educational and interface-related aterial used by the syste and the student odel. For 1 A variety of language in a Creole continuu that is interediate between the standard for (acrolect) and fors that diverge greatly fro the standard for (basilect).

3 exaple, ultiedia files related to the interface s look and feel, such as icons, logos, and those related to the learning exercises (scenario pictures, feedback pictures) are stored here together with educational aterial such as question descriptions, solutions, feedback files, topic hierarchies. Each of these assets is described by their asset etadata. The etadata descriptions define the context of use and the nature of the assets and are indispensable in the design because they facilitate reuse and exchange of copatible assets. Both the pedagogical odule and cultural heuristic coponent use these descriptions when aking instructional and enculturation-related decisions. PEDAGOGICAL MODULE The pedagogical odule serves the sae purpose as the expert odel in an ITS but was separated fro the student odel for a cleaner design. Instructional rules constantly access and update the student odel in response to input data fro the student and events captured fro the screen. They control how and when instructional feedback is given, and they anage the selection and transition of the learning activities. Any on-screen feedback given is stored in the student odel so that records of student experiences are kept up to date. Nae of Variable ID Topics_Tested Multiedia* Skill_Level Learning_Objectives Question_Description* Question Answer Hints* Description of Learning Activity Variable Unique identifier Topic(s) tested or covered in the learning activity Filenae(s) of ultiedia content used in the activity The skill level that a student ust possess for successful copletion of the activity Skills or knowledge that a student should possess upon successful copletion of the activity Descriptions, instructions, and/or scenarios that guide the student and set the context of the question Content that the student ust anipulate, odify, or select fro. Model answer/content expected fro the student Further guidance corresponding to different parts of the odel answer Table 1. Learning aterial teplate used by the pedagogical odule and cultural heuristics In order for these rules to properly scaffold the learning activities and deterine appropriate feedback for the 28 student, the learning aterials need to follow a teplate used internally or provide etadata descriptions that identify the corresponding parts identified in Table 1 below. SYSTEM IMPLEMENTATION Two web-based ITSs were ipleented based on the software architecture described in the previous section. One syste was enculturated for a Trinidad and Tobago context (Culturally Relevant Instructional Prograing Syste CRIPSY) while the other reained generic (Instructional Non-CulturAl Prograing Syste INCAPS). Both systes were built for the sae educational doain, Coputer Science prograing, and were identical fro a functional standpoint. The systes were ipleented using Java-based tools and technology which facilitated sealess integration of the various coponents into one coplete syste. At the presentation layer, HTML, javascript, and css were used to create the web pages and graphical user interfaces. The Dynaic Web Content Aggregator was ipleented using servlets that rendered and foratted the web pages. Additional servlets were used to deliver required functionality such as validation, verification of input data, and for handling log-ons and session anageent. Apache Tocat 6.0 was used as the web server environent. The pedagogical odule, student odel, and cultural heuristic coponent were ipleented using JESS (Java Expert Syste Shell) rule engines and rules. Interediary java progras were used to connect each of these self-contained units. At the data level, siple file forats were used to anage the content repository since rule engines handle data anipulation priarily using facts. The developent of enculturated assets for CRIPSY was done sei-autoatically and anually. Coputer Science (CS) education stresses the iportance of analytical prograing skills [5] because a large part of the curriculu involves reading, planning, and writing coputer progras. These skills include being able to understand code written by other people such as libraries and full progras, and being able to detect and repair errors in the syntax and logic of progra code. Proper developent of these skills requires rigorous practice sessions with written proble sets in the for of code. Basic skills of understanding progra code and detecting and repairing syntactical and logic/analytical errors were targeted using code snippets related to topics on the prograing curriculu of the target student audience. Constructivis and situated cognition was selected as the ajor instructional strategy since analytical prograing skills were being targeted and also because of the good fit between situated cognition and culturally-aware instruction The prograing exercises descriptions, parts of the exercise code and instructional hints were enculturated using subtle, careful use of cultural seiotics, specifically failiar language and cultural naes of objects and foods. As shown in the screenshots in Figure 2 above, both systes used the sae instructional content but differed in

4 the expression of the content, that is, cultural and non- cultural. A inialist interface design was used in order not to distract the students and to increase ease of use. Instructional feedback consisted of identification of correct/incorrect lines of code, hints for the incorrect lines, and general guidance. Figure 2. Screenshots of CRIPSY (top) and INCAPS (botto) featuring enculturated and non-cultural versions of the sae prograing exercise. After 30 inutes, the sessions tied-out autoatically to ensure that both groups used the systes for the sae duration. A tied post-test was then adinistered, followed by an evaluation survey. At the end of the experient, usage logs were retrieved fro the server achines. RESULTS Initial exaination of the pre-testt (P1) and post-test (P2) scores indicated positive changes in the post-test perforance of both groups of students as illustrated in Table 2 below. Paired t-tests conducted on both sets of presignificant test and post-test scores revealed statistically increases in the students prograing skills after they used the systes. Table 2 shows that the difference in the post-test scores of the test group (CRIPSY) was higher than that off the control group (INCAPS). It should be noted however, the difference between the groups scores is odestt and not statistically significant (p ) Group N P1 P2 Diff CRIPSY INCAPS T-Test EXPERIMENTAL DESIGN AND RESULTS A study was done to evaluate whether the enculturated syste, CRIPSY, was ore effective than the control syste, INCAPS, for increasing analytical prograing skill, and to gauge studentt opinion and interest in the culturally-enhanced approach taken. Participants Sixty (60) students, 31 ales and 29 feales, enrolled in the first year coputer prograing course at the University of the West Indies (U.W.I.) voluntarily participated in the study. Aged between 18 and 47 years (ean20.983, s.d.4.835), 23.3% were of African descent, 41.6% were of East-Indian descent, 1.66% were of Caucasian descent, and 33.3% were of ixed ethnicities. Procedure The students were randoly assigned to a control group (n30) and a test group (n30). A tied pre-test was adinistered to both groups, then the students logged on to the test servers in the Coputer Science laboratory at U.W.I. Each student was given a unique usernae and password that activated the respective pre-assigned syste. The student was therefore unaware of whether the syste was cultural or non-cultural prior to logging on. The test group used CRIPSY and the control group used INCAPS. 29 Tablee 2. Changes in pre-test and post-test scores for the test group (CRIPSY) and control group (INCAPS) Overall, the studentss had roughly equivalent averages for tie-on-task with the prograing exercises for the test and control groups. Greater variation was found for the nuber of correct, incorrect and total attepts ade at copleting the exercises between the test and control groupss as shown in Table 3 below. Correct Questions Correct Attepts Incorrect Attepts Total Attepts Tie-on-task CRIPSY INCAPS s.d s. d s.d s. d s.d s. d s.d s. d s.d s. d Table 3. Suary statistics extracted fro session log files for thee test group (CRIPSY) and control group (INCAPS) Analysis of the logged session data revealed positive, very significant linear correlations (p< <0.01) between the total tie spent on the exercises in the systes and the total nuber of attepts ade for both the test (r0.584) and controll groups (r0.519). Strong, extreely significant

5 correlations (p<0.001) were also found between the total nuber of correct attepts ade and the total nuber of exercises copleted successfully for both the test (r0.899) and control groups (r0.932). The total nuber of incorrect attepts and the total nuber of exercises copleted successfully were very significantly correlated for the test group (r0.519, p<0.01) but only weakly related for the control group (r0.102, p>0.1). A strong significant correlation was found between the positivity of the syste rating given by the test group students and the total nuber of attepts they ade at the exercises (r0.589, p< 0.01) whereas a weak negative correlation existed for the control group (r-0.097, p>0.1). In the subjective assessent survey, 56.2% of the test group students rated CRIPSY as really good and pretty good, 42.3% rated it as ok, and 11.5% rated it as not good. The ost popular reasons for liking CRIPSY were helpful hints, interesting and funny proble contexts, encouraging understanding of prograing errors, and the use of cultural language. The ost coon reasons for disliking CRIPSY included server glitches, lack of flexible answer forats, unreadable or too uch cultural language, and confusing proble contexts. 43.5% of the control group students rated INCAPS as really good and pretty good, and 56.5% rated it as ok. None of the control group students rated INCAPS as not good. The ost popular reasons for liking INCAPS were interesting and funny proble contexts, encouraging understanding of prograing errors, and helpful proble descriptions. The ost coon reasons for disliking INCAPS included server glitches, lack of flexible answer forats, unreadable cultural language, and distracting proble contexts. The difficulty of the exercises received ostly siilar reviews. Around 4% described the exercises as challenging fro both groups. 74% of the control group and 76% of the test group found the exercises to be neither easy nor difficult. The reaining 22% of the control group rated the exercises as easy copared to 16% of the test group. Interestingly, 4% of test group gave a rating of very easy. The instructional feedback was identical in both systes with the exception of the enculturated language used in CRIPSY. Students rated the feedback in CRIPSY as helpful (85.7%), encouraging (10.7%) and useless (3.6%) whereas INCAPS was rated as helpful (69%), encouraging (24.1%), and confusing (6.9%). ANALYSIS OF RESULTS The increases in test scores for both groups were significant and provide strong evidence for the success of Intelligent Tutoring Systes in increasing student perforance. Although only arginally larger, the group that used the enculturated syste, CRIPSY, produced larger gains copared to control group. This confirs the assuption that cultural interventions do indeed have positive effects on learners [6, 7] and provides epirical evidence in support of enculturated learning systes. In addition, the 30 sealess integration intended aongst the cultural software coponents and the traditional ITS coponents was achieved since siilar usage patterns were found in both the test and control group session logs. Cultural eleents did not detract away fro student behaviour coonly reported for ITSs since siilar correlations were found for between the total tie spent on the exercises in the systes and the total nuber of attepts ade, and between the total nuber of correct attepts ade and the total nuber of exercises copleted successfully for both experiental groups. The enculturated syste however perfored slightly better copared to the control syste since the total nuber of incorrect attepts and the total nuber of exercises copleted successfully were strongly correlated. This iplies that both correct and incorrect attepts ade with the enculturated syste proved to be beneficial for copleting the exercises. Making attepts also seeed to be encouraging for students in the test group since the positivity of their syste ratings increased with the nuber of attepts ade. This relationship was weaker for the control group which suggests that the culturally enhanced syste ay indeed have prooted a ore relaxed learning atosphere as theorized by proponents of culturally-aware instructional environents [6,7]. Both systes suffered fro parallel probles related to server glitches and software bugs. Inflexible answer forats understandably caused confusion for both sets of students since correct alternative prograing solutions were not accepted by the pedagogical odule. This resulted in soe students being blocked fro advancing to the next exercise despite having correct answers. Surprisingly, these probles did not decrease the appeal of the tutoring systes because students were eager to have access to the systes when the experient was over. Another design issue that affected students was the static cultural language used in the enculturated syste. For soe students the density of the cultural ters used distorted the text and ade the sentences difficult to read quickly and therefore difficult to understand quickly. An initial suggestion of incorporating a custoizable language density scale was proposed as way of dealing with this proble so that users ay adjust the language to suit their own preferences. This feature was iediately confired as desirable by the students. Overall, the students liked the enculturated syste priarily because of the reasons outlined in earlier studies [9, 10] enriching learning experiences and huour. The use of culture created a failiar setting and it was done in a way that was interesting to the students. Although both systes eployed an anchored instructional approach, a larger percentage of students rated the prograing exercises as easier for the enculturated syste copared to the control syste. The instructional feedback/hints were also deeed to be ore helpful by the students using the enculturated syste again despite providing siilar guidance to those used in the control syste.

6 CONCLUSION Culture is rapidly becoing an iportant consideration in the design of elearning software firstly because of the increase in the nuber of users accessing software over the Internet, and secondly because of the sheer diversity in the cultural backgrounds of these users. Conventional learning has often taken place in a localized setting with a teacher guiding one or ore students in their search for knowledge. With the advent of the Internet, this traditional setting has changed drastically since students now have access to teachers and educational aterial fro over wide distances. Consequently, these students are exposed to a variety of educational tools, teaching strategies and learning aterials which were not developed with their own personal needs in ind. This has draatic usability iplications especially when the ainstrea culture for which e-learning aterials are designed clashes with that of the users. Based on the encouraging evidence established by the study, the research discussed in this paper deonstrates a practical approach towards developing an enculturated webbased learning environent. By leveraging research fro various fields such as Intelligent Tutoring Systes and culturally-aware instruction, this work shows how the coplexity of enculturation can be anaged and how aspects of intelligent tutoring can be enculturated. Epirical evidence indicates that enculturated systes perfor as well as traditional tutoring systes and are potentially superior at creating relaxed, engaging learning atospheres for the Coputer Science prograing doain. However care ust be taken to ensure that the cultural enhanceents atch the tolerance level of the student users. Further refineent and iproveents are planned for the systes described. A liited aount of cultural autoation was undertaken, so expansion of the cultural coverage is necessary. Additional features such as deeper cultural learner profiling, adjustable language density, and greater tutoring flexibility are also part of the plans intended for this research. REFERENCES 1. Blanchard, E., Razaki, R., Frasson C. Cross-cultural adaptation of e-learning contents: A ethodology. In: Richards, G. (ed.): ELearn 2005, AACE (2005), Blanchard, E.G., Mizoguchi, R. Designing culturallyaware tutoring systes: Toward an upper ontology of culture. In E Blanchard, and D Allard (eds.), Culturally Aware Tutoring Systes, Montreal, Canada, (2008), Econoides, A.A. Culture-aware collaborative learning. Multi-cultural Education & Technology Journal 3, 4 (2008), Fleer, M. Reflecting indigenous culture in educational software design. Journal of Reading 32, 7 (1989), Gray, S., Edwards, S., Lewandowski, G., and Shende, A. Iproving student prograing skills by developing progra coprehension abilities: Panel discussion. Journal of Coputing Sciences in Colleges 20, 3 (2005), Henderson, L. Theorizing a ultiple cultures instructional design odel for e-learning and teaching. In Edundson A. (ed): Globalized E-Learning Cultural Challenges, Inforation Science Publishing: London, UK (2007), McLoughlin, C. Adapting e-learning across cultural boundaries: A fraework for quality learning, pedagogy, and interaction. In Edundson A. (ed): Globalized E-Learning Cultural Challenges, Inforation Science Publishing: London, UK (2007), Mills, C., Dalgarno, B. A Conceptual odel for gaebased Intelligent Tutoring Systes. In Proc. Ascilite s/ills.pdf (2007) 4. Mohaed, P., and Mohan, P. Student attitudes towards using culturally-oriented educational gaes to iprove prograing proficiency: An exploratory study. In: Edutainent, LNCS 5670, Springer-Verlag: Berlin (2009), Mohaed, P., and Mohan, P. Cobining digital gaes with culture: A novel approach towards boosting student interest and skill developent in Coputer Science prograing. In: 2010 Second International Conference on Mobile, Hybrid, and On-Line Learning, IEEE Coputer Society (2010), Reh, M. Developing enculturated agents- Pitfalls and strategies. In E.G. Blanchard, D. Allard (eds): Handbook of Research on Culturally-Aware Inforation Technology: Perspectivies and Models. IGI Global (2010) 7. Young, P. Integrating culture in the design of ILTS. British Journal of Educational Technology 39, 1 (2008),