The Role of Organizational Devices in ESL Readers Construction of Mental Representations of Hypertext Content

Transcription

1 Khalid Al-Seghayer 531 The Role of Organizational Devices in ESL Readers Construction of Mental Representations of Hypertext Content KHALID AL-SEGHAYER Imam University (English Department at Al-Kharj Community College) ABSTRACT The current electronic text format is inherent to the problem of text integration, or, alternatively, cohesion deficit, which greatly affects reading comprehension. The question remains as to whether well structured hypertext would enable L2 readers, particularly ESL readers, to overcome potential difficulties in integrating information and building a unified representation of text content delivered by computer. This study examines the efficacy of embedding microstructural devices (e.g., headings, reviews, and logical connectives statements) and macrostructural devices (a graphical overview map of text content) on the construction and organization of computer hypertext presentations. The study also investigates the interaction between the effect of L2 readers reading proficiency and the degree of structure in hypertext (structured vs. less structured) on ESL learners development of coherent mental representations of hypertext content. The participants, 40 ESL students, were introduced to two hypertext reading programs. The first was considered well structured hypertext because it included organizational devices and explicitly showed its underlying hypertext structure. The second was considered less structured because it included no organizational devices and did not indicate the underlying structure of its hypertext. A repeated measures design was used in this study. Forty participants were measured under two conditions: well structured and less structured hypertexts. They were identified as intermediate ESL learners based on their TOEFL scores and were classified as proficient or less proficient readers based on their scores on the reading section of the TOEFL. To assess the efficacy of each type of hypertext, multiple choice and mapping main ideas and details tests were developed and administered to participants after they had read both hypertexts. Results of both tests were analyzed using a paired-samples t-test to compare performance in the two hypertext reading programs and a two-way (proficiency level by hypertext reading program) mixed model ANOVA. The investigation yielded findings showing that well structured hypertext aided ESL readers in developing a more coherent mental representation of the hypertext content, thereby increasing their reading comprehension. The results also indicated that well structured hypertext was more helpful to less proficient readers than it was to more proficient readers. These results offer theoretical, pedagogical, and technological implications for L2 reading instructors and instructional designers. CALICO Journal, 24 (3), p-p CALICO Journal

2 532 CALICO Journal, Vol. 24, No. 3 KEYWORDS Hypertext, ESL Reading, ESL Proficiency, Text Structure, Microstructural and Macrostructural Devices INTRODUCTION Computer technology is becoming increasingly prevalent as a means of information presentation in education, particularly in second language (L2) education, and thus has come to play an important role in the field of reading. Much attention has been given in recent years to the ways in which the features of electronic reading environments promote the reading experience and change the nature of reading (Dee-Lucas, 1996; Dee-Lucas & Larkin, 1995; Foltz, 1992; Greenlee-Moore & Smith, 1996; Hofman & van Oostendorp, 1999; Lacroix, 1999; Reinking, 1988, 1995; Reinking & Schreiner, 1985). Electronic reading environments possess attributes that are unavailable in any other medium of instruction as a result of their text format, or hypertext. In hypertext, information is divided into units that serve as hypertext nodes. Each unit is displayed on an overview map or in an alphabetized list as a title that reflects its content. Units are interconnected by links, words or icons that users can select to move among units. Because of these features, readers of hypertext are not tied to a predefined path for reading but, rather, can read units in many possible sequences. Readers of hypertext also have immediate access to a variety of representational modes, including sound, still pictures, dynamic video clips, and animation, as well as access facilitators such as browsers, maps, and menus. Moreover, readers can interact with the information presented, moving to different points in a text as needed and reacting to multiple linked sources of information. The hypertext environment allows more flexibility in reading, but it may also lead to fragmented representation of the overall text if the units and links do not give readers cues to the way information is structured. When a hypertext lacks explicit cues to its structure, readers must try to reconstruct the relevant relationships on their own. Reading theory suggests that providing microstructural devices such as headings and logical connectives, as well as macrostructural devices such as graphic representations of content, can help readers structure information appropriately because such devices indicate the relationships among units, thus helping readers to integrate information into a coherent whole. Within the context of second language, in spite of hopes and claims about the potential benefits of computer technology for L2 reading, few empirical studies have examined its effectiveness. In fact, most of the issues and concerns associated with reading hypertext have been derived from studies conducted with first language (L1) readers. Studies are needed to develop a solid understanding of how computers can be used effectively in L2 reading and how L2 readers perceive the experience of hypertext reading. This study examines the effects of embedded microstructural devices (e.g., headings, reviews, and logical connective statements) and a macrostructural de-

3 Khalid Al-Seghayer 533 vice (a graphical overview map of hypertext content) on construction 1 and organization 2 in L2 readers processing of hypertext presentations. It also investigates the relationship between L2 readers proficiency with reader- and author-defined interactive visual and verbal organization devices and the effects of these devices on coherent mental representation building of hypertext content. 3 REVIEW OF RELATED LITERATURE To establish a context, this review will cover L1 and L2 hypertext reading research and L1 and L2 reading research in general. The review also includes studies on printed text because traditional reading research can provide valuable perspectives on text processing. Factors Influencing Construction and Organization of a Coherent Mental Representation of Hypertext Content The specific task of developing a coherent mental representation and the general task of comprehending a hypertext document are significantly influenced by (a) explicit representation of the text s underlying structure, (b) the overall coherence of the information, and (c) the readers reading abilities. a. Representation of Structure Knowledge of structure enables readers to predict what material the text will include and how that material will be organized. The sequential presentation that characterizes hypertext environments makes it difficult to get a complete picture of the text unless the hypertext environment includes local (microstructural) features that help readers select the most important textual information, as well as global (macrostructural) features to assist them in integrating the content into a unified representation. Local (microstructural) devices. Text-based techniques that may affect the construction and organization of a coherent mental representation are structural markers or discourse cues that clarify the structure of the passage. These cues provide a conceptual framework for the reader to use in organizing the information into a coherent overall representation. Three microstructural devices include headings, logical connectives, and reviews. Headings. Headings and subheadings are common typographical cues in traditional text. Electronic text uses two types of headings: local headings and global headings. Local headings introduce the topic of each unit and identify important information. They help readers remember important points from each passage and summarize the information within units. Global headings identify the structure of the document. They influence complex reading activities by helping readers plan reading activities and by encouraging them to examine the important information in each passage (Lacroix, 1999; Lorch, 1989). These functions have been empirically confirmed. Lacroix (1999) conducted three experiments in which she examined the effects of different textual organizations on macrostructural construction and organiza-

4 534 CALICO Journal, Vol. 24, No. 3 tion. In her third experiment, she explored the effects of headings on macrostructural construction and organization. Eighty college students were instructed to read small sets of documents presented in hypertext format under two different conditions. Half of the subjects read selected texts that incorporated local and global headings, and the other half read texts without headings. Both groups were asked to write a summary and prepare a report outline. The results indicated that subjects who read texts using global and local titles were more likely to include important statements in their summaries and that the presence of both types of headings helped readers remember the most important ideas in the texts they had read. Dee-Lucas and Larkin (1995) obtained similar results when comparing college students reading traditional text, electronic text with a structured overview, and electronic text with an unstructured overview. The results indicated that the presence of unit titles on the overviews of electronic text led to a great deal of recalled content. Dee-Lucas and Larkin proposed that unit titles served as recall aids and, as a result, increased text content recall. They contended that readers used their memories of the unit titles as recall aids for the text topic. Logical connectives. Logical connectives reveal the logical relations among ideas in a text (Mayer, 1984) and help readers construct a coherent mental representation because they explicitly indicate the relations between text units (Golding, Millis, & Hauselt, 1995; Kintsch, 1990). Previews. Preview statements outline text content and purpose, preview sentences emphasize the structure of the passage and reveal the relationships between superordinate and subordinate ideas. To date, no studies have examined the impact of including logical connectives and previews in a hypertext environment. However, some studies have looked at their use in traditional texts. Loman and Mayer (1983) examined the effect of using reviews, headings, and logical connectives on 58 high school readers understanding of expository text. Participants were divided into two groups. One group read the signaled version of the selected text and the other group read the nonsignaled version. Both versions were the same except that the signaled version had review sentences that stated the purpose of the text and used headings and logical connectives. After reading their texts, participants were required to take a recall test, a problem-solving test, a fact retention test, and a verbatim retention test. Their results showed that students who read a text containing these features recalled conceptual information and generated higher quality problem solutions than students who read the same text without the local devices. Spyridakis and Standal (1987) also investigated the effect of headings, previews, and logical connectives on reading comprehension in traditional texts. Three hundred college students read one of four passages and then answered questions examining information about details in the passages, superordinate information, and implicit relationships. Their results showed that these features helped readers comprehend the text they read. Spyridakis and Standal suggested that these techniques communicated the structure of the texts to readers and increased their understanding.

5 Khalid Al-Seghayer 535 Global (macrostructural) devices. Recognition of macrostructure is a prerequisite for success in tasks involving global comprehension and meaningful learning. Because multiple paths are possible when reading a hypertext, there is a greater navigation load on the reader than with linear text. In order to simplify the reader s task, therefore, global navigation features must be provided. One common global device in hypertext is a graphic representation or map that can help readers identify the structure of the text as a whole (Van Dijk & Kintsch, 1983). Tang (1992) explored the effect of graphic representation on reading comprehension among 45 ESL students. Comparing students who completed a tree graphic about a reading with students who did not, Tang found that graphic representation of knowledge facilitated awareness of text structure, reading comprehension, and immediate recall. The majority of participants agreed that using the graphic facilitated comprehension and recall. Tang suggested that graphics incorporated with linguistic devices enable readers to reconstruct their knowledge after reading and express it in a second language. Dee-Lucas and Larkin (1995) found that displaying the underlying structure of text through hierarchical overviews produced better memory for text topics and better breadth of recall than a menu-like content listing and that hierarchical overviews were easier for readers to recall and use. Furthermore, they reported that readers with overview lists had more difficulty determining what was important in the text as a whole. They maintained that this difficulty was due to the lack of explicit organizational information in the overview list. Hofman and van Oostendorp (1999) conducted a study in which they sought to examine whether a structural overview led to a deeper understanding of texts. Forty first-year university students were assigned to read a text under one of two presentation conditions, either a structural overview or a list. The students were asked to answer text-based questions intended to measure their recognition and inference questions designed to measure their situation model that they had constructed from the information they read. The results demonstrated that low- and high-prior-knowledge participants scored equally well on the text-based questions, whereas on the situation questions, low-prior-knowledge participants scored significantly lower on the overview condition than on the listed condition. b. Overall Coherence of Text Text coherence is a major factor in constructing a coherent mental representation of hypertext content. Developing a coherent mental representation of text content entails integrating information into a unified model that identifies the central ideas. Foltz (1992) compared readers comprehension and strategies when reading a printed text, an unembellished hypertext, and a coherent hypertext. Seventy-three undergraduate students were assigned to study an introductory economics chapter using a hierarchically designed hypertext system. After reading the text, the participants answered comprehension questions (five short-answer essays, and eight multiple-choice questions) and wrote an essay on the chapter. Participants who

6 536 CALICO Journal, Vol. 24, No. 3 read the hypertext scored better on questions that assessed their knowledge of the structure of the text; also, the more coherently the text was read, the better the reader s summary essay. In addition, the results indicated that participants relied heavily on the hypertext map for orientation. Foltz contended that readers had a strong need for coherence when studying complex materials. McNamara, E. Kintsch, Songer, and W. Kintsch (1996) conducted two experiments examining the role of text coherence in middle school students comprehension of science texts. Their findings from the first experiment indicated that participants who read a more coherent version of a text recalled more of its content than participants who read a less coherent version. In a second experiment, they found that readers who had little background knowledge on the subject matter benefited from a text that was maximally coherent at both the local and the global level, while high-knowledge readers benefited from a minimally coherent text. c. Readers Abilities in Hypertext Environments Another factor that is believed to have an impact on constructing a coherent mental representation and comprehending hypertext documents is readers readinglevel proficiencies. User characteristics have been the subject of investigation in many studies of hypertext because readers approaches to hypertext systems vary widely due to their reading styles and abilities. Leidig (1992) investigated the relationship between cognitive style (learning styles and spatial ability) and structural maps in hypertext. Participants completed a spatial ability test and then were categorized into one of the following four learning styles: (a) divergers, (b) assimilators, (c) convergers, and (d) accommodators. The 282 participants were assigned to one of four treatment groups: (a) linear text only, (b) hypertext without a navigation map, (c) hypertext with a textual navigation map, and (d) hypertext with a spatial navigation map. After reading, participants responded to a questionnaire which was designed to measure the amount of material they had learned and their levels of satisfaction with reading in hypertext environments. The obtained results yielded findings that demonstrated a significant interaction between learning styles and structural maps. Leidig found that assimilators performed well with both textual and graphical maps of the hypertext structure, whereas convergers and accommodators were less effective in similar conditions. Balcytiene (1999) compared readers of printed text with readers of hypertext to investigate the role of individual differences in using hypertext environments. Based on the research results, participants were classified into self-regulated and cue-dependent readers. Self-regulated readers seemed to benefit more from hypertext environments because (a) they were more capable of using their metacognitive skills, (b) they had less test anxiety, and (c) they were globally oriented, retrieving information for initial construction of a mental model that they later elaborated, whereas cue-dependent readers were locally oriented, reacting to the topical demands of a particular guiding question.

7 Khalid Al-Seghayer 537 McNamara et al. (1996) found that a well developed structured reading environment improved the comprehension of readers with low prior knowledge and impaired the comprehension of high-prior-knowledge readers. The authors proposed that a well developed structure reduced the amount of active processing that skilled readers were allowed to do. Conclusion to the Reviewed Related Literature This survey of relevant literature pertaining to the processes of reading and comprehending hypertext documents and the factors that may affect the development of a global coherent representation of hypertext content leads to some tentative conclusions. It also suggests some implications and point to an area of research that has not been thoroughly examined in L2 reading research. The contribution of hypertext to reading in general, and to L2 reading in particular, adds a new, nonlinear dimension to the task of reading. It also makes readers assume a different, more active role in the process of reading and introduces another element to the meaning-making process due to discontinuous text divisions. Without the inclusion of some organizational devices, text progression can be expected to be interrupted which can hamper the development of a unified integrated text representation. These factors underscore the need for research in a largely unexplored area within the field of L2 reading the impact of providing a visual and verbal conceptual orientation of the structure of hypertext at both the local and global levels on L2 readers ability to construct a coherent mental representation. The importance of this issue hinges on the fact that the prerequisite to understanding and acquiring knowledge from a hypertext document is L2 readers ability to develop a coherent representation or conceptualize a structural overview of texts. THE CURRENT STUDY In the hope of developing L2 reading comprehension, some L2 reading specialists have embraced computer technology because of its potential to support effective reading environments. Available L2 reading programs seem to focus primarily on the information presentation capabilities of the medium and pay less attention to the cognitive processing issues involved in reading electronic texts. In order to comprehend a hypertext, readers are expected to build a detailed representation of the content at the local level as they build a more general representation of the global meaning of the text. The factors that influence building a mental representation of electronic-text content have not been thoroughly explored. Little empirical research has examined the impact of providing visual and verbal conceptual orientation to the structure of hypertext at both the local and global levels on building a coherent mental representation. Hence, this study sought to determine whether embedding organizational devices in hypertext would help L2 readers develop coherent representations of hypertext content, that is, whether depicting the representation of the underlying structure of a text and displaying how it is organized could provide a conceptual

8 538 CALICO Journal, Vol. 24, No. 3 framework for readers to use in organizing information into a coherent representation. Empirical evidence of the effectiveness of such devices could aid the design of computer-based reading environments that would help readers organize information and integrate it into a coherent whole. The study s goal was outlined in two research questions. 1. Do organizational devices affect L2 readers processing of hypertext construction and organization and facilitate development of a coherent global representation of content? 2. Do L2 readers reading proficiency and the degree of structuring produce interaction effects that influence the development of mental representations of hypertext content? To answer these questions, the study tested two hypotheses. Hypothesis 1: The presence of an organizational device would significantly influence the two levels of macrostructural processing, that is, construction and organization. This would foster development of a globally coherent mental representation. Hypothesis 2: L2 readers reading proficiency will affect their ability to develop a coherent representation of hypertext content. The construction of coherent mental representations by proficient readers will remain the same under both treatment conditions, whereas for less proficient readers it will be significantly better in the well structured hypertext environment than in the less structured hypertext environment. METHOD Participants The participants in the study were 40 ESL students from 10 different language backgrounds enrolled in the English Language Institute (ELI) at the University of Pittsburgh (see Table 1). They ranged in age from 19 to 40; 27 were women, and 13 were men. At the time of the data collection, students were receiving 20 hours of intensive English instruction each week. Their academic backgrounds reflected a variety of majors, and all were planning to enroll in undergraduate or graduate studies as soon as they passed the language requirement. In order to qualify for the study, each participant had to have spent at least 2 semesters in the host environment and attained a TOEFL score between 450 and 530 (intermediate). In addition, each had to be placed in the intermediate level based on their performance on the reading section of the Michigan Test. The participants instructors were asked to confirm each participant s intermediate proficiency level in reading. Since the study employed a mixed design approach, all participants were exposed to the same treatment conditions. Although the participants had different language backgrounds, the use of a mixed study design meant that each participant served as his or her own control.

9 Khalid Al-Seghayer 539 Table 1 Characteristics of Participants Country Male Female Total Bahrain 2 2 China Japan Italy 1 1 Lebanon 1 1 Korea Morocco 1 1 Mexico 1 1 Qatar 3 3 Peru 1 1 Sweden 1 1 Taiwan Thailand Turkey 2 2 Venezuela Total Intermediate-level ESL learners were chosen because they would be able to deal with text from a more global perspective and would possess enough vocabulary and grammatical knowledge to understand the texts used in the study. In addition, a sample of intermediate-level learners would include both more proficient and less proficient readers. Based on their scores on the reading section of the TOEFL, participants were divided into two categories: proficient (score of on the reading comprehension section of the TOEFL) and less proficient (score of on the reading comprehension section of the TOEFL). The proficient group included 15 women and 5 men with an age range of 19 to 40 years. The native languages of the students in this group included Arabic (2), Chinese (5), Japanese (4), Korean (4), Spanish (2), Swedish (1), Thai (1), and Turkish (1). Sixteen were graduates, and 4 were undergraduates. The less proficient group included 12 women and 8 men ranging in age from 20 to 33. The native languages represented in this group included Arabic (5), Chinese (2), Italian (1), Japanese (6), Korean (1), Spanish (2), Thai (2), and Turkish (1). Fourteen were graduates, and 6 were undergraduates. Setting All the study s sessions took place at the Robert Henderson Language Media Center at the University of Pittsburgh. The center contains 44 computers and is the

10 540 CALICO Journal, Vol. 24, No. 3 lab that most students in the ELI use for school assignments and other related computing needs. Design This was mixed design in which the participants served as their own controls. The between-subjects factor was reading ability with two levels of proficiency (high proficiency and low proficiency), and the within-subjects factor was the use of organizational devices with two levels (presence or absence of the organizational devices). Participants read two analogous hypertexts, one with an organizational device and the other without, and then took two tests. The independent variables, well organized and less organized hypertext reading programs, were experimentally manipulated by exposing participants to both treatment conditions. The dependent variables, construction of mental representation of electronic texts at both the local and global levels, were measured by two different reading tests (see Table 2). Table 2 Study Design Reading ability Text with organizational devices included Text without organizational devices included Proficient readers X X Less proficient readers X X Procedure The experimental procedures consisted of different phases that were undertaken over the course of four meetings. The first phase, which lasted 1 hour and 35 minutes, served as an introductory session in which the participants, either individually or in small groups, met with the researcher at the Language Media Center. Each was informed about the purpose of the study and asked to fill out the informational background questionnaire and take the TOEFL test. The second phase, which lasted approximately 30 minutes, served as a practice session in which participants received a brief introduction to the hypertext reading programs and practiced reading texts similar to the experimental ones. The final phase, which lasted a total of 2 hours, was conducted in two separate 1-hour meetings that served as the experimental sessions. In each meeting, participants were asked to (a) read two analogous expository hypertexts on the computer screen under two different conditions, one with organizational devices included and one without (30 minutes for each) and (b) take the comprehension tests (10-15 minutes for each). Participants completed all tasks within a 4-hour block of time. To avoid any potential order effect, the order of the passages was varied and counterbalanced. The first 10 proficient readers read the less organized hypertext

11 Khalid Al-Seghayer 541 and took the multiple choice and the mapping tests associated with this text; the other 10 read the well organized hypertext and took the test associated with it. Then the order for this group was switched. On the other hand, the first 10 less proficient readers read the very well organized hypertext and took the two tests associated with it; the other 10 read the less organized hypertext and took the tests associated with it. Then the order for this group was switched. Instructional Materials Reading Passages Two passages were selected from Sociology: Experiencing Changing Societies (Kammeyer, Ritzer, & Yetman, 1997), an undergraduate textbook. The two passages, Human Interaction (HI) and Verbal Symbols and Language (VSL), were expository in nature and judged to be unfamiliar to most intermediate-level students. The passages were selected according to seven specific criteria: the texts (a) shared the same subject matter, (b) shared the same literary style, (c) shared the same length, (d) shared the same degree of difficulty, (e) were authentic passages, (f) required minimal background knowledge, and (g) had logical, rhetorical ordering of ideas. In this way, the practical effectiveness of organizational devices (i.e., headings, logical connectives, previews, and graphical representation of text structure in the form of overview maps) on readers ability to create a coherent representation could be detected. Consultation with Reading Teachers Two ESL reading teachers assisted in the design of the organizational devices to be included in the hypertext reading programs. The teachers were asked to suggest the best global and local titles for each unit, to write statements connecting each text unit to the next, and to write a short preview for each unit. They also performed idea-unit analyses on the reading texts, identifying one main idea and two supporting ideas for each paragraph. Converting the Selected Texts into Hypertext Documents The two experimental passages were converted into hypertext documents. Each of the eight paragraphs in each text served as a single unit, filling part of the screen along with the associated local and global titles, a preview of the unit, an overview map (well organized program) and an alphabetized list of text content (less organized program). Content, length, and syntactic structure of the paragraphs remained in their original versions. Thus, there were eight units, each one able to stand alone. Readers could move among them using links. In the case of the HI text, readers were able to click on the overview map titles to display the content of the selected unit on the screen, along with the included organizational devices. In the VSL text, they were able to click on the alphabetical list of titles to display the content of the selected unit on the screen.

12 542 CALICO Journal, Vol. 24, No. 3 Software: Overview of Content and Organization The investigator designed two hypertext reading programs (well structured and less structured) with similar introductory sections and aesthetic features but different main sections. Program designs were based on the design principles proposed by Leshin, Pollock, and Reigeluth (1992), Martin (1990), Nielsen (1990), and Thorn (1995). Both hypertext reading programs along with their tests were developed with Flash MX software (Macromedia, 2002). The audio component was recorded and processed with Digital Performer 3.02 software (MOTU, 2001). Each program contained a tutorial, an introduction, and a main section. The tutorial illustrated how to interact with the screen features and use the program effectively. In addition, in the well structured program, the tutorial prompted students to use the structure of the hypertext to guide their reading. The introduction previewed the hypertexts and explained that each was divided into units that were displayed independently on the computer screen. The purpose of the tutorial and introduction was to reduce the cognitive load imposed by the new instructional environment and to ensure that participants who were less familiar with reading hypertext were not disadvantaged. Overview of the Well Structured Hypertext Reading Program. The well structured hypertext reading program contained the HI text. After the tutorial and introduction, users saw a static global overview of unit titles organized hierarchically and displayed as interconnected semantic nodes along with an explanation of the function and organization of the overview map. The window displaying the actual text was divided into three frames: a title banner across the top, the overview map on the left, and the actual text. The frame containing the actual text included the local and global headings of each unit and two navigation buttons: Previous and Review. The Previous button took users to the previously visited point; the Review button displayed a brief preview of unit content (see Figure 1). Figure 1 Well Structured Hypertext Reading Program

13 Khalid Al-Seghayer 543 The actual text was divided into eight stand-alone units, each consisting of one word paragraph from the original text. Readers could move among the units using the links on the overview map. Instructions on how to interact with the program features were placed at the bottom right of each page. Upon completing the final unit, users had the opportunity to see all of the text presented in every unit and then to return to the overview page. A message informed users that they could select any of the units for review. Overview of the Less Structured Hypertext Reading Program. After viewing the brief introduction to the program, participants saw an unstructured, menu-like list of the unit titles typical of what readers encounter in instructional hypertext environments. The list overview, which filled the whole screen, presented the major topics and segments of the selected text without indicating the relationships among them. The second section contained the VSL text. This text was divided into smaller paragraph units so that each could stand alone, and each unit had a title reflecting its content. The window that displayed the text was divided into three frames. The top frame held the title of the text, Verbal Symbols and Language, the left frame displayed the list-like overview showing the main units comprising the text, and the right frame displayed selected text units. The navigation buttons were located at the bottom of the right frame. The button on the right took users to the next unit, and the button on the left allowed users to return to the previous unit (see Figure 2). Figure 2 Less Structured Hypertext Reading Program Evaluation Measures Evaluation measures were selected to (a) reflect the degree to which a text is represented at the local and global levels, (b) test knowledge of text structure (mapping main ideas and details), and (c) test how recognition of text information

14 544 CALICO Journal, Vol. 24, No. 3 was affected by the experimental manipulations. In addition, they were designed primarily to measure knowledge of conceptual relations rather than simple knowledge of facts or ideas. Production assessments such as free or cued written recall were avoided because it was assumed that the writing ability of the participants was not adequately developed for such measures. Knowledge of underlying textual organization was tested in the form of multiple-choice test items. Each test contained 15 questions with 4 possible answers. In order to produce an accurate measure of the amount of material learned, construction of the multiple-choice items took passage dependency into account and were designed to control for (a) breadth of coverage of the test items themselves, (b) cues in response, (c) plausibility of distractors, (d) careful wording in the test items, and (e) randomization of test items. After reading a hypertext, participants clicked on the Begin Quiz button, which took them directly to the associated multiple-choice test. Introductory instructions and instructions accompanying each question made test taking less confusing, reduced the chance of error, and allowed for an accurate measure of the time users spent on each question. Because the questions required readers to integrate information or to infer something that was not stated in the text, participants were permitted to look at the hypertext while answering the questions. After completing the multiple-choice test, participants went on to the mapping of main ideas and details (MOMID) test which examined sensitivity to text structure. Instructions for this test were presented in an initial window. Each MOMID test consisted of 24 sentences taken randomly from the hypertext, and participants identified each sentence as either the main idea or a detail of a given unit and indicated its status on a map. Participants were given the opportunity to look at the hypertext while answering the questions because the questions required readers to infer or locate main idea and detail information that was not explicitly stated. The MOMID test was constructed in consultation with ESL reading experts and was subject to a number of considerations, including (a) limited alternatives, (b) test items free of response cues, (c) plausibility of distractors, (d) careful wording of test items, and (e) randomization of test items. Delivery of the Multiple-choice and MOMID Tests After completing the multiple-choice tests, participants were asked to continue working on the MOMID tests. Once these were done, they submitted their answers by clicking on the Submit button. Their answers, along with their background information, were automatically sent to the investigator s account and database management file. After completing the MOMID tests, participants received a message which thanked them for their participation and indicated that their data had been received. Test Item Analysis Item analysis of the multiple-choice tests revealed an index of difficulty ranging from 0.55 to 0.92 for the test used in the well structured program and from 0.33 to

15 Khalid Al-Seghayer for the test used in the less structured program. The index of discriminability ranged from 0.20 to 0.55 for the test used in the well structured program and from 0.20 to 0.45 for the test used in the less structured program. For the MOMID tests, the index of difficulty ranged from 0.52 to 0.71 for the test used in the well structured program and from 0.36 to 0.60 for the test used in the less structured program. The index of discriminability ranged from 0.18 to 0.50 for the test used in the well structured program and from 0.20 to 0.50 for the test used in the less structured program. An estimate of the reliability of the internal consistency of the tests was made from a single administration of the tests by using the Kuder-Richardson formula. The internal consistency reliability was 0.74 for the multiple-choice test used in the well structured program and 0.72 for the one used in the less structured program. It was estimated as 0.70 for the MOMID test used in the well structured program and 0.68 for the one used in the less structured program. Scoring System All of the tests were of the discrete-point type, so each was scored following similar procedures. For the multiple-choice tests, each correct answer received one point and each incorrect answer received zero points. For the MOMID tests, full credit was awarded for assigning all ideas correctly, and one point was deducted for each mistake. The answers of the two tests were routed electronically to a MySQL software database with the help of customized code written in the PHP scripting language. This enabled the quiz and tracking results to be sent directly to the researcher s address for evaluation. Data Analysis Procedures The collected data was first subjected to a statistical descriptive analysis. Inferential statistics were then used to predict characteristics of similar sets of data, thus revealing how generalizable the results would be to the whole population of ESL learners. To answer the research question regarding the effect of organizational devices on L2 readers processing of hypertext, a paired-sample t-test was performed on participants scores on the questions that examined their ability to extract main ideas and comprehend content. To answer the research question regarding possible interaction between reading proficiency and degree of structuring, a twoway mixed model ANOVA was conducted. The between-subjects factor in the ANOVA was reading ability, with high proficiency and low proficiency levels. The within-subjects factor was the use of organizational devices, with devicespresent and devices-absent levels. The ANOVA analysis tested for three effects. 1. The test for the main effect of reading proficiency evaluated whether ESL learners with high reading proficiency developed more coherent mental representations of text than ESL learners with low reading proficiency, regardless of whether organizational devices were present.

16 546 CALICO Journal, Vol. 24, No The test for the main effect of an organizational device evaluated whether the effects of text that included organizational devices differed from those of text that did not include them. 3. The test for interaction between reading proficiency and use of an organizational device evaluated whether the differences between the effects of the two types of text were the same for the two reading groups. All statistical tests used an alpha level of.05 as the level of significance. RESULTS The first research question examined the effect of embedded organizational devices on developing a coherent mental representation of hypertext content. For the purposes of this study, coherent mental representation of hypertext content was defined as the ability to identify the central ideas of the text and measured by a set of questions about overall main ideas. To answer this research question, participants responses to questions about the main point of each hypertext were examined. 4 Table 3 presents the results of the paired-samples t-test, which showed that participants performed significantly better on questions that probed the central ideas of hypertexts in the presence of organizational devices (t = 3.85, p <.0005). The mean score on questions that inquired about the central ideas of the less structured hypertext was 1.12, compared to 1.68 for the well structured hypertext. For the less structured hypertext the standard deviation was 0.82, and for the well structured hypertext it was Table 3 Results of Paired-samples t-test on Participants Scores on Questions Pertaining to Overall Main Ideas (N = 40) Without organizational devices With organizational devices Test M SD M SD t p Overall main <.0005 ideas questions The second research question investigated the interaction effects between reading ability and the presence or absence of organizational devices. To answer this question, participants responses on the multiple-choice and MOMID tests were examined. Table 4 shows the means and standard deviations of participants scores on the multiple-choice tests. As can be seen, the mean for proficient readers with the less structured hypertext was and with the well structured hypertext. The standard deviation was 2.22 with the less structured hypertext and 1.76 with the well structured hypertext. By contrast, the less proficient readers mean for less structured hypertext was 6.10, and 9.45 for well structured hypertext. Their standard deviations were 1.44 with the less structured and 1.90 with the well structured hypertext.

17 Khalid Al-Seghayer 547 Table 5 presents the results of the two-way ANOVA. The effect of the interaction between proficiency level and use of organizational devices was highly significant, F (1, 38) = 21.35, p < Table 4 Means and Standard Deviations of Multiple-choice Test Scores by Proficiency Level and Device Use Organizational device LS WS Proficiency level M SD M SD Marginal Proficient (n = 20) Less proficient (n = 20) Marginal LS: Less structured hypertext reading program WS: Well structured hypertext reading program Table 5 Results of Analysis of Variance on Multiple-choice Test Scores (N = 40) Source SS df MS F p Between subjects Proficiency Error <.0005 Within subjects Device <.0005 Device X proficiency <.0005 Error Because of the presence of a significant interaction, a graph of the interaction was examined before the main effects were interpreted. As Figure 3 shows, the interaction is ordinal. Therefore, it is still appropriate to interpret the main effects. The main effect for the inclusion of the organizational devices was highly significant: F (1, 38) = 53.10, p < Regardless of proficiency level, students performed better when reading the well structured hypertext (M = 10.25) than when reading the less structured hypertext (M = 8.20). However, as Figure 4 shows, the structured hypertext had a stronger effect on the performance of less proficient readers. The main effect for proficiency level was also significant, F (1, 38) = 31.68, p <.0005.

18 548 CALICO Journal, Vol. 24, No. 3 Figure 3 Interaction between Presence or Absence of Organizational Devices and Reading Proficiency: Multiple-choice Test Multiple-choice test scores Proficient 6 6 Less proficient 5 without Organizational devices 5 with Table 6 presents means and standard deviations of participants scores on the MOMID test. It shows that the mean of proficient readers reading the less structured hypertext was 6.60 (SD = 1.88), which was close to the mean of 6.55 (SD = 1.85) for their reading of the well structured hypertext. By contrast, the less proficient readers mean when reading the less structured hypertext was 4.60 (SD = 1.64) and 7.60 (SD = 1.79) when reading the well structured hypertext. Table 6 Means and Standard Deviations of MOMID Test Scores by Proficiency Level and Device Use Organizational device LS WS Proficiency level M SD M SD Marginal Proficient (n = 20) Less proficient (n = 20) Marginal

19 Khalid Al-Seghayer 549 Table 7 presents the results of the two-way ANOVA. The effect of the interaction between reading proficiency level and use of the organizational devices was highly significant F (1, 38) = 38.03, p < Because of the presence of a significant interaction, a graph of the interaction was examined before interpreting the main effects (see Figure 4). Table 7 Results of Analysis of Variance on MOMID Test Scores (N = 40) Source SS df MS F p Between subjects Proficiency Error Within subjects Device <.0005 Device X proficiency <.0005 Error Figure 4 Interaction between Presence or Absence of Organizational Device and Reading Proficiency: MOMID Test Mapping test scores Proficient Less proficient 4.0 without Organizational devices 4.0 with

20 550 CALICO Journal, Vol. 24, No. 3 As can be seen in Figure 4, the interaction is disordinal. Therefore, interpretation of the main effects is not appropriate. Without the organizational devices, the more proficient readers performed at a higher level than the less proficient readers. However, when using the organizational devices, the more proficient readers performed at a slightly lower level than the less proficient readers. Stated in another way, the use of the organizational devices had virtually no effect on the performance of the more proficient readers but had a strong beneficial effect on the performance of the less proficient readers. When considering the results for both the multiple choice and MOMID tests, it can be seen that the well structured hypertext had a greater positive impact on the performance of the less proficient readers. DISCUSSION It can be seen from the results presented above that the well structured hypertext reading program enabled participants to develop a better and more coherent mental representation of the hypertext content and increased their reading comprehension. The results also revealed that less proficient readers benefited more from the well structured hypertext program than did proficient readers. Overall, the results confirmed the value of well organized hypertext on L2 reading, particularly for less proficient ESL readers. The results obtained are discussed in the light of the two research questions that guided this study. In reviewing them, an attempt has been made to discuss the findings in detail, determine whether they concur with those of previous studies, and suggest theoretical explanations or justifications for the findings. Global Coherent Mental Representation Participants answers to the multiple-choice questions demonstrated that they were able to recognize the central points of the hypertext far better with the well structured hypertext. In other words, they were able to develop more coherent mental representations of the well structured hypertext than of the less structured hypertext. The effect sizes on developing global coherent mental representations of the well structured hypertext content showed that Cohen s f was medium (.608), 5 suggesting that well structured hypertext made a difference in developing a coherent mental representation. The presence of an overview map that outlined the overall structure of the hypertext and organizational devices (e.g., local and global titles, a preview option, and logical connective statements) seemed to increase ESL readers chances of discovering the gist of the hypertext and, as a result, building a unified representation. The devices provided such assistance because they clarified the form of the well structured hypertext, provided information about how the text units related to each other, and encouraged the readers to learn the overall structure of the hypertext. These results concur with those of Foltz (1992), who showed that the more coherent or organized the hypertext was, the better the essays or the higher the number of macropositions generated in the participants essays. The results also