RESEARCH AIMS AND RATIONALE FOR THIS STUDY
Reading in a second language is considered to be a meaningful language-learning activity, be it on paper or on screen. Indeed, while attempting overall text comprehension, learners interact with different types of cultural, semantic, and syntactic information that can be processed and possibly learned/remembered. Moreover, many second-language specialists see reading as a pre-eminent means of acquiring new vocabulary). This kind of vocabulary learning, where the vocabulary is “picked-up” during normal L1 or L2 reading activities, is referred to in the research as “incidental” vocabulary learning .It is called “incidental” because the major purpose for the interaction with the particular environment or material is not to learn words, but to understand the message of the text and to build up a coherent text base. While reading, learners seem to “guess” , “reconstruct” or “derivePowell”, the meaning of unknown words from the context. Whenever the context fails to deliver the correct meaning of unknown words (when the context contains too many unknown words or whenever the reader lacks the adequate cultural, world or domain the use of a dictionary or of marginal (electronic) glosses is one of the factors that can promote pick-up rates .It is therefore of great importance that students make the utmost use of the dictionary definitions the software provides.
However, a study by Black, Wright, Black, & Norman (1992) indicated that readers use electronic definitions only for short-term purposes and that long-term retention is almost non-existent. Moreover, it has also been suggested that inferred meanings are remembered better than given meanings) and that inferring word meanings from context is still one of the most effective strategies for learning new words. Advocates of this kind of vocabulary learning rely on the theory of “cognitive depth” and high mental effort: the more actively we work out a solution to the problem, the more likely we are to store this information permanently. Summarising, the software programme should provide the learner with dictionary definitions in order to support the reader when the reading becomes problematic, but should also prevent the reader from excessive clicking (called “click happy behavior” by Roby, 1999, p. 98) which could lead to a more superficial, short-term learning. Therefore, the signalling-device of glosses plays an important part. Indeed, on the one hand, a condition in which the hyperlinks are highlighted could incite the users to click excessively, which could lead to a more temporary vocabulary acquisition. On the other hand, a condition where the hyperlinks are invisible could encourage the students to make a more careful use of the glosses and rely more on the context to figure out the meaning of unknown words (which in turn, could lead to more thorough retention of the vocabulary).
Nevertheless, in another line of thinking, it is also possible that the highlights indicating the hyperlinks simply attract attention to the words in question, which could have a positive effect on vocabulary acquisition. Several studies seem to indicate that when a word is made salient, this has a positive influence on its acquisition. In a study on listening comprehension for instance posits that if language items are made salient in any way (e.g., by exercises), they are remembered better than others. Brett did not focus on reading and vocabulary learning and therefore did not address the question of comprehension of the whole text. In reading research, some studies also indicate that when words are made salient, they are remembered better.Does this imply that in a condition without highlights (a graphical tool to make words salient), vocabulary learning would immediately become more difficult? And what would this mean in the long-term?
Some research has looked into the effects of how the users’ willingness to consult an online dictionary definition can be increased. A study by Black et al. (1992; similar but separate from the one mentioned above), involving 20 subjects, suggested that if the glosses are indicated by a small black spot, displayed as a superscript immediately after the glossed word, it increases the students’ willingness to consult this gloss. However, this study did not look into which effects these increased consultations might have on the reading process, actual vocabulary learning, and text comprehension.
As far as the reading task is concerned, research has established that the way in which a text is read is strongly influenced by the readers’ study goals versus knowledge .It is therefore not unlikely that a given reading task might influence the students’ use of either visible or invisible links, which might be reflected in the learning and/or reading outcomes. Yet another small-scale study by Black et al. (1992) could not establish that the involvement of a particular reading task (reading for gist or for detailed knowledge acquisition) differentiates the readers’ clicking behaviour. The five studies by Black et al. (1992) reported on in the single article relate to first-language technical-vocabulary acquisition.
The present study aims to clarify the following issues: When difficult words are highlighted in a software reading programme, does this incite the readers to click intensively, and, if so, does this have a positive effect on the vocabulary acquisition? Or does this lead to excessive clicking, which only results in short-term vocabulary retention? Or does the simple fact that the highlights make some words in the text salient increase their chances of being better candidates for storage in long-term memory? Other studies have looked into the signaling-mode of glosses and have addressed the question of salience, but these studies did not concentrate on the combined effects of marking a hyperlink (i.e., highlighting) and the reading task (general vs. specific) on vocabulary acquisition (short-term and long-term retention), text comprehension, and reading process (reading time, clicking behaviour). Indeed, does increased clicking slow down the reading? Or does reading and rereading a higher number of glosses take up as much time as the combined action of consulting glosses and deriving a word meaning from its context? Moreover, is it the case that when students consult more dictionary definitions, this also leads to a better understanding of the remainder of the text?
A Specific Reading Task Decreases Incidental Vocabulary Learning If it is the case that the specific reading task decreases the students’ willingness to click, and if it is established that increased clicking results in better vocabulary learning, then this might well cause a slight drawback on vocabulary learning.
Marking Leads to a More Superficial, Short-Term Retention of Vocabulary It is more than likely that the results of the delayed vocabulary test would be significantly lower than those immediately after reading, both in the marked and the unmarked condition. However, if, as previous research suggested readers tend to use electronic glosses for short-term purposes only and if in the marked condition readers are more inclined to consult definitions, then it could be expected that the results decrease significantly more in the marked than in the unmarked condition Marking Decreases Text Comprehension Up until now, similar investigations were limited to the aspect of language “acquisition.” To my knowledge, no study has looked into the effects of visible links on text comprehension. I hypothesised that in the marked condition, the attention of the learners would be drawn towards the highlighted words, which would distract them from the overall text and its meaning. It is indeed possible that the constant interruption of the reading process in the marked condition would hinder the students from building up a coherent representation of the text.
A Specific Reading Task Increases Text Comprehension Since the specific reading task was much more oriented towards comprehension, the group with this particular task was expected to perform better on text comprehension than the group with the general reading task.
Marking Does not Slow Down the Reading Process As far as reading time is concerned, I originally assumed that if students click more in the marked than in the unmarked condition, the reading process would slow down. However, in the introductory experiment no difference in overall reading time could be established. I expected to confirm these findings in the present experiment.
Marking has a Negative Effect on the Results of the Free Recall The students of the group with the specific reading task performed a free recall. Its results were expected to be identical to the ones from the overall comprehension test, since the free recall is considered to be a fully integrated part of it. Highlighting could thus have a negative effect on the results of the free recall, because excessive clicking could lead to a poor construction of the text base.
Marking Positively Influences the Results on the Search-and-Find Question The group with the specific reading task also completed additional search-and-find questions. As far as these questions are concerned, the students might perform better in the marked condition, since highlighted hyperlinks might help in skimming a text. For instance, if the students are asked to find the four operations a diamond undergoes when it is cut, they can hypothesise that either the main category might be highlighted. They can start skimming the text while concentrating on the highlighted words, which would then make it easier to find the answer to the question. Previous research could not present conclusive evidence to prove that subjects find a target option faster in a highlighted display than in a display without highlighting. Some researchers favour the highlighted condition and others the non-highlighted condition. Marking Negatively Influences the Concentration Level of the Reader It is not implausible that the blue highlights also have an impact on the students’ level of concentration. In fact, in the interviews concluding the introductory experiment, the subjects involved even suggested that the highlights distracted them. The highlighted hyperlinks either attract the readers’ attention and by doing so increase the interaction with the text and thus intensify the reading, or they lead to a more superficial, “click happy” behavior, where text interaction would be minimal. If students do click excessively in the marked condition and if indeed this has a negative effect on long-term vocabulary acquisition and overall text comprehension, then the concentration level of the students in the marked condition might well be significantly lower than in the unmarked condition. offers an overview of the hypotheses and expectations of the experiment Reading Materials The students involved in the experiment read two glossed French economic texts, comparable in length (about 2,000 words each), grammatical difficulty, and vocabulary load. The texts had been selected for the introductory study mentioned above, after a pilot study involving four texts, 28 students and two faculty members of French. Within this pilot study, the texts were evaluated for interest and difficulty level: the two texts with the most similar score were included in the study. The first text dealt with the diamond industry in Antwerp; the second one was about human resources in business. A word was glossed whenever one student of the pilot study failed to know the Dutch translation of this word in a vocabulary test taken after reading. This procedure lead to the creation of 109 glossed words in the first text and 116 in the second one, which is about 5-6% of the total amount of words. In the existing literature, there is no consensus on how many unknown words a text may contain in order not to disturb the global comprehension level or the learning of vocabulary: West (1941) speaks of 2%, whereas for instance Holley (1973) refers to 7%. Both studies did not include the help of glosses.
The students were asked to read the texts online. They could easily access the glosses by clicking on the defined word. A pop-up window with a Dutch translation and a French definition (separated with a horizontal line) would then appear. I chose to offer the translation and the foreign language definition, since preferences for one or the other are said to be highly individual (Jacobs, Dufon, & Fong, 1994). The pop-up window did not cover up the portion of the text in which the glossed word was found (see Roby, 1999; Stark, 1990). Context-bound explanations were given first; more general information was given at the end between brackets. This difference was explained to the students before starting their reading.
Two versions of each text were created: a marked one and an unmarked one. In the marked condition the glossed words were in blue and underlined. In the unmarked condition the hyperlinks were invisible.The glosses remained identical in both conditions. These authors give recommendations for screen design, based upon both print and computer screen research. In the investigation carried out, the following aspects were divergent from these guidelines: the full justification (instead of left justification) and the line length (83 characters). Scott Grabinger and Osman-Jouchoux propose a line length of 60 characters, while other authors find an increase in reading rate with a greater number of characters per line.
Concentration Level of the Students To establish a possible difference in the students’ concentration during the reading of the marked versus unmarked text, a small attention test for the group of the specific reading task was incorporated. During the online reading sessions, a red rectangle (4 to 3 cm, java-script) randomly appeared (with a maximum of 5 times in the 25 minutes of reading time). The students were explicitly asked to click on it as fast as possible in order to make it disappear. I expected that the longer it took the students to react, the higher their concentration level was and the higher their interaction level with the text they were reading. A similar technique was used in research on writing in order to establish the subjects’ attention capacityThe goal of this test is to disturb the students in their reading, and the longer it takes them to withdraw their attention from the text, the more they are assumed to be concentrating on their reading.
Additional Information Gathering All 60 students were allowed to take notes while reading so as not to disturb their normal reading process. They were not, however, allowed to use these notes during any of the tests. The process data were collected by recording all sessions with Hypercam,ï a software programme that registers every on-screen movement and files them in AVI-videofiles. These files were used as a means of verification. For all 60 students a small interview and some general questions on the student’s reading habits concluded each session. All students participated in three individual sessions. Each session was concluded with a small interview and some general questions. Before reading, the students received a general technical explanation (working of pop-ups, etc.).
The General Reading Task-Group In the first session, the students of this particular group read one of the two texts in one of the two conditions, took a test on text comprehension and completed an unexpected vocabulary test. In the second session, they started with the delayed vocabulary test of the text of the first session, read the other text in the other condition, took a test on text comprehension and a finished with a vocabulary test. The delayed vocabulary test of the second session took place in a brief third session. The 30 students of this group received the following (general) reading task before reading:
Read the text that will appear on the screen very thoroughly and try to understand as much as you can. After having read the text, you will receive a test on text comprehension. You can read as long as you think is necessary.
Thus, no specific time limits were set and no particular information about the testing that would follow the reading was included. gives an overview of the development of the experiment within this (general reading task) group.
The Specific Reading Task Group In the first session, the students of the specific reading task group started with a search-and-find task of one of the two texts, in one of the two conditions. Afterwards, they read the text in question, performed a free recall and took an unannounced vocabulary test and an announced comprehension test. In the second session, they took a delayed vocabulary test of the first text and read and completed the tasks of the second text. The delayed vocabulary test of the second session took place in a brief, third session.
For the search-and-find task, the students received a total of 6 minutes. Afterwards, they were assigned the following reading task, specifically mentioning the form in which they would be tested:
You will receive 25 minutes to read the text that will appear on the screen. Afterwards, you will be asked to give an oral overview of what you have read in the text. You will also receive a comprehension test with multiple-choice and open-ended questions.
As the reader can see, this particular experimental group was subjected to strict time limits, which were pre-tested with advanced learners and were adjusted to this specific group of learners. The students had the same comprehension test as the first group and they took the same vocabulary test after the free recall. In one can find an overview of the development of the experiment within the group with the specific reading task. Activities marked with an asterisk are identical to the ones of the group with the general reading task.
Within the design of this experiment, marking and time are within-subjects variables; reading task is a between-subjects variable. A within-subjects design means that each participant provides more than one response. With a between-subjects variable, every set of responses comes from a different group of subjects.
Effect of Marking and Reading Task on Clicking Behaviour
. Marking thus increases clicking and a specific reading task decreases clicking, as was hypothesised All statistical tests were performed at the .05 level, unless otherwise indicated.
Effect of Marking, Reading Task and Time on Incidental Vocabulary Learning
presents the results of the vocabulary tests, taken immediately after reading and then one week later. These results were analysed with marking and time (test immediately after reading and delayed test) as within-subjects variables and reading task as a between-subjects variable. The results show that on one hand, marking has no significant effect on vocabulary learning. On the other hand, time and reading task do have a significant effect on vocabulary learning: time; reading task, There are no interaction effects: marking reading task,marking time. Both groups thus score significantly lower in the delayed vocabulary test. The group with the general reading task scores significantly better on the vocabulary test than the group with the specific reading task. illustrate these results. Thus, marking does not specifically influence incidental vocabulary learning (contrary to what was hypothesised), whereas a specific reading task does decrease vocabulary learning (as was hypothesised). Moreover, marking does not lead to more superficial, short-term retention of vocabulary learning, since there is no significant interaction effect between time and marking (contrary to what was hypothesised). Effect of Marking and Reading Task on Text Comprehension
shows the results of the overall comprehension test, that is, the multiple-choice questions and the open-ended questions. These results were analysed with marking as a within-subjects variable and reading task as a between-subjects variable. The results show that neither marking nor reading task have a significant effect on the student’s text comprehension: marking, reading task. Thus, marking does not decrease comprehension of the text and a specific reading task does not increase text comprehension (both contrary to what was hypothesised).
Additional Effects of Marking
Total Time Spent on Reading presents the total time in seconds spent on reading, only for the group with the general reading task, since the group of the specific reading task was subjected to strict time limits. These results were analysed with marking as a within-subjects variable, showing that there is no significant effect of marking on the total time spent reading:. Thus, marking does not slow down the reading process, as was hypothesised.
Free Recall shows the results of the free recall performed by the group with the specific reading task. These results were obtained by averaging the evaluations of two independent researchers who analysed the transcriptions of the oral recalls produced by the students. Agreement between the scores of both independent researchers was measured by computing the Pearson product moment correlation coefficients significant at the .01 level). For issues in measuring These results were analysed with marking as a within-subjects variable, revealing no significant effect of marking on the free recallThese results are thus identical to the results on the overall text comprehension test: marking does not negatively influence text comprehension.
Search-and Find displays the results of the search-and-find questions in percentage. These results were obtained by counting all correctly found items. The search-and-find task only applies to the students of the group with the specific reading task. I analysed these results with marking as a within-subjects variable, which shows that there is a significant difference between the results of the marked and the unmarked condition: Marking has a negative influence on the results of the search-and-find task (contrary to what was hypothesised), since the students score significantly better in the unmarked condition.
Attention Test The values in are the results of the attention test. They represent the seconds the students with the specific reading task needed to close the red rectangle on the screen. These results were analysed with marking as a within-subjects variable, which revealed that no significant difference could be established between the marked and the unmarked condition: According to these results, marking does not negatively influence the concentration level of the students (contrary to what was hypothesised).
DISCUSSION AND CONCLUSION
In this section, the questions raised in the research aims and rationale section are revisited and discussed in light of the results of the empirical investigation presented above. The first question posed, which led to several other questions, was whether highlighted or visible links increase the readers’ willingness to consult dictionary definitions. According to the results of the present study, the answer to this question is affirmative. The student users consulted significantly more glosses in the condition with visible links. This finding confirms previous research of Black et al. (1992) concerning first-language acquisition and shorter texts, where a black spot behind the glossed words also attracted the learners’ attention and made them click to access the provided word definition. Is this particular clicking behaviour altered by the reading task? The findings of the experiment indicate that students click significantly more in the marked than in the unmarked condition, be it in the general reading task or the specific reading task group. This might be an indication of how powerful a tool highlighting is to attract the reader’s attention. However, the study also established that readers click considerably more when they are confronted with a general reading task than when they receive a specific one, while previous research (Black et al.,1992) seemed to suggest that a reading task variable does not influence the students’ clicking behaviour. It is possible that the difference between the present results and those of Black et al. are due to the time limits that were imposed on the students.
Furthermore, does the fact that students spend significantly more time consulting information in the marked than in the unmarked condition influence the incidental learning of vocabulary? On the short-term-vocabulary test (taken immediately after reading) no difference was established, which means that whichever different vocabulary-learning strategy the students used in the two conditions can be considered equally effective in the short-term. It seems plausible that in the marked condition, students turn to a vocabulary-learning strategy based upon the reading of dictionary definitions, whereas in the unmarked condition they use a combination of this particular strategy and context derivation. The findings of the present experiment indicate that not using highlights thus making the glossed words less graphically salient, does not particularly influence the incidental learning of vocabulary in a negative way. In both conditions, the students seemed to have adapted their vocabulary learning strategies to the marking situation. In the delayed vocabulary test, the students scored significantly lower than in the test taken immediately after reading. However, the results indicate that in the marked condition, where students were guided more intensively by the highlights and clicked more excessively, the vocabulary loss is not greater than in the unmarked condition. The findings thus do not suggest that readers tend to use electronic glosses for short-term purposes, as the investigations of Black et al. (1992) indicated. The findings of the investigation presented also establish that the use of either strategy does not slow down the reading process, since no difference in reading time was found between the marked and the unmarked condition.
Does the reading task have an effect on the learning of vocabulary? As indicated, reading is often seen in the research as a pre-eminent means of vocabulary learning. The findings of the empirical research presented above clearly indicate that the success of this vocabulary learning depends highly on the reading task that is set. In the experiment, the specific reading task led to significantly less incidental vocabulary learning, which is more than probably due to the time pressure and the specific orientation towards text comprehension.
Did the more intense clicking of the marked condition lead to better text comprehension? Apparently not, and this might be explained by the link that seems to exist between vocabulary knowledge and reading comprehension, which was suggested by previous research (Freebody & Anderson, 1983a, 1983b). Since both conditions result in equal learning of vocabulary, both conditions also result in equal text comprehension.13 In the experiment, text comprehension was measured in different ways: on the one hand, a general comprehension test containing multiple-choice and open-ended questions was used; on the other hand an additional free recall for the group with the specific reading task was employed. Neither of these indicated an effect of marking on text comprehension. The search-and-find question, on the other hand, was subjected to an effect of marking. These results seem to confirm part of the research on localising items in highlighted and non-highlighted displays. It is possible that the highlights confused the students and that in the unmarked condition the text structure helped them to find the items they were looking for.
Did the reading task have an effect on text comprehension? The findings of the current investigation indicate that, contrary to what was hypothesised, the introduction of the specific reading task did not lead to a more thorough understanding of the text. This might have been due to the time limits that were set. The fact that students concentrated more on the text content when they received a specific reading task is suggested by additional information from the notes they took while reading. In these notes, they wrote down exclusively content-related items, whereas the other group (general reading task) wrote down more vocabulary. Are students more concentrated in the unmarked than in the marked condition? Is it possible for them to interact more intensively with the text in the unmarked condition? The attention test that was introduced did not reveal any differences in the concentration level of the students.
The results of the current experiment do not all confirm the findings of the previous, introductory experiment. Globally, the findings are consistent. However, the present ones are much more detailed and complete (especially on the level of text comprehension). The previous experiment did indicate a better result on the short-term vocabulary test after reading the texts in the marked condition. Nevertheless, since the internal consistency scores of the current tests were higher than the previous ones, it may be assumed that the current results are more reliable. On the other hand, the p-value of this particular effect is here .059, which is not statistically significant when a 5% level of significance is maintained. However, I believe that this value indicates that further research might not be redundant.
Summarising, the present experiment made it possible to evaluate what influence the signalling-mode of electronic glosses has on vocabulary learning, text comprehension and the reading process. Indeed, when foreign-language learners read a text where the link with the gloss is visible (highlighted), they are more willing to consult the gloss. However, this increased clicking does not slow down the reading process, nor does it increase the vocabulary learned incidentally. On the contrary, when reading a text in a condition with invisible links, the students’ clicking will be less excessive and better determined, leaving room for context derivation, which, in the long-term however, does not particularly have a positive effect on vocabulary learning. The fact of highlighting or not highlighting the hyperlink does not have an impact on text comprehension either. Apparently, the readers seem to adapt their reading strategies and vocabulary learning strategies to the screen-situation they are confronted with. The reading task then does not alter the clicking behaviour of the students since they still click considerably more when visible links are presented, even when carrying out a specific reading task. However, the reading task did influence the students’ vocabulary learning: A content-oriented reading task seems to decrease the reader’s attention for vocabulary.
Nevertheless, in a future follow-up investigation, some issues of the present experiment could still be improved, for instance, the way in which the study dealt with prior knowledge. I think my reasons for not including a pre-test in the experiment are defendable. The vocabulary test was based upon results with similar students and in co-operation with the students’ teaching assistants. By asking the students whether they knew the word before, a means of verification was included. In the end, 30% of the words of the original vocabulary test was not taken into consideration, yet an internally consistent vocabulary test was kept with the remaining items. As far as text comprehension is concerned, some questions could not yet be answered. For instance, no unreliability of this particular test.