Second Language Assessment

Using ICTS to improve second-language assessment

Patricia Rosen, Memorial University of Newfoundland

Problem

The development of authentic assessment and evaluation situations for second language (L2) learners continues to be an issue (Cummins & Devesne, 2009). Aydin (2006) reported on the scarcity of data available on assessment and evaluation in L2 and Ketabi and Ketabi (2014) found that there has not been a consensus on the distinction between types of assessment in second language learning. East and King (2012) pointed out that the ‘washback’ effect is a challenge to authentic assessment as teachers focus on what is required for a final test rather than on the individual needs of learners.

Another challenge has been to create valid evaluations that reflect real life situations without seeming contrived (Laurier, 2004). Luk (2010) found that University Oral Proficiency Interviews were contrived, resulting in “…features of institutionalized and ritualized talk rather than those of ordinary conversation” (p.47). Birch and Volkov (2007) reported the difficulties in reliable assessment of classroom conversation as weaker learners were reluctant to speak and stronger students dominated conversations.

Ketabi and Ketabi (2014) observed that authentic assessment in large classrooms is particularly difficult so traditional testing, focused on ‘…gathering scores…’ (p. 438), was used most often. They noted that this could create stressful situations that hindered student performance. East and King (2012) also identified difficulties such as anxiety and helplessness related to ‘high-stakes testing’, specifically listening tests with ‘once only at normal speed’ (p. 209) input for test-takers. Wagner (2010) argued that widely used listening tests relying only on the auditory channel without visual input are not authentic because they do not allow for the interpretation of visual along with linguistic cues. Jones (2004) also demonstrated a need for incorporating both auditory and visual channels when designing second language vocabulary evaluations, which were traditionally based on either recognition or recall activities.

Role of ICTs

Laurier (2004) highlighted ways in which ICTs were a natural choice for assessment and evaluation. He recommended using ICTs for evaluation management, facilitating authentic evaluation situations, and increasing student control, specifically with e-portfolios. Cummins and Devesne (2009) also suggested that e-portfolios were a way to integrate authentic assessment practices into L2 learning environments. They pointed out that e-portfolios allowed for the inclusion of a variety of artefacts as well as links between goals, progress, and production samples. Luk (2010) concluded that digital technologies could facilitate the collection of data for oral assessment and also suggested the use of ‘oral language portfolios’ with which to organize this data.

There are a number of ways that ICTs can facilitate authentic evaluation situations (Laurier, 2004). The assessment of student conversations was facilitated through video-recording in Luk's (2010) study of school-based oral assessment through group interaction. Jones (2004) reported that using ICT to create multimedia learning and assessment activities with a blend of pictures, text, and sound led to greater vocabulary retention amongst university students. Learners could choose how to access the vocabulary in the assessments and were most successful when the learning and assessment activities were similar in design. East and King (2012) discovered that student comprehension increased when they used technology to slow down the audio of listening comprehension tests. They recommended teachers use this technique as a way to scaffold learning when preparing their students for high-stakes listening tests.

Odo (2012) suggested expanding the idea of language ability to include using technology for communication. He discovered that learners instinctively used the affordances of technology, such as highlighting and playing with font size, to increase their understanding of reading assessments. Aydin (2006) discovered that using computers for writing tests resulted in higher test scores and higher inter-rater reliability for the assessment of university ESL students. Li, Link, Ma, Yang, and Hegelheimer (2014) found that the use of ICTs for automated writing evaluation resulted in ongoing feedback on student performance and increased student revision of their writing. In Wagner’s (2007) study on the use of video for the assessment of listening comprehension among college students, he discovered that video’s affordances could allow for a more authentic evaluation experience as students were able to access non-verbal as well as verbal communication cues for comprehension.

Birch and Volkov (2007) discovered that assessing online discussion commentary of L2 university students gave learners equal opportunities to participate. They reported higher learner engagement than in face-to-face discussions and easier identification of learners at risk in order to provide them with feedback to improve their performance. Vincent-Durroux, Poussard, Lavaur, and Aparicio (2011) supported the use of online language programming and assessment for the improvement of specific skills in university ESL learners.

Obstacles

Laurier (2004) noted that technology was not a one-size-fits all solution and that what could be an appropriate ICT solution for placement testing was not necessarily the same for formative assessments or final examinations. He reported that technology solutions, such as e-portfolios, were not often used by language teachers and surmised that this was due to an uncertainty that technology could help with language learning.

Adair-Hauck, Willingham-McLain, and Youngs (2000) cautioned that using ICTs was not a way for a teacher to save time in the classroom. They reported it to be labour intensive and difficult to manage as both students and teachers needed to adapt to their use in learning and evaluation situations. Birch and Volkov (2007) also posited that assessing participation in online discussions was time consuming in terms of creating the assessment tools as well as assessing student performance.

Odo (2012) discovered that learners were quick to adapt to the use of ICTs and Birch and Volkov (2007) reported that second language students were open to assessment through online discussions. Adair-Hauck et al. (2000) noted that instructors were satisfied with both performance and attitude of their students as a result of using ICTs.

Luk (2010) suggested using ICTs only for the collection of “less sensitive speech data…” (p. 49). East and King (2012) also reported that the slowing down of listening comprehension tests could be viewed as inauthentic and not appropriate for high stakes tests. They argued that it was appropriate for scaffolding learning and was useful for preparing learners for these tests.

Works cited

Adair-Hauck, B., Willingham-McLain, L., & Youngs, B. (2000). Evaluating the integration of technology and second language learning. CALICO Journal, 17(2), 269-306.

Aydin, S. The Effect of Computers on the Test and Inter-Rater Reliability of Writing Tests of ESL Learners. The Turkish Online Journal of Educational Technology, 5(1). Retrieved from http://www.tojet.net/articles/v5i1/519.pdf

Birch, D., & Volkov, M. (2007). Assessment of online reflections: Engaging English second language (ESL) students. Australasian Journal of Educational Technology, 23(3), 291.

Cummins, P., & Davesne, C. (2009). Using electronic portfolios for second language assessment. The Modern Language Journal, 93(s1), 848-867.

East, M., & King, C. (2012). L2 learners’ engagement with high stakes listening tests: Does technology have a beneficial role to play? CALICO Journal, 29(2), 208-223.

Hardison, D.M. (2005). Contextualized computer-based L2 Prosody training: Evaluating the effects of discourse context and video input. CALICO Journal 22(2). Retrieved from http://journals.sfu.ca/CALICO/index.php/calico/article/viewFile/692/557

Jones, L. (2004). Testing L2 vocabulary recognition and recall using pictorial and written test items. Language Learning & Technology, 8(3), 122-143.

Ketabi, S., & Ketabi, S. (2014). Classroom and Formative Assessment in Second/Foreign Language Teaching and Learning. Theory And Practice In Language Studies, 4(2), 435-440.

Laurier, M. (2004). Évaluation et multimédia dans l'apprentissage d'une L2. ReCALL : The Journal of EUROCALL, 16(2), 475-487.

Li, Z., Link, S., Ma, H., Yang, H., & Hegelheimer, V. (2014). The role of automated writing evaluation holistic scores in the ESL classroom. System, 44, 66-78.

Luk, J. (2010). Talking to score: Impression management in L2 oral assessment and the co-construction of a test discourse genre. Language Assessment Quarterly, 7(1), 25-53.

Odo, D. (2012). Computer familiarity and test performance on a computer-based cloze ESL reading assessment. Teaching English with Technology, 12(3), 18-35.

Vincent-Durroux, L., Poussard, C., Lavaur, J., & Aparicio, X. (2011). Using CALL in a formal learning context to develop oral language awareness in ESL: an assessment. Recall, 23(02), 86-97.

Wagner, E. (2007). Are they watching? Test-taker viewing behavior during an L2 video listening test. Language Learning & Technology, 11(1), 67-86.

Wagner, E. (2010). Test-takers' interaction with an L2 video listening test. System, 38(2), 280-291.