Defining and evaluating quality in e-learning: lessons drawn from case studies in the field of computer-assisted language learning (CALL)

Jonathan Leakey

Research output: Chapter in Book/Report/Conference proceedingChapterpeer-review


1] Book abtract:The aim of this publication is to discuss the broad question of quality when integrating technology into teaching and learning contexts. The book draws on the experiences of researchers and tutors working in different subject disciplines in order to focus on the commonalities identified when exploiting new technologies within a distinct pedagogical environment. This resource therefore hopes to offer students and teachers an insight into the various applications of technology in teaching and learning.This book can be dissected into a number of areas, including innovative research currently being undertaken at the fore of this technological revolution in order to support integration; the employment of technologies with a link to facilitating communities of membership; the use of specific Reusable Learning Objects designed for both secondary and tertiary education respectively; the use of e-portfolios for students, teachers and information workers; and the critical evaluation of technology.This resource therefore proposes to offer students and teachers an insight into the different applications of technology in teaching and learning. It is hoped that this can be drawn on by undergraduate and postgraduate students; instructional designers; educational managers; teachers; teacher trainers; academics; media technology students with the express intention of illuminating some of the quality issues surrounding the exploitation of technology for teaching and learning purposes.2] Chapter abstract:This chapter draws largely on the thesis completed in 2008, which focused on effectiveness research and the evaluation of computer-assisted language learning (CALL). In this chapter I explore some commonalities between CALL and other disciplines to emphasise the point that a definition of quality for CALL and an evaluative framework for CALL may have relevance for other disciplines. A brief introduction to CALL evaluation and a definition of quality for CALL is given, drawing on the contribution of a number of CALL researchers who have drawn up criteria for a common agenda for evaluating CALL. My thesis pulled these together into a sequenced model for evaluation that addresses issues of internal and external validity, criteria for judging task appropriateness, and principles for determining appropriate quality control criteria. The thesis has appeared in book form (Evaluating Computer-assisted Language Learning: An Integrated Approach to Effectiveness Research for CALL, © Peter Lang AG, 2011) in which the full evaluation model is set out with its theoretical foundations and all the contributing case studies. The starting point for my framework for quality control is Chapelle’s six criteria for evaluating CALL (2001). I looked both within and beyond CALL to widen the evaluative scope to make sure the model addressed not just pedagogy but also software and digital platforms, and their interrelationship, or synergy. To this end I cross-mapped Chapelle with Mehanna (2004) on e-learning pedagogies, with Pederson (1988) and Dunkel (1991) on effectiveness research for CALL, with Ingraham and Emery (1991) and Hubbard (1988) on evaluating software for CALL, and with Clarke (2005) and Barr, Carvalho Martins, Duffner, Gillespie, and Wright (2007) on evaluating digital platforms for CALL. This model was road-tested and developed during four different language teaching case studies carried out at the University of Ulster between 2003 and 2006, culminating in an improved version which proposes a framework for integrated evaluation of CALL platforms, programs and pedagogies and a set of checklists for good practice that are ready-to-use in any CALL context. Some of the checklists are probably usable in other, non-linguistic, contexts with no changes required; the remainder should, with some adaptation, be usable beyond CALL. This chapter concludes with a brief introduction to the improved model for evaluation and some of the quality control checklists.
Original languageEnglish
Title of host publicationQuality Issues in ICT Integration: Third Level Disciplines and Learning Contexts
EditorsTríona Hourigan, Liam Murray, Elaine Riordan
Place of PublicationNewcastle upon Tyne
PublisherCambridge Scholars Publishing
ISBN (Print)1-4438-2967-6
Publication statusPublished - Jul 2011

Bibliographical note

Reference text: Barr, D. 2004. ICT – Integrating computers in teaching, creating a computer-based language-learning environment. Bern: Peter Lang AG.

Barr, D., Leakey, J. and Ranchoux, A. 2005. TOLD like it is! An evaluation of an integrated oral development model. Language-learning & Technology 9, no. 3: 55-78, (accessed October 10, 2005).

Barr, D., S. Carvalho Martins, K. Duffner, J. Gillespie, G. Toner, and V. Wright. 2007. Multimedia language-learning in UK universities - a report by the Subject Centre for Languages, Linguistics and Area Studies carried out on behalf of the Centre for Excellence in Multimedia Language-learning, University of Ulster. (accessed December 21, 2007).

Buglear, J. (2000). Stats to go – a guide to statistics for hospitality, leisure and tourism. Oxford: Butterworth-Heinemann.

Chapelle, C. 2001. Computer applications in Second Language Acquisition. Cambridge, UK: Cambridge University Press.

Chapelle, C., and J. Jamieson. 1991. Internal and external validity issues in research on CALL effectiveness. In Computer-Assisted Language-learning and Testing: Research Issues and Practice, ed. P. Dunkel, ch.2. Newbury House: New York.

Clarke, M. 2005. Moving towards the digital classroom. Paper presented at the EUROCALL 2005 conference, August 24-27, in Krakow, Poland.

Coleman, J.A., and J. Klapper, (eds.). 2005. Effective learning and teaching in modern languages. London & New York: Routledge.

Davies, G., P. Bangs, R. Frisby, and E. Walton. 2005. Setting up effective digital language laboratories and multimedia ICT suites for MFL. CILT. (accessed August 26, 2005).

Driscoll, M. 2002. Blended learning: let’s go beyond the hype. E-learning, 3, no.3, 54.

Dunkel, P. (ed.). 1991. Computer-Assisted Language-learning and testing: research issues and practice. New York: Newbury House.

Ellis, R. 1994. The study of Second Language Acquisition. Oxford: Oxford University Press.

Felix, U. 2000. The potential of CD-ROM technology for integrating language and literature: student perceptions. German as a foreign language, 2: 48-63. Melbourne: Monash University. (accessed December 16, 2003).

Felix, U. 2005a. What do meta-analyses tell us about CALL effectiveness? ReCALL, 17, no. 2: 269–288.

Felix, U. 2005b. Analysing recent CALL effectiveness research – towards a common agenda. Computer Assisted Language-learning 18, no. 1&2: 1-32.

Gill, J., P. Johnson. 1997. Research Methods for Managers, 2nd ed. London: Paul Chapman Publishing.

Hewett, T., R. Baecker, S. Card, T. Carey, J. Gasen, M. Mantei, G. Perlman, G. Strong, and W. Verplank. 1996. ACM SIGCHI curricula for Human-Computer Interaction. (accessed December 21, 2007).

Hubbard, P. 1988. Language teaching approaches, the evaluation of CALL software, and design implications. In Modern media in foreign language education: theory and implementation. Ed. W. F. Smith. Lincolnwood, Illinois: National Textbook Company, 1988: 227-252.

Ingraham, B.D., and C.R. Emery. 1991. “France Interactive”: a hypermedia approach to language training. Educational and Training Technology International, 25, no. 4: 321-33.

Laurillard, D. and S. Hewer. 1998. TELL Consortium Project Evaluation. University of Hull, UK. (accessed December 18, 2008).

Leakey, J. 2010 (forthcoming). Evaluating Computer-assisted Language Learning: An Integrated Approach to Effectiveness Research for CALL, Bern: Peter Lang AG.

Leakey, J. 2009. A qualitative and quantitative evaluation of the effects of CALL platforms, programs and pedagogy on second language acquisition. PhD thesis, University of Ulster.

Leakey, J., and A. Ranchoux. 2006. BLINGUA – A blended language-learning approach for CALL. Computer Assisted Language-learning, 19, no. 4 & 5: 357-372, (accessed December 29, 2006).

Levy, M. 1997. Computer-Assisted Language-learning, context and conceptualization. Oxford: Clarendon Press.

McCarty, W. 1995. “Cannot without the procéss of speech be told.” Keynote speech given at the EUROCALL 1995 conference, September 7-9, Valencia, (Proceedings, pp.19-34).

Mehanna, W.N. 2004. E-Pedagogy: the pedagogies of e-learning. ALT-J, Research in Learning Technology, 12, no. 3: 279-293.

Oliver, M., and K. Trigwell. 2005. Can “blended learning” be redeemed? E-Learning, 2, no. 1: 17-26.

Pallant, J. (2001). SPSS Survival Manual – A step by step guide to data analysis using SPSS for Windows (Versions 10 and 11). Maidenhead, UK: Open University Press.

Pederson, K.M. 1988. Research on CALL. In Modern media in foreign language education: theory and implementation. Ed. W. F. Smith. Lincolnwood, Illinois: National Textbook Company, 1988: 99-131.

Remenyi, D., B. Williams, A. Money, and E. Swartz. 1998. Doing research in business and management. London: Sage Publications.

Rowntree, D. 1981. Statistics without Tears – a Primer for Non-Mathematicians. Penguin Books Ltd: Harmondsworth, England

Saunders, M., A. Thornhill, P. Lewis. 2006. Research methods for business students, 4th ed. New Jersey: Prentice Hall (Pearson Education).

Smith, W.F. (ed.). 1988. Modern media in foreign language education: applications and projects. Lincolnwood, Illinois: National Textbook Company.


  • quality control
  • computer-assisted language learning
  • CALL
  • evaluation of pedagogy
  • digital platformas
  • language learning software


Dive into the research topics of 'Defining and evaluating quality in e-learning: lessons drawn from case studies in the field of computer-assisted language learning (CALL)'. Together they form a unique fingerprint.

Cite this