Reviewed by Lyndon Taylor (Sheffield Institute of Education, Sheffield Hallam University)
27 June 2017
How does Dr. Anthony Manning’s new book differ from other contributions to the field of language testing? The book does not deal with the minutiae of test item writing and analysis or to theories relating to the assessment of a particular language skill. Rather, it takes a broad view of issues worthy of serious attention by those involved in EAP test design and implementation. As the title suggests, the aim of the book is to promote assessment literacy in EAP, a cause which has featured prominently in Manning’s work from his Doctorate in Education (2013) through to his current involvement in the delivery of the item-writing ‘roadshows’ organised by the BALEAP Testing Working Group.
The first of the book’s 15 chapters guides readers in considering the purpose of their EAP tests. Other early sections deal with issues to think through when designing a test, including the target domain features to be incorporated and the questions of authenticity and specificity. Always, Manning is careful not to be dogmatic in his treatment of these themes. For instance, the literature relating to the debate around English for General Academic Purposes or English for Specific Academic Purposes is reviewed (chapter 4), but the reader needs to decide her own position with regard to this debate and bear this in mind when choosing aspects of the target language domain to sample in her own test. A number of scenarios and strategies provide tools to help the test designer in making decisions like these.
Subsequent chapters include an overview of descriptive and inferential statistical methods which may be used in test evaluation. Statistics may be a daunting area for some EAP practitioners, though their treatment here constitutes a remarkably clear introduction. Later sections consider the wide-reaching impact of tests and test results, first and foremost on the test-taker, but also on receiving departments and indeed on society as a whole. High stakes tests in particular entail a considerable degree of responsibility for those involved in test design as well as those making inferences and decisions based on test scores. Manning advocates high standards of rigour and transparency throughout the book.
The volume reads very much like a coursebook. Panels at the start of each chapter present a bulleted overview of the content areas and subsequent extension activities. Each main section provides a concise overview of research, often taking models and frameworks from wider fields such educational psychology and applied linguistics and adapting them in a clear and practical way for EAP testing purposes. The extension activities present a number of questions, dilemmas or scenarios which encourage readers to reflect on and develop their own principles or test products based on the themes covered in the chapter. ‘Assessment voices’, in which a range of EAP assessment practitioners’ thoughts and opinions are presented in speech bubbles, may be found in several sections of the book and form the bulk of the concluding chapter. This emphasis on the practical and local, I feel, is the main strength of the book – the way the reader is presented with a range of possible values and approaches and encouraged to reflect on the situation in his own setting. The extension activities, whilst being an aid to individual reflection, also lend themselves extremely well to peer discussions and I can see the book being used as the basis of CPD sessions.
Each chapter is largely self-contained, allowing readers to pick out areas of interest. However, this reviewer felt rewarded reading the book from cover to cover, as some themes introduced in earlier chapters are taken up again in later sections, adding to a sense of cohesion. One of the central themes is that of construct validity, which Manning describes as “perhaps the most influential overarching concept in modern day good practice for language testing” (p.165). Undoubtedly, a good test is one which allows us to make accurate and meaningful inferences about the skills or abilities the test taker has (or has not) demonstrated by obtaining a particular test score. Construct Validity is positioned early as a chapter title (chapter 2) though its importance is reinforced later when we read that test constructs are central to decisions about which aspects of EAP a test should sample (chapter 4). Moreover, in the section on washback (chapter 11), we are reminded that test constructs should be reflected in the content of the EAP course. The tension between construct validity and practical issues governing large scale assessments is explored in chapter 14, particularly where a test is trying to reflect features common to a variety of target language domains.
The importance of collaboration and effective communication with a variety of colleagues and stakeholders is a further strand running through the book. Chapter 6 underlines the value of working with colleagues in the test takers’ target disciplines, although it is acknowledged that there are often barriers to interdisciplinary dialogue. In Chapter 3, on test specifications, Manning advocates a team-based approach in which a blueprint document for test design is created and iteratively evaluated and revised.
There is a danger that the work involved in assessing EAP may be poorly understood by others. Each chapter closes with a page designed as a ‘model of good practice’ in relation to the chapter theme. This could be presented to a variety of stakeholders, including test-takers and admissions tutors, in the hope of promoting understanding and transparency as well as providing assurances about the rigour underlying assessment practices. This is an excellent idea which may help in elevating the status of EAP departments in their institutions.
In this new contribution to EAP bookshelves, Manning presents a framework of skills and strategies for EAP assessment practitioners in a clear, highly-readable style. It is accessible, I am sure, even for newcomers to the field of language testing, and I have no doubt that it will become a valuable tool in the promotion of assessment literacy in EAP. In short, it is essential reading for all EAP assessment practitioners.
References
Manning, A (2013) EAP teacher assessment literacy (Doctoral dissertation, University of Leicester).
