This paper addresses the intersection of testing and policy, situating test-driven impact and validation within the context of policy-led educational reform in Korea. I will briefly review the ...existing validation models. Then, arguing for an expansion of the conventional conceptualization of consequential validity research, I use Fairclough’s dialectic–relational approach in critical discourse analysis (CDA), positioned in critical and poststructuralist research tradition, to evaluate social realities, such as intended and actual impact of policy-led testing, I take, as an example, the context of the development of the National English Ability Test (NEAT) in Korea, which had been used as a means of implementing government policies. Combining Messick’s validity framework for consequential evidence, Bachman and Palmer’s argument-based approach to validation (assessment use argument, AUA), and Fairclough’s dialectic–relational approach, I will illustrate how the impact of policy-led testing is performed and interpreted as a sociopolitical and discursive phenomenon, constituted and enacted in and through “discourse.” By revisiting the previous Faircloughian research works on NEAT’s impact, I postulate that the discourses arguing for and against social impact acquire their meanings from dialectical standpoints.
For practical and theoretical purposes, tests of second language (L2) ability commonly aim to measure one overarching trait, general language ability, while simultaneously measuring multiple ...sub-traits (e.g., reading, grammar, etc.). This tension between measuring uni- and multi-dimensional constructs concurrently can generate vociferous debate about the precise nature of the construct(s) being measured. In L2 testing, this tension is often addressed through the use of a higher-order factor model wherein multidimensional traits representing subskills load on a general ability latent trait. However, an alternative modeling framework that is currently uncommon in language testing, but gaining traction in other disciplines, is the bifactor model. The bifactor model hypothesizes a general factor, onto which all items load, and a series of orthogonal (uncorrelated) skill-specific grouping factors. The model is particularly valuable for evaluating the empirical plausibility of subscales and the practical impact of dimensionality assumptions on test scores. This paper compares a range of CFA model structures with the bifactor model in terms of theoretical implications and practical considerations, framed for the language testing audience. The models are illustrated using primary data from the British Council’s Aptis English test. The paper is intended to spearhead the uptake of the bifactor model within the cadre of measurement models used in L2 language testing.
Globalized English proficiency tests such as the International English Language Testing System (IELTS) are increasingly playing the role of gatekeepers in a globalizing world. Although the use of the ...IELTS as a 'policy tool' for making decisions in the areas of study, work and migration impacts on test-takers' lives and life chances, not much is known about its own policy and policy logic. With the growing call for listening to test-takers' voices, specific policy aspects of the test have recently been scrutinized. This article seeks to contribute to this area by drawing on test-takers' perspectives on the IELTS retake policy and developing an understanding of how policies of global English tests are perceived, interpreted and given new meanings by its key stakeholders. Although the truth-value of the emic views of test-takers may be debated, their experiences and perceptions problematize the reliability claims of the testing agency by locating commercial motives at the center of their policy and thereby raising theoretical, professional and ethical questions.
Vocabulary profiling with computational tools and word lists is an established step in the development of pedagogical materials for learners of English. However, existing tools and word lists lack ...sensitivity to the orthographical, morphological, and grammatical systems of highly-inflected and declined languages. This limits the degree to which lexical profiling can be usefully implemented in the creation of materials intended for use with beginner/low-intermediate learners of such languages who have only partial knowledge of these systems.
In this article, we present MultilingProfiler, a vocabulary profiling tool designed to support nuanced profiling of texts in French, German, and Spanish. We introduce the concept of ‘bespoke’ word families tailored to the needs of learners at various stages of development, and outline key features of the tool that operationalise this concept (the functionality to select which inflected, derived, and multiword forms of headwords are included in the profile; sensitivity to orthographical systems; embedded word lists aligning with specific programs of study; and cumulative word lists that grow with learner knowledge). We present two case studies that find MultilingProfiler's features to be effective in highlighting potential mismatches between the lexical demands of texts and the expected knowledge of learners, and consider applications of the tool in research methods.
While several test concordance tables have been published, the research underpinning such tables has rarely been examined in detail. This study aimed to survey the publically available studies or ...documentation underpinning the test concordance tables of the providers of four major international language tests, all accepted by the Australian Department of Home Affairs for Australian visa purposes. To evaluate the concordance studies, we first identified the good practice principles in concordance research through a review of both the relevant literature and leading professional standards in the field of educational measurement and language assessment. Next, we reviewed the concordance studies against the identified good practice principles. Our findings revealed that the information supplied by test providers varied, with some making the full research papers available, whereas others providing little information about their underpinning research. None of the concordance studies fulfilled all the good practice principles. Based on the findings of this study, we offer recommendations for future concordance research in the field of language testing as well as suggestions for practice.
Speech therapy uses a combination of approaches to assess speech and language, including analysis of spontaneous speech, use of standardized scales and assessment by the child’s parents and/or ...preschool teachers. In this paper, we present the Teddy Talk Test. Using a sample of sixteen children aged 4 to 5 years, we examined whether the test provided representative data on child language development. The parent questionnaire included information on the parents’ education, the parents’ native language, the child’s gender and the language the parents speak with their child. It was found that the Teddy Talk Test has the characteristics of screening tests for speech and language: it is time-efficient, it covers receptive and expressive language tasks, and it allows for rapid assessment of speech and language in large numbers of children. However, since the Teddy Talk Test has not yet been evaluated, we do not know its value regarding sensitivity, specificity and predictive validity.
The article discusses the methods available to the examiner in the verification of linguistic competence, considered among the fundamental elements of communicative competence in the teaching and ...learning of foreign languages. It is one of the skills that has an impact on professional, educational life, and also on social inclusion. Its assessment plays an important role in guiding the learner to master and use their knowledge correctly and effectively in different contexts to achieve well-defined objectives. The evaluation of language skills requires good preparation of tests according to the most appropriate method, and above all, forming a team of experts for the supervision of language tests, which must be ordered and well-constructed remains essential, from a basic level to a more advanced level where the learner will be able to interact easily without making any effort and understand what he hears and reads.
PDF
