As assessment becomes more important in education, teachers must be advanced in both assessment and teaching course materials. The most recent way of assessing the performance of students in ...listening classrooms is value-added assessment using Google Sites as e-portfolios, one of among the strongest instruments used to assess the efficacy of teachers as well as recognise growth in students. Nonetheless, the use of e-portfolios in EFL students' listening classrooms and how it influences their learning enjoyment remains undetermined due to neither researchers nor students have addressed these issues in depth, and studies on using Google Sites as an e-portfolio-based assessment for teaching listening comprehensions have also been underutilised. Thus, this research aims to gain insight towards EFL students' views on using Google Sites as an e-portfolio-based value-added assessment tool in listening classrooms. It made use of a qualitative case study and included second-year university students. Students' course reflections and semi-structured interviews were both utilised to collect data, which was thereafter analysed qualitatively using coding and thematic analysis. The results of the study revealed that creating e-portfolios with Google Sites proves helpful in assessing students' learning achievement, tracking and measuring students' progress, and diagnosing deficiencies that need to be addressed. This study renders a recommendation for the use of other well-developed assessment tools in future studies.
The main reason for this research is the boredom of online learning that many students have to deal with. Moreover, this condition has existed since more than a year ago. This situation alone makes ...the researcher, who also acts as a teacher, upgrade the online learning process with the help of social media, the Instagram Live feature. The researcher hopes that the online learning process can be more fun and enjoyable by using the Instagram Life feature, which will later improve the students' speaking ability. This research investigates using the Descriptive Statistical method, which uses class control and the class experiment. The data shown are Pre-Test and Post Test data which are then processed using Wilcoxon. This way, in the final of the research, it can be concluded that the use of Instagram live can improve Speaking skills because there is a significant score difference between before and after using Instagram live as a learning process.
This paper addresses the intersection of testing and policy, situating test-driven impact and validation within the context of policy-led educational reform in Korea. I will briefly review the ...existing validation models. Then, arguing for an expansion of the conventional conceptualization of consequential validity research, I use Fairclough’s dialectic–relational approach in critical discourse analysis (CDA), positioned in critical and poststructuralist research tradition, to evaluate social realities, such as intended and actual impact of policy-led testing, I take, as an example, the context of the development of the National English Ability Test (NEAT) in Korea, which had been used as a means of implementing government policies. Combining Messick’s validity framework for consequential evidence, Bachman and Palmer’s argument-based approach to validation (assessment use argument, AUA), and Fairclough’s dialectic–relational approach, I will illustrate how the impact of policy-led testing is performed and interpreted as a sociopolitical and discursive phenomenon, constituted and enacted in and through “discourse.” By revisiting the previous Faircloughian research works on NEAT’s impact, I postulate that the discourses arguing for and against social impact acquire their meanings from dialectical standpoints.
For practical and theoretical purposes, tests of second language (L2) ability commonly aim to measure one overarching trait, general language ability, while simultaneously measuring multiple ...sub-traits (e.g., reading, grammar, etc.). This tension between measuring uni- and multi-dimensional constructs concurrently can generate vociferous debate about the precise nature of the construct(s) being measured. In L2 testing, this tension is often addressed through the use of a higher-order factor model wherein multidimensional traits representing subskills load on a general ability latent trait. However, an alternative modeling framework that is currently uncommon in language testing, but gaining traction in other disciplines, is the bifactor model. The bifactor model hypothesizes a general factor, onto which all items load, and a series of orthogonal (uncorrelated) skill-specific grouping factors. The model is particularly valuable for evaluating the empirical plausibility of subscales and the practical impact of dimensionality assumptions on test scores. This paper compares a range of CFA model structures with the bifactor model in terms of theoretical implications and practical considerations, framed for the language testing audience. The models are illustrated using primary data from the British Council’s Aptis English test. The paper is intended to spearhead the uptake of the bifactor model within the cadre of measurement models used in L2 language testing.
Validity and validation are common in large-scale language testing. These topics are fundamental because they help stakeholders in testing systems make accurate interpretations of individuals’ ...language ability and related ensuing decisions. However, there is limited information on validity and validation for classroom language testing, for which interpretations and decisions based on curriculum objectives are paramount, too. In this reflection article, I provide a critical account of these two issues as they are applied in large-scale testing. Next, I use this background to discuss and provide possible applications for classroom language education through a proposed approach for validating classroom language tests. The approach comprises the analyses of curriculum objectives, design of test specifications, analysis of test items, professional design of instruments, statistical calculations, cognitive validation and consequential analyses. I close the article with implications and recommendations for such endeavours and highlight why they are fundamental for high-quality language testing systems in classroom contexts.
Globalized English proficiency tests such as the International English Language Testing System (IELTS) are increasingly playing the role of gatekeepers in a globalizing world. Although the use of the ...IELTS as a 'policy tool' for making decisions in the areas of study, work and migration impacts on test-takers' lives and life chances, not much is known about its own policy and policy logic. With the growing call for listening to test-takers' voices, specific policy aspects of the test have recently been scrutinized. This article seeks to contribute to this area by drawing on test-takers' perspectives on the IELTS retake policy and developing an understanding of how policies of global English tests are perceived, interpreted and given new meanings by its key stakeholders. Although the truth-value of the emic views of test-takers may be debated, their experiences and perceptions problematize the reliability claims of the testing agency by locating commercial motives at the center of their policy and thereby raising theoretical, professional and ethical questions.
Vocabulary profiling with computational tools and word lists is an established step in the development of pedagogical materials for learners of English. However, existing tools and word lists lack ...sensitivity to the orthographical, morphological, and grammatical systems of highly-inflected and declined languages. This limits the degree to which lexical profiling can be usefully implemented in the creation of materials intended for use with beginner/low-intermediate learners of such languages who have only partial knowledge of these systems.
In this article, we present MultilingProfiler, a vocabulary profiling tool designed to support nuanced profiling of texts in French, German, and Spanish. We introduce the concept of ‘bespoke’ word families tailored to the needs of learners at various stages of development, and outline key features of the tool that operationalise this concept (the functionality to select which inflected, derived, and multiword forms of headwords are included in the profile; sensitivity to orthographical systems; embedded word lists aligning with specific programs of study; and cumulative word lists that grow with learner knowledge). We present two case studies that find MultilingProfiler's features to be effective in highlighting potential mismatches between the lexical demands of texts and the expected knowledge of learners, and consider applications of the tool in research methods.
Over the last two decades, the Common European Framework of Reference for languages (CEFR) has become the most influential tool of language policy-making in Europe and beyond. The publication of the ...companion volume (CEFR-CV) constitutes a new milestone for teaching, learning and assessing languages, and is a most timely reaction to common criticism of the framework. In addition to new scales, descriptors and competence levels, the CEFR-CV introduces new modalities and broadens the scope for mediation and plurilingual/cultural communication, thereby updating and extending previous construct definitions for increasingly digitized and diverse societies. Despite the CEFR’s major impact on the language testing industry, there is thus far scarce literature on how to operationalize the CEFR-CV for assessment with the expanded framework. In addition to the huge potential for innovative assessment tasks and formats, this raises questions with regard to construct definitions, task development, test quality assurance, and rating practices. This paper will focus on six noteworthy innovations of the CEFR-CV and discuss the opportunities and challenges for assessment: (1) departure from the native-speaker norm, (2) stronger consideration of digital communication, (3) interlingual mediation, (4) intralingual mediation, (5) phonological awareness, and (6) the provision of richer descriptions of lower-level learner competencies.
While several test concordance tables have been published, the research underpinning such tables has rarely been examined in detail. This study aimed to survey the publically available studies or ...documentation underpinning the test concordance tables of the providers of four major international language tests, all accepted by the Australian Department of Home Affairs for Australian visa purposes. To evaluate the concordance studies, we first identified the good practice principles in concordance research through a review of both the relevant literature and leading professional standards in the field of educational measurement and language assessment. Next, we reviewed the concordance studies against the identified good practice principles. Our findings revealed that the information supplied by test providers varied, with some making the full research papers available, whereas others providing little information about their underpinning research. None of the concordance studies fulfilled all the good practice principles. Based on the findings of this study, we offer recommendations for future concordance research in the field of language testing as well as suggestions for practice.
PDF
