Accurate placement within levels of an ESL program is crucial for optimal teaching and learning. Commercially available tests are commonly used for placement, but their effectiveness has been found ...to vary. This study uses data from the Ohio Program of Intensive English (OPIE) at Ohio University to examine the value of two commercially available tests (the TOEFL ITP and the Michigan EPT) and a locally developed writing test for accurate placement decisions. Placement accuracy was measured in terms of the relationship between test scores and (1) appropriate placement levels for individual students according to their teachers, and (2) student performance in the classes. Findings support the continued use of multiple measures for more accurate placement decisions in the study context. However, the relationship between test scores and student performance, measured by students’ grades in the actual course levels and their success in advancing to a higher course level as additional indicators of the extent to which placement tests provide an accurate indication, was weak when analyzed through multiple regression and cross-tabulation, suggesting that factors other than initial proficiency are primarily determinative of student success when students have been accurately placed.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
In this study, we examined how much change in TOEFL® Primary™ listening and reading scores can be expected in relation to the time interval between test administrations. The test records of 5213 ...young learners of English (aged 8–13 years) in Japan and Turkey who repeated the tests were analyzed to examine test scores as a function of time interval. The effect of time on test scores was analyzed with a multilevel modeling approach, allowing both initial scores and rate of change among individual test takers to vary. In addition, we examined the effects of test-taker age and test-level difficulty on test scores. Separate analyses were conducted by country for ease of interpretation, as Japan and Turkey differ with respect to the number of hours of instruction that students receive and the English-learning goals in their respective curricula. Results showed a positive rate of change, indicating that test scores increase gradually over time. However, the rate of change differed between the two countries. Furthermore, repeaters’ test scores increased with their age and with the length of time between test administrations. Findings provide empirical evidence for schools to refer to when determining the timing of re-administration of the TOEFL Primary tests to their students.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
Data from 787 international undergraduate students at an urban university in the United States were used to demonstrate the importance of separating a sample into meaningful subgroups in order to ...demonstrate the ability of an English language assessment to predict the first-year grade point average (GPA). For example, when all students were pooled in a single analysis, the correlation of scores from the Test of English as a Foreign Language (TOEFL) with GPA was .18; in a subsample of engineering students from China, the correlation with GPA was .58, or .77 when corrected for range restriction. Similarly, the corrected correlation of the TOEFL Reading score with GPA for Chinese business students changed dramatically (from .01 to .36) when students with an extreme discrepancy between their receptive (reading/listening) and productive (speaking/writing) scores were trimmed from the sample.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
A common use of language tests is to support decisions about examinees such as placement into appropriate classes. Research on placement testing has focused on English for Academic Purposes (EAP) in ...higher education contexts. However, there is little research exploring the use of language tests to place students in English as a Second Language (ESL) support classes in secondary education. The present study examined the relationship between secondary school students’ language test scores from a standardized English-language test and the placement of these students into ESL classes by their language teachers. Ninety-two ESL students in two English-medium schools took TOEFL® Junior™ Standard. For the same students, data collection also included teachers’ judgments regarding the ESL classes the students should attend. Strong correlations between test scores and the teacher-assigned ESL levels were found. Moreover, the results from the logistic regression analysis indicated a great degree of overlap between the teacher-assigned ESL levels and the levels predicted from the TOEFL Junior Standard scores. The findings of this study provide some preliminary evidence to support the use of TOEFL Junior Standard as an initial screening tool for ESL placement. The limitations and implications of these findings for ESL placement decisions in secondary education are also discussed.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
This study examined the influence of prompt characteristics on the averages of all scores given to test taker responses on the TOEFL iBTTM integrated Read-Listen-Write (RLW) writing tasks for ...multiple administrations from 2005 to 2009. In the context of TOEFL iBT RLW tasks, the prompt consists of a reading passage and a lecture.
To understand characteristics of individual prompts, 107 previously administered RLW prompts were evaluated by participants on nine measures of perceived task difficulty via a questionnaire. Because some of the RLW prompts were administered more than once, multilevel modeling analyses were conducted to examine the relationship between ratings of the prompt characteristics and the average RLW scores, while taking into account dependency among the observed average RLW scores and controlling for differences in the English ability of the test takers across administrations.
Results showed that some of the variation in the average RLW scores was attributable to differences in the English ability of the test takers that also varied across administrations. Two variables related to perceived task difficulty, distinctness of ideas within the prompt and difficulty of ideas in the passage, were also identified as potential sources of variation in the average RLW scores.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
We explored the use of technology-assisted, trialogue-based tasks to measure the English language proficiency of students learning English as a second or foreign language. A presumed benefit of the ...system for language assessment is its suitability for use in scenario-based tasks that integrate multiple language skills. This integration allows test developers to simulate real-life language use situations by administering more authentic test tasks to strengthen the link between test performance and its interpretation as an indicator of expected success in performing real-life language use tasks. We conducted a usability study with 20 English language learners (ELLs) in grades 3 to 5, representing ten different first languages. Results suggested that the trialogue-based tasks engaged students in an authentic experience and provided evidence of their English proficiency. Perception data indicated that students perceived the trialogue-based tasks positively and understood how to respond to the virtual characters. Implications are discussed for future research and the potential of using trialogue-based assessment tasks for measuring the proficiency of young ELLs.
Full text
Available for:
BFBNIB, DOBA, IZUM, KILJ, NMLJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
•Two summary writing tasks were contrasted with respect to audience.•Audience specification in a prompt influences the quality of summary writing.•The role of contextual constraints should be ...considered in designing writing tasks.
Contextual factors influence the way we speak and write. An audience is central to the rhetorical context and helps to identify the parameters of a writing task. The purpose of this study was to examine whether and how the specification of an audience influenced the summary writing produced by adult English as a Second Language (ESL) writers in a large-scale writing assessment setting. The performance of 205 test takers who completed two writing tasks was compared with respect to audience awareness measured by three aspects of writing: context statements, content, and source attribution. Both tasks required test takers to summarize source information, but they were contrasted with respect to the specification of audience in our analysis. One task was presented without reference to an audience, whereas the other task explicitly instructed test takers to address a reader who was unfamiliar with the sources they would summarize. Of the three aspects of writing, the effect of audience specifications was observed on source attribution and context statements. This was not the case for content when responses between the two tasks were compared. However, all aspects were shown to vary across writing score levels when compared within the writing condition with audience specification.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
In this article, we consolidate and present in one place what is known about quality indicators for setting standards so that stakeholders may be able to recognize the signs of standard-setting ...quality. We use the context of setting standards to associate English language test scores with language proficiency descriptions such as those presented in the Common European Framework of Reference (Council of Europe, 2001). The criteria we discuss, however, apply to any language test and language framework. Standard-setting practices are often used to associate test scores with descriptions of language proficiency, as that is one way to provide meaning to the scores. However, there are a large number of standard-setting methods and procedures, and although there are suggested guidelines on how to implement such studies, not all studies necessarily follow the guidelines or follow them to the same extent. This makes it difficult to evaluate the quality of standard-setting results. Our intention is to offer guidance to policymakers, decision makers, and researchers about important factors they may consider to evaluate the rigor and credibility of standard-setting studies conducted to associate language test scores with language frameworks.
Full text
Available for:
BFBNIB, NUK, PILJ, SAZU, UL, UM, UPUK
This study examined the relationship between scores on the TOEFL Internet-Based Test (TOEFL iBT®) and academic performance in higher education, defined here in terms of grade point average (GPA). The ...academic records for 2594 undergraduate and graduate students were collected from 10 universities in the United States. The data consisted of students’ GPA, detailed course information, and admissions-related test scores including TOEFL iBT, GRE, GMAT, and SAT scores. Correlation-based analyses were conducted for subgroups by academic status and disciplines. Expectancy graphs were also used to complement the correlation-based analyses by presenting the predictive validity in terms of individuals in one of the TOEFL iBT score subgroups belonging to one of the GPA subgroups. The predictive validity expressed in terms of correlation did not appear to be strong. Nevertheless, the general pattern shown in the expectancy graphs indicated that students with higher TOEFL iBT scores tended to earn higher GPAs and that the TOEFL iBT provided information about the future academic performance of non-native English speaking students beyond that provided by other admissions tests. These observations led us to conclude that even a small correlation might indicate a meaningful relationship between TOEFL iBT scores and GPA. Limitations and implications are discussed.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
