Rationale, aims and objectives
We examined the effect of two interventions on both the reliability and validity of regulatory judgments: adjusting the regulatory instrument and attending a consensus ...meeting.
Method
We adjusted the regulatory instrument. With a randomized controlled trial (RCT) we examined the effect of the adjustments we made to the instrument. In the consensus meeting inspectors discussed cases and had to reach consensus about the order of the cases. We used a before and after case study to assess the effect of the consensus meeting. We compared the judgments assigned in the RCT with the unadjusted instrument with the judgments assigned with the unadjusted instrument after the consensus meeting. Moreover we explored the effect of increasing the number of inspectors per regulatory visit based on the estimates of the two interventions.
Results
The consensus meeting improved the agreement between inspectors; the variance between inspectors was smallest (0.03) and the reliability coefficient was highest (0.59). Validity is assessed by examining the relation between the assigned judgments and the corporate standard and expressed by a correlation coefficient. This coefficient was highest after the consensus meeting (0.48). Adjustment of the instrument did not increase reliability and validity coefficients.
Conclusions
Participating in a consensus meeting improved reliability and validity. Increasing the number of inspectors resulted in both higher reliability and validity values. Organizing consensus meetings and increasing the number of inspectors per regulatory visit seem to be valuable interventions for improving regulatory judgments.
Psycholinguistic data are often analyzed with repeated-measures analyses of variance (ANOVA), but this paper argues that mixed-effects (multilevel) models provide a better alternative method. First, ...models are discussed in which the two random factors of participants and items are crossed, and not nested. Traditional ANOVAs are compared against these crossed mixed-effects models, for simulated and real data. Results indicate that the mixed-effects method has a lower risk of capitalization on chance (Type I error). Second, mixed-effects models of logistic regression (generalized linear mixed models, GLMM) are discussed and demonstrated with simulated binomial data. Mixed-effects models effectively solve the “language-as-fixed-effect-fallacy”, and have several other advantages. In conclusion, mixed-effects models provide a superior method for analyzing psycholinguistic data.
In this study, the authors tested the effects of Tekster Texter, a comprehensive strategy-focused writing instruction program, using a switching replication design with three measurement occassions. ...The program was implemented by fourth, fifth, and sixth grade teachers (N = 76) in 60 general education classrooms in the Netherlands. Students (n = 688) and teachers (n = 31) in Group 1 worked with Tekster during the first 8-week period, between the first and second measurement occasion. Students (n = 732) and teachers (n = 45) in Group 2 implemented Tekster during the second 8-week period, between the second and third measurement occasion. The intervention led to statistically significant improvements in the quality of students' writing. The effect size for the full sample was 0.32 and 0.40 for students who students who completed all 16 Tekster lessons. Gains shown by students in Group 1 were maintained after 8 weeks. Because writing quality was assessed in 3 genres, the findings are generalizable across students, classes, and writing tasks. Taken together, the results of this study demonstrate that a strategy-focused writing instruction program, such as Tekster, can be an effective way to improve upper-elementary students' written language skills.
Educational Impact and Implications Statement
This study shows how Tekster Texter, a strategy-focused writing instruction program, improves the writing performance of students in Grade 4 to 6. This positive effect was still visible 2 months after the intervention. As the intervention was successfully implemented by teachers in a large number of classrooms, this study suggests that Tekster is a promising approach for improving students' writing in general education.
► We examine linguistic behavior of maximizers and approximators. ► These modifiers should combine with absolute adjectives, but not with relative ones. ► Also, an adjective modifier combination ...should show a stable rating across contexts. ► However, the same combination is judged differently in different contexts. ► Therefore, linguistic behavior does not allow for a classification of adjectives.
Respondents are more likely to disagree with negative survey questions (
This text is boring.
Yes/
No) than to agree with positive ones (
This text is interesting.
Yes/
No). The size of this effect, however, varies largely between word pairs. A semantic classification of adjectives in closed scale/absolute and open scale/relative types was predicted to explain this variation. To classify survey adjectives, a judgment experiment was conducted. Language users (
N
=
173) rated sentences in which an adjective was modified by the maximizer
completely or the approximator
almost: it should be possible to combine closed scale/absolute adjectives with these modifiers, in contrast to open scale/relative adjectives for which this is not the case.
Results show that language users agree on which adjective and degree modifier combinations are acceptable and which combinations are unacceptable. Moreover, the two methods,
almost and
fully, show convergent validity. However, the rating of the same combination of a specific adjective and a specific degree modifier varies across contexts. This suggests that neither of the two methods allows for an unambiguous classification of adjectives. Hence, the distinction between closed scale/absolute and open scale/relative adjectives cannot explain variation in survey response effects. For semantics and pragmatics results indicate that context plays a crucial role in the linguistic behavior of adjectives and degree modifiers.
The aim of this study was to provide insight into the current practice of writing instruction in Dutch primary education, as a stepping stone for designing and implementing sustainable innovations ...that could satisfy both practitioners and policymakers. We investigated the extent to which three domain-specific approaches-communicative writing, process writing, and writing strategy instruction-and general features of high-quality instruction were implemented in writing lessons in the upper grades of primary schools. We also examined the learning time for writing, teachers' views on writing and writing instruction, how efficacious they feel about teaching writing, and how skilled they are in the writing instruction domain. Lastly, we explored relations between classroom practices, learning time, and teachers' beliefs and skills through correlation analysis, to identify potential aids and constraints to guide innovations in writing education. Participants were 61 teachers of 45 primary schools in the Netherlands. Data were collected through questionnaires, stimulated recall interviews and over 100 lesson observations. Results indicated that the three domain-specific approaches for writing instruction were insufficiently implemented in Dutch classrooms, as were differentiating and the teaching of learning strategies. The allocated learning time was also insufficient, but the realized learning time and the extent to which teachers promoted active learning were satisfactory, providing a strong basis for curricular improvement. Several relations were found between teachers' classroom practices, learning time, and teachers' beliefs and skills in the domain of writing instruction. Finally, we discuss options for sustainable innovations of writing instruction in this national context.
Educational Impact and Implications Statement
The aim of this study was to provide insight into the current state of writing instruction in the upper primary level in the Netherlands, as a knowledge base for designing sustainable curricular innovations including professional development programs. The results indicated that teachers evaluate their writing lessons, are able to assess the communicative effectiveness of students' texts, promote active learning, and use the allocated learning time efficiently. However, not enough writing lessons are taught in Dutch primary schools. Also, little attention is paid to communicative aspects of writing, the writing process, teaching strategies, differentiating, and tracking students' writing development. Moreover, teachers' efficacy in teaching writing is moderate. The correlational network between teachers' beliefs and skills and classroom practices suggests that innovations must take these belief systems into account to be successful and sustainable. Overall, this study provides valuable clues for designing, adjusting, and implementing innovations in writing education in a particular region which could meet the concerns of both practitioners and policymakers.
This study examined writers’ use of their first language (L1) while writing in their second language (L2). Twenty students each wrote four short argumentative essays in their L1 (Dutch) and four in ...their L2 (English) under think-aloud conditions. We analysed whether L1 use varied between writers and tasks, and whether it was related to general writing proficiency, L2 proficiency, and L2 text quality. The analysis focused on the occurrence of a number of conceptual activities, including Generating ideas, Planning, and Metacomments. Results indicate that all participants used their L1 while writing in their L2 to some extent, although this varied among conceptual activities. In addition, L2 proficiency was directly related to L2 text quality but was not related to the occurrence of conceptual activities either in L1 or L2. General writing proficiency, on the other hand, has a negative influence on L1 use during L2 writing and a positive effect on L2 use during L2 writing. L1 use during L2 writing is negatively related to L2 text quality, at least for Metacomments. Finally, L2 use appears to be positively related to L2 text quality for Goal setting, Generating ideas, and Structuring, but negatively related to L2 text quality for Self-instructions and Metacomments. The theoretical relevance of these findings is also discussed.
This study attempts to reliably measure literature reading and creative writing ability, and subsequently to determine whether a relationship exists between the two abilities. Participants were 19 ...eleventh‐grade students: 11 were known to be good readers of literature, whereas 8 were known to be poor readers of literature. Each participant read 4 literary texts and wrote 5 creative texts. Texts concerned two genres: poems and short stories. The transcriptions of reading responses and writing products were rated by different panels of 7 and 8 independent expert judges, respectively.
Multilevel analyses indicated that agreement among judges was high and that individual student performance relative to other students was fairly consistent among tasks, both for reading and writing tasks. Moreover, average reading and average writing performance appeared to be related even when the a priori selection was taken into account.
The results support our hypothesis that a positive relationship exists between literature reading and creative writing ability. Moreover, it shows that both constructs can be measured in reliable ways. Implications for further research and for the position of creative writing within the literature curriculum are discussed.
PDF
