Open Science in Education Sciences van Dijk, Wilhelmina; Schatschneider, Christopher; Hart, Sara A.
Journal of learning disabilities,
03/2021, Letnik:
54, Številka:
2
Journal Article
Recenzirano
Odprti dostop
The Open Science movement has gained considerable traction in the last decade. The Open Science movement tries to increase trust in research results and open the access to all elements of a research ...project to the public. Central to these goals, Open Science has promoted five critical tenets: Open Data, Open Analysis, Open Materials, Preregistration, and Open Access. All Open Science elements can be thought of as extensions to the traditional way of achieving openness in science, which has been scientific publication of research outcomes in journals or books. Open Science in education sciences, however, has the potential to be much more than a safeguard against questionable research. Open Science in education science provides opportunities to (a) increase the transparency and therefore replicability of research and (b) develop and answer research questions about individuals with learning disabilities and learning difficulties that were previously impossible to answer due to complexities in data analysis methods. We will provide overviews of the main tenets of Open Science (i.e., Open Data, Open Analysis, Open Materials, Preregistration, and Open Access), show how they are in line with grant funding agencies’ expectations for rigorous research processes, and present resources on best practices for each of the tenets.
The purpose of this pre-registered study (Peltier & McKenna) was to conceptually replicate if the truncation of the ordinate and DPPXYR increased analysts’ estimation of a functional relation and ...magnitude of treatment effect. Visual analysts (n = 27) evaluated eight data sets reporting null (n = 2), small (n = 2), moderate (n = 2), and large (n = 2) effects. Each data set was graphed six times with manipulations of the ordinate and DPPXYR, resulting in 48 ABAB graphs. We estimated two separate three-level mixed effect models with variations nested in datasets and nested in participants to evaluate the impact of graph characteristics for (1) confidence in determining a functional relation and (2) the estimated magnitude of the treatment effect. We included ordinate scaling and DPPXYR at level 1 and graph effect size at level 2, including all interactions. Overall, graph manipulation consistently did not impact confidence in a functional relation. Results suggest mixed findings for graph manipulation on the estimated magnitude of the treatment effect. Findings will be couched in current literature and recommendations for graph construction and future research will be discussed.
Complex research questions often need large samples to obtain accurate estimates of parameters and adequate power. Combining extant data sets into a large, pooled data set is one way this can be ...accomplished without expending resources. Measurement invariance (MI) modeling is an established approach to ensure participant scores are on the same scale. There are two major problems when combining independent data sets through MI. First, sample sizes will often be large leading to small differences becoming noninvariant. Second, not all data sets may include the same combination of measures. In this article, we present a method that can deal with both these problems and is user friendly. It is a combination of generating random normal deviates for variables missing completely in combination with assessing model fit using the root mean square error of approximation good enough principle, based on the hypothesis that the difference between groups is not zero but small. We demonstrate the method by examining MI across eight independent data sets and compare the MI decisions of the traditional and good enough approach. Our results show the approach has potential in combining educational data.
The impact of teachers on their students’ academic achievement continues to be an area of inquiry. One area not fully explored is the relation between teachers’ behavior and classroom management (CM) ...skills, student motivation, and student achievement. We examined these relations using a multi‐level structural equation model. Data included Behavior Management subscale scores of the Classroom Assessment Scoring System for 247 fourth and fifth grade teachers, indicators of academic motivation, and end‐of‐year state standardized mathematics assessment results for 4,847 students from the National Center of Teacher Effectiveness Main Study. The results indicate teachers’ CM skills have a significant direct effect on student motivation and a significant indirect effect on students’ math achievement, suggesting that students in classrooms of teachers with better CM are more motivated to learn math and have higher math achievement scores. This underscores the importance of helping teachers increase their use of evidence‐based classroom management techniques.
Open practices, such as preregistration, registered reports, open materials, open data, open analytic code, replication, open peer review, open access, and conflict-of-interest and funding ...statements, support the transparency, accessibility, and reproducibility of research and other scholarship. The purpose of this review was to examine the prevalence of these open practices in the special education literature. We reviewed a randomly selected sample of 250 articles published in special education journals in 2020. Results indicated that conflict-of-interest and funding statements were present in most articles; a small but meaningful proportion of articles provided open materials and were open access; and preregistration, registered reports, open data, open analytic code, open peer review, and replication were rarely or never observed. Recommendations for researching and supporting the use of open practices in special education scholarship are provided.
Open-science reforms, which aim to increase credibility and access of research, have the potential to benefit the research base in special education, as well as practice and policy informed by that ...research base. Awareness of open science is increasing among special education researchers. However, relatively few researchers in the field have experience using multiple open-science practices, and few practical guidelines or resources have been tailored to special education researchers to support their exploration and adoption of open science. In this article, we described and provided guidelines and resources for applying five core open-science practices—preregistration, registered reports, data sharing, materials sharing, and open-access publishing—in special education research.
Positive Behavioral Interventions and Supports (PBIS) is a multitiered system of support for behavior used in many schools across the world. Researchers have begun adapting and evaluating Tier 1 of ...PBIS for students placed in alternative education (AE) settings. The purpose of this review is to synthesize these evaluations. We systematically searched two electronic databases to find potential studies of Tier 1 PBIS in AE settings. We screened 47 full texts, 19 of which met all inclusion criteria. Most studies (16 of 19, 84%) evaluated student behavioral outcomes, while seven studies (37%) evaluated academic outcomes alone or in addition to behavioral measures. Overall, restraints and seclusions generally decreased following Tier 1 implementation, but behavioral incident changes were variable and academic outcomes only improved for students enrolled continuously throughout the studies. The findings of this review support the need for continued evaluations of Tier 1 in AE settings using more rigorous methodologies and metrics that account for rolling admissions. The observed reductions in restraints and seclusions are promising for students and practitioners given the poor outcomes associated with these consequences.
Misconceptions about the brain and its relation to education are widespread. This can lead to the implementation of ineffective methods and the waste of precious resources. To examine the extent of ...belief in neuromyths, a survey about the brain in education was conducted. Respondents (n = 169) came from special education (n = 83) and general education (n = 78), and included preservice teachers (n = 34), in-service teachers (n = 63), higher education faculty (n = 39), and educational leaders (n = 33). The survey contained 15 Myths and 18 Facts, and overall, participants were able to correctly identify approximately 66% of all the Facts. On the other hand, on average, respondents responded correctly for only one third of the Myths. The most commonly misidentified Myths were related to motor coordination exercises to improve literacy skills, the right brain-left brain myth, and learning styles. Higher education faculty were able to identify more Myths than any other group. Implications for teacher preparation and ongoing professional learning for teachers and educational leaders are discussed.
In reading intervention research, implementation fidelity is assumed to be positively related to student outcomes, but the methods used to measure fidelity are often treated as an afterthought. ...Fidelity has been conceptualized and measured in many different ways, suggesting a lack of construct validity. One aspect of construct validity is the fidelity index of a measure. This methodological case study examined how different decisions in fidelity indices influence relative rank ordering of individuals on the construct of interest and influence our perception of the relation between the construct and intervention outcomes. Data for this study came from a large state-funded project to implement multi-tiered systems of support for early reading instruction. Analyses were conducted to determine whether the different fidelity indices are stable in relative rank ordering participants and if fidelity indices of dosage and adherence data influence researcher decisions on model building within a multilevel modeling framework. Results indicated that the fidelity indices resulted in different relations to outcomes with the most commonly used fidelity indices for both dosage and adherence being the worst performing. The choice of index to use should receive considerable thought during the design phase of an intervention study.
Core reading instruction and interventions have differential effects based on student characteristics such as cognitive ability and pre-intervention skill level. Evidence for differential effect ...based on affective characteristics is scant and ambiguous; however, students with problem behavior are more often non-responsive to core reading instruction and intensive reading interventions. In this study, we estimated the range of students' behavior ratings in which a core reading instruction intervention was effective using a data set including 3,024 students in K-3. Data came from seven independent studies evaluating the Individualized Student Instruction (ISI) Tier 1 reading intervention and were pooled using integrative data analysis. We estimated Johnson-Neyman intervals of student behavior ratings that showed a treatment effect both at the within and between classroom level. ISI was effective in improving reading scores (b = 0.51, p = .020, d = 0.08). However, students with very low or very high behavior ratings did not benefit from the approaches (range of behavior rating factor scores: −0.95-2.87). At the classroom level, students in classrooms with a higher average of problem behaviors did not benefit from ISI (average classroom behavior rating factor score: 0.05-4.25). Results suggest differentiating instruction alone is not enough for students with behavior problems to grow in reading ability.
PDF
