Recent years have seen increasing interest in and attention to evidence-based practices, where the “evidence” generally comes from well-conducted randomized trials. However, while those trials yield ...accurate estimates of the effect of the intervention for the participants in the trial (known as “internal validity”), they do not always yield relevant information about the effects in a particular target population (known as “external validity”). This may be due to a lack of specification of a target population when designing the trial, difficulties recruiting a sample that is representative of a prespecified target population, or to interest in considering a target population somewhat different from the population directly targeted by the trial. This paper first provides an overview of existing design and analysis methods for assessing and enhancing the ability of a randomized trial to estimate treatment effects in a target population. It then provides a case study using one particular method, which weights the subjects in a randomized trial to match the population on a set of observed characteristics. The case study uses data from a randomized trial of school-wide positive behavioral interventions and supports (PBIS); our interest is in generalizing the results to the state of Maryland. In the case of PBIS, after weighting, estimated effects in the target population were similar to those observed in the randomized trial. The paper illustrates that statistical methods can be used to assess and enhance the external validity of randomized trials, making the results more applicable to policy and clinical questions. However, there are also many open research questions; future research should focus on questions of treatment effect heterogeneity and further developing these methods for enhancing external validity. Researchers should think carefully about the external validity of randomized trials and be cautious about extrapolating results to specific populations unless they are confident of the similarity between the trial sample and that target population.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
School-Wide Positive Behavioral Interventions and Supports (SWPBIS) is a universal prevention strategy currently implemented in >16,000 schools across the United States. SWPBIS intends to reduce ...students' behavior problems by altering staff behaviors and developing systems and supports to meet children's behavioral needs. The current study reports intervention effects on child behaviors and adjustment from an effectiveness trial of SWPBIS.
The sample of 12,344 elementary school children was 52.9% male, 45.1% African American, and 46.1% Caucasian. Approximately 49% received free or reduced-priced meals, and 12.9% received special education services at baseline. The trial used a group randomized controlled effectiveness design implemented in 37 elementary schools. Multilevel analyses were conducted on teachers' ratings of children's behavior problems, concentration problems, social-emotional functioning, prosocial behavior, office discipline referrals, and suspensions at 5 time points over the course of 4 school years.
The multilevel results indicated significant effects of SWPBIS on children's behavior problems, concentration problems, social-emotional functioning, and prosocial behavior. Children in SWPBIS schools also were 33% less likely to receive an office discipline referral than those in the comparison schools. The effects tended to be strongest among children who were first exposed to SWPBIS in kindergarten.
These findings provide support for the hypothesized reduction in behavior problems and improvements in prosocial behavior and effective emotion regulation after training in SWPBIS. The SWPBIS framework appears to be a promising approach for reducing problems and promoting adjustment among elementary school children.
Abstract Purpose As more and more youth utilize the Internet, concern about Internet harassment and its consequences for adolescents is growing. This paper examines the potential overlap in online ...and school harassment, as well as the concurrence of Internet harassment and school behavior problems. Methods The Growing Up with Media survey is a national cross-sectional online survey of 1588 youth between the ages of 10 and 15 years old. Our main measures were Internet harassment (i.e., rude or nasty comments, spreading of rumors, threatening or aggressive comments) and school functioning (i.e., academic performance; skipping school; detentions and suspensions; and carrying a weapon to school in the last 30 days). Results Although some overlap existed, 64% of youth who were harassed online did not report also being bullied at school. Nonetheless, youth harassed online were significantly more likely to also report two or more detentions or suspensions, and skipping school in the previous year. Especially concerning, youth who reported being targeted by Internet harassment were eight times more likely than all other youth to concurrently report carrying a weapon to school in the past 30 days (odds ratio = 8.0, p = .002). Conclusions Although the data do not support the assumption that many youth who are harassed online are bullied by the same (or even different) peers at school, findings support the need for professionals working with children and adolescents, especially those working in the schools, to be aware of the possible linkages between school behavior and online harassment for some youth.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK
School-Wide Positive Behavioral Interventions and Supports (SWPBIS; Sugai & Horner, 2006) is currently implemented in over 20,000 schools across the country with the goal of preventing disruptive ...behavior problems and enhancing the school climate. While previous studies have indicated significant main effects of SWPBIS on student outcomes, the program impacts likely vary as a function of children's social-emotional characteristics. This study examined variation in the effects of SWPBIS based on children's baseline pattern of behavior problems and social-emotional skills using data from a randomized controlled effectiveness trial. Data come from 12,344 elementary school children (52.9% male; 45.1% African American and 46.1% White; 49% received free or reduced-price meals; and 12.9% received special education services). The group randomized controlled effectiveness trial was implemented in 37 elementary schools. Latent profile analyses (LPA) were conducted on teachers' baseline ratings of children's behavior problems, concentration problems, social-emotional functioning, and prosocial behavior using the Teacher Observation of Classroom Adaptation-Checklist (TOCA-C; Koth, Bradshaw, & Leaf, 2009). LPA revealed 4 latent classes: high-risk (6.6%), at-risk (23.3%), normative (36.5%), and socially-emotionally skilled (33.6%). LPA membership was found to have a moderating impact on children's subsequent discipline problems and need for and use of school-based services. Findings suggest that the effects of SWPBIS on student outcomes tend to be greatest among at-risk and high-risk children.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ, UPUK
Randomized trials remain the most accepted design for estimating the effects of interventions, but they do not necessarily answer a question of primary interest: will the programme be effective in a ...target population in which it may be implemented? In other words, are the results generalizable? There has been very little statistical research on how to assess the generalizability, or `external validity', of randomized trials. We propose the use of propensity-score-based metrics to quantify the similarity of the participants in a randomized trial and a target population. In this setting the propensity score model predicts participation in the randomized trial, given a set of covariates. The resulting propensity scores are used first to quantify the difference between the trial participants and the target population, and then to match, subclassify or weight the control group outcomes to the population, assessing how well the propensity-score-adjusted outcomes track the outcomes that are actually observed in the population. These metrics can serve as a first step in assessing the generalizability of results from randomized trials to target populations. The paper lays out these ideas, discusses the assumptions underlying the approach and illustrates the metrics by using data on the evaluation of a schoolwide prevention programme called `Positive behavioral interventions and supports'.
Full text
Available for:
BFBNIB, FZAB, GIS, IJS, INZLJ, IZUM, KILJ, NLZOH, NMLJ, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBMB, UL, UM, UPUK, ZRSKP
Objective: Although schools are identified as critical for detecting youth mental disorders, little is known about whether the number of mental health providers and types of resources that they offer ...influence student mental health service use. Such information could inform the development and allocation of appropriate school-based resources to increase service use. This article examines associations of school resources with past-year mental health service use among students with 12-month "DSM-IV" mental disorders. Method: Data come from the U.S. National Comorbidity Survey Adolescent Supplement (NCS-A), a national survey of adolescent mental health that included 4,445 adolescent-parent pairs in 227 schools in which principals and mental health coordinators completed surveys about school resources and policies for addressing student emotional problems. Adolescents and parents completed the Composite International Diagnostic Interview and reported mental health service use across multiple sectors. Multilevel multivariate regression was used to examine associations of school mental health resources and individual-level service use. Results: Nearly half (45.3%) of adolescents with a 12-month "DSM-IV" disorder received past-year mental health services. Substantial variation existed in school resources. Increased school engagement in early identification was significantly associated with mental health service use for adolescents with mild/moderate mental and behavior disorders. The ratio of students to mental health providers was not associated with overall service use, but was associated with sector of service use. Conclusions: School mental health resources, particularly those related to early identification, may facilitate mental health service use and may influence sector of service use for youths with "DSM" disorders. (Contains 3 tables.)
Schoolwide Positive Behavioral Interventions and Supports (SWPBIS) is a universal, schoolwide prevention strategy that is currently implemented in over 9,000 schools across the nation to reduce ...disruptive behavior problems through the application of behavioral, social learning, and organizational behavioral principles. SWPBIS aims to alter school environments by creating improved systems and procedures that promote positive change in student behavior by targeting staff behaviors. This study uses data from a 5-year longitudinal randomized controlled effectiveness trial of SWPBIS conducted in 37 elementary schools to examine the impact of training in SWPBIS on implementation fidelity as well as student suspensions, office discipline referrals, and academic achievement. School-level longitudinal analyses indicated that the schools trained in SWPBIS implemented the model with high fidelity and experienced significant reductions in student suspensions and office discipline referrals.
Full text
Available for:
NUK, OILJ, SAZU, UKNU, UL, UM, UPUK
A positive school climate is an important component of successful and effective schools and thus is often an aim of schoolwide initiatives. Climate has traditionally been conceptualized as a ...school-level factor and is often assumed to be related to other school-level factors (e.g., school size). The current study examines variation in perceptions of climate based on individual-, classroom-, and school-level factors to determine the influence of predictors at multiple levels. Data come from 2,468 5th graders from 37 public elementary schools. Two aspects of students' perception of school climate, order and discipline, and achievement motivation are examined. Multilevel analyses in hierarchical linear modeling indicate that individual-level factors (race and sex) accounted for the largest proportion of variance in perceptions of school climate. School-level factors (e.g., school size and faculty turnover) and several classroom-level factors (e.g., characteristics of the teacher, class size, and the concentration of students with behavior problems) were also significant predictors of perceptions of climate. These findings suggest that characteristics of the classroom environment are important to consider when aiming to improve school climate.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ, UPUK
Although there is increasing awareness of the overrepresentation of ethic minority students-particularly Black students-in disciplinary actions, the extant research has rarely empirically examined ...potential factors that may contribute to these disparities. The current study used a multilevel modeling approach to examine factors at the child (e.g., teacher-rated disruptive behavior problems) and classroom or teacher levels (e.g., teacher ethnicity, level of disruptive behavior in classroom) that may contribute to the overrepresentation of minority students in office disciplinary referrals (ODRs). Data come from 6,988 children in 381 classrooms at 21 elementary schools. The analyses indicated that even after controlling for the student's level of teacher-rated behavior problems, teacher ethnicity, and other classroom factors, Black students were significantly more likely than White students to receive ODRs. Results also suggested that ethnic match between students and their teachers did not reduce the risk for referrals among Black students.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ, UPUK
PDF
