We estimate school-choice preferences revealed by the rank-ordered lists submitted by more than 22,000 applicants to a citywide lottery for more than 200 traditional and charter public schools in ...Washington, D.C. The results confirm previously reported findings that commuting distance, schoo demographics, and academic indicators play important roles in school choice and that there is considerable heterogeneity of preferences. Higher and lower income choosers respond to academic quality measures, but respond to different indicators of quality. Simulations suggest segregation by race and income would be reduced and enrollment in high-performing schools increased if policymakers were to relax school capacity constraints in individual campuses. The simulations also suggest that removing the lowest performing schools as choice options couldfurther reduce segregation and increase enrollment in high-performing schools.
Randomized experiments involving education interventions are typically implemented as cluster randomized trials, with schools serving as clusters. To design such a study, it is critical to understand ...the degree to which learning outcomes vary between versus within clusters (schools), specifically the intraclass correlation coefficient. It is also helpful to anticipate the benefits, in terms of statistical power, of collecting household data, testing students at baseline, or relying on administrative data on previous cohorts from the same school. We use data from multiple cluster-randomized trials in four Latin American countries to provide information on the intraclass correlations in early grade literacy outcomes. We also describe the proportion of variance explained by different types of covariates. These parameters will help future researchers conduct statistical power analysis, estimate the required sample size, and determine the necessity of collecting different types of baseline data such as child assessments, administrative data at the school level, or household surveys.
Random digit dial surveys with mobile phones risk under-representation of women. To address this, we compare the characteristics of women recruited directly with those of women recruited through ...referrals from male household members. The referral process improves representation of vulnerable groups, such as young women, the asset poor, and those living in areas with low connectivity. Among mobile phone users, we show a referral (rather than a direct dial) protocol includes more nationally representative proportions of women with these attributes. While seeking intra-household referrals may improve representation, we show that it does so at a higher cost.
•Passing the phone in RDD surveys improves representation of women.•Referrals reach more youth, the asset poor, and those living in low connectivity.•Costs affect scalability of referrals given low completion rates.
Randomized trials are a common way to provide rigorous evidence on the impacts of education programs. This article discusses the trade-offs associated with study designs that involve random ...assignment of students within schools and describes the experience from one such study of Teach for America (TFA). The TFA experiment faced challenges with recruitment, randomization of students, and analysis. The solutions to those challenges may be instructive for experimenters who wish to study future interventions at the student or classroom level. The article concludes that within-school random assignment studies such as the TFA evaluation are challenging but, under the right conditions, are also feasible and potentially very rewarding in terms of generating useful evidence for policy.
The Choice Architecture of School Choice Websites Glazerman, Steven; Nichols-Barrer, Ira; Valant, Jon ...
Journal of research on educational effectiveness,
04/2020, Letnik:
13, Številka:
2
Journal Article
Recenzirano
We conducted a randomized factorial experiment to determine how displaying school information to parents in different ways affects what schools they choose for their children in a hypothetical school ...district. In a sample of 3,500 low-income parents of school-aged children, a small design manipulation, such as changing the default order in which schools were presented, induced meaningful changes in the types of schools selected. Other design choices such as using icons to represent data, instead of graphs or just numbers, or presenting concise summaries instead of detailed displays, also led parents to choose schools with higher academic performance. We also examined effects on parents' understanding of the information and their self-reported satisfaction and ease of use. In some cases, there were trade-offs. For example, representing data using only numbers maximized understanding, but adding graphs maximized satisfaction at the expense of understanding.
This paper reports on a randomized experiment to study the impact of an alternative teacher preparation program, Teach for America (TFA), on student achievement and other outcomes. We found that TFA ...teachers had a positive impact on math achievement and no impact on reading achievement. The size of the impact on math scores was about 15 percent of a standard deviation, equivalent to about one month of instruction. The general conclusions did not differ substantially for subgroups of teachers, including novice teachers, or for subgroups of students. We found no impacts on other student outcomes such as attendance, promotion, or disciplinary incidents, but TFA teachers were more likely to report problems with student behavior than were their peers. The findings contradict claims that such programs allowing teachers to bypass the traditional route to the classroom harm students.
To assess nonexperimental (NX) evaluation methods in to context of welfare, job training, and employment services programs, the authors reexamined the results of twelve case studies intended to ...replicate impact estimates from an experimental evaluation by using NX methods. They found that the NX methods sometimes came close to replicating experimentally derived results but often produced estimates that differed by policy-relevant margins, which the authors interpret as estimates of bias. Although the authors identified several study design factors associated with smaller discrepancies, no combination of factors would consistently eliminate discrepancies. Even with a large number of impact estimates, the positive and negative bias estimates did not always cancel each other out. Thus, it was difficult to identify an aggregation strategy that consistently removed bias while answering a focused question about earnings impacts of a program. They conclude that although the empirical evidence from this literature can be used in the context of training and welfare programs to improve NX research designs, it cannot on its own justify the use of such designs.
We examine behavioral responses to an incentive program that offers high-performing teachers in ten school districts across the country $20,000 to transfer into the district's hardest-to-staff ...schools. We discuss behavioral responses to the program on high-performing teachers' willingness to transfer (supply) and the effect of the transfer offer on the internal dynamics of the receiving schools (demand). We found low take-up rates among the 1,514 high-performing teachers who were offered the incentive, with minimal sorting on observable characteristics. Within the new schools, transfer teachers were less likely than their counterparts in a randomized control group to require mentoring and more likely to provide mentoring themselves. No significant differences occurred in school climate, collegiality, or the way in which students were assigned to teachers, but evidence indicates that principalsmay have strategically assigned existing teachers to grades in both treatment and control schools in response to the quality of the incoming teachers.
Increasing demands by government for “evidence-led” policy raise the risk that research evidence will mislead government rather than leading to an unbiased conclusion. The need for unbiased research ...conclusions has never been greater, yet few consumers of research understand the statistical biases with which science must always struggle. This article introduces the volume's discussion of those issues with an explanation of the major threats of bias in social science research and a map of the differing scientific opinions on how to deal with those threats. The thesis of the volume is that many of these threats could be reduced by making social science more experimental. The fact that even experimental evidence contains threats of bias does not alter that claim but merely suggests another: that educated consumers of social science may be the best defense against misleading evidence of all kinds.
Those who favor expansion of consumer choice in education claim that competition would force schools to improve. Critics claim that it would sort students by race and class. A competitive market will ...provide what consumers demand, yet neither side has empirical evidence on such consumer preferences to back up their claims. Here we offer such evidence. The dissertation estimates a conditional logit model using original data collected by the author from a public school choice program in Minneapolis, Minnesota, in order to infer how families trade off the convenience of a shorter commute with school quality and peer group characteristics. The evidence suggests that at the primary school level consumer choice alone would not raise schools' academic performance, as traditionally defined. Parents in Minneapolis were not more likely to choose elementary schools with high test scores or greater value added. Rather they preferred schools relatively close to home and ones where they were better represented ethnically and racially. Simulations suggest that expanding choice could ultimately lead to severe, but not total segregation by race and ethnicity, perhaps no worse than would arise in a system of neighborhood schools. Even if consumer choice alone does not give schools incentives to raise their academic performance, the social cost in terms of racial segregation may still be outweighed by the private benefits in terms of consumer satisfaction.
PDF
