Despite increasing efforts to encourage the adoption of field experiments in marketing research (e.g., Campbell 1969; Cialdini 1980; Li et al. 2015), the majority of scholars continue to rely ...primarily on laboratory studies (Cialdini 2009). For example, of the 50 articles published in Journal of Marketing Research in 2013, only three (6%) were based on field experiments. The goal of this article is to motivate a methodological shift in marketing research and increase the proportion of empirical findings obtained using field experiments. The author begins by making a case for field experiments and offers a description of their defining features. She then demonstrates the unique value that field experiments can offer and concludes with a discussion of key considerations that researchers should be mindful of when designing, planning, and running field experiments.
Does the consumption of ideologically congruent news on social media exacerbate polarization? I estimate the effects of social media news exposure by conducting a large field experiment randomly ...offering participants subscriptions to conservative or liberal news outlets on Facebook. I collect data on the causal chain of media effects: subscriptions to outlets, exposure to news on Facebook, visits to online news sites, and sharing of posts, as well as changes in political opinions and attitudes. Four main findings emerge. First, random variation in exposure to news on social media substantially affects the slant of news sites that individuals visit. Second, exposure to counter-attitudinal news decreases negative attitudes toward the opposing political party. Third, in contrast to the effect on attitudes, I find no evidence that the political leanings of news outlets affect political opinions. Fourth, Facebook’s algorithm is less likely to supply individuals with posts from counter-attitudinal outlets, conditional on individuals subscribing to them. Together, the results suggest that social media algorithms may limit exposure to counter-attitudinal news and thus increase polarization.
Twelve percent of the Malawian population is HIV infected. Eighteen percent of sexual encounters are casual. A condom is used a third of the time. To analyze the Malawian epidemic, a choice-theoretic ...general equilibrium search model is constructed. In the developed framework, people select between different sexual practices while knowing the inherent risk. The calibrated model is used to study several policy interventions, namely, ART, circumcision, better condoms, and the treatment of other STDs. The efficacy of public policy depends upon the induced behavioral changes and equilibrium effects. The framework complements the insights from epidemiological studies and small-scale field experiments.
Firms using online advertising regularly run experiments with multiple versions of their ads since they are uncertain about which ones are most effective. During a campaign, firms try to adapt to ...intermediate results of their tests, optimizing what they earn while learning about their ads. Yet how should they decide what percentage of impressions to allocate to each ad? This paper answers that question, resolving the well-known “learn-and-earn” trade-off using multi-armed bandit (MAB) methods. The online advertiser’s MAB problem, however, contains particular challenges, such as a hierarchical structure (ads within a website), attributes of actions (creative elements of an ad), and batched decisions (millions of impressions at a time), that are not fully accommodated by existing MAB methods. Our approach captures how the impact of observable ad attributes on ad effectiveness differs by website in unobserved ways, and our policy generates allocations of impressions that can be used in practice. We implemented this policy in a live field experiment delivering over 750 million ad impressions in an online display campaign with a large retail bank. Over the course of two months, our policy achieved an 8% improvement in the customer acquisition rate, relative to a control policy, without any additional costs to the bank. Beyond the actual experiment, we performed counterfactual simulations to evaluate a range of alternative model specifications and allocation rules in MAB policies. Finally, we show that customer acquisition would decrease by about 10% if the firm were to optimize click-through rates instead of conversion directly, a finding that has implications for understanding the marketing funnel.
Data is available at
https://doi.org/10.1287/mksc.2016.1023
.
In this introduction to the symposium, I first offer an overview of the spectrum of experimental methods in economics, from laboratory experiments to the field experiments that are the subject of ...this symposium. I then offer some thoughts about the potential gains from doing economic research using field experiments and my own mental checklist of 14 steps to improve the chances of carrying out an economics field experiment successfully. PUBLICATION ABSTRACT
Does "liking" a brand on Facebook cause a person to view it more favorably? Or is "liking" simply a symptom of being fond of a brand? The authors disentangle these possibilities and find evidence for ...the latter: brand attitudes and purchasing are predicted by consumers' preexisting fondness for brands, and these are the same regardless of when and whether consumers "like" brands on social media. In addition, we explore possible second-order effects by examining whether "liking" brands might cause consumers' friends to view that brand more favorably. When consumers see that a friend has "liked" a brand, they are less likely to buy the brand relative to when they learn that a friend genuinely likes the brand in the offline sense, which is a more meaningful social endorsement. Taken together, five experiments and two meta-analyses (N > 14,000) suggest that turning "liking" into improved brand attitudes and increased purchasing by consumers and their friends may require more than just the click of a button.
In collaboration with three companies selling a diverse set of products, we conducted randomized field experiments in which experimentally tailored email ads were sent to millions of individuals. We ...found consistently that personalizing the emails by adding consumer-specific information (e.g., recipient’s name) benefited the advertisers. Importantly, such content is not likely to be informative about the advertised product or the company. In our main experiment, we found that adding the name of the message recipient to the email’s subject line increased the probability of the recipient opening it by 20% (from 9.05% to 10.80%), which translated to an increase in sales’ leads by 31% (from 0.39% to 0.51%) and a reduction in the number of individuals unsubscribing from the email campaign by 17% (from 1.2% to 1.0%). We present similar experiments conducted with other companies, which show that the effects we document extend from objectives ranging from acquiring new customers to retaining customers who have purchased from the company in the past. Our investigation of several possible mechanisms suggests that such content increases the effort consumers make in processing the other content in the rest of the advertising message. Our paper quantifies the benefits from personalization and sheds light on the role of noninformative advertising content by analyzing several detailed measures of recipient’s interaction with the message. It provides external validity to psychological mechanisms and has clear implications for the firms that are designing their advertising campaigns.
Data and the online appendix are available at
https://doi.org/10.1287/mksc.2017.1066
.
We study a form of threshold matching in fundraising where donations above a certain threshold are topped up with a fixed amount. We show theoretically that threshold matching can induce crowding in ...if appropriately personalized. In a field experiment, we explore how thresholds should be chosen depending on past donations. The optimal choice of thresholds is rather bold, approximately 75% above past donations. Additionally, we explore how thresholds should be set for new donors as a function of their personal characteristics and demonstrate the benefits of personalization as opposed to setting a general threshold that applies to all recipients of a fundraising call.
Short-run subsidies for health products are common in poor countries. How do they affect long-run adoption? A common fear among development practitioners is that one-off subsidies may negatively ...affect long-run adoption through reference-dependence: People might anchor around the subsidized price and be unwilling to pay more for the product later. But for experience goods, one-off subsidies could also boost long-run adoption through learning. This paper uses data from a two-stage randomized pricing experiment in Kenya to estimate the relative importance of these effects for a new, improved antimalarial bed net. Reduced form estimates show that a one-time subsidy has a positive impact on willingness to pay a year later inherit. To separately identify the learning and anchoring effects, we estimate a parsimonious experience-good model. Estimation results show a large, positive learning effect but no anchoring. We black then discuss the types of products and the contexts inherit for which these results may apply.
Policymakers are increasingly using norm-based messages to influence individual decision making. We partner with a metropolitan water utility to implement a natural field experiment to examine the ...effect of such messages on residential water demand. The data, drawn from more than 100,000 households, indicate that social comparison messages had a greater influence on behavior than simple prosocial messages or technical information alone. Moreover, our data suggest that social comparison messages are most effective among households identified as the least price sensitive: high users. Yet the effectiveness of such messages wanes over time. Our results thus highlight important complementarities between pecuniary and nonpecuniary strategies.
PDF
