With the rising popularity of web surveys and the increasing use of paradata by survey methodologists, assessing information stored in user agent strings becomes inevitable. These data contain ...meaningful information about the browser, operating system, and device that a survey respondent uses. This article provides an overview of user agent strings, their specific structure and history, how they can be obtained when conducting a web survey, as well as what kind of information can be extracted from the strings. Further, the user written command parseuas is introduced as an efficient means to gather detailed information from user agent strings. The application of parseuas is illustrated by an example that draws on a pooled data set consisting of 29 web surveys.
Abstract In the social and behavioral sciences, surveys are frequently used to collect data. During the COVID-19 pandemic, surveys provided political actors and public health professionals with ...timely insights on the attitudes and behaviors of the general population. These insights were key in guiding actions to fight the pandemic. However, the data quality of these surveys remains unclear because systematic knowledge about how the survey data were collected during the COVID-19 pandemic is lacking. This is unfortunate, since decades of survey research have shown that survey design impacts data. Our Survey Data Collection and the COVID-19 Pandemic (SDCCP) project deals with this research gap. We collected rich metadata on survey design for 717 social and behavioral science surveys carried out in Germany during the first two years of the COVID-19 pandemic. In this data descriptor, we present a unique resource for a systematic assessment of the survey data collection practices and quality of surveys conducted in Germany during the COVID-19 pandemic.
This article addresses the questions of whether paradata can help us to improve the models of panel attrition and whether paradata can improve the effectiveness of propensity score weights with ...respect to reducing attrition biases. The main advantage of paradata is that it is collected as a by-product of the survey process. However, it is still an open question which paradata can be used to model attrition and to what extent these paradata are correlated with the variables of interest. Our analysis used data from a seven-wave web-based panel survey that had been supplemented by three cross-sectional surveys. This split panel design allowed us to assess the magnitude of attrition bias for a large number of substantive variables. Furthermore, this design enabled us to analyze in detail the effectiveness of propensity score weights. Our results showed that some paradata (e.g., response times and participation history) improved the prediction of panel attrition, whereas others did not. In addition, not all the paradata that increased the model fit resulted in weights that effectively reduced bias. These findings highlight the importance of selecting paradata that are linked to both the survey response process and the variables of interest. This article provides a first contribution to this challenge.
Dyadic surveys aim to interview pairs of respondents, such as partners in a relationship. In dyadic surveys, it is often necessary to obtain the anchors’ consent to contact their partners and invite ...them to a survey. If the survey is operated in self-administered modes, no interviewer is present to improve the consent rate, for example, by providing convincing arguments and additional information. To overcome the challenges posed by self-administered modes for dyadic surveys and to improve consent rates, it is important to identify aspects that positively influence the likelihood of anchors giving consent to contact their partners. Ideally, these aspects are in the hands of the researchers, such as the survey design and aspects of the questionnaire. Thus, in this study, we analyzed the relationship between anchors’ survey experience and their willingness to consent to surveying their partners in self-administered modes. Based on data from the German Family Demography Panel Study (FReDA), we found that the anchors’ perceptions of the questionnaire as “interesting” or “too personal” were related to consent rates. These relationships were consistent across different survey modes and devices. Effects of other aspects of the questionnaire, such as “important for science” and “diverse” varied between modes and devices. We concluded with practical recommendations for survey research and an outlook for future research.
* This article belongs to a special issue on “Family Research and Demographic Analysis – New Insights from the German Family Demography Panel Study (FReDA)”.
Abstract
The European Values Study (EVS) was first conducted in 1981 and then repeated in 1990, 1999, 2008, and 2017, with the aim of providing researchers with data to investigate whether European ...individual and social values are changing and to what degree. The EVS is traditionally carried out as a probability-based face-to-face survey that takes around 1 hour to complete. In recent years, large-scale population surveys such as the EVS have been challenged by decreasing response rates and increasing survey costs. In the light of these challenges, six countries that participated in the last wave of the EVS tested the application of self-administered mixed-modes (Denmark, Finland, Germany, Iceland, the Netherlands, and Switzerland). With the present data brief, we will introduce researchers to the latest wave of the EVS, the implemented mode experiments, and the EVS data releases. In our view, it is pivotal for data use in substantive research to make the reasoning behind design changes and country-specific implementations transparent as well as to highlight new research opportunities.
Response probabilities are used in adaptive and responsive survey designs to guide data collection efforts, often with the goal of diversifying the sample composition. However, if response ...probabilities are also correlated with measurement error, this approach could introduce bias into survey data. This study analyzes the relationship between response probabilities and data quality in grid questions. Drawing on data from the probability-based GESIS panel, we found low propensity cases to more frequently produce item nonresponse and nondifferentiated answers than high propensity cases. However, this effect was observed only among long-time respondents, not among those who joined more recently. We caution that using adaptive or responsive techniques may increase measurement error while reducing the risk of nonresponse bias.
Survey research is still confronted by a trend of increasing nonresponse rates. In this context, several methodological advances have been made to stimulate participation and avoid bias. Yet, despite ...the growing number of tools and methods to deal with nonresponse, little is known about whether nonresponse biases show similar trends as nonresponse rates and what mechanisms (if any) drive changes in bias. Our article focuses on biases in cohort distributions in the U.S. and German general social surveys from 1980 to 2012 as one of the key variables in the social sciences. To supplement our cross-national comparison of these trends, we decompose changes into within-cohort change (WCC) and between-cohort change. We find that biases in cohort distributions have remained relatively stable and at a relatively low level in both countries. Furthermore, WCC (i.e., survey climate) accounts for the major part of the change in nonresponse bias.
The worldwide spread of the COVID-19 pandemic has disrupted the fieldwork of surveys. The data collection efforts via the face-to-face mode have been affected especially, including the ongoing ...surveys that were in the field during the COVID-19 outbreak and the planned surveys scheduled for fieldwork later in 2020. We provide an account of how COVID-19 has impacted two family studies in Germany: “The German Family Panel” (pairfam) and the “Generations and Gender Survey” (GGS) both of which will be part of the “Family Research and Demographic Analysis” (FReDA) infrastructure. Based on pairfam, we illustrate the effects of the pandemic on ongoing data collection and the measures taken to proceed with fieldwork, and we report on a special COVID-19 survey. Based on FReDA-GGS, we outline how COVID-19 has affected our planned survey schedules, what future challenges are expected when fieldwork becomes possible again, and how we have adapted our plans accordingly.
This article introduces the evolution, framework, objectives, and design of the new data infrastructure “FReDA – The German Family Demography Panel Study”, which has been funded by the German Federal ...Ministry of Education and Research (BMBF) since 2020. FReDA is rooted in the Generation and Gender Survey (GGS) and the German Family Panel (pairfam). FReDA was initiated to facilitate research on family and demography by providing a comprehensive panel study allowing for international comparisons as well as dyadic analyses through a multi-actor design. The survey covers major fields of family research, such as fertility behaviour, reproductive health, work-family conflict, dyadic division of work, gender roles, intimate relations, separation and divorce, parenting and intergenerational relations, social inequalities, family attitudes, and well-being. FReDA interviews are conducted in a self-administered web-based (CAWI) or paper-based (PAPI) manner. The infrastructure consists of two different samples. First, the new FReDA-GGS survey started in early 2021, with a wave 1 study population of individuals aged 18 to 49 years and their partners. Second, the FReDA-pairfam survey will continue the 14-wave pairfam sample from 2022 onwards. The questionnaires of both samples will be harmonised as FReDA evolves. Data accessibility, organisation, and future perspectives of the data infrastructure are described and discussed in the paper’s conclusions.
Since cash lotteries are frequently employed to stimulate participation in web surveys sampled from online access panels, the present study assesses the research gap concerning the relevance of ...respondents knowing their odds of winning these cash lotteries. Prior studies on the use of cash lotteries have not provided the odds of winning to respondents, which may have obscured their effects. Based on an experiment in which we varied the number of prizes, the size of prizes, and whether respondents knew the odds of winning, we replicated the findings of prior studies on the use of cash lotteries. Most importantly, our findings show that knowing the odds of winning cash lotteries did not affect the participation in our survey and did not influence the effects of other survey design elements. In our study, we provide survey practitioners with recommendations on the use of cash lotteries in online access panels and close with an outlook for future research on this topic.
PDF
