Because negative findings have less chance of getting published, available studies tend to be a biased sample. This leads to an inflation of effect size estimates to an unknown degree. To see how ...meta-analyses in education account for publication bias, we surveyed all meta-analyses published in the last five years in the Review of Educational Research and Educational Research Review. The results show that meta-analyses usually neglect publication bias adjustment. In the minority of meta-analyses adjusting for bias, mostly non-principled adjustment methods were used, and only rarely were the conclusions based on corrected estimates, rendering the adjustment inconsequential. It is argued that appropriate state-of-the-art adjustment (e.g., selection models) should be attempted by default, yet one needs to take into account the uncertainty inherent in any meta-analytic inference under bias. We conclude by providing practical recommendations on dealing with publication bias.
Objectives
There are dozens of screening instruments purporting to measure the (Internet) gaming disorder (IGD/GD). The two prominent diagnostic manuals, DSM‐5 and ICD‐11, list several additional ...diagnostic or clinical features and problems (e.g., neglect of sleep, neglect of daily duties, health deterioration) that should co‐occur or be caused by the IGD/GD. It remains unclear how specific IGD/GD operationalizations (different screening scales) are related to these functional impairments.
Methods
To explore this, data on six measures of IGD/GD (IGDS9‐SF, GDSS, GDT, GAMES test, two self‐assessments) and 18 additional diagnostic features were collected from a sample of 1009 players who play digital games at least 13 h per week. A network approach was utilized to determine which operationalization is most strongly associated with functional impairment.
Results
In most of the networks, IGD/GD consistently emerged as the most central node.
Conclusion
The similar centrality of IGD/GD, irrespective of its definition (DSM‐5 or ICD‐11) or operationalization, provides support for the valid comparison or synthesis of results from studies that used instruments coming from both DSM‐5 and ICD‐11 ontologies, but only if the goal is to evaluate IGD/GD relationships to other phenomena, not the relationships between the symptoms themselves.
Robust scientific knowledge is contingent upon replication of original findings. However, replicating researchers are constrained by resources, and will almost always have to choose one replication ...effort to focus on from a set of potential candidates. To select a candidate efficiently in these cases, we need methods for deciding which out of all candidates considered would be the most useful to replicate, given some overall goal researchers wish to achieve. In this article we assume that the overall goal researchers wish to achieve is to maximize the utility gained by conducting the replication study. We then propose a general rule for study selection in replication research based on the replication value of the set of claims considered for replication. The replication value of a claim is defined as the maximum expected utility we could gain by conducting a replication of the claim, and is a function of (a) the value of being certain about the claim, and (b) uncertainty about the claim based on current evidence. We formalize this definition in terms of a causal decision model, utilizing concepts from decision theory and causal graph modeling. We discuss the validity of using replication value as a measure of expected utility gain, and we suggest approaches for deriving quantitative estimates of replication value. Our goal in this article is not to define concrete guidelines for study selection, but to provide the necessary theoretical foundations on which such concrete guidelines could be built.
Translational AbstractReplication-redoing a study using the same procedures-is an important part of checking the robustness of claims in the psychological literature. The practice of replicating original studies has been woefully devalued for many years, but this is now changing. Recent calls for improving the quality of research in psychology has generated a surge of interest in funding, conducting, and publishing replication studies. Because many studies have never been replicated, and researchers have limited time and money to perform replication studies, researchers must decide which studies are the most important to replicate. This way scientists learn the most, given limited resources. In this article, we lay out what it means to think about what is the most important thing to replicate, and we propose a general decision rule for picking a study to replicate. That rule depends on a concept we call replication value. Replication value is a function of the importance of the study, and how uncertain we are about the findings. In this article we explain how researchers can think precisely about the value of replication studies. We then discuss when and how it makes sense to use replication value as a measure of how valuable a replication study would be, and we discuss factors that funders, journals, or scientists could consider when determining how valuable a replication study is.
Concerns about the veracity of psychological research have been growing. Many findings in psychological science are based on studies with insufficient statistical power and nonrepresentative samples, ...or may otherwise be limited to specific, ungeneralizable settings or populations. Crowdsourced research, a type of large-scale collaboration in which one or more research projects are conducted across multiple lab sites, offers a pragmatic solution to these and other current methodological challenges. The Psychological Science Accelerator (PSA) is a distributed network of laboratories designed to enable and support crowdsourced research projects. These projects can focus on novel research questions or replicate prior research in large, diverse samples. The PSA’s mission is to accelerate the accumulation of reliable and generalizable evidence in psychological science. Here, we describe the background, structure, principles, procedures, benefits, and challenges of the PSA. In contrast to other crowdsourced research networks, the PSA is ongoing (as opposed to time limited), efficient (in that structures and principles are reused for different projects), decentralized, diverse (in both subjects and researchers), and inclusive (of proposals, contributions, and other relevant input from anyone inside or outside the network). The PSA and other approaches to crowdsourced psychological science will advance understanding of mental processes and behaviors by enabling rigorous research and systematic examination of its generalizability.
In two studies (N.sub.1 = 430; N.sub.2 = 500) on a general Slovak population (50.3% female; age = 39.8 + or - 11.7 years), we tested a structural model outlining the effects of one's economic ...situation and its subjective perception on time and risk preferences (in tasks with hypothetical rewards), with financial literacy serving as a mediator of these relationships. Even after respecifying the model, mostly weak or inconclusive relationships were observed. We further tested the time stability of time and risk preferences. On a sample of 224 participants who completed both waves (one year apart), we observed moderate correlations in the preferences even after controlling for income change. We argue that both time and risk preferences in monetary choices appear to be stable traits and are only marginally related to one's economic situation or financial literacy. Further investigation on the effectiveness of financial literacy in shaping economic preferences is needed.
Voluntary isolation is one of the most effective methods for individuals to help prevent the transmission of diseases such as COVID-19. Understanding why people leave their homes when advised not to ...do so and identifying what contextual factors predict this non-compliant behavior is essential for policymakers and public health officials. To provide insight on these factors, we collected data from 42,169 individuals across 16 countries. Participants responded to items inquiring about their socio-cultural environment, such as the adherence of fellow citizens, as well as their mental states, such as their level of loneliness and boredom. We trained random forest models to predict whether someone had left their home during a one week period during which they were asked to voluntarily isolate themselves. The analyses indicated that overall, an increase in the feeling of being caged leads to an increased probability of leaving home. In addition, an increased feeling of responsibility and an increased fear of getting infected decreased the probability of leaving home. The models predicted compliance behavior with between 54% and 91% accuracy within each country’s sample. In addition, we modeled factors leading to risky behavior in the pandemic context. We observed an increased probability of visiting risky places as both the anticipated number of people and the importance of the activity increased. Conversely, the probability of visiting risky places increased as the perceived putative effectiveness of social distancing decreased. The variance explained in our models predicting risk ranged from < .01 to .54 by country. Together, our findings can inform behavioral interventions to increase adherence to lockdown recommendations in pandemic conditions.
Voluntary isolation is one of the most effective methods for individuals to help prevent the transmission of diseases such as COVID-19. Understanding why people leave their homes when advised not to ...do so and identifying what contextual factors predict this non-compliant behavior is essential for policymakers and public health officials. To provide insight on these factors, we collected data from 42,169 individuals across 16 countries. Participants responded to items inquiring about their socio-cultural environment, such as the adherence of fellow citizens, as well as their mental states, such as their level of loneliness and boredom. We trained random forest models to predict whether someone had left their home during a one week period during which they were asked to voluntarily isolate themselves. The analyses indicated that overall, an increase in the feeling of being caged leads to an increased probability of leaving home. In addition, an increased feeling of responsibility and an increased fear of getting infected decreased the probability of leaving home. The models predicted compliance behavior with between 54% and 91% accuracy within each country's sample. In addition, we modeled factors leading to risky behavior in the pandemic context. We observed an increased probability of visiting risky places as both the anticipated number of people and the importance of the activity increased. Conversely, the probability of visiting risky places increased as the perceived putative effectiveness of social distancing decreased. The variance explained in our models predicting risk ranged from < .01 to .54 by country. Together, our findings can inform behavioral interventions to increase adherence to lockdown recommendations in pandemic conditions.
The article tackles the practice of testing latent variable models. The analysis covered recently published studies from 11 psychology journals varying in orientation and impact. Seventy-five studies ...that matched the criterion of applying some of the latent modeling techniques were reviewed. Results indicate the presence of a general tendency to ignore the model test (χ(2)) followed by the acceptance of approximate fit hypothesis without detailed model examination yielding relevant empirical evidence. Due to reduced sensitivity of such a procedure to confront theory with data, there is an almost invariable tendency to accept the theoretical model. This absence of model test consequences, manifested in frequently unsubstantiated neglect of evidence speaking against the model, thus implies the perilous question of whether such empirical testing of latent structures (the way it is widely applied) makes sense at all.
The changes in people’s mental health have become one of the hot topics during the COVID-19 pandemic. Parents have been said to be among the most vulnerable groups in terms of the imposed ...anti-pandemic measures. The present paper analyzes the trends in mental health indicators in a sample of Slovak parents (
N
= 363) who participated in four waves of data collection over a year and a half of the COVID-19 pandemic. The mental health indicators were represented by general levels of depression and anxiety as well as COVID-related stress and anxiety. While there were only minor changes in depression and anxiety, the dynamic in COVID-related stress and especially anxiety was more noteworthy. Besides some exceptions, the results hold even after controlling for the socioeconomic situation. The gender differences in the mental health trends were found to be negligible. Overall, we observed no substantial deterioration in the mental health indicators across the four waves of the study.
Motivation and self‐concept count among the educationally most relevant factors and the evaluation of many educational interventions requires their valid measurement. The present study examined the ...psychometric properties of a shortened version of the Students' Approaches to Learning questionnaire measuring 10 distinct motivational and self‐concept constructs. This large‐scale study drew a nationally representative sample (N = 6209; 49% female) from a population of 11–12‐year‐old Czech students. The assessment of construct validity indicated (1) the structural relations between the constructs, (2) the predictive relations to scores in standardized achievement tests or teacher‐assigned grades, and (3) the distinct differences in constructs' latent means between genders and students of the academic and mainstream track were mostly consistent with theory‐derived predictions. The nomological network of the measure maps relatively well onto the observed relations. Although not intended for individual assessment, the measure allows for psychometrically sound group inferences in relatively diverse student populations.
Practitioner points
The Students' Approaches to Learning questionnaire can provide valid group inferences on 10 self‐concept and motivational factors.
Self‐concept and interest constructs appear to be highly subject‐specific.
Self‐efficacy and academic self‐concept factors showed relatively strongest predictive validity with respect to academic achievement.
PDF
