Network models are an increasingly popular way to abstract complex psychological phenomena. While studying the
structure
of network models has led to many important insights, little attention has ...been paid to how well they
predict
observations. This is despite the fact that predictability is crucial for judging the
practical relevance
of edges: for instance in clinical practice, predictability of a symptom indicates whether an intervention on that symptom through the symptom network is promising. We close this methodological gap by introducing nodewise predictability, which quantifies how well a given node can be predicted by all other nodes it is connected to in the network. In addition, we provide fully reproducible code examples of how to compute and visualize nodewise predictability both for cross-sectional and time series data.
We present the R package mgm for the estimation of k-order mixed graphical models (MGMs) and mixed vector autoregressive (mVAR) models in high-dimensional data. These are a useful extensions of ...graphical models for only one variable type, since data sets consisting of mixed types of variables (continuous, count, categorical) are ubiquitous. In addition, we allow to relax the stationarity assumption of both models by introducing time-varying versions of MGMs and mVAR models based on a kernel weighting approach. Time-varying models offer a rich description of temporally evolving systems and allow to identify external influences on the model structure such as the impact of interventions. We provide the background of all implemented methods and provide fully reproducible examples that illustrate how to use the package.
Statistical network models describing multivariate dependency structures in psychological data have gained increasing popularity. Such comparably novel statistical techniques require specific ...guidelines to make them accessible to the research community. So far, researchers have provided tutorials guiding the estimation of networks and their accuracy. However, there is currently little guidance in determining what parts of the analyses and results should be documented in a scientific report. A lack of such reporting standards may foster researcher degrees of freedom and could provide fertile ground for questionable reporting practices. Here, we introduce reporting standards for network analyses in cross-sectional data, along with a tutorial and two examples. The presented guidelines are aimed at researchers as well as the broader scientific community, such as reviewers and journal editors evaluating scientific work. We conclude by discussing how the network literature specifically can benefit from such guidelines for reporting and transparency.
Translational Abstract
In recent years, network models have become increasingly popular in the field of psychology. Such comparably novel statistical techniques require specific guidelines to make them accessible to the research community. So far, researchers have provided tutorials guiding how network analysis can be applied to psychological data. However, there is currently little guidance in determining what parts of the analyses and results should be documented in a scientific report. A lack of such reporting standards may result in researchers being confronted with too much choice in reporting their results, which in turn might provide fertile ground for questionable reporting practices. Here, we introduce reporting standards for network analyses in cross-sectional data, along with a tutorial and two examples. The presented guidelines are aimed at researchers as well as the broader scientific community, such as reviewers and journal editors evaluating scientific work. We conclude by discussing how the network literature specifically can benefit from such guidelines for reporting and transparency.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ
Statistical network models such as the Gaussian Graphical Model and the Ising model have become popular tools to analyze multivariate psychological datasets. In many applications, the goal is to ...compare such network models across groups. In this paper, I introduce a method to estimate group differences in network models that is based on moderation analysis. This method is attractive because it allows one to make comparisons across more than two groups for all parameters within a single model and because it is implemented for all commonly used cross-sectional network models. Next to introducing the method, I evaluate the performance of the proposed method and existing approaches in a simulation study. Finally, I provide a fully reproducible tutorial on how to use the proposed method to compare a network model across three groups using the R-package
mgm
.
Exploratory factor analysis (EFA) is one of the most popular statistical models in psychological science. A key problem in EFA is to estimate the number of factors. In this article, we present a new ...method for estimating the number of factors based on minimizing the out-of-sample prediction error of candidate factor models. We show in an extensive simulation study that our method slightly outperforms existing methods, including parallel analysis, Bayesian information criterion (BIC), Akaike information criterion (AIC), root mean squared error of approximation (RMSEA), and exploratory graph analysis. In addition, we show that, among the best performing methods, our method is the one that is most robust across different specifications of the true factor model. We provide an implementation of our method in the R-package fspe.
Translational Abstract
Exploratory factor analysis (EFA) is one of the most popular statistical models in psychological science. A key problem in EFA is to estimate the number of factors. In this article, we present a new method for estimating the number of factors based on minimizing the out-of-sample prediction error of candidate factor models. We show in an extensive simulation study that our method slightly outperforms existing methods and is more robust across different specifications of the true factor model than other high performing methods. We provide an implementation of our method in the R-package fspe.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ
In recent years, a growing chorus of researchers has argued that psychological theory is in a state of crisis: Theories are rarely developed in a way that indicates an accumulation of knowledge. Paul ...Meehl raised this very concern more than 40 years ago. Yet in the ensuing decades, little has improved. We aim to chart a better path forward for psychological theory by revisiting Meehl’s criticisms, his proposed solution, and the reasons his solution failed to meaningfully change the status of psychological theory. We argue that Meehl identified serious shortcomings in our evaluation of psychological theories and that his proposed solution would substantially strengthen theory testing. However, we also argue that Meehl failed to provide researchers with the tools necessary to construct the kinds of rigorous theories his approach required. To advance psychological theory, we must equip researchers with tools that allow them to better generate, evaluate, and develop their theories. We argue that formal theories provide this much-needed set of tools, equipping researchers with tools for thinking, evaluating explanation, enhancing measurement, informing theory development, and promoting the collaborative construction of psychological theories.
Moderated Network Models Haslbeck, Jonas M. B.; Borsboom, Denny; Waldorp, Lourens J.
Multivariate behavioral research,
03/2021, Volume:
56, Issue:
2
Journal Article
Peer reviewed
Pairwise network models such as the Gaussian Graphical Model (GGM) are a powerful and intuitive way to analyze dependencies in multivariate data. A key assumption of the GGM is that each pairwise ...interaction is independent of the values of all other variables. However, in psychological research, this is often implausible. In this article, we extend the GGM by allowing each pairwise interaction between two variables to be moderated by (a subset of) all other variables in the model, and thereby introduce a Moderated Network Model (MNM). We show how to construct MNMs and propose an
-regularized nodewise regression approach to estimate them. We provide performance results in a simulation study and show that MNMs outperform the split-sample based methods Network Comparison Test (NCT) and Fused Graphical Lasso (FGL) in detecting moderation effects. Finally, we provide a fully reproducible tutorial on how to estimate MNMs with the R-package mgm and discuss possible issues with model misspecification.
Gaussian mixture models (GMMs) are a popular and versatile tool for exploring heterogeneity in multivariate continuous data. Arguably the most popular way to estimate GMMs is via the ...expectation–maximization (EM) algorithm combined with model selection using the Bayesian information criterion (BIC). If the GMM is correctly specified, this estimation procedure has been demonstrated to have high recovery performance. However, in many situations, the data are not continuous but ordinal, for example when assessing symptom severity in medical data or modeling the responses in a survey. For such situations, it is unknown how well the EM algorithm and the BIC perform in GMM recovery. In the present paper, we investigate this question by simulating data from various GMMs, thresholding them in ordinal categories and evaluating recovery performance. We show that the number of components can be estimated reliably if the number of ordinal categories and the number of variables is high enough. However, the estimates of the parameters of the component models are biased independent of sample size. Finally, we discuss alternative modeling approaches which might be adopted for the situations in which estimating a GMM is not acceptable.
Time series of individual subjects have become a common data type in psychological research. The Vector Autoregressive (VAR) model, which predicts each variable by all variables including itself at ...previous time points, has become a popular modeling choice for these data. However, the number of observations in typical psychological applications is often small, which puts the reliability of VAR coefficients into question. In such situations it is possible that the simpler AR model, which only predicts each variable by itself at previous time points, is more appropriate. Bulteel et al. (2018) used empirical data to investigate in which situations the AR or VAR models are more appropriate and suggest a rule to choose between the two models in practice. We provide an extended analysis of these issues using a simulation study. This allows us to (1) directly investigate the relative performance of AR and VAR models in typical psychological applications, (2) show how the relative performance depends both on n and characteristics of the true model, (3) quantify the uncertainty in selecting between the two models, and (4) assess the relative performance of different model selection strategies. We thereby provide a more complete picture for applied researchers about when the VAR model is appropriate in typical psychological applications, and how to select between AR and VAR models in practice.
Researchers are often interested in comparing statistical network models estimated from groups that are defined by the sum-score of the modeled variables. A prominent example is an analysis that ...compares networks of individuals with and without a diagnosis of a certain disorder. Recently, several authors suggested that this practice may lead to invalid inferences by introducing Berkson's bias. In this article, we show that whether bias is present or not depends on which research question one aims to answer. We review five possible research questions one may have in mind when separately estimating network models in groups that are based on sum-scores. For each research question, we provide an illustration with a simulated bivariate example and discuss the nature of the bias, if present. We show that if one is indeed interested in the network models of the groups defined by the sum-score, no bias is introduced. However, if one is interested in differences across groups defined by a variable other than the sum-score, detecting population heterogeneity, the network model in the general population, or inferring causal relations, then bias will be introduced in most situations. Finally, we discuss for each research question how bias can be avoided.
Translational AbstractResearchers in clinical psychology and psychiatry increasingly study relationships between symptoms of mental disorders using network models. In this context, it is often interesting to compare network models across groups, for example based on gender, age, or whether individuals are diagnosed with a disorder. The latter case is different from the first two, because diagnosis groups are themselves defined by the presence or absence of symptoms. Recently, several researchers have suggested that creating groups in this way leads to biased estimates. In this article, we show that whether or not bias occurs depends on the research question at hand. We consider five different types of research questions and in each case determine what the target of our analysis is, whether bias is present, and if so, where that bias is coming from. In each case, we outline how that bias could best be dealt with or avoided, showing that each research question requires a qualitatively different approach.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ
PDF
