Traditionally, researchers have used time series and multilevel models to analyze intensive longitudinal data. However, these models do not directly address traits and states which conceptualize the ...stability and variability implicit in longitudinal research, and they do not explicitly take into account measurement error. An alternative to overcome these drawbacks is to consider structural equation models (state-trait SEMs) for longitudinal data that represent traits and states as latent variables. Most of these models are encompassed in the latent state-trait (LST) theory. These state-trait SEMs can be problematic when the number of measurement occasions increases. As they require the data to be in wide format, these models quickly become overparameterized and lead to nonconvergence issues. For these reasons, multilevel versions of state-trait SEMs have been proposed, which require the data in long format. To study how suitable state-trait SEMs are for intensive longitudinal data, we carried out a simulation study. We compared the traditional single level to the multilevel version of three state-trait SEMs. The selected models were the multistate-singletrait (MSST) model, the common and unique trait-state (CUTS) model, and the trait-state-occasion (TSO) model. Furthermore, we also included an empirical application. Our results indicated that the TSO model performed best in both the simulated and the empirical data. To conclude, we highlight the usefulness of state-trait SEMs to study the psychometric properties of the questionnaires used in intensive longitudinal data. Yet, these models still have multiple limitations, some of which might be overcome by extending them to more general frameworks.
Translational Abstract
Collecting data from individuals multiple times a day for several days is becoming more popular among psychological researchers. This kind of data, known as intensive longitudinal data, allows studying how people change over short periods of time. In the present study, we explored how to study traits and states in intensive longitudinal data. Traits conceptualize the stability of variables, whereas states conceptualize their variability. For this, we conducted a simulation study in which we studied three state-trait structural equation models (state-trait SEMs). State-trait SEMs are measurement models for longitudinal data that represent traits and states as latent variables and include measurement error terms. Our results showed that the trait-state-occasion (TSO) model outperforms the other two. We concluded that the TSO model is an additional tool to consider when analyzing intensive longitudinal data.
In this article, the practical consequences of violations of unidimensionality on selection decisions in the framework of unidimensional item response theory (IRT) models are investigated based on ...simulated data. The factors manipulated include the severity of violations, the proportion of misfitting items, and test length. The outcomes that were considered are the precision and accuracy of the estimated model parameters, the correlations of estimated ability (
θ
^
) and number-correct (
NC
) scores with the true ability (
θ
), the ranks of the examinees and the overlap between sets of examinees selected based on either
θ
,
θ
^
, or
NC
scores, and the bias in criterion-related validity estimates. Results show that the
θ
^
values were unbiased by violations of unidimensionality, but their precision decreased as multidimensionality and the proportion of misfitting items increased; the estimated item parameters were robust to violations of unidimensionality. The correlations between
θ
,
θ
^
, and
NC
scores, the agreement between the three selection criteria, and the accuracy of criterion-related validity estimates are all negatively affected, to some extent, by increasing levels of multidimensionality and the proportion of misfitting items. However, removing the misfitting items only improved the results in the case of severe multidimensionality and large proportion of misfitting items, and deteriorated them otherwise.
In first episode psychosis (FEP) baseline negative symptoms (BNS) and relapse both predict less favorable functional outcome. Relapse-prevention is one of the most important goals of treatment. Apart ...from discontinuation of antipsychotics, natural causes of relapse are unexplained. We hypothesized that BNS, apart from predicting worse functional outcome, might also increase relapse risk.
We performed a post-hoc analysis of 7-year follow-up data of a FEP cohort (n = 103) involved in a dose-reduction/discontinuation (DR) vs. maintenance treatment (MT) trial. We examined: 1) what predicted relapse, 2) what predicted functional outcome, and 3) if BNS predicted relapse, whether MT reduced relapse rates compared to DR. After remission patients were randomly assigned to DR or MT for 18 months. Thereafter, treatment was uncontrolled.
BNS and duration of untreated psychosis (DUP) predicted relapse. Number of relapses, BNS, and treatment strategy predicted functional outcome. BNS was the strongest predictor of relapse, while number of relapses was the strongest predictor of functional outcome above BNS and treatment strategy. Overall and within MT, but not within DR, more severe BNS predicted significantly higher relapse rates. Treatment strategies did not make a difference in relapse rates, regardless of BNS severity.
BNS not only predicted worse functional outcome, but also relapses during follow-up. Since current low dose maintenance treatment strategies did not prevent relapse proneness in patients with more severe BNS, resources should be deployed to find optimal treatment strategies for this particular group of patients.
In decision-making, it is important not only to use the correct information but also to combine information in an optimal way. There are robust research findings that a mechanical combination of ...information for personnel and educational selection matches or outperforms a holistic combination of information. However, practitioners and policy makers seldom use mechanical combination for decision-making. One of the important conditions for scientific results to be used in practice and to be part of policy-making is that results are easily accessible. To increase the accessibility of mechanical judgment prediction procedures, we (1) explain in detail how mechanical combination procedures work, (2) provide examples to illustrate these procedures, and (3) discuss some limitations of mechanical decision-making.
The flipped classroom is becoming more popular as a means to support student learning in higher education by requiring students to prepare before lectures and actively engaging students during ...lectures. While some research has been conducted into student performance in the flipped classroom, students' study behaviour throughout a flipped course has not been investigated. This study explored students' study behaviour throughout a flipped and a regular course by means of bi-weekly diaries. Furthermore, student references to their learning regulation were explored in course evaluations. Results from the diaries showed that students' study behaviour in the flipped course did not appear to be very different from that of students in a regular course. Furthermore, study behaviour did not appear strongly related to student performance in both the flipped and the regular course. Exploration of student references to their learning regulation in the course evaluations showed that some students experienced the flipped course design as intended to support their learning process. Other students, however, demonstrated resistance to changing their study behaviour even though changing study behaviour is expected in order to benefit from the flipped classroom. Further research on the relationship between students' learning regulation and actual study behaviour and course results is necessary to understand when and why implementing the flipped classroom is successful. Recommendations that may help more effective flipped classroom implementation include considering the prior history between students and instructor(s), the broader curriculum context, and frequent expectation communication especially with large numbers of students and non-mandatory lecture attendance.
Purpose
In Mokken scaling, the
Crit
index was proposed and is sometimes used as evidence (or lack thereof) of violations of some common model assumptions. The main goal of our study was twofold: To ...make the formulation of the
Crit
index explicit and accessible, and to investigate its distribution under various measurement conditions.
Methods
We conducted two simulation studies in the context of dichotomously scored item responses. We manipulated the type of assumption violation, the proportion of violating items, sample size, and quality. False positive rates and power to detect assumption violations were our main outcome variables. Furthermore, we used the
Crit
coefficient in a Mokken scale analysis to a set of responses to the General Health Questionnaire (GHQ-12), a self-administered questionnaire for assessing current mental health.
Results
We found that the false positive rates of
Crit
were close to the nominal rate in most conditions, and that power to detect misfit depended on the sample size, type of violation, and number of assumption-violating items. Overall, in small samples
Crit
lacked the power to detect misfit, and in larger samples power differed considerably depending on the type of violation and proportion of misfitting items. Furthermore, we also found in our empirical example that even in large samples the
Crit
index may fail to detect assumption violations.
Discussion
Even in large samples, the
Crit
coefficient showed limited usefulness for detecting moderate and severe violations of monotonicity. Our findings are relevant to researchers and practitioners who use Mokken scaling for scale and questionnaire construction and revision.
The accessibility to electronic devices and the novel statistical methodologies available have allowed researchers to comprehend psychological processes at the individual level. However, there are ...still great challenges to overcome as, in many cases, collected data are more complex than the available models are able to handle. For example, most methods assume that the variables in the time series are measured on an interval scale, which is not the case when Likert-scale items were used. Ignoring the scale of the variables can be problematic and bias the results. Additionally, most methods also assume that the time series are stationary, which is rarely the case. To tackle these disadvantages, we propose a model that combines the partial credit model (PCM) of the item response theory framework and the time-varying autoregressive model (TV-AR), which is a popular model used to study psychological dynamics. The proposed model is referred to as the time-varying dynamic partial credit model (TV-DPCM), which allows to appropriately analyze multivariate polytomous data and nonstationary time series. We test the performance and accuracy of the TV-DPCM in a simulation study. Lastly, by means of an example, we show how to fit the model to empirical data and interpret the results.
One of the goals of computerized tutoring systems is to optimize the learning of facts. Over a hundred years of declarative memory research have identified two robust effects that can improve such ...systems: the spacing and the testing effect. By making optimal use of both and adjusting the system to the individual learner using cognitive models based on declarative memory theories, such systems consistently outperform traditional methods (Van Rijn, Van Maanen, & Van Woudenberg, 2009). This adjustment process is driven by a continuously updated estimate of the rate of forgetting for each item and learner on the basis of the learner's accuracy and response time. In this study, we investigated to what extent these estimates of individual rates of forgetting are stable over time and across different materials. We demonstrate that they are stable over time but not across materials. Even though most theories of human declarative memory assume a single underlying rate of forgetting, we show that, in practice, it makes sense to assume different materials are forgotten at different rates. If a computerized, adaptive fact‐learning system allowed different rates of forgetting for different materials, it could adapt to individual learners more readily.
Robust scientific evidence shows that human performance predictions are more valid when information is combined mechanically (with a decision rule) rather than holistically (in the decision-maker's ...mind). Yet, information is often combined holistically in practice. One reason is that decision makers lack the knowledge of evidence-based decision making. In a performance prediction task, we tested whether watching an educational video on evidence-based decision making increased decision-makers' use of a decision rule and their prediction accuracy immediately after the manipulation and a month later. Furthermore, we manipulated whether participants earned incentives for accurate predictions. Existing research showed that incentives decrease decision-rule use and prediction accuracy. We hypothesized that this is the case for decision makers who did not receive educational information about evidence-based decision making, but that incentives increase decision-rule use and prediction accuracy for participants who received educational information. Our results showed that educational information increased decision-rule use. This resulted in increased prediction accuracy, but only immediately after receiving the educational information. In contrast to the existing literature, incentives slightly increased decision-rule use. We did not find evidence that this effect was larger for educated participants. Providing decision makers with educational information may be effective to increase decision-rule use in practice.
Public Significance Statement
Combining information with a decision rule results in more valid predictions than combining information holistically in the mind. Yet, decision makers rarely use decision rules in practice. This study suggests that a brief educational intervention can increase decision-makers' use of a decision rule in a human performance prediction task. Consequently, prediction accuracy increased, but only temporarily. Such an educational intervention is easily applicable and may increase evidence-based decision making in practice. But, interventions may need to be repeated for a lasting effect.
PDF
