Despite the increasing popularity of mixed-mode approaches to data collection, studies examining measurement equivalence across different survey modes in representative samples of the general ...population, particularly focusing on measures of socially sensitive psychological constructs, are sparse. In this study, we used data from a large representative sample of the Slovenian population (
N
= 9,900) collected as part of the third wave of the European Health Interview Survey (EHIS) to examine mode-inherent effects (i.e., measurement effects that cannot be neutralized by clever survey design) of the traditional interviewer-mediated face-to-face mode and the increasingly popular self-administered web mode on three measures of psychological functioning, namely the Patient Health Questionnaire Depression Scale (PHQ-8), the Satisfaction with Life Scale (SWLS), and the Mental Health Continuum – Short Form (MHC-SF). After controlling for self-selection propensities, our results showed strict measurement invariance for all three scales across the two survey modes, but statistically significant and notable differences in latent means, suggesting that individuals who respond face-to-face systematically report better psychological functioning than individuals who respond over the web. These results suggest significant mode inherent effects that cannot be attributed to measurement non-invariance of the scales between face-to-face and web survey modes, but most likely to social desirability bias in responses achieved in the presence of an interviewer. Administration mode effects must be considered when interpreting and comparing results obtained through different survey modes, particularly interviewer-mediated versus self-administered modes, especially when using measures of culturally desirable traits and behaviors, such as mental health and well-being.
Existing procedures for testing measurement invariance focus mainly on group-level comparisons rather than individual comparisons (i.e., the main conclusion typically concerns the question of ...comparability of group means). We propose a set of intuitive graphical displays called Person-Level Assessment of Measurement Invariance (PLAMI), which attempt to provide information about the effect of the lack of measurement invariance on the level of person scores. This information complements the group-level information contained in model fit indices and overall effect size indices, and should be especially useful for practitioners using tests for individual diagnostics. PLAMI uses results from the multiple-group factor analysis to estimate and visualize score distortions in terms of expected score difference between members of different groups and the probability of an unacceptably high score difference, conditional for the value of latent trait. The stability of the results can be evaluated by means of resampling. Simulation can be used for an a priori evaluation of the sample size adequacy. We illustrate the proposed plots on two real examples involving testing measurement invariance of the Machiavellianism scale, and we demonstrate the added value of PLAMI compared to the use of fit indices alone.
The purpose of this study was to examine measurement invariance of the Subjective Happiness Scale across countries, gender, and age groups and across time by multigroup confirmatory factor analysis. ...Altogether, 4,977 participants from nine European, American, and Australian countries were included in the study. Our results revealed that both configural and metric invariance held across countries, but scalar invariance was only partially confirmed with one item yielding varying intercepts in different countries. Measurement invariance was also confirmed across gender and age groups. Longitudinal measurement invariance was examined on a subsample of 478 English-speaking participants and was fully confirmed across five consecutive assessment points. Factor means were compared between groups and across time, and good convergent validity of the Subjective Happiness Scale was found in relation to a measure of temporal satisfaction with life. Overall, our results demonstrate that self-reported happiness was measured similarly in nine different countries, gender and age groups and over time, and provide a solid foundation for meaningful cross-group and cross-time comparisons in subjective happiness.
As a crisis response to the COVID-19 pandemic, many companies quickly established virtual leadership systems and enabled employees to continue their work from home. This cross-sectional research ...addresses virtual leadership efficacy assessed by the leaders and by their employees. The findings suggest that leaders evaluate themselves significantly better than their employees, and their leadership efficacy mainly depends on their previous experience of working from home and ability to use communication technologies. This research contributes to the understanding of the factors that have the biggest influence on the belief in leadership efficacy in the context of a rapidly evolving system of remote work.
Recently, a patient-reported experience measure (PREM) was developed in Slovenia to assess patients' experiences with outpatient specialist healthcare clinics. The aim of this study was to evaluate ...the psychometric properties (including factor structure, reliability, convergent validity, and response distribution) of the questionnaire.
The sample consisted of 8,406 adult participants treated in 171 specialist clinics from different medical fields. Participants voluntarily and anonymously responded to either the paper or online survey.
Descriptive statistics show meaningful response patterns with a general tendency towards favourable assessments. The psychometric analyses of the scales evaluating doctor's and nurses' work, respectively, generally showed a good fit of the unidimensional factor model as well as the Rasch model, high factor loadings and very good to satisfactory reliability. The Rasch scaling showed that these scales were most informative for patients with relatively unfavourable experience ratings.
The results are similar to those found in previous evaluations of PREMs in other countries. Given its good psychometric properties, the Slovenian PREM can be recommended for healthcare evaluations in Slovenia and as a model for the development of similar PREMs in other countries.
A sense of community plays an important role in promoting sustainable behaviors and values. Therefore, when measuring sense of community, it is important to ensure that such measurement instruments ...have excellent psychometric properties. In this study, we start from the model that sense of community is related to a person’s perception of his or her relationship with a community, particularly in terms of need fulfillment, sense of connectedness and emotional connection, and perceived impact on the community. The Brief Sense of Community Scale (BSCS) is an 8-item self-report scale designed specifically for the assessment of individuals’ relation with their neighborhood. The aim of this study was to investigate the factor structure and reliability of a new Slovenian version. A sample of 502 adults from Slovenia completed the scale. The data did not support the original hierarchical four-factor model. On the other hand, a one-factor model with autoregressive effects on the unique factors fitted well. The reliability of the sum score was excellent, while only two out of the four subscales had acceptable reliability. The results support the use of the Slovenian version of the BSCS as a unidimensional scale and allow the use of this measure in research on sense of community and possibly as a rigorous measure in further studies of sustainable development in different cultures.
When principal component solutions are compared across
two groups, a question arises whether the extracted components have the same
interpretation in both populations. The problem can be approached ...by testing
null hypotheses stating that the congruence coefficients between pairs of
vectors of component loadings are equal to 1. Chan, Leung, Chan, Ho, and Yung (1999) proposed a bootstrap
procedure for testing the hypothesis of perfect congruence between vectors of
common factor loadings. We demonstrate that the procedure by Chan et al. is both
theoretically and empirically inadequate for the application on principal
components. We propose a modification of their procedure, which constructs the
resampling space according to the characteristics of the principal component
model. The results of a simulation study show satisfactory empirical properties
of the modified procedure.
Residential satisfaction is a topic that has been extensively studied in recent decades because it can offer important insights into the quality of the residential environment. However, many ...inconsistencies and unanswered questions on this topic still persist. Because the understanding of any field of inquiry is importantly affected by the quality of the methodology and measurement instruments employed, this article explores the current state of development and investigation of the psychometric properties of one of the most widely employed methods of measuring residential satisfaction: self-assessment questionnaires that measure satisfaction by assessing satisfaction with specific aspects of the residential environment. A review of representative studies shows a general lack of properly developed and validated questionnaires, lack of sufficient reporting on the origin, development, and psychometric characteristics of the questionnaires employed, and often too little thought and effort invested in developing and validating questionnaires. Such observations are especially important for evaluating the quality of studies and their implications for residential satisfaction, and they are the points where research practice could be improved.
The aim of this longitudinal study, carried out on a sample of Slovenian-speaking toddlers, was to analyze developmental changes and stability in early vocabulary development; to establish relations ...between toddler's vocabulary and grammar; and to analyze the effects of parental education and the frequency of shared reading on toddlers' vocabulary and grammar. The sample included fifty-one toddlers, aged 1;4 at the time of the first, and 2;7 at the time of the last, assessment. Toddlers' vocabulary and grammar were assessed six times during a 15-month period using the Slovenian adaptation of the CDI. Our findings suggest great individual differences in both size and rate of toddlers' vocabulary development. Toddlers' vocabulary scores remained relatively stable across a 3-month period. Early vocabulary at 1;7 predicted vocabulary, sentence complexity, and mean length of utterance (MLU) at 2;7, while the frequency of shared reading mediated the effect of parental education on toddlers' vocabulary and grammar at 2;7.
In scoring of a multiple-choice test, the number of correct answers does not use all information available from item responses. Scoring such tests by applying empirically determined weights to the ...chosen options should provide more information on examinees' knowledge and consequently produce more valid test scores. However, existing empirical evidence on this topic does not clearly support option weighting. To overcome the limitations of the previous studies, we performed a simulation study where we manipulated the instruction to examinees, discrimination structure of distractors, test length, and sample size. We compared validity and internal consistency of number-correct scores, corrected-for-guessing scores, two variants of correlation-weighted scores and homogeneity analysis scores. The results suggest that in certain conditions the correlation-weighted scores are notably more valid than the number-correct scores. On the other hand, homogeneity analysis cannot be recommended as a scoring method. The relative performance of scoring methods strongly depends on the instructions and on distractors' properties, and only to a lesser extent on sample size and test length.
PDF
