Preliminary evaluations of behavioral interventions, referred to as pilot studies, predate the conduct of many large-scale efficacy/effectiveness trial. The ability of a pilot study to inform an ...efficacy/effectiveness trial relies on careful considerations in the design, delivery, and interpretation of the pilot results to avoid exaggerated early discoveries that may lead to subsequent failed efficacy/effectiveness trials. "Risk of generalizability biases (RGB)" in pilot studies may reduce the probability of replicating results in a larger efficacy/effectiveness trial. We aimed to generate an operational list of potential RGBs and to evaluate their impact in pairs of published pilot studies and larger, more well-powered trial on the topic of childhood obesity.
We conducted a systematic literature review to identify published pilot studies that had a published larger-scale trial of the same or similar intervention. Searches were updated and completed through December 31st, 2018. Eligible studies were behavioral interventions involving youth (≤18 yrs) on a topic related to childhood obesity (e.g., prevention/treatment, weight reduction, physical activity, diet, sleep, screen time/sedentary behavior). Extracted information included study characteristics and all outcomes. A list of 9 RGBs were defined and coded: intervention intensity bias, implementation support bias, delivery agent bias, target audience bias, duration bias, setting bias, measurement bias, directional conclusion bias, and outcome bias. Three reviewers independently coded for the presence of RGBs. Multi-level random effects meta-analyses were performed to investigate the association of the biases to study outcomes.
A total of 39 pilot and larger trial pairs were identified. The frequency of the biases varied: delivery agent bias (19/39 pairs), duration bias (15/39), implementation support bias (13/39), outcome bias (6/39), measurement bias (4/39), directional conclusion bias (3/39), target audience bias (3/39), intervention intensity bias (1/39), and setting bias (0/39). In meta-analyses, delivery agent, implementation support, duration, and measurement bias were associated with an attenuation of the effect size of - 0.325 (95CI - 0.556 to - 0.094), - 0.346 (- 0.640 to - 0.052), - 0.342 (- 0.498 to - 0.187), and - 0.360 (- 0.631 to - 0.089), respectively.
Pre-emptive avoidance of RGBs during the initial testing of an intervention may diminish the voltage drop between pilot and larger efficacy/effectiveness trials and enhance the odds of successful translation.
Abstract
Study Objectives
To examine changes in elementary aged children’s sleep and physical activity during a 1-week and a 3-week school break.
Methods
Sleep and physical activity of elementary ...children (n = 154, age = 5–9 years, 44.8% female, 65.5% African American) were collected over 7 weeks that included a 1-week break in two schools and a 3-week break in a single school. Mixed regression models estimated sleep and physical activity changes within and between groups (i.e. 1-week vs. 3-weeks) during school and school break weeks.
Results
Compared to school weeks, bed times shifted 72.7 (95% CI = 57.5, 87.9) and 75.4 (95% CI = 58.1, 92.7) minutes later on weekdays during the 1-week and 3-week break, respectively. Wake times shifted 111.6 (95% CI = 94.3, 128.9) and 99.8 (95% CI = 80.5, 119.1) minutes later on weekdays during 1-week and 3-week breaks. On weekdays during the 3-week break, children engaged in 33.1 (95% CI = 14.1, 52.2) more sedentary minutes and −12.2 (−20.2, −4.2) fewer moderate-to-vigorous physical activity minutes/day. No statistically significant changes in children’s sedentary, light, or moderate-to-vigorous physical activity (MVPA) minutes were observed on weekdays during the 1-week break. Between-group differences in the change in time sedentary (32.1—95% CI = 5.8, 58.4), and moderate-to-vigorous (−13.0—95% CI = −23.9, −2.0) physical activity were observed.
Conclusions
Children’s sleep shifted later on both 1-week and 3-week breaks. Children’s activity changed minimally on weekdays during a 1-week school break and more during a 3-week school break. Displaced sleep and reductions in activity are intervention targets for mitigating unhealthy weight gain during extended breaks from school.
Shade coffee farms in southwest Ethiopia are known to host high levels of avian biodiversity. However, these farms vary in terms of forest management, which affects their understory, mid-story, crown ...cover, and canopy closure, and hence their structural complexity. Such differences in vegetation structure can potentially affect the survival of specialist bird species, and shade coffee farms may not equally contribute to avian biodiversity conservation. This study aimed to investigate how avian community composition, richness, and the relative abundance of different bird functional guilds relate to structural differences in vegetation shaped by forest management. Bird guild classification was based on bird species forest dependence, diet type, migration status, nest type, foraging, and nesting strata, and bird communities were surveyed using the Timed Species Counts (TSCs) method. Species turnover in bird communities was evaluated using detrended correspondence analysis and redundancy analysis, whereby multiple regression models were used to examine bird guild responses to vegetation structure. Total bird species richness and relative abundance did not respond to vegetation structure. However, the richness of forest specialists and understory foragers, and the relative abundance of mid-high foragers, all positively related to tree diameter at breast height (DBH) and crown cover, whereas the relative abundance of species with medium levels of forest dependency, mid-high/canopy foragers, and open-nesters were positively related to basal area and canopy cover. This study demonstrates that the relative value of shade coffee farms for avian biodiversity conservation depends on the type of forest management, and that bigger trees with larger crown cover provide a habitat of higher quality to habitat specialist birds.
Background: Clinical decision support (CDS) for primary care has been shown to improve delivery of preventive services. However, there is little evidence for efficiency of physicians due to CDS ...assistance. In this article, we report a pilot study for measuring the impact of CDS on the time spent by physicians for deciding on preventive services and chronic disease management. Methods: We randomly selected 30 patients from a primary care practice, and assigned them to 10 physicians. The physicians were requested to perform chart review to decide on preventive services and chronic disease management for the assigned patients. The patients assignment was done in a randomized crossover design, such that each patient received 2 sets of recommendations—one from a physician with CDS assistance and the other from a different physician without CDS assistance. We compared the physician recommendations made using CDS assistance, with the recommendations made without CDS assistance. Results: The physicians required an average of 1 minute 44 seconds, when they were they had access to the decision support system and 5 minutes when they were unassisted. Hence the CDS assistance resulted in an estimated saving of 3 minutes 16 seconds (65%) of the physicians’ time, which was statistically significant (P < .0001). There was no statistically significant difference in the number of recommendations. Conclusion: Our findings suggest that CDS assistance significantly reduced the time spent by physicians for deciding on preventive services and chronic disease management. The result needs to be confirmed by performing similar studies at other institutions.
ABSTRACT
BACKGROUND
Critical reflection by faculty physicians on adverse patient events is important for changing physician’s behaviors. However, there is little research regarding physician ...reflection on quality improvement (QI).
OBJECTIVE
To develop and validate a computerized case-based learning system (CBLS) to measure faculty physicians’ reflections on adverse patient events.
DESIGN
Prospective validation study.
PARTICIPANTS
Staff physicians in the Department of Medicine at Mayo Clinic Rochester.
MAIN MEASURES
The CBLS was developed by Mayo Clinic information technology, medical education, and QI specialists. The reflection questionnaire, adapted from a previously validated instrument, contained eight items structured on five-point scales. Three cases, representing actual adverse events, were developed based on the most common error types: systems, medication, and diagnostic. In 2009, all Mayo Clinic hospital medicine, non-interventional cardiology, and pulmonary faculty were invited to participate. Faculty reviewed each case, determined the next management step, rated case generalizability and relevance, and completed the reflection questionnaire. Factor analysis and internal consistency reliability were calculated. Associations between reflection scores and characteristics of faculty and patient cases were determined.
KEY RESULTS
Forty-four faculty completed 107 case reflections. The CBLS was rated as average to excellent in 95 of 104 (91.3%) completed satisfaction surveys. Factor analysis revealed two levels of reflection:
Minimal
and
High
. Internal consistency reliability was very good (overall Cronbach’s α = 0.77). Item mean scores ranged from 2.89 to 3.73 on a five-point scale. The overall reflection score was 3.41 (standard deviation 0.64). Reflection scores were positively associated with case generalizability (p = 0.001), and case relevance (p = 0.02).
CONCLUSIONS
The CBLS is a valid method for stratifying faculty physicians’ levels of reflection on adverse patient events. Reflection scores are associated with case generalizability and relevance, indicating that reflection improves with pertinent patient encounters. We anticipate that this instrument will be useful in future research on QI among low versus high-reflecting physicians.
Rationale, aims and objectives The United States Preventive Services Task Force recommends a one‐time screening for abdominal aortic aneurysm (AAA) with ultrasonography for men aged 65 to 75 years ...who have ever smoked. However, despite a mortality rate of up to 80% for ruptured AAAs, providers order the screening for a minority of patients. We sought to determine the effect of a Web‐based point‐of‐care clinical decision support system on AAA screening rates in a primary care practice.
Methods We conducted a retrospective review of medical records of male patients aged 65 to 75 years who were seen at any of our practice sites in 2007 and 2008, before and after implementation of the clinical decision support system.
Results Overall screening rates were 31.36% in 2007 and 44.09% in 2008 (P‐value: <0.001). Of patients who had not had AAA screening prior to the visit, 3.22% completed the screening after the visit in 2007, compared with 18.24% in 2008 when the clinical support system was implemented, 5.36 times improvement (P‐value: <0.001).
Conclusions A Web‐based clinical decision support for primary care physicians significantly improved delivery of AAA screening of eligible patients. Carefully developed clinical decision support systems can optimize care delivery, ensuring that important preventive services are delivered to eligible patients.
The Together Everyone Achieves More Physical Activity (TEAM-PA) trial is a randomized controlled trial testing the efficacy of a group-based intervention for increasing physical activity (PA) among ...insufficiently active African American women.
The TEAM-PA trial uses a group cohort design, is implemented at community sites, and will involve 360 African American women. The trial compares a 10-week group-based intervention vs. a standard group-delivered PA comparison program. Measures include minutes of total PA/day using 7-day accelerometer estimates (primary outcome), and body mass index, blood pressure, waist circumference, walking speed, sedentary behavior, light physical activity, and the percentage achieving ≥150 min of moderate to vigorous PA/week (secondary outcomes) at baseline, post-intervention, and 6-months post-intervention.
The intervention integrates elements from Social Cognitive Theory, Self-Determination Theory, Group Dynamics Theory, and a focus on collectivism to evaluate different components of social affiliation (relatedness, reciprocal support, group cohesion, and collective efficacy). The intervention integrates shared goal-setting via Fitbits, group-based problem-solving, peer-to-peer positive communication, friendly competition, and cultural topics related to collectivism. Compared to the standard group-delivered PA program, participants in the intervention are expected to show greater improvements from baseline to post- and 6-month follow-up on minutes of total PA/day and secondary outcomes. Social affiliation variables (vs. individual-level factors) will be evaluated as mediators of the treatment effect.
The results of the TEAM-PA trial will determine the efficacy of the intervention and identify which aspects of social affiliation are most strongly related to increased PA among African American women.
This study was registered on Clinicaltrials.gov (# NCT05519696) in August 2022 prior to initial participant enrollment.
For-cause physical activity events reach many people. Little research has applied Self-Determination Theory to participants’ experiences in for-cause physical activity events. This qualitative study ...explored participants’ (n = 18) experiences in 5K for-cause physical activity events and intention to complete future events. Interviews were recorded and transcribed verbatim. Emergent coding assessed responses for themes. Constructs of competence and relatedness were most prevalent and an altruistic desire to support the cause. Participants highlighted feelings of community supporting their intention to complete future events. Overall, experiences aligned with Self-Determination Theory. Future studies may include altruism to understand leveraging opportunities for promoting physical activity.
Abstract
The implementation of evidence-based public health programs into practice is critical for improving health, but trainings for organizational change agents are often not scalable.
To describe ...the process of converting a training that targets faith-based organizational capacity development from an in-person to an online format.
We engaged in an iterative process to convert the training delivery mode from in-person to online that included assessing stakeholder support, consulting the literature on best practices, seeking a design team, consolidating content, designing engaging lessons, and building an online site. Feedback from end-users and other audiences was incorporated throughout. Pilot participants with characteristics like intended training users were then recruited via community and faith-based partner networks. They rated their agreement with statements about the effectiveness as well as design and functionality of each lesson and the overall training (1 = strongly disagree, 5 = strongly agree) and participated in a structured follow-up interview.
Nine pilot participants (representing 9 churches in 7 states; 6 African American, 5 with health ministries) rated the online lessons favorably (all ratings ≥ 4.5). Most (90.4%) perceived the lesson duration to be “just right” and spent 52.5 ± 9.9 minutes/lesson. Participants evaluated the overall training positively (all ratings ≥ 4.7). Lesson content, resources, multimedia, and program ideas were most-liked aspects of lessons, while content, staff responsiveness, discussion board, and pace were most-liked aspects of the overall training in open-ended and interview responses.
This paper shares a replicable process for converting training modalities from in-person to online with the goal of increased scalability.
PDF
