...there is growing recognition that the relationship between accelerometer counts and energy expenditure is highly dependent on the activities included the calibration study; and that cut-points ...derived from a single regression model or Receiver Operating Characteristic curve cannot adequately characterise physical activity intensity across a wide range of physical activities 13. ...it is important to note that the Evenson cut-points still misclassified MVPA as light-intensity physical activity 20% of the time, and that light intensity physical activities were misclassified as sedentary at least 40% of the time 12. ...given that the relationship between activity counts and energy expenditure in children under five differs substantially to that observed in adolescent youth 14, the application of the Evenson cut-points in children aged 2- to 5-years by Steene-Johannessen must be questioned. When applied to youth, machine learning approaches have shown to provide more accurate predictions of physical activity intensity 13, 16. ...to cut-point methods, which only estimate time spend in MVPA, physical activity classification models can predict time spent in specific activity types (e.g., walking, running, dancing, cycling) or broader activity classes (e.g., active games or sports) 13, 16. Because hip mounted accelerometers are typically placed on snug fitting elastic belts and worn over clothing, non-compliance and insufficient wear time were frequent problems in these studies.
Machine learning (ML) accelerometer data processing methods have potential to improve the accuracy of device-based assessments of physical activity (PA) in young children. Yet the uptake of ML ...methods by health researchers has been minimal and the use of cut-points (CP) continues to be the norm, despite evidence of significant misclassification error. The lack of studies demonstrating a relative advantage for ML approaches over CP methods maybe a key contributing factor.
The current study compared the accuracy of PA intensity predictions provided by ML classification models and previously published CPs for preschool-aged children.
In a free-living study, 31 preschool-aged children (mean age = 4.0 ± 0.9 y) wore wrist and hip ActiGraph GT3X+ accelerometers while completing a video recorded 20-minute free play session. Ground truth PA intensity was coded continuously using the Children's Activity Rating Scale (CARS). Accelerometer data was classified as sedentary (SED), light intensity (LPA), or moderate-to-vigorous intensity (MVPA) using ML random forest PA classifiers and published CPs for preschool-aged children. Performance differences were evaluated in a hold-out sample by comparing weighted kappa statistics, classification accuracy for each intensity band, and equivalence testing.
ML classification models (hip: κ = 0.76; wrist: κ = 0.72) exhibited significantly higher agreement with ground truth PA intensity than CP methods (hip: κ = 0.38-0.49; wrist: κ = 0.31-0.44). For the ML models, classification accuracy for SED and LPA ranged from 83% - 88%, while classification accuracy for MVPA ranged from 68% - 78%. For the CP's, classification accuracy ranged from 50% - 94% for SED, 19% - 75% for LPA, and 44% - 76.1% for MVPA. ML classification models showed equivalence (within ± 0.5 SD) with directly observed time in SED, LPA, and MVPA. None of the CP's exhibited evidence of equivalence.
Under free living conditions, ML classification models for hip or wrist accelerometer data provide more accurate assessments of PA intensity in young children than CP methods. The results demonstrate the relative advantage of ML methods over threshold-based approaches and adds to a growing evidence base supporting the feasibility and accuracy of ML accelerometer data processing methods.
In this study, we evaluated agreement among three generations of ActiGraph™ accelerometers in children and adolescents. Twenty-nine participants (mean age = 14.2 ± 3.0 years) completed two ...laboratory-based activity sessions, each lasting 60 min. During each session, participants concurrently wore three different models of the ActiGraph™ accelerometers (GT1M, GT3X, GT3X+). Agreement among the three models for vertical axis counts, vector magnitude counts, and time spent in moderate-to-vigorous physical exercise (MVPA) was evaluated by calculating intraclass correlation coefficients and Bland-Altman plots. The intraclass correlation coefficient for total vertical axis counts, total vector magnitude counts, and estimated MVPA was 0.994 (95% CI = 0.989-0.996), 0.981 (95% CI = 0.969-0.989), and 0.996 (95% CI = 0.989-0.998), respectively. Inter-monitor differences for total vertical axis and vector magnitude counts ranged from 0.3% to 1.5%, while inter-monitor differences for estimated MVPA were equal to or close to zero. On the basis of these findings, we conclude that there is strong agreement between the GT1M, GT3X, and GT3X+ activity monitors, thus making it acceptable for researchers and practitioners to use different ActiGraph™ models within a given study.
The absence of comparative validity studies has prevented researchers from reaching consensus regarding the application of intensity-related accelerometer cut points for children and adolescents.
...This study aimed to evaluate the classification accuracy of five sets of independently developed ActiGraph cut points using energy expenditure, measured by indirect calorimetry, as a criterion reference standard.
A total of 206 participants between the ages of 5 and 15 yr completed 12 standardized activity trials. Trials consisted of sedentary activities (lying down, writing, computer game), lifestyle activities (sweeping, laundry, throw and catch, aerobics, basketball), and ambulatory activities (comfortable walk, brisk walk, brisk treadmill walk, running). During each trial, participants wore an ActiGraph GT1M, and V˙O2 was measured breath-by-breath using the Oxycon Mobile portable metabolic system. Physical activity intensity was estimated using five independently developed cut points: Freedson/Trost (FT), Puyau (PU), Treuth (TR), Mattocks (MT), and Evenson (EV). Classification accuracy was evaluated via weighted κ statistics and area under the receiver operating characteristic curve (ROC-AUC).
Across all four intensity levels, the EV (κ=0.68) and FT (κ=0.66) cut points exhibited significantly better agreement than TR (κ=0.62), MT (κ=0.54), and PU (κ=0.36). The EV and FT cut points exhibited significantly better classification accuracy for moderate- to vigorous-intensity physical activity (ROC-AUC=0.90) than TR, PU, or MT cut points (ROC-AUC=0.77-0.85). Only the EV cut points provided acceptable classification accuracy for all four levels of physical activity intensity and performed well among children of all ages. The widely applied sedentary cut point of 100 counts per minute exhibited excellent classification accuracy (ROC-AUC=0.90).
On the basis of these findings, we recommend that researchers use the EV ActiGraph cut points to estimate time spent in sedentary, light-, moderate-, and vigorous-intensity activity in children and adolescents.
Machine learning (ML) activity classification models trained on laboratory-based activity trials exhibit low accuracy under free-living conditions. Training new models on free-living accelerometer ...data, reducing the number of prediction windows comprised of multiple activity types by using shorter windows, including temporal features such as standard deviation in lag and lead windows, and using multiple sensors may improve the classification accuracy under free-living conditions. The objective of this study was to evaluate the accuracy of Random Forest (RF) activity classification models for preschool-aged children trained on free-living accelerometer data. Thirty-one children (mean age = 4.0 ± 0.9 years) completed a 20 min free-play session while wearing an accelerometer on their right hip and non-dominant wrist. Video-based direct observation was used to categorize the children's movement behaviors into five activity classes. The models were trained using prediction windows of 1, 5, 10, and 15 s, with and without temporal features. The models were evaluated using leave-one-subject-out-cross-validation. The F-scores improved as the window size increased from 1 to 15 s (62.6%-86.4%), with only minimal improvements beyond the 10 s windows. The inclusion of temporal features increased the accuracy, mainly for the wrist classification models, by an average of 6.2 percentage points. The hip and combined hip and wrist classification models provided comparable accuracy; however, both the models outperformed the models trained on wrist data by 7.9 to 8.2 percentage points. RF activity classification models trained with free-living accelerometer data provide accurate recognition of young children's movement behaviors under real-world conditions.
Summary
The impact of screen‐based devices on children's health and development cannot be properly understood without valid and reliable tools that measure screen time within the evolving digital ...landscape. This review aimed to summarize characteristics of measurement tools used to assess screen time in young children; evaluate reporting of psychometric properties; and examine time trends related to measurement and reporting of screen time. A systematic review of articles published in English across three databases from January 2009 to April 2020 was undertaken using PROSPERO protocol (registration: CRD42019132599) and Preferred Reporting Items for Systematic Reviews and Meta‐Analyses (PRISMA) guidelines. Included articles measured screen time as outcome, exposure, or confounder in children 0–6 years. The search identified 35,868 records, 1035 full‐text articles were screened for eligibility, and 622 met inclusion criteria. Most measures (60%) consisted of one to three items and assessed duration of screen time on a usual day. Few measures assessed content (11%) or coviewing (7%). Only 40% of articles provided a citation for the measure, and only 69 (11%) reported psychometric properties—reliability n = 58, validity n = 19, reliability and validity n = 8. Between 2009 and 2019, the number of published articles increased from 28 to 71. From 2015, there was a notable increase in the proportion of articles published each year that assessed exposure to mobile devices in addition to television. The increasing number of published articles reflects increasing interest in screen time exposure among young children. Measures of screen time have generally evolved to reflect children's contemporary digital landscape; however, the psychometric properties of measurement tools are rarely reported. There is a need for improved measures and reporting to capture the complexity of children's screen time exposures.
Wrist-worn accelerometers are associated with greater compliance. However, validated algorithms for predicting activity type from wrist-worn accelerometer data are lacking. This study compared the ...activity recognition rates of an activity classifier trained on acceleration signal collected on the wrist and hip.
52 children and adolescents (mean age 13.7 ± 3.1 year) completed 12 activity trials that were categorized into 7 activity classes: lying down, sitting, standing, walking, running, basketball, and dancing. During each trial, participants wore an ActiGraph GT3X+ tri-axial accelerometer on the right hip and the non-dominant wrist. Features were extracted from 10-s windows and inputted into a regularized logistic regression model using R (Glmnet + L1).
Classification accuracy for the hip and wrist was 91.0% ± 3.1% and 88.4% ± 3.0%, respectively. The hip model exhibited excellent classification accuracy for sitting (91.3%), standing (95.8%), walking (95.8%), and running (96.8%); acceptable classification accuracy for lying down (88.3%) and basketball (81.9%); and modest accuracy for dance (64.1%). The wrist model exhibited excellent classification accuracy for sitting (93.0%), standing (91.7%), and walking (95.8%); acceptable classification accuracy for basketball (86.0%); and modest accuracy for running (78.8%), lying down (74.6%) and dance (69.4%).Potential Impact: Both the hip and wrist algorithms achieved acceptable classification accuracy, allowing researchers to use either placement for activity recognition.
PURPOSE:The purpose of this review is to address important methodological issues related to conducting accelerometer-based assessments of physical activity in free-living individuals.
METHODS:We ...review the extant scientific literature for empirical information related to the following issuesproduct selection, number of accelerometers needed, placement of accelerometers, epoch length, and days of monitoring required to estimate habitual physical activity. We also discuss the various options related to distributing and collecting monitors and strategies to enhance compliance with the monitoring protocol.
RESULTS:No definitive evidence exists currently to indicate that one make and model of accelerometer is more valid and reliable than another. Selection of accelerometer therefore remains primarily an issue of practicality, technical support, and comparability with other studies. Studies employing multiple accelerometers to estimate energy expenditure report only marginal improvements in explanatory power. Accelerometers are best placed on hip or the lower back. Although the issue of epoch length has not been studied in adults, the use of count cut points based on 1-min time intervals maybe inappropriate in children and may result in underestimation of physical activity. Among adults, 3–5 d of monitoring is required to reliably estimate habitual physical activity. Among children and adolescents, the number of monitoring days required ranges from 4 to 9 d, making it difficult to draw a definitive conclusion for this population. Face-to-face distribution and collection of accelerometers is probably the best option in field-based research, but delivery and return by express carrier or registered mail is a viable option.
CONCLUSION:Accelerometer-based activity assessments requires careful planning and the use of appropriate strategies to increase compliance.
To evaluate the accuracy of LAB EE prediction models in preschool children completing a free-living active play session. Performance was benchmarked against EE prediction models trained on free ...living (FL) data.
25 children (mean age = 4.1±1.0 y) completed a 20-minute active play session while wearing a portable indirect calorimeter and ActiGraph GT3X+ accelerometers on their right hip and non-dominant wrist. EE was predicted using LAB models which included Random Forest (RF) and Support Vector Machine (SVM) models for the wrist, and RF and Artificial Neural Network (ANN) models for the hip. Two variations of the LAB models were evaluated; 1) an "off the shelf" model without additional training; 2) models retrained on free-living data, replicating the methodology used in the original calibration study (retrained LAB). Prediction errors were evaluated in a hold-out sample of 10 children.
Root mean square error (RMSE) for the FL and retrained LAB models ranged from 0.63-0.67 kcals/min. In the hold out sample, RMSE's for the hip LAB (0.62-0.71), retrained LAB (0.58-0.62) and FL models (0.61-0.65) were similar. For the wrist placement, FL SVM had a significantly higher RMSE (0.73 ± 0.29 kcals/min) than the retrained LAB SVM (0.63 ± 0.30 kcals/min) and LAB SVM (0.64 ± 0.18 kcals/min). The LAB (0.64 ± 0.28), retrained LAB (0.64 ± 0.25), and FL (0.62 ± 0.26) RF exhibited comparable accuracy.
Machine learning EE prediction models trained on LAB and FL data had similar accuracy under free-living conditions.
Rapid developments in technology have encouraged the use of smartphones in physical activity research, although little is known regarding their effectiveness as measurement and intervention tools.
...This study systematically reviewed evidence on smartphones and their viability for measuring and influencing physical activity.
Research articles were identified in September 2013 by literature searches in Web of Knowledge, PubMed, PsycINFO, EBSCO, and ScienceDirect.
The search was restricted using the terms (physical activity OR exercise OR fitness) AND (smartphone* OR mobile phone* OR cell phone*) AND (measurement OR intervention). Reviewed articles were required to be published in international academic peer-reviewed journals, or in full text from international scientific conferences, and focused on measuring physical activity through smartphone processing data and influencing people to be more active through smartphone applications.
Two reviewers independently performed the selection of articles and examined titles and abstracts to exclude those out of scope. Data on study characteristics, technologies used to objectively measure physical activity, strategies applied to influence activity; and the main study findings were extracted and reported.
A total of 26 articles (with the first published in 2007) met inclusion criteria. All studies were conducted in highly economically advantaged countries; 12 articles focused on special populations (e.g. obese patients). Studies measured physical activity using native mobile features, and/or an external device linked to an application. Measurement accuracy ranged from 52 to 100% (n = 10 studies). A total of 17 articles implemented and evaluated an intervention. Smartphone strategies to influence physical activity tended to be ad hoc, rather than theory-based approaches; physical activity profiles, goal setting, real-time feedback, social support networking, and online expert consultation were identified as the most useful strategies to encourage physical activity change. Only five studies assessed physical activity intervention effects; all used step counts as the outcome measure. Four studies (three pre-post and one comparative) reported physical activity increases (12-42 participants, 800-1,104 steps/day, 2 weeks-6 months), and one case-control study reported physical activity maintenance (n = 200 participants; >10,000 steps/day) over 3 months.
Smartphone use is a relatively new field of study in physical activity research, and consequently the evidence base is emerging.
Few studies identified in this review considered the validity of phone-based assessment of physical activity. Those that did report on measurement properties found average-to-excellent levels of accuracy for different behaviors. The range of novel and engaging intervention strategies used by smartphones, and user perceptions on their usefulness and viability, highlights the potential such technology has for physical activity promotion. However, intervention effects reported in the extant literature are modest at best, and future studies need to utilize randomized controlled trial research designs, larger sample sizes, and longer study periods to better explore the physical activity measurement and intervention capabilities of smartphones.
PDF
