Eye behaviour provides valuable information revealing one's higher cognitive functions and state of affect. Although eye tracking is gaining ground in the research community, it is not yet a popular ...approach for the detection of emotional and cognitive states. In this paper, we present a review of eye and pupil tracking related metrics (such as gaze, fixations, saccades, blinks, pupil size variation, etc.) utilized towards the detection of emotional and cognitive processes, focusing on visual attention, emotional arousal and cognitive workload. Besides, we investigate their involvement as well as the computational recognition methods employed for the reliable emotional and cognitive assessment. The publicly available datasets employed in relevant research efforts were collected and their specifications and other pertinent details are described. The multimodal approaches which combine eye-tracking features with other modalities (e.g. biosignals), along with artificial intelligence and machine learning techniques were also surveyed in terms of their recognition/classification accuracy. The limitations, current open research problems and prospective future research directions were discussed for the usage of eye-tracking as the primary sensor modality. This study aims to comprehensively present the most robust and significant eye/pupil metrics based on available literature towards the development of a robust emotional or cognitive computational model.
Gait analysis is crucial for the detection and management of various neurological and musculoskeletal disorders. The identification of gait events is valuable for enhancing gait analysis, developing ...accurate monitoring systems, and evaluating treatments for pathological gait. The aim of this work is to introduce the Smart-Insole Dataset to be used for the development and evaluation of computational methods focusing on gait analysis. Towards this objective, temporal and spatial characteristics of gait have been estimated as the first insight of pathology. The Smart-Insole dataset includes data derived from pressure sensor insoles, while 29 participants (healthy adults, elderly, Parkinson's disease patients) performed two different sets of tests: The Walk Straight and Turn test, and a modified version of the Timed Up and Go test. A neurologist specialized in movement disorders evaluated the performance of the participants by rating four items of the MDS-Unified Parkinson's Disease Rating Scale. The annotation of the dataset was performed by a team of experienced computer scientists, manually and using a gait event detection algorithm. The results evidence the discrimination between the different groups, and the verification of established assumptions regarding gait characteristics of the elderly and patients suffering from Parkinson's disease.
•The dataset combines both objective and subjective performance evaluation.•Wide variety of eye tracking data, high number of participants.•Statistically significant correlations of eye features with ...cognitive workload tasks.•Encouraging results regarding cognitive workload level classification based on eye tracking data collected.•Publicly available to the academic community.
Background and Objective: The cognitive workload is an important component in performance psychology, ergonomics, and human factors. Publicly available datasets are scarce, making it difficult to establish new approaches and comparative studies. In this work, COLET-COgnitive workLoad estimation based on Eye-Tracking dataset is presented. Methods: Forty-seven (47) individuals’ eye movements were monitored as they solved puzzles involving visual search activities of varying complexity and duration. The participants’ cognitive workload level was evaluated with the subjective test of NASA-TLX and this score is used as an annotation of the activity. Extensive data analysis was performed in order to derive eye and gaze features from low-level eye recorded metrics, and a range of machine learning models were evaluated and tested regarding the estimation of the cognitive workload level. Results: The activities induced four different levels of cognitive workload. Multi tasking and time pressure have induced a higher level of cognitive workload than the one induced by single tasking and absence of time pressure. Multi tasking had a significant effect on 17 eye features while time pressure had a significant effect on 7 eye features. Both binary and multi-class identification attempts were performed by testing a variety of well-known classifiers, resulting in encouraging results towards cognitive workload levels estimation, with up to 88% correct predictions between low and high cognitive workload. Conclusions: Machine learning analysis demonstrated potential in discriminating cognitive workload levels using only eye-tracking characteristics. The proposed dataset includes a much higher sample size and a wider spectrum of eye and gaze metrics than other similar datasets, allowing for the examination of their relations with various cognitive states.
Parkinson's disease (PD) is one of the most prevalent neurological diseases, described by complex clinical phenotypes. The manifestations of PD include both motor and non-motor symptoms. We ...constituted an experimental protocol for the assessment of PD motor signs of lower extremities. Using a pair of sensor insoles, data were recorded from PD patients, Elderly and Adult groups. Assessment of PD patients has been performed by neurologists specialized in movement disorders using the Movement Disorder Society-Unified Parkinson's Disease Rating Scale (MDS-UPDRS)-Part III: Motor Examination, on both ON and OFF medication states. Using as a reference point the quantified metrics of MDS-UPDRS-Part III, severity levels were explored by classifying normal, mild, moderate, and severe levels of PD. Elaborating the recorded gait data, 18 temporal and spatial characteristics have been extracted. Subsequently, feature selection techniques were applied to reveal the dominant features to be used for four classification tasks. Specifically, for identifying relations between the spatial and temporal gait features on: PD and non-PD groups; PD, Elderly and Adults groups; PD and ON/OFF medication states; MDS-UPDRS: Part III and PD severity levels. AdaBoost, Extra Trees, and Random Forest classifiers, were trained and tested. Results showed a recognition accuracy of 88%, 73% and 81% for, the PD and non-PD groups, PD-related medication states, and PD severity levels relevant to MDS-UPDRS: Part III ratings, respectively.
A popular and widely suggested measure for assessing unilateral hand motor skills in stroke patients is the box and block test (BBT). Our study aimed to create an augmented reality enhanced version ...of the BBT (AR-BBT) and evaluate its correlation to the original BBT for stroke patients. Following G-power analysis, clinical examination, and inclusion-exclusion criteria, 31 stroke patients were included in this study. AR-BBT was developed using the Open Source Computer Vision Library (OpenCV). The MediaPipe's hand tracking library uses a palm and a hand landmark machine learning model to detect and track hands. A computer and a depth camera were employed in the clinical evaluation of AR-BBT following the principles of traditional BBT. A strong correlation was achieved between the number of blocks moved in the BBT and the AR-BBT on the hemiplegic side (Pearson correlation = 0.918) and a positive statistically significant correlation (p = 0.000008). The conventional BBT is currently the preferred assessment method. However, our approach offers an advantage, as it suggests that an AR-BBT solution could remotely monitor the assessment of a home-based rehabilitation program and provide additional hand kinematic information for hand dexterities in AR environment conditions. Furthermore, it employs minimal hardware equipment.
Parkinson's disease (PD) is characterized by a variety of motor and non-motor symptoms, some of them pertaining to gait and balance. The use of sensors for the monitoring of patients' mobility and ...the extraction of gait parameters, has emerged as an objective method for assessing the efficacy of their treatment and the progression of the disease. To that end, two popular solutions are pressure insoles and body-worn IMU-based devices, which have been used for precise, continuous, remote, and passive gait assessment. In this work, insole and IMU-based solutions were evaluated for assessing gait impairment, and were subsequently compared, producing evidence to support the use of instrumentation in everyday clinical practice. The evaluation was conducted using two datasets, generated during a clinical study, in which patients with PD wore, simultaneously, a pair of instrumented insoles and a set of wearable IMU-based devices. The data from the study were used to extract and compare gait features, independently, from the two aforementioned systems. Subsequently, subsets comprised of the extracted features, were used by machine learning algorithms for gait impairment assessment. The results indicated that insole gait kinematic features were highly correlated with those extracted from IMU-based devices. Moreover, both had the capacity to train accurate machine learning models for the detection of PD gait impairment.
Synthetic data generation has emerged as a promising solution to overcome the challenges which are posed by data scarcity and privacy concerns, as well as, to address the need for training artificial ...intelligence (AI) algorithms on unbiased data with sufficient sample size and statistical power. Our review explores the application and efficacy of synthetic data methods in healthcare considering the diversity of medical data. To this end, we systematically searched the PubMed and Scopus databases with a great focus on tabular, imaging, radiomics, time-series, and omics data. Studies involving multi-modal synthetic data generation were also explored. The type of method used for the synthetic data generation process was identified in each study and was categorized into statistical, probabilistic, machine learning, and deep learning. Emphasis was given to the programming languages used for the implementation of each method. Our evaluation revealed that the majority of the studies utilize synthetic data generators to: (i) reduce the cost and time required for clinical trials for rare diseases and conditions, (ii) enhance the predictive power of AI models in personalized medicine, (iii) ensure the delivery of fair treatment recommendations across diverse patient populations, and (iv) enable researchers to access high-quality, representative multimodal datasets without exposing sensitive patient information, among others. We underline the wide use of deep learning based synthetic data generators in 72.6 % of the included studies, with 75.3 % of the generators being implemented in Python. A thorough documentation of open-source repositories is finally provided to accelerate research in the field.
Display omitted
Automatic segmentation of the prostate of and the prostatic zones on MRI remains one of the most compelling research areas. While different image enhancement techniques are emerging as powerful tools ...for improving the performance of segmentation algorithms, their application still lacks consensus due to contrasting evidence regarding performance improvement and cross-model stability, further hampered by the inability to explain models' predictions. Particularly, for prostate segmentation, the effectiveness of image enhancement on different Convolutional Neural Networks (CNN) remains largely unexplored. The present work introduces a novel image enhancement method, named RACLAHE, to enhance the performance of CNN models for segmenting the prostate's gland and the prostatic zones. The improvement in performance and consistency across five CNN models (U-Net, U-Net++, U-Net3+, ResU-net and USE-NET) is compared against four popular image enhancement methods. Additionally, a methodology is proposed to explain, both quantitatively and qualitatively, the relation between saliency maps and ground truth probability maps. Overall, RACLAHE was the most consistent image enhancement algorithm in terms of performance improvement across CNN models with the mean increase in Dice Score ranging from 3 to 9% for the different prostatic regions, while achieving minimal inter-model variability. The integration of a feature driven methodology to explain the predictions after applying image enhancement methods, enables the development of a concrete, trustworthy automated pipeline for prostate segmentation on MR images.
Machine learning (ML) and artificial intelligence are emerging as important components of precision medicine that enhance diagnosis and risk stratification. Risk stratification tools for hypertrophic ...cardiomyopathy (HCM) exist, but they are based on traditional statistical methods. The aim was to develop a novel machine learning risk stratification tool for the prediction of 5-year risk in HCM. The goal was to determine if its predictive accuracy is higher than the accuracy of the state-of-the-art tools.
Data from a total of 2302 patients were used. The data were comprised of demographic characteristics, genetic data, clinical investigations, medications, and disease-related events. Four classification models were applied to model the risk level, and their decisions were explained using the SHAP (SHapley Additive exPlanations) method. Unwanted cardiac events were defined as sustained ventricular tachycardia occurrence (VT), heart failure (HF), ICD activation, sudden cardiac death (SCD), cardiac death, and all-cause death.
The proposed machine learning approach outperformed the similar existing risk-stratification models for SCD, cardiac death, and all-cause death risk-stratification: it achieved higher AUC by 17%, 9%, and 1%, respectively. The boosted trees achieved the best performing AUC of 0.82. The resulting model most accurately predicts VT, HF, and ICD with AUCs of 0.90, 0.88, and 0.87, respectively.
The proposed risk-stratification model demonstrates high accuracy in predicting events in patients with hypertrophic cardiomyopathy. The use of a machine-learning risk stratification model may improve patient management, clinical practice, and outcomes in general.
Display omitted
•Machine learning-based risk stratification is an alternative to statistical methods.•Machine learning risk-stratification model outperforms related statistical models.•Boosted trees achieved the highest risk stratification predictive accuracy.•Explanation of machine learning models confirms some of the known risk factors.
Virtual population generation is an emerging field in data science with numerous applications in healthcare towards the augmentation of clinical research databases with significant lack of population ...size. However, the impact of data augmentation on the development of AI (artificial intelligence) models to address clinical unmet needs has not yet been investigated. In this work, we assess whether the aggregation of real with virtual patient data can improve the performance of the existing risk stratification and disease classification models in two rare clinical domains, namely the primary Sjögren's Syndrome (pSS) and the hypertrophic cardiomyopathy (HCM), for the first time in the literature. To do so, multivariate approaches, such as, the multivariate normal distribution (MVND), and straightforward ones, such as, the Bayesian networks, the artificial neural networks (ANNs), and the tree ensembles are compared against their performance towards the generation of high-quality virtual data. Both boosting and bagging algorithms, such as, the Gradient boosting trees (XGBoost), the AdaBoost and the Random Forests (RFs) were trained on the augmented data to evaluate the performance improvement for lymphoma classification and HCM risk stratification. Our results revealed the favorable performance of the tree ensemble generators, in both domains, yielding virtual data with goodness-of-fit 0.021 and KL-divergence 0.029 in pSS and 0.029, 0.027 in HCM, respectively. The application of the XGBoost on the augmented data revealed an increase by 10.9% in accuracy, 10.7% in sensitivity, 11.5% in specificity for lymphoma classification and 16.1% in accuracy, 16.9% in sensitivity, 13.7% in specificity in HCM risk stratification.
Display omitted
•Data augmentation enriches patient databases with insufficient population size and statistical power.•Data augmentation yields hybrid and robust ML models for risk stratification and disease prediction.•Tree ensembles can produce virtual distributions having increased convergence with the real ones.•Data quality control is crucial prior to the data augmentation process.•Virtual population generation methods can address clinical unmet needs in rare diseases.
PDF
