Abstract Background Telerehabilitation is a promising avenue for improving patient outcomes and expanding accessibility. However, there is currently no spine-related assessment for telerehabilitation ...that covers multiple exercises. Methods We propose a wearable system with two inertial measurement units (IMUs) to identify IMU locations and estimate spine angles for ten commonly prescribed spinal degeneration rehabilitation exercises (supine chin tuck head lift rotation, dead bug unilateral isometric hold, pilates saw, catcow full spine, wall angel, quadruped neck flexion/extension, adductor open book, side plank hip dip, bird dog hip spinal flexion, and windmill single leg). Twelve healthy subjects performed these spine-related exercises, and wearable IMU data were collected for spine angle estimation and IMU location identification. Results Results demonstrated average mean absolute spinal angle estimation errors of 2.59 $$^\circ$$ ∘ and average classification accuracy of 92.97%. The proposed system effectively identified IMU locations and assessed spine-related rehabilitation exercises while demonstrating robustness to individual differences and exercise variations. Conclusion This inexpensive, convenient, and user-friendly approach to spine degeneration rehabilitation could potentially be implemented at home or provide remote assessment, offering a promising avenue to enhance patient outcomes and improve accessibility for spine-related rehabilitation. Trial registration: No. E2021013P in Shanghai Jiao Tong University.
We introduce pyannote.audio, an open-source toolkit written in Python for speaker diarization. Based on PyTorch machine learning framework, it provides a set of trainable end-to-end neural building ...blocks that can be combined and jointly optimized to build speaker diarization pipelines. pyannote.audio also comes with pre-trained models covering a wide range of domains for voice activity detection, speaker change detection, overlapped speech detection, and speaker embedding - reaching state-of-the-art performance for most of them.
La tâche de segmentation et de regroupement en locuteurs (speaker diarization) consiste à identifier "qui parle quand" dans un flux audio sans connaissance a priori du nombre de locuteurs ou de leur ...temps de parole respectifs. Les systèmes de segmentation et de regroupement en locuteurs sont généralement construits en combinant quatre étapes principales. Premièrement, les régions ne contenant pas de parole telles que les silences, la musique et le bruit sont supprimées par la détection d'activité vocale (VAD). Ensuite, les régions de parole sont divisées en segments homogènes en locuteur par détection des changements de locuteurs, puis regroupées en fonction de l'identité du locuteur. Enfin, les frontières des tours de parole et leurs étiquettes sont affinées avec une étape de re-segmentation. Dans cette thèse, nous proposons d'aborder ces quatre étapes avec des approches fondées sur les réseaux de neurones. Nous formulons d’abord le problème de la segmentation initiale (détection de l’activité vocale et des changements entre locuteurs) et de la re-segmentation finale sous la forme d’un ensemble de problèmes d’étiquetage de séquence, puis nous les résolvons avec des réseaux neuronaux récurrents de type Bi-LSTM (Bidirectional Long Short-Term Memory). Au stade du regroupement des régions de parole, nous proposons d’utiliser l'algorithme de propagation d'affinité à partir de plongements neuronaux de ces tours de parole dans l'espace vectoriel des locuteurs. Des expériences sur un jeu de données télévisées montrent que le regroupement par propagation d'affinité est plus approprié que le regroupement hiérarchique agglomératif lorsqu'il est appliqué à des plongements neuronaux de locuteurs. La segmentation basée sur les réseaux récurrents et la propagation d'affinité sont également combinées et optimisées conjointement pour former une chaîne de regroupement en locuteurs. Comparé à un système dont les modules sont optimisés indépendamment, la nouvelle chaîne de traitements apporte une amélioration significative. De plus, nous proposons d’améliorer l'estimation de la matrice de similarité par des réseaux neuronaux récurrents, puis d’appliquer un partitionnement spectral à partir de cette matrice de similarité améliorée. Le système proposé atteint des performances à l'état de l'art sur la base de données de conversation téléphonique CALLHOME. Enfin, nous formulons le regroupement des tours de parole en mode séquentiel sous la forme d'une tâche supervisée d’étiquetage de séquence et abordons ce problème avec des réseaux récurrents empilés. Pour mieux comprendre le comportement du système, une analyse basée sur une architecture de codeur-décodeur est proposée. Sur des exemples synthétiques, nos systèmes apportent une amélioration significative par rapport aux méthodes de regroupement traditionnelles.
Speaker diarization is the task of determining "who speaks when" in an audio stream that usually contains an unknown amount of speech from an unknown number of speakers. Speaker diarization systems are usually built as the combination of four main stages. First, non-speech regions such as silence, music, and noise are removed by Voice Activity Detection (VAD). Next, speech regions are split into speaker-homogeneous segments by Speaker Change Detection (SCD), later grouped according to the identity of the speaker thanks to unsupervised clustering approaches. Finally, speech turn boundaries and labels are (optionally) refined with a re-segmentation stage. In this thesis, we propose to address these four stages with neural network approaches. We first formulate both the initial segmentation (voice activity detection and speaker change detection) and the final re-segmentation as a set of sequence labeling problems and then address them with Bidirectional Long Short-Term Memory (Bi-LSTM) networks. In the speech turn clustering stage, we propose to use affinity propagation on top of neural speaker embeddings. Experiments on a broadcast TV dataset show that affinity propagation clustering is more suitable than hierarchical agglomerative clustering when applied to neural speaker embeddings. The LSTM-based segmentation and affinity propagation clustering are also combined and jointly optimized to form a speaker diarization pipeline. Compared to the pipeline with independently optimized modules, the new pipeline brings a significant improvement. In addition, we propose to improve the similarity matrix by bidirectional LSTM and then apply spectral clustering on top of the improved similarity matrix. The proposed system achieves state-of-the-art performance in the CALLHOME telephone conversation dataset. Finally, we formulate sequential clustering as a supervised sequence labeling task and address it with stacked RNNs. To better understand its behavior, the analysis is based on a proposed encoder-decoder architecture. Our proposed systems bring a significant improvement compared with traditional clustering methods on toy examples.
Korean spruce （Picea koraiensis Sieb. E1 Zucc.） is one of the main afforestation species in northern China. Seedling quality is a critical factor at planting time. To test whether the synergistic ...growth enhancement of Scotch pine （P. sylvestris var. mongolica） seedlings brought by the plant beneficial fungus Trichoderma virens （J.H. Mill., Giddens and A.A. Foster） Arx and ectomycorrhizal fungus （Suillus luteus （L.） Roussel.） can also benefit Korean spruce seed- lings, we examined the effects of S. luteus and T. virens on the growth of P. koraiensis seedlings and drought resistance of P. sylvestris var. mongolica in peat soils. The two fungi were added to sterilized peat soil in pots, and the plants were grown for 4 months. Seedling growth and physiological variables, including mycorrhizal colonization rate of roots, biomass, and chlorophyll content, were examined. The colonization rate of the mycorrhizal fungus on P. koraiensis exceeded 65 %, and the synergism between S. luteus and T. virens enhanced most of the variables for P. koraiensis seedlings after inoculation with S. luteus then 30 days later with T. virens as in our published results for seedlings of P. sylvestris var. mongolica. When seedlings of P. sylvestris var. mongolica were inoculated with this sequence, they became more drought tolerant. T. virens also induced S. luteus to produce -l,3-glucanase and chitinase. This inocu- lation sequence at planting can thus improve the quality of P. sylvestris var. mongolica and P. koraiensis seedlings and substantiates our previous results.
Recent work in cross-lingual contextual word embedding learning cannot handle multi-sense words well. In this work, we explore the characteristics of contextual word embeddings and show the link ...between contextual word embeddings and word senses. We propose two improving solutions by considering contextual multi-sense word embeddings as noise (removal) and by generating cluster level average anchor embeddings for contextual multi-sense word embeddings (replacement). Experiments show that our solutions can improve the supervised contextual word embeddings alignment for multi-sense words in a microscopic perspective without hurting the macroscopic performance on the bilingual lexicon induction task. For unsupervised alignment, our methods significantly improve the performance on the bilingual lexicon induction task for more than 10 points.
More and more neural network approaches have achieved considerable improvement upon submodules of speaker diarization system, including speaker change detection and segment-wise speaker embedding ...extraction. Still, in the clustering stage, traditional algorithms like probabilistic linear discriminant analysis (PLDA) are widely used for scoring the similarity between two speech segments. In this paper, we propose a supervised method to measure the similarity matrix between all segments of an audio recording with sequential bidirectional long short-term memory networks (Bi-LSTM). Spectral clustering is applied on top of the similarity matrix to further improve the performance. Experimental results show that our system significantly outperforms the state-of-the-art methods and achieves a diarization error rate of 6.63% on the NIST SRE 2000 CALLHOME database.
This paper describes the speaker diarization systems developed for the Second DIHARD Speech Diarization Challenge (DIHARD II) by the Speed team. Besides describing the system, which considerably ...outperformed the challenge baselines, we also focus on the lessons learned from numerous approaches that we tried for single and multi-channel systems. We present several components of our diarization system, including categorization of domains, speech enhancement, speech activity detection, speaker embeddings, clustering methods, resegmentation, and system fusion. We analyze and discuss the effect of each such component on the overall diarization performance within the realistic settings of the challenge.
PDF
