In recent years, there have been notable advancements in technology, deep learning, and pose detection. One significant application of these advancements pertains to the real-time detection of sign ...language from video sources. The motivation behind this research stems from the pressing societal need to enhance the quality of life for individuals with speech impairments. Given the current prominence of online meetings, exacerbated by the COVID-19 pandemic, there is a growing need for systems that can provide individuals with speech impairments greater independence in communication, eliminating the requirement for a human translator. This research proposal advocates for a solution that leverages PoseNet algorithms for the extraction of key pose points, which are subsequently employed within LSTM models for the predictive modeling of sign language gestures. This research paper aims to make several notable contributions to the field of assistive technology and human-computer interaction. The achieved accuracy stands at an impressive 98%, underscoring the robustness and precision of our proposed system.
Emotions have an important role in education. Affective development, attitudes, and emotions in learning are measured using affective assessment. This method is the right way to determine the ...student’s affective development. However, the process did not run optimally because the teacher found it difficult to collect student’s affective data. This paper describes the development of a system that can assist teachers in carrying out affective assessment. The system was developed using a v-model that aligns the verification phase with the validation. The use of the system is carried out during learning activities. The emotion detection system detects through body gestures using PoseNet to generate emotional data for each student. The detection results are then processed and displayed on an information system in the form of a website for affective assessment. The accuracy of emotion detection got validation values of 84.4% and 80.95% after being tested at school. In addition, the acceptance test with the usability aspect of the system by the teacher got a score of 77.56% and a score of 79.85% by the students. Based on several tests carried out, this developed system can assist the process of implementing affective assessment.
Abstract Pose estimation is a computer vision task used to detect and estimate the pose of a person or an object in images or videos. It has some challenges that can leverage advances in computer ...vision research and others that require efficient solutions. In this paper, we provide a preliminary review of the state‐of‐the‐art in pose estimation, including both traditional and deep learning approaches. Also, we implement and compare the performance of Hand Pose Estimation (HandPE), which uses PoseNet architecture for hand sign problems, for an ASL dataset by using different optimizers based on 10 common evaluation metrics on different datasets. Also, we discuss some related future research directions in the field of pose estimation and explore new architectures for pose estimation types. After applying the PoseNet model, the experiment results showed that the accuracy achieved was 99.9%, 89%, 97%, 79%, and 99% for the ASL alphabet, HARPET, Yoga, Animal, and Head datasets, comparing those with common optimizers and evaluation metrics on different dataset.
To prevent convulsions and falls of patients in the absence of medical staff, it is crucial to monitor their physical condition in hospital wards. However, several unresolved challenges in human ...joint recognition remain, such as object occlusion, human self-occlusion and complex backgrounds, resulting in difficulties in its practical application. In this paper, a multi-LiDAR system is proposed to obtain a multi-view human body point cloud. An improved V2V-Posenet model was introduced to detect the actual position of the human joint. In this system, each point cloud was spliced into a full point cloud and voxelized into the model. We also used a random voxel zero setting for data enhancement, constraining the relative length between human joints into a loss function and three-dimensional Gaussian filtering in a heat map for model learning. The improved model exhibited excellent performance in detecting human joints in hospital wards. The experimental results showed that the improved model achieved 91.6 % mean average precision, compared to 80.1 % for the original model and 77.4 % for the comparison algorithm A2J-Posenet. The speed of the improved model meets the requirements for real-time target detection.
Sensor-based human activity recognition (HAR) is a method for observing a person's activity in an environment. With this method, it is possible to monitor remotely. HAR can analyze a person's gait, ...whether normal or abnormal. Some of its applications may use several sensors mounted on the body, but this method tends to be complex and inconvenient. One alternative to wearable sensors is using video. One of the most commonly used HAR platforms is PoseNET. PoseNET is a sophisticated platform that can detect the skeleton and joints of the body, which are then known as joints. However, a method is still needed to process the raw data from PoseNET to detect subject activity. Therefore, this research proposes a way to detect abnormalities in gait using empirical mode decomposition and the Hilbert spectrum and transforming keys-joints, and skeletons from vision-based pose detection into the angular displacement of walking gait patterns (signals). Joint change information is extracted using the Hilbert Huang Transform to study how the subject behaves in the turning position. Furthermore, it is determined whether the transition goes from normal to abnormal subjects by calculating the energy in the time-frequency domain signal. The test results show that during the transition period, the energy of the gait signal tends to be higher than during the walking period.
In this paper, we propose a skeleton-based method to identify violence and aggressive behavior. The approach does not necessitate high-processing equipment and it can be quickly implemented. Our ...approach consists of two phases: feature extraction from image sequences to assess a human posture, followed by activity classification applying a neural network to identify whether the frames include aggressive situations and violence. A video violence dataset of 400 min comprising a single person's activities and 20 h of video data including physical violence and aggressive acts, and 13 classifications for distinguishing aggressor and victim behavior were generated. Finally, the proposed method was trained and tested using the collected dataset. The results indicate the accuracy of 97% was achieved in identifying aggressive conduct in video sequences. Furthermore, the obtained results show that the proposed method can detect aggressive behavior and violence in a short period of time and is accessible for real-world applications.
This paper investigate an improved detection method that estimates the acceleration of the head and shoulder key point position and position change using the skeleton key point information extracted ...using PoseNet from the image obtained from the low-cost 2D RGB camera, and improves the accuracy of fall judgment. This paper propose a fall detection method based on the post-fall characteristics of the post-fall, the speed of changes in the main point of the human body, and the change in the width and height ratio of the body's bounding box. The public data set was used to extract human skeletal features and train deep learning, GRU, and as a result of experiments, this paper find the following feature extraction methods. High classification accuracy can be achieved, and the proposed method showed a 99.8% fall detection success rate more effectively than the conventional method using raw skeletal data.
The fusion of augmented reality (AR) and deep learning technologies has ushered in a transformative era in the realm of real-time physical activity monitoring. This research paper introduces a system ...that harnesses the capabilities of PoseNet-based skeletal keypoint extraction and deep neural networks to achieve unparalleled accuracy and real-time functionality in the identification and classification of a wide spectrum of physical activities. With an impressive accuracy rate of 98% within 100 training epochs, the system proves its mettle in precise activity recognition, making it invaluable in domains such as fitness training, physical education, sports coaching, and home-based fitness. The system's real-time feedback mechanism, bolstered by AR technology, not only enhances user engagement but also motivates users to optimize their exercise routines. This paper not only elucidates the system's architecture and functionality but also highlights its potential applications across diverse fields. Furthermore, it delineates the trajectory of future research avenues, including the development of advanced feedback mechanisms, exploration of multi-modal sensing techniques, personalization for users, assessment of long-term impacts, and endeavors to ensure accessibility, inclusivity, and data privacy. In essence, this research sets the stage for the evolution of real-time physical activity monitoring, offering a compelling framework to improve fitness, physical education, and athletic training while promoting healthier lifestyles and the overall well-being of individuals worldwide.
Human pose estimation (HPE) has become a prevalent research topic in computer vision. The technology can be applied in many areas, such as video surveillance, medical assistance, and sport motion ...analysis. Due to higher demand for HPE, many HPE libraries have been developed in the last 20 years. In the last 5 years, more and more skeleton-based HPE algorithms have been developed and packaged into libraries to provide ease of use for researchers. Hence, the performance of these libraries is important when researchers intend to integrate them into real-world applications for video surveillance, medical assistance, and sport motion analysis. However, a comprehensive performance comparison of these libraries has yet to be conducted. Therefore, this paper aims to investigate the strengths and weaknesses of four popular state-of-the-art skeleton-based HPE libraries for human pose detection, including OpenPose, PoseNet, MoveNet, and MediaPipe Pose. A comparative analysis of these libraries based on images and videos is presented in this paper. The percentage of detected joints (PDJ) was used as the evaluation metric in all comparative experiments to reveal the performance of the HPE libraries. MoveNet showed the best performance for detecting different human poses in static images and videos.
Celotno besedilo
Dostopno za:
CEKLJ, DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, UILJ, UKNU, UL, UM, UPUK
This paper suggests the Exergaming approach of computer game control for a human motion in sport computer games in order to increase the level of physical activity of students in online classes, ...which takes into account the initial knowledge of the specialised courses of the IT curriculums. The approach is based on open access repository of computer programs in the Scratch-based block language (47 games for 28 summer sports, 16 games for 10 winter sports). Scratch programs for human motion recognition are based on the PoseNet neural network, which allows changing the principle of controlling game characters through the keyboard and mouse. In October 2023, ninety two first-year students majoring in Computer Science participated in a five-week online Exergaming as second part of the Physical Education course, among them 70 students (76%) successfully completed all stages. 57% of students felt motivated to continue using the Exergame approach and demonstrated the desire to achieve decent results, that indicates skill “Positive Thinking” in the discipline “Physical Education”, seven Hard Skills and four Soft Skills of Computing Curricula 2020 were indicated to compensate for the lack of classroom learning.
PDF
