Sign language is a visual language used by deaf people. One difficulty of sign language recognition is that sign instances of vary in both motion and shape in three-dimensional (3D) space. In this ...research, we use 3D depth information from hand motions, generated from Microsoft's Kinect sensor and apply a hierarchical conditional random field (CRF) that recognizes hand signs from the hand motions. The proposed method uses a hierarchical CRF to detect candidate segments of signs using hand motions, and then a BoostMap embedding method to verify the hand shapes of the segmented signs. Experiments demonstrated that the proposed method could recognize signs from signed sentence data at a rate of 90.4%.
Sleep favors the reactivation and consolidation of newly acquired memories. Yet, how our brain selects the noteworthy information to be reprocessed during sleep remains largely unknown. From an ...evolutionary perspective, individuals must retain information that promotes survival, such as avoiding dangers, finding food, or obtaining praise or money. Here, we test whether neural representations of rewarded (compared to non-rewarded) events have priority for reactivation during sleep. Using functional MRI and a brain decoding approach, we show that patterns of brain activity observed during waking behavior spontaneously reemerge during slow-wave sleep. Critically, we report a privileged reactivation of neural patterns previously associated with a rewarded task (i.e., winning at a complex game). Moreover, during sleep, activity in task-related brain regions correlates with better subsequent memory performance. Our study uncovers a neural mechanism whereby rewarded life experiences are preferentially replayed and consolidated while we sleep.
In the last three decades, the development of functional magnetic resonance imaging (fMRI) has significantly contributed to the understanding of the brain, functional brain mapping, and resting-state ...brain networks. Given the recent successes of deep learning in various fields, we propose a 3D-CNN-LSTM classification model to diagnose health conditions with the following classes: condition normal (CN), early mild cognitive impairment (EMCI), late mild cognitive impairment (LMCI), and Alzheimer's disease (AD). The proposed method employs spatial and temporal feature extractors, wherein the former utilizes a U-Net architecture to extract spatial features, and the latter utilizes long short-term memory (LSTM) to extract temporal features. Prior to feature extraction, we performed four-step pre-processing to remove noise from the fMRI data. In the comparative experiments, we trained each of the three models by adjusting the time dimension. The network exhibited an average accuracy of 96.4% when using five-fold cross-validation. These results show that the proposed method has high potential for identifying the progression of Alzheimer's by analyzing 4D fMRI data.
Artificial intelligence technologies and vision systems are used in various devices, such as automotive navigation systems, object-tracking systems, and intelligent closed-circuit televisions. In ...particular, outdoor vision systems have been applied across numerous fields of analysis. Despite their widespread use, current systems work well under good weather conditions. They cannot account for inclement conditions, such as rain, fog, mist, and snow. Images captured under inclement conditions degrade the performance of vision systems. Vision systems need to detect, recognize, and remove noise because of rain, snow, and mist to boost the performance of the algorithms employed in image processing. Several studies have targeted the removal of noise resulting from inclement conditions. We focused on eliminating the effects of raindrops on images captured with outdoor vision systems in which the camera was exposed to rain. An attentive generative adversarial network (ATTGAN) was used to remove raindrops from the images. This network was composed of two parts: an attentive-recurrent network and a contextual autoencoder. The ATTGAN generated an attention map to detect rain droplets. A de-rained image was generated by increasing the number of attentive-recurrent network layers. We increased the number of visual attentive-recurrent network layers in order to prevent gradient sparsity so that the entire generation was more stable against the network without preventing the network from converging. The experimental results confirmed that the extended ATTGAN could effectively remove various types of raindrops from images.
Sign language spotting is the task of detecting and recognizing signs in a signed utterance, in a set vocabulary. The difficulty of sign language spotting is that instances of signs vary in both ...motion and appearance. Moreover, signs appear within a continuous gesture stream, interspersed with transitional movements between signs in a vocabulary and nonsign patterns (which include out-of-vocabulary signs, epentheses, and other movements that do not correspond to signs). In this paper, a novel method for designing threshold models in a conditional random field (CRF) model is proposed which performs an adaptive threshold for distinguishing between signs in a vocabulary and nonsign patterns. A short-sign detector, a hand appearance-based sign verification method, and a subsign reasoning method are included to further improve sign language spotting accuracy. Experiments demonstrate that our system can spot signs from continuous data with an 87.0 percent spotting rate and can recognize signs from isolated data with a 93.5 percent recognition rate versus 73.5 percent and 85.4 percent, respectively, for CRFs without a threshold model, short-sign detection, subsign reasoning, and hand appearance-based sign verification. Our system can also achieve a 15.0 percent sign error rate (SER) from continuous data and a 6.4 percent SER from isolated data versus 76.2 percent and 14.5 percent, respectively, for conventional CRFs.
Recently, three-dimensional (3D) content used in various fields has attracted attention owing to the development of virtual reality and augmented reality technologies. To produce 3D content, we need ...to model the objects as vertices. However, high-quality modeling is time-consuming and costly. Drawing-based modeling is a technique that shortens the time required for modeling. It refers to creating a 3D model based on a user’s line drawing, which is a 3D feature represented by two-dimensional (2D) lines. The extracted line drawing provides information about a 3D model in the 2D space. It is sometimes necessary to generate a line drawing from a 2D cartoon image to represent the 3D information of a 2D cartoon image. The extraction of consistent line drawings from 2D cartoons is difficult because the styles and techniques differ depending on the designer who produces the 2D cartoons. Therefore, it is necessary to extract line drawings that show the geometric characteristics well in 2D cartoon shapes of various styles. This paper proposes a method for automatically extracting line drawings. The 2D cartoon shading image and line drawings are learned using a conditional generative adversarial network model, which outputs the line drawings of the cartoon artwork. The experimental results show that the proposed method can obtain line drawings representing the 3D geometric characteristics with a 2D line when a 2D cartoon painting is used as the input.
•Combine manual and non-manual signals to recognize sign language.•Apply a hierarchical CRF to discriminate between signs and fingerspelling.•Recognize a facial expression to analyze the specific ...lexical meaning of signed utterance.
The sign language is composed of two categories of signals: manual signals such as signs and fingerspellings and non-manual ones such as body gestures and facial expressions. This paper proposes a new method for recognizing manual signals and facial expressions as non-manual signals. The proposed method involves the following three steps: First, a hierarchical conditional random field is used to detect candidate segments of manual signals. Second, the BoostMap embedding method is used to verify hand shapes of segmented signs and to recognize fingerspellings. Finally, the support vector machine is used to recognize facial expressions as non-manual signals. This final step is taken when there is some ambiguity in the previous two steps. The experimental results indicate that the proposed method can accurately recognize the sign language at an 84% rate based on utterance data.
Although posttranscriptional regulation of RNA metabolism is increasingly recognized as a key regulatory process in plant response to environmental stresses, reports demonstrating the importance of ...RNA metabolism control in crop improvement under adverse environmental stresses are severely limited. To investigate the potential use of RNA-binding proteins (RBPs) in developing stress-tolerant transgenic crops, we generated transgenic rice plants (Oryza sativa) that express Arabidopsis thaliana glycine-rich RBP (AtGRP) 2 or 7, which have been determined to harbor RNA chaperone activity and confer stress tolerance in Arabidopsis, and analyzed the response of the transgenic rice plants to abiotic stresses. AtGRP2- or AtGRP7-expressing transgenic rice plants displayed similar phenotypes comparable with the wild-type plants under high salt or cold stress conditions. By contrast, AtGRP2- or AtGRP7-expressing transgenic rice plants showed much higher recovery rates and grain yields compared with the wild-type plants under drought stress conditions. The higher grain yield of the transgenic rice plants was due to the increases in filled grain numbers per panicle. Collectively, the present results show the importance of posttranscriptional regulation of RNA metabolism in plant response to environmental stress and suggest that GRPs can be utilized to improve the yield potential of crops under stress conditions.
Visual interpretation of gestures can be useful in accomplishing natural human-robot interaction (HRI). Previous HRI research focused on issues such as hand gestures, sign language, and command ...gesture recognition. Automatic recognition of whole-body gestures is required in order for HRI to operate naturally. This presents a challenging problem, because describing and modeling meaningful gesture patterns from whole-body gestures is a complex task. This paper presents a new method for recognition of whole-body key gestures in HRI. A human subject is first described by a set of features, encoding the angular relationship between a dozen body parts in 3-D. A feature vector is then mapped to a codeword of hidden Markov models. In order to spot key gestures accurately, a sophisticated method of designing a transition gesture model is proposed. To reduce the states of the transition gesture model, model reduction which merges similar states based on data-dependent statistics and relative entropy is used. The experimental results demonstrate that the proposed method can be efficient and effective in HRI, for automatic recognition of whole-body key gestures from motion sequences
PDF
