Attention has arguably become one of the most important concepts in the deep learning field. It is inspired by the biological systems of humans that tend to focus on the distinctive parts when ...processing large amounts of information. With the development of deep neural networks, attention mechanism has been widely used in diverse application domains. This paper aims to give an overview of the state-of-the-art attention models proposed in recent years. Toward a better general understanding of attention mechanisms, we define a unified model that is suitable for most attention structures. Each step of the attention mechanism implemented in the model is described in detail. Furthermore, we classify existing attention models according to four criteria: the softness of attention, forms of input feature, input representation, and output representation. Besides, we summarize network architectures used in conjunction with the attention mechanism and describe some typical applications of attention mechanism. Finally, we discuss the interpretability that attention brings to deep learning and present its potential future trends.
This letter adopts long short-term memory (LSTM) to predict sea surface temperature (SST), and makes short-term prediction, including one day and three days, and long-term prediction, including ...weekly mean and monthly mean. The SST prediction problem is formulated as a time series regression problem. The proposed network architecture is composed of two kinds of layers: an LSTM layer and a full-connected dense layer. The LSTM layer is utilized to model the time series relationship. The full-connected layer is utilized to map the output of the LSTM layer to a final prediction. The optimal setting of this architecture is explored by experiments and the accuracy of coastal seas of China is reported to confirm the effectiveness of the proposed method. The prediction accuracy is also tested on the SST anomaly data. In addition, the model's online updated characteristics are presented.
In recent years, circular RNAs (circRNAs) have been found to play an essential regulatory role in hepatocellular carcinoma (HCC) through various mechanisms, particularly the endogenous competitive ...RNA (ceRNA) mechanism. Therefore, it is significant to explore the circRNAs in hepatoma. In this study, we constructed the ceRNA and survival network using Cytoscape. We also used R, Perl software, and multiple online databases and platforms, including Gene Ontology (GO) and Kyoto Encyclopedia of Genes and Genomes (KEGG), to perform overall survival, immune cell infiltration, immune checkpoints, pathway activity, and anticancer drug sensitivity analysis of the genes. Finally, the receiver operator characteristic curve (ROC) analysis was performed to identify the diagnosis value of the genes. KEGG analysis revealed the T cell receptor signaling pathway as the main enrichment pathway. A total of 29 genes related to survival and prognosis were screened out. The findings suggest that ZNF544, WDR76, ACTG1, RASSF3, E2F3, ASRGL1, and POGK are associated with multilevel immune cell infiltration. Additionally, immune checkpoint analysis screened out the ACTG1, E2F3, RASSF3, and WDR76. It was also revealed that the WDR76, E2F3, ASRGL1, and POGK mainly activated the cell cycle and DNA damage response (DDR) pathway. The results suggest that the sensitivity toward trametinib, refametinib (RDEA119), and selumetinib correlates to the expression of WDR76. ROC analysis showed that the area under the curve (AUC) of all genes in the regulatory axis was greater than 0.7. The identified hsa_circ_0000417/hsa_circ_0002688/hsa_circ_0001387--hsa-miR-199a-5p--WDR76 regulatory axis may provide new insights into the progression, clinical diagnosis, and treatment of HCC.
Underwater pulse waveform recognition is an important method for underwater object detection. Most existing works focus on the application of traditional pattern recognition methods, which ignore the ...time- and space-varying characteristics in sound propagation channels and cannot easily extract valuable waveform features. Sound propagation channels in seawater are time- and space-varying convolutional channels. In the extraction of the waveform features of underwater acoustic signals, the effect of high-accuracy underwater acoustic signal recognition is identified by eliminating the influence of time- and space-varying convolutional channels to the greatest extent possible. We propose a hash aggregate discriminative network (HADN), which combines hash learning and deep learning to minimize the time- and space-varying effects on convolutional channels and adaptively learns effective underwater waveform features to achieve high-accuracy underwater pulse waveform recognition. In the extraction of the hash features of acoustic signals, a discrete constraint between clusters within a hash feature class is introduced. This constraint can ensure that the influence of convolutional channels on hash features is minimized. In addition, we design a new loss function called aggregate discriminative loss (AD-loss). The use of AD-loss and softmax-loss can increase the discriminativeness of the learned hash features. Experimental results show that on pool and ocean datasets, which were collected in pools and oceans, respectively, by using acoustic collectors, the proposed HADN performs better than other comparative models in terms of accuracy and mAP.
Inflammatory bowel disease (IBD) is a chronic, recurrent gastrointestinal disorder with elusive etiology. Interleukin-12 (IL-12) and IL-23 have emerged as key proinflammatory mediators/cytokines in ...IBD pathogenesis. Ustekinumab (UST), targeting IL-12 and IL-23, has demonstrated promising efficacy and safety in the treatment of IBD. Recently, UST has become increasingly favored as a potential first-line treatment option. This review delineates UST's mechanism of action, its clinical applications in IBD, including the response rates, strategies for dose optimization for case of partial or lost response, and potential adverse events. This review aims to offer a comprehensive understanding of UST's role as a therapeutic option in IBD management.
RNA splicing is the process of forming mature mRNA, which is an essential phase necessary for gene expression and controls many aspects of cell proliferation, survival, and differentiation. Abnormal ...gene-splicing events are closely related to the development of tumors, and the generation of oncogenic isoform in splicing can promote tumor progression. As a main process of tumor-specific splicing variants, alternative splicing (AS) can promote tumor progression by increasing the production of oncogenic splicing isoforms and/or reducing the production of normal splicing isoforms. This is the focus of current research on the regulation of aberrant tumor splicing. So far, AS has been found to be associated with various aspects of tumor biology, including cell proliferation and invasion, resistance to apoptosis, and sensitivity to different chemotherapeutic drugs. This article will review the abnormal splicing events in colorectal cancer (CRC), especially the tumor-associated splicing variants arising from AS, aiming to offer an insight into CRC-targeted splicing therapy.
Inflammatory bowel disease (IBD), a general term encompassing Crohn’s disease (CD) and ulcerative colitis (UC), and other conditions, is a chronic and relapsing autoimmune disease that can occur in ...any part of the digestive tract. While the cause of IBD remains unclear, it is acknowledged that the disease has much to do with the dysregulation of intestinal immunity. In the intestinal immune regulatory system, Cholesterol-25-hydroxylase (CH25H) plays an important role in regulating the function of immune cells and lipid metabolism through catalyzing the oxidation of cholesterol into 25-hydroxycholesterol (25-HC). Specifically, CH25H focuses its mechanism of regulating the inflammatory response, signal transduction and cell migration on various types of immune cells by binding to relevant receptors, and the mechanism of regulating lipid metabolism and immune cell function via the transcription factor Sterol Regulator-Binding Protein. Based on this foundation, this article will review the function of CH25H in intestinal immunity, aiming to provide evidence for supporting the discovery of early diagnostic and treatment targets for IBD.
Glycemic variability (GV) has been associated with vascular complications in patients with diabetes. However, the relationship between GV and risk of atrial fibrillation (AF) remains not fully ...determined. We therefore conducted a systematic review and meta-analysis to evaluate the above association.
Medline, Embase, Web of Science, Wanfang, and China National Knowledge Infrastructure were searched for longitudinal follow-up studies comparing the incidence of AF between patients with higher versus lower GV. A random-effects model incorporating the potential heterogeneity was used to pool the results.
Nine cohort studies with 6,877,661 participants were included, and 36,784 (0.53%) participants developed AF during follow-up. Pooled results showed that a high GV was associated with an increased risk of AF (risk ratio RR: 1.20, 95% confidence interval CI: 1.11 to 1.30, p < 0.001, I
= 20%). Subgroup analyses suggested consistent association between GV and AF in prospective (RR: 1.29, 95% CI: 1.05 to 1.59, p = 0.01) and retrospective studies (RR: 1.18, 95% CI: 1.08 to 1.29, p = 0.002), in diabetic (RR: 1.24, 95% CI: 1.03 to 1.50, p = 0.03) and non-diabetic subjects (RR: 1.13, 95% CI: 1.00 to 1.28, p = 0.05), in studies with short-term (RR: 1.25, 95% CI: 1.11 to 1.40, p < 0.001) and long-term GV (RR: 1.18, 95% CI: 1.05 to 1.34, p = 0.006), and in studies with different quality scores (p for subgroup difference all > 0.05).
A high GV may predict an increased risk of AF in adult population.
Atrial fibrosis is a crucial contributor to initiation and perpetuation of atrial fibrillation (AF). This study aimed to identify a circRNA-miRNA-mRNA competitive endogenous RNA (ceRNA) regulatory ...network related to atrial fibrosis in AF, especially to validate hsa_circ_0000672/hsa_miR-516a-5p/TRAF6 ceRNA axis in AF preliminarily. The circRNA-miRNA-mRNA ceRNA network associated with AF fibrosis was constructed using bioinformatic tools and literature reviews. Left atrium (LA) low voltage was used to represent LA fibrosis by using LA voltage matrix mapping. Ten controls with sinus rhythm (SR), and 20 patients with persistent AF including 12 patients with LA low voltage and 8 patients with LA normal voltage were enrolled in this study. The ceRNA regulatory network associated with atrial fibrosis was successfully constructed, which included up-regulated hsa_circ_0000672 and hsa_circ_0003916, down-regulated miR-516a-5p and five up-regulated hub genes (KRAS, SMAD2, TRAF6, MAPK11 and SMURF1). In addition, according to the results of Kyoto Encyclopedia of Genes and Genomes (KEGG) pathway analysis, these hub genes were clustered in TGF-beta and MAPK signaling pathway. In the patients with persistent AF, hsa_circ_0000672 expression in peripheral blood monocytes was significantly higher than those in controls with SR by quantitative real-time polymerase chain reaction (p-value < 0.001). Furthermore, hsa_circ_0000672 expression was higher in peripheral blood monocytes of persistent AF patients with LA low voltage than those with LA normal voltage (p-value = 0.002). The dual-luciferase activity assay confirmed that hsa_circ_0000672 exerted biological functions as a sponge of miR-516a-5p to regulate expression of its target gene TRAF6. Hsa_circ_0000672 expression in peripheral blood monocytes may be associated with atrial fibrosis. The hsa_circ_0000672 may be involved in atrial fibrosis by indirectly regulating TRAF6 as a ceRNA by sponging miR-516a-5p.
Since about 100 years ago, to learn the intrinsic structure of data, many representation learning approaches have been proposed, either linear or nonlinear, either supervised or unsupervised, either ...“shallow” or “deep”. Particularly, deep architectures are widely applied for representation learning in recent years, and have delivered top results in many tasks, such as image classification, object detection and speech recognition. In this paper, we review the development of data representation learning methods. Specifically, we investigate both traditional feature learning algorithms and state-of-the-art deep learning models. The history of data representation learning is introduced, while available online resources (e.g., courses, tutorials and books) and toolboxes are provided. At the end, we give a few remarks on the development of data representation learning and suggest some interesting research directions in this area.
PDF
