Humans can naturally and effectively find salient regions in complex scenes. Motivated by this observation, attention mechanisms were introduced into computer vision with the aim of imitating this ...aspect of the human visual system. Such an attention mechanism can be regarded as a dynamic weight adjustment process based on features of the input image. Attention mechanisms have achieved great success in many visual tasks, including image classification, object detection, semantic segmentation, video understanding, image generation, 3D vision, multimodal tasks, and self-supervised learning. In this survey, we provide a comprehensive review of various attention mechanisms in computer vision and categorize them according to approach, such as channel attention, spatial attention, temporal attention, and branch attention; a related repository
https://github.com/MenghaoGuo/Awesome-Vision-Attentions
is dedicated to collecting related work. We also suggest future directions for attention mechanism research.
The class activation maps are generated from the final convolutional layer of CNN. They can highlight discriminative object regions for the class of interest. These discovered object regions have ...been widely used for weakly-supervised tasks. However, due to the small spatial resolution of the final convolutional layer, such class activation maps often locate coarse regions of the target objects, limiting the performance of weakly-supervised tasks that need pixel-accurate object locations. Thus, we aim to generate more fine-grained object localization information from the class activation maps to locate the target objects more accurately. In this paper, by rethinking the relationships between the feature maps and their corresponding gradients, we propose a simple yet effective method, called LayerCAM. It can produce reliable class activation maps for different layers of CNN. This property enables us to collect object localization information from coarse (rough spatial localization) to fine (precise fine-grained details) levels. We further integrate them into a high-quality class activation map, where the object-related pixels can be better highlighted. To evaluate the quality of the class activation maps produced by LayerCAM, we apply them to weakly-supervised object localization and semantic segmentation. Experiments demonstrate that the class activation maps generated by our method are more effective and reliable than those by the existing attention methods. The code will be made publicly available.
Salient object segmentation, edge detection, and skeleton extraction are three contrasting low-level pixel-wise vision problems, where existing works mostly focused on designing tailored methods for ...each individual task. However, it is inconvenient and inefficient to store a pre-trained model for each task and perform multiple different tasks in sequence. There are methods that solve specific related tasks jointly but require datasets with different types of annotations supported at the same time. In this paper, we first show some similarities shared by these tasks and then demonstrate how they can be leveraged for developing a unified framework that can be trained end-to-end. In particular, we introduce a selective integration module that allows each task to dynamically choose features at different levels from the shared backbone based on its own characteristics. Furthermore, we design a task-adaptive attention module, aiming at intelligently allocating information for different tasks according to the image content priors. To evaluate the performance of our proposed network on these tasks, we conduct exhaustive experiments on multiple representative datasets. We will show that though these tasks are naturally quite different, our network can work well on all of them and even perform better than current single-purpose state-of-the-art methods. In addition, we also conduct adequate ablation analyses that provide a full understanding of the design principles of the proposed framework. To facilitate future research, source code will be released.
We extensively compare, qualitatively and quantitatively, 41 state-of-the-art models (29 salient object detection, 10 fixation prediction, 1 objectness, and 1 baseline) over seven challenging data ...sets for the purpose of benchmarking salient object detection and segmentation methods. From the results obtained so far, our evaluation shows a consistent rapid progress over the last few years in terms of both accuracy and running time. The top contenders in this benchmark significantly outperform the models identified as the best in the previous benchmark conducted three years ago. We find that the models designed specifically for salient object detection generally work better than models in closely related areas, which in turn provides a precise definition and suggests an appropriate treatment of this problem that distinguishes it from other problems. In particular, we analyze the influences of center bias and scene complexity in model performance, which, along with the hard cases for the state-of-the-art models, provide useful hints toward constructing more challenging large-scale data sets and better saliency models. Finally, we propose probable solutions for tackling several open problems, such as evaluation scores and data set bias, which also suggest future research directions in the rapidly growing field of salient object detection.
Detecting and segmenting salient objects from natural scenes, often referred to as salient object detection, has attracted great interest in computer vision. While many models have been proposed and ...several applications have emerged, a deep understanding of achievements and issues remains lacking. We aim to provide a comprehensive review of recent progress in salient object detection and situate this field among other closely related areas such as generic scene segmentation, object proposal generation, and saliency for fixation prediction. Covering 228 publications, we survey i) roots, key concepts, and tasks, ii) core techniques and main modeling trends, and iii) datasets and evaluation metrics for salient object detection. We also discuss open problems such as evaluation metrics and dataset bias in model performance, and suggest future research directions.
Delving Deep Into Label Smoothing Zhang, Chang-Bin; Jiang, Peng-Tao; Hou, Qibin ...
IEEE transactions on image processing,
2021, Volume:
30
Journal Article
Peer reviewed
Open access
Label smoothing is an effective regularization tool for deep neural networks (DNNs), which generates soft labels by applying a weighted average between the uniform distribution and the hard label. It ...is often used to reduce the overfitting problem of training DNNs and further improve classification performance. In this paper, we aim to investigate how to generate more reliable soft labels. We present an Online Label Smoothing (OLS) strategy, which generates soft labels based on the statistics of the model prediction for the target category. The proposed OLS constructs a more reasonable probability distribution between the target categories and non-target categories to supervise DNNs. Experiments demonstrate that based on the same classification models, the proposed approach can effectively improve the classification performance on CIFAR-100, ImageNet, and fine-grained datasets. Additionally, the proposed method can significantly improve the robustness of DNN models to noisy labels compared to current label smoothing approaches. The source code is available at our project page: https://mmcheng.net/ols/
The precise and large-scale identification of intact glycopeptides is a critical step in glycoproteomics. Owing to the complexity of glycosylation, the current overall throughput, data quality and ...accessibility of intact glycopeptide identification lack behind those in routine proteomic analyses. Here, we propose a workflow for the precise high-throughput identification of intact N-glycopeptides at the proteome scale using stepped-energy fragmentation and a dedicated search engine. pGlyco 2.0 conducts comprehensive quality control including false discovery rate evaluation at all three levels of matches to glycans, peptides and glycopeptides, improving the current level of accuracy of intact glycopeptide identification. The N-glycoproteome of samples metabolically labeled with
N/
C were analyzed quantitatively and utilized to validate the glycopeptide identification, which could be used as a novel benchmark pipeline to compare different search engines. Finally, we report a large-scale glycoproteome dataset consisting of 10,009 distinct site-specific N-glycans on 1988 glycosylation sites from 955 glycoproteins in five mouse tissues.Protein glycosylation is a heterogeneous post-translational modification that generates greater proteomic diversity that is difficult to analyze. Here the authors describe pGlyco 2.0, a workflow for the precise one step identification of intact N-glycopeptides at the proteome scale.
Tin‐based halide perovskite materials have been successfully employed in lead‐free perovskite solar cells, but the overall power conversion efficiencies (PCEs) have been limited by the high carrier ...concentration from the facile oxidation of Sn2+ to Sn4+. Now a chemical route is developed for fabrication of high‐quality methylammonium tin iodide perovskite (MASnI3) films: hydrazinium tin iodide (HASnI3) perovskite film is first solution‐deposited using presursors hydrazinium iodide (HAI) and tin iodide (SnI2), and then transformed into MASnI3 via a cation displacement approach. With the two‐step process, a dense and uniform MASnI3 film is obtained with large grain sizes and high crystallization. Detrimental oxidation is suppressed by the hydrazine released from the film during the transformation. With the MASnI3 as light harvester, mesoporous perovskite solar cells were prepared, and a maximum power conversion efficiency (PCE) of 7.13 % is delivered with good reproducibility.
High‐quality, pinhole‐free CH3NH3SnI3 films are achieved from pristine NH2NH3SnI3 perovskite, and the oxidation of Sn2+ to Sn4+ can be efficiently suppressed owing to the reduction agent hydrazine generated inside the films in the conversion. With the CH3NH3SnI3 film as light absorber, mesoporous MASnI3 perovskite solar cells were fabricated with a maximum PCE of 7.13 %.
Full text
Available for:
BFBNIB, FZAB, GIS, IJS, KILJ, NLZOH, NUK, OILJ, SAZU, SBCE, SBMB, UL, UM, UPUK
Driven by the persisting poor understanding of the sluggish kinetics of the hydrogen evolution reaction (HER) on Pt in alkaline media, a direct correlation of the interfacial water structure and ...activity is still yet to be established. Herein, using Pt and Pt–Ni nanoparticles we first demonstrate a strong dependence of the proton donor structure on the HER activity and pH. The structure of the first layer changes from the proton acceptors to the donors with increasing pH. In the base, the reactivity of the interfacial water varied its structure, and the activation energies of water dissociation increased in the sequence: the dangling O−H bonds < the trihedrally coordinated water < the tetrahedrally coordinated water. Moreover, optimizing the adsorption of H and OH intermediates can re‐orientate the interfacial water molecules with their H atoms pointing towards the electrode surface, thereby enhancing the kinetics of HER. Our results clarified the dynamic role of the water structure at the electrode–electrolyte interface during HER and the design of highly efficient HER catalysts.
On nickel–platinum alloy nanoparticles under alkaline conditions, the reactivity of interfacial water varies with its structure and the order of water dissociation. The inclusion of nickel re‐orientates interfacial water molecules with their hydrogen atoms pointing towards the electrode surface, thereby enhancing the kinetics of the hydrogen evolution reaction (HER).
Full text
Available for:
BFBNIB, FZAB, GIS, IJS, KILJ, NLZOH, NUK, OILJ, SAZU, SBCE, SBMB, UL, UM, UPUK
Recent evidences showed that long noncoding RNAs (lncRNAs) are frequently dysregulated and play important roles in various cancers. Clear cell renal cell carcinoma (ccRCC) is one of the leading cause ...of cancer-related death, largely due to the metastasis of ccRCC. However, the clinical significances and roles of lncRNAs in metastatic ccRCC are still unknown.
lncRNA expression microarray analysis was performed to search the dysregulated lncRNA in metastatic ccRCC. quantitative real-time PCR was performed to measure the expression of lncRNAs in human ccRCC samples. Gain-of-function and loss-of-function experiments were performed to investigate the biological roles of lncRNAs on ccRCC cell proliferation, migration, invasion and in vivo metastasis. RNA pull-down, RNA immunoprecipitation, chromatin immunoprecipitation, and western blot were performed to explore the molecular mechanisms underlying the functions of lncRNAs.
The microarray analysis identified a novel lncRNA termed metastatic renal cell carcinoma-associated transcript 1 (MRCCAT1), which is highly expressed in metastatic ccRCC tissues and associated with the metastatic properties of ccRCC. Multivariate Cox regression analysis revealed that MRCCAT1 is an independent prognostic factor for ccRCC patients. Overexpression of MRCCAT1 promotes ccRCC cells proliferation, migration, and invasion. Depletion of MRCCAT1 inhibites ccRCC cells proliferation, migration, and invasion in vitro, and ccRCC metastasis in vivo. Mechanistically, MRCCAT1 represses NPR3 transcription by recruiting PRC2 to NPR3 promoter, and subsequently activates p38-MAPK signaling pathway.
MRCCAT1 is a critical lncRNA that promotes ccRCC metastasis via inhibiting NPR3 and activating p38-MAPK signaling. Our results imply that MRCCAT1 could serve as a prognostic biomarker and therapeutic target for ccRCC.
Full text
Available for:
DOBA, IZUM, KILJ, NUK, PILJ, PNG, SAZU, SIK, UILJ, UKNU, UL, UM, UPUK
PDF
