Learning 3D global features by aggregating multiple views has been introduced as a successful strategy for 3D shape analysis. In recent deep learning models with end-to-end training, pooling is a ...widely adopted procedure for view aggregation. However, pooling merely retains the max or mean value over all views, which disregards the content information of almost all views and also the spatial information among the views. To resolve these issues, we propose Sequential Views To Sequential Labels (SeqViews2SeqLabels) as a novel deep learning model with an encoder-decoder structure based on recurrent neural networks (RNNs) with attention. SeqViews2SeqLabels consists of two connected parts, an encoder-RNN followed by a decoder-RNN, that aim to learn the global features by aggregating sequential views and then performing shape classification from the learned global features, respectively. Specifically, the encoder-RNN learns the global features by simultaneously encoding the spatial and content information of sequential views, which captures the semantics of the view sequence. With the proposed prediction of sequential labels, the decoder-RNN performs more accurate classification using the learned global features by predicting sequential labels step by step. Learning to predict sequential labels provides more and finer discriminative information among shape classes to learn, which alleviates the overfitting problem inherent in training using a limited number of 3D shapes. Moreover, we introduce an attention mechanism to further improve the discriminative ability of SeqViews2SeqLabels. This mechanism increases the weight of views that are distinctive to each shape class, and it dramatically reduces the effect of selecting the first view position. Shape classification and retrieval results under three large-scale benchmarks verify that SeqViews2SeqLabels learns more discriminative global features by more effectively aggregating sequential views than state-of-the-art methods.
Extracellular vesicles (EVs) contain specific proteins, lipids, and nucleic acids that can be passed to other cells as signal molecules to alter their function. However, there are many problems and ...challenges in the conversion and clinical application of EVs. Storage and protection of EVs is one of the issues that need further research. To adapt to potential clinical applications, this type of problem must be solved. This review summarizes the storage practices of EVs in recent years, and explains the impact of temperature on the quality and stability of EVs during storage based on current research, and explains the potential mechanisms involved in this effect as much as possible.
Extracellular vesicles (EVs) contain specific proteins, lipids, and nucleic acids that can be passed to other cells as signal molecules to alter their function. However, there are many problems and challenges in the conversion and clinical application of EVs. Storage and protection of EVs is one of the issues that need further research. To adapt to potential clinical applications, this type of problem must be solved. Here, we briefly review EVs' biogenesis, contents, subtypes and effect of storage temperature on its quality and stability, as well as application of cryoprotectants in EVs cryopreservation, and specifically focus on the mechanism by which storage temperature affects the quality and stability of EVs.
Fine-grained 3D shape classification is important for shape understanding and analysis, which poses a challenging research problem. However, the studies on the fine-grained 3D shape classification ...have rarely been explored, due to the lack of fine-grained 3D shape benchmarks. To address this issue, we first introduce a new 3D shape dataset (named FG3D dataset) with fine-grained class labels, which consists of three categories including airplane, car and chair. Each category consists of several subcategories at a fine-grained level. According to our experiments under this fine-grained dataset, we find that state-of-the-art methods are significantly limited by the small variance among subcategories in the same category. To resolve this problem, we further propose a novel fine-grained 3D shape classification method named FG3D-Net to capture the fine-grained local details of 3D shapes from multiple rendered views. Specifically, we first train a Region Proposal Network (RPN) to detect the generally semantic parts inside multiple views under the benchmark of generally semantic part detection. Then, we design a hierarchical part-view attention aggregation module to learn a global shape representation by aggregating generally semantic part features, which preserves the local details of 3D shapes. The part-view attention module hierarchically leverages part-level and view-level attention to increase the discriminability of our features. The part-level attention highlights the important parts in each view while the view-level attention highlights the discriminative views among all the views of the same object. In addition, we integrate a Recurrent Neural Network (RNN) to capture the spatial relationships among sequential views from different viewpoints. Our results under the fine-grained 3D shape dataset show that our method outperforms other state-of-the-art methods. The FG3D dataset is available at https://github.com/liuxinhai/FG3D-Net.
The task of point cloud upsampling aims to acquire dense and uniform point sets from sparse and irregular point sets. Although significant progress has been made with deep learning models, ...state-of-the-art methods require ground-truth dense point sets as the supervision, which makes them limited to be trained under synthetic paired training data and not suitable to be under real-scanned sparse data. However, it is expensive and tedious to obtain large numbers of paired sparse-dense point sets as supervision from real-scanned sparse data. To address this problem, we propose a self-supervised point cloud upsampling network, named SPU-Net, to capture the inherent upsampling patterns of points lying on the underlying object surface. Specifically, we propose a coarse-to-fine reconstruction framework, which contains two main components: point feature extraction and point feature expansion, respectively. In the point feature extraction, we integrate the self-attention module with the graph convolution network (GCN) to capture context information inside and among local regions simultaneously. In the point feature expansion, we introduce a hierarchically learnable folding strategy to generate upsampled point sets with learnable 2D grids. Moreover, to further optimize the noisy points in the generated point sets, we propose a novel self-projection optimization associated with uniform and reconstruction terms as a joint loss to facilitate the self-supervised point cloud upsampling. We conduct various experiments on both synthetic and real-scanned datasets, and the results demonstrate that we achieve comparable performances to state-of-the-art supervised methods.
3D shape reconstruction from multiple hand-drawn sketches is an intriguing way to 3D shape modeling. Currently, state-of-the-art methods employ neural networks to learn a mapping from multiple ...sketches from arbitrary view angles to a 3D voxel grid. Because of the cubic complexity of 3D voxel grids, however, neural networks are hard to train and limited to low resolution reconstructions, which leads to a lack of geometric detail and low accuracy. To resolve this issue, we propose to reconstruct 3D shapes from multiple sketches using direct shape optimization (DSO), which does not involve deep learning models for direct voxel-based 3D shape generation. Specifically, we first leverage a conditional generative adversarial network (CGAN) to translate each sketch into an attenuance image that captures the predicted geometry from a given viewpoint. Then, DSO minimizes a project-and-compare loss to reconstruct the 3D shape such that it matches the predicted attenuance images from the view angles of all input sketches. Based on this, we further propose a progressive update approach to handle inconsistencies among a few hand-drawn sketches for the same 3D shape. Our experimental results show that our method significantly outperforms the state-of-the-art methods under widely used benchmarks and produces intuitive results in an interactive application.
Learning discriminative shape representation directly on point clouds is still challenging in 3D shape analysis and understanding. Recent studies usually involve three steps: first splitting a point ...cloud into some local regions, then extracting the corresponding feature of each local region, and finally aggregating all individual local region features into a global feature as shape representation using simple max-pooling. However, such pooling-based feature aggregation methods do not adequately take the spatial relationships (e.g. the relative locations to other regions) between local regions into account, which greatly limits the ability to learn discriminative shape representation. To address this issue, we propose a novel deep learning network, named Point2SpatialCapsule, for aggregating features and spatial relationships of local regions on point clouds, which aims to learn more discriminative shape representation. Compared with the traditional max-pooling based feature aggregation networks, Point2SpatialCapsule can explicitly learn not only geometric features of local regions but also the spatial relationships among them. Point2SpatialCapsule consists of two main modules. To resolve the disorder problem of local regions, the first module, named geometric feature aggregation , is designed to aggregate the local region features into the learnable cluster centers, which explicitly encodes the spatial locations from the original 3D space. The second module, named spatial relationship aggregation , is proposed for further aggregating the clustered features and the spatial relationships among them in the feature space using the spatial-aware capsules developed in this article. Compared to the previous capsule network based methods, the feature routing on the spatial-aware capsules can learn more discriminative spatial relationships among local regions for point clouds, which establishes a direct mapping between log priors and the spatial locations through feature clusters. Experimental results demonstrate that Point2SpatialCapsule outperforms the state-of-the-art methods in the 3D shape classification, retrieval and segmentation tasks under the well-known ModelNet and ShapeNet datasets.
Efficient electron transfer from photosensitizer to catalytic sites is crucial for effective artificial photosynthesis, yet it remains a significant challenge. Herein, it is reported that simple ...fluorination of the organic linkers in the MIL‐101(Fe) photocatalyst results in a remarkable threefold increase in the photocatalytic CO2‐to‐CO conversion rate (688 µmol g−1 h−1) compared to the pristine counterpart (230 µmol g−1 h−1). It is unveiled that, instead of directly modifying the electron structure of MIL‐101(Fe), the fluorinated linkers enhance the interaction between the discrete photocatalyst and photosensitizer (Ru(bpy)32+, bpy = 2,2'‐bipyridine) via hydrogen bonding, thereby facilitating their intermolecular electron transfer. Most importantly, it is also demonstrated that this performance boosting strategy can be applied to other Fe‐based metal–organic frameworks (MOFs) photocatalysts such as MIL‐53(Fe) and MIL‐88(Fe). The present work not only underscores the fluorination of organic linkers as a generic promising approach to enhance the photocatalytic performance of MOF‐based catalysts, but also holds significant implications for photosynthesis and catalytic processes reliant on intermolecular electron transfer as an important step.
This study demonstrates a threefold increase in the photocatalytic CO2‐to‐CO conversion rate by incorporating fluorinated linkers in Fe‐based metal organic framework photocatalysts. The enhanced electron transfer is attributed to strengthened interaction via hydrogen bonding between the photocatalyst and photosensitizer. This strategy holds promise for developing high‐performance photocatalysts.
Cross-modal retrieval using deep neural networks aims to retrieve relevant data between the two different modalities. The performance of cross-modal retrieval is still unsatisfactory for two ...problems. First, most of the previous methods failed to incorporate the common knowledge among modalities when predicting the item representations. Second, the semantic relationships indicated by class label are still insufficiently utilized, which is an important clue for inferring similarities between the cross modal items. To address the above issues, we propose a novel cross memory network with pair discrimination (CMPD) for image-text cross modal retrieval, where the main contributions lie in two-folds. First, we propose the cross memory as a set of latent concepts to capture the common knowledge among different modalities. It is learnable and can be fused into each modality through attention mechanism, which aims to discriminatively predict representations. Second, we propose the pair discrimination loss to discriminate modality labels and class labels of item pairs, which can efficiently capture the semantic relationships among these modality labels and class labels. Comprehensive experimental results show that our method outperforms the state-of-the-art approaches in image-text retrieval.
Learning 3D global features by aggregating multiple views is important. Pooling is widely used to aggregate views in deep learning models. However, pooling disregards a lot of content information ...within views and the spatial relationship among the views, which limits the discriminability of learned features. To resolve this issue, 3D to Sequential Views (3D2SeqViews) is proposed to more effectively aggregate the sequential views using convolutional neural networks with a novel hierarchical attention aggregation. Specifically, the content information within each view is first encoded. Then, the encoded view content information and the sequential spatiality among the views are simultaneously aggregated by the hierarchical attention aggregation, where view-level attention and class-level attention are proposed to hierarchically weight sequential views and shape classes. View-level attention is learned to indicate how much attention is paid to each view by each shape class, which subsequently weights sequential views through a novel recursive view integration. Recursive view integration learns the semantic meaning of view sequence, which is robust to the first view position. Furthermore, class-level attention is introduced to describe how much attention is paid to each shape class, which innovatively employs the discriminative ability of the fine-tuned network. 3D2SeqViews learns more discriminative features than the state-of-the-art, which leads to the outperforming results in shape classification and retrieval under three large-scale benchmarks.
As the largest container and resource of metals, sediment has a special role in the fate of metals. Factors influencing bioavailability of heavy metals in sediment have never been comprehensively ...considered and the sediment properties still fail to understand and even controversial. In this review, the mechanisms of sediment properties such as acid-volatile sulfides (AVS), organic matter, texture (clay, silt or sand) and geology, organism behaviors as well as those influencing the bioavailability of metals were analyzed. Under anoxic condition, AVS mainly reduce the solubility and toxicity of metals, while organic matters, Fe–Mn oxides, clay or silt can stabilize heavy metals in elevated oxidative–reductive potential (ORP). Other factors including the variation of pH, redox potential, aging as well as nutrition and the behavior of benthic organism in sediment also largely alter metals mobility and distribution. These factors are often inter-related, and various toxicity assessment methods used to evaluate the bioavailability of trace metals have been also discussed. Additionally, we expect that some novel synthetic materials like polysulfides, nano-materials, provide the substantial amendments for metals pollution in sediment.
•The influencing mechanisms of sediment geochemical properties on heavy metals bioavailability were analyzed.•AVS stabilize heavy metals in anoxic state but OM and Fe-Mn oxides immobilize metals under aerobic condition.•Considering the sediment geochemical properties is needed when establish the model of bioavailability.
PDF
