•We propose a novel ensemble model for bankruptcy prediction.•We use Extreme Gradient Boosting as an ensemble of decision trees.•We propose a new approach for generating synthetic features to improve ...prediction.•The presented method is evaluated on real-life data of Polish companies.
Bankruptcy prediction has been a subject of interests for almost a century and it still ranks high among hottest topics in economics. The aim of predicting financial distress is to develop a predictive model that combines various econometric measures and allows to foresee a financial condition of a firm. In this domain various methods were proposed that were based on statistical hypothesis testing, statistical modeling (e.g., generalized linear models), and recently artificial intelligence (e.g., neural networks, Support Vector Machines, decision tress). In this paper, we propose a novel approach for bankruptcy prediction that utilizes Extreme Gradient Boosting for learning an ensemble of decision trees. Additionally, in order to reflect higher-order statistics in data and impose a prior knowledge about data representation, we introduce a new concept that we refer as to synthetic features. A synthetic feature is a combination of the econometric measures using arithmetic operations (addition, subtraction, multiplication, division). Each synthetic feature can be seen as a single regression model that is developed in an evolutionary manner. We evaluate our solution using the collected data about Polish companies in five tasks corresponding to the bankruptcy prediction in the 1st, 2nd, 3rd, 4th, and 5th year. We compare our approach with the reference methods.
Deep generative architectures provide a way to model not only images but also complex, 3-dimensional objects, such as point clouds. In this work, we present a novel method to obtain meaningful ...representations of 3D shapes that can be used for challenging tasks, including 3D points generation, reconstruction, compression, and clustering. Contrary to existing methods for 3D point cloud generation that train separate decoupled models for representation learning and generation, our approach is the first end-to-end solution that allows to simultaneously learn a latent space of representation and generate 3D shape out of it. Moreover, our model is capable of learning meaningful compact binary descriptors with adversarial training conducted on a latent space. To achieve this goal, we extend a deep Adversarial Autoencoder model (AAE) to accept 3D input and create 3D output. Thanks to our end-to-end training regime, the resulting method called 3D Adversarial Autoencoder (3dAAE) obtains either binary or continuous latent space that covers a much broader portion of training data distribution. Finally, our quantitative evaluation shows that 3dAAE provides state-of-the-art results for 3D points clustering and 3D object retrieval.
•Comprehensive studies of VAEs and AAEs in context of 3D point cloud generation.•Selecting proper cost function for EMD achieves proper ELBO for variational calculus.•We show that AAE can learn 3D points generation, retrieval, and clustering.•Extending AAE framework to obtain compact representation of 3D point clouds.
Abstract
Motivation
Automated selection of signals in protein NMR spectra, known as peak picking, has been studied for over 20 years, nevertheless existing peak picking methods are still largely ...deficient. Accurate and precise automated peak picking would accelerate the structure calculation, and analysis of dynamics and interactions of macromolecules. Recent advancement in handling big data, together with an outburst of machine learning techniques, offer an opportunity to tackle the peak picking problem substantially faster than manual picking and on par with human accuracy. In particular, deep learning has proven to systematically achieve human-level performance in various recognition tasks, and thus emerges as an ideal tool to address automated identification of NMR signals.
Results
We have applied a convolutional neural network for visual analysis of multidimensional NMR spectra. A comprehensive test on 31 manually annotated spectra has demonstrated top-tier average precision (AP) of 0.9596, 0.9058 and 0.8271 for backbone, side-chain and NOESY spectra, respectively. Furthermore, a combination of extracted peak lists with automated assignment routine, FLYA, outperformed other methods, including the manual one, and led to correct resonance assignment at the levels of 90.40%, 89.90% and 90.20% for three benchmark proteins.
Availability and implementation
The proposed model is a part of a Dumpling software (platform for protein NMR data analysis), and is available at https://dumpling.bio/.
Supplementary information
Supplementary data are available at Bioinformatics online.
We introduce NodeFlow, a flexible framework for probabilistic regression on tabular data that combines Neural Oblivious Decision Ensembles (NODEs) and Conditional Continuous Normalizing Flows (CNFs). ...It offers improved modeling capabilities for arbitrary probabilistic distributions, addressing the limitations of traditional parametric approaches. In NodeFlow, the NODE captures complex relationships in tabular data through a tree-like structure, while the conditional CNF utilizes the NODE’s output space as a conditioning factor. The training process of NodeFlow employs standard gradient-based learning, facilitating the end-to-end optimization of the NODEs and CNF-based density estimation. This approach ensures outstanding performance, ease of implementation, and scalability, making NodeFlow an appealing choice for practitioners and researchers. Comprehensive assessments on benchmark datasets underscore NodeFlow’s efficacy, revealing its achievement of state-of-the-art outcomes in multivariate probabilistic regression setup and its strong performance in univariate regression tasks. Furthermore, ablation studies are conducted to justify the design choices of NodeFlow. In conclusion, NodeFlow’s end-to-end training process and strong performance make it a compelling solution for practitioners and researchers. Additionally, it opens new avenues for research and application in the field of probabilistic regression on tabular data.
•We propose a comprehensible model for credit risk assessment using a scoring table.•We use Restricted Boltzmann Machine to determine scoring points in a scoring table.•We deal with the imbalanced ...data by applying the geometric mean criterion.•The quality of the presented method is evaluated on four credit scoring datasets.
Credit scoring is the assessment of the risk associated with a consumer (an organization or an individual) that apply for the credit. Therefore, the problem of credit scoring can be stated as a discrimination between those applicants whom the lender is confident will repay credit and those applicants who are considered by the lender as insufficiently reliable. In this work we propose a novel method for constructing comprehensible scoring model by applying Classification Restricted Boltzmann Machines (ClassRBM). In the first step we train the ClassRBM as a standalone classifier that has ability to predict credit status but does not contain interpretable structure. In order to obtain comprehensible model, first we evaluate the relevancy of each of binary features using ClassRBM and further we use these values to create the scoring table (scorecard). Additionally, we deal with the imbalanced data issue by proposing a procedure for determining the cutting point using the geometric mean of specificity and sensitivity. We evaluate our approach by comparing its performance with the results gained by other methods using four datasets from the credit scoring domain.
Contemporary deep neural networks offer state-of-the-art results when applied to visual reasoning, e.g., in the context of 3D point cloud data. Point clouds are an important data type for the precise ...modeling of three-dimensional environments, but effective processing of this type of data proves to be challenging. In the world of large, heavily-parameterized network architectures and continuously-streamed data, there is an increasing need for machine learning models that can be trained on additional data. Unfortunately, currently available models cannot fully leverage training on additional data without losing their past knowledge. Combating this phenomenon, called catastrophic forgetting, is one of the main objectives of continual learning. Continual learning for deep neural networks has been an active field of research, primarily in 2D computer vision, natural language processing, reinforcement learning, and robotics. However, in 3D computer vision, there are hardly any continual learning solutions specifically designed to take advantage of point cloud structure. This work proposes a novel neural network architecture capable of continual learning on 3D point cloud data. We utilize point cloud structure properties for preserving a heavily compressed set of past data. By using rehearsal and reconstruction as regularization methods of the learning process, our approach achieves a significant decrease of catastrophic forgetting compared to the existing solutions on several most popular point cloud datasets considering two continual learning settings: when a task is known beforehand, and in the challenging scenario of when task information is unknown to the model.
•We introduce a new continual learning model designed for 2D & 3D point cloud data.•For rehearsal purposes, we utilize only a tiny portion of the original data.•We can significantly mitigate the catastrophic forgetting for up to 20 tasks.•The experiments show superb results on the most popular benchmark datasets.
Few-Shot learning aims to train models which can adapt to previously unseen tasks based on small amounts of data. One of the leading Few-Shot learning approaches is Model-Agnostic-Meta-Learning ...(MAML), which learns the general weights of the meta-model that are later adapted to downstream tasks. However, MAML’s main limitation lies in that the update procedure is realized by gradient-based optimization, which cannot always modify weights to the essential level in one or even a few iterations. Moreover, using many gradient steps results in time-consuming optimization and inference procedures. In this paper, we propose HyperMAML, a novel generalization of MAML, where the update procedure is also a part of the model. Namely, we replace gradient descent with a trainable Hypernetwork which updates the weights. Consequently, the model can generate significant updates whose range is not limited to a fixed number of gradient steps. Experiments show that HyperMAML outperforms MAML in most cases and performs comparably to state-of-the-art techniques in standard Few-Shot learning benchmarks.
•We propose a Few-Shot learning model that produces task-specific parameter updates.•HyperMAML does not require loss calculation or backpropagation to update parameters.•Our approach offers Few-Shot accuracy superior to MAML and its numerous variants.
Our study addresses the need for universal monitoring solutions given the diverse environmental impacts of surface mining operations. We present a solution combining remote sensing and machine ...learning techniques, utilizing a dataset of over 2000 satellite images annotated with ten distinct labels indicating mining area components. We tested various approaches to develop comprehensive yet universal machine learning models for mining area segmentation. This involved considering different types of mines, raw materials, and geographical locations. We evaluated multiple satellite data set combinations to determine optimal outcomes. The results suggest that radar and multispectral data fusion did not significantly improve the models’ performance, and the addition of further channels led to the degradation of the metrics. Despite variations in mine type or extracted material, the models’ effectiveness remained within an Intersection over Union value range of 0.65–0.75. Further, in this research, we conducted a detailed visual analysis of the models’ outcomes to identify areas requiring additional attention, contributing to the discourse on effective mining area monitoring and management methodologies. The visual examination of models’ outputs provides insights for future model enhancement and highlights unique segmentation challenges within mining areas.
•We propose a novel generative model for 3D point clouds.•The model utilizes two normalizing flows - one produces an object descriptor and conditions the other to produce the shape.•Our method is ...insensitive to the ordering and cardinality and reduces training speed in comparison to other approaches.•Performed experiments show that the proposed model is a new state-of-the-art method for point cloud generation.
In this paper, we propose a simple yet effective method to represent point clouds as sets of samples drawn from a cloud-specific probability distribution. This interpretation matches intrinsic characteristics of point clouds: the number of points and their ordering within a cloud is not important as all points are drawn from the proximity of the object boundary. We postulate to represent each cloud as a parameterized probability distribution of points in space, which is defined by a generative neural network. The network operates by composing several spatial transformations of point locations. Once trained, it provides a natural framework for point cloud manipulation. For instance we can decouple cloud shape from its orientation and provide routines for aligning a new cloud into a default spatial orientation. To exploit similarities between same-class objects and to improve model performance, we turn to weight sharing: networks that model densities of points belonging to objects in the same family share all parameters with the exception of a small, object-specific embedding vector. We show that these embedding vectors capture semantic relationships between objects. Our method leverages generative invertible flow networks to learn embeddings as well as to generate point clouds. Thanks to this formulation and contrary to similar approaches, we are able to train our model in an end-to-end fashion. As a result, our model offers competitive or superior quantitative results on benchmark datasets, while enabling unprecedented capabilities to perform cloud manipulation tasks, such as point cloud registration and regeneration, by a generative network.
PDF
