For prognostics and health management of industrial systems, machine remaining useful life (RUL) prediction is an essential task. While deep learning-based methods have achieved great successes in ...RUL prediction tasks, large-scale neural networks are still difficult to deploy on edge devices owing to the constraints of memory capacity and computing power. In this article, we propose a lightweight and adaptive knowledge distillation (KD) framework to alleviate this problem. First, multiple teacher models are compressed into a student model through KD to improve the industrial prediction accuracy. Second, a dynamic exiting method is studied to enable an adaptive inference on the distilled student model. Finally, we develop a reparameterization scheme to further lessen the student network. Experiments on two turbofan engine degradation datasets and a bearing degradation dataset demonstrate that our method significantly outperforms the state-of-the-art KD methods and enables the distilled model with an adaptive inference ability.
This paper presents a novel jerk minimization algorithm in the context of multi-axis flank CNC machining. The toolpath of the milling axis in a flank milling process, a ruled surface, is ...reparameterized by a B-spline function, whose control points and knot vector are unknowns in an optimization-based framework. The total jerk of the tool’s motion is minimized, implying the tool is moving as smooth as possible, without changing the geometry of the given toolpath. Our initialization stage stems from measuring the ruling distance metric (RDM) of the ruled surface. We show on several examples that this initialization reliably finds close initial guesses of jerk-minimizers and is also computationally efficient. The applicability of the presented approach is illustrated by some practical case studies.
Display omitted
•We present a novel jerk minimization algorithm for multi-axis flank CNC machining.•The toolpath of the milling axis is reparameterized by a B-spline function.•The jerk of the motion is optimized without changing the geometry of the toolpath.•Our initialization stems from measuring the ruling distance metric of the toolpath.•The applicability of the presented method is illustrated by several case studies.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Designing efficient deep learning models for 3D point clouds is an important research topic. Point-voxel convolution (Liu et al. in NeurIPS, 2019) is a pioneering approach in this direction, but it ...still has considerable room for improvement in terms of performance, since it has quite a few layers of simple 3D convolutions and linear point-voxel feature fusion operations. To resolve these issues, we propose a novel reparameterizable point-voxel convolution (RepPVConv) block. First, RepPVConv adopts two reparameterizable 3D convolution modules to extract more informative voxel features without introducing any extra computational overhead for inference. The rationale is that the reparameterizable 3D convolution modules are trained in high-capacity modes but are reparameterized into low-capacity modes during inference while losslessly maintaining the original performance. Second, RepPVConv attentively fuses the reparameterized voxel features with those of points. Since the proposed approach operates in a nonlinear manner, descriptive reparameterized voxel features can be better utilized. Extensive experimental results show that RepPVConv-based networks are efficient in terms of both GPU memory consumption and computational complexity and significantly outperform the state-of-the-art methods.
Full text
Available for:
EMUNI, FIS, FZAB, GEOZS, GIS, IJS, IMTLJ, KILJ, KISLJ, MFDPS, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, SBMB, SBNM, UKNU, UL, UM, UPUK, VKSCE, ZAGLJ
It is hard to accomplish fast semantic segmentation on large remote sensing images, since current neural networks with numerous parameters often rely on significant computational resources. Our team ...proposes an improved fast semantic segmentation model based on short-term dense-connection network (RepSTDC). We introduce a structure reparameterization and coordinate attention into STDC networks. By structure reparameterization, we transform the multi-branch structure into a comparable single-branch configuration during the inference process. By replacing the traditional channel attention with a coordinate attention mechanism, we enhance the attention mechanism with considering channel relationships and long-distance position information, and then it saves the memory usages. We conducted thorough experiments to assess the efficacy of network components of RepSTDC on the several benchmark datasets. Additionally, we compared our proposed approach with state-of-the-art methods. Our RepSTDC model can well balance the accuracy performances, computing speed, and memory usage in most cases. It achieves fast segmentation by significantly reducing parameters but without obviously compromising performances compared to other methods.
Full text
Available for:
BFBNIB, GIS, IJS, IZUM, KILJ, KISLJ, NUK, PILJ, PNG, SAZU, UL, UM, UPUK
Integrated solutions for analytics over relational databases are of great practical importance as they avoid the costly repeated loop data scientists have to deal with on a daily basis: select ...features from data residing in relational databases using feature extraction queries involving joins, projections, and aggregations; export the training dataset defined by such queries; convert this dataset into the format of an external learning tool; and train the desired model using this tool. These integrated solutions are also a fertile ground of theoretically fundamental and challenging problems at the intersection of relational and statistical data models.
This article introduces a unified framework for training and evaluating a class of statistical learning models over relational databases. This class includes ridge linear regression, polynomial regression, factorization machines, and principal component analysis. We show that, by synergizing key tools from database theory such as schema information, query structure, functional dependencies, recent advances in query evaluation algorithms, and from linear algebra such as tensor and matrix operations, one can formulate relational analytics problems and design efficient (query and data) structure-aware algorithms to solve them.
This theoretical development informed the design and implementation of the AC/DC system for structure-aware learning. We benchmark the performance of AC/DC against R, MADlib, libFM, and TensorFlow. For typical retail forecasting and advertisement planning applications, AC/DC can learn polynomial regression models and factorization machines with at least the same accuracy as its competitors and up to three orders of magnitude faster than its competitors whenever they do not run out of memory, exceed 24-hour timeout, or encounter internal design limitations.
Full text
Available for:
IZUM, KILJ, NUK, PILJ, SAZU, UL, UM, UPUK
Bayesian Deep Net GLM and GLMM Tran, M.-N.; Nguyen, N.; Nott, D. ...
Journal of computational and graphical statistics,
01/2020, Volume:
29, Issue:
1
Journal Article
Peer reviewed
Open access
Deep feedforward neural networks (DFNNs) are a powerful tool for functional approximation. We describe flexible versions of generalized linear and generalized linear mixed models incorporating basis ...functions formed by a DFNN. The consideration of neural networks with random effects is not widely used in the literature, perhaps because of the computational challenges of incorporating subject specific parameters into already complex models. Efficient computational methods for high-dimensional Bayesian inference are developed using Gaussian variational approximation, with a parsimonious but flexible factor parameterization of the covariance matrix. We implement natural gradient methods for the optimization, exploiting the factor structure of the variational covariance matrix in computation of the natural gradient. Our flexible DFNN models and Bayesian inference approach lead to a regression and classification method that has a high prediction accuracy, and is able to quantify the prediction uncertainty in a principled and convenient way. We also describe how to perform variable selection in our deep learning method. The proposed methods are illustrated in a wide range of simulated and real-data examples, and compare favorably to a state of the art flexible regression and classification method in the statistical literature, the Bayesian additive regression trees (BART) method. User-friendly software packages in Matlab, R, and Python implementing the proposed methods are available at
https://github.com/VBayesLab
.
Full text
Available for:
BFBNIB, GIS, IJS, KISLJ, NUK, PNG, UL, UM, UPUK
Inverse design algorithms are the basis for realizing high-performance, freeform nanophotonic devices. Current methods to enforce geometric constraints, such as practical fabrication constraints, are ...heuristic and not robust. In this work, we show that hard geometric constraints can be imposed on inverse-designed devices by reparameterizing the design space itself. Instead of evaluating and modifying devices in the physical device space, candidate device layouts are defined in a constraint-free latent space and mathematically transformed to the physical device space, which robustly imposes geometric constraints. Modifications to the physical devices, specified by inverse design algorithms, are made to their latent space representations using backpropagation. As a proof-of-concept demonstration, we apply reparameterization to enforce strict minimum feature size constraints in local and global topology optimizers for metagratings. We anticipate that concepts in reparameterization will provide a general and meaningful platform to incorporate physics and physical constraints in any gradient-based optimizer, including machine learning-enabled global optimizers.
Full text
Available for:
IJS, KILJ, NUK, PNG, UL, UM
The most puzzling aspect of the ‘strange metal’ behavior of correlated electron compounds is that the linear in temperature resistivity often extends down to low temperatures, lower than natural ...microscopic energy scales. We consider recently proposed deconfined critical points (or phases) in models of electrons in large dimension lattices with random nearest-neighbor exchange interactions. The criticality is in the class of Sachdev–Ye–Kitaev models, and exhibits a time reparameterization soft mode representing gravity in dual holographic theories. We compute the low temperature resistivity in a large M limit of models with SU(M) spin symmetry, and find that the dominant temperature dependence arises from this soft mode. The resistivity is linear in temperature down to zero temperature at the critical point, with a co-efficient universally proportional to the product of the residual resistivity and the co-efficient of the linear in temperature specific heat. We argue that the time reparameterization soft mode offers a promising and generic mechanism for resolving the strange metal puzzle.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Deep neural network models significantly outperform classical algorithms in the hyperspectral image (HSI) classification task. These deep models improve generalization but incur significant ...computational demands. This article endeavors to alleviate the computational distress in a depthwise manner through the use of morphological operations. We propose the adaptive morphology filter (AMF) to effectively extract spatial features like the conventional depthwise convolution layer. Furthermore, we reparameterize AMF into its equivalent form, i.e., a traditional binary morphology filter, which drastically reduces the number of parameters in the inference phase. Finally, we stack multiple AMFs to achieve a large receptive field and construct a lightweight AMNet for classifying HSIs. It is noteworthy that we prove the deep stack of depthwise AMFs to be equivalent to structural element decomposition. We test our model on five benchmark datasets. Experiments show that our approach outperforms state-of-the-art methods with fewer parameters (<inline-formula> <tex-math notation="LaTeX">{\approx }10 k </tex-math></inline-formula>). The codes will be publicly available at https://github.com/zhu-xlab/Adaptive-Morphology-Filter .
PDF
