Deep neural networks (DNNs) have achieved state-of-the-art performance in a number of domains but suffer intensive complexity. Network quantization can effectively reduce computation and memory costs ...without changing network structure, facilitating the deployment of DNNs on mobile devices. While the existing methods can obtain good performance, low-bit quantization without time-consuming training or access to the full dataset is still a challenging problem. In this paper, we develop a novel method named Compressorbased non-uniform quantization (CNQ) method to achieve non-uniform quantization of DNNs with few unlabeled samples. Firstly, we present a compressor-based fast nonuniform quantization method, which can accomplish nonuniform quantization without iterations. Secondly, we propose to align the feature maps of the quantization model with the pre-trained model for accuracy recovery. Considering the property difference between different activation channels, we utilize the weighted-entropy perchannel to optimize the alignment loss. In the experiments, we evaluate the proposed method on image classification and object detection. Our results outperform the existing post-training quantization methods, which demonstrate the effectiveness of the proposed method.
We consider the commutative limit of matrix geometry described by a large-N sequence of some Hermitian matrices. Under some assumptions, we show that the commutative geometry possesses a Kaehler ...structure. We find an explicit relation between the Kaehler structure and the matrix configurations which define the matrix geometry. We also discuss a relation between the matrix configurations and those obtained from the geometric quantization.
The post-training quantization (PTQ) is a common technology to improve the efficiency of embedded neural network accelerators. Existing PTQ schemes for CNN activations usually rely on calibration ...dataset with good data representation to reduce quantization overflow in inference, which is not always effective due to large variation and uncertainty of the inference input data in practice. This paper proposes an adaptive PTQ method for activations (AQA), which monitors the quantization overflow of activations, adaptively updates the quantization parameters, and re-quantizes the activations on-the-fly when the overflow degree is over a threshold. The key challenges in implementing the AQA method are to reduce the associated side-effects in increasing computational complexity, processing time and hardware resource usage. We propose a series of design optimizations for quantization overflow monitor, quantization parameters update and re-quantization to successfully address the challenges. The proposed AQA method is implemented in a CNN accelerator and evaluated on VGG16, ResNet18 and MobileNetV2 on several datasets. Experiment results show that the adaptation method makes the models' inference accuracy stable over various quantization overflow degrees, while the static quantization method suffers from significant accuracy degradation. The costs introduced by the adaptation method include 5% power consumption increase and 4% throughput degradation.
Traditional deep learning models are trained at a centralized server using data samples collected from users. Such data samples often include private information, which the users may not be willing ...to share. Federated learning (FL) is an emerging approach to train such learning models without requiring the users to share their data. FL consists of an iterative procedure, where in each iteration the users train a copy of the learning model locally. The server then collects the individual updates and aggregates them into a global model. A major challenge that arises in this method is the need of each user to repeatedly transmit its learned model over the throughput limited uplink channel. In this work, we tackle this challenge using tools from quantization theory. In particular, we identify the unique characteristics associated with conveying trained models over rate-constrained channels, and propose a suitable quantization scheme for such settings, referred to as universal vector quantization for FL (UVeQFed). We show that combining universal vector quantization methods with FL yields a decentralized training system in which the compression of the trained models induces only a minimum distortion. We then theoretically analyze the distortion, showing that it vanishes as the number of users grows. We also characterize how models trained with conventional federated averaging combined with UVeQFed converge to the model which minimizes the loss function. Our numerical results demonstrate the gains of UVeQFed over previously proposed methods in terms of both distortion induced in quantization and accuracy of the resulting aggregated model.
The novel Computing-In-Memory (CIM) technology has demonstrated significant potential in enhancing the performance and efficiency of convolutional neural networks (CNNs). However, due to the low ...precision of memory devices and data interfaces, an additional quantization step is necessary. Conventional NN quantization methods fail to account for the hardware characteristics of CIM, resulting in inferior system performance and efficiency. This paper proposes CIMQ, a hardware-efficient quantization framework designed to improve the efficiency of CIM based NN accelerators. The holistic framework focuses on the fundamental computing elements in CIM hardware: inputs, weights and outputs (or activations, weights and partial sums in NNs) with four innovative techniques. Firstly, bit-level sparsity induced activation quantization is introduced to decrease dynamic computation energy. Secondly, inspired by the unique computation paradigm of CIM, an innovative array-wise quantization granularity is proposed for weight quantization. Thirdly, partial sums are quantized with a reparametrized clipping function to reduce the required resolution of analog-to-digital converters (ADCs). Finally, to improve the accuracy of quantized neural networks (QNNs), the post-training quantization (PTQ) is enhanced with a random quantization dropping strategy. The effectiveness of the proposed framework has been demonstrated through experimental results on various NNs and datasets (CIFAR10, CIFAR100, ImageNet). In typical cases, the hardware efficiency can be improved up to 222% with a 58.97% improvement in accuracy compared to conventional quantization methods.
For uniform scalar quantization, the error distribution is approximately a uniform distribution over an interval (which is also a 1-dimensional ball). Nevertheless, for lattice vector quantization, ...the error distribution is uniform not over a ball, but over the basic cell of the quantization lattice. In this paper, we construct vector quantizers with periodic properties, where the error is uniformly distributed over the n-ball, or any other prescribed set. We then prove upper and lower bounds on the entropy of the quantized signals. We also discuss how our construction can be applied to give a randomized quantization scheme with a nonuniform error distribution.
The quantization of the output of a binary-input discrete memoryless channel to a smaller number of levels is considered. An algorithm, which finds an optimal quantizer, in the sense of maximizing ...mutual information between the channel input and quantizer output is given. This result holds for arbitrary channels, in contrast to previous results for restricted channels or a restricted number of quantizer outputs. In the worst case, the algorithm complexity is cubic M 3 in the number of channel outputs M. Optimality is proved using the theorem of Burshtein, Della Pietra, Kanevsky, and Nádas for mappings, which minimize average impurity for classification and regression trees.
This paper investigates the performance of limited-fronthaul cell-free massive multiple-input multiple-output (MIMO) taking account the fronthaul quantization and imperfect channel acquisition. Three ...cases are studied, which we refer to as Estimate&Quantize, Quantize&Estimate, and Decentralized, according to where channel estimation is performed and exploited. Maximum-ratio combining (MRC), zero-forcing (ZF), and minimum mean-square error (MMSE) receivers are considered. The Max algorithm and the Bussgang decomposition are exploited to model optimum uniform quantization. Exploiting the optimal step size of the quantizer, analytical expressions for spectral and energy efficiencies are presented. Finally, an access point (AP) assignment algorithm is proposed to improve the performance of the decentralized scheme. Numerical results investigate the performance gap between limited fronthaul and perfect fronthaul cases, and demonstrate that exploiting relatively few quantization bits, the performance of limited-fronthaul cell-free massive MIMO closely approaches the perfect-fronthaul performance.
On the Best Lattice Quantizers Agrell, Erik; Allen, Bruce
IEEE transactions on information theory,
12/2023, Letnik:
69, Številka:
12
Journal Article
Recenzirano
Odprti dostop
A lattice quantizer approximates an arbitrary real-valued source vector with a vector taken from a specific discrete lattice. The quantization error is the difference between the source vector and ...the lattice vector. In a classic 1996 paper, Zamir and Feder show that the globally optimal lattice quantizer (which minimizes the mean square error) has white quantization error: for a uniformly distributed source, the covariance of the error is the identity matrix, multiplied by a positive real factor. We generalize the theorem, showing that the same property holds (i) for any lattice whose mean square error cannot be decreased by a small perturbation of the generator matrix, and (ii) for an optimal product of lattices that are themselves locally optimal in the sense of (i). We derive an upper bound on the normalized second moment (NSM) of the optimal lattice in any dimension, by proving that any lower- or upper-triangular modification to the generator matrix of a product lattice reduces the NSM. Using these tools and employing the best currently known lattice quantizers to build product lattices, we construct improved lattice quantizers in dimensions 13 to 15, 17 to 23, and 25 to 48. In some dimensions, these are the first reported lattices with normalized second moments below the best known upper bound.
PDF
