•Impulsive controller is given for synchronization of delayed NNs with stochastic perturbation.•The impulsive controller is given via LMI technique.•Time delays may be nondifferentiable under ...stochastic NNs.
In this paper, an impulsive controller is designed to achieve the exponential synchronization of chaotic delayed neural networks with stochastic perturbation. By using the impulsive delay differential inequality technique that was established in recent publications, several sufficient conditions ensuring the exponential synchronization of chaotic delayed networks are derived, which can be easily checked by LMI Control Toolbox in Matlab. A numerical example and its simulation is given to demonstrate the effectiveness and advantage of the theory results.
Deep learning based semi-supervised learning (SSL) algorithms have led to promising results in recent years. However, they tend to introduce multiple tunable hyper-parameters, making them less ...practical in real SSL scenarios where the labeled data is scarce for extensive hyper-parameter search. In this paper, we propose a novel meta-learning based SSL algorithm (Meta-Semi) that requires tuning only one additional hyper-parameter, compared with a standard supervised deep learning algorithm, to achieve competitive performance under various conditions of SSL. We start by defining a meta optimization problem that minimizes the loss on labeled data through dynamically reweighting the loss on unlabeled samples, which are associated with soft pseudo labels during training. As the meta problem is computationally intensive to solve directly, we propose an efficient algorithm to dynamically obtain the approximate solutions. We show theoretically that Meta-Semi converges to the stationary point of the loss function on labeled data under mild conditions. Empirically, Meta-Semi outperforms state-of-the-art SSL algorithms significantly on the challenging semi-supervised CIFAR-100 and STL-10 tasks, and achieves competitive performance on CIFAR-10 and SVHN.
This paper studies a loss-averse newsvendor problem with reference dependence, where both demand and yield rate are stochastic. We obtain the loss-averse newsvendor’s optimal ordering policy and ...analyze the effects of loss aversion, reference dependence, random demand and yield on it. It is shown that the loss-averse newsvendor’s optimal order quantity and expected utility decreases in loss aversion level and reference point. Then, that this order quantity may be larger than the risk-neutral one’s if the reference point is less than a negative threshold. In addition, although the effect of random yield leads to an increase in the order quantity, the loss-averse newsvendor may order more than, equal to or less than the classical one, which significantly depends on loss aversion level and reference point. Numerical experiments were conducted to demonstrate our theoretical results.
Dynamic Neural Networks: A Survey Han, Yizeng; Huang, Gao; Song, Shiji ...
IEEE transactions on pattern analysis and machine intelligence,
2022-Nov.-1, 2022-11-1, 20221101, Volume:
44, Issue:
11
Journal Article
Peer reviewed
Open access
Dynamic neural network is an emerging research topic in deep learning. Compared to static models which have fixed computational graphs and parameters at the inference stage, dynamic networks can ...adapt their structures or parameters to different inputs, leading to notable advantages in terms of accuracy, computational efficiency, adaptiveness, etc. In this survey, we comprehensively review this rapidly developing area by dividing dynamic networks into three main categories: 1) sample-wise dynamic models that process each sample with data-dependent architectures or parameters; 2) spatial-wise dynamic networks that conduct adaptive computation with respect to different spatial locations of image data; and 3) temporal-wise dynamic models that perform adaptive inference along the temporal dimension for sequential data such as videos and texts. The important research problems of dynamic networks, e.g., architecture design, decision making scheme, optimization technique and applications, are reviewed systematically. Finally, we discuss the open problems in this field together with interesting future research directions.
In this article, the finite-time synchronization problem of delayed complex dynamical networks (CDNs) with impulses is studied, where two types of impulses, namely, synchronizing impulses and ...desynchronizing impulses, are fully considered, respectively. Since the existence of impulses makes the discontinuity of the states, which means that the classical result for finite-time stability is inapplicable in such a case, the key challenge is how to guarantee the finite-time stability and estimate the settling time in impulse sense. We apply impulsive control theory and finite-time stability theory to CDNs and establish some sufficient conditions for finite-time synchronization, where two kinds of memory controllers are designed for synchronizing impulses and desynchronizing impulses, respectively. Moreover, the upper bounds for settling time of synchronization, which depends on the impulse sequences, are effectively estimated. It shows that the synchronizing impulses can shorten the settling time of synchronization; conversely, the desynchronizing impulses can delay it. Finally, the theoretical analysis is verified by two simulation examples.
Reinforcement learning (RL) is a promising technique for designing a model-free controller by interacting with the environment. Several researchers have applied RL to autonomous underwater vehicles ...(AUVs) for motion control, such as trajectory tracking. However, the existing RL-based controller usually assumes that the unknown AUV dynamics keep invariant during the operation period, limiting its further application in the complex underwater environment. In this article, a novel meta-RL-based control scheme is proposed for trajectory tracking control of AUV in the presence of unknown and time-varying dynamics. To this end, we divide the tracking task for AUV with time-varying dynamics into multiple specific tasks with fixed time-varying dynamics, to which we apply meta-RL for training to distill the general control policy. The obtained control policy can transfer to the testing phase with high adaptability. Inspired by the line-of-sight (LOS) tracking rule, we formulate each specific task as a Markov decision process (MDP) with a well-designed state and reward function. Furthermore, a novel policy network with an attention module is proposed to extract the hidden information of AUV dynamics. The simulation environment with time-varying dynamics is established, and the simulation results reveal the effectiveness of our proposed method.
Least squares support vector machines (LS-SVMs) express the training in terms of solving a system of linear equations or an equivalent quadratic program (QP) with one linear equality constraint, in ...contrast to a QP with lower and upper bounds and one linear equality constraint for conventional support vector machines (SVMs). But for large scale problems, the presence of the linear equality constraint impedes the applications of some well developed methods. In this paper, we first eliminate the linear equality constraint of the QP in training LS-SVM and make it an unconstrained one, then propose a fast iterative single data approach with stepsize acceleration to the unconstrained QP. As a result of combining the selection rule of variables with the coordinate descent approach, the proposed approach is superior to the successive over-relaxation (SOR) method. Meanwhile updating only one variable at each iteration makes the proposed approach simpler and more flexible than the sequential minimal optimization (SMO) method. Computational experiment results on several benchmark data sets show that the proposed approach is more efficient than the existing single data approach and the SMO methods.
► The minimization problem for LS-SVM is transformed into an unconstrained one. ► We suggest an iterative single data approach to training the unconstrained LS-SVM. ► The stepsize accelerating implementation is incorporated to speed up the training process.
Motivated by the real-life scheduling problem in a steel-wire factory in China, this paper studies the problem of minimizing the maximum lateness on a single machine with family setups. In view of ...the NP-hard nature of the problem, neighborhood properties of the problem are investigated. It is found that the traditional move-based neighborhood is inefficient to search. Then a new neighborhood, which is based on batch destruction and construction, is developed. A simulated annealing algorithm with the new neighborhood is proposed. Experiments are carried out on the randomly generated problems and the real-life instances from a factory in China. Computational results show that the proposed algorithm can obtain better near optimal solutions than the existing algorithm.
► We transform the minimization problem for LS-SVM into an unconstrained one. ► Reduced formulations for LS-SVM are proposed. ► The times of using conjugate gradient method are reduced to one instead ...of two.
As a promising method for pattern recognition and function estimation, least squares support vector machines (LS-SVM) express the training in terms of solving a linear system instead of a quadratic programming problem as for conventional support vector machines (SVM). In this paper, by using the information provided by the equality constraint, we transform the minimization problem with a single equality constraint in LS-SVM into an unconstrained minimization problem, then propose reduced formulations for LS-SVM. By introducing this transformation, the times of using conjugate gradient (CG) method, which is a greatly time-consuming step in obtaining the numerical solution, are reduced to one instead of two as proposed by
Suykens et al. (1999). The comparison on computational speed of our method with the CG method proposed by Suykens et al. and the first order and second order SMO methods on several benchmark data sets shows a reduction of training time by up to 44%.
This paper proposes a three-phase algorithm (TPA) for the flowshop scheduling problem with blocking (BFSP) to minimize makespan. In the first phase, the blocking nature of BFSP is exploited to ...develop a priority rule that creates a sequence of jobs. Using this as the initial sequence and a variant of the NEH-insert procedure, the second phase generates an approximate solution to the problem. Then, utilizing a modified simulated annealing algorithm incorporated with a local search procedure, the schedule generated in the second phase is improved in the third phase. A pruning procedure that helps evaluate most solutions without calculating their complete makespan values is introduced in the local search to further reduce the computational time needed to solve the problem. Results of the computational experiments with Taillard's benchmark problem instances show that the proposed TPA algorithm is relatively more effective and efficient in minimizing makespan for the BFSP than the state-of-the-art procedures. Utilizing these results, 53 out of 60 new tighter upper bounds have been found for large-sized Taillard's benchmark problem instances.
PDF
