A new learning rule (Precise-Spike-Driven (PSD) Synaptic Plasticity) is proposed for processing and memorizing spatiotemporal patterns. PSD is a supervised learning rule that is analytically derived ...from the traditional Widrow-Hoff rule and can be used to train neurons to associate an input spatiotemporal spike pattern with a desired spike train. Synaptic adaptation is driven by the error between the desired and the actual output spikes, with positive errors causing long-term potentiation and negative errors causing long-term depression. The amount of modification is proportional to an eligibility trace that is triggered by afferent spikes. The PSD rule is both computationally efficient and biologically plausible. The properties of this learning rule are investigated extensively through experimental simulations, including its learning performance, its generality to different neuron models, its robustness against noisy conditions, its memory capacity, and the effects of its learning parameters. Experimental results show that the PSD rule is capable of spatiotemporal pattern classification, and can even outperform a well studied benchmark algorithm with the proposed relative confidence criterion. The PSD rule is further validated on a practical example of an optical character recognition problem. The results again show that it can achieve a good recognition performance with a proper encoding. Finally, a detailed discussion is provided about the PSD rule and several related algorithms including tempotron, SPAN, Chronotron and ReSuMe.
Evolutionary algorithms (EAs) have become one of the most effective techniques for multi-objective optimization, where a number of variation operators have been developed to handle the problems with ...various difficulties. While most EAs use a fixed operator all the time, it is a labor-intensive process to determine the best EA for a new problem. Hence, some recent studies have been dedicated to the adaptive selection of the best operators during the search process. To address the exploration versus exploitation dilemma in operator selection, this paper proposes a novel operator selection method based on reinforcement learning. In the proposed method, the decision variables are regarded as states and the candidate operators are regarded as actions. By using deep neural networks to learn a policy that estimates the <inline-formula><tex-math notation="LaTeX">Q</tex-math></inline-formula> value of each action given a state, the proposed method can determine the best operator for each parent that maximizes its cumulative improvement. An EA is developed based on the proposed method, which is verified to be more effective than the state-of-the-art ones on challenging multi-objective optimization problems.
Objective: Deep learning (DL) techniques have been introduced to assist doctors in the interpretation of medical images by detecting image-derived phenotype abnormality. Yet the privacy-preserving ...policy of medical images disables the effective training of DL model using sufficiently large datasets. As a decentralized computing paradigm to address this issue, federated learning (FL) allows the training process to occur in individual institutions with local datasets, and then aggregates the resultant weights without risk of privacy leakage. Methods: We propose an effective federated multi-task learning (MTL) framework to jointly identify multiple related mental disorders based on functional magnetic resonance imaging data. A federated contrastive learning-based feature extractor is developed to extract high-level features across client models. To ease the optimization conflicts of updating shared parameters in MTL, we present a federated multi-gate mixture of expert classifier for the joint classification. The proposed framework also provides practical modules, including personalized model learning, privacy protection, and federated biomarker interpretation. Results: On real-world datasets, the proposed framework achieves robust diagnosis accuracies of 69.48 <inline-formula><tex-math notation="LaTeX">\pm</tex-math></inline-formula> 1.6%, 71.44 <inline-formula><tex-math notation="LaTeX">\pm</tex-math></inline-formula> 3.2%, and 83.29 <inline-formula><tex-math notation="LaTeX">\pm</tex-math></inline-formula> 3.2% in autism spectrum disorder, attention deficit/hyperactivity disorder, and schizophrenia, respectively. Conclusion: The proposed framework can effectively ease the domain shift between clients via federated MTL. Significance: The current work provides insights into exploiting the advantageous knowledge shared in related mental disorders for improving the generalization capability of computer-aided detection approaches.
For accurate prognostics, users have to determine the current health of the system and predict future degradation pattern of the system. An increasingly popular approach toward tackling prognostic ...problems involves the use of switching models to represent various degradation phases, which the system undergoes. Such approaches have the advantage of determining the exact degradation phase of the system and being able to handle nonlinear degradation models through piecewise linear approximation. However, limitations of such existing methods include, limited applicability due to the discretization of predicted remaining useful life, insufficient robustness due to the use of single models and others. This paper circumvents these limitations by proposing a hybrid of ensemble methods with switching methods. The proposed method first implements a switching Kalman filter (SKF) to classify between various linear degradation phases, then predict the future propagation of fault dimension using appropriate Kalman filters for each phase. This proposed method achieves both continuous and discrete prediction values representing the remaining life and degradation phase of the system, respectively. The proposed framework is shown via a case study on benchmark simulated aeroengine data sets. The evaluation of the proposed framework shows that the proposed method achieves better accuracy and robustness against noise compared with other methods reported in the literature. The results also indicate the effectiveness of the SKF in detecting the switching point between various degradation modes.
Primates perform remarkably well in cognitive tasks such as pattern recognition. Motivated by recent findings in biological systems, a unified and consistent feedforward system network with a proper ...encoding scheme and supervised temporal rules is built for solving the pattern recognition task. The temporal rules used for processing precise spiking patterns have recently emerged as ways of emulating the brain's computation from its anatomy and physiology. Most of these rules could be used for recognizing different spatiotemporal patterns. However, there arises the question of whether these temporal rules could be used to recognize real-world stimuli such as images. Furthermore, how the information is represented in the brain still remains unclear. To tackle these problems, a proper encoding method and a unified computational model with consistent and efficient learning rule are proposed. Through encoding, external stimuli are converted into sparse representations, which also have properties of invariance. These temporal patterns are then learned through biologically derived algorithms in the learning layer, followed by the final decision presented through the readout layer. The performance of the model with images of digits from the MNIST database is presented. The results show that the proposed model is capable of recognizing images correctly with a performance comparable to that of current benchmark algorithms. The results also suggest a plausibility proof for a class of feedforward models of rapid and robust recognition in the brain.
Recently, multi and many-objective evolutionary algorithms (MOEAs) embedded with clustering techniques to enhance their environmental selection show promising performance for tackling multi and ...many-objective optimization problems (MOPs) with irregular Pareto fronts (PFs). However, the similarity metric used for clustering cannot reflect the diversity of solutions fairly when it is measured based on the direction distances of solutions to an ideal point. Consequently, it may mislead the environmental selection when solving MOPs with convex PFs. To alleviate this issue, this paper suggests an MOEA using clustering with a flexible similarity metric to run the environmental selection. In our approach, we first use a simple yet effective method to roughly predict the concavity or convexity of the target problem. Then, a flexible reference point is set to define the direction distances of solutions, which can properly measure the similarity between solutions and fairly show their diversity for solving MOPs with various PF shapes. After that, we use a hierarchical clustering method with this flexible similarity metric to run the environmental selection on all solutions, which will properly classify them into N clusters (N is the population size). Finally, in each cluster, one solution with the best convergence value will survive to compose the new population. When compared to twenty-seven competitive MOEAs in solving nineteen benchmark MOPs and sixteen real-world engineering MOPs with various PF shapes, the experimental results demonstrate that the proposed algorithm has significant advantages.
Evolutionary multiobjective optimization in dynamic environments is a challenging task, as it requires the optimization algorithm converging to a time-variant Pareto optimal front. This paper ...proposes a dynamic multiobjective optimization algorithm which utilizes an inverse model set to guide the search toward promising decision regions. In order to reduce the number of fitness evalutions for change detection purpose, a two-stage change detection test is proposed which uses the inverse model set to check potential changes in the objective function landscape. Both static and dynamic multiobjective benchmark optimization problems have been considered to evaluate the performance of the proposed algorithm. Experimental results show that the improvement in optimization performance is achievable when the proposed inverse model set is adopted.
Facing the increasing worldwide prevalence of mental disorders, the symptom-based diagnostic criteria struggle to address the urgent public health concern due to the global shortfall in ...well-qualified professionals. Thanks to the recent advances in neuroimaging techniques, functional magnetic resonance imaging (fMRI) has surfaced as a new solution to characterize neuropathological biomarkers for detecting functional connectivity (FC) anomalies in mental disorders. However, the existing computer-aided diagnosis models for fMRI analysis suffer from unstable performance on large datasets. To address this issue, we propose an efficient multitask learning (MTL) framework for joint diagnosis of multiple mental disorders using resting-state fMRI data. A novel multiobjective evolutionary clustering algorithm is presented to group regions of interests (ROIs) into different clusters for FC pattern analysis. On the optimal clustering solution, the multicluster multigate mixture-of-expert model is used for the final classification by capturing the highly consistent feature patterns among related diagnostic tasks. Extensive simulation experiments demonstrate that the performance of the proposed framework is superior to that of the other state-of-the-art methods. Moreover, the potential for practical application of the framework is also validated in terms of limited computational resources, real-time analysis, and insufficient training data. The proposed model can identify the remarkable interpretative biomarkers associated with specific mental disorders for clinical interpretation analysis.
Spikes play an essential role in information transmission in central nervous system, but how neurons learn from them remains a challenging question. Most algorithms studied how to train spiking ...neurons to process patterns encoded with a sole assumption of either a rate or a temporal code. Is there a general learning algorithm capable of processing both codes regardless of the intense debate on them within neuroscience community? In this paper, we propose several threshold-driven plasticity algorithms to address the above question. In addition to formulating the algorithms, we also provide proofs with respect to several properties, such as robustness and convergence. The experimental results illustrate that our algorithms are simple, effective and yet efficient for training neurons to learn spike patterns. Due to their simplicity and high efficiency, our algorithms would be potentially beneficial for both software and hardware implementations. Neurons with our algorithms can also detect and recognize embedded features from a background sensory activity. With the as-proposed algorithms, a single neuron can successfully perform multicategory classifications by making decisions based on its output spike number in response to each category. Spike patterns being processed can be encoded with both spike rates and precise timings. When afferent spike timings matter, neurons will automatically extract temporal features without being explicitly instructed as to which point to fire.
Artificial neural networks (ANN) have become the mainstream acoustic modeling technique for large vocabulary automatic speech recognition (ASR). A conventional ANN features a multi-layer architecture ...that requires massive amounts of computation. The brain-inspired spiking neural networks (SNN) closely mimic the biological neural networks and can operate on low-power neuromorphic hardware with spike-based computation. Motivated by their unprecedented energy-efficiency and rapid information processing capability, we explore the use of SNNs for speech recognition. In this work, we use SNNs for acoustic modeling and evaluate their performance on several large vocabulary recognition scenarios. The experimental results demonstrate competitive ASR accuracies to their ANN counterparts, while require only 10 algorithmic time steps and as low as 0.68 times total synaptic operations to classify each audio frame. Integrating the algorithmic power of deep SNNs with energy-efficient neuromorphic hardware, therefore, offer an attractive solution for ASR applications running locally on mobile and embedded devices.
PDF
