The architecture of multilayer perceptron (MLP) neural networks dictates the network's performance. However, aiming at the specific classification problems, suitable architectures of MLPs must be ...determined beforehand. More often, the design decisions rely on a trial and error learning and the experience knowledge. To automatically design an MLP's network architecture and optimize network parameters, a multiobjective bilevel programming model is built. In this model, a multiobjective optimization problem is constructed in the upper level for obtaining a set of Pareto optimal architectures of the MLPs, considering network complexity, training error rate, and validation error rate, while a single-objective optimization problem is established in the lower level to search for the optimum network parameters for a given network architecture. For dealing with this model efficiently, a novel multiobjective hierarchical learning algorithm (MOHLA) is proposed, in which an integer-coding NSGA-II is developed as the upper-level optimizer for a set of Pareto optimal network structures of the MLPs, while a non-iterative method is regarded as the lower-level solver for the MLPs' connection parameters. After a set of trained MLPs is obtained finally by using MOHLA, a selective ensemble strategy is adopted for improving identification accuracy. Three types of multiobjective bilevel programming models are investigated and compared in the experiments. Moreover, the proposed MOHLA is compared with several state-of-the-art learning approaches on various classification problems. The experimental results confirm that MOHLA performs well.
A digital twin is a digital replica of a physical entity to which it is remotely connected. A digital twin can provide a rich representation of the corresponding physical entity and enables ...sophisticated control for various purposes. Although the concept of the digital twin is largely known, designing digital twins based systems has not yet been fully explored. In practice, digital twins can be applied in different ways leading to different architectural designs. To guide the architecture design process, we provide a pattern-oriented approach for architecting digital twin-based systems. To this end, we propose a catalog of digital twin architecture design patterns that can be reused in the broad context of systems engineering. The patterns support the various phases in the systems engineering life cycle process, and are described using a well-defined pattern documentation template. For illustrating the application of digital twin patterns, we adopt a multi-case study approach in the agriculture and food domain.
The rapid development of miniaturized electronic devices has greatly stimulated the endless pursuit of high-performance on-chip micro-supercapacitors (MSCs) delivering both high energy and power ...densities. To this end, an advanced three-dimensional (3D) microelectrode architecture design offers enormous opportunities due to high mass loading of active materials, large specific surface areas, fast ion diffusion kinetics, and short electron transport pathways. In this review, we summarize the recent advances in the rational design of 3D architectured microelectrodes including 3D dense microelectrodes, 3D nanoporous microelectrodes, and 3D macroporous microelectrodes. Furthermore, the emergent microfabrication strategies are discussed in detail in terms of charge storage mechanisms and structure–performance correlation for on-chip MSCs. Finally, we conclude with a perspective on future opportunities and challenges in this thriving field.
Liquid state machine(LSM) is an attractive spiking neural network (SNN) for Network-on-Chip(NoC)-based neuromorphic platforms due to their biological characteristics and hardware efficiency. But the ...randomly connected topology of the liquid in LSM and lots of communication spike bring different dataflow and communication congestion on the NoC-based platform. Aiming to design an accurate and communication optimized LSM architecture, we have to explore the LSM and NoC architecture design space. Enormous design space and the gap between LSM/NoC design space bring challenges to find out the optimal pair of LSM/NoC architecture design.
To face the above challenge, we propose a multi-objective LSM/NoC architecture co-design framework, which fast and efficiently explores the design space of LSM/NoC to generate an optimal LSM architecture with low latency on NoC-based platform.
Evaluation results show that our framework can generate LSM architecture suitable for execution on NoC-based platform with reduced runtime and negligible reduced accuracy. Compared with state-of-the-art LSM designs with the fixed NoC structure, we achieve 2.5x∼3.0x latency reduction or average 3.1x energy reduction. For fair comparison, compared with state-of-the-art LSM designs with our NOC architecture search process, our framework can achieve 1.25x∼1.41x lower latency and 1.16x∼1.87x lower energy together with only average 0.65% accuracy loss.
The internet of vehicle (IoV) orchestration is an emerging technology in heterogeneous vehicles to contrivance diverse intelligent transportation applications. The roadside unit (RSU) plays a vital ...role during service provisioning. Vehicle‐to‐vehicle and vehicle‐to‐infrastructure communications have consistently accomplished the services in a vehicular network. However, persisting the increased vehicles' quality of experience and network vendors' utilities and which RSUs have to select for effective, reliable service are critical open research challenges to consolidate RSU services to enhance network service utility rate. In this article, we design a deep learning‐inspired RSU Service Consolidation Approach based on two‐models to enhance the service reliability by formulating the RSU coverage issue with the RSU Migration model and content delivery issue with Linear Programming‐based Multicast model. Adaptive Packet‐Error measurement system to optimize service reliability rate at the edge of cooperative vehicular network based on content correlation. The performance and efficiency are examined based on MATLAB. The simulation outcome shows RSC approach has low execution cost by 39%, service reliability rate by 71% than the state‐of‐art approaches.
Industrial control systems (ICSs) are facing serious and evolving security threats because of a variety of malicious attacks. Deep learning-based intrusion detection systems (IDSs) have been widely ...considered as one of promising security solutions for ICSs, but these deep neural networks for IDSs in ICSs have been designed manually, which are extremely dependent on expert experience with numerous model parameters. This paper makes the first attempt to develop an automatic architecture design method of convolutional neural networks (CNNs) based on differential evolution (abbreviated as DE-CNN) for the intrusion detection issue in ICSs. The first phase of the proposed DE-CNN is the off-line architecture optimization of the CNNs constructed by three basic units such as ResNetBlockUnit, DenseNetBlockUnit, and PoolingUnit, including encoding the architecture parameters of a CNN as a population, evaluating the fitness of the population by the validation accuracy and the number of CNN model parameters, implementing the evolutionary process including mutation and crossover operations, and selecting the best individual from the population. Then, the optimal CNN model obtained by the off-line optimization of DE-CNN is deployed for the online IDSs. The experimental results on two intrusion detection datasets in ICSs including SWaT and WADI have demonstrated the superiority of the proposed DE-CNN to the state-of-the-art manually-designed and neuroevolution-based methods under both unsupervised and supervised learning in terms of Precision, Recall, F1-Score and the number of the CNN model parameters.
Modules are requisite for the realization of modular reconfigurable manipulators. The design of modules in literature mainly revolves around geometric aspects and features such as lengths, ...connectivity and adaptivity. Optimizing and designing the modules based on dynamic performance is considered as a challenge here. The present paper introduces an Architecture-Prominent-Sectioning (APS) strategy for the planning of architecture of modules such that a reconfigurable manipulator possesses minimal joint torques during its operations. Proposed here is the transferring of complete structure into an equivalent system, perform optimization and map the resulting arrangement into possible architecture. The strategy has been applied on a set of modular configurations considering three-primitive-paths. The possibility of getting advanced/complex shapes is also discussed to incorporate the idea of a modular library.
Designing effective architectures is one of the key factors behind the success of deep neural networks. Existing deep architectures are either manually designed or automatically searched by some ...Neural Architecture Search (NAS) methods. However, even a well-designed/searched architecture may still contain many nonsignificant or redundant modules/operations (e.g., some intermediate convolution or pooling layers). Such redundancy may not only incur substantial memory consumption and computational cost but also deteriorate the performance. Thus, it is necessary to optimize the operations inside an architecture to improve the performance without introducing extra computational cost. To this end, we have proposed a Neural Architecture Transformer (NAT) method which casts the optimization problem into a Markov Decision Process (MDP) and seeks to replace the redundant operations with more efficient operations, such as skip or null connection. Note that NAT only considers a small number of possible replacements/transitions and thus comes with a limited search space. As a result, such a small search space may hamper the performance of architecture optimization. To address this issue, we propose a Neural Architecture Transformer++ (NAT++) method which further enlarges the set of candidate transitions to improve the performance of architecture optimization. Specifically, we present a two-level transition rule to obtain valid transitions, i.e., allowing operations to have more efficient types (e.g., convolution<inline-formula><tex-math notation="LaTeX">{\to }</tex-math> <mml:math><mml:mo>→</mml:mo></mml:math><inline-graphic xlink:href="tan-ieq1-3086914.gif"/> </inline-formula>separable convolution) or smaller kernel sizes (e.g., <inline-formula><tex-math notation="LaTeX">5{\times }5 {\to } 3{\times }3</tex-math> <mml:math><mml:mrow><mml:mn>5</mml:mn><mml:mo>×</mml:mo><mml:mn>5</mml:mn><mml:mo>→</mml:mo><mml:mn>3</mml:mn><mml:mo>×</mml:mo><mml:mn>3</mml:mn></mml:mrow></mml:math><inline-graphic xlink:href="tan-ieq2-3086914.gif"/> </inline-formula>). Note that different operations may have different valid transitions. We further propose a Binary-Masked Softmax (BMSoftmax) layer to omit the possible invalid transitions. Last, based on the MDP formulation, we apply policy gradient to learn an optimal policy, which will be used to infer the optimized architectures. Extensive experiments show that the transformed architectures significantly outperform both their original counterparts and the architectures optimized by existing methods.
PDF
