•Two new mutation strategies to enhance the DE performance are proposed.•A new adaptive approach to key control parameters are defined.•The proposed approach has superior classiffication ...accuracy.•The proposed algorithm’s performance is acceptable in feature subset selection.
In classification problems, normally there exists a large number of features, but not all of them contributing to the improvement of classification performance. These redundant features make the classification problem time consuming and often result in poor performance. Feature selection methods have been proposed to reduce the number of features, minimize computational cost, and maximize classification accuracy. As a wrapper-based approach, evolutionary algorithms have been widely applied in feature subset selection tasks. However, some of them trap into local minima, especially when the number of features increases, while others are not efficient in computational time. This paper proposes a Modified Differential Evolution (DE) approach to Feature Selection (MDEFS) by utilizing two new mutation strategies to create a feasible balance between exploration and exploitation and maintain the classification performance in an acceptable range concerning both the number of features and accuracy. Some modifications are made to the standard DE crossover and its key control parameters to enhance the proposed algorithm’s capabilities further. The proposed method has been compared to several state-of-the-art methods utilizing standard datasets from the UCI repository and results of the experiments demonstrate the superiority of the proposed approach.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
Quantification of the stationary points and the associated basins of attraction of neural network loss surfaces is an important step towards a better understanding of neural network loss surfaces at ...large. This work proposes a novel method to visualise basins of attraction together with the associated stationary points via gradient-based stochastic sampling. The proposed technique is used to perform an empirical study of the loss surfaces generated by two different error metrics: quadratic loss and entropic loss. The empirical observations confirm the theoretical hypothesis regarding the nature of neural network attraction basins. Entropic loss is shown to exhibit stronger gradients and fewer stationary points than quadratic loss, indicating that entropic loss has a more searchable landscape. Quadratic loss is shown to be more resilient to overfitting than entropic loss. Both losses are shown to exhibit local minima, but the number of local minima is shown to decrease with an increase in dimensionality. Thus, the proposed visualisation technique successfully captures the local minima properties exhibited by the neural network loss surfaces, and can be used for the purpose of fitness landscape analysis of neural networks.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
The problem of formation control of a team of mobile robots based on the virtual and behavioral structures is considered in this paper. In the virtual structure, each mobile robot is modeled by an ...electric charge. The mobile robots move toward a circle, and due to repulsive forces between the identical charges, regular polygon formations of the mobile robots will be realized. For swarm formation, a virtual mobile robot is located at the center of the circle, and other mobile robots follow it. In the introduced approach, each mobile robot finds its position in the formation autonomously, and the formation can change automatically in the case of change in the number of the mobile robots. This paper also proposes a technique for avoiding obstacles based on the behavioral structure. In this technique, when a mobile robot gets close to an obstacle, while moving toward its target, a rotational potential field is applied to lead the mobile robot to avoid the obstacle, without locating in local minimum positions.
Neural networks with rectified linear unit (ReLU) activation functions (a.k.a. ReLU networks) have achieved great empirical success in various domains. Nonetheless, existing results for learning ReLU ...networks either pose assumptions on the underlying data distribution being, e.g., Gaussian, or require the network size and/or training size to be sufficiently large. In this context, the problem of learning a two-layer ReLU network is approached in a binary classification setting, where the data are linearly separable and a hinge loss criterion is adopted. Leveraging the power of random noise perturbation, this paper presents a novel stochastic gradient descent (SGD) algorithm, which can provably train any single-hidden-layer ReLU network to attain global optimality, despite the presence of infinitely many bad local minima, maxima, and saddle points in general. This result is the first of its kind, requiring no assumptions on the data distribution, training/network size, or initialization. Convergence of the resultant iterative algorithm to a global minimum is analyzed by establishing both an upper bound and a lower bound on the number of non-zero updates to be performed. Moreover, generalization guarantees are developed for ReLU networks trained with the novel SGD leveraging classic compression bounds. These guarantees highlight a key difference (at least in the worst case) between reliably learning a ReLU network as well as a leaky ReLU network in terms of sample complexity. Numerical tests using both synthetic data and real images validate the effectiveness of the algorithm and the practical merits of the theory.
•Damage detection in structures by applying using artificial neural network (ANN) and cuckoo search (CS) algorithm.•Structural damage localization and quantification in bridges and beam-like ...structures.•Improve ANN training parameters using CS.•ANN combined with CS (ANN-CS) is accurate and cost effective.
This paper presents a new approach for damage detection in structures by applying a flexible combination based on an artificial neural network (ANN) and cuckoo search (CS) algorithm. ANN has become one of the most powerful tools employing computational intelligence techniques to tackle complex problems in numerous fields. However, due to the application of backpropagation algorithms based on gradient descent, a major drawback of ANN is the common problem of local minima that acts as a great hindrance to the search for the best solution. To overcome this disadvantage, we propose to combine ANN with evolutionary algorithms based on global search techniques. This paper employs CS to improve ANN training parameters (weight and bias) by minimizing the difference between real and desired outputs and then using these parameters to generate the network. Two numerical models, comprising a steel beam calibrated using experimental measurements and a large-scale truss bridge, are used to assess the robustness of the proposed approach. The results demonstrate that ANN combined with CS (ANN-CS) is accurate and requires a lower computational time than ANN, and evolutionary algorithm (EA) alone in terms of structural damage localization and quantification.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
In exploratory factor analysis, factor rotation algorithms can converge to local solutions (i.e., local minima) when they are initiated from different starting points. To better understand this ...problem, we performed three studies that investigated the prevalence and correlates of local solutions with five factor rotation algorithms: varimax, oblimin, entropy, and geomin (orthogonal and oblique). In total, we simulated 16,000 data sets and performed more than 57 million factor rotations to examine the influence of (a) factor loading size, (b) number of factor indicators, (c) factor cross loadings, (d) factor correlation size, (e) factor loading standardization, (f) sample size, and (g) model approximation error on the frequency of local solutions in factor rotation. We also examined local solutions in an exploratory factor analysis of an open source data set that included 54 personality items. Across three studies, all five algorithms converged to local solutions under some conditions with geomin (orthogonal and oblique) producing the highest number of local solutions. Follow-up analyses showed that, when factor rotations produced multiple solutions, the factor pattern with the maximum hyperplane count (rather than the lowest complexity value) was typically closest in mean squared error to the population factor pattern.
Translational Abstract
In exploratory factor analysis, factor rotation algorithms can converge to local solutions (i.e., local minima) when they are initiated from different starting points. To better understand this problem, we performed three studies that investigated the prevalence and correlates of local solutions with five factor rotation algorithms: varimax, oblimin, entropy, and geomin (orthogonal and oblique). In total, we simulated 16,000 data sets and performed more than 57 million factor rotations to examine the influence of (a) factor loading size, (b) number of factor indicators, (c) factor cross loadings, (d) factor correlation size, (e) factor loading standardization, (f) sample size, and (g) model approximation error on the frequency of local solutions in factor rotation. We also examined local solutions in an exploratory factor analysis of an open source data set that included 54 personality items. Across three studies, all five algorithms converged to local solutions under some conditions with geomin (orthogonal and oblique) producing the highest number of local solutions. Follow-up analyses showed that, when factor rotations produced multiple solutions, the factor pattern with the maximum hyperplane count (rather than the lowest complexity value) was typically closest in mean squared error to the population factor pattern.
Full text
Available for:
CEKLJ, FFLJ, NUK, ODKLJ, PEFLJ, UPUK
In this paper, we prove that depth with nonlinearity creates no bad local minima in a type of arbitrarily deep ResNets with arbitrary nonlinear activation functions, in the sense that the values of ...all local minima are no worse than the global minimum value of corresponding classical machine-learning models, and are guaranteed to further improve via residual representations. As a result, this paper provides an affirmative answer to an open question stated in a paper in the conference on Neural Information Processing Systems 2018. This paper advances the optimization theory of deep learning only for ResNets and not for other network architectures.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
The cooperation of tailoring fiber angles and designing its macro layout is an excellent approach to improve the performance of composite structures and achieve lighter weight. This article proposes ...a novel discrete–continuous optimization model for the concurrent optimization of the macro structural topology and local fiber angle for the fiber-reinforced composite structure. Therein, the novel modeling consists of a two-step strategy, where the normal distribution function is adopted to form the discrete material interpolation scheme firstly and then the continuous angle design on fine angle subintervals is conducted. Because it requires the fixed discrete design variables regardless of the number of angle subintervals and simultaneously achieves the continuous fiber angle design, it can save computational cost and provide an infinite design domain. Furthermore, the subdividing angle interval reduces the risks of falling into the local optimum solution for continuous angle design. Besides, the fiber discontinuity may cause stress concentration and manufacturing problems in the intersection of discontinuous fiber paths. Therefore, a linear density technique is used to achieve fiber continuity. Finally, a concurrent topology optimization model is built based on the proposed novel model. And several numerical examples are well-organized and presented to illustrate the validity of the proposed model.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
PDF
