We consider the problem of learning an unknown function f⋆ on the d-dimensional sphere with respect to the square loss, given i.i.d. samples {(yi, xi)}i≤n where xi is a feature vector uniformly ...distributed on the sphere and yi = f⋆ (xi) + εi. We study two popular classes of models that can be regarded as linearizations of two-layers neural networks around a random initialization: the random features model of Rahimi–Recht (RF); the neural tangent model of Jacot–Gabriel–Hongler (NT). Both these models can also be regarded as randomized approximations of kernel ridge regression (with respect to different kernels), and enjoy universal approximation properties when the number of neurons N diverges, for a fixed dimension d. We consider two specific regimes: the infinite-sample finite-width regime, in which n = ∞ while d and N are large but finite, and the infinite-width finite-sample regime in which N = ∞ while d and n are large but finite. In the first regime, we prove that if dℓ+δ ≤ N ≤ dℓ+1−δ for small δ > 0, then RF effectively fits a degree-ℓ polynomial in the raw features, and NT fits a degree-(ℓ + 1) polynomial. In the second regime, both RF and NT reduce to kernel methods with rotationally invariant kernels. We prove that, if the sample size satisfies dℓ+δ ≤ n ≤ dℓ+1−δ, then kernel methods can fit at most a degree-ℓ polynomial in the raw features. This lower bound is achieved by kernel ridge regression, and near-optimal prediction error is achieved for vanishing ridge regularization.
•A new Takagi-Sugeno system based Kernel ridge regression (TS-KRR) was proposed.•The TS-KRR strategy is implemented for both adaptive and offline identification.•The TS-KRR was integrated with the ...GPC to control discrete-time nonlinear systems.•The proposed controller showed good results in TS fuzzy GPC with offline modeling.•The adaptive TS fuzzy GPC showed good results in dealing with disturbances.
In this paper, a novel fuzzy Generalized Predictive Control (GPC) is proposed for discrete-time nonlinear systems via Takagi-Sugeno system based Kernel Ridge Regression (TS-KRR). The TS-KRR strategy approximates the unknown nonlinear systems by learning the Takagi-Sugeno (TS) fuzzy parameters from the input-output data. Two main steps are required to construct the TS-KRR: the first step is to use a clustering algorithm such as the clustering based Particle Swarm Optimization (PSO) algorithm that separates the input data into clusters and obtains the antecedent TS fuzzy model parameters. In the second step, the consequent TS fuzzy parameters are obtained using a Kernel ridge regression algorithm. Furthermore, the TS based predictive control is created by integrating the TS-KRR into the Generalized Predictive Controller. Next, an adaptive, online, version of TS-KRR is proposed and integrated with the GPC controller resulting an efficient adaptive fuzzy generalized predictive control methodology that can deal with most of the industrial plants and has the ability to deal with disturbances and variations of the model parameters. In the adaptive TS-KRR algorithm, the antecedent parameters are initialized with a simple K-means algorithm and updated using a simple gradient algorithm. Then, the consequent parameters are obtained using the sliding-window Kernel Recursive Least squares (KRLS) algorithm. Finally, two nonlinear systems: A surge tank and Continuous Stirred Tank Reactor (CSTR) systems were used to investigate the performance of the new adaptive TS-KRR GPC controller. Furthermore, the results obtained by the adaptive TS-KRR GPC controller were compared with two other controllers. The numerical results demonstrate the reliability of the proposed adaptive TS-KRR GPC method for discrete-time nonlinear systems.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UL, UM, UPCLJ, UPUK, ZRSKP
Reservoir computing has emerged in the last decade as an alternative to gradient descent methods for training recurrent neural networks. Echo State Network (ESN) is one of the key reservoir computing ...“flavors”. While being practical, conceptually simple, and easy to implement, ESNs require some experience and insight to achieve the hailed good performance in many tasks. Here we present practical techniques and recommendations for successfully applying ESNs, as well as some more advanced application-specific modifications.
With randomly generated weights between input and hidden layers, a random vector functional link network is a universal approximator for continuous functions on compact sets with fast learning ...property. Though it was proposed two decades ago, the classification ability of this family of networks has not been fully investigated yet. Through a very comprehensive evaluation by using 121 UCI datasets, the effect of bias in the output layer, direct links from the input layer to the output layer and type of activation functions in the hidden layer, scaling of parameter randomization as well as the solution procedure for the output weights are investigated in this work. Surprisingly, we found that the direct link plays an important performance enhancing role in RVFL, while the bias term in the output neuron had no significant effect. The ridge regression based closed-form solution was better than those with Moore–Penrose pseudoinverse. Instead of using a uniform randomization in −1,+1 for all datasets, tuning the scaling of the uniform randomization range for each dataset enhances the overall performance. Six commonly used activation functions were investigated in this work and we found that hardlim and sign activation functions degenerate the overall performance. These basic conclusions can serve as general guidelines for designing RVFL networks based classifiers.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
15.
Genomic selection in hybrid breeding Zhao, Yusheng; Mette, Michael F.; Reif, Jochen C.
Plant breeding,
February 2015, Volume:
134, Issue:
1
Journal Article
Peer reviewed
While hybrid breeding is widely applied in outbreeding species, for many self‐pollinating crop plants, it has only recently been established. This may have had its reason in the limitations of ...methods available for hybrid performance prediction, in particular when established heterotic pools were absent. Genomic selection has been suggested as a promising approach to resolve these limitations. In our review, we briefly introduce the principles of genomic selection as an extension of marker‐assisted selection using genome‐wide high‐density molecular marker data and discuss the advantages and limitations of currently used algorithms. Including the outcome from a recent extended approach to hybrid wheat as a timely example, we summarize current progress in empirical studies on the application of genomic selection for prediction of hybrid performance. Here, we put emphasis on the factors affecting the accuracy of prediction, pointing in particular to the relevance of relatedness, genotype x environment interaction and experimental design. Finally, we discuss future research needs and potential applications.
Full text
Available for:
BFBNIB, DOBA, FZAB, GIS, IJS, IZUM, KILJ, NLZOH, NUK, OILJ, PILJ, PNG, SAZU, SBCE, SBMB, UILJ, UKNU, UL, UM, UPUK
•Biased estimators may be powerful tools in extreme learning machine studies.•Ridge based regression estimators may outperform the extreme machine learning.•Choice of ridge regularization parameter ...is effective to obtain more stable results.•Generalization performance of ELM depend on selection of regularization parameter.
The extreme learning machine (ELM) which is a single layer feedforward neural network provides extremely fast training speed and good generalization performance. The ELM however, has its respective drawback: it is known to be sensitive to the ill-conditioned data. To overcome the ill-conditioning problem in ELM, ELM based on ridge regression (RR-ELM) was proposed. Since RR-ELM is a biased method, ELM based on almost unbiased ridge regression (AUR-ELM) was accordingly proposed to reduce the bias in a certain extent. RR-ELM and AUR-ELM introduced in the existence of multicollinearity, depend on the regularization parameter. The regularization parameter affects the performance of both RR-ELM and AUR-ELM. There is no consensus on the selection of the regularization parameter. Although there are various methods in linear regression to select the regularization parameter, only one method based on the selection minimizing the mean squared error was used in RR-ELM. In this study, AIC, BIC and CV criteria in the context of RR-ELM and AUR-ELM were proposed as alternative methods for the selection of the regularization parameter. An experimental study was conducted on eight data sets which are widely known and used in machine learning studies. The analyzes are considered as purposive for regression studies which are the most important fields of expert systems and machine learning. The results obtained demonstrate that the selection method of the regularization parameter is significantly effective on both the generalization and particularly stability performance of RR-ELM and AUR-ELM when compared to ELM
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
With the in-depth implementation of the rural revitalization strategy, it is of great significance to study the supervision and governance mechanism of national audit in the implementation of the ...rural revitalization strategy. As the main components with smaller contribution rates will be discarded when evaluating and analyzing the influencing factors of rural revitalization, it may affect the analytical judgment of the actual problem. Therefore, based on stepwise regression, this paper adopts ridge regression to correct multicollinearity, with the help of the ridge trace diagram to judge the correlation between independent variables intuitively and quickly, and through the ridge calculation as much as possible to retain the independent variables that have a greater impact on rural revitalization. The principal component-ridge regression model is proposed because it takes into account the advantages and disadvantages of various regression methods. To solve the problem of unstable regression coefficients caused by multiple covariances, to analyze the important influencing factors of state auditing on rural revitalization according to the regression coefficients, and to establish the governance optimization path of state auditing to promote rural revitalization. According to the regression results, the government audit variable has a significance level of 1%. The comprehensive governance function of government audit has a regression coefficient of -0.038 that shows significant and negative results at a 1% confidence level. The performance scores of rural sustainability as well as financial dimensions are in the range of 0.8 to 1, which is a good level.
We consider the problem of reconstructing a function from a finite set of noise-corrupted samples. Two kernel algorithms are analyzed, namely kernel ridge regression and ɛ-support vector regression. ...By assuming the ground-truth function belongs to the reproducing kernel Hilbert space of the chosen kernel, and the measurement noise affecting the dataset is bounded, we adopt an approximation theory viewpoint to establish deterministic, finite-sample error bounds for the two models. Finally, we discuss their connection with Gaussian processes and two numerical examples are provided. In establishing our inequalities, we hope to help bring the fields of non-parametric kernel learning and system identification for robust control closer to each other.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
As China takes great efforts to cap its total energy consumption, it is important to understand the future energy use in all sectors. This paper aims to present a long-term prediction of energy use ...in China’s construction and building sectors (CBS) up to the year 2100. A STIRPAT (Stochastic Impacts by Regression on Population, Affluence, and Technology) model is used to establish the relationship between six socioeconomic and technological factors and China’s CBS energy consumption. Based on the statistical data from 2000 to 2016, ridge regression is applied to derive the coefficients of the STIRPAT model to counter the impact of multicollinearity on regression results. The projections are performed for three scenarios: a benchmark scenario, an intensive scenario, and an extensive scenario. The results show that for all three scenarios, the overall trend of China’s CBS energy consumption is to continuously increase from the present, reach a peak in the range between 1155 and 1243 million tons of standard coal equivalent (Mtce) in 2050, and then decrease to 942–1116 Mtce in 2100. The above projection and the associated STIRPAT model are valuable for developing policies on construction and buildings to control the total energy use in China.
•Use a STIRPAT model to predict China’s energy use in construction and buildings.•Three scenarios and six socioeconomic and technological factors are considered.•The projected energy will increase until 2050 and then decrease gradually.•A peak energy use between 1155 and 1243 Mtce is projected to occur in 2050.
Full text
Available for:
GEOZS, IJS, IMTLJ, KILJ, KISLJ, NLZOH, NUK, OILJ, PNG, SAZU, SBCE, SBJE, UILJ, UL, UM, UPCLJ, UPUK, ZAGLJ, ZRSKP
PDF
