The reciprocal elastic net Alhamzawi, Rahim; Alhamzawi, Ahmed; Mallick, Himel
Communication in statistics. Case studies and data analysis,
10/2023, Letnik:
9, Številka:
4
Journal Article
Recenzirano
A new penalized likelihood method (reciprocal elastic net) is put forward for regularization and variable selection. Our proposal is based on a new class of reciprocal penalty functions, combining ...the strengths of the reciprocal LASSO regularization and ridge regression. We formulate the reciprocal elastic net problem as an equivalent reciprocal LASSO problem on augmented data, facilitating a direct utilization of the reciprocal LASSO algorithm to generate the entire reciprocal elastic net solution path. We further present the reciprocal adaptive elastic net, fusing the merits of ridge regression with the adaptively weighted reciprocal LASSO regularization. These methods, illustrated through simulated examples and real data analysis, demonstrate satisfactory performance in various diversified scenarios compared to published methods. Finally, we propose Bayesian methods to solve the reciprocal elastic net and reciprocal adaptive elastic net models using Gibbs samplers.
We consider estimation in a high-dimensional linear model with strongly correlated variables. We propose to cluster the variables first and do subsequent sparse estimation such as the Lasso for ...cluster-representatives or the group Lasso based on the structure from the clusters. Regarding the first step, we present a novel and bottom-up agglomerative clustering algorithm based on canonical correlations, and we show that it finds an optimal solution and is statistically consistent. We also present some theoretical arguments that canonical correlation based clustering leads to a better-posed compatibility constant for the design matrix which ensures identifiability and an oracle inequality for the group Lasso. Furthermore, we discuss circumstances where cluster-representatives and using the Lasso as subsequent estimator leads to improved results for prediction and detection of variables. We complement the theoretical analysis with various empirical results.
The use of the multinomial logit model is typically restricted to applications with few predictors, because in high-dimensional settings maximum likelihood estimates tend to deteriorate. A ...sparsity-inducing penalty is proposed that accounts for the special structure of multinomial models by penalizing the parameters that are linked to one variable in a grouped way. It is devised to handle general multinomial logit models with a combination of global predictors and those that are specific to the response categories. A proximal gradient algorithm is used that efficiently computes stable estimates. Adaptive weights and a refitting procedure are incorporated to improve variable selection and predictive performance. The effectiveness of the proposed method is demonstrated by simulation studies and an application to the modeling of party choice of voters in Germany.
•Bitcoin returns determinants.•Lasso regression allows both variable selection and regularization in the analysis.•Search intensity (Google), gold returns and policy uncertainty are found to be the ...most important.
We examine the significance of twenty-one potential drivers of bitcoin returns for the period 2010–2017 (2533 daily observations). Within a LASSO framework, we examine the effects of factors such as stock market returns, exchange rates, gold and oil returns, FED’s and ECB’s rates and internet trends on bitcoin returns for alternate time periods. Search intensity and gold returns emerge as the most important variables for bitcoin returns.
Variable selection plays an important role in high dimensional statistical modelling which nowadays appears in many areas and is key to various scientific discoveries. For problems of large scale or ...dimensionality p, accuracy of estimation and computational cost are two top concerns. Recently, Candes and Tao have proposed the Dantzig selector using L₁-regularization and showed that it achieves the ideal risk up to a logarithmic factor log(p). Their innovative procedure and remarkable result are challenged when the dimensionality is ultrahigh as the factor log(p) can be large and their uniform uncertainty principle can fail. Motivated by these concerns, we introduce the concept of sure screening and propose a sure screening method that is based on correlation learning, called sure independence screening, to reduce dimensionality from high to a moderate scale that is below the sample size. In a fairly general asymptotic framework, correlation learning is shown to have the sure screening property for even exponentially growing dimensionality. As a methodological extension, iterative sure independence screening is also proposed to enhance its finite sample performance. With dimension reduced accurately from high to below sample size, variable selection can be improved on both speed and accuracy, and can then be accomplished by a well-developed method such as smoothly clipped absolute deviation, the Dantzig selector, lasso or adaptive lasso. The connections between these penalized least squares methods are also elucidated.
We study the asymptotic properties of the Adaptive LASSO (adaLASSO) in sparse, high-dimensional, linear time-series models. The adaLASSO is a one-step implementation of the family of folded concave ...penalized least-squares. We assume that both the number of covariates in the model and the number of candidate variables can increase with the sample size (polynomially or geometrically). In other words, we let the number of candidate variables to be larger than the number of observations. We show the adaLASSO consistently chooses the relevant variables as the number of observations increases (model selection consistency) and has the oracle property, even when the errors are non-Gaussian and conditionally heteroskedastic. This allows the adaLASSO to be applied to a myriad of applications in empirical finance and macroeconomics. A simulation study shows that the method performs well in very general settings with t-distributed and heteroskedastic errors as well with highly correlated regressors. Finally, we consider an application to forecast monthly US inflation with many predictors. The model estimated by the adaLASSO delivers superior forecasts than traditional benchmark competitors such as autoregressive and factor models.
Ordering of regression or classification coefficients occurs in many real-world applications. Fused Lasso exploits this ordering by explicitly regularizing the differences between neighboring ...coefficients through an
ℓ
1
norm regularizer. However, due to nonseparability and nonsmoothness of the regularization term, solving the fused Lasso problem is computationally demanding. Existing solvers can only deal with problems of small or medium size, or a special case of the fused Lasso problem in which the predictor matrix is the identity matrix. In this paper, we propose an iterative algorithm based on the split Bregman method to solve a class of large-scale fused Lasso problems, including a generalized fused Lasso and a fused Lasso support vector classifier. We derive our algorithm using an augmented Lagrangian method and prove its convergence properties. The performance of our method is tested on both artificial data and real-world applications including proteomic data from mass spectrometry and genomic data from array comparative genomic hybridization (array CGH). We demonstrate that our method is many times faster than the existing solvers, and show that it is especially efficient for large
p
, small
n
problems, where
p
is the number of variables and
n
is the number of samples.
PDF
