Summary
Penalization of the likelihood by Jeffreys’ invariant prior, or a positive power thereof, is shown to produce finite-valued maximum penalized likelihood estimates in a broad class of binomial ...generalized linear models. The class of models includes logistic regression, where the Jeffreys-prior penalty is known additionally to reduce the asymptotic bias of the maximum likelihood estimator, and models with other commonly used link functions, such as probit and log-log. Shrinkage towards equiprobability across observations, relative to the maximum likelihood estimator, is established theoretically and studied through illustrative examples. Some implications of finiteness and shrinkage for inference are discussed, particularly when inference is based on Wald-type procedures. A widely applicable procedure is developed for computation of maximum penalized likelihood estimates, by using repeated maximum likelihood fits with iteratively adjusted binomial responses and totals. These theoretical results and methods underpin the increasingly widespread use of reduced-bias and similarly penalized binomial regression models in many applied fields.
Technological advances have resulted in the introduction of new products and manufacturing processes that have a large number of characteristics and variables to be monitored to ensure the product ...quality. Simultaneous monitoring of variables under such a high-dimensional environment is quite challenging. Traditional approaches are less sensitive to the out-of-control signals in high-dimensional processes, especially when only a few variables are responsible for abnormal changes in the process output. Recently, variable selection-based charts are proposed to overcome the drawbacks of the traditional approaches. These approaches adopt diagnosis procedures to identify a small subset of potentially changed variables. However, the detection capability of these charts may be low in cases when the size of the shift is relatively small in a highly correlated data structure, which is critical in modern industry. Moreover, the complexity of computation would dramatically increase as the dimension of the process parameters increases due to the diagnosis procedure, which is inappropriate for online monitoring. This article proposes a new penalized likelihood-based approach via
norm regularization, which does not select variables but rather "shrinks" all process mean estimates toward zero. A closed-form solution of the proposed approach along with probability distributions of the monitoring statistic under null and alternative hypotheses are obtained, which makes the proposed chart significantly efficient to monitor high-dimensional processes. In addition, we explore theoretical properties of the approach and present several extensions of the proposed chart and its integration with other existing charts. Finally, we compare the performance of the approach with existing methods and present a case study of a high-speed milling process.
When developing risk prediction models on datasets with limited sample size, shrinkage methods are recommended. Earlier studies showed that shrinkage results in better predictive performance on ...average. This simulation study aimed to investigate the variability of regression shrinkage on predictive performance for a binary outcome. We compared standard maximum likelihood with the following shrinkage methods: uniform shrinkage (likelihood-based and bootstrap-based), penalized maximum likelihood (ridge) methods, LASSO logistic regression, adaptive LASSO, and Firth’s correction. In the simulation study, we varied the number of predictors and their strength, the correlation between predictors, the event rate of the outcome, and the events per variable. In terms of results, we focused on the calibration slope. The slope indicates whether risk predictions are too extreme (slope < 1) or not extreme enough (slope > 1). The results can be summarized into three main findings. First, shrinkage improved calibration slopes on average. Second, the between-sample variability of calibration slopes was often increased relative to maximum likelihood. In contrast to other shrinkage approaches, Firth’s correction had a small shrinkage effect but showed low variability. Third, the correlation between the estimated shrinkage and the optimal shrinkage to remove overfitting was typically negative, with Firth’s correction as the exception. We conclude that, despite improved performance on average, shrinkage often worked poorly in individual datasets, in particular when it was most needed. The results imply that shrinkage methods do not solve problems associated with small sample size or low number of events per variable.
Q.Clear, a Bayesian penalized-likelihood reconstruction algorithm for PET, was recently introduced by GE Healthcare on their PET scanners to improve clinical image quality and quantification. In this ...work, we determined the optimum penalization factor (beta) for clinical use of Q.Clear and compared Q.Clear with standard PET reconstructions.
A National Electrical Manufacturers Association image-quality phantom was scanned on a time-of-flight PET/CT scanner and reconstructed using ordered-subset expectation maximization (OSEM), OSEM with point-spread function (PSF) modeling, and the Q.Clear algorithm (which also includes PSF modeling). Q.Clear was investigated for β (B) values of 100-1,000. Contrast recovery (CR) and background variability (BV) were measured from 3 repeated scans, reconstructed with the different algorithms. Fifteen oncology body (18)F-FDG PET/CT scans were reconstructed using OSEM, OSEM PSF, and Q.Clear using B values of 200, 300, 400, and 500. These were visually analyzed by 2 scorers and scored by rank against a panel of parameters (overall image quality; background liver, mediastinum, and marrow image quality; noise level; and lesion detectability).
As β is increased, the CR and BV decreases; Q.Clear generally gives a higher CR and lower BV than OSEM. For the smallest sphere reconstructed with Q.Clear B400, CR is 28.4% and BV 4.2%, with corresponding values for OSEM of 24.7% and 5.0%. For the largest hot sphere, Q.Clear B400 yields a CR of 75.2% and a BV of 3.8%, with corresponding values for OSEM of 64.4% and 4.0%. Scorer 1 and 2 ranked B400 as the preferred reconstruction in 13 of 15 (87%) and 10 of 15 (73%) cases. The least preferred reconstruction was OSEM PSF in all cases. In most cases, lesion detectability was highest ranked for B200, in 9 of 15 (67%) and 10 of 15 (73%), with OSEM PSF ranked lowest. Poor lesion detectability on OSEM PSF was seen in cases of mildly (18)F-FDG-avid mediastinal nodes in lung cancer and small liver metastases due to background noise. Conversely, OSEM PSF was ranked second highest for lesion detectability in most pulmonary nodule evaluation cases. The combined scores confirmed B400 to be the preferred reconstruction.
Our phantom measurement results demonstrate improved CR and reduced BV when using Q.Clear instead of OSEM. A β value of 400 is recommended for oncology body PET/CT using Q.Clear.
GAMLSS is a general framework for fitting regression type models where the distribution of the response variable does not have to belong to the exponential family and includes highly skew and ...kurtotic continuous and discrete distribution. GAMLSS allows all the parameters of the distribution of the response variable to be modelled as linear/non-linear or smooth functions of the explanatory variables. This paper starts by defining the statistical framework of GAMLSS, then describes the current implementation of GAMLSS in R and finally gives four different data examples to demonstrate how GAMLSS can be used for statistical modelling.
Determining how to select the tuning parameter appropriately is essential in penalized likelihood methods for high dimensional data analysis. We examine this problem in the setting of penalized ...likelihood methods for generalized linear models, where the dimensionality of covariates p is allowed to increase exponentially with the sample size n. We propose to select the tuning parameter by optimizing the generalized information criterion with an appropriate model complexity penalty. To ensure that we consistently identify the true model, a range for the model complexity penalty is identified in the generlized information criterion. We find that this model complexity penalty should diverge at the rate of some power of log(p) depending on the tail probability behaviour of the response variables. This reveals that using the Akaike information criterion or Bayes information criterion to select the tuning parameter may not be adequate for consistently identifying the true model. On the basis of our theoretical study, we propose a uniform choice of the model complexity penalty and show that the approach proposed consistently identifies the true model among candidate models with asymptotic probability 1. We justify the performance of the procedure proposed by numerical simulations and a gene expression data analysis.
Episodic radiations in the fly tree of life Wiegmann, Brian M; Trautwein, Michelle D; Winkler, Isaac S ...
Proceedings of the National Academy of Sciences - PNAS,
04/2011, Volume:
108, Issue:
14
Journal Article
Peer reviewed
Open access
Flies are one of four superradiations of insects (along with beetles, wasps, and moths) that account for the majority of animal life on Earth. Diptera includes species known for their ubiquity (Musca ...domestica house fly), their role as pests (Anopheles gambiae malaria mosquito), and their value as model organisms across the biological sciences (Drosophila melanogaster). A resolved phylogeny for flies provides a framework for genomic, developmental, and evolutionary studies by facilitating comparisons across model organisms, yet recent research has suggested that fly relationships have been obscured by multiple episodes of rapid diversification. We provide a phylogenomic estimate of fly relationships based on molecules and morphology from 149 of 157 families, including 30 kb from 14 nuclear loci and complete mitochondrial genomes combined with 371 morphological characters. Multiple analyses show support for traditional groups (Brachycera, Cyclorrhapha, and Schizophora) and corroborate contentious findings, such as the anomalous Deuterophlebiidae as the sister group to all remaining Diptera. Our findings reveal that the closest relatives of the Drosophilidae are highly modified parasites (including the wingless Braulidae) of bees and other insects. Furthermore, we use micro-RNAs to resolve a node with implications for the evolution of embryonic development in Diptera. We demonstrate that flies experienced three episodes of rapid radiation--lower Diptera (220 Ma), lower Brachycera (180 Ma), and Schizophora (65 Ma)--and a number of life history transitions to hematophagy, phytophagy, and parasitism in the history of fly evolution over 260 million y.
Penalized likelihood methods are fundamental to ultrahigh dimensional variable selection. How high dimensionality such methods can handle remains largely unknown. In this paper, we show that in the ...context of generalized linear models, such methods possess model selection consistency with oracle properties even for dimensionality of nonpolynomial (NP) order of sample size, for a class of penalized likelihood approaches using folded-concave penalty functions, which were introduced to ameliorate the bias problems of convex penalty functions. This fills a long-standing gap in the literature where the dimensionality is allowed to grow slowly with the sample size. Our results are also applicable to penalized likelihood with the L 1 -penalty, which is a convex function at the boundary of the class of folded-concave penalty functions under consideration. The coordinate optimization is implemented for finding the solution paths, whose performance is evaluated by a few simulation examples and the real data analysis.
Ordered subset expectation maximization (OSEM) is the most widely used algorithm for clinical PET image reconstruction. OSEM is usually stopped early and post-filtered to control image noise and does ...not necessarily achieve optimal quantitation accuracy. As an alternative to OSEM, we have recently implemented a penalized likelihood (PL) image reconstruction algorithm for clinical PET using the relative difference penalty with the aim of improving quantitation accuracy without compromising visual image quality. Preliminary clinical studies have demonstrated visual image quality including lesion conspicuity in images reconstructed by the PL algorithm is better than or at least as good as that in OSEM images. In this paper we evaluate lesion quantitation accuracy of the PL algorithm with the relative difference penalty compared to OSEM by using various data sets including phantom data acquired with an anthropomorphic torso phantom, an extended oval phantom and the NEMA image quality phantom; clinical data; and hybrid clinical data generated by adding simulated lesion data to clinical data. We focus on mean standardized uptake values and compare them for PL and OSEM using both time-of-flight (TOF) and non-TOF data. The results demonstrate improvements of PL in lesion quantitation accuracy compared to OSEM with a particular improvement in cold background regions such as lungs.
Sparse estimation via penalized likelihood (PL) is now a popular approach to learn the associations among a large set of variables. This paper describes an R package called lslx that implements PL ...methods for semi-confirmatory structural equation modeling (SEM). In this semi-confirmatory approach, each model parameter can be specified as free/fixed for theory testing, or penalized for exploration. By incorporating either a L1 or minimax concave penalty, the sparsity pattern of the parameter matrix can be efficiently explored. Package lslx minimizes the PL criterion through a quasi-Newton method. The algorithm conducts line search and checks the first-order condition in each iteration to ensure the optimality of the obtained solution. A numerical comparison between competing packages shows that lslx can reliably find PL estimates with the least time. The current package also supports other advanced functionalities, including a two-stage method with auxiliary variables for missing data handling and a reparameterized multi-group SEM to explore population heterogeneity.
PDF
