The successful development of new drugs with a companion diagnostic based on genomic alteration of an oncogene has led to rethinking of all phases on clinical development of cancer drugs. We ...critically review some of the new clinical trial designs for biomarker‐based cancer drug development. We try to clarify the objectives of the new designs and examine completed trials using these designs to evaluate what has been learned about these designs.
The established molecular heterogeneity of human cancers necessitates the development of new paradigms to serve as a reliable basis for precision medicine. The assumptions underlying some of the ...conventional approaches to clinical trial design and analysis are no longer appropriate because of the molecular heterogeneity of tumors of a given primary site. This article reviews some clinical trial designs that have been actively applied in the codevelopment of therapeutics and predictive biomarkers to inform their use in oncology. These include the enrichment design, the basket design, and the umbrella design. Oncology leads most other therapeutic areas in development of personalized or precision medicine. Personalized or precision medicine is practiced daily in oncology on the basis of tumor genomics and may evolve in other therapeutic areas as it has in oncology, rather than according to inherited polymorphisms as so often imagined. Consequently, some of the clinical trial designs described here may serve as a possible blueprint for therapeutic development in fields other than oncology.
Many cancer treatments benefit only a minority of patients who receive them. This results in an enormous burden on patients and on the health care system. The problem will become even greater with ...the increasing use of molecularly targeted agents whose benefits are likely to be more selective unless the drug development process is modified to include co-development of companion diagnostics. Whole genome biotechnology and decreasing costs of genome sequencing make it increasingly possible to achieve an era of predictive medicine in oncology therapeutics. The challenges are numerous and substantial but are not primarily technological. They involve organizing publicly funded diagnostics of deregulated pathways, adopting new paradigms for drug development, and developing incentives for industry to incur the complexity and expense of co-development of drugs and companion diagnostics. This article reviews some designs for phase III clinical trials that may facilitate movement to a more predictive oncology.
Cross-validation (CV) is an effective method for estimating the prediction error of a classifier. Some recent articles have proposed methods for optimizing classifiers by choosing classifier ...parameter values that minimize the CV error estimate. We have evaluated the validity of using the CV error estimate of the optimized classifier as an estimate of the true error expected on independent data.
We used CV to optimize the classification parameters for two kinds of classifiers; Shrunken Centroids and Support Vector Machines (SVM). Random training datasets were created, with no difference in the distribution of the features between the two classes. Using these "null" datasets, we selected classifier parameter values that minimized the CV error estimate. 10-fold CV was used for Shrunken Centroids while Leave-One-Out-CV (LOOCV) was used for the SVM. Independent test data was created to estimate the true error. With "null" and "non null" (with differential expression between the classes) data, we also tested a nested CV procedure, where an inner CV loop is used to perform the tuning of the parameters while an outer CV is used to compute an estimate of the error. The CV error estimate for the classifier with the optimal parameters was found to be a substantially biased estimate of the true error that the classifier would incur on independent data. Even though there is no real difference between the two classes for the "null" datasets, the CV error estimate for the Shrunken Centroid with the optimal parameters was less than 30% on 18.5% of simulated training data-sets. For SVM with optimal parameters the estimated error rate was less than 30% on 38% of "null" data-sets. Performance of the optimized classifiers on the independent test set was no better than chance. The nested CV procedure reduces the bias considerably and gives an estimate of the error that is very close to that obtained on the independent testing set for both Shrunken Centroids and SVM classifiers for "null" and "non-null" data distributions.
We show that using CV to compute an error estimate for a classifier that has itself been tuned using CV gives a significantly biased estimate of the true error. Proper use of CV for estimating true error of a classifier developed using a well defined algorithm requires that all steps of the algorithm, including classifier parameter tuning, be repeated in each CV loop. A nested CV procedure provides an almost unbiased estimate of the true error.
Oncologists need improved tools for selecting treatments for individual patients. The development of therapeutically relevant prognostic markers has traditionally been slowed by poor study design, ...inconsistent findings, and lack of proper validation studies. Microarray expression profiling provides an exciting new technology for relating tumor gene expression to patient outcome, but it also provides increased challenges for translating initial research findings into robust diagnostics that benefit patients and physicians in therapeutic decision making. This article attempts to clarify some of the misconceptions about the development and validation of multigene expression signature classifiers and highlights the steps needed to move genomic signatures into clinical application as therapeutically relevant and robust diagnostics.
Abstract Objective This study investigated the effects of sleep extension on tennis serving accuracy, as well as daytime sleepiness in college varsity tennis players. Methods Twelve (seven females ...and five males) healthy students on a college varsity tennis team maintained their habitual sleep–wake schedule for a one-week baseline period followed by a one-week sleep extension period. Participants were requested to sleep at least nine hours, including naps, during the sleep extension period. Serving accuracy was assessed when participants were sleep deprived (prior to the sleep extension period) and after the sleep extension period. Levels of daytime sleepiness were monitored via the Epworth Sleepiness Scale and the Stanford Sleepiness Scale , and caffeine consumption was recorded throughout the study. Results Participants slept significantly more in the second week — the sleep extension week — compared with the first week — the baseline week (8.85 vs. 7.14 h; p < 0.05). Following the sleep extension period, accuracy of the tennis serves improved significantly (35.7% vs. 41.8%; p < 0.05), and the Epworth Sleepiness Scale and Stanford Sleepiness Scale scores declined significantly (12.15 vs. 5.67; p < 0.05 and 3.56 vs. 2.67; p < 0.05, respectively). Conclusions This study demonstrates that an increase in sleep of approximately 2 h per night significantly increased athletic performance in college varsity tennis players.
Background Both the validity and the reproducibility of microarray-based clinical research have been challenged. There is a need for critical review of the statistical analysis and reporting in ...published microarray studies that focus on cancer-related clinical outcomes. Methods Studies published through 2004 in which microarray-based gene expression profiles were analyzed for their relation to a clinical cancer outcome were identified through a Medline search followed by hand screening of abstracts and full text articles. Studies that were eligible for our analysis addressed one or more outcomes that were either an event occurring during follow-up, such as death or relapse, or a therapeutic response. We recorded descriptive characteristics for all the selected studies. A critical review of outcome-related statistical analyses was undertaken for the articles published in 2004. Results Ninety studies were identified, and their descriptive characteristics are presented. Sixty-eight (76%) were published in journals of impact factor greater than 6. A detailed account of the 42 studies (47%) published in 2004 is reported. Twenty-one (50%) of them contained at least one of the following three basic flaws: 1) in outcome-related gene finding, an unstated, unclear, or inadequate control for multiple testing; 2) in class discovery, a spurious claim of correlation between clusters and clinical outcome, made after clustering samples using a selection of outcome-related differentially expressed genes; or 3) in supervised prediction, a biased estimation of the prediction accuracy through an incorrect cross-validation procedure. Conclusions The most common and serious mistakes and misunderstandings recorded in published studies are described and illustrated. Based on this analysis, a proposal of guidelines for statistical analysis and reporting for clinical microarray studies, presented as a checklist of “Do's and Don'ts,” is provided.
Combination therapy programs are the hallmark of the successful treatment of all forms of human malignancies. In this issue of Cell, Palmer and Sorger present data suggesting that cell culture ...results indicative of synergistic anticancer drug interactions rarely translate clinically and that the results of combination therapies in mouse models or human clinical trials, even if successful, are best explained by the independent activities of the individually administered drugs.
Combination therapy programs are the hallmark of the successful treatment of all forms of human malignancies. In this issue of Cell, Palmer and Sorger present data suggesting that cell culture results indicative of synergistic anticancer drug interactions rarely translate clinically and that the results of combination therapies in mouse models or human clinical trials, even if successful, are best explained by the independent activities of the individually administered drugs.
PDF
