Those working in healthcare today are challenged more than ever before to quickly and efficiently learn from data to improve their services and delivery of care. There is broad agreement that ...healthcare professionals working on the front lines benefit greatly from the visual display of data presented in time order.
To describe the run chart-an analytical tool commonly used by professionals in quality improvement but underutilised in healthcare.
A standard approach to the construction, use and interpretation of run charts for healthcare applications is developed based on the statistical process control literature.
Run charts allow us to understand objectively if the changes we make to a process or system over time lead to improvements and do so with minimal mathematical complexity. This method of analyzing and reporting data is of greater value to improvement projects and teams than traditional aggregate summary statistics that ignore time order. Because of its utility and simplicity, the run chart has wide potential application in healthcare for practitioners and decision-makers. Run charts also provide the foundation for more sophisticated methods of analysis and learning such as Shewhart (control) charts and planned experimentation.
Abstract
Objective
Motivated by the coronavirus disease 2019 (covid-19) pandemic, we developed a novel Shewhart chart to visualize and learn from variation in reported deaths in an epidemic.
Context
...Without a method to understand if a day-to-day variation in outcomes may be attributed to meaningful signals of change—rather than variability we would expect—care providers, improvement leaders, policy-makers, and the public will struggle to recognize if epidemic conditions are improving.
Methods
We developed a novel hybrid C-chart and I-chart to detect within a geographic area the start and end of exponential growth in reported deaths. Reported deaths were the unit of analysis owing to erratic reporting of cases from variability in local testing strategies. We used simulation and case studies to assess chart performance and define technical parameters. This approach also applies to other critical measures related to a pandemic when high-quality data are available.
Conclusions
The hybrid chart detected the start of exponential growth and identified early signals that the growth phase was ending. During a pandemic, timely reliable signals that an epidemic is waxing or waning may have mortal implications. This novel chart offers a practical tool, accessible to system leaders and frontline teams, to visualize and learn from daily reported deaths during an epidemic. Without Shewhart charts and, more broadly, a theory of variation in our epidemiological arsenal, we lack a scientific method for a real-time assessment of local conditions. Shewhart charts should become a standard method for learning from data in the context of a pandemic or epidemic.
BACKGROUND Quality improvement (QI) efforts have become widespread in healthcare, however there is significant variability in their success. Differences in context are thought to be responsible for ...some of the variability seen.
To develop a conceptual model that can be used by organisations and QI researchers to understand and optimise contextual factors affecting the success of a QI project.
10 QI experts were provided with the results of a systematic literature review and then participated in two rounds of opinion gathering to identify and define important contextual factors. The experts subsequently met in person to identify relationships among factors and to begin to build the model.
The Model for Understanding Success in Quality (MUSIQ) is organised based on the level of the healthcare system and identifies 25 contextual factors likely to influence QI success. Contextual factors within microsystems and those related to the QI team are hypothesised to directly shape QI success, whereas factors within the organisation and external environment are believed to influence success indirectly.
The MUSIQ framework has the potential to guide the application of QI methods in healthcare and focus research. The specificity of MUSIQ and the explicit delineation of relationships among factors allows a deeper understanding of the mechanism of action by which context influences QI success. MUSIQ also provides a foundation to support further studies to test and refine the theory and advance the field of QI science.
Decision-makers need signals for action as the coronavirus disease 2019 (COVID-19) pandemic progresses. Our aim was to demonstrate a novel use of statistical process control to provide timely and ...interpretable displays of COVID-19 data that inform local mitigation and containment strategies. Healthcare and other industries use statistical process control to study variation and disaggregate data for purposes of understanding behavior of processes and systems and intervening on them. We developed control charts at the county and city/neighborhood level within one state (California) to illustrate their potential value for decision-makers. We found that COVID-19 rates vary by region and subregion, with periods of exponential and non-exponential growth and decline. Such disaggregation provides granularity that decision-makers can use to respond to the pandemic. The annotated time series presentation connects events and policies with observed data that may help mobilize and direct the actions of residents and other stakeholders. Policy-makers and communities require access to relevant, accurate data to respond to the evolving COVID-19 pandemic. Control charts could prove valuable given their potential ease of use and interpretability in real-time decision-making and for communication about the pandemic at a meaningful level for communities.
Effective quality improvement (QI) depends on rigorous analysis of time-series data through methods such as statistical process control (SPC). As use of SPC has become more prevalent in health care, ...QI practitioners must also be aware of situations that warrant special attention and potential modifications to common SPC charts, which include skewed continuous data, autocorrelation, small persistent changes in performance, confounders, and workload or productivity measures. This article reviews these situations and provides examples of SPC approaches for each.
As the globe endures the coronavirus disease 2019 (COVID-19) pandemic, we developed a hybrid Shewhart chart to visualize and learn from day-to-day variation in a variety of epidemic measures over ...time.
Countries and localities have reported daily data representing the progression of COVID-19 conditions and measures, with trajectories mapping along the classic epidemiological curve. Settings have experienced different patterns over time within the epidemic: pre-exponential growth, exponential growth, plateau or descent and/ or low counts after descent. Decision-makers need a reliable method for rapidly detecting transitions in epidemic measures, informing curtailment strategies and learning from actions taken.
We designed a hybrid Shewhart chart describing four 'epochs' ((i) pre-exponential growth, (ii) exponential growth, (iii) plateau or descent and (iv) stability after descent) of the COVID-19 epidemic that emerged by incorporating a C-chart and I-chart with a log-regression slope. We developed and tested the hybrid chart using international data at the country, regional and local levels with measures including cases, hospitalizations and deaths with guidance from local subject-matter experts.
The hybrid chart effectively and rapidly signaled the occurrence of each of the four epochs. In the UK, a signal that COVID-19 deaths moved into exponential growth occurred on 17 September, 44 days prior to the announcement of a large-scale lockdown. In California, USA, signals detecting increases in COVID-19 cases at the county level were detected in December 2020 prior to statewide stay-at-home orders, with declines detected in the weeks following. In Ireland, in December 2020, the hybrid chart detected increases in COVID-19 cases, followed by hospitalizations, intensive care unit admissions and deaths. Following national restrictions in late December, a similar sequence of reductions in the measures was detected in January and February 2021.
The Shewhart hybrid chart is a valuable tool for rapidly generating learning from data in close to real time. When used by subject-matter experts, the chart can guide actionable policy and local decision-making earlier than when action is likely to be taken without it.
Conducting studies for learning is fundamental to improvement. Deming emphasised that the reason for conducting a study is to provide a basis for action on the system of interest. He classified ...studies into two types depending on the intended target for action. An enumerative study is one in which action will be taken on the universe that was studied. An analytical study is one in which action will be taken on a cause system to improve the future performance of the system of interest. The aim of an enumerative study is estimation, while an analytical study focuses on prediction. Because of the temporal nature of improvement, the theory and methods for analytical studies are a critical component of the science of improvement.
Could medical research and quality improvement studies be more productive with greater use of multifactor study designs?
Drawing on new primary sources and the literature, we examine the roles of A. ...Bradford Hill and Ronald A. Fisher in introducing the design of experiments in medicine.
Hill did not create the randomized controlled trial, but he popularized the idea. His choice to set aside Fisher's advanced study designs shaped the development of clinical research and helped the single-treatment trial to become a methodological standard.
Multifactor designs are not widely used in medicine despite their potential to make improvement initiatives and health services research more efficient and effective. Quality managers, health system leaders, and directors of research institutes could increase productivity and gain important insights by promoting a broader use of factorial designs to study multiple interventions simultaneously and to learn from interactions.
To understand county-level variation in case fatality rates of COVID-19, a statewide analysis of COVID-19 incidence and fatality data was performed, using publicly available incidence and case ...fatality rate data of COVID-19 for all 67 Alabama counties and mapped with health disparities at the county level. A specific adaptation of the Shewhart p-chart, called a funnel chart, was used to compare case fatality rates. Important differences in case fatality rates across the counties did not appear to be reflective of differences in testing or incidence rates. Instead, a higher prevalence of comorbidities and vulnerabilities was observed in high fatality rate counties, while showing no differences in access to acute care. Funnel charts reliably identify counties with unexpected high and low COVID-19 case fatality rates. Social determinants of health are strongly associated with such differences. These data may assist in public health decisions including vaccination strategies, especially in southern states with similar demographics.
PDF
