Analysis for Decision-Making, “How To” Brief #1
Reporting deaths from COVID-19 is important to monitor impact of the disease and guide response efforts. But confirmed COVID-19 deaths alone are unlikely to capture the full extent of the disease’s burden on a population. Analysis of excess mortality can help provide a more complete picture needed to support evidence-based policymaking.
TARGET AUDIENCE: Analytic staff who are responsible for producing data-driven insights to inform decisions on how countries, states and cities respond to the COVID-19 pandemic.
PURPOSE: To provide a general road map that analysts can follow to produce and present findings on mortality patterns related to COVID-19 in their jurisdiction.
WHY should you do this analysis?
- Some COVID-19 deaths may be incorrectly assigned a more general cause.
- People who die at home may never have been tested for the virus that causes COVID-19.
- Excess deaths may occur because health services are overtaxed by pandemic response. Though these deaths are not directly caused by COVID-19, they can be attributed indirectly to the outbreak (e.g. due to societal or health care system disruptions) and are relevant to policy response.
To monitor the outbreak comprehensively and adjust response strategies accordingly, health ministries should supplement tracking of confirmed COVID-19 deaths by comparing deaths during the pandemic — from all causes and from respiratory diseases — to prior mortality levels. This calculation of excess mortality shows if there is an increase in mortality — from COVID-19 or other causes — which can help to indicate the scale, as well as the indirect impact, of the outbreak.
Reviewing all-cause mortality data can help ministries answer key operational question such as:
- Is there a difference between current level of all-cause mortality and the expected range?
- Is that difference increasing or decreasing over time?
- Are there specific types of mortality (e.g. in home vs. in facilities, or among defined demographic groups) that are significantly higher than the usual baseline?
WHEN should you do this analysis?
This analysis can commence immediately with the identification and processing of baseline data. When an outbreak is anticipated or ongoing, current data should be processed on an ongoing weekly basis.
HOW do you complete this analysis?
Estimating the level of excess mortality is a relatively simple two-step analysis: (1) establish the baseline level of mortality and (2) compare current observed levels of mortality to baseline levels.
Establish baseline level of mortality
Mortality patterns demonstrate variations week to week within a year, even in the absence of an extraordinary event like the COVID-19 pandemic. When estimating the baseline mortality, analysts can account for this variation by doing the following3:
- Assemble week-by-week data on total deaths for several prior years (preferably the previous five, for example, 2015-2019).4
- Calculate the average5 weekly deaths across the years (i.e., the average of week 1 rates, week 2 rates, etc.).
- Estimate a confidence interval around the average number of deaths.
Compare baseline and observed mortality
The previous step yields a baseline mortality and a “normal” range of variability around that baseline (i.e., the confidence interval). Ongoing analysis can determine whether (and by how much) observed weekly mortality levels during the outbreak are outside the expected range. This comparison can be extended to specific demographic groups, geographic sub-units and causes (e.g. respiratory diseases), as data permits. Further analysis can also be done to calculate excess mortality from natural causes only.
HOW can you effectively interpret findings for decision-making?
The difference between the baseline and current mortality burden can be presumed to be the excess mortality that is related to the COVID-19 pandemic. Further analysis by age, sex and geography can identify the populations that are most affected. Analysis by cause (e.g. respiratory diseases or external causes) can further help to understand the mortality trends. Excess mortality should be interpreted in conjunction with data on public health and social measures that have been implemented as these may affect mortality patterns. Examining excess mortality burden in comparison to confirmed COVID deaths can also provide insights into gaps in disease surveillance.
A visualization that communicates these findings to decision-makers would simply show: (a) the range for the expected level of mortality and (b) the trajectory of actual mortality.
Weekly deaths in 2020 by age group, Switzerland 6
HOW are the results used?
Excess mortality provides a measure of the overall impact of COVID-19 on a population and health care system. Confirmed COVID-19 deaths alone may underestimate the true impact of the pandemic when testing is limited and cases remain undiagnosed. Excess mortality analysis on the other hand provides data on deaths that are both directly and indirectly related to COVID-19. The additional deaths identified in the excess mortality analysis include unconfirmed COVID-19 deaths as well as deaths from other conditions that occur due to disruptions in health care services. In conjunction with indicators related to testing, cases, hospitalizations and other measures, excess mortality from all causes can not only provide more complete information on the burden of the COVID-19 pandemic, it should help to inform public health action. In combination with the aforementioned indicators, excess mortality can be used to advocate for the importance of control measures to lessen the overall impact of COVID-19. If the number of observed deaths exceeds the expected threshold, decision-makers should examine relevant indicators to assess whether the health care system is responding adequately to the need for medical services or requires strengthening.
However, excess mortality measures have limitations that must be explored to confirm the results. The following questions should be asked and the answers may be used to interpret and adjust the results.
- Reporting lag – What is the time from death occurring to the inclusion of the death in the reported numbers?
- Underreporting – What proportion of deaths are not reported? If the number of observed deaths is below the expected threshold, decision-makers should explore whether these potential limitations are driving the results.
- Changes in other causes of death – Has there been a decrease in deaths due to external causes that might explain the decrease? Stratify the observed deaths according to Natural vs External causes.
- Vital Strategies, et al (2020). Excel mortality calculator. NY: Vital Strategies (https://staging.epidemics.plumbweb.io/covid19/resources/excess-mortality/)
- Vital Strategies, et al (2020). Revealing the Toll of COVID-19: A Technical Package for Rapid Mortality Surveillance and Epidemic Response. New York: Vital Strategies (https://staging.epidemics.plumbweb.io/covid19/resources/rapid-mortality-surveillance/)
- Analysts can use an Excel spreadsheet tool to perform this process. (https://vital.box.com/v/ExcessMortalityCalculator)
- Where historical baseline data are not available, expected number of deaths can be estimated using additional information about the age and gender distribution of the population.
- Other methods can also be used to obtain the baseline, such as the forecasting function in Excel.
- Federal Statistical Office. Mortality, causes of death. 2020 [cited 2020 April 4]; Available from: https://www.bfs.admin.ch/bfs/en/home/statistics/health/state-health/mortality-causes-death.html.