Beyond Averages: Health Equity Requires Consideration Of Distributions & Stratifications

Standard measures and models for health and healthcare are typically based on a per-patient average expected return in health for each intervention investment. While this approach is an excellent starting place, it's limited and avoids addressing 3 essential questions:

1. How is the expected impact different across the spectrum of potential participants?
2. What outcomes data is needed beyond standard measures of cost and effectiveness?
3. Who isn't represented in the data used to populate the models?

These 3 questions are critical to generating useful models for cost-effectiveness toward advancing health equity objectives. When designed well, addressing these questions can simultaneously improve the overall validity and usefulness of data analysis models while providing a foundation for more equitable impact.

Averages hide disparities and may lead to statistical issues

Cost-effectiveness models often focus on expected values and averages. A typical cost-effectiveness model might produce the expected or average incremental cost-effectiveness ratio as its key output. By focusing on only this average, we can easily miss major issues across the distribution of effects. In some cases, we will see substantial errors in the modeling.

Different types of people will almost certainly have different expected impacts in cost and effectiveness. Nearly all healthcare services and interventions are tied in complex ways to the larger healthcare and socioeconomic system. As a result, some interventions and services may work better for historically underserved communities who may have greater care needs.

Conversely, some interventions may require additional support to be effective for underserved populations. If we look exclusively at the average and fail to account for the diversity of people impacted, we may miss key social factors in an intervention's cost-effectiveness. In extreme cases, a small group of people with very particular social, health, and economic conditions may drive effects (or lack thereof). Equity requires that we go beyond just averages and consider these effects across the spectrum of people, particularly for the underserved.

Statistical thinking beyond averages

Looking exclusively at averages without considering distributions can also lead to errors. One common error is mistakenly assuming that the ratio of averages is the average ratio, which is particularly problematic because typical cost-effectiveness measures are ratio measures. This can be addressed more directly by looking at the distributions of cost-effectiveness measures across the population, rather than just the averages.

In some ways, looking at the distributions of effects across the population is similar to considering the distribution of uncertainty. For both purposes, assessing a central value only can be misleading and less informative for decision-making.

Statistical stratification to evaluate health equity

Another practical way to address these problems is to pre-define social stratifications to examine, such as race, gender, location, or income. The exercise of thinking through how different groups might be impacted can be valuable, providing testable hypotheses and information for decision making. In turn, those persons can better care for patients through more data-informed decisions based on social determinants of health.

As with clinical trial design, testing these hypotheses comes with a cost: in most cases, studies will not be sufficiently powered or designed to examine these issues unless it is built into the design initially. Examining more factors inherently requires more statistical power, requiring that analysts plan for these extra analyses from the beginning. In general, best practices include running a small number of key stratified analyses and comparisons across different social groups, considering any remaining groups as secondary or for exploratory analyses. When collecting primary data for a cost-effectiveness study, it may also be valuable to stratify the population by those metrics in the recruitment phase.

Consider those outside the data and model

Researchers should consider 2 different forms of "missingness":

1. People who are not represented in the underlying datasets for model parameters
2. People who are not represented in the profile of people that are expected to be reached

The types of clinical trials and studies that are typically used to populate model parameters tend to draw from more privileged and less disparaged groups. Similarly, cost data, such as those from claims databases, will tend to draw from those who are highly engaged with the healthcare system. As a result, most cost-effectiveness models tend to undercount the historically underserved.

Related to equity concerns, take note that populations that most need to be included are also the least likely to already exist in the data.

Know which measures indicate cost and effectiveness

Quality-adjusted life-years (QALYs) are often used as the standard basis for cost-effectiveness models. However, like any measure, QALYs have built-in inherent assumptions. Which types of health issues are and are not adjusted for in QALYs are decisions that are built into the measure. Similarly, the functions and processes that are often used to adjust quality are reflected in the preferences and experiences of those who were involved in the elicitation process. In both cases, the individuals contributing to and making those decisions are more likely to come from highly privileged and highly engaged backgrounds. As a result, these quality weightings may not be reflective of the preferences and experiences of the individuals with greatest health equity interest.

3 steps to evaluate a cost-effectiveness ratio

1. One of the most important aspects for addressing equity in cost-effectiveness modeling is the analysis team itself. Recruit people from underserved populations into core leadership and design a team that can adequately guide the model toward addressing equity needs.
2. Ensure that health equity issues are a part of the initial study design, considering potential equity-related biases at each stage. Designing with equity in mind typically requires more time for design, a large pool of complex data, and a wider variety of considerations.
3. Make statistical adjustments to the data as appropriate so that the core study design has adequate support. In general, it is better to use study design in conjunction with planned statistical adjustments, if necessary, rather than rely entirely on statistics to make up for weak design. This issue can be alleviated by using number of methods like equity- and target-based weighting.

Finally, concentrate results reporting on the distributions of effects across the population rather than the average cost-effectiveness measures. This approach could include reporting on differences among groups or showing the complete distribution of expected effects.

Cost-effective health equity solutions

Designing cost-effectiveness models for equity-based considerations requires thinking beyond standard metrics and procedures. Reconsider assumptions, expand statistical approaches, and enhance data science practices. RTI Health Advance supports health equity and data analysis teams to step outside the standard analytical boxes to address equity head on. Reach out to our team to learn more about how we can help.