Algorithmic Bias In Clinical Decision-Making Models

Uncovering biases in predictive models in healthcare

Widely used across the many facets of healthcare, predictive analytics adopt various forms and functions such as: a statistical model that uses patterns observed in historical data to predict the likelihood or probability of a future outcome; or an algorithm that either assigns weights or segments individuals, encounters, and observations into risk buckets/cohorts based on historical or baseline data.

Mainstream predictive healthcare tools in use

Some well-known and validated examples of such predictive tools include the Johns Hopkins Adjusted Clinical Groups (ACG) system, the 3M Clinical Risk Groups (CRGs) system, and the Charlson Comorbidity Index (CCI). Each uses historical or baseline data for the purpose of stratifying populations to identify those at highest risk. The use cases and potential benefit gained from adopting predictive analytics in healthcare organizations can be quite extensive. For example, through predictive analytics, organizations can proactively identify individuals who are at high risk of developing chronic conditions and thus target these individuals for early intervention, timely diagnosis, and prevention of adverse health outcomes that can be both financially and physically costly to the individual.

The role of predictive models in healthcare

There is a myriad of validated predictive analytic tools that have been adopted in healthcare and proven to provide a benefit to population health management. Healthcare systems across the US are subject to financial penalties under the value-based Center for Medicare and Medicaid Services (CMS) Readmissions Reduction Program (HRRP).

Predictive modeling and readmission rates

Hospitals in the HRRP must work to ensure that their 30-day readmission rates are below the national average for cohorts of patients being treated for conditions, including myocardial infarction, congestive heart failure, pneumonia, chronic obstructive pulmonary disease (COPD), total knee replacement, and total hip replacement. Therefore, many healthcare organizations rely on predictive models or algorithms like the model developed by van Walraven et al. to target individuals who are at high risk for a 30-day readmission. This algorithm is referred to as the LACE Index representing Length of stay (L), Acuity of admission (A), Comorbidities (C), and recent emergency department use (E).

Predictive modeling and outcome improvement

Beyond the example of readmissions, predictive models and algorithms have an effective track record for helping healthcare organizations improve outcomes in other facets of healthcare. This includes an electronic medical record (EMR)-based model predicting patients who are at high risk of not appearing for a scheduled clinic visit. Another is the Sepsis “Sniffer" Algorithm (SSA), a predictive analytic tool initiating a digital alert in the EMR system for patients at high risk for developing sepsis during or following an inpatient encounter.

Given their ability to proactively identify high-risk patients more proactively, many healthcare organizations rely on the output of predictive models or algorithms when engaging in clinical decision making, such as triggering patients who may need additional resources from care management or identifying patients with poor prognosis and who might benefit from earlier treatment interventions.

Predictive algorithms support VBC and health equity

As the current healthcare system is shifting its focus to a more value-based care approach, there is also much more emphasis on improving health equity. CMS just recently released its 2022-2023 health equity framework outlining a strategy to reduce health disparities and outcomes and to eliminate barriers to CMS-supported benefits, services, and coverage in the next 10 years. Amidst the prioritization of improving health equity, it will be imperative to dissect predictive models or algorithms that are currently in use in healthcare to uncover any algorithmic biases that can be damaging to health equity efforts. Also, future predictive analytic tools will need to include methods to address algorithmic bias in development and validation phases to ensure that the use of these models and algorithms will not widen disparities.

Challenges of algorithmic bias in healthcare

The term algorithmic bias was first introduced by Panch and colleagues as “the instances when the application of an algorithm compounds existing inequities in socioeconomic status, race, ethnic background, religion, gender, disability, or sexual orientation to amplify them and adversely impact inequities in health systems." The underlying reasons for the presence of algorithmic biases that can be present in predictive models and algorithms can vary. Reasons may include the quality of the methods applied to the development of the model like poor sample size, how missing data is handled, proper validation of model in testing cohorts, and/or properly addressing model overfitting.

Source of algorithmic bias is inherent

Some underlying reasons for algorithmic biases can be more inherent such as underrepresentation of historically vulnerable and underrepresented populations in training and testing data used for algorithm and model development. Historically and economically marginalized populations can include those who lack the social capacity to represent themselves, such as children, incarcerated individuals, students, and those living in poverty.

These persons may be from an ethnic minority or gender population who have undergone historical discriminations and continue to be subject to systemic structural biases. In data used to train and test predictive models or algorithms, a lack of equal representation from historically marginalized persons skews to include persons from more protected groups. This flaw leads to a predictive tool that can be highly biased in stratification (classification of risk across populations) and misclassification of risk (wrong assignment of risk).

Implications of algorithmic bias in healthcare

In clinical decision-making processes, there can be biases in patients who may be triggered for a referral to a home-based care program, or to receive a digital care monitoring device, or to be identified as eligible for clinical trial recruitment.

Obermeyer and colleagues uncovered biases in one of the largest commercial-risk predictive algorithms used to target patients for high-risk care management programs. Their findings concluded that Black patients were found to be significantly sicker than White patients with the same risk score. Black patients in the same risk score of White patients were found to have more severe and less controlled hypertension, diabetes, renal failure, anemia, and cholesterol.

Vyas et al. describe that the continued use of nationally adopted algorithms, such as the Kidney Donor Risk Index (KDRI) and the vaginal birth after cesarean (VBAC) algorithm, will continue to contribute to disparities. The KDRI reduces the likelihood of donation from a Black donor, marking them as high risk of graft failure, thus leading to wait-time disparities among Black patients given their likelihood to receive kidneys from Black donors. Similarly, the VBAC algorithm that lowers the likelihood or estimate for VBAC success for individuals of color could widen disparities, especially since there are already high disparities in maternal mortality rates.

Countering bias in predictive analytics

Leaning on clinical decisions made from the output of biased predictive analytics will thwart our efforts to improve health equity. What can be done to ensure that current and future predictive models and algorithms are bias free or at the least, very minimally biased? Given the burgeoning growth of Artificial Intelligence (AI) and machine-based learning models, there have been frameworks and methods developed to measure and address algorithmic bias.

Framework 1

One objective framework for detecting algorithmic bias that is of great scientific rigor and quality is posed by Paulus and Kent, which is purposed in mitigating unfairness in predictive analytical tools. The first step suggested in their framework to reduce algorithmic bias in predictive models used to aid in clinical decision-making processes for balancing harms-benefits is the inclusion of representative samples for model development and validation.

Training and testing cohorts that mirror their target populations will ensure a representative sample. Subsequently, subgroup validity should be diagnosed by evaluating the discrimination and calibration of the model performance overall and among and within key social factors, such as race and gender. Intersectionality is an important concept to include in these steps to detect and uncover biases within social strata (e.g., differences between social classes within a race or ethnic group).

Framework 2

Additionally, Paulus and Kent outline a second framework for reducing algorithmic unfairness among models and algorithms used for rationing. There are 2 approaches that can be applied for reducing algorithmic unfairness.

1. The first is restricting model inputs to well-established causal risk factors. This approach includes being cautious about what variables are used to generate model outputs and including those with a causal relationship with the outcome and avoiding proxies.
2. The second concept is examining the distribution of fairness in model outputs. This approach includes applying fairness criteria, such as examining that the probability of correct classification conditional on the modeled outcome is the same for all values of race, ethnicity, or gender; or assessing the average prediction for patients with the outcome which ensures that sensitivity and false negative rates are stable across all values of race, ethnicity, and gender.

Selecting bias-free measures

Beyond the framework outlined by Paulus and Kent, choosing outcomes and measures that are free of inherent biases is another crucial part of predictive model and algorithm development. For example, individuals who do not access and engage with the healthcare system may have lower costs, and therefore, using costs as a surrogate for prediction of future healthcare outcomes and utilization could lead to widening disparities between those who access and engage with the system vs those who do not.

Additionally, Wang and colleagues have offered a checklist that can be implemented to critically assess if a predictive analytical model or algorithm is introducing any biases. One guiding component of the checklist is answering 11 questions to determine the presence of 6 sources of potential bias including:

1. Label bias
2. Model bias
3. Population bias
4. Measurement bias
5. Missing validation bias
6. Human use bias

RTI Health Advance supports the battle against bias in predictive analytics

For big data analytics and health equity to cooperate simultaneously, we must begin to acknowledge and account for the inherent biases that predictive analytics can introduce. It will require a systematic and rigorous evaluation of predictive model and algorithm solutions to ensure that disparate outcomes are not widened, and that the predictive solution stratifies and segments populations bias free. Please contact us to learn more about how we can help you to detect algorithmic biases in your predictive analytical tools and provide solutions to overturn biased estimates and outputs.