Lightweight Evaluation Methodology For Value-Based Care

Not Your Father’s Evaluation Methodology: A Nimble Path to Value-Based Care

As commercial healthcare payers and providers endeavor to improve care and lower costs, they must consider an increasing number of technologies, services, and methods. Many claim the ability to accelerate the shift to value-based care. However, these promises are challenging to validate without a standardized repository on the efficacy of new technologies and methods. Therefore, payers and providers are increasingly adopting lightweight evaluation methodologies to generate rigorous evidence of impact and help prioritize investments.

Current Environment for Value-Based Care Tools and Tech

Healthcare payers and providers are inundated with choices for technologies, data analysis tools, and processes promising to efficiently deliver high-quality care. Although new technologies can be valuable tools, resources can be drained when choosing between myriad options that claim to help organizations shift to VBC without precise knowledge of the most effective options.

According to a 2021 report released by the IQVIA Institute for Human Data Science, more than 350,000 health apps are available for consumers. Over 90,000 were released in 2020 alone. These include general and disease-specific digital health tools, digital therapeutics, wearables, sensors, and digital biomarkers.

In addition to these consumer-facing products, payers and providers also have an overwhelming number of options for collecting patient data, predicting, and visualizing trends, and measuring quality performance.

Misinformed Decisions Are Detrimental to the Mission

There is no clear, comprehensive, and validated source of information about the impact of alternative products and methods. Unsure of the most efficient ways to review options, payers and providers are hamstrung in their efforts to select optimal technologies.

To the detriment of patients, technology investment decisions are often rooted in biased educated guesses, influenced by marketing pitches, or shaped by logistical pressures. As a result, we’ve seen choices made without stakeholders possessing a clear understanding of the likely impact on key outcomes.

Evaluation of Value-Based Investments Can Be Efficient and Impactful

The advisors at RTI Health Advance champion rigorous, light-touch, and inexpensive evaluations that offer an antidote to this problem. Evaluations needn’t involve large teams, span multiple years, or consume considerable resources to accurately estimate the business impact of adopting a new technology or method.

We help sort through the morass of potential value-based investments to ensure that every dollar payers and providers spend demonstratively improves patient health. Organizations like yours are increasingly adopting academic evaluation methods adapted for commercial use to project the financial and health impacts of new technologies, consumer health products, or data management solutions before making a full-fledged investment.

When considering evaluating an intervention before a full-scale rollout it is critical to select an approach that delivers rigorous evidence at the right budget within an acceptable timeframe.

Here are some of the most common evaluation approaches used in the commercial health care market to inform strategic decisions:

Opportunity Evaluation Method: Randomized Controlled Trial

Randomized controlled trials (RCTs) are the “gold standard” in evaluation design. These studies most clearly establish a causal link between a program/initiative and an outcome. A purely randomized evaluation establishes a control group by randomly offering access to services to one subpopulation and tracking outcomes for the treatment and control group. 

RCTs are often the most expensive among the evaluation designs reviewed in this article because they require active manipulation of program/technology access. However, when an RCT is logistically and financially feasible, this method can capture an unbiased estimate of an intervention’s impact on total cost of care, healthcare quality, healthcare equity, or any measurable outcome.

Besides cost, another consideration is that because RCTs rely on withholding access to the intervention from the control group, a pure randomized experiment can only be implemented when there is no requirement to offer the entire population access to the technology/program. When this is not tenable, a similar effect can be achieved by randomizing any feature of the enrollment process that impacts the probability of user uptake (such as by randomly assigning cohorts to differing types of marketing outreach).

Opportunity Evaluation Method: Retrospective Analysis

When conducting an RCT is not feasible, payers and providers can often use historical data to conduct evaluations of similar rigor. For example, if a digital tool, business strategy, or data analytics system has been deployed on a limited basis (or even if a similar tool has been tried), it is typically possible to use quasi-experimental approaches such as propensity score matching and regression discontinuity to construct a control group. The key requirement is data on outcomes (such as medical claims for cost outcomes) and indicators of when the intervention, technology, or process being evaluated was introduced.

Retrospective analyses can blend internal and external data to yield similarly precise measurements of an intervention’s impact on healthcare cost or quality relative to randomized trials. Sometimes quasi-experimental studies involve additional data pre-processing steps necessary to construct a control group from real-world data. However, because these studies rely on historical data, they are typically more rapid than RCTs.

Opportunity Evaluation Method: Forecasting 

Sometimes projecting a technology’s future impact is more important than estimating its historical impact. Forecasting is particularly useful when historical data may be unreliable or when a technology is being introduced to a new population or combined in a novel fashion with other interventions.

Forecasting models typically leverage intermediate estimates of a program or technology’s impact gleaned from internal or external research to project long-term impacts, simulate outcomes in various scenarios, and display cost savings. While quantitative forecasting allows organizations to weigh a greater variety of alternatives than can be feasibly evaluated separately, it hinges upon obtaining trustworthy estimates of each model parameter. The reliability of the forecast depends on the collective precision of its inputs.

Opportunity Evaluation Method: Qualitative Analysis

Quantitative analyses can estimate the impact on an outcome but cannot elucidate why or how a technology works. Understanding how technologies function for users is critical for scaling up, tweaking, or retargeting an intervention.

Incorporating qualitative analyses into any of the above approaches can help to unpack this “black box.” Qualitative investigation can include interviews or focus groups with users, real-world observation, or surveys. While this does tend to increase evaluation costs, it ultimately can greatly improve a study’s impact by helping translate research evidence into impactful business strategy. Qualitative analysis is particularly useful for test-iterate-implement research lifecycles.

VBC Technology Evaluations Help You Choose Wisely

Many payers and providers have sufficient data on members, costs, methods, and alternative technologies available to apply the evaluation methods described above to evaluate technology investments more critically. 

When implementing a brand-new value-based care solution or technology without precedent, minor modifications to product roll-out can enable a critical evaluation of product efficacy, which subgroups it works best for, and how or if it should be scaled. 

The number of new technologies and methods promising to deliver value-based care will only increase. The health care evaluation experts at RTI Health Advance can help plan, implement, and interpret lightweight evaluations critical to improving patient health and lowering costs.

Contact us to learn more about applying these methods to your payer or provider organization.