Digital Health App Scorecards: Assessing For Purchase, Development, Or FDA Approval

There are over 350K mobile health apps currently on the market with about 200 being added each day. Of those considered digital therapeutics (DTx), only 35 to 40 have been approved by the FDA since 2017; also known as Prescription Digital Therapeutics (PDTx) or Software as a Medical Device (SaMD). Because digital health applications are largely unregulated and evidence generation can be expensive, individuals, payers, and providers are often challenged in their efforts to assess effectiveness, value, reliability, and quality based on the claims alone. One approach that can help with assessing digital health apps and digital therapeutics is the use of a scorecard methodology. A scorecard approach provides a standards-based framework and a visual tool for collecting, summarizing, and analyzing performance data based on an evaluation rubric and scoring range.

Objective evaluation of digital health solutions is challenging

With rapid growth in the number of digital therapeutics and digital health apps, it is challenging to find an objective assessment of effectiveness. It is difficult for healthcare leaders to effectively assess the value of digital health apps and platforms to meet their specific needs.

Digital health scorecards offer a more objective evaluation

Our research shows that a tailored approach to assessment-via-scorecard is preferred by the payer and provider organizations we contacted. The value of the right scorecard ensures that healthcare organizations, employers, and software developers make data-driven decisions that align with their clinical and business objectives.

Accelerating digital health adoption requires assessment standards

While investment in digital health solutions is high, and growing, adoption by provider and payer organizations has been slow. Currently, only 40% of payers are covering DTx. A survey conducted by Pear Therapeutics and Avalere highlights it's potential with 80% of payers believing DTx may be effective as a complementary therapy, and 67.5% thought that FDA authorization of DTx is 'very important' when deciding to cover them.

To accelerate adoption of digital health solutions, there needs to be clear evaluation criteria that enable healthcare and clinical leaders to determine if a particular solution or technology is right for them.

"Having a standard way to evaluate and compare widely varying options is needed. A tool with unbiased information about client experience, a reality check of how well they delivered against what was promised, and the ease of implementation/degree to which integration and experience were seamlessness would be especially useful."- RTI Health Advance health plan interview

RTI Health Advance research suggests that a digital health scorecard method can be useful for payers and providers

RTI Health Advance worked with a team of students at the UNC Gillings School of Public Health to develop a capstone project that included brief surveys and interviews with patient advocates, healthcare provider, and payer organizations to understand their needs for evaluating digital health, including DTx and PDTx solutions, and the challenges they face with objective evaluation of available technologies.

Top three desires for evaluating digital health, digital therapeutics, and prescription digital therapeutics tools

The team identified several challenges with digital health apps that stand in the way of healthcare organizations integrating digital health apps into their workflows: the need for clinical evidence of their claims, better data integration and management, and greater usability, for patients and providers

Figure 1 highlights a few direct quotes taken from interviews with a patient advocacy group, a large employer, and a payer organization that frame the top three priorities for a scorecard – quality of evidence, usability, and how easily the data flows.

Payers and providers also want a voice in what criteria are important to them in the evaluation

Our survey found that stakeholders want to be certain that the criteria that are important to them are assessed in evaluating digital health solutions. The most preferred approach was to use a reliable scorecard framework that can be tailored to their clinical, operational, and business objectives as shown in figure 2.

Digital health scorecard: Use a standard framework and tailor for specific needs

Existing digital health scorecards provide tools with limitations

Several scorecard models exist, including Johns Hopkins, AHRQ, the Mobile App Rating Scale (MARS), the App Quality Assessment Tool or Health-Related Apps (AQUA), the THESIS approach, Digital Technology Assessment Criteria (DTAC) from the UK's NHS, and ORCHA also from the UK.

Some assessment tools incorporate therapeutic concepts though many limit their review to key technical elements such as usability or data privacy and security. Other assessment tools focus on technology integration or architecture but do not assess clinical effectiveness claims. Ideally, a basic scorecard would assess key technical aspects and usability and also allow users to tailor the measures to assess other key priorities around clinical, consumer, engagement, or equity dimensions.

Creating or tailoring digital therapeutics scorecard criteria

As mentioned, while some frameworks may feature advanced application quality assessments, no scorecard or assessment tool provides a comprehensive application quality assessment, incorporating end-user requirements alongside technical, usability, and clinical dimensions which is why tailoring any baseline assessment for individual healthcare organizations or programs is critical.

When tailoring any available digital health scorecard or evaluation tool, here are three areas that we suggest you consider – clinical risk, stickiness, and inclusivity.

Purpose and clinical risk

Clarity on desired outcomes, e.g., clinical, quality of life, education, and engagement, is paramount. What are the goals of the care plan and does this application or therapeutic address those goals? The evidence required would depend upon the claims made.

Most important to digital health app adoption is safety in addition to quality. One study found that more than 75% of people with blood pressure in the hypertensive range received falsely reassuring results. An analysis of diabetes apps revealed that just five out of 280 (<2%) demonstrated a meaningful clinical benefit.

Decreasing friction and increasing stickiness

Usability is critical if the patient is going to be engaged enough with the app to effect a change in behavior. The process needs to be sticky and self-reinforcing to maintain patient interest. An application may work well in a clinical trial where the patients are engaged in the study, but it may not work as well in a real-world setting like a health clinic or in-home use.

Usability is paramount to achieve stickiness and adoption. Patients need to be able to engage easily with the technology. The most frequently cited challenge by senior adults is lack of usability and “fit" with lifestyle. The technology needs to fit into the flow of people's lives. And it needs to be self-reinforcing to keep individuals engaged long enough to gain benefit.

Ensuring inclusivity and accounting for diversity

Using digital health solutions can help extend programs to historically underserved patients and can augment in-person healthcare with digital communication, education, and remote care management. Digital health apps may also provide opportunities to tailor care for people experiencing health disparities and inequities.

Evaluate digital health through data-driven analysis and tailored assessment criteria

Our team provides expertise across the digital health continuum from decisions about market access, pricing, and regulatory guidance to evidence generation and communication. We work to ensure that any digitally-enabled healthcare tool delivers on cost, quality, safety, and equity objectives.

Learn more about RTI Health Advance Digital Health services and connect with us.