One major challenge for health systems is measuring the return on investment (ROI) of implementing artificial intelligence (AI) technologies. While AI has the potential to address workforce shortages, provider burnout, and administrative tasks, it can be difficult to quantify the financial impact of these tools. This challenge has become even more pressing as healthcare organizations face increased financial pressures due to changes in government policies affecting insurance coverage and reimbursement.
According to experts at the recent HLTH 2025 conference in Las Vegas, measuring the ROI of AI in healthcare is complex due to the diverse factors that influence financial outcomes. Micky Tripathi, Chief AI Implementation Officer at the Mayo Clinic, highlighted the difficulty of applying traditional ROI metrics to AI investments in healthcare delivery. While some AI tools, such as revenue cycle management products, may show direct financial benefits by improving metrics like time to collections, the impact of other tools, such as AI documentation assistants, is less clear.
Despite the challenges in measuring financial returns, there are other ways to assess the value of AI in healthcare. For example, AI scribes that assist with documentation can help reduce clinician burnout by streamlining administrative tasks. This can have significant cost-saving implications, as the expense of replacing a physician is much higher than retaining them within the health system. Additionally, AI tools like ambient scribes can improve efficiency by enabling providers to complete notes more quickly, see more patients, and bill for services more accurately.
Furthermore, patient satisfaction is an important metric that can be positively impacted by AI technologies in healthcare. Patients appreciate the presence of their providers during appointments when AI tools handle documentation, leading to a more enjoyable and transparent care experience. By vocalizing their thought process and care plans during exams, providers using AI scribes can enhance communication with patients and ensure better understanding of treatment plans.
Overall, health systems are exploring creative ways to measure the impact of AI investments beyond traditional ROI metrics. By focusing on outcomes such as provider retention, efficiency gains, and patient satisfaction, healthcare organizations can demonstrate the value of AI in addressing some of the industry’s biggest challenges.
As the healthcare sector continues to prioritize AI investments, the industry is hopeful that these technologies will lead to tangible improvements in quality of care, operational efficiency, and financial performance. By carefully evaluating the impact of AI tools on various aspects of healthcare delivery, organizations can maximize the benefits of these technologies and drive positive outcomes for patients and providers alike.
Implementing an AI tool in a healthcare setting requires a well-thought-out plan that includes clear goals and metrics for evaluating its success. Without a solid evaluation strategy, it can be challenging to determine whether the project was a win, whether financially or by other important measures.
According to Mouneer Odeh, chief data and AI officer at Cedars-Sinai, the first step in evaluating an AI tool is to define what success looks like for the project. This involves clearly outlining the objectives of the tool and why it is being implemented in the first place.
At Cleveland Clinic, AI tools are evaluated using two types of metrics, as explained by Sonja O’Malley, the general manager of business development and licensing at Cleveland Clinic Innovations. Quantitative measures, such as reducing no-show rates or cutting documentation time compared to a baseline metric, are important indicators of success. Additionally, qualitative criteria, like feedback on patient or clinician experience, play a crucial role in assessing the impact of the tool.
While financial return on investment is a key consideration for evaluating AI tools, Anika Gardenhire, chief digital and transformation officer at Ardent Health, emphasizes the importance of looking beyond direct causation. Sometimes, the benefits of an AI tool may not be immediately apparent in financial terms but can have a significant impact on patient care or clinician efficiency.
When determining the success of an AI tool, it is essential to consider all associated costs, including implementation expenses and the time clinicians spend on the project. If a tool only breaks even financially and does not provide significant benefits to patients or clinicians, it may not be worth continuing with the project.
Ultimately, evaluating an AI tool in healthcare requires a comprehensive approach that takes into account both quantitative and qualitative measures of success. By clearly defining goals, setting measurable metrics, and considering all associated costs, healthcare organizations can determine the true impact of AI tools and make informed decisions about their implementation.
