By Spencer Young, CEO at MedReview  

When the American Medical Association raises concerns about the use of AI in healthcare, we listen.

Among the topics explored at the 2026 AMA Annual Meeting was the increasing use of AI in medicine, from generating clinical notes to informing care and billing decisions.

One example the organization cites is health plans’ increasing use of AI tools to deny, reduce, or terminate coverage or payments for medical care. Yet these tools often rely on datasets that don’t reflect the most current updates in the science or clinical guidelines across specialties, increasing the risk of errors. 

In response, the AMA underscored their commitment to a human-in-the-loop (HITL) policy which ensures that a physician makes the final call on clinical reviews.

What does this mean for health plans looking to efficiently, yet accurately, validate their payment decisions? And what impact could this have on us all as patients? Let’s take a look.

The role and risks of AI in payment integrity

Every year, health plans are at risk for millions of dollars in mispaid claims. The ability to identify – and rectify – inconsistencies between the medical care provided and how it’s documented and billed is at the core of a strong payment integrity strategy.  This in turn helps health plans promote accurate billing, meet regulatory guidelines, and build trust with its healthcare provider network.

Using AI as part of the payment integrity process makes sense, thanks to the technology’s ability to process millions of claims in seconds, expose hidden fraud patterns, and eliminate labor-intensive administrative workflows.

Yet these algorithms can miss context, misinterpret clinical nuances, or flag legitimate claims as erroneous. Without human oversight, this can lead to friction and frustration between providers and health plans, delayed reimbursements, and patient dissatisfaction.

It can also have a negative impact on patient care. For example, in a recently released AMA survey, 61% of physicians said they fear that payers’ use of unregulated AI is increasing prior authorization denials. This often leads to patients abandoning or delaying necessary medical care. 

A HITL approach balances AI efficiency with expert clinical and operational reviews, helping mitigate risks while ensuring fairness, accuracy, and transparency.

How HITL helps improve AI models

HITL integrates human expertise directly into AI workflows. In payment integrity, this means that rather than fully automating decisions, approaches that utilize HITL ensure that clinicians, auditors, and coders remain active participants in key analyses by validating data, reviewing anomalies, and making context-driven clinical judgments.

This framework creates a continuous feedback loop. AI flags potential fraud or errors, humans provide clinical and regulatory insight that support or invalidate these findings, and their corrections improve future AI performance. The result is a system that not only works faster but also gets smarter and more reliable over time.

A human-in-the-loop approach strengthens the reliability of payment decisions by:

• Ensuring accuracy and data quality – Human experts verify anomalies, resolve ambiguities, and prevent systemic errors that AI might reinforce if left unchecked. 
• Empowering insights into complex, unstructured data – Clinical notes, imaging reports, and nuanced coding often require human interpretation. HITL ensures that context and medical judgment are not lost in translation when evaluating claims.
• Building trust and transparency – HITL mitigates the “black box” problem by introducing explainable, defensible human review. Provider abrasion is reduced when humans, not algorithms, handle disputes and clearly communicate findings.
• Enabling adaptation to fraud and regulatory change – Fraud schemes evolve constantly, as do compliance requirements. Human reviewers can train AI to recognize new fraud patterns or apply updated regulations far faster than automation alone.
• Upholding ethics and accountability – HITL ensures that critical decisions about payments and care remain accountable to qualified professionals, aligning with requirements such as the Department of Health and Human Services’ HTI-1 rule mandating algorithmic transparency.

AI’s impact on patient care and trust 

For all its advancements, AI remains more like a smart toddler, not a wise expert. Models may guess because they are rewarded for providing an answer and may reflect structural inequities and bias found in the data. AI-driven financial decisions directly impact patient care, risking delays, stress, denied care, and lawsuits. That can – and quite frankly, should – lead to distrust for healthcare organizations and the people they serve.

In fact, a recent JAMA report shows that two-thirds of adults in the United States have little trust that healthcare systems will use AI responsibly. And consumers already rate health insurers among the least trusted sectors of the healthcare system.

Collectively, we have a responsibility to use AI in ways that balance speed with accuracy, and reject blind convenience for the sake of trust. Approaches where human expertise is augmented by AI – from clinical care to decision transparency – are essential to achieving this goal. There’s just too much at stake.

About the author

As Chief Executive Officer of MedReview, Spencer is ultimately responsible for the organization’s day-to-day operations, as well as its long-term strategic growth plans. Spencer has more than 35 years of experience and expertise in risk management, operations, process improvement and growth strategies.

Prior to joining MedReview, Spencer worked in operations and strategic business development at several companies. Spencer was President of Health Data Insights and Senior Vice President of Clinical Operations at HMS Holdings Corp. where he was responsible for overseeing the strategies and operations of the clinical teams across the enterprise. Previously, Spencer served as Vice President of Payment Integrity Services at UnitedHealthcare. To learn more about MedReview’s physician-approved payment integrity solutions, visit www.medreview.us