Governance at Scale: How Health Plans Should Risk-Tier Their LLMs

In utilization management (UM), few metrics speak louder—or cut deeper—than overturn rates. When a significant share of denied claims are later approved on appeal, it’s rarely just about an individual decision. It’s a reflection of something bigger: inconsistent policy interpretation, reviewer variability, documentation breakdowns, or outdated clinical criteria.

Regulators have taken notice. CMS and NCQA increasingly treat appeal outcomes as a diagnostic lens into whether a payer’s UM program is both fair and clinically grounded.¹ High overturn rates now raise questions not just about accuracy, but about governance.

In Medicare Advantage alone, more than 80 % of appealed denials were overturned in 2023 — a statistic that underscores how often first-pass decisions fail to hold up under scrutiny.² The smartest health plans have started to listen. They’re treating appeals not as administrative noise—but as signals.

What Overturned Denials Are Really Saying

Every overturned denial tells a story. It asks, implicitly: Was the original UM decision appropriate, consistent, and well-supported?

Patterns in appeal outcomes can expose weaknesses that internal audits often miss. For example:

• Repeated overturns for a single service category often signal misaligned or outdated policies.

• Overturns concentrated among certain reviewers may point to training or workflow inconsistencies.

• Successful appeals after peer-to-peer discussions often reveal documentation or communication gaps between provider and plan.

These trends mirror national data showing that many initial denials are overturned once additional clinical details are provided, highlighting communication—not medical necessity—as the core failure.³ The takeaway is simple but powerful: Appeal data is feedback—from providers, from regulators, and from your own operations—about how well your UM program is working in the real world.

The Systemic Signals Behind High Overturn Rates

When you look beyond the surface, overturned denials trace back to five systemic fault lines common across payer organizations:

1. Policy Rigor vs. Flexibility
   Medical necessity criteria must balance evidence-based precision with real-world adaptability. Policies written without clinical nuance—or not updated frequently enough—tend to generate denials that can’t stand up under appeal.

2. Reviewer Variability
   Even with clear policies, human interpretation introduces inconsistency. Differences in specialty expertise, decision fatigue, or tool usage can lead to unpredictable outcomes.

3. Provider Documentation Gaps
   Many initial denials are simply the result of incomplete records. When appeals are approved after additional information surfaces, the problem isn’t inappropriate care—it’s communication failure.

4. Operational Friction
   Lag times between intake, review, and notification can distort first-pass decisions. Data fragmentation between UM, claims, and provider portals compounds the issue.

5. Weak Feedback Governance
   Too often, appeal outcomes are logged but not analyzed. Mature UM programs close the loop—using overturned denials to retrain reviewers, refine policies, and target provider outreach.

Federal oversight agencies have long flagged this issue: an OIG review found that Medicare Advantage plans overturned roughly three-quarters of their own prior authorization denials, suggesting systemic review flaws and weak first-pass decision integrity.⁴

Turning Appeals into a Feedback Engine

Leading payers are reframing appeals from a reactive function to a proactive improvement system.
They’re building analytics that transform overturn data into actionable intelligence:

• Policy Calibration: Tracking which criteria most often lead to successful appeals reveals where policies may be too restrictive or outdated.

• Reviewer Performance: Overlaying overturn trends with reviewer data helps identify where training or peer review support is needed.

• Provider Partnership: By sharing de-identified appeal insights, plans can help provider groups strengthen documentation and pre-service submissions.

• Regulatory Readiness: Demonstrating a closed-loop feedback process strengthens NCQA compliance and positions the plan as an adaptive, learning organization.

This approach turns what was once a compliance burden into a continuous-learning advantage.

From Reversal to Reform

High overturn rates are not just a symptom—they’re an opportunity. Each reversed denial offers a data point that, aggregated and analyzed, can make UM programs more consistent, more transparent, and more clinically aligned.

The goal isn’t to eliminate appeals. It’s to make sure every appeal teaches the organization something useful—about process integrity, provider behavior, and the evolution of clinical practice.

When health plans start to see appeals as mirrors rather than metrics, UM stops being a gatekeeping exercise and becomes a governance discipline.

The Bottom Line

Overturned denials aren’t administrative noise—they’re operational intelligence. They show where your policies, people, and processes are misaligned, and where trust between payer and provider is breaking down.

For forward-thinking plans, this is the moment to reimagine UM as a learning system.
At Mizzeto, we help health plans turn appeal data into strategic insight—linking overturned-denial analytics to reviewer training, policy governance, and compliance reporting. Because in utilization management, every reversal has a lesson—and the best programs are the ones that listen.

SOURCES ‍

1. National Committee for Quality Assurance (NCQA). Overview of Proposed Updates to Utilization Management Accreditation 2026
2. Kaiser Family Foundation (KFF). “Nearly 50 Million Prior Authorization Requests Were Sent to Medicare Advantage Insurers in 2023"
3. American Medical Association (AMA). “Prior Authorization Denials Up Big in Medicare Advantage"
4. U.S. Department of Health & Human Services, Office of Inspector General (OIG). Some Medicare Advantage Organization Denials of Prior Authorization Requests Raise Concerns About Beneficiary Access to Medically Necessary Care

‍

Not all intelligence is created equal. As health plans race to integrate large language models (LLMs) into clinical documentation, prior authorization, and member servicing, a deceptively simple question looms: Which model actually works best for healthcare?

The answer isn’t about which LLM is newest or largest — it’s about which one is most aligned to the realities of regulated, data-sensitive environments. For payers and providers, the right model must do more than generate text. It must reason within rules, protect privacy, and perform reliably under the weight of medical nuance

Understanding the Core Question

For payers and providers alike, the decision isn’t simply “which LLM performs best,” but “which model can operate safely within healthcare’s regulatory, ethical, and operational constraints.”

Healthcare data is complex — part clinical, part administrative, and deeply contextual. General-purpose LLMs like GPT-4, Claude 3, and Gemini Ultra excel in reasoning and summarization, but their performance on domain-specific medical content still requires rigorous evaluation.¹ Meanwhile, emerging healthcare-trained models such as Med-PaLM 2, LLaMA-Med, and BioGPT promise higher clinical accuracy — yet raise questions about transparency, dataset provenance, and deployment control.

Analyzing the Factors That Matter

Evaluating an LLM for healthcare use comes down to five dimensions:

1. Data Security and Privacy: Models must support on-premise or private cloud deployment, with PHI never leaving the payer’s-controlled environment.

2. Domain Adaptation: Can the model be fine-tuned or context-trained on medical ontologies, payer workflows, or prior authorization rules?

3. Explainability: Does it provide confidence scores, citations, or audit logs for generated content — essential for regulatory defense and trust?

4. Integration Readiness: Can it interact with existing data ecosystems like QNXT, HealthEdge, or EPIC via APIs or orchestration layers?

5. Cost and Scalability: Beyond performance, can it operate efficiently at enterprise scale without prohibitive inference costs?

The Case for General-Purpose Models

Models like OpenAI’s GPT-4 and Anthropic’s Claude 3 dominate enterprise use because of their versatility, mature APIs, and strong compliance track records. GPT-4, for instance, underpins several FDA-compliant tools for clinical documentation and prior authorization automation.²

Advantages include:

• Maturity and security: Vendors offer HIPAA-aligned enterprise environments, audit trails, and SOC-2 compliance.

• Cross-domain adaptability: They integrate easily across payer workflows — intake, summarization, or correspondence.

• Rapid iteration: Frequent updates and strong partner ecosystems reduce implementation lag.

But there are caveats. General models sometimes “hallucinate” clinical or regulatory facts, especially when interpreting EHR data. Without domain fine-tuning or strong prompt governance, output quality can drift.

The Case for Healthcare-Specific LLMs

A growing ecosystem of medical-domain LLMs is changing the landscape. Google’s Med-PaLM 2 demonstrated near-clinician accuracy on the MedQA benchmark, outperforming GPT-4 in structured reasoning about medical questions. Open-source options like BioGPT (Microsoft) and ClinicalCamel are being tested for biomedical text mining and claims coding support.

Advantages include:

• Higher clinical grounding: Trained on PubMed, clinical guidelines, and biomedical literature.

• Explainability: Some models provide citation-based reasoning or evidence chains.

• On-premise deployability: Open-source variants allow PHI-safe environments.

Yet, the trade-offs are real:

• Limited generalization: These models can underperform on administrative or financial text.

• Resource demands: Fine-tuning and maintenance require specialized infrastructure and talent.

• Regulatory uncertainty: Validation for real-world payer use remains early-stage.

Synthesizing the Middle Ground

The emerging consensus is hybridization. Many payers and health systems are adopting dual-model architectures:

• A general-purpose model (e.g., GPT or Claude) for summarization, knowledge extraction, and conversational interfaces.³

• A domain-specific, internally governed model (often LLaMA or Mistral–based) for compliance-sensitive tasks involving PHI, clinical logic, or audit documentation.

This “governed ensemble” strategy balances innovation and oversight — leveraging the cognitive power of frontier models while preserving control where it matters most.

The key isn’t picking a single best model. It’s building the right model governance stack — version control, prompt audit trails, human-in-the-loop review, and strict access controls. Healthcare’s best LLM is not the one that knows the most, but the one that knows its limits.

The Bottom Line

Choosing an LLM for healthcare isn’t a procurement exercise — it’s a governance decision. Plans should evaluate models the way they would evaluate clinical interventions: by evidence, reliability, and risk tolerance.

The best LLMs for healthcare are those that combine precision, provenance, and privacy — not those that simply perform best in general benchmarks. Success lies in orchestrating intelligence responsibly, not in adopting it blindly.

At Mizzeto, we help payers design AI ecosystems that strike this balance. Our frameworks support multi-model orchestration, secure deployment, and audit-ready oversight — enabling health plans to innovate confidently without compromising compliance or control. Because in healthcare, intelligence isn’t just about what a model can say — it’s about what a plan can trust.

‍

‍

SOURCES

1. Assessing the use of the novel tool Claude 3 in comparison to ChatGPT 4.0
2. Use of GPT-4 to analyze medical records of patients with extensive investigations and delayed diagnosis
3. Benefits, Limits, and Risks of GPT-4 as an AI Chatbot for Medicine

Every payer today faces the same dilemma: automate or fall behind. But as health plans modernize claims, prior authorization, and member servicing workflows, a harder question emerges — should automation be built in-house, or outsourced to specialized partners?

It’s not a new question, but it’s never been more consequential. The industry’s next wave of competitiveness will hinge not on whether payers automate, but how they do it — and whether their automation strategy aligns with scale, compliance, and differentiation goals.

The Core Question

At its heart, the decision to build or buy automation is a test of strategic identity. Is automation a core capability, something that defines how a plan competes and operates — or is it a commodity, a function that can be standardized and sourced efficiently from outside partners?

For some payers, automation is mission-critical — a differentiator in member experience and operational agility. For others, it’s infrastructure: vital, but not unique. That distinction shapes everything that follows.

The Case for Building In-House

Building automation internally appeals to payers seeking control, customization, and intellectual ownership. It allows them to define workflows in ways that reflect their unique mix of products, regions, and compliance requirements.

Advantages include:

• Alignment with proprietary processes: In-house development ensures automation mirrors the plan’s rules, data models, and legacy integrations.

• Data governance and security: Sensitive PHI and analytics stay within the enterprise perimeter.

• Strategic flexibility: Internal teams can iterate faster and adapt automation to emerging needs without vendor dependency.

• Institutional learning: Each build deepens internal knowledge of systems, workflows, and decision logic — a long-term competitive asset.

But building comes at a cost. It demands high technical maturity, deep domain expertise, and cross-department coordination.¹ Development cycles can stretch months or years, and maintaining the systems consumes scarce IT resources. For many plans, the real bottleneck isn’t willingness — it’s capacity.

The Case for Partnering

Outsourcing automation to experienced partners offers a different calculus — one built on speed, scalability, and proven expertise.

Key advantages:

• Faster time-to-value: Pre-built frameworks and tested integrations allow quicker deployment.

• Regulatory assurance: Partners often stay ahead of evolving CMS, HIPAA, and interoperability mandates.²

• Access to specialized talent: Few payers can sustain teams with expertise in both healthcare operations and advanced automation technologies.

• Cost predictability: Subscription or managed-service models reduce capital expense and limit the risk of project overruns.

The trade-off is dependency. Vendor-managed solutions can limit flexibility, especially when plans want unique configurations or when data must flow through external systems.³ Integration complexity and long-term lock-in can also undercut initial savings.

The Hybrid Middle Ground

The best strategies often blend both approaches. Leading payers are moving toward hybrid automation models — building internal frameworks for strategic functions (e.g., utilization management, clinical decisioning) while partnering for standardized tasks (e.g., claims intake, document processing, member correspondence).

This model captures the best of both worlds: retaining control where differentiation matters, outsourcing where scale and efficiency dominate. It also creates optionality — the ability to evolve as organizational maturity, regulatory requirements, or vendor ecosystems shift.

In practical terms:

• Build when automation defines your strategic advantage or touches sensitive clinical workflows.

• Buy when automation is repeatable, compliance-driven, or infrastructure-heavy.

• Blend when speed and learning are equally important.

The Decision Framework

For CEOs and CIOs, the build-vs-buy question is not purely technical — it’s strategic. A sound framework includes:

1. Mission alignment: Does the automation initiative advance core differentiation or just maintain parity?

2. Capability audit: Do internal teams have the skill, bandwidth, and governance maturity to sustain it?

3. Regulatory horizon: Will external vendors adapt faster to rule changes or interoperability mandates?

4. Cost vs. value timeline: How does total cost of ownership compare across three, five, and seven years?

5. Data ownership: Who controls the insights, algorithms, and audit trails — and how secure are they?

These questions clarify whether automation should be a center of excellence or a service partnership.

The Bottom Line

Automation is no longer optional. But how payers approach it will separate the efficient from the exceptional. Building offers control; buying offers speed. The smartest plans will use both — designing architectures that evolve with the industry while maintaining ownership of what truly differentiates them.

At Mizzeto, we help payers strike that balance. Our modular automation frameworks integrate with core systems like QNXT, Facets, and HealthEdge, enabling plans to retain strategic control while accelerating execution. Whether building, buying, or blending, we help payers turn automation into a competitive advantage — not just an operational upgrade.

‍

SOURCES

1. Toolkit: Addressing the Administrative Burden of Prior Authorization
2. CMS Interoperability and Prior Authorization Final Rule
3. Building Interoperable Healthcare Systems - One Size Doesn't Fit All

Few issues in healthcare generate as much consensus — and as much frustration — as prior authorization. Providers say it delays care and drives burnout. Patients say it creates barriers and confusion. Payers defend it as a necessary check on cost and safety. For decades, the debate has been stuck in a cycle of promises: that reforms are coming, that automation will help, that balance is possible.

That cycle is beginning to break. Starting in 2025, new CMS rules will tighten prior authorization response times, mandate public reporting of approval data, and require API-based interoperability across Medicare Advantage, Medicaid, CHIP, and ACA exchange plans.¹ At the same time, several large payers — including Humana, Cigna, and UnitedHealthcare — have announced major cuts to prior authorization requirements.

The question is no longer if prior authorization will change. It’s how much value those changes will deliver.

For payer CEOs, the core challenge is shifting from promise to proof: measuring whether reforms translate into measurable returns in cost, efficiency, provider satisfaction, and member outcomes.

Where the Value Lies

Prior authorization touches nearly every stakeholder. That’s why ROI must be assessed on multiple fronts:

• Operational efficiency: Every hour a nurse spends processing prior auth requests is an hour not spent on clinical judgment. Automating intake, routing, and documentation reduces this administrative drag.²

• Provider satisfaction: According to an AMA survey, 94% of physicians reported care delays due to prior authorization,³ and 30% said it had led to a serious adverse event for a patient. Reforms that cut down unnecessary requests or speed up turnaround times directly improve the provider relationship.

• Member experience: Delays erode trust. Streamlined prior auth can improve satisfaction scores, reduce appeals, and strengthen retention.

• Medical cost management: The original purpose of prior authorization was cost containment. Eliminating it wholesale risks overutilization, but smart reforms — especially paired with gold-carding or risk-based contracting — can maintain oversight while cutting waste.

Each of these levers can be measured. The trick is deciding which metrics matter most for executives and regulators alike.

Early Evidence

The industry doesn’t have to speculate. Early experiments in trimming prior authorization already show ROI.

• Humana announced in 2023 it would remove prior authorization for 1,000 services — nearly 20% of its total requirements.⁴ The company reported significant reductions in provider complaints and faster turnaround on the cases that still required review.

• Cigna followed by cutting prior auth on 600 procedures, citing the need to “reduce friction” with providers. Early internal analyses showed reduced processing costs without a spike in utilization.⁵

• UnitedHealthcare said it would eliminate PA for 20% of procedures in 2024. Aetna announced similar streamlining.

At the same time, automation is showing measurable impact. Plans deploying AI-assisted intake have reported reductions of 50–70% in manual review time, according to case studies published by AHIP.⁶

Together, these reforms point to a clear ROI pathway: fewer requests → lower admin burden → happier providers → equal or better utilization control.

Measuring What Matters

To move beyond anecdotes, payers need a measurement framework. CEOs should ask their teams:

• How much administrative time have we saved? (Nurse hours, FTE cost equivalents, processing turnaround).

• How has provider satisfaction shifted? (Net promoter scores, complaint volumes, participation rates).

• What’s the member impact? (Grievances filed, appeal rates, CAHPS scores).

• Are medical costs stable? (Utilization trends in services with PA removed vs. those retained).

• What’s the compliance dividend? (Alignment with CMS’s transparency reporting requirements, reduced audit risk).

By tracking these measures over time, plans can prove whether reforms deliver more than good headlines.

The Strategic Risks

Of course, cutting prior authorization is not risk-free.

• Overutilization creep: Without oversight, services like imaging or specialty drugs may see cost spikes.

• Uneven execution: If PA cuts are applied inconsistently, providers may still face confusion — and complain even louder.

• Regulatory mismatch: CMS requires reporting on all PA activity, even as payers reduce requirements. Plans must ensure they still have the infrastructure to measure what’s left.

The risk is not in reform itself, but in reform without data discipline.

From Compliance to Advantage

The true opportunity lies in harmonizing reforms with technology. CMS’s interoperability rule requires plans to build FHIR APIs and expose prior authorization metrics publicly. Instead of treating that as a reporting burden, payers can use the same infrastructure to create real-time dashboards for providers, track ROI metrics internally, and demonstrate performance externally.

Done right, this flips prior authorization from a compliance headache to a competitive differentiator. A plan that can show regulators, providers, and members that reforms improved experience and held costs steady will win trust in a way that rules alone can’t mandate.

The Bottom Line

The era of promises is ending. Between CMS mandates and payer-led reforms, prior authorization is undergoing its most significant transformation in decades. The real test is not whether requirements are reduced or APIs built — it’s whether these changes deliver measurable ROI in efficiency, satisfaction, and outcomes.

For CEOs, the call to action is clear: build the measurement framework now, so when reforms hit full stride in 2025–2027, you’ll have proof — not just promises — to show regulators, providers, and members alike.

At Mizzeto, we help health plans design and implement these measurement frameworks, from integrating API data feeds to creating dashboards that track ROI across operations. Reform is inevitable. Proof is optional. The plans that can show it will lead.

SOURCES

1. CMS Interoperability & Prior Authorization Final Rule

2. Fixing Prior Auth - American Medical Association

3. AMA Survey Indicates Prior Authorization Wreaks Havoc On Patient Care

4. Humana Accelerates Efforts to Eliminate Prior Authorization Requirements

5. Cigna Healthcare Removes 25 Percent of Medical Services From Prior Authorization

6. Improving Prior Authorization for Patients & Providers - AHIP

‍