Your quality team listens to maybe five calls per agent each month. It scores each one in isolation, hands back a number, and calls that coaching. For a VP of Operations, that’s the quiet problem with call center QA sampling. The sample is too small to be fair, and it isn’t even predictive. SQM Group found that 81% of traditional QA scores didn’t correlate with customers’ own satisfaction ratings.1 So you’re coaching agents on noise.
And the sample has a second hole. It’s almost all in English. The calls your members place in Spanish, Mandarin, or Vietnamese rarely reach a reviewer, because the people scoring calls read and speak English.
So the problem compounds. The sample is tiny, it’s graded call-by-call, and it’s almost all English. The fix isn’t a better sample. It’s no sample: score 100% of calls, in every language, and read agents on their whole body of work.
What a 1% Sample Actually Measures
Start with the math. SQM finds that most call centers evaluate five or more calls per agent a month.2 Against the several hundred calls an agent handles in that time, that’s about 1% of their work, often less. So for most agents, 99% of what they do is never scored.
That alone would be a problem. The bigger one is which calls get picked, and what the score means.
Reviewers tend to grab calls that are easy to score. Short ones. Clean ones. The calls where nothing went wrong. The messy, high-stakes calls, the ones you’d most want to learn from, are the ones that get skipped.
Then each call is graded on its own. Did the agent use the greeting? Hit the compliance script? Close properly? These are real things. But they describe how one call was handled, not whether the agent is serving members well.
That’s why the scores drift from reality. SQM found that only 17% of agents believe their QA program improves customer satisfaction.1 The people being measured can already see the gap.
And coaching inherits the gap. A supervisor pulls up two or three scored calls and builds a development plan on them. If those calls aren’t representative, the plan is guesswork dressed up as data.
The Calls Your QA Never Hears
Now add language. More than 25 million people in the United States are limited English proficient.3 Across Medicare Advantage, Medicaid, and the Marketplace, they’re a large share of who you serve.
Those members call. Their calls get handled. Then they vanish from quality review.
The reason is structural, not deliberate. Most QA reviewers work in English. Scoring a call in Vietnamese or Mandarin means finding a reviewer who speaks it, which most plans can’t do at scale. So the sample quietly narrows to English, and a whole population of interactions goes unscored.
Here’s what makes that dangerous. CMS already grades these calls, even if you don’t.
Each year, from roughly February through June, CMS places test calls to Part C and Part D call centers. It checks whether an interpreter can be reached, whether TTY works, and whether the agent answers plan questions accurately. It does this in English plus six more languages, including Spanish, Mandarin, Vietnamese, and Tagalog.4 Those results feed CMS’s compliance monitoring.
So CMS secret-shops your agents in languages your own QA never reviews. If you want the fuller case for why sampling leaves most of a call center invisible, we made it in Are Health Plans Really Listening to All Their Members? The point for agent performance is narrower. You can’t coach what you never hear, and you’re accountable for it anyway.
The Agents Your Sample Misses
The agents who carry the most non-English calls are, by design, the least sampled. Their calls get skipped, so their scores rest on whatever English calls happened to get pulled. Your hardest-working bilingual agents end up judged on the least representative slice of their work.
QA sampling also misattributes. When a member calls back about the same issue, a handful of scored calls can’t tell you whether the agent mishandled it or a broken process behind the agent did. SQM’s data on calls that aren’t resolved on the first try traces 49% of the errors to the organization and 38% to the agent.5 Coach the agent for an organization problem, and you’ve fixed nothing.
And a monthly sample rarely catches a pattern. If an agent mishandles one specific scenario, and it comes up on a small share of calls, the odds it lands in that 1% are low. The behavior you’d most want to coach is the behavior least likely to be seen.
So the agents carrying your hardest, highest-risk calls are the ones your sample is least likely to cover. That’s not a coaching program. It’s a lottery.
The difference between the two approaches comes down to how much of an agent’s work you actually look at.
How to Score Agents on Every Call
Moving off sampling isn’t about scoring harder. It’s about scoring everything. Four shifts get you there.
First, capture every call. Record 100% of interactions, in every language, and keep the transcript. SQM itself recommends recording all calls, not a slice.1 You can’t analyze what you didn’t keep.
Second, score consistently against defined dimensions. Apply the same rubric to every call, so an agent’s score reflects the rubric, not which reviewer they drew or which calls came up. Define what you’re measuring: member sentiment and risk, communication and empathy, compliance.
Third, include every language by default. Translation has to be built in, not a special request. If non-English calls need a separate workflow, they’ll keep getting skipped. A Tagalog call should get scored the same day, the same way, as an English one.
Fourth, coach on patterns, not anecdotes. Once every call is scored, you stop asking “how was this one call” and start asking “what does this agent do across a hundred calls.” Repeated phrasing that confuses members. A compliance step that slips under pressure. That’s coaching you can act on, and it’s the kind that moves CAHPS scores a thin sample never could.
What to Look for in a Solution
When you evaluate call center QA tools, look for five things.
- Every call scored, not a monthly sample. Coverage of 100% of interactions, so no agent is judged on a lucky few.
- Every language, not just English. Translation and scoring built in for all languages, so non-English calls are reviewed on the same footing as English ones.
- Patterns across calls, not calls in isolation. The ability to group an agent’s calls and surface recurring behavior, not just grade one interaction at a time.
- Objective scoring, not reviewer roulette. A consistent rubric applied the same way to every call and every agent.
- Plan-owned data, not vendor custody. The scores, transcripts, and intelligence sit with the plan, not locked inside an outsourcer’s reporting.
That last one matters most. When your QA is run by the same vendor that staffs your calls, the vendor is grading its own work. Independent, plan-owned scoring is the only version you can trust.
Mizzeto’s Call Center Intelligence Tool was built for this. It scores 100% of calls across every language and evaluates agents on its Agent Empathy & Communication dimension, so coaching rests on an agent’s whole body of work, not a sample. The data stays yours.
What Changes When You Score Everything
Coaching stops being guesswork. Supervisors work from an agent’s full record, not a handful of cherry-picked calls. Your strongest non-English agents finally get credit, and your struggling ones get help you can actually target. And the CMS test calls that used to be a once-a-year gamble become something you can see coming, because you’re already scoring the same calls CMS does.
That’s the real cost of call center QA sampling. Not just unfair scores, but a coaching program built on a sliver of the truth. Scoring every call fixes the input. If you want to see it on your own calls, send us a sample. We’ll return scored transcripts before you commit to anything.
References
- SQM Group. “Call Center Quality Monitoring: Best Practices.” sqmgroup.com. https://www.sqmgroup.com/resources/library/blog/call-center-quality-monitoring-best-practices
- SQM Group. “Call Center Quality Assurance Tips.” sqmgroup.com. https://www.sqmgroup.com/resources/library/blog/call-center-quality-assurance-tips
- U.S. Census Bureau. American Community Survey, 2019-2023 5-Year Estimates, Table S1601, Language Spoken at Home. data.census.gov.
- Centers for Medicare & Medicaid Services. “2022 Part C and Part D Call Center Monitoring: Timeliness and Accuracy & Accessibility Studies.” December 16, 2021. https://www.cms.gov/files/document/2022partcandpartdcallcentermonitoring.pdf
- SQM Group. “Call Center FCR Benchmark 2024 Results by Industry.” sqmgroup.com. https://www.sqmgroup.com/resources/library/blog/call-center-fcr-benchmark-2024-results-by-industry




















