Call center speech analytics is software that transcribes your recorded calls, applies categories to the text, and reports how often each category appears. It answers questions your queue reports cannot, such as what callers ask that nobody has an answer for. What it will not do is work well on audio it cannot hear, or on categories you have not written.
Two things decide whether it earns its cost, and neither one is the vendor. The first is your audio, because transcription accuracy on telephone conversation is nothing like the figures quoted from benchmark tests. The second is your category set, because a category that fires on the wrong calls produces a list nobody reads twice. Below are the accuracy numbers by audio condition, a starter category set with the triggers and the decision each one changes, and the arithmetic that tells you whether a category is readable at your call volume.
What is speech analytics?
Speech analytics is the practice of converting recorded or live calls to text, searching that text for defined categories, and reporting the results against your call metadata. Call recording stores the audio. Speech analytics makes the words countable, so you can ask what share of calls contained something and get a number instead of an anecdote.
It is a different instrument from quality assurance. QA scores a call against a standard and needs a reviewer's judgment, which is why it works on a sample. Speech analytics counts occurrences of a pattern across the whole population and needs no judgment at all, which is why it scales and also why it cannot tell you whether a call was handled well. The scoring half of that work, including the weights and the sampling arithmetic, is in the guide to call center quality assurance.
Vendors split the category into two products. Post-call analytics runs after the call ends and reports on history. Real-time analytics runs during the call and puts prompts in front of the agent while they talk. They are priced separately, so check which one a quote covers before you compare it with anything else.
How does speech analytics work?
There are four steps, and each one can fail on its own. Audio goes to a speech recognition model, which returns a transcript with timings and usually a speaker label. The transcript goes through a category layer, which is either phrase rules, a trained classifier, or a language model prompt. Matches get attached to the call record. Reporting counts them by agent, reason, day or queue.
The failure modes stack in a particular order, and people tend to debug the last step first. If the transcript is wrong, every category above it is wrong, and no amount of reporting work fixes that. If the transcript is fine and the category is badly specified, you get a list of calls that is technically correct and operationally useless. Only once both of those are sound does the reporting layer matter.
Why speech analytics accuracy depends on your audio, not the vendor
Published accuracy figures come from benchmark test sets, and the gap between the easy sets and telephone conversation is wide enough to change what you can build on top of it.
The clearest comparison is in the Whisper paper, which measured models across many test sets (Radford et al., Robust Speech Recognition via Large-Scale Weak Supervision, 2022, Table 2). Word error rate is the share of words the system got wrong, so lower is better.
| Test set | What the audio is | wav2vec 2.0 Large, no LM | Whisper Large V2 |
|---|---|---|---|
| LibriSpeech Clean | Read audiobook speech | 2.7% | 2.7% |
| Switchboard | Telephone calls, newly acquainted pairs on an assigned topic | 28.3% | 13.8% |
| CallHome | Telephone calls, friends and family, open ended | 34.8% | 17.6% |
Read the first row against the last. The wav2vec model was trained on clean read speech and scores 2.7% there, and on natural telephone conversation better than one word in three came back wrong. Whisper, trained on much more varied audio, roughly halves the telephone error and is still off by about one word in six. Your calls sit closer to the CallHome row than the LibriSpeech row, because your callers talk the way people actually talk.
People find phone audio hard too. When Microsoft measured professional human transcribers on the same NIST 2000 test set, they made errors on 5.9% of words in the Switchboard portion and 11.3% in the CallHome portion, where friends and family members have open-ended conversations (Xiong et al., Achieving Human Parity in Conversational Speech Recognition, 2016). Natural conversation is roughly twice as hard to transcribe as a structured one, for people and for software alike.
Then there is what your own phone platform does to the audio before anyone transcribes it. Google's speech recognition documentation is specific about this: capture at 16,000 Hz or higher, because lower sampling rates may reduce accuracy, but do not re-sample, since telephony audio is natively around 8,000 Hz and should be sent at its own rate. The same page warns that recording or transmitting through mp3, mp4, m4a, mu-law, a-law or other lossy codecs may reduce accuracy (Best practices, Cloud Speech-to-Text documentation, read 2026-10-19). Google ships a separate telephony model for audio from phone calls, typically recorded at 8 kHz, for that reason (Transcription models, read 2026-10-19).
An accuracy figure on its own therefore tells you very little. Ask which audio it was measured on, at what sample rate, through which codec, and whether both voices were mixed into a single channel. A number measured on clean audio describes the demo you were shown rather than the calls you run.
Write your category set before you buy anything
The category set is the part you own, the part no vendor can write for you, and the part that decides whether the reports get read. Each category needs a trigger, a decision it changes, and an honest note about where it breaks. A category that changes no decision is a number on a dashboard nobody opens.
Copy this set, cut what does not apply, and add the two or three things your center argues about most.
| Category | What triggers it | The decision it changes | Where it breaks |
|---|---|---|---|
| Repeat contact on the same reason | A second call from the same number inside a set window with the same disposition | Which first-call procedure to rewrite | Shared numbers: switchboards, households, one office line |
| No answer available | Agent phrases such as "let me find out" or "I'm not sure" | What gets added to the knowledge base, in priority order | Polite filler that means nothing, so pair it with call duration |
| Promise of a callback or credit | "Someone will call you", "I'll get that waived" | Whether the promised follow-up actually happened | Needs a join to your follow-up record, or it only counts promises |
| Self-service already failed | "I tried online", "the website would not let me" | Which web or app step to fix, and which calls were avoidable | Callers blame the website for things it does not do |
| Data captured by spelling or repetition | The same name, address or number read back more than twice | Whether your capture step or your lookup is the problem | Transcription errors inflate this one, so sample before trusting it |
| Required disclosure absent | Your own required phrase not present in the call | Whether this is coaching or a script nobody can say naturally | Wording drift: agents paraphrase, so match on meaning or allow variants |
| Competitor or price named | Your competitor list, plus price and quote language | What sales and marketing hear about objections | Generic mentions with no intent behind them |
| Long mid-call silence | A gap over a set length while the call is connected | Which lookup or tool is slow enough to cost handle time | Needs audio timings, not transcript text alone |
Two things are worth saying about using it. The required-disclosure row is the one place simple phrase matching beats everything else, because you are looking for an exact sentence you wrote yourself. And resist adding sentiment as a category on its own. Sentiment tells you a call went badly without telling you why, so it works as a filter on top of the other categories and poorly as a category by itself.
The arithmetic that decides whether a category is readable
A category produces a list of calls. Whether that list is worth opening depends on how rare the thing is, not on how good the detector is. This is where a speech analytics program usually stalls, often with somebody concluding that the software does not work.
Here is the arithmetic, with illustrative figures. Say you run 20,000 calls a month and you want cancellation intent. Suppose it genuinely appears on 2% of calls, which is 400 calls. Your phrase set catches 80% of the real ones, so 320. It also fires on 3% of the other 19,600 calls, which is 588 more.
Flagged calls = 320 + 588 = 908
Actually relevant = 320
Precision = 320 / 908 = 35%
Two of every three calls on that list are not cancellations. A list like that gets abandoned inside a month, and the conclusion people draw is that the tool is wrong, when the detector did roughly what was asked of it.
Now change one thing. Run the same phrase set only over calls already dispositioned as billing, say 3,000 of them, where cancellation intent really runs at 10%, so 300 calls. Catch 80% and you get 240. The same 3% false positive rate on the other 2,700 gives 81.
Flagged calls = 240 + 81 = 321
Actually relevant = 240
Precision = 240 / 321 = 75%
That is the same detector in the same month, producing a list someone will actually work through. The only thing that changed was the denominator. Written as a formula, where p is how common the thing is, r is the share of real cases caught, and f is the share of everything else that trips the trigger:
precision = (p x r) / (p x r + (1 - p) x f)
The practical rule falls straight out of it. For anything rarer than a few percent of your volume, narrow the population before you tune the trigger. Filter by call reason, by queue, by time of day, or by a metric threshold first, then apply the category inside that slice. A category running against every call is only readable when the thing you are looking for is common.
Which questions need transcripts and which your reports already answer
Before you buy, sort your questions into the ones transcripts answer and the ones your existing platform already answers. The second list is usually longer than people expect, and paying a second vendor for it is how a speech analytics budget gets spent without anything changing.
| The question | Where the answer already is | Needs transcript analysis? |
|---|---|---|
| How many callers abandoned, and when | Your queue and ACD reports | No |
| Which call reasons take the longest | Dispositions joined to handle time | No, if your dispositions are reliable |
| Are we staffed for the peak intervals | Volume forecast and an Erlang calculation | No |
| Which agents need coaching on accuracy | A scored sample and a calibrated reviewer | No, scoring takes judgment |
| Why calls in one reason run long | Transcripts of the long ones in that reason | Yes |
| What share of callers mention a given thing | Transcripts, with the arithmetic above | Yes |
| What callers ask that we have no answer for | Transcripts, filtered to the no-answer phrases | Yes |
| Whether the promised follow-up happened | Transcripts joined to your follow-up record | Yes, and the join is the hard part |
| Whether the required disclosure was said | Transcripts, matched against your own phrase | Yes |
If your dispositions are unreliable, the second and fourth rows move into the transcript column, and a good first use of analytics is checking disposition accuracy against what was actually said. That exercise is worth doing on its own, because every report built on dispositions inherits their errors. The definitions those reports rest on are in the guide to call center metrics and KPIs.
Do you need a separate speech analytics product?
Three questions settle it, in this order.
Does your phone or agent platform already transcribe every call and attach the transcript to the call record? Many now do. If yours does, you already have the expensive part.
Can you get transcripts and call metadata out in bulk, through an API or an export, with the reason, duration, outcome and caller number attached? If you can, a spreadsheet or a few SQL queries will answer most of the questions in the table above, and you will find out which categories matter before paying for a category engine. If you cannot, that export gap is the thing to fix first, with your current vendor.
Do you need prompts in front of the agent while the call is live? That is a separate product at a separate price, and it is worth buying once you know which prompts to show. Post-call analysis is what tells you.
On Telvana, every call is recorded and transcribed, with summaries, sentiment, outcome tags and analytics by agent and day. Tagging is automatic across four outcomes: transferred, booked, voicemail and needs follow-up. A workspace per team or client program keeps that reporting separate, which matters for an outsourcer running several client programs under one account. What none of it does is choose your categories or decide what a finding means. That part stays with your team.
What to ask a speech analytics vendor
- What word error rate do you measure on audio like ours, at our sample rate and through our codec, and how was that measured?
- Can we export transcripts and call metadata in bulk, through an API, including the matches and the reason each call matched?
- Who writes and edits categories, our team or yours, and what does a change cost in money and in days?
- Are categories phrase rules, a trained classifier, or a language model prompt, and can we see the evidence behind a given match?
- Is this post-call, real time, or both, and how is each one priced?
- What is redacted from transcripts and audio, at what point, and can we control it?
- How long are audio and transcripts kept, who can see them, and what happens to them when we leave?
Press hardest on the second one. Export is what decides whether you can change your mind later.
Frequently asked questions
What is speech analytics in a call center?
It is software that transcribes recorded or live calls, applies defined categories to the text, and reports how often each appears alongside your call data. It makes what was said countable across every call, rather than across the handful a reviewer listens to.
How does speech analytics work?
Audio is transcribed by a speech recognition model, the transcript is matched against categories built from phrase rules, a classifier or a language model prompt, matches are attached to the call record, and the results are reported by agent, reason or interval. Transcription quality limits everything above it.
What is the difference between speech analytics and voice analytics?
The terms are used interchangeably in practice, and products sold under either name often do the same work. Where a distinction is drawn, speech analytics means analysis of the words and voice analytics takes in acoustic properties such as tone, pace and silence. Ask what a given product measures rather than relying on the label.
Is speech analytics AI?
The transcription step uses machine learning models, so that part is. The category layer may be nothing more than a list of phrases, or it may be a trained classifier or a language model. Ask which, because a phrase list is predictable and auditable while a model is neither, and the two fail in different ways.
What are examples of speech analytics?
Counting how often callers say they already tried the website, finding calls where an agent promised a callback, checking whether a required disclosure was said, spotting repeat contacts on the same reason, and measuring how often callers have to spell a name twice. Each one points at a specific fix.
Do you still need speech analytics if every call is already transcribed?
Not necessarily. If you can export transcripts with the call record, start by querying them directly for the few questions you actually have. Buy a category engine when the list of questions is long enough, and stable enough, that maintaining the queries yourself has become the bottleneck.
Where to start
Pick one question your center argues about and cannot settle from a report, then pull a week of transcripts for the narrowest slice of calls where the answer would live. Read 30 of them yourself. You come out of it with a trigger worth automating, a realistic sense of how clean your transcripts are, and an estimate of how common the thing is, which is the number the arithmetic above needs. That is a day of work, and it will tell you more than a demo will.
Telvana builds AI agents that answer calls, texts and web chats for contact centers, and transfers to your team under rules you set. Every call comes back recorded, transcribed, summarized and tagged by outcome. Pricing starts from $399 a month based on call volume, plus a one-time setup fee. Book a demo with your top call types and you will hear an agent handle one. The rest of this cluster is in the metrics and operations guides, including how average handle time behaves once you start reading it as a signal.