That’s the conversation we keep having with teams building AI meeting assistants, transcription tools, and voice agents. Someone opens a ticket: “The summary is wrong,” or “the model invented a sentence nobody said.” The team spends a day tuning the prompt. The real bug was three layers down, in the audio path, before any text ever reached the model.
Speech-to-text is a pipeline, not a single step
It’s easy to think of STT as one black box: audio in, transcript out. In production, it’s a chain of several distinct stages, and each one can quietly corrupt the data passed to the next.
A rough version of that chain looks like this: raw audio is captured on the client, a voice activity detector (VAD) decides which parts of the stream contain speech, that speech travels over the network to your infrastructure, an STT engine turns it into text, and that text is handed to an LLM for summarisation or written into a database.
Every stage after audio capture assumes that the stage before it did its job correctly. None of them can tell you when that assumption breaks.
Where it actually breaks: VAD
VAD exists to save bandwidth and compute. Instead of streaming continuous silence, the client only sends audio when someone is likely speaking.
The problem is that “likely speaking” is a threshold, and thresholds are wrong at the edges. A VAD tuned too aggressively clips the first syllable of a sentence, because it takes a few milliseconds to recognise that speech has started. It cuts off trailing words during a pause. It misreads background noise, a keyboard, an air conditioner, a second voice bleeding through a laptop mic, as silence or as speech.
None of this produces an error. It produces a shorter, subtly wrong audio stream that gets passed downstream as if it were complete.
Where it also breaks: the network
Assume VAD did its job perfectly and captured exactly the right audio. It still has to get from the client to your STT engine, usually over UDP, usually across a network you don’t control.
Packet loss and jitter are normal there, not exceptional. A few percent packet loss is common on real-world Wi-Fi and mobile connections. Without jitter buffering and packet loss concealment on the receiving end, those gaps arrive as literal silence spliced into the audio, or as words rearranged out of order.
An STT engine, given that input, doesn’t fail loudly. It does what it’s designed to do: produce its best guess at text for whatever audio it received. The gap becomes a dropped word. The reordering becomes a nonsensical clause. The output is fluent, confident, and wrong, which is exactly what makes it hard to catch downstream.
Garbage text doesn’t stay contained
This is the part that turns a network issue into a product issue. The transcript from a degraded audio stream doesn’t get flagged as low-confidence and routed for review. It gets treated as ground truth.
An LLM summarising a meeting transcript has no way to know that “we’re launching Tuesday” was actually “we’re not launching Tuesday” with a dropped word. It summarises what it was given. A structured extraction step, writing action items to a database, has no way to know that a name was garbled by packet loss. It writes what it was given.
By the time someone notices, the error looks like an LLM hallucination or a prompt engineering problem. The actual defect happened before any tokens were generated, in a part of the stack most application teams never look at.
Why is this an infrastructure problem, not a prompt problem
You can’t prompt your way out of missing audio. If the input to your STT engine is wrong, no amount of instruction tuning on the summarisation step fixes it, because the information genuinely isn’t there anymore.
Fixing it means fixing the pipeline itself: VAD tuned with enough padding to not clip real speech, jitter buffering and packet loss handling on the network path before audio reaches the STT engine, and ideally some signal, confidence scores, packet loss stats, that flows downstream so your application layer knows when a transcript came from a clean stream versus a degraded one.
This is exactly the layer CoveKit sits at. Teams building voice AI products don’t want to become experts in jitter buffers and TURN relay behaviour. CoveKit handles the real-time transport and network reliability underneath your STT and LLM stack, so the audio your transcription engine sees is the audio your users actually spoke.
If your summaries or transcripts have been quietly wrong and you’ve been debugging the model, it’s worth checking the pipeline first.