The difference between a transcription system and a real-time voice agent isn't speed. It's what happens in the few hundred milliseconds while the person is still speaking — and what the system can do while it is speaking itself.
Words appear as you say them, not after you finish. Recognition is streaming, with state kept between chunks — it doesn't re-transcribe every time. Partial text trails your voice by ~1.1 s (the 1120 ms chunk); once you stop speaking, the rest comes out in 92 ms.
92 ms to text, after you stopIt listens to the pitch slope, not a silence timer. A firm fall: you've finished. Flat or rising: you have more to say.
560 ms · 17 points fewer wrongly cut pausesYou talk over it and it goes quiet. And it remembers exactly how much you heard, not how much it managed to generate.
speech synthesis stops in 1–3 msIt says a short bridge phrase, sends the job to another thread and keeps talking. It stays interruptible while the tool works.
Each agent with its own voice, language, instructions, tools and knowledge base. Tenants isolated from each other, dozens of agents in the same process.
You close the tab, the network drops, the server restarts — the conversation picks up where it left off. Tools finish their work and the result waits for you.
ChatGPT Live has none of the last three: the app has no public API, session length isn't published, and it doesn't work between your visits. Full feature inventory →
All synthesized on the CPU, with no external service. Speech recognition runs on the Mac's integrated GPU (M3 Max) — no dedicated graphics card.
Switching happens mid-conversation, without reconnecting — recognition, synthesis and instructions change at once, but the voice does not.
The language model writes digits. Nobody says "two three four eight zero lei". Before synthesis, the text passes through a function that spells out every number — with the rule that fits the context: amounts as numbers, times as times, phone numbers digit by digit.
what the model writes
Suma totală este de 23480 de lei, iar programul e de la 09:00 până la 22:00.
„Suma totală este de douăzeci și trei de mii patru sute optzeci de lei, iar programul e de la ora nouă până la ora douăzeci și două."
"The total is 23,480 lei, and opening hours are from 9:00 to 22:00."
A time isn't read as "zero nine zero zero". An amount isn't read digit by digit. The same
function decides which rule applies where.
what the model writes
Vă sun la 069123456 pe 15.09.2026, comanda nr. 4712.
„Vă sun la zero șase nouă unu doi trei patru cinci șase pe cincisprezece septembrie două mii douăzeci și șase, comanda nr. patru mii șapte sute doisprezece."
"I'll call you at 069123456 on 15 September 2026, order no. 4712."
The phone number — digit by digit, the only useful form on the phone. The date — in words.
The order number — as a whole number. Three different rules in the same sentence.
„Îmi pare rău, în această înserare târzie nu găsesc nicio șansă."
"I'm sorry, on this late evening I see no chance."
ă, â, î, ș, ț in the same sentence. In Romanian a diacritic changes the word, not its look:
„țară" (country) and „tara" are two different words.
„Ați dori să vă sun înapoi mâine dimineață?"
"Would you like me to call you back tomorrow morning?"
The same pitch slope the system reads to know whether you've finished speaking, this
time produced.
Speech recognition and synthesis run on your machine. Only text goes out — never audio.
Where a patient's or client's voice isn't allowed to leave the building, this isn't a technical preference. It's the condition for being able to use anything at all.
A voice vendor was acquired, the platform shut down, customers' voice clones were deleted. What runs on your side stays.
The audio components aren't billed per minute. The only variable cost left is the language model.
Almost every voice agent decides you've finished after a fixed number of milliseconds of silence. That's a timer, not a detector — and it cuts people off mid-sentence. Grai looks at the pitch slope over the last half second.
Measured on real human conversations, in English (CANDOR: 80 clips, 98 pauses, offline replay). An "error" means a wrongly cut pause: at 560 ms the rate drops from 69.4% to 52.0%. The cost, 0.42 ms per decision, is measured on an M3 Max. At the same error rate as a fixed 820 ms threshold, the intonation-based decision waits 560 ms — and makes more decisions, so the gain doesn't come from deciding less often.
Word error rate (WER), FLEURS Romanian (test split, 200 utterances). Lower is better. The "how" column is what makes the table readable — a model that sees the whole file, offline, in the cloud, isn't solving the same problem as one that outputs text while you're still speaking.
NVIDIA declares 25.90% for exactly this model, on the same test, with the same 1120 ms chunk — we measured 26.15% (95% CI 23.89–28.42). The model saw little Romanian in training: NVIDIA's model card places Romanian in its «broad-coverage» tier, the second of three, and lists FLEURS among its training data. Our figure agrees with the card, but may be optimistic; we are checking it with a memorization probe. On the same 200 utterances, SpeD — an open Romanian model, offline, running locally on CPU — scores 6.70%. That is the offline ceiling we are working toward. Russian is a different language and a different test (FLEURS Russian): the same model, in the same run, scores 9.15%.
Text normalization fixed on 5 September 2026 (thousands separators, times of day): 26.31% → 26.15% for Romanian, 9.30% → 9.15% for Russian. Same transcripts, rescored. All figures, with methodology →
Every claim about others is taken from their documentation, not inferred. Where they don't publish a figure, it says "unpublished" — we don't put a guess in its place.
| Criterion | Grai | ChatGPT Live | Rest of the market |
|---|---|---|---|
| Romanian | yes | yes, but Romanian isn't on their top-fluency list; for less common languages, their system card acknowledges gaps in accent and fluency | 5 of 6 major labs don't support it at all; Google's model card acknowledges that language detection fails on non-native accents |
| Audio stays on your server | yes, entirely | no | only expensive on-premise, from two vendors |
| Available as an API | soon | the app has no API; their Realtime API is a separate product | yes |
| Session length | survives page reloads, disconnects and server restarts | not published for the app; the Realtime API cuts off at 60 minutes | only one vendor has true resumption |
| Works between visits | yes | no | nobody |
| Native full-duplex | no — cascaded, with barge-in | yes | mostly cascaded |
| Accuracy on Romanian | 26.15% measured | unpublished | 3.0–5.28% declared |
| Time to response | 2667 ms measured p50 · n=29 · first sound of the answer |
unpublished | 1.26 s at the top of the competition HumDial · a different quantity, not subtractable |
| Running cost | 0 | per minute | $0.40–1.20/hour |
It's the winner's latency at HumDial, ICASSP 2026 — the only challenge where full-duplex is actually measured on conversation, not on isolated inference. The "under 300 ms" figures in press releases measure something else. And the challenge gives 80% of the score to behavior and 20% to speed — because a fast agent that cuts people off mid-sentence is worse than a slow one that waits. The 1.26 s and our 2667 ms are different quantities, measured differently: one is not subtracted from the other.
The full architecture: which model does what, where they connect, what runs locally and what doesn't.
See the architecture → FeaturesEverything the system does today, with the measured figure next to each item. And what it doesn't do yet.
See the list → EvidenceEvery measurement, with the chunk size, the sample and the confidence interval.
See the figures → StudyFive methodology traps, with examples from our own mistakes.
Read the study → RoadmapThree training runs: streaming recognition learned from the offline model, the turn-taking decision, our own voice.
See the roadmap → VoiceWhat's possible today, how long it takes, and the licensing and biometric data obligations.
See the details → CollaborationRecorded Romanian speech with a clean license. What we're looking for and what we offer in return.
See what we need →The same engines, behind one interface: agents as records, voice and language per agent, isolated tenants, a one-line widget. We open access as we train speech recognition on Romanian.