Grai · Model
A cascaded real-time system: three models working in a chain, plus the layer that makes them behave like a single conversation.
There are two ways to build a voice agent. Native full-duplex — a single model that takes audio in and puts audio out, the way ChatGPT Live does. And cascaded — speech recognition, then a language model, then speech synthesis: three separate pieces.
Grai is cascaded. That is a choice, not an accidental limitation: the pieces can be swapped independently, two of the three run locally, and the running cost of the audio is zero. The price is that it cannot make sounds while it listens — the pause between your turn and its turn is real.
Almost the entire commercial market is cascaded, including ElevenLabs Agents and most providers. Native full-duplex, in production and with Romanian, is something nobody has today.
| Piece | Model | Where it runs | License | Romanian in model card | Measured, p50 |
|---|---|---|---|---|---|
| Speech recognition (ASR) | Nemotron 3.5 ASR streaming, 0.6 billion parameters | local, on the integrated GPU (M3 Max, MLX) | OpenMDW-1.1 | broad-coverage | 92 ms |
| Language model (LLM) | swappable, through an OpenAI-compatible gateway | external | the provider's | — | 2180 ms |
| Speech synthesis (TTS) | Supertonic 3, 99 million parameters, 31 languages | local, on the CPU | OpenRAIL-M | — | 305 ms |
| Supertonic status: upstream archived 23 July 2026; runs locally, with no upstream development; a replacement is under evaluation. | |||||
The three figures: p50, n=17, language model gpt-5.4-nano, filler turned off. 92 ms is the ASR flush after you stop talking; partial text while you speak arrives with the delay of the 1120 ms chunk (~1.1 s). In the same run, the turn-taking decision takes 586 ms and the first sound arrives at 3172 ms p50 — the language model is ~69% of the total. “Broad-coverage” is the second of three tiers in NVIDIA's model card: the model saw little Romanian in training. No dedicated graphics card.
The only piece that leaves your machine is the language model, and it receives text, not audio. It can be local too, if you have the hardware — then nothing goes out.
The models above are borrowed and can be replaced. What is ours — and what makes the difference between a demo and an agent that can answer the phone — is the layer between them.
Decides when you have finished speaking, from the slope of your intonation. It tells short acknowledgements (“mhm”) apart from taking the turn, raises the threshold when you dictate numbers, and calibrates to your pace in syllables per second.
1,019 lines · 86 testsMeasures pitch, energy, pace, hesitation, whisper — normalized against your own baseline, not in absolute terms. The F0 slope over the last 600 ms is the signal that decides the turn.
731 lines · 1.9 ms per turnRemembers across conversations, with revision over time: “is coming on Friday” becomes “came on Friday” once Friday has passed. Deletion works on disk, not just in the interface.
976 lines · 104 testsIn both directions. From people: “douăzeci și trei de mii patru sute optzeci” (twenty-three thousand four hundred eighty) becomes 23480 before the model. To people: 069123456 becomes “zero șase nouă unu…”, digit by digit. Times, dates, amounts, each with its own rule.
1,280 lines · 98 testsThe agent lives outside the connection. You close the tab, the network drops, the server restarts — the conversation continues. Tools finish their work and the result waits for you in the queue.
632 linesLimits per client and per address, with your position in the queue instead of a flat refusal. The speech recognizer is leased per turn, not per connection — that is where the capacity comes from.
506 lines · 57 tests16,082 lines of code, of which 4,112 are tests. The figures next to each block are counted from the source, not estimated.
When it needs information from outside, most agents go silent and wait. Grai forks: it says a short bridge phrase, sends the job to another thread, and stays interruptible.
The hard part of barge-in is not stopping. Synthesis runs faster than playback, so at the moment you talk over it there is audio left in the buffer that never reached your ear.
Without correction, the model believes it said everything — and on the next turn says “as I already told you” about something you never heard. The history is cut at the number of milliseconds that actually came out of the speaker. The stop itself takes between 1 and 3 milliseconds.
The performance figures above have measurements behind them, with sample, conditions and, where it can be computed, a confidence interval.