How Parakeet turns sound into text on the Neural Engine
Parakeet is NVIDIA's speech recognition model. It's the accuracy option in Outspoken. Not the fastest to first word, but consistently the most correct on messy audio, accents, and technical vocabulary. Understanding how it's built explains why Outspoken treats it as a batch engine rather than a streaming one.
A transducer, not a decoder
The usual mental model for speech recognition is an encoder that turns audio into features and a decoder that turns features into text, the same shape as machine translation. Parakeet's TDT variant, a token-and-duration transducer, works differently. A joint network combines the encoder's audio representation with a prediction network at every timestep and scores the next token from that combination. There's no decoder attending back over the whole sequence.
The duration part is what makes TDT fast. Instead of stepping through audio one frame at a time waiting to decide there's nothing to emit, the model predicts how many frames it can skip before the next token is likely. Silence and steady phonemes get consumed in one jump instead of many wasted steps.
This matters for on-device latency. A frame-synchronous transducer without duration prediction spends compute on every frame whether or not there's a token to emit. TDT spends its compute where the information actually is.
Why Outspoken runs it as a batch engine
Outspoken's TranscriptionEngine protocol has one method, transcribe, with callbacks for partial results and download progress. Apple's SpeechAnalyzer is the only implementation that calls the partial-result callback with anything meaningful. Parakeet and Whisper both produce a transcript once, after dictation stops.
That's a reflection of how these models are built, not a missing feature. SpeechAnalyzer is engineered by Apple for live captioning and dictation, low per-frame latency as a first-class goal, at some cost to raw accuracy. Parakeet, running through FluidAudio's CoreML conversion, is optimized for accuracy over the whole utterance. You get better results feeding it a complete audio buffer and letting the transducer walk it than force-feeding incremental slices and asking for a guess every hundred milliseconds. Outspoken's engine picker is really a latency-versus-accuracy choice wearing a settings screen.
Getting it onto the Neural Engine
Parakeet doesn't ship as a PyTorch checkpoint in the app. It's converted to CoreML ahead of time, which is what lets it run on Apple's Neural Engine instead of burning CPU or the discrete GPU path. FluidAudio's AsrModels.load(from:) loads that converted model directly from a bundled directory in the iOS build. Outspoken enforces offline loading once it's bundled, blocking the framework from silently trying to re-download weights it already has locally. That matters for the privacy story and for airplane-mode reliability.
One detail looks cosmetic and isn't: the bundled folder has to be named exactly parakeet-tdt-0.6b-v3-coreml. FluidAudio derives its internal model paths from that folder name at load time. Rename it for tidiness and loading breaks silently, no clear error. The kind of constraint that shows up once, expensively, and gets written down forever after.
What this buys you
Apple's engine for a conversation with your cursor, where every second of latency is felt. Parakeet for something you'll actually read back: a commit message, a long prompt, a paragraph. Worth an extra beat to get the words right the first time.