On-device speech in 60 megabytes
Keyboard extensions run under one of the tightest memory budgets on the platform. Roughly 60 to 80MB. iOS's jetsam process kills anything over that. No warning, no graceful degradation, just gone.
A speech recognition model doesn't fit in that budget. Parakeet is a few hundred megabytes. Whisper's smaller checkpoints still don't clear the bar. The honest answer to "can a keyboard extension do on-device speech" looked like no.
The way out wasn't a smaller model. It was not running the model in the extension at all.
Out of process by design
Apple's SpeechAnalyzer, the framework behind Outspoken's Apple engine, runs its model out of process. The extension talks to it through an API boundary. The weights live in a system process that isn't subject to the extension's jetsam limit. From the extension's point of view, transcription is a service call, not a local computation. The memory the model needs never has to be memory the keyboard owns.
That constraint ruled out the other two engines for the keyboard target entirely. Parakeet, via FluidAudio, and Whisper, via WhisperKit, both load their weights in process. Fine for the main app, where there's no jetsam ceiling. Useless for an extension. A compilation flag, OUTSPOKEN_HEAVY_ENGINES, gates those two engines out of every app extension target at the source level. It has to be an explicit flag rather than a canImport check, because all targets in the project share one build products directory. A canImport guard would see the framework is present and wrongly assume it's usable.
The permission problem
SpeechAnalyzer needs RequestsOpenAccess to get microphone and App Group access from inside a keyboard. Keyboard extensions can't present their own permission prompts. There's no TCC dialog surface available to them. So the microphone grant has to happen in the container app first, before the keyboard extension ever runs. If a user installs just the keyboard and skips the app, dictation silently can't start. The extension isn't broken. It's waiting on a permission it structurally cannot ask for itself.
Finalized text only
SpeechAnalyzer streams results as you speak. Two flavors: volatile, still resolving, and finalized, locked in. Inserting every update into the text field means retyping words as the model revises its guess. Visually noisy, and on a keyboard, disorienting, since the cursor jumping around is the user's own cursor.
Outspoken's keyboard only inserts finalized text. Nothing ever has to be deleted and retyped in front of the user. Text accretes forward as the model commits to it.
What crosses the process boundary
The keyboard and the container app don't talk to each other directly. They share an App Group. Settings, a heartbeat flag so the app knows the keyboard is alive, and a pending-intent flag all live in a shared UserDefaults suite.
Dictation history is trickier. It's stored in SwiftData, and SwiftData doesn't push changes across process boundaries. A dictation made in the keyboard doesn't appear in the app's history list until the app itself refetches, which happens on becoming active, not continuously. A small compromise, and the honest one. Cross-process live sync would have added real complexity for a feature almost nobody needs.
None of this is exotic engineering. Mostly a series of small refusals. Refusing to load a model where it can't fit. Refusing to assume a permission prompt is available. Refusing to retype text the user already saw committed. The constraint didn't get solved so much as respected.