Skip to main content
SpeakoFlow uses your GPU when it can and falls back to the CPU otherwise. The defaults pick the best available device on every platform, so you rarely need to change anything.

​
Speech acceleration

The acceleration control sits under Settings → Advanced, inside the Experimental features group. Turn Experimental features on to see it. One dropdown covers both the backend and the device:
  • Auto (default). Uses the best available GPU, falling back to CPU when there is none.
  • A specific device. Pins speech inference to that GPU. Each detected GPU shows its name and memory, for example NVIDIA GeForce RTX 4070 (12.0 GB). Pick one when you want transcription off the GPU your game or render job is using.
  • CPU. Forces CPU inference. Slower, but sidesteps every driver problem.
This setting drives both transcription engines (the current GGUF engine and the older ONNX engine).

​
CPU baseline on x86-64

The official release builds target a fixed instruction set: AVX2, FMA, F16C, and BMI2. If your processor lacks these (some older Xeons), SpeakoFlow refuses to load a Whisper model rather than crashing, and tells you what is missing.

​
ONNX acceleration

A separate ONNX dropdown appears only when your build has more than one ONNX backend compiled in. It applies to the older-engine models that run on ONNX Runtime: Parakeet, Canary, Moonshine, SenseVoice, GigaAM, and Cohere. On Windows the official build includes DirectML, and Auto resolves to DirectML. That is an intentional override: the underlying library’s own Auto skips DirectML, which would silently pin every ONNX model to the CPU. On macOS and Linux the official builds compile no ONNX GPU provider, so ONNX models run on the CPU regardless.

​
The built-in LLM engine

The built-in local engine (llama.cpp) powers the assistant and AI cleanup when you use the “On this computer” provider. It runs two separate processes:
  • Assistant on port 11435, with vision projector support and a full context window.
  • Cleanup on port 11436, with no projector and a smaller context window.
Both processes use the GPU. On a machine with a dedicated GPU and an integrated one, the engine pins itself to the dedicated GPU. It does this by matching its own device list against the app’s probe by adapter name, so even when the two enumerations disagree on numbering, the right GPU is selected. On machines with only one GPU, or only integrated graphics, this is automatic. The engine starts without a Vulkan runtime if one is not installed. It falls back to CPU, which is slower but works.

​
Unload timeouts

Each role has its own unload timeout: The timeout options are: Never, Immediately, After 2 / 5 / 10 / 15 minutes, After 1 hour. With debug mode on, an extra After 15 seconds option appears for testing.
  • Never. Keeps the model resident. Fastest first response, highest idle memory.
  • Immediately. Unloads after every use. Lowest idle memory, a reload cost every time.
  • After 5 minutes. Covers a normal working rhythm. Several dictations or questions in a row keep the model warm, and the memory comes back when you walk away.
The assistant’s engine is pre-warmed when you start a recording, so most reloads stay invisible. The cleanup engine warms up with a real prefill at recording start, so page-in and GPU shader compilation happen during recording rather than on your first cleanup. You can also unload the speech model on demand from the system tray menu.

​
Context size

The assistant’s local engine launches with a context window of 8,192 tokens by default. You can adjust it from 512 to 32,768 in steps of 512, in the assistant’s settings. Context is the biggest driver of how much memory the engine holds. The small models this feature targets do not need more than 8K. Raising it raises memory use even when your conversations are short. A vision-capable model loads its multimodal projector alongside the weights, so budget for more than the model size alone.
A dedicated GPU always wins over an integrated one, even when the integrated one reports a larger shared-memory budget.For the GGUF speech engine, both Auto and a specific GPU resolve to an automatic backend with a guaranteed CPU fallback. Neither can leave you unable to transcribe on a machine with no usable GPU.
On Linux only, SpeakoFlow enumerates compute devices by launching a short-lived copy of itself rather than doing it in-process.The vendored whisper.cpp/ggml is compiled with -march=native on the build machine. On Linux, packages are routinely built on one machine and run on another, so loading ggml’s Vulkan backend on a CPU narrower than the build machine’s can raise SIGILL, an illegal-instruction crash that would take the whole app down at launch.Running the probe in a child process turns “the app will not start” into “no GPU is listed”. If the child crashes, hangs (killed after 30 seconds), or produces nothing parsable, SpeakoFlow logs a warning and carries on with CPU transcription.On x86-64, if your CPU lacks FMA3 support, GPU enumeration is skipped outright, because ggml’s Vulkan backend uses FMA3 internally.Windows and macOS keep the in-process call.
Two independent settings, both off by default.Always-on microphone (Settings → Dictation → Output). Keeps the microphone stream warm so the first words are never clipped by the moment it takes to open the device. Uses a little more battery, and on some systems keeps the microphone-in-use indicator lit.Keep mic open between transcriptions (Settings → Advanced → Experimental). Leaves the audio stream open after a transcription finishes, which makes back-to-back dictation faster. It can affect Bluetooth audio quality, since the headset never gets to leave its microphone profile.The first warms the microphone up before recording. The second avoids tearing it down afterwards. They can be used together.

Transcription models

The full catalog and how to choose.

Models you already have

Add a model file or link a folder.
Last modified on August 30, 2026