Skip to main content
SpeakoFlow uses your GPU when it can and falls back to the CPU otherwise. The defaults pick the best available device on every platform, so you rarely need to change anything. Intel Macs and the Windows on ARM build run speech on the processor. So does the x64 Windows build when it runs emulated on a Windows on ARM PC, because the graphics path under emulation is not reliable enough for speech models.

​
Speech processing

Where speech runs is one dropdown, Speech processing, in Settings → Advanced → Performance. Every local speech model uses it, Whisper .bin files included, because they all run on the same engine (transcribe.cpp).
  • Auto (default). Uses your graphics card, the dedicated one if you have both a dedicated and an integrated one. With no usable graphics card it uses the processor.
  • A specific graphics card. Each detected card shows its name and memory, for example NVIDIA GeForce RTX 4070 (12.0 GB). Pick one when you want transcription off the card your game or render job is using. If that card is later missing (an unplugged eGPU, a driver that stopped reporting it), SpeakoFlow falls back to Auto rather than failing.
  • Processor (CPU). Slower, but works everywhere. Pick it if dictation crashes or your graphics driver has problems.
Changing it unloads the speech model, so your next dictation loads it on the device you chose.
Updating from an older version carries your choice over: Auto stays Auto and CPU stays Processor. A specific graphics card picked in the old Whisper acceleration setting goes back to Auto once, because the new engine lists devices differently. Pick it again from the dropdown. The old ONNX acceleration setting is gone with the engine it belonged to.

​
The built-in LLM engine

The built-in local engine (llama.cpp) powers the assistant and AI cleanup when you use the “On this computer” provider. It runs two separate processes:
  • Assistant on port 11435, with vision projector support and a full context window.
  • Cleanup on port 11436, with no projector and a smaller context window.
Both processes use the GPU. On a machine with a dedicated GPU and an integrated one, the engine pins itself to the dedicated GPU. It does this by matching its own device list against the app’s probe by adapter name, so even when the two enumerations disagree on numbering, the right GPU is selected. On machines with only one GPU, or only integrated graphics, this is automatic. The engine starts without a Vulkan runtime if one is not installed. It falls back to CPU, which is slower but works.

​
Unload timeouts

Each role has its own unload timeout: The timeout options are: Never, Immediately, After 2 / 5 / 10 / 15 minutes, After 1 hour. With debug mode on, an extra After 15 seconds option appears for testing.
  • Never. Keeps the model resident. Fastest first response, highest idle memory.
  • Immediately. Unloads after every use. Lowest idle memory, a reload cost every time.
  • After 5 minutes. Covers a normal working rhythm. Several dictations or questions in a row keep the model warm, and the memory comes back when you walk away.
The assistant’s engine is pre-warmed when you start a recording, so most reloads stay invisible. The cleanup engine warms up with a real prefill at recording start, so page-in and GPU shader compilation happen during recording rather than on your first cleanup. You can also unload the speech model on demand from the system tray menu.

​
Context size

The assistant’s local engine launches with a context window of 8,192 tokens by default. You can adjust it from 512 to 32,768 in steps of 512, in the assistant’s settings. Context is the biggest driver of how much memory the engine holds. The small models this feature targets do not need more than 8K. Raising it raises memory use even when your conversations are short. A vision-capable model loads its multimodal projector alongside the weights, so budget for more than the model size alone.
A dedicated GPU always wins over an integrated one. For speech, Auto takes the first graphics card that starts, every dedicated one before any integrated one, and falls back to the processor when none works. The built-in LLM engine prefers the dedicated card too, even when the integrated one reports a larger shared-memory budget.A saved card that is no longer there falls back to Auto, and Auto falls back to the processor, so a machine without a graphics card still transcribes.
On Linux only, SpeakoFlow lists compute devices by launching a short-lived copy of itself rather than doing it in-process.Listing the devices loads the graphics driver’s Vulkan code, and on Linux drivers vary more than anywhere else. A driver that crashes during that step would otherwise take the whole app down at launch.Running the probe in a child process turns “the app will not start” into “no GPU is listed”. If the child crashes, hangs (killed after 30 seconds), or produces nothing parsable, SpeakoFlow logs a warning and carries on with CPU transcription.On x86-64, if your CPU lacks FMA3 support, GPU listing is skipped outright, because the Vulkan backend uses FMA3 internally and would crash.Windows and macOS keep the in-process call.
Two independent settings, both off by default.Always-on microphone (Settings → Dictation → Output). Keeps the microphone stream warm so the first words are never clipped by the moment it takes to open the device. Uses a little more battery, and on some systems keeps the microphone-in-use indicator lit.Keep mic open between transcriptions (Settings → Advanced → Experimental). Leaves the audio stream open after a transcription finishes, which makes back-to-back dictation faster. It can affect Bluetooth audio quality, since the headset never gets to leave its microphone profile.The first warms the microphone up before recording. The second avoids tearing it down afterwards. They can be used together.

Transcription models

The full catalog and how to choose.

Models you already have

Add a model file or link a folder.
Last modified on October 20, 2026