> ## Documentation Index
> Fetch the complete documentation index at: https://www.speakoflow.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Screen vision

> Ask the SpeakoFlow assistant about what is on your screen. Covers the three screen access modes, when the frame is captured, image quality per provider, and the macOS Screen Recording permission.

Ask about what is on your screen and the assistant answers with that context. It only looks when you let it.

Set it up in *Settings → Assistant → Screen vision*. You need a vision-capable model. Most curated local models qualify (the catalog marks them "Vision"), as do the mainstream cloud multimodal models.

## Screen access

| Mode                 | What happens                                                                                                                                                                                     |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| **Off**              | Screen capture is disabled. The panel's camera and region controls are hidden.                                                                                                                   |
| **Manual** (default) | You decide. Click the camera in the panel and your screen is attached to every message until you turn it off again, or snip just a region.                                                       |
| **Agent decides**    | The model gets a `capture_screen` tool and may take one screenshot per message when your question clearly refers to something on screen. It is told not to capture for self-contained questions. |

Every screenshot that goes to the model is shown in the conversation, in both active modes, so there is always a visible record of when the assistant looked.

<Warning>
  **macOS.** Screen vision needs macOS Screen Recording permission. Grant it in *System Settings → Privacy & Security → Screen Recording*, then restart SpeakoFlow for it to take effect.
</Warning>

## What actually gets sent

SpeakoFlow captures the monitor under your mouse cursor, so on a multi-monitor desk you get the screen you are actually working on.

Only a compact thumbnail is stored in the conversation and in [History](/docs/personalize/history). The full-resolution frame is sent to the model once and never written to disk. Thumbnails show inline in the panel and can be clicked to enlarge.

<AccordionGroup>
  <Accordion title="When the frame is grabbed for a voice question">
    *When I ask, When the message sends. Default: When I ask. Shown in Manual mode only.*

    This changes the timing for voice questions only, where there is a real gap between starting to ask and finishing.

    * **When I ask** (default). Grabs the frame the moment you press the hotkey, so it captures what you were looking at when you started talking, not whatever is on screen after you finish.
    * **When the message sends**. Grabs it after you stop talking and the speech is transcribed.

    Typed messages always capture on send, since the panel is already in front of you either way.
  </Accordion>

  <Accordion title="How much image quality each provider gets">
    The frame is JPEG-compressed down to a budget picked from your provider, because "send the sharpest possible image" and "the request succeeds" are not the same goal.

    | Provider                   | Budget   | Why                                                                                                   |
    | -------------------------- | -------- | ----------------------------------------------------------------------------------------------------- |
    | Azure                      | Smallest | Azure's gateway rejects oversized request bodies outright.                                            |
    | Built-in or a local server | Middle   | Keeps on-screen text legible without spending a huge share of a small local context on vision tokens. |
    | Other cloud providers      | Largest  | Sharpest, so fine text, code, and error messages stay readable.                                       |
  </Accordion>

  <Accordion title="What happens if your model cannot read images">
    The panel says so and names the problem rather than silently dropping the screenshot. Switch to a vision-capable model, or ask again with screen vision off.
  </Accordion>
</AccordionGroup>