Profpatsch/users/Profpatsch/mastodon-alt-text

View full contents ↓

mastodon-alt-text(1)

mastodon-alt-text - generate Mastodon alt text for the image in the clipboard

mastodon-alt-text [-addr host:port] [-browser bin]
mastodon-alt-text serve [-addr host:port] [-idle duration]

mastodon-alt-text takes the image on the X11 CLIPBOARD selection, asks Gemini 2.5 Flash for three alt texts, and opens a browser window to pick between them. The chosen text is copied to both the PRIMARY and CLIPBOARD selections, ready to paste into Mastodon's alt text field. Any of them can then be refined by saying what should change, which produces three new ones to choose from, or the prompt they were all written to can be edited and the three generated again.

Three are generated rather than one because judging alt text is a matter of taste: whether to name what a thing is or describe how it looks, how much of the background matters, whether text in the image should be quoted. Picking from a few is faster than re-running until one is good, and saying what is wrong with the closest one is faster still.

Invoked bare – from a keybinding – it restarts the user service and points a browser at it. The work happens in the service; see SERVICE.

-addr host:port

Address to bind (serve) or to open (the launcher). By default serve takes an ephemeral port and the launcher reads whichever one it got, so neither has a port number built in.

-browser bin

Browser to open the picker with, resolved on PATH. Default chromium.

-idle duration

(serve only) Exit once no page has been connected for this long. Default 60s.

The page shows the image and three cards, which fill in as the generations land rather than all at once at the end. Each card gives its length, and says so if it runs over Mastodon's 1500 character limit.

Clicking a card copies it. Picking again just overwrites the selections, so it costs nothing to change your mind. A generation that fails is shown as an error on its own card and does not disturb the other two.

Nothing closes the window: you close it when you are done, and the server notices the browser has gone and exits by itself.

Under the image is the prompt all three were written to, prefilled with the default and editable. Changing it and pressing Regenerate – or Ctrl-Enter, or simply leaving the field after an edit – produces three new texts.

This is the whole prompt rather than an extra line appended to a hidden one, so the default's clauses can be contradicted rather than only added to: asking for a different language, or for more than 1500 characters, works because the sentence that said otherwise is right there to be edited. The other side of that is that the default's three clauses are load-bearing. "Output only the alt text" is what suppresses the "Here is the alt text:" preamble, and deleting it brings the preamble back. reset to default, which appears once the text differs, puts the original back without generating anything.

Regenerate is enabled whether or not the prompt was edited, so it doubles as a reroll: the three differ by sampling alone (see CAVEATS), so asking the same question again is a reasonable move when all three were bad rather than the brief being wrong. An empty prompt is refused instead of being quietly replaced by the default, which would leave the field disagreeing with what was sent.

Regenerating starts a new conversation: the image and the new prompt, and nothing else. Refinements made before it are discarded rather than carried over – see Refining for why that is the useful behaviour rather than a limitation.

If none of the three is quite right, pick the closest one, type what should change – "shorter", "do not mention the background", "name the species" – and press Refine. Three fresh takes on that instruction replace the cards. This can be repeated as often as you like.

Refining works from the variant you picked, so a pick is required first. That is also why nothing is lost when the cards are replaced: the text a round started from was copied to the clipboard when it was chosen, and it stays there until you pick again.

Instructions accumulate rather than replace one another. A round that asked for a shorter text is still in force when a later round asks for something else, so constraints can be built up one at a time instead of having to be described all at once. The image is part of the conversation throughout, so each round still describes the picture rather than paraphrasing the previous description.

Only the chosen variant is remembered; the two you did not pick are discarded and never shown to the model. Rounds run one at a time, and both Refine and Regenerate are disabled while one is generating.

Refining and regenerating are opposites, and which one to reach for follows from that. Refining is a conversation: it needs a pick, it accumulates, and it drifts. Regenerating throws the conversation away and asks the original question differently: it needs no pick, works before anything has been chosen, and cannot drift because there is no history left to drift through. Use Refine to adjust a text that is nearly right; change the prompt when all three are answering the wrong question.

The server runs as the systemd user unit mastodon-alt-text.service, which the launcher restarts on every invocation. Restart rather than start, because systemctl start on an already-running unit does nothing, and each run has to re-read the clipboard.

The server binds an ephemeral port rather than a fixed one. A single browser window on loopback has no need of a well-known number, and claiming one only creates a collision with whatever else happens to want it. The port that was assigned is written to $XDG_RUNTIME_DIR/mastodon-alt-text.port, which the launcher reads.

The unit is Type=notify, so systemctl restart returns only once the port is accepting connections and has been published. The browser is spawned after that returns, which is what removes the need for it to retry against a server that is not up yet, and what makes the port file certain to be the current run's rather than a leftover.

There is no [Install] section. The service is only ever started on demand; enabling it would start a server with nothing to caption.

An open page holds a server-sent events stream, and that stream is the liveness signal. While one is connected the server stays up; when the last one goes the process exits after the idle timeout, or immediately reconnects if the browser was only reloading. The timeout also runs from startup, so a browser that never manages to connect does not leave a server behind.

The timeout is measured on a monotonic clock, which does not advance while the machine is suspended. Closing the lid on a closed picker therefore does not expire the server mid-sleep; it exits a minute after the machine wakes.

Images are converted to WebP before upload, which for a typical PNG screenshot is a twentyfold reduction – 408 KB to 18 KB in one measurement – and upload dominates the request.

The generations run concurrently, so three cost about as long as the slowest one rather than their sum: around two seconds in total. Model thinking is disabled, as it adds seconds and buys nothing for captioning an image that is already in the request.

The unit sets KillMode=process, which is required rather than hardening. An X11 selection is not stored anywhere: it is owned by a live process that hands over the data when another client asks for it, which is why xclip(1) stays resident after writing one. Under the default KillMode=control-group systemd kills that process along with the server, and the text quietly disappears from the clipboard about a minute after it was copied.

Reading the image consults the TARGETS selection first. Asking xclip(1) for image/png when the clipboard holds text does not fail – it returns the text, which then reaches magick(1) and surfaces as "no decode delegate for this image format" rather than saying there is no image.

GEMINI_API_KEY

API key for Gemini. If unset, the key is read from pass(1) at internet/ai.google.dev/gemini/api-keys/gemini-2.5-flash. Under systemd that needs GNUPGHOME set so gpg-agent(1) can be reached; the shipped unit does this.

~/.config/systemd/user/mastodon-alt-text.service

The user unit, copied from the source tree.

$XDG_RUNTIME_DIR/mastodon-alt-text.port

The address the running server bound, written before it reports readiness and removed when it exits. Falls back to /tmp if the variable is unset.

The mastodon-alt-text utility exits 0 on success, and >0 if an error occurs.

Failures that happen after the server is up – an empty clipboard, a rejected API key – are reported in the browser window instead, since that is where you are looking. The service still exits successfully once it goes idle.

Take a screenshot, then caption it:

$ import png:- | xclip -selection clipboard -t image/png

$ mastodon-alt-text

Use a different browser:

$ mastodon-alt-text -browser firefox

Run the server in the foreground, without systemd, to see what it is doing:

$ mastodon-alt-text serve -addr 127.0.0.1:9999 -idle 5m

magick(1), pass(1), systemctl(1), xclip(1)

Profpatsch

The launcher passes --app to the browser and does not wait for it. With a browser already running, the URL is handed to the existing process and the launcher returns at once, so a window that fails to open is not reported: the symptom is a service that starts and then quietly goes away a minute later.

Alt text is generated from an image alone, with no idea of the post it belongs to, so it describes rather than explains. For a screenshot of a conversation or a chart, expect to edit the result.

The three generations differ only in sampling temperature, raised above the default so they do not come back nearly identical. They are all written to the one prompt, so all three can share a misreading of the image, and no amount of rerolling will shake it – that is what editing the prompt is for.

Refinements chain, and a long chain drifts. Each round works from the previous wording as much as from the picture, so by the fourth or fifth a detail that was only ever visible in the image can quietly disappear. When that happens it is quicker to start over than to refine back: press Regenerate, which drops the accumulated conversation and goes back to the image. The last text you picked is on the clipboard either way.