What our local AI actually runs at

This page reports what the machines behind this blog actually do, measured rather than estimated. It is re-run monthly and the numbers below are dated. Everything here describes a setup that runs in production every night — not a rig assembled to produce an article.

Last run: 2026-08-25. 3 repeats per measurement, median reported. Last reviewed: 2026-09-04 — re-checked against every render since, and unchanged.

The two machines, and what each one is measured on

There are two, and they do different jobs. An M3 Max with 128 GB of unified memory runs the language model. An RTX 5060 Ti with 16 GB runs image generation, and nothing else. Both are measured here, each on the job it actually does: tokens per second below, seconds per image further down.

That division is the first useful finding, and it is not a compromise — it is what a single consumer GPU forces on you. While ComfyUI holds the 5060 Ti's memory for image work, there is no room left for a language model: measured on this machine, Ollama was left with 150 MB of VRAM and a 20-billion-parameter model fell back to running on the CPU entirely. So the card is not benchmarked for LANGUAGE work — publishing a tokens-per-second figure for a GPU that will never serve a language model in this setup would be measuring a configuration that does not exist. What it is asked to do instead is measured properly, and that is the image section.

Language model throughput

gpt-oss:20b on the M3 Max 128GB

20.9B parameters, MXFP4 quantisation, 13.8 GB on disk, 13.09 GB resident — the whole model fits in memory.

gpt-oss:120b on the M3 Max 128GB

116.8B parameters, MXFP4 quantisation, 65.4 GB on disk, 64.68 GB resident — the whole model fits in memory.

Two things stand out. Generation speed barely moves with prompt length — it sits near the same figure whether the prompt is ninety tokens or sixteen hundred, because generation is bound by memory bandwidth rather than by how much there is to read. Prompt processing does the opposite and climbs sharply with length, because a longer prompt parallelises better. If you are choosing hardware for long-context work, the second number is the one that changes your day.

Image generation, on the 16 GB card

The section above explains why the 5060 Ti is not benchmarked for language work. This is what it is benchmarked for: 5 image models across 12 measured configurations, 425 renders in total, on one card with 16 GB of VRAM.

Every row is one CONFIGURATION, not one model. The same model at a different quantisation, resolution, step count or post-processing setting is a different row, because each of those changes the number and a table that hides them is not measuring anything you could reproduce.

The quantisation is worth about 10% of the render

Flux.2 dev at 24 steps and 1920x1072, same card, same finishing settings: Q4_K_M takes 260.3 s across 49 renders and Q6_K takes 285.6 s across 48. 25.3 seconds, for weights that differ only in how they are packed. An earlier version of this page reported a single figure for both — 280.5 s — which was neither of them.

What the finishing chain costs

4 configurations were measured both with and without the upscale-and-grain pass that runs in the same graph. It adds between 10 and 15 seconds, near enough the same regardless of how long the render itself took — so it is a fixed toll, and it dominates a fast model's total while disappearing into a slow one's. Any benchmark that does not say whether post-processing is inside its clock is not comparable with any other.

The comparison with no step-count excuse in it: HiDream-I1 renders a larger 2048x1152 image in 25.1 seconds at 28 steps, while Flux.2 dev needs 260.3 seconds for a smaller 1920x1072 one at 24 — 10 times longer to produce fewer pixels.

How the image numbers were measured

• One row per render, recorded by the renderer. Every figure comes from ComfyUI's own execution record: the graph it ran, the model file it loaded, the latent it sampled, and start-to-finish timestamps. Nothing is inferred from a log line and nothing is re-run to produce a nicer table.
• Real work, not a staged sweep. These are renders this setup actually did — article images, comparison nights, and other production work on the same card. That is why the run counts are uneven: a configuration used more gets measured more.
• Median wall clock of every render at that configuration, not the best one. The run count is in the table so you can see how thin a row is, and a configuration with fewer than 5 renders is not published at all.
• Grouped by the whole configuration. Machine, model file, quantisation, step count, resolution and post-processing. Two of those look like the same model and are not: Flux.2 at Q4_K_M and at Q6_K are separate rows, and an earlier version of this page averaged them into one figure that matched neither.
• Post-processing is inside the clock. Where the column says yes, the same graph also ran a 4x upscaler, Poisson noise and film grain after the render, and those seconds are included — which is why the same model appears twice with a gap between the two.
• One machine. Only renders on the card named above are in the table. Renders on the M3 Max go in its own section rather than being averaged into this one.

Held back from the table: Flux.2 dev Q4_K_M at 8 steps, 1024x1024 (2 render(s), 5 needed); HiDream-I1 mxfp8 at 28 steps, 1024x576 (1 render(s), 5 needed); Flux.1 dev — 77 render(s) on this card, not published: not one of the blog's image models; Flux.2 dev Q6_K at 24 steps, 1920x1080 — 11 render(s) on 192.168.1.194:8188, a different machine. Measured, just not on terms this table can state.

Image figures from render manifest (per-render provenance), RTX 5060 Ti 16GB.

How the language numbers were measured

• Counted, not estimated. Measurements come from Ollama's native generate endpoint, which reports token counts and durations directly. Tokens per second is arithmetic on those, never wall-clock divided by a tokeniser's guess.
• Time to first token is the first token of any kind, including the model's reasoning. Waiting for the first visible word measures something else.
• Cold cache. Every run uses a unique prefix, because the prompt cache will otherwise be mistaken for throughput — with it reused, one run reported almost twenty-six thousand tokens per second of prompt processing, which was the cache being read back rather than the machine working.
• Warm-up discarded, then the median of the remaining runs. Not the best of them.
• Deterministic: temperature zero, fixed seed, fixed generation limit.
• Contention checked. A run is discarded if image generation starts on the same machine, and the share of the model actually resident in memory is recorded rather than assumed.

Changes to the hardware itself

The machines are the premise of every figure above, so when they change it is recorded here rather than quietly absorbed.

• 2026-09-04 — image models measured: added Flux.1 dev, Flux.2 dev, Z-Image Base
• 2026-09-04 — image models measured: no longer present Flux.2

Corrections

• 2026-09-04 — A Flux.1-dev row briefly appeared here. Those renders happen on this card but belong to a different project, at a different resolution; this page reports the models the blog's own image pipeline runs. Removed, and the renders are still recorded — just not claimed here.
• 2026-09-04 — The image table was rebuilt from per-render provenance, and it corrects two errors of identity. Rows were previously grouped by model NAME and step count, taken from a log that names a model class rather than the file loaded. That merged Flux.2 at Q4_K_M with Flux.2 at Q6_K into one figure of 280.5 s describing neither (they are 260.3 s and 285.6 s), merged renders made with and without in-graph post-processing, and reported 55 renders of Z-Image BASE under the name Z-Image Turbo — a different, undistilled model. The same renders, regrouped by the configuration that actually produced them.

Changelog

This is the first run. Each monthly re-run is added here, so a change in the numbers is visible rather than silently overwritten.

Measured 2026-08-25 on hardware in Los Angeles, reviewed 2026-09-04. Re-run monthly. If a figure here looks wrong, it probably is — tell me and I will re-measure it.