§ 001 / Lead entry
Local-first AI systems for builders running open models on controlled compute.
I publish benchmark rows, failure logs, and build notes for open models running on controlled compute. AgentGuard stays available for budget, loop, timeout, and rate limits. This journal starts from an RTX 5090 workstation in Nashville, but the entries hold for local rigs, rented accelerators, private cloud, and GPU-backed clusters.
| Model | Quant | Workload | Gen tok/s | Prompt tok/s | Peak VRAM |
|---|---|---|---|---|---|
| llama3.1:8b | Q4_K_M | short-gen-256 | 228.9 | 679 | 7.2 GB |
| llama3.1:8b | Q4_K_M | long-context-summarize | 206.7 | 12,109 | 7.8 GB |
| llama3.1:8b | Q4_K_M | agent-code-task-512 | 227.8 | 1,028 | 7.8 GB |
| gemma4:26b | Q4_K_M | short-gen-256 | 198.8 | 15 (cold) | 19.9 GB |
| gemma4:26b | Q4_K_M | long-context-summarize | 180.2 | 6,179 | 20.2 GB |
| gemma4:26b | Q4_K_M | agent-code-task-512 | 207.4 | 197 | 20.2 GB |
Changing num_ctx between requests forces a full model reload. On a 26B that is 140 seconds per swap. Pin your context size.
Every entry carries the model, quant, prompt, hardware, and result. Failed runs stay in the record: the June timeout that preceded this sweep is still published beside it. The archive is the proof surface for local-first work on controlled compute.
Read the 5090 Reports