The build log.

Engineering

Vision-language models on a consumer GPU: four models, 4,004 images, one RTX 3060

Four open-weight VLMs, 4,004 requests, 1,001 COCO images. Per-request throughput, VRAM, and power draw: 24.1p total electricity.

Engineering

Self-hosted inference on a single consumer GPU: 14 models on an RTX 3060

What does self-hosted inference cost in practice? 14 open-weight models, full telemetry, one GPU: 5.6p electricity cost 280k tokens.

Engineering

Local LLM Serving on a £170 GPU: vLLM, Ollama, and Marigold Compared

Three serving stacks, one model, one cheap GPU. A 14.7x speedup in our own code, and a VRAM problem most benchmarks don't mention.

Engineering

Impact of Serialisation Format on LLM Task Performance.

A benchmark across four small models found the format effect everyone talks about mostly wasn't there. The task was the variable that mattered, and it broke in the same way for every model we tried.

Engineering

Good Practice and Silent Errors in Distributed Inference

Distributed inference fails quietly. Three practices -- joined observability, consistent container monitoring, and classified error states -- determine whether those failures are findable.

Engineering

Open-weight model inference on dedicated UK hardware: benchmark results

Throughput and latency measurements across 23 instruct models, 8 embedding models, 3 text-eval models, and 7 TTS models. 738 completed jobs, zero errors, on private AWS infrastructure in eu-west-2.