Engineering
The build log.
Benchmarks, architecture notes, and technical writing.
— Engineering
Vision-language models on a consumer GPU: four models, 4,004 images, one RTX 3060
Four open-weight VLMs, 4,004 requests, 1,001 COCO images. Per-request throughput, VRAM, and power draw: 24.1p total electricity.
— Engineering
Self-hosted inference on a single consumer GPU: 14 models on an RTX 3060
What does self-hosted inference cost in practice? 14 open-weight models, full telemetry, one GPU: 5.6p electricity cost 280k tokens.
— Engineering
Local LLM Serving on a £170 GPU: vLLM, Ollama, and Marigold Compared
Three serving stacks, one model, one cheap GPU. A 14.7x speedup in our own code, and a VRAM problem most benchmarks don't mention.
— Engineering
Impact of Serialisation Format on LLM Task Performance.
A benchmark across four small models found the format effect everyone talks about mostly wasn't there. The task was the variable that mattered, and it broke in the same way for every model we tried.
— Engineering
Good Practice and Silent Errors in Distributed Inference
Distributed inference fails quietly. Three practices -- joined observability, consistent container monitoring, and classified error states -- determine whether those failures are findable.
— Engineering
Open-weight model inference on dedicated UK hardware: benchmark results
Throughput and latency measurements across 23 instruct models, 8 embedding models, 3 text-eval models, and 7 TTS models. 738 completed jobs, zero errors, on private AWS infrastructure in eu-west-2.