PyPI version Latest tag  Open source · Python · Open-weight models

A self-hosted LLM server and inference API.

Serve HuggingFace models over a typed HTTP API, on your own hardware.

Marigold runs text, image, and audio models locally from Docker Compose. Chat and embeddings are served on OpenAI-compatible endpoints; image generation, speech and the other model types use the typed HTTP API. No request reaches OpenAI, Anthropic, or any other external provider; runs air-gapped once models are cached. 

pip install bayis-marigold
git clone https://github.com/bayinfosys/marigold-examples
marigold cache init
marigold platform start
marigold package create marigold-examples/chat -o /tmp
marigold package install /tmp/chat-<version>.tar.gz
marigold cache populate chat
marigold application start chat
Read the setup guide View on GitHub

Runs on hardware you control. No request leaves the machine, no third-party model provider sits in the path, nothing is retained outside the directory you point it at. For regulated sectors, that resolves the data question before procurement starts. Architecture detail →

Applications, not just endpoints

01

Models are the stable layer

One shared weight cache, independent of which application is running. Downloaded once, reused by everything that needs it.

02

Applications are the thin layer

A models.yaml, a marigold.toml, and application code -- a package. Marigold runs it; it never runs the package's own code, by design.

03

Why this split matters

We've written about this at length: models as a fat protocol, applications as thin clients on top -- the same shape that made early internet protocols durable. Read the essay →

What runs on it

Content ingestion and moderation

Extract text, embed, classify, and gate restricted content before it reaches an index. Text and image embeddings in one pipeline enable cross-modal search.

img2txt text-embed instruct image-embed

Quality-gated image generation

Generate, score for safety, aesthetic quality, and prompt alignment. Regenerate automatically until thresholds are met or attempts run out.

txt2img image-eval image-text-eval

Visual conformance checking

Compare an observation image against a reference by structural embedding. On deviation, segment both, diff the masks, and describe the discrepancy in natural language.

image-embed img2mask img2txt instruct

Document briefing and audio delivery

Extract entities, classify, summarise, and convert to speech in multiple languages from one workflow submission. Output: a summary plus audio per language.

img2txt instruct tts

Batch tabular classification

Classify large volumes of unlabelled rows against a small labelled set. Low-confidence predictions go to an instruct model for explanation. No retraining.

tabular-classify instruct

Site monitoring with change detection

Embed and compare images from multiple locations in parallel. On change, describe it and assemble a report, delivered as text and audio on a schedule.

image-embed img2txt instruct tts

What workflows can do

Model types

  • text-embedding, image-embedding
  • instruct (chat / instruction-following)
  • txt2img, img2txt, img2mask, depth
  • tts, txt2audio
  • text-eval, image-eval, image-text-eval
  • text-similarity

Workflow features

  • Shared state across steps
  • Sequential and parallel execution
  • Conditional branching on output values
  • Halt and reject with a structured result
  • Prompt interpolation from prior outputs
  • Per-step audit trail -- status, timing, output, run count

Workflow execution is built on runfox and json-logic-path, both open source on PyPI.

Marigold (bayis-marigold on PyPI) is a self-hosted inference server. The Marigold depth-estimation model from ETH Zurich is a separate project.

Guides

Run a local LLM server

Install the CLI, start the platform with Docker Compose, cache a model and run a first application.

Cursor local model

Point Cursor, or VS Code with Continue, at a self-hosted model, and what each editor sends where.

Ollama alternative

Where Marigold and Ollama differ: model types, API surface, request handling, and measured throughput.

Stop rebuilding your inference stack per engagement.

Open source. Clone it, run it on your own hardware.

View on GitHub Read the setup guide

Questions? ed@bayis.co.uk