PyPI version Latest tag  Open source · Python · Open-weight models

A self-hosted inference API for open-weight models.

Serve HuggingFace models over a typed HTTP API, on your own hardware.

Marigold runs text, image, and audio models locally behind an OpenAI-compatible API -- chat, embeddings, image generation, and more, served from Docker Compose. No request ever reaches OpenAI, Anthropic, or any other external provider; runs fully airgapped once models are cached. 

pip install bayis-marigold
git clone https://github.com/bayinfosys/marigold-examples
marigold deployment start marigold-examples/chat
See what it runs View on GitHub

Runs on hardware you control. No request leaves the machine, no third-party model provider sits in the path, nothing is retained outside the directory you point it at. For regulated sectors, that resolves the data question before procurement starts. Architecture detail →

Applications, not just endpoints

01

Models are the stable layer

One shared weight cache, independent of which application is running. Downloaded once, reused by everything that needs it.

02

Applications are the thin layer

A models.yaml, a marigold.toml, and application code -- a package. Marigold runs it; it never runs the package's own code, by design.

03

Why this split matters

We've written about this at length: models as a fat protocol, applications as thin clients on top -- the same shape that made early internet protocols durable. Read the essay →

What runs on it

Content ingestion and moderation

Extract text, embed, classify, and gate restricted content before it reaches an index. Text and image embeddings in one pipeline enable cross-modal search.

img2txt text-embed instruct image-embed

Quality-gated image generation

Generate, score for safety, aesthetic quality, and prompt alignment. Regenerate automatically until thresholds are met or attempts run out.

txt2img image-eval image-text-eval

Visual conformance checking

Compare an observation image against a reference by structural embedding. On deviation, segment both, diff the masks, and describe the discrepancy in natural language.

image-embed img2mask img2txt instruct

Document briefing and audio delivery

Extract entities, classify, summarise, and convert to speech in multiple languages from one workflow submission. Output: a summary plus audio per language.

img2txt instruct tts

Batch tabular classification

Classify large volumes of unlabelled rows against a small labelled set. Low-confidence predictions go to an instruct model for explanation. No retraining.

tabular-classify instruct

Site monitoring with change detection

Embed and compare images from multiple locations in parallel. On change, describe it and assemble a report, delivered as text and audio on a schedule.

image-embed img2txt instruct tts

What workflows can do

Model types

  • text-embedding, image-embedding
  • instruct (chat / instruction-following)
  • txt2img, img2txt, img2mask, depth
  • tts, txt2audio
  • text-eval, image-eval, image-text-eval
  • text-similarity

Workflow features

  • Shared state across steps
  • Sequential and parallel execution
  • Conditional branching on output values
  • Halt and reject with a structured result
  • Prompt interpolation from prior outputs
  • Per-step audit trail -- status, timing, output, run count

Workflow execution is built on runfox and json-logic-path, both open source on PyPI.

Stop rebuilding your inference stack per engagement.

Open source. Clone it, run it on your own hardware.

View on GitHub Read the setup guide

Questions? ed@bayis.co.uk