Deployment
Marigold is the same platform at every size. Applications do not change
between a laptop and a cloud account: only marigold-host.toml does –
where the database is, where outputs are stored, and where the workers
run. In every shape the data stays on infrastructure you control.
Three shapes
| Shape | Database | Workers | Outputs |
|---|---|---|---|
| Laptop or home box | the bundled Postgres | the local GPU or CPU | the cache directory |
| Startup | a managed Postgres such as Supabase | one or more rented GPU instances | an S3 bucket |
| Institution | AWS RDS or Azure Database for PostgreSQL, in your own account | GPU instances in your own account | S3 or Azure Blob Storage, in your own account |
The parts
| Part | What it holds | Where it can run |
|---|---|---|
| Database | the job queue, results, the model catalogue, usage | the bundled Postgres, or any Postgres 17 |
| API | the HTTP entry point for applications and clients | beside the database, or anywhere that reaches it |
| Workers | model execution, one model loaded at a time per worker | any host with a GPU, or a CPU, that reaches the database |
| Model cache | downloaded weights | one per worker host |
| Object storage | images, audio and video outputs | the cache directory, S3, or Azure Blob Storage |
Workers take jobs from the queue in the database. Adding a GPU host adds a worker; nothing else changes, and applications see more throughput.
Laptop or home box
The default: everything under one cache directory, started by
marigold application start. See Getting started.
External database
Point every host at the same Postgres:
# marigold-host.toml
[database]
url = "postgresql://marigold:<password>@db.example.internal:5432/marigold"
With a database URL set, the platform does not start its own Postgres.
Managed services that work as they are:
- Supabase, for a small team: use the direct connection string, not the transaction pooler, since workers hold long-lived connections.
- AWS RDS for PostgreSQL, in your own VPC.
- Azure Database for PostgreSQL, flexible server, in your own virtual network.
Keep the database private to the network the API and workers run in.
Object storage
Outputs go to the cache directory by default. For several hosts, use a bucket every host can reach.
S3, or any S3-compatible service:
# marigold-host.toml
[storage]
outputs = "s3"
bucket = "marigold-outputs" # must already exist
region = "eu-west-2"
# endpoint = "http://..." # an S3-compatible service; unset for AWS S3
On AWS, give the instances an IAM role with read and write on the bucket rather than access keys.
Azure Blob Storage:
# marigold-host.toml
[storage]
outputs = "azure"
container = "marigold-outputs" # must already exist
with AZURE_STORAGE_ACCOUNT=<account> in the shell environment.
On Azure, give the instances a managed identity with the Storage Blob Data Contributor role on the container.
The worker writes outputs; the API only reads them. See Configuration for every storage key and the secrets each store reads from the shell.
GPU worker hosts
Each worker host runs a worker against the shared database, with its own model cache:
# marigold-host.toml on a worker host
[platform]
compose_files = ["cpu", "gpu"] # a worker only: no API, no Postgres
[database]
url = "postgresql://marigold:<password>@db.example.internal:5432/marigold"
[storage]
location = "/data/marigold"
Populate each host’s cache with the models it should serve, then start it:
marigold cache populate <application>
marigold platform start
A host serves the models in its own cache. Hosts with different GPUs can hold different models: large instruct models on the large cards, embeddings and evals on smaller ones.
The API host
The API host runs ["core"] with the same [database] and storage
settings, and no worker. Applications run beside it, or on any host
that reaches it.
Marigold has no authentication of its own. Outside a private network,
put your own in front of the API – a reverse proxy, a load balancer
with your identity provider – and have it set X-User-Id for each
caller. See Configuration.
Restricting instances
Marigold limits what each container can use: memory, CPUs and
processes per service from [resources], and hardened application
instances (see Configuration). It
leaves the network to the host. Both platform networks are ordinary
Docker bridges, so application instances can reach the internet.
To restrict that, use the host’s own controls:
- a security group, network ACL or host firewall denying outbound traffic from the machine, with exceptions for model downloads and object storage;
- private endpoints (AWS VPC endpoints, Azure Private Link) for object storage and the database;
- Docker’s
DOCKER-USERiptables chain, for rules on container traffic alone.
[network] bind decides who reaches the API: 127.0.0.1, the
default, admits this machine only; 0.0.0.0 admits any host on the
network, each trusted by its X-User-Id.
Air gaps
Every shape can run without outbound access once the container images and models are in place: mirror the images into your registry, populate the model caches, and close the egress. Workers make no network calls and load models only from their cache.
Help
Bay Information Systems builds Marigold applications for clients, sets up internal registries and mirrors, deploys the platform on a client’s own servers or cloud account, and works with design partners on what the platform does next: see Services.