OpenAI-compatible inference, done properly.

One-line swap

Change the base URL. Keep the code.

The model parameter maps to a Marigold model name. The rest of the request is unchanged.

Before

from openai import OpenAI

client = OpenAI(api_key="sk-...")

response = client.chat.completions.create(
    model="gpt-4.1-mini",
    messages=[{"role": "user", "content": "Summarise this document: ..."}]
)

After

from openai import OpenAI

client = OpenAI(
    base_url="https://api.marigold.run/v1",
    api_key="your-api-key"
)

response = client.chat.completions.create(
    model="qwen/qwen2.5-7b-instruct",
    messages=[{"role": "user", "content": "Summarise this document: ..."}]
)

Raw HTTP

curl https://api.marigold.run/v1/chat/completions \
  -H "Authorization: Bearer your-api-key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "qwen/qwen2.5-7b-instruct",
    "messages": [{"role": "user", "content": "Hello"}]
  }'

Native async API

Beyond chat completions

The native API covers the full model registry: embeddings, image-to-text, text-to-speech, depth estimation, segmentation, and eval. Submit a job, poll for the result, retrieve binary outputs from storage.

Submit and poll

import requests, time

API = "https://api.marigold.run"
KEY = {"Authorization": "Bearer your-api-key"}

job = requests.post(f"{API}/infer", headers=KEY, json={
    "model_type": "text-embedding",
    "model_name": "sentence-transformers/all-MiniLM-L6-v2",
    "input": "The quick brown fox"
}).json()

while True:
    result = requests.get(f"{API}/infer/{job['job_id']}", headers=KEY).json()
    if result["status"] == "complete":
        print(result["output"]["embedding"])
        break
    time.sleep(0.5)

IDE and editor support

Works wherever a custom OpenAI base URL is accepted

Cursor, Continue, Cline, and Aider all support a custom OpenAI-compatible endpoint. Set the base URL to https://api.marigold.run/v1 and your Marigold API key. Full configuration instructions are on the IDE setup page.

IDE setup guide →