Configuration & providers

Everything in Loom is configured by flags + environment variables (with an optional .env or --config YAML). Loom is built as four swappable ports — search, MLOps, model-builder, and model — and each is selected the same way: a flag, an env var, and a sensible default. Secrets are read only from the environment, never passed on the command line.

How configuration works

A single LoomConfig is the source of truth for provider selection, model routing, and budget. It is resolved with a clear precedence — later layers override earlier ones:

• dataclass defaults (what you get with nothing set);
• an optional YAML file (--config path.yaml);
• the environment / a local .env;
• explicit flag overrides on the command line.

So a flag always wins over an env var, which wins over YAML, which wins over the built-in default. Every verb also self-documents its flags:

loom <verb> --help        # lists every flag for that verb

Environment variables & flags

The knobs you will reach for most. Each pairs an env var with its flag (where one exists), the default, and what it controls:

Endpoints & the model knobs

Two providers point at OpenAI-compatible endpoints, configured by env var rather than a flag:

• OPENAI_BASE_URL — the endpoint the nim provider uses (defaults to integrate.api.nvidia.com);
• LOOM_MODEL_BASE_URL — the endpoint the openai-compat provider uses (LiteLLM / vLLM / Ollama, etc.).

The budget is more than steps: it carries steps, draft count, debug probability, and max debug depth — but --steps / LOOM_BUDGET_STEPS is the one you set day to day. Start small and raise it once a run works.

Swappable ports

Loom is ports & adapters: every verb runs through an interface, never a concrete backend, so any layer is drop-in replaceable. There are four seams, each with a default adapter and a one-line way to swap it.

Search — the brain (AIDE)

The search provider runs the ML-iteration loop behind loom run / /loom-optimize: it proposes candidate solutions, runs each one for real through the MLOps port, scores the result against your metric, and returns the best. The default adapter is AIDE tree-search (pinned by commit for reproducibility). Swap it with --search.

MLOps — the muscle (Metaflow)

The execution provider runs the work and owns the datastore. There are two built-in adapters, and the choice changes what's reachable:

The local path is the on-ramp — no Metaflow, no GPU, no cloud. The lifecycle verbs need --mlops metaflow because they run flows, not in-process candidates. Where artifacts physically live (local or S3/minio) is an implementation detail Metaflow owns, configured through METAFLOW_PROFILE / METAFLOW_* — see the data model.

Model-builder — the training engine (NeMo)

The model-builder is the heavy training/serving backend behind loom train. You state intent in data-science vocabulary — --objective {next-event|masked-field|contrastive}, --budget {probe|small|full}, --backbone, --metric — and the adapter compiles that to backend config; no .nemo/Megatron nouns ever surface. The default is NeMo AutoModel; the seam is swappable, with DeepSpeed, PyTorch FSDP, Megatron, and HF Accelerate as planned peers (they compare with each other as distributed-training engines — not with Metaflow, which orchestrates the GPU step that runs whichever one you pick).

Select the builder with LOOM_MODEL_BUILDER_PROVIDER. The torch-free CPU local stand-in actually builds a backbone end-to-end (a real PPMI + SVD embedding model) — no GPU, sub-2s, deterministic — which makes it ideal for dev and CI:

# No GPU target set: refuses cleanly, shows the cost plan, never launches.
loom train --dataset IngestDataset/123 --objective next-event --budget full
#   STATUS : REFUSED_NO_GPU_TARGET  (set LOOM_GPU_TARGET, or use the CPU stand-in)

# The CPU stand-in builds a backbone for real — dev + CI.
LOOM_MODEL_BUILDER_PROVIDER=local \
  loom train --dataset IngestDataset/123 --objective next-event --budget probe
#   STATUS : BUILT  (-> a backbone pathspec + @card)

Model — the LLM backend

The model provider decides which model the brain talks to and how it authenticates. You pick one per role — --code-provider (writes solutions) and --feedback-provider (the judge that scores) — or set both at once with --model-provider. Each adapter configures AIDE's existing LLM backends via config/env; Loom never forks AIDE and never handles your secrets. The full table is below.

Model providers

The eight built-in model providers, with how each authenticates and whether it can serve the judge role. The judge always uses tool/function calling, so a feedback route must be tool-capable.

Picking and mixing providers

Set the same provider for both roles, or split them — a cheap local model can write code while a stronger model judges:

# Native OpenAI for both roles
export OPENAI_API_KEY="sk-..."; export LOOM_CODE_MODEL="gpt-4o"; export LOOM_FEEDBACK_MODEL="gpt-4o"
loom run --data ./task --goal "..." --metric "..." --model-provider openai-api

# NVIDIA NIM — a real NeMo touchpoint, zero GPU infra
export NVIDIA_API_KEY="nvapi-..."
loom run --data ./task --goal "..." --metric "..." --model-provider nim

# Mix roles: a local model writes code, Claude judges
export OPENAI_BASE_URL="http://localhost:11434/v1"   # Ollama, say
loom run --data ./task --goal "..." --metric "..." \
  --code-provider openai-compat --feedback-provider anthropic-api

OpenRouter, NIM & OpenAI-compatible routing

AIDE routes purely by model name, which creates two sharp edges the providers route around for you:

• OpenRouter: AIDE's dedicated OpenRouter backend can't do tool calling, and the judge always passes a function spec — so the openrouter provider forces the OpenAI-compatible path (sets OPENAI_BASE_URL, copies the key into OPENAI_API_KEY) and best-effort checks the feedback slug's tool support.
• Native OpenAI model names make the OpenAI backend hardcode api.openai.com and ignore OPENAI_BASE_URL — so openai-api deliberately leaves it unset, while nim and openai-compat use a non-OpenAI model name plus the base URL to reach a different endpoint.

Subscription bridges

The two *-subscription providers are CLI bridges: Loom drives the claude / codex CLI you already installed and logged in, never seeing your credentials. Metering comes from a separate subscription pool — a heavy AIDE run is many calls and can exhaust it, so for sustained runs an API key is the better fit. loom run pre-flights both (a missing binary or login yields an actionable hint, not a traceback).

The Loom gateway (loom-proxy)

The loom-proxy provider routes through the Loom gateway — a thin Anthropic-passthrough that holds the real vendor key server-side, so callers present only a LOOM_API_KEY. Run it as a server and point clients at it:

# Server side: the gateway holds the REAL vendor key; callers present only a LOOM_API_KEY.
export ANTHROPIC_API_KEY="sk-ant-..."     # stays on the server
export LOOM_API_KEY="loom-..."            # the key callers must present (allowlist)
loom proxy serve --host 127.0.0.1 --port 8088

# Client side (another shell, only the LOOM key):
export LOOM_API_KEY="loom-..."
loom run --data ./task --goal "..." --metric "..." --model-provider loom-proxy

Two env vars configure the client side: LOOM_API_BASE (default http://127.0.0.1:8088) and LOOM_PROXY_LOG_PATH (default learnings/proxy_calls.jsonl). When a hosted gateway is detected — LOOM_API_BASE set to a non-loopback URL, or LOOM_PROXY_DEFAULT truthy — and you haven't explicitly chosen a provider, Loom defaults both roles to loom-proxy. Precedence: an explicit --code/feedback/model-provider (or LOOM_CODE/FEEDBACK_PROVIDER) always wins; else a hosted gateway ⇒ loom-proxy; else anthropic-api. A loopback base is local dev, so it never flips the default.

Keyless by default

Most of Loom needs no model key at all. The read-only and lifecycle verbs work without one:

• read-only datasets, eda, viz, report, ops, validate, doctor;
• workspace-write features, ingest, train (the local stand-in), telemetry.

Only the LLM verbs — run / pipeline, which drive the AIDE search brain — and the natural-language planning need a model key. A missing key yields an actionable line (set ANTHROPIC_API_KEY … or pick a --model-provider), never a crash. The default model is Claude, so the quickest path is an ANTHROPIC_API_KEY; inside the agent you can also run /login.

Adding a provider

Because every verb speaks an interface and the controller resolves providers by name from a registry, adding a backend is one adapter class plus a one-line registration — never a core change. Each new provider is constructed uniformly as Provider(config), reads endpoints/profiles from the config, reads secrets from the environment at the point of use, and lazy-imports any heavy dependency so the core import stays light. Once registered, you select it exactly like a built-in:

loom run --search my-brain              # LOOM_SEARCH_PROVIDER=my-brain
loom run --mlops my-runtime            # LOOM_MLOPS_PROVIDER=my-runtime
loom run --model-provider my-llm        # LOOM_CODE_PROVIDER / LOOM_FEEDBACK_PROVIDER
LOOM_MODEL_BUILDER_PROVIDER=my-trainer loom train ...

A new model-builder must additionally pass the golden conformance suite (the tokenize → pretrain → embed → finetune → evaluate round-trip, valid artifact pathspecs, manifest honesty). The full provider model and step-by-step is in docs/architecture.md in the repository.