Models and providers

Providers#

A provider's type is the API it speaks: anthropic, openai (any OpenAI-compatible Chat Completions API), responses (the OpenAI Responses API), gemini (Google's generateContent API) or bedrock (AWS Bedrock Converse, signed with the standard AWS credential chain). For well-known services, preset: fills in the type, base URL and key variable, and a provider named after a preset uses it without saying so:

providers:
  groq: {}                    # same as preset: groq
  fast:
    preset: cerebras
  local:
    preset: ollama            # no key needed
  gemini:
    keys_env: [GEMINI_API_KEY, GEMINI_API_KEY_2]
  aws:
    type: bedrock
    region: us-east-1         # AWS credentials from the usual chain; keys_env may name Bedrock API keys instead

Any field you set wins over the preset. habakkuk models lists each provider with its credential status and, where the API can list them, its models.

Retries, key rotation and fallback#

Errors are classified (rate limit, overload, auth, billing, context overflow, bad request, network). Transient ones retry with jittered backoff; rate limits and auth failures rotate to the next key in keys_env, with a cooldown per key; and errors another model can fix fall back down the agent's fallbacks. See Architecture: models.

Subscription logins#

A provider can use an account sign-in instead of keys:

habakkuk auth login chatgpt     # ChatGPT plan: device code, then the Responses API the plan uses
habakkuk auth login copilot     # GitHub Copilot plan: GitHub device code, then Copilot's OpenAI-compatible API
habakkuk auth list
habakkuk auth logout chatgpt

providers:
  chatgpt:
    auth: chatgpt             # type responses is implied
  copilot:
    auth: copilot             # type openai is implied

Tokens are kept in $HABAKKUK_HOME/auth.json (owner-only, replaced atomically) and refreshed shortly before they expire, under a file lock so two Habakkuk processes never refresh at once. There is no Claude subscription login: Anthropic's terms do not allow third-party apps to use Claude plan credentials, so use an API key.

Switching models#

In a chat, /model shows the models the conversation uses and /model provider/model switches this conversation only (the agent's own models stay as fallbacks); /model default switches back. The choice is stored per session.

Usage and cost#

Every model call's tokens (input, output, cache read and write) are recorded with its provider, model, agent and session in usage.db. habakkuk usage [-days 30] [-agent A] [-session S] prints totals and cost by model and by agent; /usage in a chat shows the current conversation. Costs come from a built-in table of list prices (standard tier, US dollars per million tokens), with long-prompt tiers where providers have them; calls through a subscription login count as $0. Add or correct prices in config:

pricing:
  llama-4-scout: {input: 0.11, output: 0.34}
  openrouter/anthropic/claude-sonnet-5-5: {input: 2, output: 10, cache_read: 0.1, cache_write: 2.5}

Spending limits#

Hard caps on what model calls cost, so a schedule or a runaway tool loop cannot run up a bill overnight. Set them for the whole install and per agent, in US dollars (numbers or strings: 5, "5.00"):

timezone: America/Toronto   # days start at 00:00 here, months on the 1st

budget:                     # the whole install
  daily: 5
  monthly: 100
  per_turn: 1.50            # one turn: its tool loop, compaction, title and sub-agents
  unknown_price: allow      # allow | block (see below)

agents:
  habakkuk:
    budget: {daily: 2, per_turn: 0.50}   # this agent's own calls

• Every model call is checked first: turns, sub-agents, compaction, automatic titles, the approval reviewer, memory review and meeting notes all go through the same router, which asks the limits before sending. Once a limit is reached the call is not sent. A call already under way is not cut off, so spend can pass a limit by what the calls running at that moment cost, and then stops.
• Spend is kept in memory, loaded from usage.db at start and updated as each call finishes, so the check costs nothing measurable. Prices are the ones habakkuk usage uses.
• A capped turn ends cleanly with a reply saying which limit is used up, when it resets and where to raise it, for example: Paused: the daily budget of $5.00 for agent habakkuk is used up; it resets at 00:00 America/Toronto. Raise it with agents.habakkuk.budget.daily in config.yaml. The job finishes; it is not retried. A per-turn limit stops only that turn; the next message starts a fresh allowance.
• Background work tells you once. A scheduled task whose agent is out of budget is skipped without starting, and its chat gets one notice per limit and period; later skips are only logged. A board card is blocked with the reason. Memory review is skipped and tried again later.
• Who pays. A call counts toward the agent that made it and toward the install. A sub-agent's calls count toward the sub-agent's daily and monthly limits, and toward the per-turn limit of the turn that delegated to it.
• Warning at 80%. When a daily or monthly limit passes 80%, the chat whose call crossed it gets one warning per period (the log, when there is no chat).
• Unknown prices. A model missing from the price table counts as $0, with one warning per model in the log; add it under pricing:. With unknown_price: block, calls to such models are refused while any limit applies (the next model in the chain is tried).
• Changing limits in config.yaml applies within seconds, without a restart.

See spend against the limits with habakkuk usage (today and this month for the install and each agent), /budget in a chat (this conversation's agent and the install), or GET /api/budget (API).

Speech-to-text#

Voice notes from chats and habakkuk talk are transcribed through a provider of type openai with an /audio/transcriptions endpoint:

transcription:
  provider: openai          # a provider with type: openai
  model: gpt-4o-mini-transcribe   # required
  prompt: "Names and words to spell right: Habakkuk, Toronto"   # optional