Multi-LLM Providers

Provider Overview

All providers share the same configuration shape. Only provider, model, and (where required) api_key need to change when switching.

Configuration

Switch providers by changing agent.provider in config.yaml. All other agent settings (memory, tools, Telegram) remain unchanged.

agent:
  provider: openai          # change this one value
  model: gpt-4o             # pick a model for that provider
  api_key: ${OPENAI_API_KEY}
  max_tokens: 4096
  temperature: 0.7

Utility Model

Each provider has a utility model — a cheaper, faster model used automatically for memory summarization and compaction. Teleton selects a sensible default per provider. Override it explicitly if needed:

agent:
  provider: anthropic
  model: claude-opus-4-6
  utility_model: claude-haiku-4-5-20251001  # optional override

Tool Limits

Anthropic and Claude Code providers have unlimited tool calls per turn — ideal for complex agentic workflows that call many tools in sequence. All other providers are capped at 128 tool calls per turn, which is sufficient for the vast majority of use cases.

Common Parameters

Anthropic (Claude)

The default provider. Claude models offer unlimited tool calls, making them the best choice for complex multi-step agentic workflows. Key prefix: sk-ant-api03-.

agent:
  provider: anthropic
  model: claude-opus-4-6
  api_key: ${ANTHROPIC_API_KEY}
  max_tokens: 4096

Available models:

Console: console.anthropic.com

Claude Code (Auto)

A special variant of the Anthropic provider that automatically reads OAuth credentials from a local Claude Code installation. No API key required — Teleton reads the token directly from disk and rotates it on expiry or 401 errors.

agent:
  provider: claude-code
  model: claude-opus-4-6
  # api_key is optional — used only as fallback if auto-detection fails

Credential resolution order:

1. Return cached token if still valid
2. Linux / Windows: read ~/.claude/.credentials.json (or $CLAUDE_CONFIG_DIR/.credentials.json)
3. macOS: read from Keychain (service Claude Code-credentials), fall back to credentials file
4. If all else fails, use api_key from config
5. Throw if nothing works — run claude login to authenticate

Same models as anthropic, same unlimited tool limit. OAuth tokens use the prefix sk-ant-oat01-.

Get Claude Code: claude.ai/code

OpenAI

Access GPT-4o, GPT-5, o3, and other OpenAI models. Key prefix: sk-. Tool calls capped at 128 per turn.

agent:
  provider: openai
  model: gpt-4o
  api_key: ${OPENAI_API_KEY}

Available models:

Console: platform.openai.com

Google (Gemini)

Gemini models with up to 1M context window. No key prefix requirement. Tool calls capped at 128 per turn.

agent:
  provider: google
  model: gemini-2.5-flash
  api_key: ${GOOGLE_API_KEY}

Available models:

Console: aistudio.google.com

xAI (Grok)

Grok models from xAI with very large context windows and vision capabilities. Key prefix: xai-.

agent:
  provider: xai
  model: grok-3
  api_key: ${XAI_API_KEY}

Available models:

Console: console.x.ai

Groq

Ultra-fast inference on open-source models via Groq's custom hardware. Key prefix: gsk_. Best for latency-sensitive applications.

agent:
  provider: groq
  model: llama-3.3-70b-versatile
  api_key: ${GROQ_API_KEY}

Available models:

Console: console.groq.com

OpenRouter

Multi-model gateway — access hundreds of models from many providers with a single API key. Model IDs use the format provider/model-name. Key prefix: sk-or-.

agent:
  provider: openrouter
  model: anthropic/claude-opus-4.5
  api_key: ${OPENROUTER_API_KEY}

Available models (selection):

Any model listed on openrouter.ai/models can be used directly as the model value.

Console: openrouter.ai

Moonshot (Kimi K2.5)

Moonshot AI's Kimi K2.5 model accessed via the kimi-coding API at api.kimi.com/coding. Multimodal with a 262K context window. Key prefix: sk-.

agent:
  provider: moonshot
  model: k2p5
  api_key: ${MOONSHOT_API_KEY}

Note: The model ID k2p5 is the config alias for Kimi K2.5. The provider uses a specialized kimi-coding API endpoint internally — do not use generic Moonshot platform model IDs.

Available models:

Console: platform.moonshot.ai

Mistral AI

Mistral models including Devstral (coding-optimized) and Magistral (reasoning). No key prefix requirement.

agent:
  provider: mistral
  model: devstral-small-2507
  api_key: ${MISTRAL_API_KEY}

Available models:

Console: console.mistral.ai

Cerebras

High-speed inference on Cerebras' Wafer-Scale Engine hardware. Key prefix: csk-. Excellent throughput for large models.

agent:
  provider: cerebras
  model: qwen-3-235b-a22b-instruct-2507
  api_key: ${CEREBRAS_API_KEY}

Available models:

Console: cloud.cerebras.ai

ZAI (Zhipu)

GLM models from Zhipu AI (ZAI). Features FREE flash variants with 200K context. No key prefix requirement.

agent:
  provider: zai
  model: glm-4.7
  api_key: ${ZAI_API_KEY}

Available models:

Console: z.ai

MiniMax

MiniMax M2 and M2.5 series models with a 204K context window. No key prefix requirement.

agent:
  provider: minimax
  model: MiniMax-M2.5
  api_key: ${MINIMAX_API_KEY}

Available models:

Console: platform.minimax.io

HuggingFace

Access models hosted on HuggingFace Inference API. Model IDs use the org/model-name format. Token prefix: hf_.

Note: The environment variable is HF_TOKEN, not HUGGINGFACE_API_KEY.

agent:
  provider: huggingface
  model: deepseek-ai/DeepSeek-V3.2
  api_key: ${HF_TOKEN}

Available models:

Console: huggingface.co

Cocoon Network (Decentralized)

Decentralized inference network — no API key required. Payments are made in TON. Requires the cocoon-cli proxy running locally. Teleton connects to it over a local HTTP port.

Architecture note: Cocoon uses XML-based tool injection rather than the standard JSON tool calling API. This is handled transparently inside Teleton.

agent:
  provider: cocoon
  model: Qwen/Qwen3-32B

cocoon:
  port: 10000  # cocoon-cli local proxy port (default: 10000)

Setup:

1. Install and start cocoon-cli from cocoon.network
2. Fund your Cocoon wallet with TON
3. Set agent.provider: cocoon in config — no API key needed
4. Configure cocoon.port if you changed the default proxy port

Available models depend on what Cocoon Network offers at runtime. Qwen/Qwen3-32B is the recommended default.

Website: cocoon.network

Local LLM

Run any OpenAI-compatible local model server: Ollama, vLLM, LM Studio, llama.cpp, or any other server that exposes the /v1/chat/completions endpoint. No API key or internet connection required.

base_url is required for this provider.

agent:
  provider: local
  model: llama3.2          # model tag as known to Ollama
  base_url: http://localhost:11434/v1

agent:
  provider: local
  model: auto              # auto = discovered at runtime
  base_url: http://localhost:1234/v1

agent:
  provider: local
  model: mistralai/Mistral-7B-Instruct-v0.3
  base_url: http://localhost:8000/v1
  api_key: none            # some servers require a placeholder key

Notes:

• Setting model: auto tells Teleton to query the server's /v1/models endpoint and use the first available model.
• Tool calling support depends on the local model and server. Not all local models support function calling.
• For Ollama, pull the model first: ollama pull llama3.2

Model Catalog

Teleton ships with a catalog of 80+ pre-defined model IDs across all 15 providers, used by the WebUI setup wizard and CLI onboard flow. You are not limited to this list — any model ID accepted by the provider's API can be set as agent.model directly in config.yaml.

The catalog is defined in src/config/model-catalog.ts. Each entry includes the model value, display name, and a short description with context window and approximate pricing.

agent:
  provider: openrouter
  model: nvidia/nemotron-nano-9b-v2  # any valid provider model ID works