Skip to content

Use CrewAI with the Gateway

CrewAI is a framework for building multi-agent AI systems. Connect it to the Gateway to run your crews through your regional endpoint with full data residency and budget controls.

Prerequisites

  • CrewAI installed (pip install crewai or uv add crewai)
  • An LLM API key

Connect CrewAI to the Gateway

Gateway model IDs, like claude-4-5-sonnet, aren't on CrewAI's list of OpenAI models. You have to tell CrewAI you're using a custom OpenAI-compatible endpoint. If you don't, it tries to fall back to LiteLLM and fails with Unable to initialize LLM ... LiteLLM fallback package is not installed.

Replace [your-region] with your region code (e.g. ch103, us103).

from crewai import LLM

llm = LLM(
    model="openai/claude-4-5-sonnet",
    api_key="your-amazee-ai-api-key",
    base_url="https://llm.[your-region].amazee.ai/v1",
)

Passing base_url makes CrewAI use its built-in OpenAI client against the Gateway.

Option 2 — Environment variables

Set these before running your crew:

export OPENAI_API_KEY="your-amazee-ai-api-key"
export OPENAI_API_BASE="https://llm.[your-region].amazee.ai/v1"
export MODEL="openai/claude-4-5-sonnet"

Leave llm off your agents and CrewAI picks up MODEL and OPENAI_API_BASE on its own. If you build an LLM object yourself, add custom_openai=True:

from crewai import LLM

llm = LLM(model="openai/claude-4-5-sonnet", custom_openai=True)

Option 3 — YAML projects (crewai create crew --classic)

Newer CrewAI versions create a JSON project by default. Use --classic to get config/agents.yaml. If there's no .env in the project, create one.

Put the three variables from Option 2 in your project's .env. In config/agents.yaml, leave llm out. Each agent then uses MODEL and OPENAI_API_BASE from .env.

researcher:
  role: Researcher
  goal: ...
  backstory: ...

Don't set llm: openai/claude-4-5-sonnet in agents.yaml. CrewAI builds that without your base URL and fails with LiteLLM fallback package is not installed. A nested provider:/config: block fails too (TypeError: unhashable type: 'dict'). To give one agent a different model, set it in crew.py: Agent(config=..., llm=LLM(model="openai/claude-4-5-haiku", custom_openai=True)). Keep your API key in .env, not in YAML.

Verify

Run your crew and check the My usage card on the AI keys page to confirm requests are flowing through your regional endpoint.

Troubleshooting

Unable to initialize LLM ... LiteLLM fallback package is not installed
CrewAI didn't know you were using a custom endpoint. Pass base_url= (Option 1), or add custom_openai=True (Option 2). Installing crewai[litellm] also works, but you don't need it.
Auth errors
Make sure api_key is passed to the LLM constructor, or that OPENAI_API_KEY is set to your Gateway key.
Model not found or 403 team_model_access_denied
The model ID doesn't exist or isn't enabled for your key. Copy an exact ID from GET /v1/models and add the openai/ prefix (e.g. openai/claude-4-5-sonnet).