Use Open WebUI with the Gateway¶

Open WebUI is a self-hosted AI interface that supports any OpenAI-compatible backend. Connect it to the Gateway to run Claude and other models through your regional endpoint, with optional pgvector support for retrieval-augmented generation (RAG).

Prerequisites¶

• Open WebUI running (installation guide)
• An LLM API key and endpoint from my.amazee.io

Connect the Gateway LLM¶

1. Open Admin Panel → Settings → Connections
2. Under OpenAI API, click + (Add Connection)
3. Fill in:

1. Click Verify Connection, then Save

You should now see your available models (e.g. claude-4-5-sonnet, claude-4-5-haiku) in the model selector.

You can also set the connection with environment variables at startup:

environment:
  - OPENAI_API_BASE_URL=https://llm.[your-region].amazee.ai/v1
  - OPENAI_API_KEY=your-api-key-here

Open WebUI reads these two variables only on first start, then stores them in its database. To change them later, use the Admin Panel.

See Available Models for the full list.

Use the Gateway for embeddings¶

By default, Open WebUI does not send embeddings to the Gateway. It embeds documents locally on the Open WebUI host with a small sentence-transformers model. To keep embeddings in your region, route them through the Gateway.

In the UI, open Admin Panel → Settings → Documents. Set the embedding engine to OpenAI and enter:

Or set these environment variables:

environment:
  - RAG_EMBEDDING_ENGINE=openai
  - RAG_EMBEDDING_MODEL=titan-embed-text-v2:0
  - RAG_OPENAI_API_BASE_URL=https://llm.[your-region].amazee.ai/v1
  - RAG_OPENAI_API_KEY=your-api-key-here

Like the connection variables, these are read only on first start. To change them later, use Admin Panel → Settings → Documents.

If you change the embedding model later, re-upload your documents.

Connect pgvector (optional)¶

The Gateway provides a managed pgvector database for embeddings and RAG workflows. To get pgvector credentials for your workspace, contact ai.support@amazee.io.

Open WebUI configures vector databases via environment variables. Add these at startup:

VECTOR_DB=pgvector
PGVECTOR_DB_URL=postgresql://USERNAME:PASSWORD@vectordb.[your-region].amazee.ai:5432/DATABASE

With Docker Compose:

environment:
  - OPENAI_API_BASE_URL=https://llm.[your-region].amazee.ai/v1
  - OPENAI_API_KEY=your-api-key-here
  - RAG_EMBEDDING_ENGINE=openai
  - RAG_EMBEDDING_MODEL=titan-embed-text-v2:0
  - RAG_OPENAI_API_BASE_URL=https://llm.[your-region].amazee.ai/v1
  - RAG_OPENAI_API_KEY=your-api-key-here
  - VECTOR_DB=pgvector
  - PGVECTOR_DB_URL=postgresql://USERNAME:PASSWORD@vectordb.[your-region].amazee.ai:5432/DATABASE

Restart Open WebUI after changing the vector database settings.

If your database user can't create extensions, set PGVECTOR_CREATE_EXTENSION=false and ask support to enable the vector extension for you.

Verify your setup¶

LLM

Send a test message. You should get a response from the model you selected.

Embeddings

Add a document to a collection in Open WebUI. If you set the embedding engine to OpenAI with your Gateway URL, the request shows up in your usage on my.amazee.io.

pgvector

Enable RAG on a collection and ask a question about a document you've uploaded. A working pgvector connection returns grounded answers.

Troubleshooting¶

No models appear after saving the connection

Double-check the base URL includes /v1. Try https://llm.[your-region].amazee.ai/v1/models in your browser — you should get a JSON model list (you'll need your API key as a Bearer token to authenticate).

Embeddings fail

Confirm the embedding model ID under Admin Panel → Settings → Documents matches an available model. See Available Models.

pgvector connection refused

Verify the host, port (5432), username, password, and database name from your portal. The pgvector instance must be in the same region as your LLM endpoint.