Bring Your Own LLM

Why Use a Custom LLM?

Benefit	Description
Customisation	Use domain-specific or fine-tuned models for better accuracy in your vertical
Data Privacy	Keep sensitive data within your own infrastructure or private cloud
Compliance	Meet strict regulatory requirements around data residency and auditing
Cost Control	Optimise expenses by choosing competitive providers or using your own hardware

Requirements

Your LLM API must implement the OpenAI Chat Completions specification:

Endpoint: POST /chat/completions

Payload: Must accept a messages: [] array

Streaming: Must support Server-Sent Events (SSE) for real-time interaction

Optional — required only for specific features:

Function Calling — needed if your agent uses tools

Vision — needed for multimodal inputs

Setup Options

Via Developer Studio

Navigate to Create New Agent.

Find the Agent Config section.

Select Bring your own LLM from the Provider dropdown.

Enter your API URL, Model Name, and API Token.

Bring Your Own LLM configuration in Developer Studio — Developer Studio — LLM provider configuration

Via API

Set provider to "custom" within the avatars.config.llm object:

curl --request POST \
  --url https://api.trugen.ai/v1/ext/agent \
  --header 'Content-Type: application/json' \
  --header 'x-api-key: YOUR_API_KEY' \
  --data @- <<EOF
{
  "agent_name": "Custom LLM Agent",
  "config": { "timeout": 240 },
  "avatars": [
    {
      "config": {
        "llm": {
          "provider": "custom",
          "model": "MODEL_NAME",
          "url": "API_BASE_URL",
          "token": "YOUR_EXTERNAL_TOKEN"
        },
        "stt": { "provider": "deepgram", "model": "nova-3" },
        "tts": { "provider": "elevenlabs", "voice_id": "ZUrEGyu8GFMwnHbvLhv2" }
      },
      "persona_name": "Custom LLM Agent",
      "persona_prompt": "You are a helpful AI agent."
    }
  ]
}
EOF

Browse all built-in models across Groq, Azure OpenAI, Google, and more.

Introduction

AI Teammates

Why Use a Custom LLM?

Requirements

Setup Options

Via Developer Studio

Via API

What’s Next?

Available LLMs

Prompting Strategies

​Why Use a Custom LLM?

​Requirements

​Setup Options

​Via Developer Studio

​Via API

​What’s Next?

Available LLMs

Prompting Strategies

Why Use a Custom LLM?

Requirements

Setup Options

Via Developer Studio

Via API

What’s Next?