LLM Provider Configuration

Sovereign Workflows uses AI models for workflow steps like text summarization, classification, structured extraction, and autonomous agent reasoning. Before any of those features work, you need to tell the platform which AI providers to call and how to authenticate with them.

License Required

AI features require a license with LLM capabilities enabled. Check your license details in the Licensing section to confirm availability.

Supported Providers

The platform ships with built-in support for five LLM providers. Each has a dedicated client implementation that handles authentication and API format differences automatically.

Provider	Provider ID	Authentication	Use Case
OpenAI	`openai`	Bearer token	GPT-4o, GPT-4o mini, and other OpenAI models
Google Gemini	`gemini`	Bearer token	Gemini 2.5 Pro, Gemini 2.5 Flash, and other Gemini models
Anthropic	`anthropic`	`x-api-key` header	Claude 4, Claude 3.5 Sonnet, and other Claude models
Azure OpenAI	`azure-openai`	`api-key` header	GPT models deployed through your Azure OpenAI resource
Ollama	(any local ID)	Optional	Llama, Mistral, and other open-weight models for air-gapped deployments

Tool Calling

OpenAI, Gemini, Azure OpenAI, and Anthropic all support tool calling (function calling). This is required for autonomous agent nodes and the MCP Server. If you are using agent features, make sure the models you configure have the Tools capability.

For detailed setup instructions for each provider, see Provider Setup.

The Configuration File

LLM configuration lives in a single JSON file mounted into containers at startup. The default path inside the container is:

/etc/workflows/llm-config.json

You can override this path by setting the Llm:ConfigFilePath configuration key (via environment variable or appsettings).

The file has three main sections: Providers (which AI services to connect to), Profiles (named presets that map to a provider and model), and optional Context/RAG settings.

Loading Priority

Config file — if a file exists at the configured path, it is loaded and used
Environment variables / appsettings — if no file is found, the loader checks the Llm:Providers configuration section
Built-in defaults — if neither source provides configuration, an embedded default is used

Mount Your Config File

The built-in defaults exist only so the service starts without crashing. For production use, always mount your own llm-config.json. The default configuration will not have valid API keys.

Full Configuration Example

This example configures OpenAI and Google Gemini with multiple profiles:

{
  "Providers": [
    {
      "ProviderId": "openai",
      "BaseUrl": "https://api.openai.com/v1",
      "ApiKeySecretName": "OPENAI_API_KEY",
      "IsLocal": false,
      "Models": [
        {
          "ModelId": "gpt-4o",
          "MaxInputTokens": 128000,
          "MaxOutputTokens": 16384,
          "Capabilities": "Chat,Tools"
        },
        {
          "ModelId": "gpt-4o-mini",
          "MaxInputTokens": 128000,
          "MaxOutputTokens": 16384,
          "Capabilities": "Chat,Tools"
        }
      ]
    },
    {
      "ProviderId": "gemini",
      "BaseUrl": "https://generativelanguage.googleapis.com/v1beta/openai",
      "ApiKeySecretName": "GEMINI_API_KEY",
      "IsLocal": false,
      "Metadata": {
        "ChatCompletionsPath": "/chat/completions"
      },
      "Models": [
        {
          "ModelId": "gemini-2.5-pro",
          "MaxInputTokens": 1048576,
          "MaxOutputTokens": 65536,
          "Capabilities": "Chat,Tools"
        }
      ]
    }
  ],

  "Profiles": [
    {
      "ProfileId": "default",
      "PreferredProviderId": "gemini",
      "PreferredModelId": "gemini-2.5-pro",
      "Temperature": 0.7,
      "MaxOutputTokens": 4096,
      "ToolsEnabled": false
    },
    {
      "ProfileId": "agentic",
      "PreferredProviderId": "openai",
      "PreferredModelId": "gpt-4o",
      "Temperature": 0.3,
      "MaxOutputTokens": 8192,
      "ToolsEnabled": true
    }
  ]
}

Provider Fields

Field	Required	Description
`ProviderId`	Yes	Unique identifier (e.g., `openai`, `gemini`, `anthropic`, `azure-openai`)
`BaseUrl`	Yes	The API base URL for this provider
`ApiKeySecretName`	Yes	Environment variable name that holds the API key — the key itself never goes in this file
`IsLocal`	No	Set to `true` for self-hosted models (Ollama). Used by routing strategies. Defaults to `false`
`Metadata`	No	Provider-specific settings (Gemini: `ChatCompletionsPath`, Azure: `ApiVersion` + `DeploymentId`)
`Models`	Yes	Array of models available from this provider

Model Fields

Field	Required	Description
`ModelId`	Yes	Model identifier sent to the provider API (e.g., `gpt-4o`, `gemini-2.5-pro`)
`MaxInputTokens`	Yes	Maximum prompt tokens the model accepts
`MaxOutputTokens`	Yes	Maximum completion tokens the model can produce
`Capabilities`	Yes	Comma-separated: `Chat`, `Tools`, `JsonOutput`, `Streaming`. Agent features require `Tools`

API Key Setup

API keys are always provided through environment variables. The ApiKeySecretName field tells the platform which variable to read — the actual key never appears in the config file.

Variable	Provider	How to Obtain
`OPENAI_API_KEY`	OpenAI	platform.openai.com/api-keys
`GEMINI_API_KEY`	Google Gemini	aistudio.google.com/apikey
`ANTHROPIC_API_KEY`	Anthropic	console.anthropic.com/settings/keys
`AZURE_OPENAI_API_KEY`	Azure OpenAI	Azure Portal, under your OpenAI resource's Keys section
`OLLAMA_API_KEY`	Ollama (local)	Usually not required — set to any non-empty value if your instance requires auth

Never Hardcode Keys

API keys should exist only in environment variables or a secrets manager. Never put them in config files, docker-compose files checked into source control, or container images.

Keys can be supplied through any mechanism supported by .NET configuration: environment variables, Docker secrets mounted as files, or a secrets manager that injects values into the environment.

Docker Configuration

Two services consume the LLM configuration: the LLM Connector (executes LLM workflow steps) and the MCP Server (handles agent chat sessions). Both need the config file mounted and the API key environment variables set.

Add to your .env file:

# Required: at least one provider key
GEMINI_API_KEY=your-gemini-key

# Optional: additional providers
OPENAI_API_KEY=sk-your-openai-key
ANTHROPIC_API_KEY=sk-ant-your-anthropic-key
AZURE_OPENAI_API_KEY=your-azure-key

Which Services Need Keys?

Only the LLM Connector and MCP Server need LLM API keys. The Engine API, Executor Worker, and other services do not call LLM providers directly.

Next Steps

Provider Setup — detailed configuration for each supported provider
Profiles & Routing — balance cost and quality with profiles, configure per-node overrides
AI-Powered Workflows — learn how to add AI steps to your workflows