Build agents

Model providers

A provider registration opens a model route in the host. An agent definition selects that route by name. The definition never contains the credential.

Mental model: register once, select per agent

flowchart LR
    accTitle: Model provider selection flow
    accDescr: Host configuration registers a named provider, each agent binds that provider and model, and the resolved chat client performs the request.
    C["Host configuration<br/>credential and endpoint"] --> R["Use... registration"]
    R --> N["Provider registry<br/>stable provider name"]
    D["Agent definition<br/>ModelBinding"] --> N
    N --> P["IChatClient pipeline"]
    P --> M["Selected provider and model"]

This separation has two useful effects. You can register several providers in one process. You can also move an agent to another provider without moving a secret into the database or the console.

Package	Registration	Provider name in `ModelBinding`	Surface
`Tracon.OpenAI`	`UseOpenAI()`	`openai`	OpenAI Chat Completions
`Tracon.OpenAI`	`UseOpenAI()`	`openai-responses`	OpenAI Responses
`Tracon.OpenAI`	`UseOpenAICompatible(name, …)`	your `name`	Compatible Chat Completions
`Tracon.Anthropic`	`UseAnthropic()`	`anthropic`	Anthropic Messages
`Tracon.Google`	`UseGoogle()`	`google`	Gemini Developer API
`Tracon.Azure`	`UseAzureOpenAI()`	`azure-openai`	Azure OpenAI Chat Completions

The Tracon meta package includes Tracon.OpenAI. Add the Anthropic, Google, or Azure package only when the host uses it.

Register providers

Read credentials from configuration. Put their values in user-secrets, environment variables, or a secret manager.

dotnet user-secrets set "Tracon:Providers:OpenAI:ApiKey" "<key>"
dotnet user-secrets set "Tracon:Providers:Anthropic:ApiKey" "<key>"
dotnet user-secrets set "Tracon:Providers:Google:ApiKey" "<key>"
dotnet user-secrets set "Tracon:Providers:AzureOpenAI:ApiKey" "<key>"

using Tracon;

var tracon = builder.AddTracon()
    .UseOpenAI(builder.Configuration.GetSection(OpenAIProviderOptions.SectionName))
    .UseAnthropic(builder.Configuration.GetSection(AnthropicProviderOptions.SectionName))
    .UseGoogle(builder.Configuration.GetSection(GoogleProviderOptions.SectionName))
    .UseAzureOpenAI(builder.Configuration.GetSection(AzureOpenAIProviderOptions.SectionName));

Each options type validates at startup. Missing required configuration stops the host before the first run. A named compatible endpoint can be keyless, and Azure can use a credential factory instead of an API key.

Live voice providers

OpenAI can also register an ILiveVoiceProvider, which is a different kind of registration again: it opens an outbound connection billed by the second, so it is a separate call rather than a flag on UseOpenAI().

tracon
    .UseOpenAI(builder.Configuration.GetSection(OpenAIProviderOptions.SectionName))
    .UseOpenAILive(builder.Configuration.GetSection(OpenAILiveOptions.SectionName))
    .UseLiveVoice();

The provider runs the whole spoken conversation and carries the audio straight to the browser; your agents handle the work it delegates. The API key stays on the server. See provider-hosted live voice for what that trade gives up.

Image generation providers

OpenAI, Azure OpenAI, and Google can also register an IImageGenerator. These are separate from chat-model registrations because an image model or Azure deployment is not safely inferred from an agent’s chat model.

tracon
    .UseOpenAIImages(options =>
    {
        options.Enabled = true;
        options.Model = "gpt-image-1";
    });

// Azure uses an image deployment name.
// tracon.UseAzureOpenAIImages(options => options.Model = "image-deployment");

// Google uses a provider image model. It does not support WIDTHxHEIGHT in this surface.
// tracon.UseGoogleImages(options => options.Model = "your-imagen-model");

Each extension shares the authenticated client factory already created by UseOpenAI, UseAzureOpenAI, or UseGoogle; it adds no second credential path or package. The extensions register generators by provider name. When more than one is registered, set Tracon:Images:Provider to openai, azure-openai, or google to select the generator and matching price table. An unkeyed IImageGenerator that your application registers remains the fallback for a custom provider.

UseOpenAICompatible() does not imply image support. Register a supported image extension only after you verify that its provider API supports the selected model.

Then bind an agent to one stable provider name:

tracon.AddAgent(new AgentDefinition
{
    Name = "support",
    Instructions = "Resolve support requests. State uncertainty clearly.",
    Model = new ModelBinding
    {
        Provider = AnthropicProviderNames.Anthropic,
        Model = "your-current-model-name",
        MaxOutputTokens = 2_048,
    },
});

Use a current model name from the provider. Tracon does not pin one for you.

Provider registration is host configuration, not a console operation. The console never accepts or displays provider secrets. Its Models screen shows the catalog and cached health for providers that the host already registered. The agent editor can select a provider and model, but it does not currently edit ProviderSettings; set vendor-specific keys in code or through the management HTTP API.

OpenAI and compatible endpoints

UseOpenAI() always registers both official OpenAI routes. The OpenAIProviderOptions.EnableResponsesSurface option does not change this behavior. That option applies only to compatible endpoints.

tracon.UseOpenAICompatible("ollama", options =>
{
    options.Endpoint = new Uri("http://localhost:11434/v1");
    // A local server can run without an API key.
});

A compatible registration creates only the Chat Completions route by default. Set EnableResponsesSurface = true only if the server implements /v1/responses. The second provider is then named {name}-responses.

The name must match [a-z0-9][a-z0-9-]{0,31}. The names openai and openai-responses are reserved. An absolute Endpoint is required. A compatible server with no key is valid; Tracon supplies only the fixed placeholder required by the OpenAI client library.

Provider-specific settings

Portable settings live directly on ModelBinding: Temperature, TopP, MaxOutputTokens, ReasoningEffort, and ResponseFormat. Vendor-only settings live in ProviderSettings. Unknown keys fail compilation instead of being ignored.

using System.Text.Json;

var anthropicBinding = new ModelBinding
{
    Provider = AnthropicProviderNames.Anthropic,
    Model = "your-current-claude-model",
    MaxOutputTokens = 4_096,
    ProviderSettings = new Dictionary<string, JsonElement>(
        StringComparer.OrdinalIgnoreCase)
    {
        [AnthropicProviderNames.PromptCachingSetting] =
            JsonSerializer.SerializeToElement(true),
        [AnthropicProviderNames.ThinkingBudgetTokensSetting] =
            JsonSerializer.SerializeToElement(2_048),
    },
};

var googleBinding = new ModelBinding
{
    Provider = GoogleProviderNames.Google,
    Model = "your-current-gemini-model",
    ProviderSettings = new Dictionary<string, JsonElement>(
        StringComparer.OrdinalIgnoreCase)
    {
        [GoogleProviderNames.SafetyHarassmentSetting] =
            JsonSerializer.SerializeToElement("BLOCK_ONLY_HIGH"),
        [GoogleProviderNames.ThinkingBudgetTokensSetting] =
            JsonSerializer.SerializeToElement(512),
        [GoogleProviderNames.ThinkingIncludeThoughtsSetting] =
            JsonSerializer.SerializeToElement(true),
    },
};

Anthropic supports anthropic.promptCaching and anthropic.thinking.budgetTokens. Prompt caching is off by default. When thinking is enabled, its budget must be smaller than the effective output limit. Temperature must be absent or 1.

Google supports five google.safety.* thresholds plus google.thinking.budgetTokens and google.thinking.includeThoughts. The thinking budget range is -1..65535; -1 lets the model decide and 0 disables thinking. Valid safety values are BLOCK_LOW_AND_ABOVE, BLOCK_MEDIUM_AND_ABOVE, BLOCK_ONLY_HIGH, BLOCK_NONE, and OFF. A safety-filtered empty response becomes a failed run with error type content_filtered.

Azure OpenAI supports no ProviderSettings keys. The package deliberately rejects them.

Azure OpenAI uses deployments

For azure-openai, ModelBinding.Model is the deployment name, not the base model name. A wrong deployment usually produces 404 even when provider health is good.

Use an API key, or supply an Azure Core credential factory. Managed identity wins if both are present.

// Add Azure.Identity to the consumer project for DefaultAzureCredential.
using Azure.Identity;

tracon.UseAzureOpenAI(options =>
{
    options.Endpoint = new Uri("https://my-resource.openai.azure.com/");
    options.CredentialFactory = static () => new DefaultAzureCredential();
    options.DefaultDeployment = "support-production";
});

Tracon.Azure depends on Azure.Core, not Azure.Identity. The consumer chooses the credential implementation. Azure OpenAI Responses, On Your Data, and Azure AI Foundry Agents are not exposed by this provider.

Per-tenant credentials (BYOK)

By default every tenant shares the credential a Use...() call registered at startup. A multi-tenant host can instead let each tenant bring its own key — its usage and its bill stay separate from every other tenant’s.

BYOK is an optional provider capability, not a universal contract. A provider opts in by implementing ITenantCredentialModelProvider in addition to IModelProvider; the four built-in providers (OpenAI, Anthropic, Google, Azure OpenAI) all do. A provider that does not implement it simply never sees a tenant credential, and Tracon never falls back to the setup-time key on its behalf: a tenant binding saved against a provider that does not support BYOK fails every run with a stable provider_credential_unsupported error instead of silently billing the tenant’s traffic to the host’s own account.

A tenant’s binding stores only the name of a configuration key, never the value:

dotnet user-secrets set "Tracon:ProviderKeys:Acme:OpenAI" "<acme's key>"

curl -X PUT "http://localhost:5081/tracon/api/tenants/acme/providers/openai" \
  -H "Authorization: Bearer $TRACON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"apiKeyConfigurationName": "Tracon:ProviderKeys:Acme:OpenAI"}'

The name must sit under the configured prefix (default Tracon:ProviderKeys:); a name outside it is rejected with 400 both when it is saved and again when it is resolved. GET /api/tenants/acme/providers reports whether the name currently resolves to a value (resolved: true/false) — never the value itself. A tenant with no binding for a provider keeps using the global setup-time credential; nothing changes until a binding is written. A binding that exists but resolves to no value does not fall back to the global credential silently — the run fails with a clear error instead, so a misconfigured tenant is never billed to the wrong account.

Restrict which providers a tenant’s agents may call with an egress policy:

curl -X PUT "http://localhost:5081/tracon/api/tenants/acme/egress" \
  -H "Authorization: Bearer $TRACON_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"allowedProviders": ["openai", "anthropic"]}'

A tenant with no saved policy is unrestricted — saving one is an additive restriction, not a default wall. An agent definition naming a provider outside the saved list is rejected at compile time, before any request reaches the network; the same check also protects PUT .../providers/{provider} itself, so both surfaces agree. The console’s Settings screen exposes both panels; no field there accepts a credential value, only a configuration key name and an optional endpoint override.

A provider without a package

AddModelProvider() registers a provider Tracon does not ship a package for. Implement IModelProvider — a stable Name, a Models catalog, and CreateChatClient(ModelBinding) returning a raw IChatClient — and register it. If the provider should also accept a per-tenant credential (see Per-tenant credentials above), additionally implement ITenantCredentialModelProvider, whose CreateChatClient(ModelBinding, ModelProviderCredential) takes a non-null credential; a provider that does not implement it is never called with one — Tracon fails the run instead of falling back to the setup-time key on its behalf:

public sealed class ContosoModelProvider(HttpClient httpClient, string setupApiKey)
    : ITenantCredentialModelProvider
{
    public string Name => "contoso";

    public IReadOnlyList<ModelDescriptor> Models { get; } =
        [new ModelDescriptor { Name = "contoso-large", SupportsTools = true }];

    public IChatClient CreateChatClient(ModelBinding binding)
        => new ContosoChatClient(httpClient, binding.Model, setupApiKey);

    public IChatClient CreateChatClient(ModelBinding binding, ModelProviderCredential credential)
        => new ContosoChatClient(httpClient, binding.Model, credential.ApiKey);
}

services.AddHttpClient();

tracon.AddModelProvider(services =>
    new ContosoModelProvider(services.GetRequiredService<HttpClient>(), setupApiKey));

A provider that never wants to support BYOK simply implements only IModelProvider and stops there — it still works with the setup-time credential for every tenant.

A complete, runnable version of this provider — including per-tenant credentials and provider-settings validation — lives in the repository at samples/Tracon.Samples.CustomModelProvider. It takes Tracon by PackageReference only, which is what makes it a proof rather than an illustration.

The runtime contract

The rules below are what the registry and the compile path actually rely on. An implementation that breaks one still compiles and still passes its own unit tests; it misbehaves under a real deployment. Verify them by deriving ModelProviderContract from the Tracon.Testing.Contracts.Xunit package rather than by reading carefully. If your provider supports BYOK, derive its opt-in ModelProviderCredentialContract too. Its required assertion must inspect the provider request boundary (for example a recording transport), not merely a different client object; otherwise a new wrapper can still send the setup-time key and bill the wrong tenant.

Lifetime and threading. Your provider is a singleton. It must not capture a scoped service, and the provider set is fixed once the container is built. CreateChatClient is called concurrently and must be thread-safe. If you cache a client per credential in a ConcurrentDictionary, remember that GetOrAdd may run its factory more than once for the same key under a race and discard the extras — so building a client must be side-effect free.

Return a raw client. UseFunctionInvocation(), OpenTelemetry, the content guard, the circuit breaker, the concurrency limiter, attachment resolution, response caching, the fallback chain and content-filter detection are all added by ModelProviderRegistry around whatever you return. Building any of them yourself is not merely redundant: the registry places the content guard directly above your client so that every turn of the tool-call loop is inspected. With your own inner loop, the turn that carries a tool result back into the model runs beneath the guard — the exact path prompt injection takes — while the reply text and the tool-call count stay identical, so nothing else reveals it.

Tracon never disposes the client you return. It is built once per compiled agent and held by it; the compiled agent implements neither IDisposable nor IAsyncDisposable, and evicting one from the compile cache drops the reference without disposing. You own the lifetime of what you return, and it must tolerate never being disposed — which is why the shipped providers return clients backed by a long-lived, shared SDK client.

Naming. Name is matched against ModelBinding.Provider case-insensitively. Registering two providers under one name is not last-one-wins: the host fails at startup.

The catalog is not an allow list. See the next section.

Credentials. CreateChatClient(ModelBinding, ModelProviderCredential) — the ITenantCredentialModelProvider overload — is called only when a tenant supplies a credential and your provider implements that interface; without it, Tracon never calls your provider with a tenant credential at all, and it does not fall back to calling the plain CreateChatClient(ModelBinding) overload with the tenant’s request either — the run fails closed with provider_credential_unsupported instead. If you do implement the interface, the key must never fall back to your setup-time key: a tenant that supplied a credential is billed on it, or the call fails. An endpoint may fall back, so a globally configured base address still applies when a tenant overrides only its key.

How failures are classified

Tracon decides whether to move to the next Fallbacks link by reading the exception’s type name and message text across the whole exception graph, because the core holds no compile-time reference to any provider SDK’s exception types. What that means when you write a provider:

Failure	Next fallback link is tried
A timeout — a `TimeoutException` in the graph, or a message that says it timed out	Yes
`OperationCanceledException` with no timeout signal, while the caller’s token is cancelled	No — the caller asked to stop
Message carries `HTTP 401` or `HTTP 403`	No — switching providers would hide a configuration mistake
Message carries `429`, `too many requests`, or `rate limit`	Yes
Message carries `HTTP 5xx`	Yes
A transport/SDK-client type with no HTTP status in the message	Yes — treated as a connection failure
Anything else	No — the retry set is closed and positive

Two consequences are worth stating plainly. A provider-side safety filter is not an exception here: return an ordinary ChatResponse with ChatFinishReason.ContentFilter and Tracon raises TraconContentFilteredException itself. Throwing instead makes the circuit breaker count a healthy provider as failing. And a TraconException thrown from CreateChatClient is wrapped as a compilation error naming the agent, while every other exception propagates raw — so use it for configuration or binding problems the host author can act on.

Model catalog is metadata, not permission

Every provider options type has a Models collection. It drives the console model picker, capability hints, and cost calculation. It is not an allow list. A definition can use a model that is absent from the catalog.

The four capability flags do not carry equal weight, and only one of them is a gate:

Flag	What it does
`SupportsStructuredOutput`	Enforced. An agent requesting JSON output fails compilation when the model is found in the catalog with this explicitly `false`. A model absent from the catalog is not checked. See Structured output
`SupportsTools`	Advisory metadata — nothing enforces it at run time
`SupportsStreaming`	Advisory metadata — nothing enforces it at run time
`SupportsReasoning`	Advisory metadata — nothing enforces it at run time

The advisory flags still drive the console’s picker and capability hints, so an inaccurate catalog misleads a human even where it cannot fail a run.

Response caching

Enable it per agent. Tracon forces the cache key with three inputs beyond the messages and options themselves: the tenant, the sorted tool names, and the provider — a hit never crosses a tenant boundary and never lands on an agent with a different tool set, even when the prompt and instructions are otherwise identical.

builder.Services.AddDistributedMemoryCache(); // or a real distributed cache: Redis, SQL Server, ...

tracon.AddAgent(new AgentDefinition
{
    Name = "cached-support",
    Instructions = "Resolve support requests. State uncertainty clearly.",
    Model = new ModelBinding
    {
        Provider = AnthropicProviderNames.Anthropic,
        Model = "your-current-model-name",
        ResponseCache = new ResponseCacheSettings { Enabled = true, Lifetime = TimeSpan.FromMinutes(10) },
    },
});

A hit skips the model call entirely: no token usage, no cost, and no new trace span for that turn. It does not skip the tool-call loop — if the cached response carries a tool call, the tool still runs; a hit is not a shortcut around side effects. A run’s own usage field reports null for a hit, not 0: Tracon distinguishes “not measured” from “measured as zero” everywhere it reports usage.

Turning ResponseCache.Enabled on without an IDistributedCache registered fails validation and fails to compile the agent; the error names the missing registration. Nothing runs uncached silently. A store failure (a timeout, an oversized payload the store rejects) is logged and treated as a miss on read, or simply dropped on write — a cache problem never fails a call that would otherwise have succeeded.

Concurrent tool calls

By default, independent tool calls returned in the same turn run one after another. Turn AllowConcurrentToolCalls on to run them at the same time instead — useful when a turn calls several independent, I/O-bound tools and their combined latency matters more than a small increase in peak concurrency.

Model = new ModelBinding
{
    Provider = AnthropicProviderNames.Anthropic,
    Model = "your-current-model-name",
    AllowConcurrentToolCalls = true,
},

Off by default: with no change, calls still run one at a time exactly as they do today. Turn it on only for tools whose bodies are safe to run concurrently with themselves — a tool that shares mutable state across calls without its own synchronization should not opt in.

Check a prompt against the context window before running it

The pre-flight check is TraconPreflightOptions, bound from Tracon:Preflight. It is off until Enabled is set, and ReserveRatio decides how much of the window is held back for the answer.

Outgoing concurrency is TraconModelConcurrencyOptions, bound from Tracon:ModelConcurrency: MaxConcurrentCallsPerProvider caps how many calls Tracon has in flight against one provider at a time.

ContextWindowTokens on a catalog ModelDescriptor powers two features: derivation for ContextWindow compaction, and an optional pre-flight check on POST /api/agents/{name}/run that rejects an oversized prompt before any provider is called.

{
  "Tracon": {
    "Preflight": {
      "Enabled": true,
      "ReserveRatio": 0.2
    }
  }
}

Preflight.Enabled is off by default: a wrong estimate stops a run that would have succeeded, and that risk needs an explicit opt-in. ReserveRatio (default 0.2) sets aside a share of the window for the answer; a prompt estimated above the remaining budget returns 400 with the counted and allowed token numbers, and no provider is contacted.

The count is approximate — it uses a single fixed OpenAI encoding regardless of the bound provider, because Anthropic and Google publish no equivalent offline tokenizer. Diagnose the estimate for any agent, independent of whether the check is enabled, with:

curl -X POST "http://localhost:5081/tracon/api/agents/support/estimate" \
  -H "Authorization: Bearer $TRACON_TOKEN" \
  -H 'Content-Type: application/json' \
  -d '{"message":"..."}'

contextWindowTokens and allowedPromptTokens come back null when the agent’s model is not in the catalog — a missing catalog entry is never treated as a rejection, since there is nothing to compare the prompt against.

Defaults and operational behavior

Setting	Default or rule
Model fallback	Provider options can define a default model or Azure deployment for programmatic use; the management HTTP API still requires `model.model`
Sampling fields	`null` uses the provider default
Anthropic output limit	`DefaultMaxOutputTokens = 4096` because Anthropic requires `max_tokens`
Compatible Responses route	Off
Health cache	60 seconds
Background health checks	Off; checks run on request unless an interval is configured
Circuit breaker	On; 5 consecutive failures; one half-open attempt after 30 seconds
Model catalog	Empty until the host supplies entries
Pre-flight context-window check	Off; `POST /api/agents/{name}/estimate` still works when off
Fallback chain	Empty; an unavailable primary throws, same as before this feature existed
Tenant provider binding	None; every tenant uses the global setup-time credential until one is saved
Tenant egress policy	Unrestricted; saving one is an additive restriction, never a default wall
Allowed configuration prefix for a binding	`Tracon:ProviderKeys:`; a name outside it is rejected with `400`
Response cache	Off; a binding with `ResponseCache.Enabled = true` and no registered `IDistributedCache` fails to compile
Response cache lifetime	10 minutes, when caching is enabled
Concurrent tool calls	Off; independent tool calls in one turn run one after another

Force a current, cost-free reachability check with:

curl -H "Authorization: Bearer $TRACON_TOKEN" \
  "http://localhost:5081/tracon/api/models/health/openai?refresh=true"

The health call reads a model list. It does not run a completion.

Troubleshooting

“Provider is not registered.” Confirm that the matching Use...() call ran and that ModelBinding.Provider uses the exact stable name from the table above.

The host fails during startup. Check the provider’s configuration section. Keep the secret value out of appsettings.json, but make sure the environment or secret manager supplies it. Azure also requires an absolute resource endpoint.

The model does not appear in the console. Add it to the provider’s Models collection. The absence does not stop a definition from using it.

Azure health is good, but a run returns 404. Health verifies the resource and credential, not a deployment. Check the deployment name in ModelBinding.Model.

A compatible run has no token count or cost. The upstream server probably omitted streaming usage. Configure no estimate unless you can label it as an estimate.

A compatible provider returns 402. Some gateways reserve credit against the maximum possible output. Set a realistic ModelBinding.MaxOutputTokens value.

Anthropic rejects a thinking request. Keep the thinking budget below the output limit. Remove temperature or set it to 1.

A working prompt gets rejected by the pre-flight check. The token estimate is approximate. Raise ReserveRatio toward zero, or call /estimate to see the counted value against the model’s real ContextWindowTokens before deciding.