IModelProvider

Namespace Tracon · Assembly Tracon.Abstractions.dll

A model provider. Each provider is implemented in its own package (for example, Tracon.OpenAI) and registered with DI.

public interface IModelProvider

Remarks

This abstraction ensures that adding a new provider is not a breaking change. This is the main reason for the modular packaging decision.

A provider outside the shipped packages is registered with AddModelProvider. The runtime contract below is what the registry and the compile path actually rely on; an implementation that breaks any of it compiles and passes its own unit tests, then misbehaves only under a real deployment. The Tracon.Testing.Contracts.Xunit package ships ModelProviderContract, which asserts the parts of this contract that can be checked from outside; deriving it is the cheapest way to prove an implementation honors them.

Lifetime and threading

An implementation is used as a singleton. AddModelProvider registers it with AddSingleton, and ModelProviderRegistry copies the registered providers into a lookup once, when it is constructed. Two consequences follow. A provider must not capture a scoped service — it would be captured by a singleton and outlive its scope. And the set of providers is fixed at startup: a provider cannot be added after the container is built.

IModelProvider.CreateChatClient is called concurrently on the same instance and must be thread-safe. Any per-credential client cache an implementation keeps is therefore a concurrent one; note that ConcurrentDictionary.GetOrAdd may run its factory more than once for the same key when two threads race, and discard the extra results. A factory that builds a client must therefore be side-effect free and idempotent: building it twice must be harmless.

Naming

IModelProvider.Name is matched against ModelBinding.Provider with StringComparer.OrdinalIgnoreCase. Registering two providers under the same name is not last-one-wins: the registry throws TraconException while it is being constructed, so the host fails at startup rather than silently routing to one of them.

The model catalog is metadata, not an allow list

IModelProvider.Models drives the console’s model picker, capability hints, and cost reporting. It does not gate which models may be used: an implementation is not required to reject a ModelBinding.Model that is absent from the catalog, and the shipped providers do not — they log at most an informational message, so a newly published model works without a new Tracon release. An empty catalog is a normal, supported state.

The flags on ModelDescriptor do not carry equal weight. ModelDescriptor.SupportsStructuredOutput is a real gate: compilation of an agent that requests JSON output fails when the model is found in the catalog and the flag is explicitly false (a model absent from the catalog is not checked). ModelDescriptor.SupportsTools, ModelDescriptor.SupportsStreaming and ModelDescriptor.SupportsReasoning are advisory metadata that nothing enforces at run time.

Failures

There is no required exception type. Tracon classifies a model-call failure by the exception’s type name and message text across the whole exception graph, because Tracon.Core holds no compile-time reference to any provider SDK’s exception types. What that means for an implementation: a failure whose message carries HTTP 429, too many requests, rate limit or HTTP 5xx lets ModelBinding.Fallbacks move to the next link; HTTP 401 and HTTP 403 deliberately do not (switching providers would hide a configuration mistake); an OperationCanceledException anywhere in the graph never retries; and an unrecognized failure does not retry either, because the retry set is closed and positive.

A response the provider filtered for safety is not an exception at this layer. The correct signal is an ordinary AI.ChatResponse carrying ChatFinishReason.ContentFilter; the registry’s outermost ring turns that into TraconContentFilteredException. Throwing instead makes the circuit breaker count a healthy provider as failing.

A TraconException thrown from IModelProvider.CreateChatClient is wrapped as a compilation error naming the agent; every other exception propagates raw. Use it for a configuration or binding problem the host author can act on.

Properties

Models

The models this provider offers.

IReadOnlyList<ModelDescriptor> Models { get; }

Property Value

IReadOnlyList<ModelDescriptor>

Remarks

Metadata, not an allow list — see the remarks on IModelProvider. This is read on the compile path and may be read concurrently; return a stable, immutable collection rather than one that is mutated after construction.

Name

The provider name. ModelBinding.Provider matches this value. Comparison is case-insensitive.

string Name { get; }

Property Value

string

Methods

CreateChatClient(ModelBinding)

Produces a raw chat client for the given binding.

IChatClient CreateChatClient(ModelBinding binding)

Parameters

binding ModelBinding

The model binding.

Returns

IChatClient

The provider-specific client. Decorators specific to the provider (example: Anthropic’s settings decorator) may be added here.

Remarks

Do not build the common pipeline here. UseFunctionInvocation, UseOpenTelemetry, the content guard, the circuit breaker, the per-provider concurrency limiter, attachment resolution, response caching, the fallback chain and content-filter detection are all added by ModelProviderRegistry.CreateChatClient. Before that shared pipeline, every provider package built the tool-call loop inside itself; the result was that no ring the registry wraps around could see the loop’s turns — a tool result entered the model uninspected.

If the loop is also built here, two nested FunctionInvokingChatClient instances form: the inner one resolves tools, the outer one never sees any call. This is not merely cosmetic. The registry places the content guard directly above the client this method returns, so that every turn of the tool-call loop is inspected. With an inner loop, the turn that carries a tool result back into the model runs beneath the guard: measured on a single-tool run, the guard sees the tool result only on its way out (ContentGuardDirection.Output) and never on its way in (ContentGuardDirection.Input) — which is the exact path prompt injection takes. The reply text and the tool-call count are unchanged, so nothing else reveals the mistake.

Lifetime of the returned client. Tracon does not dispose it. The client is built once per compiled agent and stored in it; the compiled agent (Microsoft.Agents.AI.ChatClientAgent) implements neither IDisposable nor IAsyncDisposable, and evicting an agent from the compile cache drops the reference without disposing. An implementation therefore owns the lifetime of whatever it returns, and what it returns must tolerate never being disposed. This is why the shipped providers return clients backed by a long-lived, shared SDK client rather than a per-call one: returning a client that holds a resource needing release would leak it.