# IEvalEvaluatorFactory

> Tracon documentation. Tracon is a .NET package family that adds a control
> plane on Microsoft Agent Framework, published as NuGet packages that run
> inside your own ASP.NET Core application.
>
> Page: https://tracon.dev/api/tracon.ievalevaluatorfactory/
> Summary: IEvalEvaluatorFactory: The extension point that builds the AI.IAgentEvaluator an eval suite is graded with.

<div class="api-provenance">Namespace Tracon · Assembly <code>Tracon.Core.dll</code></div>

The extension point that builds the `AI.IAgentEvaluator` an eval
suite is graded with.

```csharp
public interface IEvalEvaluatorFactory
```

## Remarks

Tracon grades a suite with MAF's `AI.LocalEvaluator`, built
from the checks the suite declares. This interface is the seam around that
choice: a consumer registers its own factory to grade with a different
evaluator, or to wrap the built-in one.

The factory receives the suite's compiled checks rather than being a bare
`AI.IAgentEvaluator` singleton. A suite declares its checks and
Tracon refuses to run a suite that declares none; an evaluator that
never saw them would silently grade something else. A factory that ignores
the argument takes over grading completely, and that is then a deliberate
choice rather than an accident.

The factory is a singleton and is called once per eval run. It must be
thread-safe. Registering none keeps today's behaviour exactly.

The built-in `toolCallArgsMatch` check reads a case's expected tool
calls from the very `AI.EvalItem` instance the run builds. The item's
`ExpectedToolCalls` carries them too, in the form Microsoft Agent
Framework reads, where an object or array value is its JSON text. An evaluator
that copies the items before it hands them to the checks leaves that check with
the second form only: an expected object then compares as text and does not
equal the object a call carried. Pass the items through unchanged, or compare
in a check of your own.

**Tenant mode:** tenant-independent. The factory sees only the
suite's checks, never a tenant identifier or tenant data; the run it grades
carries the tenant, and the eval store applies it.

**Delivery:** no delivery guarantee applies. This is a
synchronous factory call on the eval job's own thread, not a dispatch
mechanism. It is called again when a job's lease expires and the job is
retried, so it must be cheap and free of side effects.

## Methods

### <a id="Tracon_IEvalEvaluatorFactory_Create_System_Collections_Generic_IReadOnlyList_Microsoft_Agents_AI_EvalCheck__"></a> Create\(IReadOnlyList<EvalCheck\>\)

Creates the evaluator that grades one eval run.

```csharp
IAgentEvaluator Create(IReadOnlyList<EvalCheck> checks)
```

#### Parameters

`checks` [IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<EvalCheck\>

The suite's compiled checks. Never empty.

#### Returns

 IAgentEvaluator

The evaluator. Must not be [null](https://learn.microsoft.com/dotnet/csharp/language-reference/keywords/null).