# IEvalStore

> Tracon documentation. Tracon is a .NET package family that adds a control
> plane on Microsoft Agent Framework, published as NuGet packages that run
> inside your own ASP.NET Core application.
>
> Page: https://tracon.dev/api/tracon.ievalstore/
> Summary: IEvalStore: The store for evaluation (eval) suites, cases, and runs.

<div class="api-provenance">Namespace Tracon · Assembly <code>Tracon.Abstractions.dll</code></div>

The store for evaluation (eval) suites, cases, and runs.

```csharp
public interface IEvalStore
```

## Remarks

**DI lifetime — singleton.** Registered as a singleton with
`TryAdd`; a consumer's own registration wins.

## Methods

### <a id="Tracon_IEvalStore_AddCaseAsync_System_Guid_Tracon_EvalCaseDraft_System_Threading_CancellationToken_"></a> AddCaseAsync\(Guid, EvalCaseDraft, CancellationToken\)

Adds a SINGLE case to the suite. `Seq` is generated atomically by
the store; the caller does not compute it. If the same
[EvalCaseDraft.SourceRunId](/api/tracon.evalcasedraft/#Tracon_EvalCaseDraft_SourceRunId) is added a second time, the
existing case is returned with `Created: false`.

```csharp
ValueTask<EvalCaseAddResult> AddCaseAsync(Guid suiteId, EvalCaseDraft draft, CancellationToken cancellationToken = default)
```

#### Parameters

`suiteId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The suite identifier.

`draft` [EvalCaseDraft](/api/tracon.evalcasedraft/)

The draft of the case to add.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalCaseAddResult](/api/tracon.evalcaseaddresult/)\>

The added (or already existing) case, and whether it was created.

### <a id="Tracon_IEvalStore_CompleteRunAsync_Tracon_EvalRunCompletion_System_Threading_CancellationToken_"></a> CompleteRunAsync\(EvalRunCompletion, CancellationToken\)

Finalizes a run and writes its summary counters.

```csharp
ValueTask CompleteRunAsync(EvalRunCompletion completion, CancellationToken cancellationToken = default)
```

#### Parameters

`completion` [EvalRunCompletion](/api/tracon.evalruncompletion/)

The finalization information.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask)

The completion task.

### <a id="Tracon_IEvalStore_CreateRunAsync_Tracon_EvalRun_System_Threading_CancellationToken_"></a> CreateRunAsync\(EvalRun, CancellationToken\)

Creates a new run record (state [EvalRunStatus.Pending](/api/tracon.evalrunstatus/)).

```csharp
ValueTask<EvalRun> CreateRunAsync(EvalRun run, CancellationToken cancellationToken = default)
```

#### Parameters

`run` [EvalRun](/api/tracon.evalrun/)

The run record.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalRun](/api/tracon.evalrun/)\>

The run record with its identifier assigned.

### <a id="Tracon_IEvalStore_DeleteSuiteAsync_System_String_System_String_System_Threading_CancellationToken_"></a> DeleteSuiteAsync\(string, string, CancellationToken\)

Deletes the suite and all of its cases/runs (cascade).

```csharp
ValueTask<bool> DeleteSuiteAsync(string tenantId, string name, CancellationToken cancellationToken = default)
```

#### Parameters

`tenantId` [string](https://learn.microsoft.com/dotnet/api/system.string)

The tenant identifier.

`name` [string](https://learn.microsoft.com/dotnet/api/system.string)

The suite name.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[bool](https://learn.microsoft.com/dotnet/api/system.boolean)\>

[true](https://learn.microsoft.com/dotnet/csharp/language-reference/builtin-types/bool) if the suite was deleted.

### <a id="Tracon_IEvalStore_DiffRunsAsync_Tracon_EvalRunDiffQuery_System_Threading_CancellationToken_"></a> DiffRunsAsync\(EvalRunDiffQuery, CancellationToken\)

Aligns two completed runs of the same suite, case by case.

```csharp
ValueTask<EvalRunDiff?> DiffRunsAsync(EvalRunDiffQuery query, CancellationToken cancellationToken = default)
```

#### Parameters

`query` [EvalRunDiffQuery](/api/tracon.evalrundiffquery/)

Which two runs to compare, and which page of the result to return.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalRunDiff](/api/tracon.evalrundiff/)?\>

The difference; [null](https://learn.microsoft.com/dotnet/csharp/language-reference/keywords/null) if either run does not exist or
belongs to another tenant.

#### Exceptions

 [EvalRunDiffUnavailableException](/api/tracon.evalrundiffunavailableexception/)

The two runs exist but cannot be compared: they measure different
suites, one of them has not completed, or one of them holds results for
fewer cases than its summary counts because retention removed some of
them.
**An empty diff is never returned in their place** — it
would read as "nothing changed" and turn a CI gate green over a
regression. Implementations delegate to
[EvalRunDiffBuilder.Build](/api/tracon.evalrundiffbuilder/#Tracon_EvalRunDiffBuilder_Build_Tracon_EvalRun_Tracon_EvalRun_System_Collections_Generic_IReadOnlyList_Tracon_EvalCaseResult__System_Collections_Generic_IReadOnlyList_Tracon_EvalCaseResult__System_Int32_System_Int32_) rather than restating the rules.

### <a id="Tracon_IEvalStore_GetRunAsync_System_String_System_Guid_System_Threading_CancellationToken_"></a> GetRunAsync\(string, Guid, CancellationToken\)

Fetches a single run record.

```csharp
ValueTask<EvalRun?> GetRunAsync(string tenantId, Guid evalRunId, CancellationToken cancellationToken = default)
```

#### Parameters

`tenantId` [string](https://learn.microsoft.com/dotnet/api/system.string)

The tenant identifier.

`evalRunId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The run identifier.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalRun](/api/tracon.evalrun/)?\>

The record; [null](https://learn.microsoft.com/dotnet/csharp/language-reference/keywords/null) if it does not exist or belongs to another tenant.

### <a id="Tracon_IEvalStore_GetRunByJobIdAsync_System_String_System_Guid_System_Threading_CancellationToken_"></a> GetRunByJobIdAsync\(string, Guid, CancellationToken\)

Fetches the run a job record produced.

```csharp
ValueTask<EvalRun?> GetRunByJobIdAsync(string tenantId, Guid jobId, CancellationToken cancellationToken = default)
```

#### Parameters

`tenantId` [string](https://learn.microsoft.com/dotnet/api/system.string)

The tenant identifier.

`jobId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The job identifier.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalRun](/api/tracon.evalrun/)?\>

The record; [null](https://learn.microsoft.com/dotnet/csharp/language-reference/keywords/null) if it does not exist.

### <a id="Tracon_IEvalStore_GetSuiteAsync_System_String_System_String_System_Threading_CancellationToken_"></a> GetSuiteAsync\(string, string, CancellationToken\)

Fetches the suite matching the given name within the tenant.

```csharp
ValueTask<EvalSuite?> GetSuiteAsync(string tenantId, string name, CancellationToken cancellationToken = default)
```

#### Parameters

`tenantId` [string](https://learn.microsoft.com/dotnet/api/system.string)

The tenant identifier.

`name` [string](https://learn.microsoft.com/dotnet/api/system.string)

The suite name.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalSuite](/api/tracon.evalsuite/)?\>

The suite; [null](https://learn.microsoft.com/dotnet/csharp/language-reference/keywords/null) if it does not exist.

### <a id="Tracon_IEvalStore_ListCaseResultsAsync_System_String_System_Guid_System_Threading_CancellationToken_"></a> ListCaseResultsAsync\(string, Guid, CancellationToken\)

Lists a run's case results.

```csharp
ValueTask<IReadOnlyList<EvalCaseResult>> ListCaseResultsAsync(string tenantId, Guid evalRunId, CancellationToken cancellationToken = default)
```

#### Parameters

`tenantId` [string](https://learn.microsoft.com/dotnet/api/system.string)

The tenant identifier.

`evalRunId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The run identifier.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<[EvalCaseResult](/api/tracon.evalcaseresult/)\>\>

The results.

### <a id="Tracon_IEvalStore_ListCasesAsync_System_Guid_System_Threading_CancellationToken_"></a> ListCasesAsync\(Guid, CancellationToken\)

Lists a suite's cases, ordered by sequence number.

```csharp
ValueTask<IReadOnlyList<EvalCase>> ListCasesAsync(Guid suiteId, CancellationToken cancellationToken = default)
```

#### Parameters

`suiteId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The suite identifier.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<[EvalCase](/api/tracon.evalcase/)\>\>

The cases.

### <a id="Tracon_IEvalStore_ListSuitesAsync_System_String_System_Threading_CancellationToken_"></a> ListSuitesAsync\(string, CancellationToken\)

Lists all of a tenant's suites.

```csharp
ValueTask<IReadOnlyList<EvalSuite>> ListSuitesAsync(string tenantId, CancellationToken cancellationToken = default)
```

#### Parameters

`tenantId` [string](https://learn.microsoft.com/dotnet/api/system.string)

The tenant identifier.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<[EvalSuite](/api/tracon.evalsuite/)\>\>

The suites, ordered by name.

### <a id="Tracon_IEvalStore_MarkRunRunningAsync_System_Guid_System_Nullable_System_Int32__System_String_System_Threading_CancellationToken_"></a> MarkRunRunningAsync\(Guid, int?, string?, CancellationToken\)

Transitions a run to [EvalRunStatus.Running](/api/tracon.evalrunstatus/) and records
the agent version and model identifier being measured.

```csharp
ValueTask MarkRunRunningAsync(Guid evalRunId, int? agentVersion, string? modelId, CancellationToken cancellationToken = default)
```

#### Parameters

`evalRunId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The run identifier.

`agentVersion` [int](https://learn.microsoft.com/dotnet/api/system.int32)?

The agent definition version being measured.

`modelId` [string](https://learn.microsoft.com/dotnet/api/system.string)?

The model identifier being measured.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask)

The completion task.

### <a id="Tracon_IEvalStore_QueryRunsAsync_Tracon_EvalRunQuery_System_Threading_CancellationToken_"></a> QueryRunsAsync\(EvalRunQuery, CancellationToken\)

Lists runs by filter. The newest record is returned first.

```csharp
ValueTask<IReadOnlyList<EvalRun>> QueryRunsAsync(EvalRunQuery query, CancellationToken cancellationToken = default)
```

#### Parameters

`query` [EvalRunQuery](/api/tracon.evalrunquery/)

The filter.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<[EvalRun](/api/tracon.evalrun/)\>\>

The records.

### <a id="Tracon_IEvalStore_RecordCaseResultAsync_Tracon_EvalCaseResult_System_Threading_CancellationToken_"></a> RecordCaseResultAsync\(EvalCaseResult, CancellationToken\)

Records a case result.

```csharp
ValueTask RecordCaseResultAsync(EvalCaseResult result, CancellationToken cancellationToken = default)
```

#### Parameters

`result` [EvalCaseResult](/api/tracon.evalcaseresult/)

The case result.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask)

The completion task.

### <a id="Tracon_IEvalStore_ReplaceCasesAsync_System_Guid_System_Collections_Generic_IReadOnlyList_Tracon_EvalCase__System_Threading_CancellationToken_"></a> ReplaceCasesAsync\(Guid, IReadOnlyList<EvalCase\>, CancellationToken\)

Replaces all of a suite's cases with the given list.

```csharp
ValueTask<IReadOnlyList<EvalCase>> ReplaceCasesAsync(Guid suiteId, IReadOnlyList<EvalCase> cases, CancellationToken cancellationToken = default)
```

#### Parameters

`suiteId` [Guid](https://learn.microsoft.com/dotnet/api/system.guid)

The suite identifier.

`cases` [IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<[EvalCase](/api/tracon.evalcase/)\>

The new case list. Sequence numbers are assigned by list order.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[IReadOnlyList](https://learn.microsoft.com/dotnet/api/system.collections.generic.ireadonlylist\-1)<[EvalCase](/api/tracon.evalcase/)\>\>

The cases with identifiers assigned.

### <a id="Tracon_IEvalStore_SaveSuiteAsync_Tracon_EvalSuite_System_Threading_CancellationToken_"></a> SaveSuiteAsync\(EvalSuite, CancellationToken\)

Creates or updates the suite.

```csharp
ValueTask<EvalSuite> SaveSuiteAsync(EvalSuite suite, CancellationToken cancellationToken = default)
```

#### Parameters

`suite` [EvalSuite](/api/tracon.evalsuite/)

The suite to save.

`cancellationToken` [CancellationToken](https://learn.microsoft.com/dotnet/api/system.threading.cancellationtoken)

The cancellation token.

#### Returns

 [ValueTask](https://learn.microsoft.com/dotnet/api/system.threading.tasks.valuetask\-1)<[EvalSuite](/api/tracon.evalsuite/)\>

The suite with its identifier and timestamps assigned.