.NET API
EvalCase
Tracon.Abstractions.dllA single test case inside an EvalSuite.
public sealed record EvalCase : IEquatable<EvalCase>- Inheritance
- object ← EvalCase
- Implements
- IEquatable<EvalCase>
- Inherited Members
- object.GetType(), object.ToString(), object.Equals(object?), object.Equals(object?, object?), object.ReferenceEquals(object?, object?), object.GetHashCode()
Constructors
Section titled “Constructors”EvalCase()
Section titled “ EvalCase()”public EvalCase()Properties
Section titled “Properties”Context
Section titled “ Context”Text given to the model as extra context.
public string? Context { get; init; }Property Value
Section titled “Property Value”Remarks
Section titled “Remarks”The agent receives it through the document channel, ahead of
EvalCase.Query: the case run builds the messages a production run
with documents sends (see AgentRunDocument). Plain text
goes as one document named context. Text made only of document
channel blocks - what promoting a run with documents stores here, one
-----BEGIN TRACON DOCUMENT----- block per document - goes back as
one document per block, unchanged, so the case repeats the run’s
messages.
The evaluator receives the same text as its context, for checks that judge an answer against reference material.
ExpectedOutput
Section titled “ ExpectedOutput”The expected output. Referenced by the containsExpected check.
public string? ExpectedOutput { get; init; }Property Value
Section titled “Property Value”ExpectedToolCalls
Section titled “ ExpectedToolCalls”The tool calls this case expects, by name and optionally by arguments.
public IReadOnlyList<EvalExpectedToolCall> ExpectedToolCalls { get; init; }Property Value
Section titled “Property Value”IReadOnlyList<EvalExpectedToolCall>
Remarks
Section titled “Remarks”Referenced by the toolCallArgsMatch check. An empty list expects
nothing, so that check passes. The toolCalled check does not read
this field: it takes its tool names from its own tools field in the
suite’s checks definition.
The case identifier.
public Guid Id { get; init; }Property Value
Section titled “Property Value”Parameters
Section titled “ Parameters”Values for the target agent’s AgentDefinition.Parameters schema. null for an agent that declares no parameters.
public IReadOnlyDictionary<string, string>? Parameters { get; init; }Property Value
Section titled “Property Value”IReadOnlyDictionary<string, string>?
Remarks
Section titled “Remarks”Evaluating a parameterized agent without this field would run every
case against unresolved {{name}} placeholders left literally in
the instructions text - the same missing-parameter check that
POST /api/agents/{name}/run applies also runs here, and a case
missing a required value fails outright rather than running with a
broken prompt.
PromotedAt
Section titled “ PromotedAt”The promotion time. null when hand-written.
public DateTimeOffset? PromotedAt { get; init; }Property Value
Section titled “Property Value”The query text sent to the agent.
public required string Query { get; init; }Property Value
Section titled “Property Value”The sequence number within the suite (starts at 0).
public required int Seq { get; init; }Property Value
Section titled “Property Value”SourceKind
Section titled “ SourceKind”The reason for promotion. null when hand-written.
public EvalCaseSource? SourceKind { get; init; }Property Value
Section titled “Property Value”SourceRunId
Section titled “ SourceRunId”The run the case was generated from. null when hand-written.
public Guid? SourceRunId { get; init; }Property Value
Section titled “Property Value”Guid?
SuiteId
Section titled “ SuiteId”The identifier of the suite this case belongs to.
public required Guid SuiteId { get; init; }Property Value
Section titled “Property Value”Methods
Section titled “Methods”Equals(object?)
Section titled “ Equals(object?)”public override bool Equals(object? obj)Parameters
Section titled “Parameters”obj object?
Returns
Section titled “Returns”Equals(EvalCase?)
Section titled “ Equals(EvalCase?)”public bool Equals(EvalCase? other)Parameters
Section titled “Parameters”other EvalCase?
Returns
Section titled “Returns”GetHashCode()
Section titled “ GetHashCode()”public override int GetHashCode()Returns
Section titled “Returns”ToString()
Section titled “ ToString()”public override string ToString()Returns
Section titled “Returns”Operators
Section titled “Operators”operator ==(EvalCase?, EvalCase?)
Section titled “ operator ==(EvalCase?, EvalCase?)”public static bool operator ==(EvalCase? left, EvalCase? right)Parameters
Section titled “Parameters”left EvalCase?
right EvalCase?
Returns
Section titled “Returns”operator !=(EvalCase?, EvalCase?)
Section titled “ operator !=(EvalCase?, EvalCase?)”public static bool operator !=(EvalCase? left, EvalCase? right)Parameters
Section titled “Parameters”left EvalCase?
right EvalCase?