.NET API
EvalRunDiff
Tracon.Abstractions.dllThe case-by-case difference between two completed EvalRun records of the same suite.
public sealed record EvalRunDiff : IEquatable<EvalRunDiff>Inheritance
Section titled “Inheritance”Implements
Section titled “Implements”Inherited Members
Section titled “Inherited Members”object.GetType(), object.ToString(), object.Equals(object?), object.Equals(object?, object?), object.ReferenceEquals(object?, object?), object.GetHashCode()
Remarks
Section titled “Remarks”A single “how many broke” number is not enough to act on: the engineer on call wants to know which case broke. Every aligned case therefore lands in exactly one EvalCaseDiffKind bucket, and each entry carries both sides’ agent run identifier, so a regression is one click away from the two conversations that produced it.
Cases are aligned by EvalCaseResult.CaseId, and case
content is not snapshotted per run. If a case’s
EvalCase.Query is edited between the two runs while its
identifier is kept, the two sides describe different questions and the diff
silently compares them as one. Through the HTTP surface this cannot happen
(PUT /api/evals/{name}/cases replaces cases and assigns fresh
identifiers, so an edit shows up as a EvalCaseDiffKind.Removed
plus an EvalCaseDiffKind.Added entry); it can only happen when
a consumer calls IEvalStore.ReplaceCasesAsync directly and
preserves identifiers. Nothing here detects that, and no guarantee is
offered against it.
The counters are for the whole diff and do not shrink with paging; EvalRunDiff.Cases is the requested page.
Constructors
Section titled “Constructors”EvalRunDiff()
Section titled “ EvalRunDiff()”public EvalRunDiff()Properties
Section titled “Properties”AddedCount
Section titled “ AddedCount”The number of cases present only in the candidate run.
public required int AddedCount { get; init; }Property Value
Section titled “Property Value”Baseline
Section titled “ Baseline”The run being compared against (the older, known-good side).
public required EvalRun Baseline { get; init; }Property Value
Section titled “Property Value”Candidate
Section titled “ Candidate”The run being judged.
public required EvalRun Candidate { get; init; }Property Value
Section titled “Property Value”The requested page of aligned cases, ordered so that the buckets an engineer looks for come first: EvalCaseDiffKind.Regressed, EvalCaseDiffKind.StillFailing, EvalCaseDiffKind.Added, EvalCaseDiffKind.Fixed, EvalCaseDiffKind.Removed, then EvalCaseDiffKind.Unchanged; within a bucket by case identifier.
public required IReadOnlyList<EvalCaseDiff> Cases { get; init; }Property Value
Section titled “Property Value”FixedCount
Section titled “ FixedCount”The number of cases that failed on the baseline and pass now.
public required int FixedCount { get; init; }Property Value
Section titled “Property Value”RegressedCount
Section titled “ RegressedCount”The number of cases that passed on the baseline and fail now.
public required int RegressedCount { get; init; }Property Value
Section titled “Property Value”RemovedCount
Section titled “ RemovedCount”The number of cases present only in the baseline run.
public required int RemovedCount { get; init; }Property Value
Section titled “Property Value”StillFailingCount
Section titled “ StillFailingCount”The number of cases that failed on both sides.
public required int StillFailingCount { get; init; }Property Value
Section titled “Property Value”TotalCases
Section titled “ TotalCases”The number of aligned cases across both runs, ignoring paging.
public required int TotalCases { get; init; }Property Value
Section titled “Property Value”UnchangedCount
Section titled “ UnchangedCount”The number of cases that passed on both sides.
public required int UnchangedCount { get; init; }Property Value
Section titled “Property Value”Methods
Section titled “Methods”Equals(object?)
Section titled “ Equals(object?)”public override bool Equals(object? obj)Parameters
Section titled “Parameters”obj object?
Returns
Section titled “Returns”Equals(EvalRunDiff?)
Section titled “ Equals(EvalRunDiff?)”public bool Equals(EvalRunDiff? other)Parameters
Section titled “Parameters”other EvalRunDiff?
Returns
Section titled “Returns”GetHashCode()
Section titled “ GetHashCode()”public override int GetHashCode()Returns
Section titled “Returns”ToString()
Section titled “ ToString()”public override string ToString()Returns
Section titled “Returns”Operators
Section titled “Operators”operator ==(EvalRunDiff?, EvalRunDiff?)
Section titled “ operator ==(EvalRunDiff?, EvalRunDiff?)”public static bool operator ==(EvalRunDiff? left, EvalRunDiff? right)Parameters
Section titled “Parameters”left EvalRunDiff?
right EvalRunDiff?
Returns
Section titled “Returns”operator !=(EvalRunDiff?, EvalRunDiff?)
Section titled “ operator !=(EvalRunDiff?, EvalRunDiff?)”public static bool operator !=(EvalRunDiff? left, EvalRunDiff? right)Parameters
Section titled “Parameters”left EvalRunDiff?
right EvalRunDiff?