EvalCaseDiffKind

Namespace Tracon · Assembly Tracon.Abstractions.dll

The bucket an aligned case falls into within an EvalRunDiff.

[JsonConverter(typeof(JsonStringEnumConverter<EvalCaseDiffKind>))]
public enum EvalCaseDiffKind

Fields

Added = 4

Present only in the candidate run.

Fixed = 1

Failed on the baseline, passes on the candidate.

Regressed = 2

Passed on the baseline, fails on the candidate.

Removed = 5

Present only in the baseline run.

StillFailing = 3

Failed on both sides.

Unchanged = 0

Passed on both sides.

Remarks

EvalCaseDiffKind.Added and EvalCaseDiffKind.Removed are deliberately separate from the four outcome buckets and are never folded into them: adding a case to a suite moves the pass rate without anything having regressed, and a gate that cannot tell those apart raises a false alarm. Written as a name in JSON; the value order must not change — only append.