HTTP API
EvalRunDiff
The case-by-case difference between two completed EvalRun records of the same suite.
Shape: object
| Property | Required | Type | Description | Rules |
|---|---|---|---|---|
baseline |
yes | EvalRun |
The run being compared against (the older, known-good side). | — |
candidate |
yes | EvalRun |
The run being judged. | — |
cases |
yes | array of EvalCaseDiff |
The requested page of aligned cases, ordered so that the buckets an engineer looks for come first: EvalCaseDiffKind.Regressed, EvalCaseDiffKind.StillFailing, EvalCaseDiffKind.Added, EvalCaseDiffKind.Fixed, EvalCaseDiffKind.Removed, then EvalCaseDiffKind.Unchanged; within a bucket by case identifier. | — |
totalCases |
yes | integer (int32) |
The number of aligned cases across both runs, ignoring paging. | pattern `^-?(?:0\ |
unchangedCount |
yes | integer (int32) |
The number of cases that passed on both sides. | pattern `^-?(?:0\ |
fixedCount |
yes | integer (int32) |
The number of cases that failed on the baseline and pass now. | pattern `^-?(?:0\ |
regressedCount |
yes | integer (int32) |
The number of cases that passed on the baseline and fail now. | pattern `^-?(?:0\ |
stillFailingCount |
yes | integer (int32) |
The number of cases that failed on both sides. | pattern `^-?(?:0\ |
addedCount |
yes | integer (int32) |
The number of cases present only in the candidate run. | pattern `^-?(?:0\ |
removedCount |
yes | integer (int32) |
The number of cases present only in the baseline run. | pattern `^-?(?:0\ |