Last 11 runs. 6 silent failures caught, 1 routed to review.
| # | time (UTC) | source | status | failed checks | payload |
|---|---|---|---|---|---|
| 11 | 2026-09-07T20:06:51Z | strict-demo | fail | empty_output: Output is empty / whitespace only. | show raw JSON"" |
| 10 | 2026-09-07T20:06:51Z | research-agent | pass | — | show raw JSON"Acme Corp Q3 2026 filing found: revenue up 12% year over year, two sources attached." |
| 9 | 2026-09-07T20:06:51Z | research-agent | fail | possible_loop: Semantic thrash: web_search called 3x with near-identical arguments after an empty result (similarity 1.00 >= 0.85) (possible loop). | show raw JSON"Could not find the Acme Corp Q3 2026 filing yet, retrying with a different phrasing." |
| 8 | 2026-09-07T20:06:51Z | research-agent | fail | possible_loop: Tool call repeated 4x with identical arguments: web_search({"q": "Acme Corp Q3 2026 filing"}) (possible loop). | show raw JSON"Searching for Acme Corp Q3 2026 filings. Searching for Acme Corp Q3 2026 filings. Searching for Acme Corp Q3 2026 filings. Searching for Acme Corp Q3 2026 filings." |
| 7 | 2026-09-07T20:06:51Z | api-normalizer | fail | schema_drift: Output shape changed vs. baseline: type changed: tags (string -> array) | show raw JSON{
"title": "Widget",
"tags": [
"blue",
"small"
],
"meta": {
"id": 41
}
} |
| 6 | 2026-09-07T20:06:51Z | api-normalizer | pass | — | show raw JSON{
"title": "Widget",
"tags": "blue,small",
"meta": {
"id": 41
}
} |
| 5 | 2026-09-07T20:06:51Z | weekly-research | review | — | show raw JSON{
"summary": "Acme Corp Q3 2026: preliminary figures suggest revenue up about 10%.",
"sources": [
{
"url": "https://example.com/acme-prelim"
}
],
"confidence": "medium"
} |
| 4 | 2026-09-07T20:06:51Z | weekly-research | fail | contract_violation: output does not mention required input: 'Acme Corp'; output does not mention required input: 'Q3 2026' | show raw JSON{
"summary": "Global SaaS market grew 9% last quarter according to analysts.",
"sources": [
{
"url": "https://example.com/saas-market"
},
{
"url": "https://example.com/x"
}
],
"confidence": "medium"
} |
| 3 | 2026-09-07T20:06:51Z | weekly-research | pass | — | show raw JSON{
"summary": "Acme Corp Q3 2026: revenue up 12% year over year, driven by enterprise renewals.",
"sources": [
{
"url": "https://example.com/acme-q3"
},
{
"url": "https://example.com/acme-ir"
}
],
"confidence": "high"
} |
| 2 | 2026-09-07T20:06:51Z | demo | fail | schema_drift: Output shape changed vs. baseline: type changed: age (number -> string) | show raw JSON{
"name": "Ada",
"email": "ada@example.com",
"age": "2026-09-05"
} |
| 1 | 2026-09-07T20:06:51Z | demo | pass | — | show raw JSON{
"name": "Ada",
"email": "ada@example.com",
"age": 36
} |
Schema drift: the first run per source sets the baseline shape (keys + types). To re-learn after an intentional change: POST /schema/reset?source=NAME with your X-API-Key. Per-run detail: GET /run/{id}. Review: a passing run can be routed to a human via contract.review (if_missing / if_contains / sample_rate) or payload review:true; reject with add_rule to harden the contract. Dead-letter: the response is synchronous, branch on status; or send strict:true (env SANECHECK_STRICT=1) and a failed run answers HTTP 422 so your node's error output fires. Loops: meta.tool_calls repeated with identical arguments, near-identical arguments (canonicalized + Jaccard >= 0.85 vs the last 3 calls), or the same line repeating in the output, flag possible_loop. Contracts (job drift): send a 'contract' object with required fields / must_contain, or store one via POST /contract?source=NAME.
SaneCheck v0.5.2