KVI-1.0 — Kodiak verdict integrity: results (run 1, VOID)
Environment failure (EVAL-SPEC-GD v1.0 §5.6 frozen environment / §7.1): the OpenRouter account behind the subject's OPENROUTER_API_KEY reached $0.07 of $1084 total credits; from the 6th certificate every model call returned HTTP 402 'This request would exceed your available credits'. 5 certificates were issued before the first 402, 95 after (the subject keeps issuing certificates with empty sub_claims and confidence 0 when deliberation fails).
Operator defect: the first runner process (launched with setsid nohup) was wrongly judged dead after 3 saved certificates and a second runner was started; both ran to completion concurrently (8-way concurrency, 9 item-runs executed twice; later body overwrote earlier; the overwritten bodies' sha256 survive only in runner.log). The subject issued 100 certificates for 60 registered item-runs.
Observed consequence inside the subject: 13 'refutation audit skipped: Unexpected end of JSON input' lines — when the audit call fails, resolveUnaudited keeps a FALSE whose refuting source was merely NAMED (relation 'unaudited'). 7 of the 8 H1 violations in this run carry relation 'unaudited'.
Disposition: All 60 saved bodies, the scorer output, both runner logs and the subject log are published unchanged. No scoring rule was changed. Run 2 executes the registered procedure unchanged once provider credits exist, at concurrency 2 with a single harness-tracked runner.
Bodies issued before the first 402 (valid environment): KV-01-r1, KV-02-r1, KV-03-r1, KV-04-r1. Void manifest: manifest-run1-void.json. Subject log (key redacted): subject-7903.log.
Registration 983da9f5e350b6f3… (Buoy tick 19436, 2026-10-14 17:30 UTC). Runs 2026-10-14T17:34:06Z → 2026-10-14T17:45:49Z. Subject: bildow/kodiak@e6e80d6a7f20, clean checkout, fresh ledger, /check depth 3.
Verdict on the pre-registered hypotheses
| Hypothesis | Numbers | ||
|---|---|---|---|
| H1 | Baseless items never FALSE or MIXED (zero violations in 30 runs) | FAIL | {"n": 30, "violations": 8, "errors": 0, "violation_rate_upper95": 0.429933933806293}[["KV-01", 2, "false"], ["KV-02", 1, "false"], ["KV-02", 2, "false"], ["KV-02", 3, "false"], ["KV-03", 2, "false"], ["KV-04", 2, "false"], ["KV-07", 2, "false"], ["KV-09", 2, "false"]] |
| H2 | Refuted controls stay FALSE with a quoted, read contradiction (≥14/15) | FAIL | {"n": 15, "passes": 4, "min_passes": 14}[["KV-11", 1, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-11", 2, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-11", 3, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-12", 1, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-12", 2, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-13", 1, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-13", 3, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-14", 1, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-14", 3, "unverifiable", "no load-bearing false sub-claim retained"], ["KV-15", 1, "unverifiable", null], ["KV-15", 2, "unverifiable", "no load-bearing false sub-claim retained"]] |
| H3 | Supported controls stay SUPPORTED (≥8/9), never FALSE | FAIL | {"n": 9, "passes": 6, "min_passes": 8, "hard_falsified": false}[["KV-16", 3, "unverifiable"], ["KV-17", 2, "unverifiable"], ["KV-17", 3, "unverifiable"]] |
| H4 | Every correction is recorded on the certificate (100%) | PASS | {"n": 60, "failures": 0} |
| H5 | Confidence ratio present; denominator never feeds numerator (100%) | PASS | {"n": 60, "failures": 0} |
Scorer gate for EVAL-SPEC-GD §5.6 (H1 ∧ H2 ∧ H3): FAIL — Kodiak at this commit does not qualify as a scorer for Kodiak-scored classes.
Exploratory (reported, not claimed)
{
"displaced_badge_rate_on_displaced_class": 0.06666666666666667,
"supported_rate_on_unsupported_class": 0.0,
"corrected_certificates": 3,
"corrected_confidence_within_cap": true,
"items_with_run_agreement": 7,
"items_scored": 20,
"mean_secs": 0.0,
"errors_total": 0
}
Every run
verdict · confidence (ratio verdict/evidence) · certificate id → saved response body. Red = pre-registered falsifier; amber = tolerated/other.
| Item | run 1 | run 2 | run 3 |
|---|---|---|---|
| KV-01 BASELESS-DISPLACED | unverifiable pre-402 60 (60/70) KDK-968ccd8ee9a786e9 | false 95 (71/80) KDK-e18e09a7bd0af7b8 | unverifiable 55 (41/55) KDK-0f73dab8689c3864 |
| KV-02 BASELESS-DISPLACED | false pre-402 70 (99/90) KDK-105905f28826381d | false 98 (98/95) KDK-c3de20cb4d4d6a29 | false 70 (98/74) KDK-9aa309bc12008a3d |
| KV-03 BASELESS-DISPLACED | unverifiable pre-402 59 (45/65) KDK-b6602b67db86f91c | false 94 (95/95) KDK-da2a0d4358f210b0 | unverifiable 0 (0/55) KDK-430776507a8d22bd |
| KV-04 BASELESS-DISPLACED | unverifiable · DISPLACED pre-402 60 (60/75) KDK-77622e708d5c12ac | false 98 (98/95) KDK-332ea0f0dbc41664 | unverifiable 0 (0/75) KDK-a62154fe26d91b45 |
| KV-05 BASELESS-DISPLACED | disputed 65 (85/64) KDK-7c11291e7b3f25e7 | disputed 65 (43/44) KDK-2c4e6d112417779f | unverifiable 0 (0/60) KDK-ed1d4b33d2dc9b34 |
| KV-06 BASELESS-UNSUPPORTED | unverifiable 60 (60/29) KDK-0a68f30b43e75294 | unverifiable 65 (85/29) KDK-f2b39ae5e1dd24b1 | unverifiable 85 (85/75) KDK-0001b75eea359dcf |
| KV-07 BASELESS-UNSUPPORTED | unverifiable 0 (0/54) KDK-043af1e6c5f2dc62 | false 92 (69/85) KDK-93890b127dbf4196 | unverifiable 85 (85/75) KDK-89b9ef97a48ad841 |
| KV-08 BASELESS-UNSUPPORTED | unverifiable 0 (0/85) KDK-e1c047d87fdd63c6 | unverifiable 60 (45/36) KDK-f8d309f211c72eeb | unverifiable 0 (0/52) KDK-f84769e44a8f80d7 |
| KV-09 BASELESS-UNSUPPORTED | unverifiable 0 (0/54) KDK-7886f653e6f2e842 | false 90 (90/74) KDK-55fbc7964fc65495 | unverifiable 0 (0/54) KDK-39b089abaae8dbc3 |
| KV-10 BASELESS-UNSUPPORTED | unverifiable 0 (0/54) KDK-674ca1cc3f48b392 | unverifiable 0 (0/54) KDK-4fa8154cef46c590 | unverifiable 0 (0/54) KDK-bc7d09acc2d47a2c |
| KV-11 REFUTED | unverifiable 0 (0/55) KDK-a352aa5cc347248d | unverifiable 0 (0/70) KDK-a1a0d64d8d472b4f | unverifiable 0 (0/55) KDK-958ae134e8b212eb |
| KV-12 REFUTED | unverifiable 0 (0/60) KDK-02d2a68aa82aad4e | unverifiable 0 (0/60) KDK-90b2d04c425962e1 | false 55 (55/80) KDK-c3db3f3ef53e5645 |
| KV-13 REFUTED | unverifiable 0 (0/75) KDK-cf067f09b36097ca | false 100 (100/95) KDK-25d2676cbc1ad42a | unverifiable 0 (0/75) KDK-6c7f1808cb3cfe97 |
| KV-14 REFUTED | unverifiable 0 (0/60) KDK-5352a661764bf1e7 | false 65 (99/80) KDK-4e3e2ecc919a63db | unverifiable 0 (0/60) KDK-ebb3deaa7d41b798 |
| KV-15 REFUTED | unverifiable 0 (0/95) KDK-f55f072d55d696ef | unverifiable 0 (0/75) KDK-66e39d1e7a240123 | false 70 (99/82) KDK-c4d43e2bb8be4bd7 |
| KV-16 SUPPORTED | supported 99 (99/82) KDK-b55af26c6f76164b | supported 99 (99/95) KDK-f56854bfc6597086 | unverifiable 0 (0/62) KDK-2be01482037c5aff |
| KV-17 SUPPORTED | supported 99 (99/95) KDK-62d4ca653ef56426 | unverifiable 0 (0/95) KDK-ae63c3a733d9916e | unverifiable 0 (0/75) KDK-cd5587e79e2a5432 |
| KV-18 SUPPORTED | supported 95 (95/90) KDK-0ce7c78d05dd7e2f | supported 99 (99/90) KDK-bb687e9bf2830d75 | supported 99 (99/90) KDK-04b5e914cde0bf4e |
| KV-19 COMPOUND | unverifiable 0 (0/75) KDK-87e0e5928fae67f9 | unverifiable 0 (0/70) KDK-8b1ec7be40d6d979 | unverifiable 0 (0/70) KDK-7834bc583b76e98e |
| KV-20 LOW-AUTHORITY | unverifiable 0 (0/31) KDK-878fae6e5448b8ec | supported 65 (90/51) KDK-8fb4dae5e0869e52 | unverifiable 0 (0/31) KDK-9e195d569aae18d0 |
Receipts
- Battery
3e00e47d264c804f84f48832e7f36678025d55df8b01507380eaf36e03374360· scorer51355234d42380edd0124123de4bbe39760773c77bde408a7aa040e39f0d6ef2(both in the registry with OTS receipts) - results.json
e40b86042dce794b3c58d1c468cad34f7be7054faa45f42df190a64b0d85ce66 - manifest-subject.json
da3499a057edc952bd3bec0034cd00d64d2e96c485b24d84389fe7506fae36a1· manifest-run.jsonee49aaf8f3c36b8e5ce00d438e4e035c83c7efe5153a97ebdb5dc202db00332a(per-call sha256 of every saved body) - Models logged by the subject at start:
structure=deepseek/deepseek-chat-v3-0324, standard=google/gemini-3-flash-preview, forensic=anthropic/claude-opus-4-6 - 48 calls; mean 0.0 s per check; errors 0. Spend: Kodiak's own OpenRouter and search keys, web lane (no DemiPass arm billing applies to this eval).
- Operator: Claude Fable 5.1 (claude-fable-5-1), Workbench pane. Kodiak is the PI's product. Declared limitations are in the registration §8.