ADR 0032 — Restoring the postmortem write path, and the first real floor recalibration

Status: Accepted Date: 2026-08-17 Audience: public Context phase: Postmortem persistence + ADR 0012 floor calibration Related: ADR 0011 (outcome classification + horizons), ADR 0012 (floor recalibration formula), ADR 0015 (engine-math gate), ADR 0031 (observability; signals.provider segmentation) Implements: issue #73, PR #74

Context

signal_outcomes has not accepted a row since 2026-05-27. For 81 days the daily analysis stage reported status: "ok" while writing nothing.

The consequence reaches further than a stale table. signal_outcomes is the only input to recomputePillarFloors, so ADR 0012's recalibration has never run against real data in production. All five pillars still carry their Phase E seed:

pillar | floor_value | sample_size | precision_60d | updated_at
P1..P5 |    0.70     |      0      |     null      | 2026-05-04

Three defects had to coincide:

1. The dedup read was unbounded. PostgREST caps a select at 1,000 rows and does not report truncating. The table crossed that line on the exact day the writes stopped:

   day	inserted	cumulative
   2026-05-26	87	950
   2026-05-27	88	1,038 ← crosses the cap; last insert ever

2. The invisible rows poisoned the batch. The 38 pairs the truncated read missed were re-queued and violated UNIQUE (signal_id, horizon_days). Postgres rejects the whole statement, so every valid row died with them.

3. The error was swallowed. It went to console.error and nowhere else — not to pipeline_errors, not to the stage status, not to the response. This is the same failure shape ADR 0031 was written to eliminate for the signals stage, recurring one stage over.

The live run on 2026-08-14 shows what was being discarded each day:

"postmortem": { "evaluated": 1537, "inserted": 0,
  "by_classification": { "true_positive": 221, "false_positive": 82,
                         "neutral_hit": 290, "neutral_miss": 352,
                         "insufficient_data": 592 } }

945 signals classified correctly, none persisted.

Decision

1 · Fix the write path (PR #74). Page the dedup read explicitly; replace the batch insert with a chunked upsert carrying onConflict: "signal_id,horizon_days", ignoreDuplicates: true; throw on write failure instead of logging and continuing.

The upsert is the structural change: existingKey becomes an optimisation rather than a correctness guarantee, so anything a read misses is dropped by the database instead of taking the statement down with it. A truncated read can no longer cause data loss under any circumstances.

2 · Accept that the floors will move, and record the expectation here. Nothing in this change touches recalibrateFloor, SAMPLE_FLOOR_MIN, HORIZONS, classifyOutcome, realizedReturn, or ENGINE_VERSION (f.1.0). /strategy-gate therefore returns PASS as a pass-through: the engine math is byte-identical.

But the inputs change from empty to populated, so the floors will move for the first time. On the 2026-08-14 figures:

precision = 221 / (221 + 82) = 72.9%
floor     = 0.7 - 0.5 * (0.729 - 0.5) = 0.586   (ADR 0012, clamped [0.5, 0.9])

The floors are currently too high, not too low. The engine has been holding a 0.70 bar that the evidence does not support, discarding signals it should have counted. Expect decision volume to rise, not fall.

This is recorded as an ADR even though the gate does not compel one, because otherwise the production floors would move with no written explanation — and ADR 0012 is precisely the decision this affects.

Consequences

Timing. Merging does not move the floors. recomputePillarFloors runs monthly (0 22 1 * *), so the first real recalibration is 2026-09-01 22:00 UTC. It uses a 60-day window at the 5-day horizon and is gated at SAMPLE_FLOOR_MIN = 20 outcomes per pillar — pillars below that threshold do not move. There is a two-week observation window between merge and any change.

The first recalibration reads a surge, not a series. The first post-fix run persists roughly 1,500 rows at once. The 2026-09-01 recalibration will therefore compute from a backlog rather than a steady accumulation. Reviewing the per-pillar precision before that run is advisable; pausing one cycle is reasonable and costs nothing.

Provider segmentation (ADR 0031) is preserved. provider is carried onto every outcome row, so precision stays segmentable by model. This matters immediately: a Sonnet 4.6 → Sonnet 5 classifier swap is under evaluation in PR #72, and its track record must not pool with the incumbent's.

Known limitation, deliberately not fixed here. insufficient_data remains terminal. The dedup key blocks re-evaluation once any row exists for a (signal_id, horizon_days) pair, so a signal scored before its horizon elapsed keeps that verdict permanently — even after the price bars arrive. The first post-fix run will therefore lock roughly 592 rows that would otherwise resolve later.

Fixing it means either not persisting insufficient_data at all, or upserting with update-on-conflict. Both change the append-only audit semantics the postmortem was built on, so it warrants its own decision rather than riding along with a persistence fix. Tracked in issue #73.

Detection is improved but not complete. A write failure now degrades the stage to partial, which is queryable in pipeline_runs and visible to alerting. The verbatim message still does not reach pipeline_errors — that needs ctx.recordError threaded into runAnalysisStage, an observability-seam change beyond this fix. There is still no watchdog rule for "no outcomes written in N days", which is what would have caught this on day two rather than day 81.

Alternatives considered

Keep insert and simply paginate the dedup read. Rejected. It fixes the present trigger and leaves the mechanism intact: any future gap between what the read returns and what the table holds — a race, a replica lag, a second writer — reproduces the same total batch loss. The upsert removes the failure mode rather than the trigger.

Backfill with a one-shot script. Unnecessary. The daily run already evaluates 1,537 signals every weekday; once the write works it persists them on the next pass. Only the ~900 stale rows from May need explicit remediation.

Treat this as a rule change and require a backtest. Rejected as uninformative. The backtest replays engine decisions over history and cannot observe a floor value that has never varied; per the open question in CLAUDE.md, per-day staged replay is still deferred, so the harness could not measure this even in principle. Separately, backtests.meta carries no sor_basket return, so the Δ-vs-basket comparison /strategy-gate specifies is not currently computable for any change — worth closing before the next genuine rule change.