ADR 0033 — Signals dedup restoration, and retiring the unbounded-read class

Status: Accepted Date: 2026-08-24 Audience: public Context phase: Daily signals pipeline + read-path integrity Related: ADR 0011/0012 (outcomes + floor recalibration), ADR 0015 (engine-math gate), ADR 0031 (provider failover + observability), ADR 0032 (postmortem write path — the first instance of this defect class)

Context

The owner-visible symptom: /news showed no scored signal after Aug 14, and scoring looked unreliable since late July. The pipeline, meanwhile, reported green and wrote ~170 signals every weekday.

Both things were true. The daily scan never stopped writing signals — it had been mis-scoring since June 10, and the page the owner reads had been mis-rendering since roughly the same week. One defect class explains all of it, the same one ADR 0032 fixed for postmortem writes: unbounded PostgREST selects are silently capped at 1,000 rows. No error, no truncation flag — the missing rows simply don't exist to the caller.

What actually happened, in sequence

Window	Event
Jun 9	signals crosses 1,000 rows. ingestSignals' dedup read (select news_item_id from signals, unranged) now returns only the oldest 1,000 ids.
Jun 10 → Aug 21	Everything scored recently looks unscored. The same news is re-scored every weekday: 2,338 duplicate signal rows across 1,011 news items (up to 7 copies each), ~130 wasted classifier calls/day, and 64 signal_outcomes rows attached to duplicates. Runs report ok/partial throughout — a re-score looks exactly like a score.
Jul 20–28	News fetch outage (fixed in #66). No news rows exist for this window; nothing to backfill.
Jul 29–30	Recovery; catch-up burst scored by the OpenRouter fallback (ADR 0031 segmentation applies).
Jul 31 → Aug 6	Provider account outage: 200-of-200 classifier failures daily (failover shipped mid-window, #68/#69). Leftover: 123 unscored items Jul 29–Aug 20.
Aug 8 →	Classifier occasionally emits invented pillars ("P8", "P9") on multi-stock roundup stories. The thesis_pillar_t enum rejects the insert; the item stays unscored and retries — and fails — every day.
Aug 14 →	Owner observes /news showing nothing scored. The page's badge join reads ALL signals unranged → capped at the oldest 1,000 → every recent item renders unscored. The exact onset the owner perceived tracks the read-side truncation, not any change in scoring.
Aug 19	One missed signals stage (HTTP failure, workflow self-reported red, caught up next day). Unrelated to the above.

The audit

Once the class was identified, every growth-prone read was audited. Confirmed truncating in production on the day of the audit:

• lib/signals.ts — dedup read (the origin of the duplicates).
• lib/news-feed.ts — all three /news badge joins (the owner's symptom).
• lib/cost.ts — MTD ledger read: August already had 2,197 rows, so the weekly pulse and budget alerts under-counted real spend.
• lib/postmortem.ts — candidate read (4,918 rows in the 120d window → only ~20% of eligible signals evaluated) and the prices_daily read (~3,600 rows → truncated bars can shift trading-day indexing, not just shrink coverage).
• lib/cadence.ts / lib/decisions.ts — 14d rollup-window reads (1,529 rows in-window at audit time, dup-inflated).

Bounded-but-fragile (fixed in the same pass): lib/signals-feed.ts 14d/7d summaries, lib/stock-notes.ts weekly-note signals.

Verified safe: reads with explicit .limit(), head-only counts (lib/heartbeat.ts, lib/observability/freshness.ts), time-boxed small windows (lib/digest.ts .limit(50)).

Decision

1. One paging primitive. lib/paged-read.ts (selectAllPaged) is the only sanctioned way to read a whole window: loops .range() pages of 1,000, requires a stable order key, THROWS on error (a failed read must never degrade to a smaller set — that is this incident), throws on maxPages overflow rather than return a known-incomplete set, and preserves the PostgREST error code for strict callers.

2. Dedup fails closed and the backlog is reachable. ingestSignals pages the scored-id read, and pages candidates newest-first until the run's limit of unscored items is found or the table is exhausted. Fresh news keeps priority; backlog drains with leftover budget.

3. Uniqueness is enforced by the database. Migration 20260824090000_signals_dedupe_unique.sql removes the 2,338 duplicates — keeping the EARLIEST signal per news item, the row that would have existed absent the bug — and adds UNIQUE (news_item_id). The insert path treats 23505 as a benign "someone scored it first" skip. A future dedup regression now surfaces as a no-op, not silent duplicates. Deleted rows are preserved in signals_dedupe_backup_20260824 (RLS-locked) until verified, then dropped. The delete cascades the 64 outcome rows attached to duplicates — intended, since their return windows were anchored at re-score time.

4. Invalid pillars are coerced, not fatal. A model-emitted pillar outside P1–P5 means "none of the five"; it is persisted as null and counted in a new pillar_coerced stage metric so classifier drift stays visible.

5. Owner-decided backfill scope (recorded here because it feeds the engine):

   • Score the entire unscored backlog, including ~1,060 pre-Jul-29 items (mostly a 907-item bulk fetch from Jun 8, published May 22–Jun 7). The owner explicitly accepts that these signals carry created_at = scoring time and therefore enter the current 14d decision windows.
   • No Finnhub backfetch for Jul 20–28 — that hole in the news record is accepted as permanent.
   • No backfilled exclusion flag. Outcome windows for backfilled signals are anchored at scoring time, up to ~3 months after the news moved the price; the owner accepts this skew feeding the Sep 1 first floor recalibration (ADR 0032). If the floors move implausibly, the recalibration window (windowDays) still bounds the damage and this decision can be revisited before Oct 1.

Engine-math statement (ADR 0015)

lib/decisions, lib/cadence, lib/postmortem are touched, but no threshold, weight, window, formula, or cadence changes. The edits make the reads those functions already performed COMPLETE instead of silently truncated. /strategy-gate applies as a pass-through, ADR 0032 precedent: the floors and decisions may move because the inputs are now whole — that is the intended effect of a bug fix, not a change to the math.

Consequences

• Post-cleanup, signals reflects one verdict per news item (~2,580 rows); 14d windows drop from ~1,500 (dup-inflated) to their true size — until the backfill burst temporarily swells them, per the owner's decision above.
• August MTD cost restates upward (the ledger was always right; the read wasn't). The backfill itself costs ≈ $3.5–4 against the $10/mo cap.
• Postmortem now evaluates every eligible signal; expect evaluated to jump and insufficient_data classifications to fall.
• Monitoring gap, deliberately not closed here: a run that re-scores already-scored items looks healthy to every current alert. With the unique index the failure mode is now a spike of benign 23505 skips instead of data corruption; a watchdog rule on "scored but net-new signals ≈ 0 for N days" is left as follow-up.