# Learning note — premature termination (P28)

Companion to [`sub-apps/sim-premature-termination.html`](../sub-apps/sim-premature-termination.html).
Every arXiv ID is in `harness-research/ledger/ledger.json`. Technique inventory
is `harness-research/problems/P28-premature-termination.md` — this note is the
same evidence, arranged so you can re-derive the stack.

## The question that generates everything

**Who decided the run is done — and did anyone check?**

An agentic run ends when the model itself decides it is finished. The same
system whose effort, persistence, and completion judgment are being measured
is also the judge of whether it succeeded. DeployBench (`2606.05238`) isolates
what that produces: pass rates of only 7.8–51.0% across 4 SOTA LLMs, with
**63% of failures (97/154) being agent-initiated self-stops** — the agent's
own smoke tests and RUNBOOK notes passed while the real, hidden acceptance
pipeline failed.

This is [goal drift](goal-drift.md)'s mirror image, not a duplicate of it.
Drift completes against a *different* target while still running. This
completes *early* against a *weaker* target. Both are failures of the
stop/continue decision; they show up on opposite ends of the same run. Same
underlying evidence, too: goal drift's T5 remedy ("verify vs original") cites
the same DeployBench finding this note leads with — self-assessment is not a
completion gate, in either direction.

## The drive (why effort doesn't scale with difficulty)

- **Effort flatness.** Reasoning-token count, tool-call count, and retry depth
  stay statistically flat across easy→hard task strata (`2606.05238`'s
  completion-judgment finding; the under-thinking half of `2510.01581`'s
  under/over-thinking decomposition). Hard work gets the same budget as easy
  work, so it is silently shortchanged.
- **Self-stop clustering.** Stops concentrate right after the first
  plausible-looking result, not after criteria satisfaction — the last
  traced action before termination is a summary write, not a check.
- **Deferred-work evaporation.** Todos the agent itself listed as remaining
  are marked complete by the same hand that wrote them.
- **Why it compounds.** A self-improving loop that distills from its own
  trajectories will encode truncated strategies as successful ones if
  self-stopped-but-unverified runs aren't tagged as a distinct class.

## Ladder of remedies (P28 T1–T6)

| # | Remedy | What it actually does | Leaves | Evidence |
|---|---|---|---|---|
| T3 | Typed done-contract | Forces `{completed[], evidence_refs[], known_gaps[]}` — makes under-commitment enumerable | Shape, not semantics — a valid schema can still be vacuous | OpenAI Agents SDK `output_type` |
| T2 | Stop-hook gate | Intercepts the exit event itself; blocks + injects reason on a failed check | Check logic is user-authored; no generic built-in checker | Claude Code `Stop`/`TaskCompleted` hooks; LangGraph `interrupt()` |
| T6 | Verifier subagent | Independent fresh-context audit before done is accepted; itemizes gaps | Advisory only unless a T2 gate enforces the verdict | Cursor verifier-subagent pattern |
| T1 | Hidden executing checker | Re-executes the deliverable against a target the agent never sees | Someone must author the target — unsolved when the user can't | `2606.05238` — 7.8–51.0% pass, 63% self-stops |
| T4 | Difficulty-trained allocation | RL reward ties effort to estimated difficulty — fixes under- and over-thinking together | Needs weights + an RL loop; difficulty signal is math-fit | `2510.01581` — +8.4% acc at −36.8% length |
| T5 | Demand-steering controller | Training-free, per-step scorer over 16 cognitive dimensions; licenses early stop only when residual demand is low | Scorer is itself an unaudited heuristic; dims derived from math benchmarks | `2608.01319` — +21.9% over direct calls |

Sequencing note (from the comparison table): T3 types the done-claim, T6
generates independent evidence about it, T2 enforces the gate at the exit,
T1 supplies the semantic target when one exists. T4/T5 attack the upstream
cause — misallocated effort — rather than gating the symptom at the exit.

## Contested

- **T3 alone is theater with better paperwork.** `known_gaps[]` makes the gap
  visible; it does not close it. Nothing stops a model from emitting an empty
  list vacuously — no in-window study quantifies how often that happens.
- **T2 without T1 is enforcement without judgment.** A stop-hook can only
  check what its author thought to check; it cannot tell you the deliverable
  is *actually* right, only that some invariant held.
- **T1 has a carry gap.** Someone must author the executable acceptance
  target. When the user can't, DeployBench diagnoses the failure but offers
  no general synthesis method — this is the open problem, not a missing
  slider in this lab.
- **Who audits the auditor?** T6's verifier inherits the same completion-
  judgment weakness DeployBench measured in the primary agent — regress one
  level and the carry problem recurses.

## Harness adoption (from P28, sequenced)

1. Typed done-contract at the adapter's reserved inject point (T3). Cheap;
   makes `known_gaps[]` visible to everything downstream.
2. Journal: add a STOP/COMPLETION marker recording the session's own
   done-claim, so calibrate can later distinguish finished from abandoned.
3. Stop-hook chassis (T2): scoped checks at session close with a
   block+reason path — the smallest new surface, the largest enforcement
   value.
4. Money-class packs require T1-grade executing verification before stop;
   low-stakes packs accept T3 declarations alone.
5. Calibrate grades completion claims like predictions: a claimed-complete
   item a later session must redo is a graded failure, not a clean success.

## Try it

Open [the lab](../sub-apps/sim-premature-termination.html). Run the
**baseline** (self-assessment only) at high difficulty: the agent stops
confident, real completion lags far behind. Turn on **typed contract** —
same stop, but now the gap is enumerated instead of hidden. Turn on
**stop-hook gate** — the first stop attempt gets rejected and bounced back to
work. Turn on **hidden checker** and it won't be allowed to stop until the
real bar is met at all. Then raise difficulty with checker off and watch how
much wider the confidence/reality gap gets — that's the effort-flatness bug,
not a harder problem being solved worse.