Stopping repeated agent work with distinct-progress fingerprints

A simple current-information request once ran for 44 steps. The agent fetched the same pages, summarized them again, and continued because every repeated fetch increased the raw source count. The runtime interpreted a larger ledger as progress even when the set of work had not changed.

We also saw the opposite failure. A model emitted a progress sentence such as "I am checking the official sources" and the runtime treated it as the final answer.

Both failures sit at the boundary between work and termination. We addressed them with two separate controls: typed final-answer eligibility and a fingerprint over distinct work.

A checkpoint cannot finalize a run

Progress updates and final answers have different semantics. A checkpoint tells the user what is happening while the run remains active. A final answer claims that the run reached a terminal state.

Lightcap represents that distinction with types:

MessageKind:
  Checkpoint | PartialAnswer | FinalAnswer | FailureReport

TerminalStatus:
  completed | partial | blocked | continue

continue is a sentinel, not a terminal status. A message classified as Checkpoint cannot end the loop.

For answer-producing work, finalAnswerEligibility() receives primitive facts from the run ledger. It does not scan the proposed answer for phrases that sound finished.

finalAnswerEligibility({
  isAnswerProducing,
  attemptedWebGrounding,
  eligibleEvidenceCount,
  groundedAnswer,
  honestNoSource,
  hasUnresolvedBlockers,
  recovering,
  forced,
});

The state machine has four important outcomes:

• A grounded answer may finalize as completed.
• An unresolved blocker with no active recovery finalizes as blocked.
• An explicit source-limited result finalizes as partial or blocked, depending on whether any eligible evidence exists.
• An ungrounded answer that is not forced remains a checkpoint and the run continues.

If a safety or deadline boundary forces the loop to stop without a grounded answer, the runtime replaces the candidate with an honest partial or blocked fallback. Forced termination does not convert weak evidence into completion.

Build and edit tasks use separate artifact checks. The evidence rule above applies to answer-producing runs, where eligible sources and grounded claims determine whether the result is ready.

Why the original progress signal failed

The first no-progress detector included a raw sourceCount and a rolling list of tool calls. That design had a monotonicity bug.

Suppose the agent fetched three pages in a cycle:

A, B, C, A, B, C, A, B, C

The count grew from three to nine, so every iteration changed the fingerprint. The stall counter reset even though the run had learned nothing after the first cycle.

The same problem appears with commands. Retrying npm test after the same failure should not count as new progress merely because another command record was appended.

Progress is a set of distinct work

progressFingerprint() now canonicalizes four ledger dimensions:

• files, keyed by path, operation, and line changes;
• successful commands, keyed by command and outcome;
• tools, keyed by tool identity and result;
• sources, keyed by URL, query, or a stable tool summary.

Each dimension becomes a sorted set before it is serialized. Repeating an identical successful action produces the same fingerprint.

return stableJson({
  files: [...fileKeys].sort(),
  commands: [...commandKeys].sort(),
  tools: [...toolKeys].sort(),
  sources: [...sourceKeys].sort(),
});

Failures need an additional normalization step. A struggling agent may express the same failed operation through several surface commands: change PYTHONPATH, reinstall the package, reclone the repository, then try another wrapper. If all attempts end with the same typed failure, treating each command string as progress still permits a loop.

Failed commands are therefore keyed by a normalized outcome. Successful commands retain their distinct command identity. This avoids flattening legitimate work while collapsing repeated attempts that reach the same dead end.

The bounded stall decision

After each progress check, the loop compares the new fingerprint with the last one:

if (nextFingerprint === lastProgressFingerprint) {
  noProgressIterations += 1;
} else {
  lastProgressFingerprint = nextFingerprint;
  noProgressIterations = 0;
}

if (noProgressIterations >= 4) {
  forceFinish = true;
}

Four consecutive checks with no new distinct work trigger a deterministic wrap-up. The runtime does not ask the looping model whether it believes it is finished.

That boundary interacts with task state. If a build is stalled but still has required files that have not been written, the runtime can issue a bounded rescue instruction before forcing completion. This prevents the stall guard from cutting off a build merely because the model narrated the next file instead of calling the write tool. Rescue attempts are capped, so the exception cannot recreate an infinite loop.

A separate monologue guard handles iterations with no action call and no ledger progress. This is orthogonal to repeated tool outcomes: one catches talk without action, the other catches action without new work.

Regression tests for identity, not volume

The loop-stop tests pin the property that the original counter violated.

• Fetching one URL once and ten times produces the same fingerprint.
• Cycling through A, B, and C produces no new progress after the three distinct URLs exist.
• Adding a genuinely new URL changes the fingerprint.
• Repeating the same failed command does not change the fingerprint.
• Running a different successful command does change it.

Checkpoint tests separately verify that an ungrounded candidate stays continue, a grounded answer becomes completed, and a forced ungrounded stop becomes an honest partial or blocked result.

These are unit-level invariants. The 44-step incident is a concrete failure trace, not an aggregate performance study. We have not published a distribution showing how often the four-check threshold is optimal across task classes.

Limits and false-stop risk

Set-based progress is stricter than a raw counter, but it is only as good as its identity keys.

If a tool summary includes volatile timestamps, two equivalent outcomes may look different. If normalization is too aggressive, two genuinely different debugging attempts may collapse into one failure key. The implementation preserves distinct successful commands and uses typed diagnostics for failed tools to reduce both risks.

The fixed threshold also encodes a tradeoff. A lower number ends loops faster but may interrupt a slow recovery. A higher number spends more time and tokens before admitting that the state is unchanged. Build-rescue and recovery state provide bounded exceptions; they do not remove the need to measure false stops.

Finally, a changed fingerprint proves that the ledger changed. It does not prove that the task moved closer to success. Artifact checks, evidence eligibility, and admissibility still decide whether the new work matters.

Reliable termination requires both halves. Typed final states prevent a checkpoint from becoming an answer. Distinct-progress fingerprints prevent repeated activity from buying unlimited time. Neither asks prose to carry a control decision that belongs in runtime state.