fix(session): retire stale truncation watermark on new committed turn (#3831) (#3890)

* fix(session): retire stale truncation watermark on new committed turn (#3831)

retry_last / undo_last / the Edit-truncate handler set truncation_watermark
to suppress the *replaced* tail from the append-only state.db merge.
Session.save() deliberately never auto-clears it (#2914), but nothing retired
it when the user then sent a genuinely NEW turn either — so it froze at the old
edit boundary. A frozen watermark then dropped post-watermark state.db rows
whenever the sidecar was later reconstructed empty (recovery/reconcile),
permanently losing the turns sent after the edit (state.db still had them).

Retire a POSITIVE watermark to None once the new user turn is COMMITTED to
session.messages — at the success-merge (3 sites), eager-checkpoint, error/
recovery materialization, and cold-load repair commit points. Not at chat-start:
in deferred mode the new row isn't in messages yet, so a merge in that window
would resurrect the replaced tail (the max-sidecar guard hasn't risen past the
old boundary). Once committed, max_sidecar_timestamp rises past the replaced
tail and the merge suppresses it without the watermark, so retiring is safe.

Cleared to None, never 0.0 — 0.0 is the truncate-to-empty sentinel (#2914) that
must keep blocking all state replay, so the clear is falsy-gated.

Closes #3831

* chore(changelog): clarify watermark-retirement timing to commit-time

Greptile review noted the original phrase "retires the watermark at the
start of a new user turn" was timing-imprecise. The retirement actually
fires when the new turn is durably committed to session.messages —
at the agent-result merge, the eager user-message checkpoint, or the
cold-load recovery commit. Reword for accuracy; semantics unchanged.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

* test(#3831): add regression tests for the two inline watermark-clear paths (greptile P2)

Cover the error/cancel materialization path (_materialize_pending_user_turn_before_error)
and the eager first-turn checkpoint path (_checkpoint_user_message_for_eager_session_save),
which inline the falsy-gated watermark clear instead of calling the tested helper.
The error path is precisely the #3831 failure mode (recovery/reconcile after a
crash), so a dedicated regression test closes that gap. Both assert a positive
watermark clears to None while the 0.0 truncate-to-empty sentinel (#2914) is
preserved.

---------

Co-authored-by: nesquena-hermes <[email protected]>
Co-authored-by: Nathan Esquenazi <nesquena@gmail.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>

api/models.py

@@ -389,6 +389,12 @@ def _append_recovered_pending_turn(session, *, timestamp: int | None = None) ->
         recovered['attachments'] = list(session.pending_attachments)
     session.messages.append(recovered)
     _append_recovered_turn_to_context(session, recovered)
+    # The new user turn is now committed to messages (#3831): retire a positive
+    # truncation watermark from a prior retry/undo/edit so it can't freeze at the
+    # old edit boundary and drop these post-edit turns on a later empty-sidecar
+    # reconcile. None, never 0.0 (the truncate-to-empty sentinel, #2914).
+    if getattr(session, 'truncation_watermark', None):
+        session.truncation_watermark = None
     return recovered