Event-log recovery: cursor consistency, decoder agreement, and resume correctness

Two positions that should have agreed

The projection tracked both a byte offset into the event file and the ordinal expected at that position. The recorded failure was concrete:

expected ordinal 31217, got 31218

The expected record existed earlier in the source file. The cursor pointed past it. The investigation also found duplicate or regressed ordinals elsewhere, so the problem could not be reduced to one missing line.

A complete JSON parsing pass at that stage found no malformed records. That narrowed the problem: syntactic validity was intact, while the ordering assumptions used to materialize history were not.

This is why file growth, valid JSON, and an advancing UI need separate checks. Each describes a different part of the system.

The projection cursor is a compound state

A byte offset and an ordinal are two coordinates for the same logical position. Updating either one independently can produce a state that is individually well-formed but impossible to consume consistently.

For an abstract append-only log, let b be the byte position of the next record to consume and n its expected logical ordinal. A cursor is C = (b, n). A successful materialization step needs to keep the projected records and the next cursor mutually consistent. Otherwise, a crash can leave either an advanced cursor without its corresponding rows or committed rows behind an old cursor.

The incident did not establish that such a transaction failure originally caused the mismatch. This model explains the invariant the diagnosis checked. At the recorded cursor, the next source record did not have the expected ordinal, even though that ordinal existed elsewhere in the file.

Rebuilding a projection can reconcile derived state with a chosen interpretation of the source. It cannot, by itself, establish that the source's own ordering is valid. That became important once duplicate and regressed ordinals were observed.

Parseability is weaker than decoder agreement

The raw source passed JSON parsing, but a generic JSON parser is not the same thing as the application's typed event decoder. A syntactically valid numeric value can still fail a narrower schema, and two readers can disagree about whether a record participates in resume-state reconstruction.

That is the significance of the decimal token-count observation. If one path accepts a record and another fails to account for it while reconstructing the next ordinal, both may operate on the same file while deriving different continuation state. This describes the decoder/resume hypothesis investigated in the retained record; the local observations did not prove every step of that mechanism.

The broader requirement is that readers responsible for projection, reverse scanning, and continuation agree on record semantics. Agreement on JSON syntax alone is insufficient. Any policy for unknown or malformed event types must also preserve a coherent account of ordering; silently skipping a record can have consequences beyond losing that record's display content.

Repair the derived state with the source preserved

The recovery work backed up the relevant data and operated on the projection and its ordering interpretation. Preserving the canonical record mattered: changing the original transcript would make later diagnosis harder and could conceal the conditions that produced the mismatch.

A separate repair process also needed to wait for the application to exit. An initial attempt watched stale process state and remained waiting. The practical correction was a standalone script with dynamic process polling and visible status output, so its progress could be observed independently of the application being repaired.

The retained recovery summary records 151,613 processed records and 5,042 ordinal corrections. That was evidence that historical materialization completed. It was not yet evidence that the application would produce a valid continuation.

Reopening was the decisive test

After the application reopened and the session resumed, a new record reused an ordinal. The failure had recurred at the producer/runtime boundary.

The replay had repaired the existing view. It had not repaired the behavior that created the next ordering error. Calling the incident fixed at the end of the offline run would have missed the condition the user actually cared about: continuing work without the history becoming stale again.

The recurrence followed a record containing a decimal token count and resembled a decoder/resume problem identified during the investigation. That was a useful lead. It did not prove a generic concurrency race or establish that all long conversations would fail the same way.

Idempotence does not establish a valid continuation

For a fixed source snapshot L, a deterministic reconstruction procedure R should converge to the same logical projection when run again. Ignoring operational metadata, the intended property is:

R(L, R(L, P)) = R(L, P)

Here P is the prior projection. This is a specification-level property, not an idempotence test result claimed by the incident report. It describes one useful obligation for recovery tooling: applying the same reconstruction again should not keep changing its logical answer.

Continuation imposes a different obligation. If A appends a new event using resume state reconstructed by the live application, then A must produce a source that the normal projection path can consume. An offline repair can be deterministic and still coexist with a broken A.

That was the practical boundary exposed by reopening. The repair's completed record count described work over existing data. The next appended record tested the application's reconstruction of future ordering, and it failed.

For a live stream, even a statement that the projection reached the end of the file needs a reference point. The file may grow during inspection. A useful comparison fixes an observed source boundary and checks whether the projection reaches that boundary without ordering errors. Otherwise, concurrent growth can be confused with projection lag or recovery failure.

Recovery has to include continuation

The acceptance condition became more specific: new events must receive valid, increasing ordinals; the projection must consume them; and a later reopen must preserve the newly visible history.

A replay checks whether old events can be interpreted under a chosen recovery procedure. Resume checks whether the live application can continue the stream. Reopen checks whether the visible result survives persistence and reload. Passing the first says little about the other two unless they are observed.

The recorded outcome was therefore partial recovery with a reproduced continuation failure. That result is less satisfying than a green completion message, but it identifies the remaining defect much more precisely.

Evidence note: based on the September 8 local diagnostic report and retained recovery summary. The original diagnostic report was reread for this article; the repair was not rerun. These observations describe the affected installation at that time and do not establish the behavior of current Codex releases.