[domiwei]

Problem

Caplin's fork graph writes two kinds of dump files and calls File.Sync() on each one:

• the head state, in DumpBeaconStateOnDisk (cl/phase1/forkchoice/fork_graph/fork_graph_disk_fs.go), every 4th slot (dumpSlotFrequency = 4);
• the execution payload envelope, in DumpEnvelopeOnDisk (same file), after Gloas.

On a Hoodi node (1.14M validators) one head-state dump writes about 143 MB. dumpMs is p50 ~540 ms, p90 ~1.06 s and max 2.65 s. Measured locally, encoding takes ~4 ms and snappy + write ~100 ms, so most of the node time is likely writeback and fsync. The split on the node was not measured separately.

Every dump holds a shared lock, and the head dump after fork choice also holds a lock and a transaction, so every millisecond counts:

• every dump holds stateDumpLock (fork_graph_disk_fs.go:122). The same lock is needed by fork-graph replay (readBeaconStateFromDisk, :57), by req/resp and API envelope reads (ReadEnvelopeFromDisk, :205), and after Gloas by envelope import (DumpEnvelopeOnDisk, :354), which runs while it holds the fork-choice f.mu (cl/phase1/forkchoice/on_execution_payload.go:780, :1176);
• the head dump after fork choice runs inside syncedData.ViewHeadState, which holds the synced-data read lock (cl/phase1/stages/forkchoice.go:469, cl/beacon/synced_data/synced_data.go:167). A waiting writer makes new readers queue behind it;
• it also runs inside the stage's indiciesDB RW transaction (cl/phase1/stages/forkchoice.go:498-518).

Why removing fsync is safe

The durability that fsync gives (data survives a power loss or a kernel crash) is never used, because nothing reads these files after such an event:

• Both files live in path.Join(dirs.Tmp, "caplin-forkchoice"). Caplin removes and recreates this directory on every start, before it builds the fork graph (cmd/caplin/caplin1/run.go:365-384). This is the only production fork-graph filesystem; the spectests use afero.NewMemMapFs() (cl/spectest/consensus_tests/fork_choice.go:314). The removal came in the same PR that added the dump and its Sync (Forkchoice states saved on disk #8546). Comments in cl/phase1/stages/forward_sync.go call the forward-sync dumps "restart checkpoints", but they are also wiped on start; the real restart anchor is the finalized-state save.

Inside the running process the files stay usable without fsync:

• Reads see written data through the page cache, fsynced or not.
• All content reads and writes of these files hold stateDumpLock (fork_graph_disk_fs.go:57, :122, :205, :354). Prune removals and the hasBeaconState existence check run outside it (fork_graph_disk.go:755, :850-858); this is unchanged.
• If a head-state dump cannot be read, replay walks back to an older dump when the block is still kept (fork_graph_disk.go:639-645), or returns no state when only the header is kept, for example for the anchor (:623-633). This is the same as for any missing dump.
• Envelopes are still written to a .tmp file, closed with a checked Close, and renamed (fork_graph_disk_fs.go:377-427). Readers never see a partial file, and a write, flush, close or rename error keeps the existing final file (TestDumpEnvelopeFailurePreservesExistingFinal).

History: the head-state Sync came with the original feature (#8546, which gives no rationale; its error was always returned) and was moved in #8851. The envelope dump copied the pattern in #18956, and #23548 moved envelopes to tmp+rename and kept the Sync. No change relies on the durability of these files.

Trade-off: fsync is also the point where the kernel reliably reports a delayed writeback error (EIO). close(2) on a local filesystem usually does not report it. Such an error may now go unnoticed by the dump and show up later, for example when the data is read again after the pages were evicted. Before, the dump itself failed. A head-state read error makes replay fall back as described above. An envelope that fails to decode is marked invalid and can be fetched again. If the re-read itself returns EIO, the envelope stays marked as present, so normal ingestion skips it while the error lasts: until a later read succeeds, the anchor envelope is rewritten, or the block is pruned. This needs a failing disk, and it is accepted here for the cost above.

What changed

• DumpBeaconStateOnDisk: removed Sync(). The file was closed by an unchecked defer. It is now closed once in a defer that joins the Close error into the returned error, like the envelope dump, so errors reported at close (for example on NFS) are not lost.
• DumpEnvelopeOnDisk: removed Sync(). The order stays write, flush, checked Close, Rename.

What is not changed

• The finalized-state save under dirs.CaplinLatest (saveFinalizedStateOnDiskIfNeeded / replaceDurableFile, cl/phase1/stages/forkchoice.go:372-430) is the restart anchor. It keeps its file fsync, rename and directory fsync.
• Every dump still holds stateDumpLock, and the head dump after fork choice still runs under the synced-data read lock and inside the indiciesDB RW transaction. This PR only makes the dumps shorter.

Follow-ups (not in this PR):

• Move the head-state dump and the finalized-state save after tx.Commit(). This also fixes a silent rollback: today a failed dump makes postForkchoiceOperations return an error, so the canonical-chain and finalized-slot writes in the same transaction are rolled back.
• Gloas envelope import holds the fork-choice f.mu while it waits for stateDumpLock (on_execution_payload.go:780 → :1176), so a long head-state dump still stalls fork choice. Either release f.mu around persistence or give envelopes their own lock and buffer.

Tests

• TestForkGraphDumpsDoNotRequireSync: with an afero file whose Sync() fails, NewForkGraphDisk (forced anchor dump) and DumpEnvelopeOnDisk must succeed, and the state and the envelope must read back with equal SSZ hashes. Fails on main with the injected Sync error.
• TestDumpBeaconStateReturnsCloseError: with an afero file whose Close() fails, DumpBeaconStateOnDisk must return that error and close the file exactly once. Fails on main because the error was dropped.
• TestDumpEnvelopeFailurePreservesExistingFinal: removed the "sync" stage, because Sync is no longer called. The open, write, close and rename stages still pass.
• go test -race ./cl/phase1/forkchoice/... passes. go test ./cl/... passes except cl/spectest Test/mainnet_config, which needs the downloaded mainnet fixtures (not present locally, unrelated to this change). make lint is clean.

Backport

Not planned for release/3.8.

[Copilot]

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

[Copilot]

🟢 Approval recommended

The focused changes preserve flushing, locking, and atomic envelope replacement, with targeted regression coverage and no unresolved findings.

No open issues 🎉

_{Review effort: Balanced}

[yperbasis]

Approving.

Non-blocking:

1. The two "Dump state periodically for restart checkpoints" comments in cl/phase1/stages/forward_sync.go now contradict the argument this PR rests on, since the dumps are wiped on start. Worth rewording them here.
2. Please file tracking issues for the two follow-ups in the description (dump after tx.Commit() to stop the silent rollback; f.mu held while envelope import waits for stateDumpLock) so they do not get lost.
3. release/3.8 has the same fsync calls and this commit cherry-picks onto it cleanly. Mainnet carries a similar validator count, so a [r3.8] backport looks worthwhile.