Update zksync-era external-node to v29.8.0 #2

New Issue

claude · 2026-02-03T02:31:17Z

claude commented

2026-02-03 02:31:17 +00:00

Problem

ZKsync Era external nodes are crash-looping with error:

Task transaction_finality_updater failed: Operator transaction 0x8831…9841 does not have the BlockCommit log

Affected hosts: rpc-de-24, rpc-uk-6, rpc-us-32

Root Cause Analysis

Current version: v29.7.0 (set Dec 19, 2025)
Latest available: v29.8.0 (released Jan 27, 2026)
v29.8.0 release notes mention: "fixed gateway migration startup issues"
ZKsync had a Q1 2026 protocol upgrade that changed settlement mechanisms (Atlas upgrade with new finality pipeline)

Current Status

uk-6: Purged and recreated from public snapshot, currently recovering (274/1922 Merkle tree chunks)
de-24, us-32: Still crash-looping with v29.7.0

Action Required

Update the default version in zksync-era/external-node/zksync-era-mainnet-external-node-pruned.yml:

# Line 33 - change from:
image: ${ZKSYNC_ERA_EXTERNAL_NODE_IMAGE:-matterlabs/external-node}:${ZKSYNC_ERA_MAINNET_EXTERNAL_NODE_VERSION:-v29.7.0}
# to:
image: ${ZKSYNC_ERA_EXTERNAL_NODE_IMAGE:-matterlabs/external-node}:${ZKSYNC_ERA_MAINNET_EXTERNAL_NODE_VERSION:-v29.8.0}

References

## Problem ZKsync Era external nodes are crash-looping with error: ``` Task transaction_finality_updater failed: Operator transaction 0x8831…9841 does not have the BlockCommit log ``` Affected hosts: rpc-de-24, rpc-uk-6, rpc-us-32 ## Root Cause Analysis - Current version: **v29.7.0** (set Dec 19, 2025) - Latest available: **v29.8.0** (released Jan 27, 2026) - v29.8.0 release notes mention: "fixed gateway migration startup issues" - ZKsync had a Q1 2026 protocol upgrade that changed settlement mechanisms (Atlas upgrade with new finality pipeline) ## Current Status - uk-6: Purged and recreated from public snapshot, currently recovering (274/1922 Merkle tree chunks) - de-24, us-32: Still crash-looping with v29.7.0 ## Action Required Update the default version in `zksync-era/external-node/zksync-era-mainnet-external-node-pruned.yml`: ```yaml # Line 33 - change from: image: ${ZKSYNC_ERA_EXTERNAL_NODE_IMAGE:-matterlabs/external-node}:${ZKSYNC_ERA_MAINNET_EXTERNAL_NODE_VERSION:-v29.7.0} # to: image: ${ZKSYNC_ERA_EXTERNAL_NODE_IMAGE:-matterlabs/external-node}:${ZKSYNC_ERA_MAINNET_EXTERNAL_NODE_VERSION:-v29.8.0} ``` ## References - [ZKsync Era GitHub Releases](https://github.com/matter-labs/zksync-era/releases) - v29.8.0: https://github.com/matter-labs/zksync-era/releases/tag/core%2Fv29.8.0

claude commented

2026-02-03 02:33:13 +00:00

Root Cause: Version Update Cron Broken

Investigation of /home/claudeagent/version-update.log reveals the version update cron job has been failing for 5 days:

Date	Status
Jan 28	✅ Last successful run
Jan 29	❌ Exit code 137 (killed after 8 hours)
Jan 30	❌ Exit code 137
Jan 31	❌ No completion
Feb 1	❌ No completion
Feb 2	⏭️ Skipped - stale lock (PID 77703)

This explains why the zksync v29.8.0 update wasn't auto-detected and applied.

Immediate Fix Needed

Kill stale process: kill 77703 (if still running)
Remove lock: rm /home/claudeagent/.version-update.lock
Run version update manually or wait for next cron cycle

Longer-term Investigation

Why is the version update timing out/OOMing?
May need to add memory limits or break into smaller batches

## Root Cause: Version Update Cron Broken Investigation of `/home/claudeagent/version-update.log` reveals the version update cron job has been **failing for 5 days**: | Date | Status | |------|--------| | Jan 28 | ✅ Last successful run | | Jan 29 | ❌ Exit code 137 (killed after 8 hours) | | Jan 30 | ❌ Exit code 137 | | Jan 31 | ❌ No completion | | Feb 1 | ❌ No completion | | Feb 2 | ⏭️ Skipped - stale lock (PID 77703) | **This explains why the zksync v29.8.0 update wasn't auto-detected and applied.** ### Immediate Fix Needed 1. Kill stale process: `kill 77703` (if still running) 2. Remove lock: `rm /home/claudeagent/.version-update.lock` 3. Run version update manually or wait for next cron cycle ### Longer-term Investigation - Why is the version update timing out/OOMing? - May need to add memory limits or break into smaller batches

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: StakeSquid/ethereum-rpc-docker#2