Generic nitro v3.10.1 strictly refuses DA on a DAC=false chain, so DAC=true
override did not work (confirmed live: node still errored 'AnyTrust DA usage
set to false'). ApeChain's canonical config is DAC=false; its official build
public.ecr.aws/i6b2w2n6/nitro-node:apechain-v3.5.6 accepts DAC=false + AnyTrust
DAS. Revert DAC to false (canonical) + keep das rest-aggregator.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live node was 'forked', looping 'no AnyTrust reader configured for AnyTrust
message (header byte 0x88)'. ApeChain (Caldera) posts AnyTrust DA certs; the
earlier 'pure rollup' fix removed the DAS reader. Set chain-config
DataAvailabilityCommittee=true (matching every other AnyTrust chain in repo,
required by nitro v3.10.1 when DA enabled) and add das rest-aggregator
https://apechain.calderachain.xyz/rest-aggregator (from official Caldera node config).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OP Sepolia network-wide Karst activation. op-node v1.19.0 (pinned) and
conduit-op-reth:latest are Karst-aware; op-node v1.17.0 introduced Karst.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nitro crash-looped: 'AnyTrust DA usage for this chain is set to false but
--node.da.anytrust.enable is set to true'. apechain posts data to Arbitrum One,
not a DAS. Regenerated without DA flags; added parent-chain-is-arbitrum:true to
the chain-info. Derives batches from the Arbitrum One parent + sequencer feed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ApeChain chain 33139, parent Arbitrum One (42161), Caldera infra, $APE gas.
baseConfig.json chain-info from the official Constellation replica nodeConfig
(ArbOS 31, owner 0x5737..601c, full rollup contract set). das/feed/sequencer
point at Caldera; parent-chain = ARBITRUM_ONE_EXECUTION_RPC (no L1 beacon).
pruned-pebble-path + archive-pebble-hash profiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The doma --filter run regenerated this global aggregate from only the
doma subset, emptying it. Restore the complete file; doma resolves via
its dshackle label so no reference-endpoint entry is required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conduit OP-stack L2 on Ethereum mainnet L1. Canyon-at-genesis; op-geth
ingests the Conduit genesis.json directly (no op-reth hash pitfall).
Config served from Conduit slug doma-mainnet-qvzsfv8nv0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mirror the restore-volumes.sh --no-slowdisk capability for live/backup clones.
Both scripts gate the target /slowdisk static-file offload on the target's
SLOWDISK env (case-insensitive, matches the Python-templated 'True') and accept
a --no-slowdisk flag that forces the offload off for one run. When SLOWDISK is
on but the target /slowdisk is too small for the static files, the clone warns
and aborts, telling the operator to re-run with --no-slowdisk.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
find -type d skipped static dirs that are already offloaded (a symlink to /slowdisk is type
l, not d) — so show-static-file-size.sh reported zero static for an offloaded node (e.g. bob
on de-35), and backup-node.sh would drop them from the manifest (breaking re-offload on the
next restore). Match dirs AND symlinks now (root-level entries too).
show-static-file-size.sh also tags each static dir with its location:
[OFFLOADED -> /slowdisk/...], [on-disk], or [BROKEN SYMLINK ...].
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The SLOWDISK value is emitted by Python templates as capitalized booleans (True/False), so
match "True" (also accept manual lowercase "true"); anything else = offload off. --no-slowdisk
sets SLOWDISK=False to override.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Use the single SLOWDISK env var from .env (already set to "true" on the dedicated-extra-disk
hosts, e.g. us-35). Offload runs only when SLOWDISK is exactly the literal "true" (case
matters) and the --no-slowdisk flag was not passed. --no-slowdisk forces it off even when
SLOWDISK=true (extra disk full). Replaces the prior NO_SLOWDISK inversion.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Static-file offload gating, clearer semantics:
- NO_SLOWDISK defaults to TRUE (offload OFF) — safe default everywhere.
- A host with a real dedicated extra disk at /slowdisk sets NO_SLOWDISK=false in its .env
to ENABLE the offload.
- New --no-slowdisk CLI flag forces NO_SLOWDISK=true, overriding the .env false — for when
the extra disk exists but is full. Flag is parsed position-independently; the positional
args ($1 compose, $2 remote source) are preserved.
Offload runs only when NO_SLOWDISK is false AND the flag was not passed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A `/slowdisk` directory exists on hosts even when it is just a folder on the root disk
(no dedicated extra disk) — offloading there gives no benefit. Source the host .env and
require SLOWDISK to be set (operator sets it only on hosts with a real extra disk mounted
at /slowdisk) before activating the static-file -> /slowdisk symlink offload. Unset =
normal extract everywhere. Target path stays /slowdisk (the fixed in-container mount).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Verified static-file dirs across the reachable fleet via per-host cursor inspection
(de-13/14/16/22/27/30/31/32/35, us-16/41, uk-4). Findings:
- nitro freezers are network-prefixed (arbitrum-one/nitro/l2chaindata/ancient ... up to
1.3 TB), so the old chain/ and data/ prefixes matched nothing;
- missing: snapshots (erigon3/op-erigon/cosmos), lightchaindata/ancient, nested l2geth
geth/geth/chaindata/ancient, aztec archiver, nitro classic-msg/ancient;
- bare `ancient` matched nothing (all are nested).
List rewritten to canonical entries. Matching (backup-node.sh manifest + show-static-
file-size.sh) now: entry with NO slash = root-level only (so `snapshots` does NOT catch
postgres pg_logical/snapshots), entry WITH a slash = path-suffix via find -path "*/X"
(matches any prefix: network-prefixed nitro, nested datadirs). The manifest now records
the CONCRETE per-volume path so restore-volumes.sh recreates the exact symlink.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
restore-volumes.sh: pre-create static-file symlinks from the backup's .txt manifest so
the immutable ancient/freezer dirs land on /slowdisk (SSD) and extract THROUGH the
symlinks via tar --keep-directory-symlink (was --dereference, which clobbered them);
hot state stays on the primary disk. Cleans stale /slowdisk targets first (no leak on
re-restore). Safe fallbacks: no /slowdisk / no manifest / no static paths -> normal
extract. Reth excluded (reth dropped whole-dir static-file symlinks).
volume-utils.sh: add delete_slowdisk_targets_for_key() — follows a volume's symlinks and
sweeps the rpc_<key>__data_ pattern under /slowdisk (matches delete-volumes.sh).
cleanup-volumes.sh: free the /slowdisk static data before docker volume rm (was leaking),
and fix the fragile substring used/unused match to an exact name match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- op-geth: drop stray trailing ` \` on --hvm.genesisheader and
--tbc.leveldbhome (the backslash was decoded as part of the hVM
genesis header -> "invalid byte U+0020 ' '" crash)
- op-node: add OP_NODE_OVERRIDE_ECOTONE/CANYON/DELTA=1725868497
(rollup.json carries only regolith_time; without these the later
fjord override tripped "fork fjord set but prior fork ecotone missing")
- op-node: OP_NODE_BSS_WS no longer wrapped in literal quotes
- bump to Hemi's current pins (mainnet is past isthmus/holocene):
op-geth 05d4d8f, op-node 7c70d2d
Values mirror hemilabs/hemi-node mainnet scripts/gen.sh + config.json.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Consensus service renamed arc-testnet-consensus -> arc-testnet-node: it is now
generated by templates/nodes/malachite.yml (node: malachite) instead of being
hacked into the arc client template's indexer block. Container body unchanged.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
op-node's default engine-API RPC timeout is short; on nodes whose EL responds
slowly to engine_newPayload/forkChoiceUpdated (e.g. boba-mainnet op-reth pruned
during catch-up) op-node times out driving the EL and the execution head freezes
while op-node keeps deriving from L1. Raising the engine RPC timeout to 120s
prevents this class of stall fleet-wide.
Regenerated from env/op/node.env: adds the env var to every op-node service
(116 geth + 40 reth + 22 erigon composes); additions only, no other changes.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
genesis.json had no jovianTime, so op-geth/op-reth never activated Jovian
and op-node rejected Jovian-era L1-info txs ("unexpected length"). Added
jovianTime=1772548201 to the genesis config (EL source of truth for both
geth and reth). Removed OP_NODE_OVERRIDE_JOVIAN from the op-node env: the
local rollup.json already carries jovian_time, making the override redundant.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
op-reth rejects geth's --nodiscover flag, which the op-geth base template
emits for node_sync nodes. Use reth's --disable-discovery instead.
- hashkeychain-testnet: now node_sync/consensus-layer (was execution-layer
with an empty DB, which could not backfill) + --disable-discovery
- xlayer mainnet/testnet (archive+pruned): --nodiscover -> --disable-discovery
(these were latently broken on restart)
Fix lives in the op-reth.yml base template so all reth nodes are covered;
the redundant override in op-reth.boba.yml was removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Chain 53302 is legacy Canyon-at-genesis (block 0 has no Shanghai
withdrawalsRoot). Upstream op-reth always injects it and computes the wrong
genesis hash 0xfa6f21ae vs canonical 0x7274a90e, so it crash-loops against
the DB. conduit-op-reth has legacy-Canyon genesis compatibility for 53302.
Regenerated from context.yml (reth client_image override) + op-reth.yml
template (historicalrpc for superseed-sepolia fresh-sync backfill).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conduit's downloaded genesis.json/rollup.json are faulty (rollup.json
lacks chain_op_config + post-Fjord fork times, crashing op-node v1.19.0).
The fixed configs live in op/bob/sepolia/. Removed the client_genesis /
node_rollup_json Conduit URLs from context.yml and regenerated, so all 7
bob-sepolia composes (geth + reth) now bind-mount ./op/bob/sepolia:/config
and drop the curl-from-Conduit init containers.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
op-node was crashing post-Jovian and the rollup/genesis configs were missing
two fields present in the live sequencer (sepolia.superseed.xyz
optimism_rollupConfig):
- rollup.json: add pectra_blob_schedule_time=1742486400 (L1 Sepolia Pectra)
- genesis.json: eip1559DenominatorCanyon 0 -> 250 (match chain_op_config)
Verified against the live sequencer. Fixes op-node 'unexpected length: 178'
L1-info parse error. karst_time not yet published by the sequencer (Karst
Sepolia ~2026-06-17 16:00 UTC) - to be added once available.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conduit's rollup.json lacked chain_op_config and fork times required by op-node v1.19.0, causing a crash loop. Mount op/bob/sepolia locally (genesis + rollup) and drop Conduit init containers.
Co-authored-by: Cursor <cursoragent@cursor.com>
bob-sepolia is past the Jovian hardfork (jovian_time=1772548201). Conduit's
downloaded rollup.json omits jovian_time, so op-node misparsed the 178-byte
Jovian L1-info deposit txs ("data is unexpected length: 178") and the node
stalled at genesis. Add OP_NODE_OVERRIDE_JOVIAN to all bob-sepolia variants.
Remove op/bob.by: an unused local genesis/rollup backup. Source of truth is
the genesis/rollup downloaded from the Conduit URLs in context.yml.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Circle's arc-execution/arc-consensus images are distroless with only the
`arc` user and no `root` entry in /etc/passwd, so `user: root` failed with
"unable to find user root". Run by numeric uid 0:0 instead (no passwd
lookup needed; root can write the root-owned named volumes).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- superseed-sepolia: add holocene/isthmus/jovian fork times to rollup.json (CL)
and genesis.json (EL). jovian_time=1768978801 is superseed's chain-specific
activation (verified live via sepolia.superseed.xyz optimism_rollupConfig);
the official superseed-xyz/node repo + the old committed config both lacked it,
causing op-node "failed to parse L1 info deposit tx ... unexpected length: 178".
- op-reth v2.3.0 -> v2.3.1 across all op-reth composes (Karst-mandatory release,
Sepolia activation Jun 17 2026 16:00 UTC).
- op-sepolia: --chain=optimismsepolia -> optimism-sepolia (invalid chain spec
prevented op-reth from starting).
- soneium-minato-sepolia (reth/geth/erigon): L1 upstream vars now resolve to
ETHEREUM_SEPOLIA_* (were ETHEREUM_MINATO-SEPOLIA_*, nonexistent -> no L1 RPC);
state-cache shell var de-hyphenated; pectra blob-schedule override now emitted.
NOTE: superseed karst_time not yet published by the chain; must be added before
Jun 17 2026 once superseed announces it.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
- sync-status.sh: resolve protocol family from the compose x-upstreams chain
label via the registry (reference-rpc-endpoint.sh --protocol) instead of
path-substring guessing; legacy path detection kept as fallback for
composes without a resolved label. Unknown families report
'unsupported protocol: X' honestly instead of a bogus eth_chainId error.
Aztec reference lookup falls back to slug urls when rollup_version is
not in the registry.
- reference-rpc-endpoint.sh: new --chain <slug> (urls by registry key,
works for idless non-EVM entries) and --protocol <slug> modes; existing
chainid and --rollup-version lookups unchanged.
- mantle-sepolia: chainid 5001 -> 5003 (verified live: 0x138b), label and
--networkid now correct
- metis-sepolia: label resolves via registry override (drpc chains.yaml
carries wrong id 59901; live chain is 59902, verified via official RPC)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- aztec-testnet, solana, ripple x-upstreams now carry their drpc slug as
chain label (was empty -> dshackle ignored the upstreams)
- reference-rpc-endpoint.json: every chain now has a protocol field
(eth/bitcoin/solana/ripple/aztec/...) for protocol-family dispatch in
scripts, plus https://{slug}.drpc.org appended as LAST fallback url
(independent endpoints stay preferred; overrides win entirely)
- registry grew 247 -> 333 entries (all drpc protocols imported, not just eth)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Aztec testnet rollup was redeployed on Sepolia; 3.0.2 computes the old
genesis archive root and crash-loops. 4.3.1 matches the deployed rollup
and bootstraps from snapshot. Verified live on rpc-de-13.
Includes regen drift from current templates for other stale compose files.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The old script grouped backups with cut -d'-' -f1-3, which mis-grouped
volume keys and permanently deleted ~360 archives on 2026-06-11. Replace
with safe retention: group by full volume key regex, keep 3 newest per key,
move excess to /backup/trash, and protect uploading-*, broken_*, and
deprecated-* prefixes.
Co-authored-by: Cursor <cursoragent@cursor.com>
Ronin migrated from standalone sidechain to OP-stack L2 operated by
Conduit, with EigenDA V2 for data availability. New composes follow
conduitxyz/ronin-migration-reth-docker: conduit-op-reth fork +
op-node v1.16.5 + eigenda-proxy 2.7.0 relay.
- op/ronin/{mainnet,saigon}: Conduit genesis (bedrockBlock patched to
55577500 / 45528550) and rollup.json (alt_da GenericCommitment injected)
- saigon is a NEW chain post-migration: chainid 202601, L1 sepolia
- also regenerated: xlayer-testnet composes (dshackle chain label was
corrupted to 'ronin' by an override typo), native ronin/ronin composes
(upstream labels follow new chain ids), reference-rpc-endpoint.json
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Superchain Sepolia hardfork ~2026-06-19 drops op-geth support; op-erigon is
already broken post-hardfork. op-reth is the migration target for Superchain
members. Generates archive-trace, minimal-trace, and pruned-trace variants
via the 'full_minimal_trace' profile (mirroring base-sepolia / op-sepolia).
Pre-existing template change: bind-mount './op/celo/sepolia:/config' migrated to
parameterizable named docker volumes (CELO_SEPOLIA_OP_GETH_*_CONFIG /
CELO_SEPOLIA_OP_GETH_*_NODE_CONFIG with sensible defaults). Surfaced when
running update.sh on top of the current rpc/ submodule HEAD.
