Reduce stages.transaction_lookup.chunk_size from 5M to 500k in reth.toml
to prevent multi-hour pipeline stalls during large checkpoint windows on
op-reth-pruned-trace (rpc-de-32).
Mirror vanilla reth.yml's client_command_dbopts ({{ '- --full' if db_type in
['pruned','minimal'] else '' }}): the arc execution client now omits --full for the
archive variant (full history) and keeps it for pruned/minimal. Previously --full was
hardcoded so both variants ran identical pruned-sync flags.
- EL client renamed arc -> reth (it IS a reth fork); template arc.yml -> reth.arc.yml
so find_template picks it as the arc-network reth override. Path is now
arc/reth/arc-testnet-reth-{pruned,archive}-trace.yml (was arc/arc/arc-testnet-arc-full).
- profiles: full -> pruned[trace] + archive[trace] (there is no 'full' node type).
- removed the {{short_name}}-snapshots init container (arc-snapshots download, ~84 GB) and
the consensus-init depends_on that pointed at it. Chain data is seeded OUT-OF-BAND, never
via a compose service. Services: arc-testnet (EL) / arc-testnet-node (malachite CL) +
consensus-init / sockets-init.
- git rm orphaned arc/arc/arc-testnet-arc-full.yml.
kona's NodeMode enum (crates/node/service/src/service/mode.rs) has only
Validator/Sequencer and parses via FromStr against the PascalCase display
strings. There is no 'verifier' mode, so the old value failed to parse and
kona-node crash-looped on startup. Validator is also kona's default.
Source change is env/op/kona.env (global default for all op-stack kona nodes),
so this regenerates KONA_NODE_MODE across all 38 op/*/...--kona compose files.
Add PRECONFIRMATION_SERVER_JWT_SECRET and --preconfirmation.jwtSecret=/jwtsecret
to archive hoodi driver services so taiko-client v2.4.0 can start the
preconfirmation block server when PRECONFIRMATION_SERVER_PORT is enabled.
Co-authored-by: Cursor <cursoragent@cursor.com>
Update EL (op-reth) and CL (op-node) image defaults in all seven
base-mainnet compose profiles. Also aligns OP_NODE_RPC_PORT with
env/op/node.env (9545) on regen.
Co-authored-by: Cursor <cursoragent@cursor.com>
Rename SHASTA_INBOX to INBOX (v2.4.0 reads --inbox from INBOX env var
and no longer honors PACAYA_INBOX/SHASTA_INBOX). Remove unused PACAYA_INBOX.
Co-authored-by: Cursor <cursoragent@cursor.com>
In xlayer/op-node:v0.1.5.1, the =true path triggers an infinite
'Found highest L2 block with canonical L1 origin. Skip further sanity check
and jump to the safe head' loop that prevents block derivation entirely.
Setting =false restores normal startup: op-node advances through the check
and begins deriving blocks. No newer op-node image exists (v0.1.5.1 is latest).
Affects all xlayer-mainnet op-node variants (geth, reth) as they share
the same op-node consensus layer.
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
- Update heimdall init.sh with heimdall-109 chain_id and mainnet seeds
- Update cometbft.Dockerfile with v1.0.7-bone version
- Add bor.Dockerfile for building shibaone/bor from source at v1.3.9-bone
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Add cronosd cometbft.Dockerfile and init.sh for Cronos EVM mainnet.
- Dockerfile: layer cometbft-common.sh + init.sh onto upstream cronos image
- init.sh: adapted from haqq pattern with EVM JSON-RPC on 8545, WS on 8546,
CometBFT RPC on 26657, P2P on 10521, chain-id cronosmainnet_25-1
- Statesync via ct_configure_statesync, genesis from official repo
- Pruning: custom with keep-recent=100, interval=19
Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
Conduit's testnet rollup.json (doma-dev-ix58nm4rnd) omits chain_op_config,
crashing op-node v1.19.0 with 'cfg.Rollup.ChainOpConfig is nil'. Vendor a
complete rollup.json (chain_op_config eip1559 6/50/250, confirmed identical
in mainnet rollup.json and testnet genesis optimism config) plus the
post-fjord fork times baked in, mounted via custom_config so op-node uses
the local file instead of curling Conduit's incomplete one.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Doma testnet (Conduit slug doma-dev-ix58nm4rnd) on Conduit's purpose-built
conduit-op-reth image - better-tested for Conduit chains than upstream op-geth.
Fork times granite/holocene/isthmus/jovian (missing from Conduit's bare
rollup.json) supplied via env/op/doma/testnet/node.env. Experiment to validate
a clean canonical sync before redoing mainnet on op-reth.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conduit's bare rollup.json for doma-mainnet omits jovian_time entirely, but the
EL genesis activates Jovian at 1769065201. Without the override op-node has no
Jovian fork time and mis-derives at the boundary (chain forks off canonical).
Same fix pattern as bob-sepolia. Override comes from env/op/doma/mainnet/node.env.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bepolia activated the Fusaka/Fulu hard fork at 2026-05-27 16:00 UTC (unix
1779897600), which ships new EVM-inflation-withdrawal values. beacon-kit
v1.3.4 (pre-fork) crash-loops replaying the fork-boundary block 20513284:
'first withdrawal is not the EVM inflation withdrawal' (RestartCount 3742 on
us-41). The EL (bera-reth v1.4.0) already speaks the Fusaka engine API
(newPayloadV4P11 etc.) and accepted the post-fork block, so only the CL is
behind. v1.4.0-rc3 sets the Fulu fork time + new inflation values. Applies
to all berachain chains' beacon-kit default; only bepolia is deployed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The container ran as user 0:994 and accessed the docker socket via group
membership, but the host docker group GID is auto-assigned and varies per
host (e.g. uk-8 is 988, not 994), so the hardcoded gid silently breaks
telegraf's docker input wherever it differs (uk-8 was in a restart loop:
permission denied on /var/run/docker.sock). Run as root (0:0) with
entrypoint [telegraf] to skip the image's gosu privilege-drop, so telegraf
reads the socket as its owner regardless of the host docker gid. Works
uniformly fleet-wide; no regression on hosts where the gid happened to match.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
op-geth was stuck at genesis (block 0, 0 L2 peers for 31h) because execution-layer
snap-sync had no peers. node_sync=true -> OP_NODE_SYNCMODE=consensus-layer (derive
L2 from L1) + geth --syncmode=full. Diagnosed by cursor on de-13.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
netdev (netlink) was reading the container's own veth (idle) -> node_network_*
showed ~0 on every host. Host network lets it see real host interfaces, so
bandwidth (egress vs tariff quota) works. Removed networks/expose (exclusive
with network_mode:host).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds diskIO to cadvisor --enable_metrics so per-container disk read/write
(container_fs_reads/writes_bytes_total) is exposed — lets node attribution
name which node is the noisy IO neighbor, not just flag the host.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
prometheus-scrape.metrics_path was '/' which made prometheus-docker-sd scrape
the HTML root and fail with 'INVALID is not a valid start token', leaving the
targets up=0. Fixes per-container (cadvisor) + host (node_exporter) metrics so
they can be wired into the DRPC insights MCP for per-node resource attribution.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Mint a Viewer Grafana service-account token on a host and print it to stdout,
so each box's local Grafana (dshackle + node_exporter + cadvisor) can be wired
into the DRPC insights MCP (grafana_servers.yaml). Idempotent SA, fresh token
per call.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Match the dRPC gateway's per-chain "how many blocks behind is ok" model instead of a
fixed 2s/5s timestamp tolerance:
- check-health.sh: compare the reference head vs local head by BLOCK NUMBER and classify
with the chain's dRPC lag thresholds (LAGGING_LAG/SYNCING_LAG, in blocks, from
chains.yaml). dRPC uses the two thresholds inconsistently across chains (sometimes
lagging<syncing, sometimes the reverse) so the smaller is the online boundary and the
larger the syncing/drop boundary. Defaults 2/6 when a chain has no thresholds.
- multicurl.sh: also skip responses with result:null (a lagging endpoint lacking the
requested block) so the fallback reference URLs are actually tried. Previously the first
endpoint's {"result":null} was accepted as success -> fallbacks never ran, and the null
reference hash made check-health report false "forked" (the online/forked flapping).
- sync-status.sh: resolve the lag thresholds (by drpc slug or chain id) and export
LAGGING_LAG/SYNCING_LAG.
- reference-rpc-endpoint.sh: add --lags and --block-time-ms lookups.
- reference-rpc-endpoint.json: regenerated with per-chain block_time_ms + lagging_lag +
syncing_lag (additive).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Minimal BSC nodes lack the state to serve eth_getProof and return
"header not found" (especially while catching up after a restart). The
drpc gateway probes eth_getProof and marks the WHOLE upstream unavailable
on failure, so e.g. bsc-mainnet-bsc-minimal on us-32 showed unavailable in
US-East despite the node being online and at head.
Disable eth_getProof on bsc minimal upstreams only (network=bsc,
db_type=minimal) so they stay available for every other method. Archive/
pruned bsc and all other nodes keep serving getProof.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
X Layer mainnet/testnet op-reth now use the OKX xlayer/op-reth:v0.0.4.1 fork
(generic op-reth can't read the X Layer DB format), the image's built-in
xlayer-<network> chain spec, --rpc.legacy-url for post-snapshot history gap-fill,
and the OKX archive-compatible reth.toml (light pruning only: merkle_changesets
distance=10064). Switched the reth profile from full_trace to archive_trace and
deleted the pruned variants — a "pruned" compose over archive snapshot data
crash-loops on a block-height mismatch. Requires the official OKX reth snapshot
pre-loaded into the volume; do NOT sync from scratch.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
op-node was following the sequencer tip in execution-layer mode while op-reth stayed
at genesis (blocks buffered, never canonical). Switch to consensus-layer sync so
op-node drives L1 derivation, add op-reth --disable-discovery, and use debug_geth
(eth_getBlockReceipts) for the L1 RPC kind — "basic" per-tx receipts hit "receipt 0
has unexpected nil block number" on the Sepolia L1 endpoint. OP_NODE_OVERRIDE_JOVIAN
=1772548201 is belt-and-suspenders (jovian_time already in rollup.json). Applies to
all bob-sepolia EL variants. Regenerated from the vibe-node generator.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
op-reth/op-geth sepolia now run op-node consensus-layer sync (empty DB is never
P2P-backfilled) with --disable-discovery, derive L1 via debug_geth against an
operator-set full-history Sepolia RPC (SUPERSEED_SEPOLIA_L1_EXECUTION_RPC, falling
back to ETHEREUM_SEPOLIA_EXECUTION_RPC), and serve a genesis.json with the invalid
top-level excessBlobGas/blobGasUsed nulls removed (eip1559DenominatorCanyon=250
retained). Regenerated from the vibe-node generator.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Per cursor cosmos handoff. (1) Traefik WS router rule Headers(Upgrade,websocket) is
case-SENSITIVE -> clients sending 'Upgrade: WebSocket' (python websocket-client) fall
through to the RPC router (200/400 not 101). rpc-client.yml now emits
HeadersRegexp(Upgrade,(?i)websocket) for all split-WS chains; regenerated cosmos+avalanche.
(2) Refactor into cometbft-common.sh: ct_configure_statesync skips if data/application.db
exists; new ct_ensure_wasm seeds CosmWasm/IBC-08 wasm (statesync omits them). init.sh calls
both. README documents wasm/version/WS gotchas. 45 other split-WS composes get HeadersRegexp
on their next regen.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Our L1 geth (e.g. rpc-de-31 ethereum-mainnet-geth) returns null for the
per-tx receipt path on historical blocks but serves eth_getBlockReceipts
fine. op-node's 'basic' kind only uses the per-tx batch, so chains
deriving from genesis (e.g. Katana) hit the broken path immediately;
chains already synced past it don't. 'standard' uses eth_getBlockReceipts
and keeps the per-tx batch as fallback (strict superset of 'basic'), so
it is safe for every OP-stack node.
Flips OP_NODE_L1_RPC_KIND default across all 176 op-node compose files.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
dshackle's WS head subscription hit the RPC root (got HTTP 200, not a WS upgrade).
Set client_ws_path=/websocket -> rpc-client.yml emits the avalanche-style priority-100
WS router (Header Upgrade:websocket -> replacepath /websocket) so WS reaches CometBFT's
/websocket. RPC router unchanged (stripprefix -> root).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The polkachu seed was refusing -> 0 peers -> statesync couldn't discover snapshots.
Use client_staticpeers (cosmos-hub is a CLIENT in our stack) with 8 live peers from
polkachu /net_info. Template now reads client_staticpeers/client_bootnodes (standard attrs).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CometBFT config.toml uses underscore keys; the sed targeted hyphens (rpc-servers etc.)
so they never matched -> rpc_servers stayed empty -> 'at least 2 RPC servers required,
got 0'. Now section-anchored to [statesync] + [_-] tolerant. (This is also why sei
never held chainhead — same hyphen bug in its bespoke init.)
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
CometBFT RPC :26657 (the dshackle upstream) is served via rpc.laddr regardless;
minimal 'gaiad start --home --minimum-gas-prices' is the valid invocation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ct_configure_statesync read .block.header.height (null — CometBFT wraps in .result) so
statesync silently skipped -> gaiad fell back to genesis replay (panic). Use
'.result.X // .X' (robust to wrapped/unwrapped). Same fallback in check-health --cosmos.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
First pure-cosmos chain (validates the §2 cometbft-common.sh lib via real gaiad init).
Single-binary CometBFT client serving :26657 (dshackle chain=cosmos-hub). Statesync
bootstrap (polkachu) since genesis-replay across gaia governance upgrades is impractical.
gzipped genesis. v25.3.2 (live chain version; registry's v27.4.0 not on ghcr). Params
(chain_id/genesis_url/statesync_rpc/seeds/min_gas) from context.yml.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Cosmos-hub has NO EVM RPC — eth_blockNumber checks don't apply. Add a --cosmos
handler that probes the CometBFT /status method: sync_info.catching_up=false =>
online, else syncing; optional head-gap check vs the drpc reference. sync-status.sh
dispatches protocol=cosmos -> check-health.sh --cosmos. eth/starknet/aztec untouched.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
morphnode panics (open /db/data/...: no such file) without the data dir + initial
validator state. EL healthy; this unblocks the CL.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
morphnode --mainnet does NOT self-generate config (crash: Config File not found in
/db/config). Add init container that seeds the committed morph/mainnet/node-data/config/
(config.toml + genesis.json, from run-morph-node) into /db/config (no-clobber). Node
renamed morph-node -> morph (cleaner; vol _morph, MORPH_MAINNET_MORPH_* env).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
EL morph-geth (go-ethereum 2.2.3, --morph --morph-mpt, archive leveldb/hash/mpt,
pure EL driven via engine API) + CL morph-node (node:0.5.7, env-driven, derives from
eth-mainnet L1). Params from authoritative morph-l2/run-morph-node. Fresh L1 derivation.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Generalizes the berachain beacon-kit init pattern into cometbft-common.sh (sourced
helper lib) so morph-node, gaiad/cosmos batch, shibarium and the EVM-cosmos chains
(haqq/sei/zero-gravity) become thin init.sh overrides. Validated against the real
init flows of beacon-kit, haqq, sei, zero-gravity. README.cometbft.md documents the
pattern (per-chain Dockerfile + thin init.sh + context). berachain stays on its own
beacon-kit.yml (untouched).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Generic nitro v3.10.1 strictly refuses DA on a DAC=false chain, so DAC=true
override did not work (confirmed live: node still errored 'AnyTrust DA usage
set to false'). ApeChain's canonical config is DAC=false; its official build
public.ecr.aws/i6b2w2n6/nitro-node:apechain-v3.5.6 accepts DAC=false + AnyTrust
DAS. Revert DAC to false (canonical) + keep das rest-aggregator.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Live node was 'forked', looping 'no AnyTrust reader configured for AnyTrust
message (header byte 0x88)'. ApeChain (Caldera) posts AnyTrust DA certs; the
earlier 'pure rollup' fix removed the DAS reader. Set chain-config
DataAvailabilityCommittee=true (matching every other AnyTrust chain in repo,
required by nitro v3.10.1 when DA enabled) and add das rest-aggregator
https://apechain.calderachain.xyz/rest-aggregator (from official Caldera node config).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
OP Sepolia network-wide Karst activation. op-node v1.19.0 (pinned) and
conduit-op-reth:latest are Karst-aware; op-node v1.17.0 introduced Karst.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
nitro crash-looped: 'AnyTrust DA usage for this chain is set to false but
--node.da.anytrust.enable is set to true'. apechain posts data to Arbitrum One,
not a DAS. Regenerated without DA flags; added parent-chain-is-arbitrum:true to
the chain-info. Derives batches from the Arbitrum One parent + sequencer feed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ApeChain chain 33139, parent Arbitrum One (42161), Caldera infra, $APE gas.
baseConfig.json chain-info from the official Constellation replica nodeConfig
(ArbOS 31, owner 0x5737..601c, full rollup contract set). das/feed/sequencer
point at Caldera; parent-chain = ARBITRUM_ONE_EXECUTION_RPC (no L1 beacon).
pruned-pebble-path + archive-pebble-hash profiles.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The doma --filter run regenerated this global aggregate from only the
doma subset, emptying it. Restore the complete file; doma resolves via
its dshackle label so no reference-endpoint entry is required.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Conduit OP-stack L2 on Ethereum mainnet L1. Canyon-at-genesis; op-geth
ingests the Conduit genesis.json directly (no op-reth hash pitfall).
Config served from Conduit slug doma-mainnet-qvzsfv8nv0.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
claude
