From 92cdb31d17bf4984b8a77bcffcadad3266391a7c Mon Sep 17 00:00:00 2001 From: Claude Agent Date: Wed, 12 Aug 2026 19:33:48 +0000 Subject: [PATCH] zero-gravity: init guard must check the CONFIG volume, not a data-volume proxy MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit aristotle @ rpc-de-32 crash-looped after its build was fixed: priv_validator_state.json found in /root/.0g. Continuing! Already initialized, continuing! panic: ProvideBlsSigner (node-core/components/signer.go:46): key file does not exist at path: /root/.0g/config/priv_validator_key.json $DATA_DIR and $CONFIG_DIR are SEPARATE docker volumes (_zerog and _config), but the guard tested only $DATA_DIR/priv_validator_state.json — inferring the state of the config volume from a file on the data volume. Worse, the copy order wrote that guard file BEFORE the two key files, so an init interrupted between them left the data volume with the guard and the config volume without the keys. Every later start then took the 'Already initialized' branch, copied nothing, and panicked. Unrecoverable by restarting: the node wedges permanently. aristotle got there via the repeated build failures earlier today. Fix, two parts: - guard on everything 0gchaind needs to boot (both config keys AND the data state file), not one proxy file on the wrong volume; - write the keys FIRST and the guard file LAST, so an interrupted init re-runs on the next start instead of latching into the wedged state. Tested locally against the real 0gchaind binary (aristotle v1.0.6), sandboxed HOME: scenario before after fresh volumes keys created keys created (no regression) wedged (state, no keys) keys MISSING keys created (unwedges) partial (state+node_key) keys MISSING keys created healthy (all present) no re-init no re-init, and a SENTINEL written into priv_validator_key.json survives -> never clobbers an initialized node The 'wedged' row reproduces aristotle's production failure exactly. Note this regenerates node identity on an affected node (node_key.json, priv_validator_key.json). These are RPC nodes that never sign, so a fresh identity is harmless; chain data on the _zerog volume is untouched. Co-Authored-By: Claude Opus 5 (1M context) --- zero-gravity/scripts/init.sh | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-) diff --git a/zero-gravity/scripts/init.sh b/zero-gravity/scripts/init.sh index c79c6457..5b67b027 100644 --- a/zero-gravity/scripts/init.sh +++ b/zero-gravity/scripts/init.sh @@ -61,23 +61,41 @@ env # seems to be the same for all the 0g chains -if [ ! -f "$DATA_DIR/priv_validator_state.json" ]; then - echo "priv_validator_state.json not found in $HOME_DIR. Proceeding with initialization steps..." +# $DATA_DIR and $CONFIG_DIR are SEPARATE docker volumes (_zerog and _config), +# so the presence of a file in one says nothing about the other. Guard on everything +# 0gchaind actually needs to boot, and write the copies so the LAST file written is the +# one that would make a re-run skip. +# +# The previous version guarded solely on $DATA_DIR/priv_validator_state.json and copied it +# BEFORE the two key files. An init interrupted between those copies therefore left the +# data volume with the guard file and the config volume without the keys, so every later +# start took the "Already initialized" branch, copied nothing, and 0gchaind panicked: +# +# panic: error calling provider ProvideBlsSigner (node-core/components/signer.go:46): +# key file does not exist at path: /root/.0g/config/priv_validator_key.json +# +# That state is unrecoverable by restarting - the node wedges permanently. Hit on +# aristotle @ rpc-de-32 on 2026-08-12 after repeated build failures interrupted its init. +if [ ! -f "$CONFIG_DIR/priv_validator_key.json" ] || \ + [ ! -f "$CONFIG_DIR/node_key.json" ] || \ + [ ! -f "$DATA_DIR/priv_validator_state.json" ]; then + echo "0g: node identity incomplete (config keys and/or priv_validator_state.json). Initializing..." TMP_DIR=$(mktemp -d) # You can add any additional initialization logic here if needed if /0g/bin/0gchaind init ${MONIKER} --chaincfg.chain-spec ${CHAIN_SPEC} --home $TMP_DIR; then cp -r /0g/0g-home/0gchaind-home/config/* $CONFIG_DIR - cp $TMP_DIR/data/priv_validator_state.json $DATA_DIR + # Keys first, guard file last: if this is interrupted the next start re-runs + # initialization instead of latching into the wedged state described above. cp $TMP_DIR/config/node_key.json $CONFIG_DIR cp $TMP_DIR/config/priv_validator_key.json $CONFIG_DIR + cp $TMP_DIR/data/priv_validator_state.json $DATA_DIR else echo "Already initialized, continuing!" >&2 fi rm -rf $TMP_DIR # delete tmp dir else - echo "priv_validator_state.json found in $HOME_DIR. Continuing!" >&2 - echo "Already initialized, continuing!" >&2 + echo "0g: node identity complete (config keys + priv_validator_state.json). Continuing!" >&2 fi exec /0g/bin/0gchaind $@