static-file list: fleet-verified rewrite + root-vs-suffix matching

Verified static-file dirs across the reachable fleet via per-host cursor inspection
(de-13/14/16/22/27/30/31/32/35, us-16/41, uk-4). Findings:
- nitro freezers are network-prefixed (arbitrum-one/nitro/l2chaindata/ancient ... up to
  1.3 TB), so the old chain/ and data/ prefixes matched nothing;
- missing: snapshots (erigon3/op-erigon/cosmos), lightchaindata/ancient, nested l2geth
  geth/geth/chaindata/ancient, aztec archiver, nitro classic-msg/ancient;
- bare `ancient` matched nothing (all are nested).

List rewritten to canonical entries. Matching (backup-node.sh manifest + show-static-
file-size.sh) now: entry with NO slash = root-level only (so `snapshots` does NOT catch
postgres pg_logical/snapshots), entry WITH a slash = path-suffix via find -path "*/X"
(matches any prefix: network-prefixed nitro, nested datadirs). The manifest now records
the CONCRETE per-volume path so restore-volumes.sh recreates the exact symlink.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 03:22:26 +00:00
parent 345538954d
commit a9e8fba794
3 changed files with 38 additions and 22 deletions

View File

@@ -36,15 +36,24 @@ generate_volume_metadata() {
# Check each static file path
if [[ -f "$static_file_list" ]]; then
while IFS= read -r path; do
# Check if the path exists
if [[ -e "$prefix/_data/$path" ]]; then
# Get the size
size=$(du -sL "$prefix/_data/$path" 2>/dev/null | awk '{print $1}')
# Format size in human-readable format
size_formatted=$(echo "$(( size * 1024 ))" | numfmt --to=iec --suffix=B --format="%.2f")
# Write to metadata file
echo "$size_formatted $path" >> "$metadata_file"
[[ -z "$path" ]] && continue
# Match rule (see static-file-path-list.txt): an entry with NO slash is root-level
# only (so `snapshots` won't catch postgres pg_logical/snapshots); an entry WITH a
# slash is a path-suffix, so any prefix matches (network-prefixed nitro freezers,
# nested l2geth geth/geth/...). Record the CONCRETE relative path so restore-volumes
# can recreate the exact static-file -> /slowdisk symlink.
local matches=() m rel
if [[ "$path" == */* ]]; then
while IFS= read -r m; do matches+=("$m"); done < <(find "$prefix/_data" -type d -path "*/$path" 2>/dev/null)
else
[[ -d "$prefix/_data/$path" ]] && matches+=("$prefix/_data/$path")
fi
for m in "${matches[@]}"; do
rel="${m#"$prefix/_data/"}"
size=$(du -sL "$m" 2>/dev/null | awk '{print $1}')
size_formatted=$(echo "$(( ${size:-0} * 1024 ))" | numfmt --to=iec --suffix=B --format="%.2f")
echo "$size_formatted $rel" >> "$metadata_file"
done
done < "$static_file_list"
fi
}

View File

@@ -26,16 +26,22 @@ for key in $keys; do
# Skip empty lines
[[ -z "$path" ]] && continue
# Check if the path exists
if [[ -e "$prefix/_data/$path" ]]; then
# Print the size of the file or directory
size=$(du -sL "$prefix/_data/$path" 2>/dev/null | awk '{print $1}')
static_size=$((static_size + size))
# Format size in human-readable format
size_formatted=$(echo "$(( size * 1024 ))" | numfmt --to=iec --suffix=B --format="%.2f")
# Print the detected path with size to stderr (one per line)
echo "$size_formatted $prefix/_data/$path" >&2
# Match rule: an entry with NO slash is root-level only (depth-1 under _data),
# so e.g. `snapshots` won't catch postgres pg_logical/snapshots. An entry WITH a
# slash is a path-suffix, so any prefix matches (network-prefixed nitro freezers
# like arbitrum-one/nitro/l2chaindata/ancient, nested l2geth geth/geth/...).
matches=()
if [[ "$path" == */* ]]; then
while IFS= read -r d; do matches+=("$d"); done < <(find "$prefix/_data" -type d -path "*/$path" 2>/dev/null)
else
[[ -d "$prefix/_data/$path" ]] && matches+=("$prefix/_data/$path")
fi
for m in "${matches[@]}"; do
size=$(du -sL "$m" 2>/dev/null | awk '{print $1}')
static_size=$((static_size + ${size:-0}))
size_formatted=$(echo "$(( ${size:-0} * 1024 ))" | numfmt --to=iec --suffix=B --format="%.2f")
echo "$size_formatted $m" >&2
done
done < "$static_file_list"
fi
done

View File

@@ -1,6 +1,7 @@
static_files
chain/nitro/l2chaindata/ancient
data/nitro/l2chaindata/ancient
ancient
geth/chaindata/ancient
bor/chaindata/ancient
snapshots
archiver
chaindata/ancient
lightchaindata/ancient
nitro/l2chaindata/ancient
nitro/classic-msg/ancient