restore/cleanup: implement static-file -> /slowdisk offload + free it on removal

restore-volumes.sh: pre-create static-file symlinks from the backup's .txt manifest so
the immutable ancient/freezer dirs land on /slowdisk (SSD) and extract THROUGH the
symlinks via tar --keep-directory-symlink (was --dereference, which clobbered them);
hot state stays on the primary disk. Cleans stale /slowdisk targets first (no leak on
re-restore). Safe fallbacks: no /slowdisk / no manifest / no static paths -> normal
extract. Reth excluded (reth dropped whole-dir static-file symlinks).

volume-utils.sh: add delete_slowdisk_targets_for_key() — follows a volume's symlinks and
sweeps the rpc_<key>__data_ pattern under /slowdisk (matches delete-volumes.sh).

cleanup-volumes.sh: free the /slowdisk static data before docker volume rm (was leaking),
and fix the fragile substring used/unused match to an exact name match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-06-15 03:05:04 +00:00
parent f39e09dac0
commit 345538954d
3 changed files with 110 additions and 52 deletions

View File

@@ -1,3 +1,8 @@
#!/bin/bash
# List (default) or, with --remove-from-disk, delete the docker volumes that are NOT
# referenced by the current COMPOSE_FILE. Removal also frees the static-file data that was
# offloaded to /slowdisk behind the volume's symlinks (delete_slowdisk_targets_for_key),
# so nothing leaks on /slowdisk.
BASEPATH="$(dirname "$0")"
source $BASEPATH/.env
source $BASEPATH/volume-utils.sh
@@ -9,7 +14,7 @@ used_volumes=()
for part in "${parts[@]}"; do
volumes=$(get_volume_keys "$BASEPATH/$part")
# Convert volumes to an array
# Prefix each compose volume key with rpc_ to match docker's volume names.
prefix="rpc_"
IFS=$'\n' read -r -d '' -a volumes_array <<< "$(printf "%s\n" "${volumes[@]}" | sed "/^$/! s/^/$prefix/")"
@@ -18,21 +23,29 @@ done
on_disk=($(docker volume ls --format '{{.Name}}' | grep '^rpc_'))
unused_volumes=()
# A volume counts as "used" only on an EXACT name match. The previous substring test
# ([[ "${used_volumes[@]}" =~ "$element" ]]) could mis-classify a volume whose name is a
# substring of another (e.g. ...-pruned vs ...-pruned-trace) and wrongly purge a live one.
is_used() {
local v=$1 u
for u in "${used_volumes[@]}"; do
[[ "$u" == "$v" ]] && return 0
done
return 1
}
unused_volumes=()
for element in "${on_disk[@]}"; do
# Check if the element exists in array2
if [[ ! "${used_volumes[@]}" =~ "$element" ]]; then
# If not, add it to the difference array
unused_volumes+=("$element")
fi
is_used "$element" || unused_volumes+=("$element")
done
if [ "$1" = "--remove-from-disk" ]; then
# Iterate over volumes in the difference array and remove them from disk
for volume in "${unused_volumes[@]}"; do
docker volume rm "$volume"
done
# Remove each unused volume AND the /slowdisk static data behind its symlinks.
for volume in "${unused_volumes[@]}"; do
echo "removing unused volume: $volume"
delete_slowdisk_targets_for_key "${volume#rpc_}"
docker volume rm "$volume"
done
else
printf '%s\n' "${unused_volumes[@]}"
fi