Commit Graph

32 Commits

Author SHA1 Message Date
rob
6bb0b19f45 Harden restore-volumes.sh against silent restore truncation and incomplete-download skips
- BUG 1: Add error checking after tar extraction for both LOCAL and REMOTE-CACHE branches
  - Check exit status of tar -I zstd -xf commands
  - Print error to stderr and exit non-zero on failure
  - Prevents silent truncation where corrupt/incomplete backup extracts partial data
  - Mirrors existing remote-STREAM branch error handling
- BUG 2: Fix REMOTE branch to resume incomplete aria2c downloads
  - Check for presence of <file>.aria2 control file as incomplete signal
  - aria2c -c continues/resumes download when .aria2 file exists
  - Only skip download when file exists AND no .aria2 control file remains
  - aria2 deletes .aria2 sidecar on successful completion, making it a reliable signal
- Maintain all existing flags: aria2c -c -Z -x8 -j8 -s8 -d
- Preserve reth guard logic and static-file offload behavior unchanged

Generated by Mistral Vibe.
Co-Authored-By: Mistral Vibe <vibe@mistral.ai>
2026-06-17 16:22:50 +00:00
57b5757a85 restore: SLOWDISK gate matches Python-templated "True" (capitalized)
The SLOWDISK value is emitted by Python templates as capitalized booleans (True/False), so
match "True" (also accept manual lowercase "true"); anything else = offload off. --no-slowdisk
sets SLOWDISK=False to override.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 04:58:13 +00:00
6c110c08ed restore: gate offload on SLOWDISK=true (exact) with --no-slowdisk override
Use the single SLOWDISK env var from .env (already set to "true" on the dedicated-extra-disk
hosts, e.g. us-35). Offload runs only when SLOWDISK is exactly the literal "true" (case
matters) and the --no-slowdisk flag was not passed. --no-slowdisk forces it off even when
SLOWDISK=true (extra disk full). Replaces the prior NO_SLOWDISK inversion.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 04:50:06 +00:00
b9efcfe34d restore: invert gate to NO_SLOWDISK (default true) + add --no-slowdisk override flag
Static-file offload gating, clearer semantics:
- NO_SLOWDISK defaults to TRUE (offload OFF) — safe default everywhere.
- A host with a real dedicated extra disk at /slowdisk sets NO_SLOWDISK=false in its .env
  to ENABLE the offload.
- New --no-slowdisk CLI flag forces NO_SLOWDISK=true, overriding the .env false — for when
  the extra disk exists but is full. Flag is parsed position-independently; the positional
  args ($1 compose, $2 remote source) are preserved.

Offload runs only when NO_SLOWDISK is false AND the flag was not passed.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 04:43:47 +00:00
3caa4ab873 restore: gate static-file offload on SLOWDISK env var (.env), not just /slowdisk presence
A `/slowdisk` directory exists on hosts even when it is just a folder on the root disk
(no dedicated extra disk) — offloading there gives no benefit. Source the host .env and
require SLOWDISK to be set (operator sets it only on hosts with a real extra disk mounted
at /slowdisk) before activating the static-file -> /slowdisk symlink offload. Unset =
normal extract everywhere. Target path stays /slowdisk (the fixed in-container mount).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 04:39:48 +00:00
345538954d restore/cleanup: implement static-file -> /slowdisk offload + free it on removal
restore-volumes.sh: pre-create static-file symlinks from the backup's .txt manifest so
the immutable ancient/freezer dirs land on /slowdisk (SSD) and extract THROUGH the
symlinks via tar --keep-directory-symlink (was --dereference, which clobbered them);
hot state stays on the primary disk. Cleans stale /slowdisk targets first (no leak on
re-restore). Safe fallbacks: no /slowdisk / no manifest / no static paths -> normal
extract. Reth excluded (reth dropped whole-dir static-file symlinks).

volume-utils.sh: add delete_slowdisk_targets_for_key() — follows a volume's symlinks and
sweeps the rpc_<key>__data_ pattern under /slowdisk (matches delete-volumes.sh).

cleanup-volumes.sh: free the /slowdisk static data before docker volume rm (was leaking),
and fix the fragile substring used/unused match to an exact name match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-15 03:05:04 +00:00
goldsquid
1aef140a9a if the backup storage is local do not use webdav 2026-06-06 11:36:58 +07:00
goldsquid
91ef991773 make the volume scripts ignore ephemeral volumes 2026-06-06 09:57:48 +07:00
goldsquid
0d38493c9e smol fixes 2025-12-16 13:11:14 +07:00
goldsquid
cfe0b50ae1 ipv4 2025-12-13 15:22:51 +07:00
goldsquid
752be20f18 delete nodekeys when restoring 2025-11-15 11:11:50 +07:00
squidbear
57a2dae63e fix 2025-03-31 18:47:32 +02:00
Sebastian
1b7ddcbb50 big fix 2025-01-30 12:27:40 +01:00
Sebastian
58fe579555 respect exiting symlinks 2025-01-28 07:41:42 +01:00
Sebastian
a81c05cd5d update to extract without storing the archive when /backup doesn't exist 2024-12-21 07:05:07 +01:00
Sebastian
a91ae0ddd0 fix 2024-09-16 13:10:00 +02:00
Sebastian
9fcd4f9a6b fix 2024-09-16 13:03:53 +02:00
Sebastian
88ec7cbd31 fix 2024-09-16 12:54:43 +02:00
Sebastian
4603a04c3b new feature 2024-09-16 12:16:04 +02:00
Sebastian
6593ddd9e3 fix it 2024-05-16 05:08:27 +02:00
Sebastian
475fcb4c21 check the sync status before reporting a node as available to serve 2024-03-25 07:19:20 +01:00
Sebastian
bcae174cb0 fix 2024-03-25 00:03:22 +01:00
Sebastian
abbb294c4a fix 2024-03-24 16:16:04 +01:00
Sebastian
09001707be fix 2024-03-24 15:29:29 +01:00
Sebastian
253f5bebf9 jo 2024-03-24 08:07:25 +01:00
Sebastian
ff6f4c1e87 look at local volumes too 2024-03-24 07:09:20 +01:00
Sebastian
57549f3cc3 better 2024-03-24 05:31:46 +01:00
Sebastian
a97aa7238b update 2024-03-24 04:53:58 +01:00
Sebastian
fa09bd4748 more sophisticated 2024-03-24 04:51:34 +01:00
Sebastian
99eaf640f3 jo 2024-03-23 04:42:19 +01:00
Sebastian
d27693b1e5 shuffle around 2024-03-22 04:40:07 +01:00
Sebastian
ed96d394dd renamed 2024-03-21 13:28:47 +01:00