show-status.sh could never report failure. The ansible task that wraps it
('Execute show-status.sh and fail on failure') therefore always passed, on every
host, regardless of node state. Three bugs, each masking the next:
1. $? read too late. `code=0` sits between the sync-status.sh call and
`if [ $? -ne 0 ]`. A plain assignment succeeds and overwrites $? with 0, so
the condition was ALWAYS false and the else branch always taken.
2. The else branch was inverted. It is the sync-status-SUCCEEDED path, yet it set
`code=1; any_failure=true` — marking healthy nodes as failures.
3. any_failure could never propagate. check_sync_status runs backgrounded (`&`),
i.e. in a subshell, so `any_failure=true` inside it is discarded; and the
`wait "$pid"` loop threw away each job's exit status.
(3) hid (1) and (2): a script that believed every node had failed still exited 0,
so nobody saw it.
Fix: capture rc immediately; restore the intended logic (success => 0, syncing or
lagging => tolerated, anything else => failure); propagate failure in the PARENT
via `wait "$pid" || any_failure=true`, since the subshell cannot.
Verified with a stubbed sync-status.sh:
scenario before after
all online 0 0
one syncing 0 0 (tolerated)
one lagging 0 0 (tolerated)
one ERROR 0 1
ALL error 0 1
Behaviour for healthy fleets is unchanged; only genuine failures now surface.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
3.2 KiB
Executable File
3.2 KiB
Executable File