reload_dshackle: cut gateway connections as soon as the SIGHUP handler reports completion (~30ms) instead of after a fixed 3s settle

The reload is in-process and takes 10-30 ms; the fixed sleep was the window in
which a removed upstream was already unrouted in dshackle while the gateway still
held the old announcements and kept sending queries it could no longer answer.
Poll the log for 'Reloading config has been completed' (or the fail/drop
markers), SETTLE stays the upper bound. Operator 2026-09-19.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01KjqejYycVyVvrxV9aTDhgX
This commit is contained in:
rob
2026-09-19 09:50:03 +00:00
parent 89d8036c6a
commit 6f4b297733

View File

@@ -208,8 +208,19 @@ for CID in $(docker ps -q -f "name=dshackle"); do
RC=1
continue
fi
sleep "$SETTLE"
LOGS=$(docker logs --since "$T0" "$CID" 2>&1)
# Poll for the handler's completion line instead of sleeping SETTLE: the reload itself
# takes ~10-30 ms, but the fixed 3 s sleep was the window in which a removed upstream
# was already gone while the gateway still held the old announcements (operator
# 2026-09-19). SETTLE is now the upper bound only.
DONE_MARK="Reloading config has been completed"
LOGS=""
for _ in $(seq 1 $(( SETTLE * 20 ))); do
LOGS=$(docker logs --since "$T0" "$CID" 2>&1)
if echo "$LOGS" | grep -qF "$DONE_MARK" || echo "$LOGS" | grep -qF "$FAIL_MARK" || echo "$LOGS" | grep -qF "$DROP_MARK"; then
break
fi
sleep 0.05
done
REASON=""
if echo "$LOGS" | grep -qF "$FAIL_MARK"; then
REASON=$(echo "$LOGS" | grep -F "$FAIL_MARK" | head -1)