drpc: accesslog-shipper sidecar (NDJSON tail -> splits listener)

Decoupled by design: dshackle writes the file regardless; a dead listener
or slow link stalls only the busybox sidecar. Retry resumes at file end
(bounded loss); the listener-side prometheus reconciliation surfaces gaps.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
rob
2026-07-09 13:10:10 +00:00
parent 00356d77e6
commit dc14e9cf2b

View File

@@ -1,12 +1,26 @@
services:
# Streams the dshackle access log (NDJSON, one line per response) to the splits
# listener on the insights host. Deliberately decoupled from dshackle via the
# file: a dead listener or slow link only ever stalls this sidecar, never the
# gateway. tail -F follows logrotate copytruncate; on disconnect the retry
# loop resumes at the file END (bounded loss, no duplication floods) — the
# per-gateway prometheus-vs-lines reconciliation on the listener side makes
# any gap visible within the hour.
accesslog-shipper:
image: busybox:1.36
restart: unless-stopped
volumes:
- ./dshackle_logs:/logs:ro
command: sh -c 'while true; do tail -F /logs/access_log.jsonl 2>/dev/null | nc ${SPLITS_LISTENER_HOST:-78.47.193.225} ${SPLITS_LISTENER_PORT:-9102}; echo "shipper: listener connection lost, retrying in 10s"; sleep 10; done'
dshackle:
image: drpcorg/dshackle:0.79.10
restart: unless-stopped
volumes:
- ./main_configs:/etc/dshackle
# dshackle accessLog target (main_configs/dshackle.yaml accessLog.filename=/logs/...).
# Host-side dir so the splits tailer can aggregate gateway->region request shares
# (earnings misattribution fix, operator + claude-machine design 2026-07-09).
# Host-side dir so the shipper sidecar above can stream gateway->region request
# shares (earnings misattribution fix, operator + claude-machine design 2026-07-09).
- ./dshackle_logs:/logs
expose:
- 8080