solution: docs for health check

docs/06-monitoring.adoc

@@ -119,4 +119,61 @@ This dashboard contains:
 
 - JSON RPC upstream conn seconds 50,75,90,99 percentiles
 
+== Health Checks
 
+Dshackle provides a http endpoint to check status of the servers.
+This check is compatible with https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-startup-probes/#http-probes[Kubernetes Liveness and Readiness Probes].
+
+By default, it's disabled, and you have to set up which blockchain are required to be available to consider Dshackle alive.
+
+.Example config:
+[source,yaml]
+----
+health:
+  port: 8082 # <1>
+  host: 127.0.0.1 # <2>
+  path: /health # <3>
+  blockchains: # <4>
+    - chain: ethereum # <5>
+      min-available: 2 # <6>
+    - chain: bitcoin
+      min-available: 1
+----
+
+<1> (optional) port to bind the Health server.
+Default: `8082`
+<2> (optional) host to bind the Health server.
+Default: `127.0.0.1`
+<3> (optional) path on the server.
+Default: `/health`.
+I.e., `http://127.0.0.1:8082/health` with default config
+<4> list of blockchain to check availability
+<5> a Blockchain to check
+<6> minimum available (i.e., fully synced) Upstreams for that blockchain
+
+With the config above the server is considered healthy if:
+
+- Dshackle has connected to at least two valid Ethereum upstreams
+- **and** at least one valid Bitcoin upstream.
+
+When the server is healthy is responds with `OK` and 200 as HTTP Status Code.
+When any of the checks failed, it responds with a short description and 503 as HTTP Status Code.
+
+Example of a response for an unhealthy server that doesn't have enough upstreams for a Ethereum Classic Blockchain.
+
+.GET http://127.0.0.1:8082/health
+----
+ETHEREUM_CLASSIC UNAVAILABLE
+----
+
+Optionally, the server can be called with `?detailed` query, which provides a more detailed response:
+
+.GET http://127.0.0.1:8082/health?detailed
+----
+ETHEREUM_CLASSIC UNAVAILABLE
+BITCOIN AVAILABLE
+  local-btc-1 OK with lag=0
+ETHEREUM AVAILABLE
+  local-eth-1 OK with lag=0
+  local-eth-2 OK with lag=0
+----

docs/reference-configuration.adoc

@@ -31,6 +31,14 @@ monitoring:
     port: 8081
     path: /metrics
 
+health:
+  port: 8082
+  host: 127.0.0.1
+  path: /health
+  blockchains:
+   - chain: ethereum
+     min-availability: 1
+
 cache:
   redis:
     enabled: true
@@ -164,6 +172,10 @@ See <<tls>> section
 | Setup Prometheus monitoring.
 See <<monitoring>> section
 
+| `health`
+|
+| Setup Health Check endpoint See <<health>> section
+
 | `proxy`
 |
 | Setup HTTP proxy that emulates all standard JSON RPC requests.
@@ -285,6 +297,52 @@ _Reserved for future use_, in case of multiple different types of endpoints.
 
 |===
 
+[#health]
+== Health Check endpoint
+
+[source,yaml]
+----
+health:
+  port: 8082
+  host: 127.0.0.1
+  path: /health
+  blockchains:
+    - chain: ethereum
+      min-available: 2
+    - chain: bitcoin
+      min-available: 1
+----
+
+[cols="2a,2a,5"]
+|===
+| Option | Default Value | Description
+
+| `port`
+| `8082`
+| HTTP port to bind the server
+
+| `host`
+| `127.0.0.1`
+| HTTP host to bind the server
+
+| `path`
+| `/health`
+| HTTP path to respond on requests
+
+| `blockchains`
+|
+| List of blockchains that must be available to consider the server _healthy_
+
+| `[blockchain].chain`
+|
+| Blockchain id
+
+| `[blockchain].min-available`
+| 1
+| How many _available_ upstreams for the blockchain is required to pass
+
+|===
+
 [#proxy]
 == Proxy config