== Upstreams Configuration

Dshackle can connect to multiple independent APIs ("upstreams") and provides an unified API on top of it.

Supported upstream protocols:

- JSON RPC
- Websockets
- gRPC (i.e. can connect to another Dshackle)

Those protocols can be configures with additional security, TLS and authentication.

=== Notes on upstream configuration

==== Ethereum

- Most of Ethereum nodes support WebSocket connection, in addition to the JSON RPC.
If it's available on your node, it's suggested to configure both JSON RPC and WebSocket connection

==== Ethereum PoS

- The support for PoS Ethereum isn't fully implemented yet. However, we support it in a simplified way.
As we can no longer rely on the difficulty parameter of a block for fork choice algorithm we implement upstream rating.
In short terms, we just consider the upstream with the highest rating to be always correct when reporting its head.
Unless it's down then we will fallback on the upstream with the second highest rating, etc.

==== Bitcoin

- Bitcoind needs to be configured to index/track addresses that you're going to request.
Make sure you configure it to index your addresses with `importaddress`.
If you request balance for an address that is not indexed then it returns 0 balance.
- To track all transactions you need to setup index for transactions, which is disabled by default.
Run it with `-reindex` option, or set `txindex=1` in the config.

=== Example Configuration

.upstreams.yaml
[source,yaml]
----
version: v1

cluster:
  defaults:
  - chains:
      - ethereum
    options:
      min-peers: 10
  - chains:
      - kovan
    options:
      min-peers: 2
  upstreams:
    - id: us-nodes
      chain: auto
      connection:
        grpc:
          host: 35.226.252.117
          port: 443
          tls:
            ca: ca.crt
            certificate: client.crt
            key: client.p8.key
    - id: infura-eth
      chain: ethereum
      role: fallback
      labels:
        provider: infura
      options:
        disable-validation: true
      connection:
        ethereum:
          rpc:
            url: "https://mainnet.infura.io/v3/${INFURA_USER}"
            basic-auth:
              username: ${INFURA_USER}
              password: ${INFURA_PASSWD}
          ws:
            url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
            basic-auth:
              username: ${INFURA_USER}
              password: ${INFURA_PASSWD}
    - id: ethereum-pos
      chain: ropsten
      connection:
        ethereum-pos:
            execution:
              rpc:
                url: ${ROPSTEN_NODE_RPC_URL}
              ws:
                url: ${ROPSTEN_NODE_WS_URL}
----

There are two main segments for upstreams configuration:

- _upstreams_ - a list of API to connect to, with all configuration specific for upstream and chain
- and _default options_ as common configuration options applied to all nodes in that group

In the example above we have:

- default configuration for _Ethereum Mainnet_ which accepts upstream as valid when it not in fast synchronization mode and has at least 10 peers.For _Kovan Testnet_ nodes the requirements are much relieved
- as upstreams it has 2 configurations
* balancer connects to another Dshackle/another machine by using gRPC protocol
** accepts (i.e. proxies) any blockchain available on that remote
** verifies TLS certificate of the server
** uses client certificate for authentication, i.e. remote server is accepting only clients authenticated by a certificate
* connects to Infura provided _Ethereum Mainnet_
** as a _fallback_ upstream, which means that it's used only if `us-nodes` fails
** configuration is using placeholders for `${INFURA_USER}` and `${INFURA_PASSWD}` which will be replaced with corresponding environment variables values
** uses Basic Authentication to authenticate requests on Infura
** label `[provider: infura]` is set for that particular upstream, which can be selected during a request.For example for some requests you may want to use nodes with that label only, i.e. _"send that tx to infura nodes only"_, or _"read only from archive node, with label [archive: true]"_
** upstream validation (peers, sync status, etc) is disabled for that particular upstream

=== Roles and Fallback upstream

By default, the Dshackle connects to each upstream in a Round-Robin basis, i.e. sequentially one by one.
If you need more gradual control over the order of which upstream is used and when you can assign following roles:

- `primary` (default role if nothing specified)
- `secondary`
- `fallback`

Where `primary` and `secondary` are considered here a _standard_ upstreams, and `fallback` is used on failure of standard upstreams.
I.e. the Dshackle always starts with making requests to standard upstreams.
If all of them failed, if responses are inconsistent (ex. for `eth_getTransactionCount`), or when it needs to broadcast to a wider network (`sendrawtransaction`), then upstreams with role `fallback` cames to use.

The internal request order is (goes to next only if all upstreams on current step a not available or failed):

1. tries with primary upstreams
2. tries with secondary upstream
3. ... delay (100ms at first, increased each iteration)
4. tries with primary upstreams
5. tries with secondary upstream
6. tries with fallback upstreams
7. ... go to step 3

Steps 3-6 are repeated until a valid response received, or a timeout for the original request is reached.

In general:
- you set role `secondary` for upstream in another cluster/datacenter - you set role `fallback` for an external upstream which may be provided by a third party, and you want to use it as a last resort

=== Configuration options

Options (default or as part of upstream config):

[cols="2,1,5a"]
|===
| Option | Default | Description

| `disable-validation` | false | if `true` then Dshackle will not try to verify status of the upstream (could be useful for a trusted cloud
provider such as Infura, but disabling it is not recommended for a normal node)
| `min-peers` | 3 | specify minimum amount of connected peers, Dshackle will not use upstream with less than specified number
| `timeout` | 60 | timeout in seconds after which request to the upstream will be discarded (and may be retried on an another upstream)
| `balance` | `true` for ethereum, `false` for bitcoin | specify if this node should be used to fetch balance for an address
|===

=== Connection type

Dshackle currently supports

- `rpc` a standard Ethereum JSON RPC
- `ws` websocket connection (supposed to be used in addition to `rpc` connection)
- `grpc` connects to another Dshackle instance

=== Bitcoin Methods

.By default an ethereum upstream allows call to the following JSON RPC methods:
- `getbestblockhash`
- `getblock`
- `getblocknumber`
- `getblockcount`
- `gettransaction`
- `getrawtransaction`
- `gettxout`
- `getreceivedbyaddress`
- `listunspent`
- `sendrawtransaction`

.Plus following methods are answered directly by Dshackle
- `getmemorypool`
- `getconnectioncount`
- `getnetworkinfo`

=== Ethereum Methods

.By default, an ethereum upstream allows calls to the following JSON RPC methods:
- `eth_gasPrice`
- `eth_call`
- `eth_estimateGas`
- `eth_getBlockTransactionCountByHash`
- `eth_getUncleCountByBlockHash`
- `eth_getBlockByHash`
- `eth_getTransactionByHash`
- `eth_getTransactionByBlockHashAndIndex`
- `eth_getStorageAt`
- `eth_getCode`
- `eth_getUncleByBlockHashAndIndex`
- `eth_getTransactionCount`
- `eth_blockNumber`
- `eth_getBalance`
- `eth_sendRawTransaction`
- `eth_getBlockTransactionCountByNumber`
- `eth_getUncleCountByBlockNumber`
- `eth_getBlockByNumber`
- `eth_getTransactionByBlockNumberAndIndex`
- `eth_getTransactionReceipt`
- `eth_getUncleByBlockNumberAndIndex`
- `eth_feeHistory`
- `eth_getLogs`

.Plus following methods are answered directly by Dshackle
- `net_version`
- `net_peerCount`
- `net_listening`
- `web3_clientVersion`
- `eth_protocolVersion`
- `eth_syncing`
- `eth_coinbase`
- `eth_mining`
- `eth_hashrate`
- `eth_accounts`

It's possible to enable additional methods that are available on upstream, or disable an existing method.
For that purpose there is `methods` configuration:

[source, yaml]
----
upstreams:
  - id: my-node
    chain: ethereum
    labels:
      archive: true
    methods:
      enabled:
        - name: trace_transaction
      disabled:
        - name: eth_getBlockByNumber
----

Such configuration option allows executing method `trace_transaction` and also disables `eth_getBlockByNumber` on that particular upstream.
If a client requests to execute method `trace_transaction` then it will be scheduled to that upstream (or any upstream with such method enabled).

NOTE: It's especially useful when used together with upstream labels.If an archive upstream has label `archive: true` it's possible to specify that the client wants to execute method `trace_transaction` only on an archive node(s), which has complete historical data for tracing.

=== Static Methods

You can overwrite existing methods or add new ones using a static response:

[source,yaml]
----
upstreams:
  - id: my-node
    chain: ethereum
    methods:
      enabled:
        - name: net_version
          static: "\"100000\""
        - name: eth_chainId
          static: "0x186a0"
        - name: eth_custom_array
          static: '["custom_array_response"]'
        - name: eth_custom_bool
          static: "false"
----

=== Authentication

==== TLS

All connection types can use TLS secured connection, with optional client certificate authentication:

- `ca` path to certificate required from remote server
- optional `certificate` and `key` for client authentication.

NOTE: Please note that `key` must be encoded with _PKCS 8_

==== Basic Authentication

For JSON RPC and Websockets a Basic Authentication can be used:

- `username` - username
- `password` - password
