== Upstreams Configuration

Dshackle can connect to multiple independent APIs ("upstreams") and provides an unified API on top of it.

Supported upstream protocols:

- JSON RPC
- Websockets
- gRPC (i.e. can connect to another Dshackle)

Those protocols can be configures with additional security, TLS and authentication.

=== Notes on upstream configuration

==== Ethereum

- Most of Ethereum nodes support WebSocket connection, in addition to the JSON RPC.
If it's available on your node, it's suggested to configure both JSON RPC and WebSocket connection

==== Bitcoin

- Bitcoind needs to be configured to index/track addresses that you're going to request.
Make sure you configure it to index your addresses with `importaddress`.
If you request balance for an address that is not indexed then it returns 0 balance.
- To track all transactions you need to setup index for transactions, which is disabled by default.
Run it with `-reindex` option, or set `txindex=1` in the config.

=== Example Configuration

.upstreams.yaml
[source,yaml]
----
version: v1

cluster:
  defaults:
  - chains:
      - ethereum
    options:
      min-peers: 10
  - chains:
      - kovan
    options:
      min-peers: 2
  upstreams:
    - id: us-nodes
      chain: auto
      connection:
        grpc:
          host: 35.226.252.117
          port: 443
          tls:
            ca: ca.crt
            certificate: client.crt
            key: client.p8.key
    - id: infura-eth
      chain: ethereum
      labels:
        provider: infura
      options:
        disable-validation: true
      connection:
        ethereum:
          rpc:
            url: "https://mainnet.infura.io/v3/${INFURA_USER}"
            basic-auth:
              username: ${INFURA_USER}
              password: ${INFURA_PASSWD}
          ws:
            url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
            basic-auth:
              username: ${INFURA_USER}
              password: ${INFURA_PASSWD}
----

There're two main segments for upstreams configuration:

- _upstreams_ - a list of API to connect to, with all configuration specific for upstream and chain
- and _default options_ as common configuration options applied to all nodes in that group

In the example above we have:

- default configuration for _Ethereum Mainnet_ which accepts upstream as valid when it not in fast synchronization mode
and has at least 10 peers.For _Kovan Testnet_ nodes the requirements are much relieved
- as upstreams it has 2 configurations
* balancer connects to another Dshackle/another machine by using gRPC protocol
** accepts (i.e. proxies) any blockchain available on that remote
** verifies TLS certificate of the server
** uses client certificate for authentication, i.e. remote server is accepting only clients authenticated by a
certificate
* connects to Infura provided _Ethereum Mainnet_
** configuration is using placeholders for `${INFURA_USER}` and `${INFURA_PASSWD}` which will be replaced with
corresponding environment variables values
** uses Basic Authentication to authenticate requests on Infura
** label `[provider: infura]` is set for that particular upstream, which can be selected during a request.For example for
some requests you may want to use nodes with that label only, i.e. _"send that tx to infura nodes only"_,
or _"read only from archive node, with label [archive: true]"_
** upstream validation (peers, sync status, etc) is disabled for that particular upstream

=== Configuration options

Options (default or as part of upstream config):

[cols="2,1,5a"]
|===
| Option | Default | Description

| `disable-validation` | false | if `true` then Dshackle will not try to verify status of the upstream (could be useful for a trusted cloud
   provider such as Infura, but disabling it is not recommended for a normal node)
| `min-peers` | 3 | specify minimum amount of connected peers, Dshackle will not use upstream with less than specified number
| `timeout` | 60 | timeout in seconds after which request to the upstream will be discarded (and may be retried on an another upstream)
|===

=== Connection type

Dshackle currently supports

- `rpc` a standard Ethereum JSON RPC
- `ws` websocket connection (supposed to be used in addition to `rpc` connection)
- `grpc` connects to another Dshackle instance

=== Bitcoin Methods

.By default an ethereum upstream allows call to the following JSON RPC methods:
- `getbestblockhash`
- `getblock`
- `getblocknumber`
- `getblockcount`
- `gettransaction`
- `getrawtransaction`
- `gettxout`
- `getreceivedbyaddress`
- `listunspent`
- `sendrawtransaction`

.Plus following methods are answered directly by Dshackle
- `getmemorypool`
- `getconnectioncount`
- `getnetworkinfo`

=== Ethereum Methods

.By default an ethereum upstream allows calls to the following JSON RPC methods:
- `eth_gasPrice`
- `eth_call`
- `eth_estimateGas`
- `eth_getBlockTransactionCountByHash`
- `eth_getUncleCountByBlockHash`
- `eth_getBlockByHash`
- `eth_getTransactionByHash`
- `eth_getTransactionByBlockHashAndIndex`
- `eth_getStorageAt`
- `eth_getCode`
- `eth_getUncleByBlockHashAndIndex`
- `eth_getTransactionCount`
- `eth_blockNumber`
- `eth_getBalance`
- `eth_sendRawTransaction`
- `eth_getBlockTransactionCountByNumber`
- `eth_getUncleCountByBlockNumber`
- `eth_getBlockByNumber`
- `eth_getTransactionByBlockNumberAndIndex`
- `eth_getTransactionReceipt`
- `eth_getUncleByBlockNumberAndIndex`


.Plus following methods are answered directly by Dshackle
- `net_version`
- `net_peerCount`
- `net_listening`
- `web3_clientVersion`
- `eth_protocolVersion`
- `eth_syncing`
- `eth_coinbase`
- `eth_mining`
- `eth_hashrate`
- `eth_accounts`

It's possible to enable additional methods that are available on upstream, or disable an existing method. For that purpose
there is `methods` configuration:

[source, yaml]
----
upstreams:
  - id: my-node
    chain: ethereum
    labels:
      archive: true
    methods:
      enabled:
        - name: trace_transaction
      disabled:
        - name: eth_getBlockByNumber
----

Such configuration option allows to execute method `trace_transaction` and also disables `eth_getBlockByNumber` on that
particular upstream. If a client requests to execute method `trace_transaction` then it will be scheduled to that upstream (or
any upstream with such method enabled).

NOTE: It's especially useful when used together with upstream labels. If an archive upstream has label `archive: true` it's
possible to specify that the client wants to execute method `trace_transaction` only on an archive node(s), which has
complete historical data for tracing.

=== Authentication

==== TLS

All connection types can use TLS secured connection, with optional client certificate authentication:

- `ca` path to certificate required from remote server
- optional `certificate` and `key` for client authentication.

NOTE: Please note that `key` must be encoded with _PKCS 8_

==== Basic Authentication

For JSON RPC and Websockets a Basic Authentication can be used:

- `username` - username
- `password` - password