solution: basic implementation for Proxy endpoint

This commit is contained in:
Igor Artamonov
2020-03-19 23:34:32 -04:00
parent 7455949541
commit 26f8dd294e
27 changed files with 1309 additions and 135 deletions

View File

@@ -29,32 +29,41 @@ And for a request:
- Is it for concrete data (_block #100_) or the latest (_get balance_)?
- Is result a static value (_just block_) or may vary depending on network and node (_latest nonce_)?
- Does it need to be repeated over multiple nodes (_broadcast transaction_)?
- Request can also specify which subset of nodes should be able to execute the request by selecting node _Labels_
(see "link:08-quorum-and-selectors.adoc[Quorum and Selectors]")
- Request can also specify which subset of nodes should be able to execute the request by selecting node _Labels_ (see "link:08-quorum-and-selectors.adoc[Quorum and Selectors]")
Based on these factors, Dshackle executes the request on a most optimal node. For most of the simple requests, it just gets a node that is synchronized
to the point that must have a response for that particular request. If node failed, returned an invalid response, returned
an empty response when shouldn't then Dshackle tries again on another node, or on the same node after awhile (default is
200ms between failover repeats)
Based on these factors, Dshackle executes the request on a most optimal node.
For most of the simple requests, it just gets a node that is synchronized to the point that must have a response for that particular request.
If node failed, returned an invalid response, returned an empty response when shouldn't then Dshackle tries again on another node, or on the same node after awhile (default is 200ms between failover repeats)
=== Proxy
Dshackle provides access with standard JSON RPC protocol, functioning as a proxy to upstreams.
It provides:
- Routes only realy/alive upstreams, i.e., synchronized and with enough peers
- Load Balancing
- Request Retry on upstream errors
- Local Caching (memory and Redis, see link:09-caching.adoc[Caching])
- Broadcasting and Quorum for requests
=== gRPC protocol
Dshackle uses gRPC protocol for communications, because:
Dshackle native protocol is based on gRPC, which provides many additional features:
- provides additional parameters on top of upstreams JSON RPC requests
- based on HTTP/2 with low latency, compression, server push, pipelining and multiplexing
- gRPC has binding and code generators for most of the languages and frameworks
- easy to support TLS encryption and authentication
Dshackle has extra methods and functionality on top of standard APIs of upstreams, and it's more flexible to wrap
original APIs, such as JSON, into gRPC, and Protobuf + has additional data provided by Dshackle. It also allowed
to push new data from the server or send responses asynchronously, immediately after it gets executed on an upstream.
gRPC protocol is more flexible, and many of the Dshackle features are accessible mostly through that native protocol.
Dshackle has extra methods and functionality on top of standard APIs of upstreams, and it's more flexible to wrap original APIs, such as JSON, into gRPC, and Protobuf + has additional data provided by Dshackle.
It also allowed to push new data from the server or send responses asynchronously, immediately after it gets executed on an upstream.
=== Distributed Load Balancing
Dshackle servers can connect to each other through a secure encrypted and authenticated connection.
It allows to build a network of nodes deployed to different regions or run blockchain nodes outside of the main
network.
It allows to build a network of nodes deployed to different regions or run blockchain nodes outside of the main network.
Later is especially important for blockchain nodes, as they require an open firewall with incoming connections for P2P, and
to execute untrusted code ("smart contracts") at the same time. With Dshackle, it's possible to separate insecure nodes from

View File

@@ -20,6 +20,14 @@ version: v1
port: 2449
tls:
enabled: false
proxy:
host: 0.0.0.0
port: 8545
routes:
- id: eth
blockchain: ethereum
- id: kovan
blockchain: kovan
upstreams:
config: "upstreams.yaml"
----
@@ -71,9 +79,30 @@ export INFURA_USER=...
.Run Dshackle
[source,bash]
----
docker run -p 2449:2449 -v $(pwd):/etc/dshackle -e "INFURA_USER=$INFURA_USER" emeraldpay/dshackle
docker run -p 2449:2449 -p 8545:8545 -v $(pwd):/etc/dshackle -e "INFURA_USER=$INFURA_USER" emeraldpay/dshackle
----
==== Access using JSON RPC
Dshackle implements standard JSON RPC interface, providing additional caching layer, upstream readiness/liveness checks, retry and other features for building Fault Tolerant services.
.Request using Curl
[source,bash]
----
curl --request POST \
--url http://localhost:8545/eth \
--header 'content-type: application/json' \
--data '{"jsonrpc":"2.0", "method":"eth_getBalance", "id":1, "params":["0x690b2bdf41f33f9f251ae0459e5898b856ed96be", "latest"]}'
----
.Output
[source,bash]
----
{"jsonrpc":"2.0","id":1,"result":"0x72fa5e0181"}
----
==== Access using gRPC
.Connect and listen for new blocks on Ethereum Mainnet
[source,bash]
----
@@ -100,5 +129,5 @@ grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plai
...
----
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet. It's a method provided
by Dshackle, provided in additional to methods provided by RPC JSON of underlying nodes.
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet.
It's a method provided by Dshackle, available in additional to methods provided by RPC JSON of underlying nodes.

View File

@@ -3,8 +3,8 @@
Dshackle server tries to load its configuration from `/etc/dshackle/dshackle.yaml`, if it can't find a file at that path
it tries to load file `dshackle.yaml` from current working directory. If none of them found server fails to run with an error.
[source,yaml]
.Example dshackle.yaml configuration:
[source,yaml]
----
version: v1
port: 2449
@@ -22,8 +22,33 @@ upstreams:
It configures following:
- server is listening on `0.0.0.0:2449`
- server is listening with gRCP API on `0.0.0.0:2449`
- TLS is enabled
- server certificate is located at `server.crt` with the key for it at `server.p8.key`
- the server requires a client authentication by TLS client certificate signed by `ca.crt` certificate
- no JSON RPC is configured
- upstreams configuration is configured in the file `upstreams.yaml`
=== Enabling JSON RPC proxy
.Example proxy:
[source,yaml]
----
version: v1
port: 2449
proxy:
host: 0.0.0.0
port: 8080
routes:
- id: eth
blockchain: ethereum
upstreams:
config: "upstreams.yaml"
----
With that configuration Dshackle starts a JSON RPC proxy:
- JSON RPC server is listening on `0.0.0.0:8080`
- `http://0.0.0.0:8080/eth` provides access to Ethereum API routed to an available upstream

View File

@@ -1,4 +1,4 @@
== Client Libraries
== Client Libraries using native gRPC-based protocol
=== Protobuf