StakeSquid/dshackle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

solution: open source it

Browse Source
This commit is contained in:
Igor Artamonov
2019-08-22 21:07:12 -04:00
parent f8edcf5762
commit d873130567
98 changed files with 2173 additions and 60 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
docs/00-intro.adoc Normal file
View File

@@ -0,0 +1,30 @@
== What is Dshackle
Dshackle is a L7 Load Balancer for Blockchain APIs with automatic discovery, health checking, secure access, TLS with
client authentication, and many other features. It can be configured as an edge proxy, middle proxy or API gateway.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers,
such as Geth, Parity, Infura, etc), automatically verifies their availability and the current status of the network,
it routes requests to available node, and makes sure the response is consistent and/or data successfully broadcasted to
the networks.
Example use cases:
- Query for a _transaction_ (block, etc) tries to find it on different nodes and/or retry until it's found or there is
a consistent answer from upstreams
- Getting _nonce_ to send a transaction makes sure it's larges value over several nodes
- Sending _transaction_ distributes it to several nodes in parallel
Availability and fault tolerance:
- Dshackle connects to several upstreams via JSON RPC, Websockets or gRPC protocols
- It verifies if a node ("upstream") is fully synchronized (not in initial sync mode), has enough peers and its height
is not behind other nodes
- If an upstream lags behind, lost peers, started to resync, or simply goes down then Dshackle temporarily excludes it from
routing and returns back when the the upstream's problem is fixed
Main goals:
- stable and fault tolerant access to blockchain nodes
- secure connections and authentication
- allow to build scalable APIs with nodes distributed over multiple data centers

17
docs/01-architecture-intro.adoc Normal file
View File

@@ -0,0 +1,17 @@
== Architecture Overview
Dshackle is built using/based on:
- Java Virtual Machine, Java 8+
- Kotlin
- Spring Framework, Boot and Reactor
It's own communication protocol is based on Protobuf and gRPC, though it connects to other nodes using different protocols,
such as JSON RPC or Websockets.
Dshackle connects to actual blockchain nodes and provides an aggregated API automatically directing requests to
currently healthy nodes. It's built using _reactive_ and non-blocking code and provides low latency and high efficient
API.
Dshackle servers can be connected to each other as well, or work in parallel, to provide scalable and fault tolerant
connection to blockchains. All connections can be secured and authenticated with client/server certificates.

104
docs/02-quick-start.adoc Normal file
View File

@@ -0,0 +1,104 @@
== Quick Start
=== Prerequisites
Dshackle is designed for cloud environment and supposed to be used withing Docker and/or Kubernetes. However it's a JVM
based application and therefore can be used in most of standard environment where Java Virtual Machine can be installed.
We're going to use Docker image for this quick start.
For demo access we use gRPCurl tool, which can be installed from https://github.com/fullstorydev/grpcurl
=== Configuration
NOTE: you can find following example configuration in demo/quick-start directory of the project
Create file `dshackle.yaml` with following content:
[source,yaml]
----
version: v1
port: 9001
tls:
enabled: false
upstreams:
config: "upstreams.yaml"
----
This very basic config says that:
- application should listen on 0.0.0.0:9001
- TLS security should be disabled (_never use in production!_)
- read upstreams configuration from file `upstreams.yaml` in current directory
Now create file `upstreams.yaml`:
[source,yaml]
----
version: v1
upstreams:
- id: infura-eth
chain: ethereum
connection:
ethereum:
rpc:
url: "https://mainnet.infura.io/v3/${INFURA_USER}"
ws:
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
- id: infura-kovan
chain: kovan
connection:
ethereum:
rpc:
url: "https://kovan.infura.io/v3/${INFURA_USER}"
----
This config:
- setups 2 upstreams, one for Ethereum Mainnet and another for Kovan Testnet (both upstreams are configured for Infura for demo purposes, but you can use other compatible endpoints)
- for Ethereum Mainnet it connects using JSON RPC and Websockets connections, for Kovan just JSON RPC is used
- Infura authentication config is omitted for this demo
- `${INFURA_USER}` value can be provided through environment variables
==== Run as docker
Official Docker image you can find at: emeraldpay/dshackle
.Setup Infura username
[source,bash]
----
export INFURA_USER=...
----
.Run Dshackle
[source,bash]
----
docker run -p 9001:9001 -v $(pwd):/config -w /config -e "INFURA_USER=$INFURA_USER" emeraldpay/dshackle
----
.Connect and listen for new blocks on Ethereum Mainnet
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:9001 io.emeraldpay.api.Blockchain/SubscribeHead
----
.Output would be like
----
{
"chain": "CHAIN_ETHEREUM",
"height": 8396159,
"blockId": "fc58a258adccc94466ae967b1178eea721349b0667f59d5fe1b0b436460bce75",
"timestamp": 1566423564000,
"weight": "AnMcf2VJB5kOSQ=="
}
{
"chain": "CHAIN_ETHEREUM",
"height": 8396160,
"blockId": "787899711b862b77df8d2faa69de664048598265a9f96abf178d341076e200e0",
"timestamp": 1566423574000,
"weight": "AnMch35tO6hSGg=="
}
...
...
----
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet. It's a method provided
by Dshackle, provided in additional to methods provided by RPC JSON of underlying nodes.

25
docs/03-server-config.adoc Normal file
View File

@@ -0,0 +1,25 @@
== Server Configuration
[source,yaml]
----
version: v1
port: 9001
tls:
enabled: true
server:
certificate: "server.crt"
key: "server.p8.key"
client:
require: true
ca: "ca.crt"
upstreams:
config: "upstreams.yaml"
----
Configures following:
- server is listening on `0.0.0.0:9001`
- TLS is enabled
- server certificate in located at `server.crt` with key for it at `server.p8.key`
- server requires a client authentication by TLS client certificate signed by `ca.crt` certificate
- upstreams configuration is configured in file `upstreams.yaml`

115
docs/04-upstream-config.adoc Normal file
View File

@@ -0,0 +1,115 @@
== Upstreams Configuration
Dshackle can connect to multiple independent APIs ("upstreams") and provides an unified API on top of it.
Supported upstream protocols:
- JSON RPC
- Websockets
- gRPC (i.e. can connect to another Dshackle)
Those protocols can be configures with additional security, TLS and authentication.
=== Example
.upstreams.yaml
[source,yaml]
----
version: v1
defaultOptions:
- chains:
- ethereum
options:
min-peers: 10
- chains:
- kovan
options:
min-peers: 2
upstreams:
- id: us-nodes
chain: auto
connection:
grpc:
host: 35.226.252.117
port: 443
tls:
ca: ca.crt
certificate: client.crt
key: client.p8.key
- id: infura-eth
chain: ethereum
labels:
provider: infura
options:
disable-validation: true
connection:
ethereum:
rpc:
url: "https://mainnet.infura.io/v3/${INFURA_USER}"
basic-auth:
username: ${INFURA_USER}
password: ${INFURA_PASSWD}
ws:
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
basic-auth:
username: ${INFURA_USER}
password: ${INFURA_PASSWD}
----
There're two main segments for upstreams configuration:
- _upstreams_ - a list of API to connect to, with all configuration specific for upstream and chain
- and _default options_ as common configuration options applied to all nodes in that group
In the example above we have:
- default configuration for _Ethereum Mainnet_ which accepts upstream as valid when it not in fast synchronization mode
and has at least 10 peers. For _Kovan Testnet_ nodes the requirements are much relieved
- as upstreams it has 2 configurations
* balancer connects to another Dshackle/another machine by using gRPC protocol
** accepts (i.e. proxies) any blockchain available on that remote
** verifies TLS certificate of the server
** uses client certificate for authentication, i.e. remote server is accepting only clients authenticated by a
certificate
* connects to Infura provided _Ethereum Mainnet_
** configuration is using placeholders for `${INFURA_USER}` and `${INFURA_PASSWD}` which will be replaced with
corresponding environment variables values
** uses Basic Authentication to authenticate requests on Infura
** label `[provider: infura]` is set for that particular upstream, which can be selected during a request. For example for
some requests you may want to use nodes with that label only, i.e. _"send that tx to infura nodes only"_,
or _"read only from archive node, with label [archive: true]"_
** upstream validation (peers, sync status, etc) is disabled for that particular upstream
=== Configuration options
Options (default or as part of upstream config):
- `disable-validation` - if `true` then Dshackle would not try to verify status of the upstream (useful for a trusted cloud
provider such as Infura, but is not recommended for an own node)
- `min-peers` - do not use upstream with less than specified connected peers
=== Connection type
Dshackle currently supports
- `rpc` a standard Ethereum JSON RPC
- `ws` websocket connection (supposed to be used in addition to `rpc` connection)
- `grpc` connects to another Dshackle instance
=== Authentication
==== TLS
All connection types can use TLS secured connection, with optional client certificate authentication:
- `ca` path to certificate required from remote server
- optional `certificate` and `key` for client authentication. Please note that `key` is encoded with _PKCS 8_
==== Basic Authentication
For JSON RPC and Websockets a Basic Authentication can be used:
- `username` - username
- `password` - password

3
docs/05-start.adoc Normal file
View File

@@ -0,0 +1,3 @@
== Launch a server
TBD

159
docs/06-methods.adoc Normal file
View File

@@ -0,0 +1,159 @@
== Methods
IMPORTANT: Dshackle provides an unified API based on gRPC and Protobuf.
Dshackle provides an unified API based on gRPC and Protobuf. I.e. it's not standard JSON RPC but client libraries could
be generated for all major libraries, and there're official libraries for Java and Javascript.
=== gRPC definition
[source,proto]
----
service Blockchain {
rpc SubscribeHead (Chain) returns (stream ChainHead) {}
rpc SubscribeBalance (BalanceRequest) returns (stream AddressBalance) {}
rpc SubscribeTxStatus (TxStatusRequest) returns (stream TxStatus) {}
rpc GetBalance (BalanceRequest) returns (stream AddressBalance) {}
rpc NativeCall (NativeCallRequest) returns (stream NativeCallReplyItem) {}
rpc Describe (DescribeRequest) returns (DescribeResponse) {}
rpc SubscribeStatus (StatusRequest) returns (stream ChainStatus) {}
}
----
=== Wrapped JSON RPC methods
Standard JSON RPC methods provided by Ethereum nodes are available wrapped into gRPC/Protobuf with additional features.
To call standard methods you use `NativeCall` method.
.NativeCallRequest
[source,proto]
----
message NativeCallRequest {
ChainRef chain = 1;
repeated NativeCallItem items = 2;
Selector selector = 3;
int32 quorum = 4;
AvailabilityEnum min_availability = 5;
}
message NativeCallItem {
uint32 id = 1;
string method = 3;
bytes payload = 4;
}
----
Where:
- `chain` target chain (see reference for ids)
- `items` as a list of independent requests, which may be executed in different nodes in parallels or in different order, with:
* `method` - a JSON RPC standard name, ex: `eth_getBlockByHash`
* `payload` - list of parameters for the methods, encoded as JSON string, ex. `["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]`
- `Selector` and `AvailabilityEnum` are described in reference, in short they allow to specify which nodes are allowed
to execute the request.
.NativeCallReplyItem
[source,proto]
----
message NativeCallReplyItem {
uint32 id = 1;
bool succeed = 2;
bytes payload = 3;
bytes error = 4;
}
----
Where:
- `payload` is JSON response for a particular call, encoded into a string (when `succeed` is true)
- or `error` if request failed (`succeed` is false)
NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response. It allows to build
non-blocking queries
=== SubscribeHead
Subscribes to new blocks on the specified chain. Returns stream of blocks right after it was accepted (and verified by
Dshackle) by any of the upstreams.
.ChainHead
[source,proto]
----
message ChainHead {
ChainRef chain = 1;
uint64 height = 2;
string block_id = 3;
uint64 timestamp = 4;
bytes weight = 5;
uint64 reorg = 6;
}
----
Where:
- `chain` - chain id
- `height` - block number
- `block_id` - block hash, as a string (please note that it doesn't have `0x` prefix)
- `timestamp` - timestamp of that block
- `weight` - total network difficulty on that block, as raw bytes
- `reorg` - number of reorganized blocks, if reorg happened
=== SubscribeBalance or GetBalance
Subscribes to changes of the balance for a single address or a set of addresses
.Request
----
message BalanceRequest {
Asset asset = 1;
AnyAddress address = 2;
}
message AnyAddress {
oneof addr_type {
SingleAddress address_single = 1;
MultiAddress address_multi = 2;
XpubAddress address_xpub = 3;
ReferenceAddress address_ref = 4;
}
}
----
.Response
----
message AddressBalance {
Asset asset = 1;
SingleAddress address = 2;
string balance = 3;
}
----
==== SubscribeTxStatus
Subscribes to transaction confirmations. Allows to send a transactions and then listen to all changes until it
gets enough confirmations. Changes is `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED`
.Request
----
message TxStatusRequest {
ChainRef chain = 1;
string tx_id = 2;
uint32 confirmation_limit = 3;
}
----
.Response (stream of)
----
message TxStatus {
string tx_id = 1;
bool broadcasted = 2;
bool mined = 3;
BlockInfo block = 4;
uint32 confirmations = 5;
}
----

83
docs/07-authentication.adoc Normal file
View File

@@ -0,0 +1,83 @@
== Authentication and encryption
=== Server authentication
Dshackle server supports both server and client certificate authentication, it's strongly recommended to use TLS to
connect to Dshackle server. More to that, the server uses gRPC which is based on HTTP/2, and most of 3rd party tools and
libraries expect it to be encrypted.
Please note that for most of use cases for Dshackle, which is designed to be an internal load balancer, a self-signed
certificates would be enough. In the example below a https://github.com/square/certstrap[certstrap] tool is used to
generate certificates, but traditional `openssl` tool can be used as well.
==== Setup Server certificate
.Generate a server certificate
----
SERVER_CA="ca.myhost.dev"
SERVER_IP="127.0.0.1"
ORG="My Company"
ORG_UNIT="Blockchain"
certstrap init --common-name "$SERVER_CA" --passphrase "" -o "$ORG" -ou "$ORG_UNIT CA"
certstrap request-cert -ip $SERVER_IP --passphrase "" -o "$ORG" -ou "$ORG_UNIT Server"
certstrap sign $SERVER_IP --CA $SERVER_CA
openssl pkcs8 -topk8 -inform PEM -outform PEM -in out/$SERVER_IP.key -out out/$SERVER_IP.p8.key -nocrypt
----
.Update dshackle.yaml to have:
----
version: v1
port: 9001
tls:
enabled: true
server:
certificate: out/127.0.0.1.crt
key: out/127.0.0.1.p8.key
----
.Verify that server uses the certificate
----
openssl s_client -alpn h2 -connect 127.0.0.1:9001 -CAfile out/ca.myhost.dev.crt
----
With the configuration above the server listen using TLS and the server identity can be verified by a client against public
server certificate. Please note that a server certificate doesn't prevent from connection by an unauthorized client, it only
verifies the server and encrypts a connection.
==== Setup Client certificate
To have authentication in both ways you'll need to configure client side certificates as well, at that case the server
will also verify each incoming connection and allow to connect only by a client with a trusted certificate.
.Generate a client certificate
----
CLIENT_CA="client-ca.myhost.dev"
CLIENT_ID="client_1"
ORG="My Company"
ORG_UNIT="Client"
certstrap init --common-name "$CLIENT_CA" --passphrase "" -o "$ORG" -ou "$ORG_UNIT CA"
certstrap request-cert --common-name "$CLIENT_ID" --passphrase ""
certstrap sign "$CLIENT_ID" --CA $CLIENT_CA
----
.Update dshackle.yaml to have:
----
version: v1
port: 9001
tls:
enabled: true
server:
certificate: out/127.0.0.1.crt
key: out/127.0.0.1.p8.key
client:
require: true
ca: out/client-ca.myhost.dev.crt
----
.Verify connection with client certificate
----
openssl s_client -alpn h2 -connect 127.0.0.1:9001 -CAfile out/ca.myhost.dev.crt -cert out/client_1.crt -key out/client_1.key
----

7
docs/08-quorum-and-selectors.adoc Normal file
View File

@@ -0,0 +1,7 @@
== Quorum
TBD
== Selectors
TBD

9
docs/09-caching.adoc Normal file
View File

@@ -0,0 +1,9 @@
== Caching
=== In memory cache
Dshackle keeps latest blocks in memory (by default 64 blocks)
=== Redis cache
TBD

19
docs/10-client-libraries.adoc Normal file
View File

@@ -0,0 +1,19 @@
== Client Libraries
=== Java Client
TBD
=== EtherJar Java library
TBD
=== Javascript Client
TBD
=== Protobuf
Original Protobuf definitions
TBD

11
docs/99-ending.adoc Normal file
View File

@@ -0,0 +1,11 @@
== Links
- Github: https://github.com/emeraldpay/dshackle
== Chat
image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge]
== Support
Contact splix@emeraldpay.io if you want to integrate Dshackle into your project or want to sponsor the development.

18
docs/README.adoc Normal file
View File

@@ -0,0 +1,18 @@
= Dshackle Documentation
include::00-intro.adoc[]
== Table of Content
. link:01-architecture-intro.adoc[Architecture Overview]
. link:02-quick-start.adoc[Quick Start]
. link:03-server-config.adoc[Server Configuration]
. link:04-upstream-config.adoc[Upstreams Configuration]
. link:05-start.adoc[How to launch a server]
. link:06-methods.adoc[API methods]
. link:07-authentication.adoc[Authentication]
. link:08-quorum-and-selectors.adoc[Quorum and Selectors]
. link:09-caching.adoc[Caching]
. link:10-client-libraries.adoc[Client Libraries]
include::99-ending.adoc[]

53
docs/tls.adoc
View File

@@ -1,53 +0,0 @@
== Setting Up TLS/SSL Security
=== Server Authentication
.Example generating of server certificate
----
SERVER_CA="ca.myhost.dev"
SERVER_IP="127.0.0.1"
ORG="My Company"
ORG_UNIT="Blockchain"
certstrap init --common-name "$SERVER_CA" --passphrase "" -o "$ORG" -ou "$ORG_UNIT CA"
certstrap request-cert -ip $SERVER_IP --passphrase "" -o "$ORG" -ou "$ORG_UNIT Server"
certstrap sign $SERVER_IP --CA $SERVER_CA
openssl pkcs8 -topk8 -inform PEM -outform PEM -in out/$SERVER_IP.key -out out/$SERVER_IP.p8.key -nocrypt
----
.Setup application.properties
----
ssl=true
ssl.cert=out/127.0.0.1.crt
ssl.key=out/127.0.0.1.p8.key
----
.Verify server certificate
----
openssl s_client -alpn h2 -connect 127.0.0.1:8090 -CAfile out/ca.myhost.dev.crt
----
=== Client Authentication (optional)
.Example generating of client certifiacte
----
CLIENT_CA="client-ca.myhost.dev"
CLIENT_ID="client_1"
ORG="My Company"
ORG_UNIT="Client"
certstrap init --common-name "$CLIENT_CA" --passphrase "" -o "$ORG" -ou "$ORG_UNIT CA"
certstrap request-cert --common-name "$CLIENT_ID" --passphrase ""
certstrap sign "$CLIENT_ID" --CA $CLIENT_CA
----
.Setup application.properties
----
ssl.client.cert=out/client-ca.myhost.dev.crt
----
.Vetify connection with client certificate
----
openssl s_client -alpn h2 -connect 127.0.0.1:8090 -CAfile out/ca.myhost.dev.crt -cert out/client_1.crt -key out/client_1.key
----