StakeSquid/dshackle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

solution: minot changes to docs

Browse Source
This commit is contained in:
Igor Artamonov
2019-10-25 21:36:42 -04:00
parent 6aed2395b6
commit bc6fd4d1db
6 changed files with 67 additions and 55 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
README.adoc
View File

@@ -8,23 +8,35 @@ image:https://img.shields.io/docker/pulls/emeraldpay/dshackle?style=flat-square[
image:https://img.shields.io/github/license/emeraldpay/dshackle.svg?style=flat-square&maxAge=2592000["License", link="https://github.com/emeraldpay/dshackle/blob/master/LICENSE"]
image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge]
Dshackle is a Blockchain API Load Balancer with automatic discovery and health checking, authentication and TLS termination.
It can be used as an edge proxy, middle proxy or API gateway.
Dshackle is an L7 Blockchain API Load Balancer with automatic discovery and health checking, authentication, and TLS termination.
It is designed to work as an edge proxy, middle proxy, or API gateway.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers,
such as Geth, Parity, Infura, etc), it automatically verifies their availability and the current status of the network,
executes commands making sure that the response is consistent and/or data successfully broadcasted to the network.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers, such
as Geth, Parity, Infura, etc), it automatically verifies their availability and the current status of the network, executes
commands making sure that the response is consistent and/or data successfully broadcasted to the network.
Project goals:
The main goals of the project is to:
- stable and fault tolerant access to blockchain nodes
- secure connections and authentication
- allows to build scalable APIs with nodes distributed over multiple data centers
- separate application services and blockchain nodes
- make blockchain access compatible with modern microservice oriented architecture
- provide secure access to a blockchain, on both protocol and data level
Dshackle connects to several upstreams via JSON RPC, Websockets or gRPC protocols. It verifies if a node ("upstream") is
fully synchronized (not in initial sync mode), has enough peers and its height is not behind other nodes. If an upstream
lags behind others, lost peers, started to resync, or simply went down then Dshackle temporarily excludes it from
requests and returns back when the the upstream problem is fixed.
The main features and advantages are:
- leveraging gRPC and HTTP2 protocols, with server push and asynchronous communications, to simplify and optimize standard
access patterns
- targeting Kubernetes architecture
- automatically distributing access to API through multiple different target nodes, taking into account their current
availability and status
- allowing to build a mesh network of routers in different regions sharing a set of underlying nodes, with automatic
rebalancing and smart routing
- caching data on the edge
- providing monitoring (ex. Prometheus) and externalizable logging
- configurable access authentication and authorization, including TLS certificates
Dshackle connects to several upstreams via JSON RPC, Websockets, or gRPC protocols. It verifies if a node ("upstream") is
fully synchronized (not in initial sync mode), has enough peers, and its height is not behind other nodes. If upstream lags
behind others, lost peers, started to resync or went down, then Dshackle temporarily excludes it from requests and returns
when the upstream problem is fixed.
image::call-schema.png[alt="Call Schema",width=100%,align="center"]

31
docs/00-intro.adoc
View File

@@ -1,30 +1,29 @@
== What is Dshackle
Dshackle is a L7 Load Balancer for Blockchain APIs with automatic discovery, health checking, secure access, TLS with
client authentication, and many other features. It can be configured as an edge proxy, middle proxy or API gateway.
Dshackle is an L7 Blockchain API Load Balancer with automatic discovery and health checking, authentication, and TLS termination.
It is designed to work as an edge proxy, middle proxy, or API gateway.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers,
such as Geth, Parity, Infura, etc), automatically verifies their availability and the current status of the network,
it routes requests to available node, and makes sure the response is consistent and/or data successfully broadcasted to
the networks.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers, such
as Geth, Parity, Infura, etc), it automatically verifies their availability and the current status of the network, executes
commands making sure that the response is consistent and/or data successfully broadcasted to the network.
Example use cases:
- Query for a _transaction_ (block, etc) tries to find it on different nodes and/or retry until it's found or there is
a consistent answer from upstreams
- Getting _nonce_ to send a transaction makes sure it's larges value over several nodes
- Sending _transaction_ distributes it to several nodes in parallel
- Read a _transaction_ - Dshackle tries to find it on different nodes and retries until it's found or there is a
consistent answer from upstreams that the transaction doesn't exist
- Get _nonce_ to send a transaction - Dshackle makes sure it finds the highest value across several nodes
- Send _transaction_ - Dshackle distributes it to several nodes in parallel
Availability and fault tolerance:
- Dshackle connects to several upstreams via JSON RPC, Websockets or gRPC protocols
- Dshackle connects to several upstreams via JSON RPC, Websockets or gRPC protocol
- It verifies if a node ("upstream") is fully synchronized (not in initial sync mode), has enough peers and its height
is not behind other nodes
- If an upstream lags behind, lost peers, started to resync, or simply goes down then Dshackle temporarily excludes it from
routing and returns back when the the upstream's problem is fixed
- If upstream lags behind, lost peers, started to resync, or goes down then Dshackle temporarily excludes it from
routing and returns back when the upstream's problem is fixed
Main goals:
- stable and fault tolerant access to blockchain nodes
- secure connections and authentication
- allow to build scalable APIs with nodes distributed over multiple data centers
- separate application services and blockchain nodes
- make blockchain access compatible with modern microservice oriented architecture
- provide secure access to a blockchain, on both protocol and data level

28
docs/01-architecture-intro.adoc
View File

@@ -6,10 +6,10 @@ Dshackle is built using/based on:
- Kotlin
- Spring Framework, Boot and Reactor
It's own communication protocol is based on Protobuf and gRPC, though it connects to other nodes using different protocols,
Its own communication protocol is based on Protobuf and gRPC, though it connects to other nodes using different protocols,
such as JSON RPC or Websockets.
Dshackle connects to actual blockchain nodes and provides an aggregated API automatically directing requests to
Dshackle connects to actual blockchain nodes and provides an aggregated API, automatically directing requests to
currently healthy nodes. It's built using _reactive_ and non-blocking code and provides low latency and high efficient
API.
@@ -21,20 +21,20 @@ connection to blockchains. All connections can be secured and authenticated with
Dshackle tries to find an optimal upstream for each particular request, to do so it consider following factors of a node:
- Current height
- Is it fully synchronized, missing few (less than 6) blocks or in process of initial sync?
- Is it fully synchronized, missing few (less than 6) blocks, or in the process of initial sync?
- How many peers it has?
And for a request:
- Is it for a concrete data (_block #100_) or latest (_get balance_)?
- Is it for concrete data (_block #100_) or the latest (_get balance_)?
- Is result a static value (_just block_) or may vary depending on network and node (_latest nonce_)?
- Does it need to be repeat over multiple nodes (_broadcast transaction_)?
- Does it need to be repeated over multiple nodes (_broadcast transaction_)?
- Request can also specify which subset of nodes should be able to execute the request by selecting node _Labels_
(see "link:08-quorum-and-selectors.adoc[Quorum and Selectors]")
Based on these factors Dshackle execute request on a most optimal node. For most of the simple requests it just gets a node which is synchronized
to the point that must have response for that particular request. If node failed, returned invalid response, returned
empty response when shouldn't then Dshackle tries again on another node, or on the same node after awhile (default is
Based on these factors, Dshackle executes the request on a most optimal node. For most of the simple requests, it just gets a node that is synchronized
to the point that must have a response for that particular request. If node failed, returned an invalid response, returned
an empty response when shouldn't then Dshackle tries again on another node, or on the same node after awhile (default is
200ms between failover repeats)
=== gRPC protocol
@@ -47,8 +47,8 @@ Dshackle uses gRPC protocol for communications, because:
- easy to support TLS encryption and authentication
Dshackle has extra methods and functionality on top of standard APIs of upstreams, and it's more flexible to wrap
original APIs, such as JSON, into gRPC and Protobuf + have additional data provided by Dshackle. It also allowed
to push new data from the server, or send responses asynchronously, immediately after it gets executed on an upstream.
original APIs, such as JSON, into gRPC, and Protobuf + has additional data provided by Dshackle. It also allowed
to push new data from the server or send responses asynchronously, immediately after it gets executed on an upstream.
=== Distributed Load Balancing
@@ -56,9 +56,9 @@ Dshackle servers can connect to each other through a secure encrypted and authen
It allows to build a network of nodes deployed to different regions or run blockchain nodes outside of the main
network.
Later is especially important for blockchain nodes, as they require open firewall with incoming connections for P2P, and
to execute untrusted code ("smart contracts") on the same time. With Dshackle it's possible to separate unsecure nodes from
business application.
Later is especially important for blockchain nodes, as they require an open firewall with incoming connections for P2P, and
to execute untrusted code ("smart contracts") at the same time. With Dshackle, it's possible to separate insecure nodes from
the business application.
But in general Dshackle allows to run a fault tolerant load balancing to build scalable and fail safe systems on
But in general, Dshackle allows running a fault tolerant load balancing to build scalable and fail safe systems on
blockchain.

6
docs/02-quick-start.adoc
View File

@@ -2,12 +2,12 @@
=== Prerequisites
Dshackle is designed for cloud environment and supposed to be used withing Docker and/or Kubernetes. However it's a JVM
based application and therefore can be used in most of standard environment where Java Virtual Machine can be installed.
Dshackle is designed for the cloud environment and supposed to be used withing Docker and/or Kubernetes. However, it's a JVM
based application and, therefore, can be used in most of the standard environments where Java Virtual Machine can be installed.
We're going to use Docker image for this quick start.
For demo access we use gRPCurl tool, which can be installed from https://github.com/fullstorydev/grpcurl
For demo access, we use gRPCurl tool, which can be installed from https://github.com/fullstorydev/grpcurl
=== Configuration

16
docs/07-authentication.adoc
View File

@@ -2,13 +2,13 @@
=== Server authentication
Dshackle server supports both server and client certificate authentication, it's strongly recommended to use TLS to
connect to Dshackle server. More to that, the server uses gRPC which is based on HTTP/2, and most of 3rd party tools and
Dshackle server supports both server and client certificate authentication, and it's strongly recommended to use TLS to
connect to Dshackle server. More to that, the server uses gRPC, which is based on HTTP/2, and most of 3rd party tools and
libraries expect it to be encrypted.
Please note that for most of use cases for Dshackle, which is designed to be an internal load balancer, a self-signed
certificates would be enough. In the example below a https://github.com/square/certstrap[certstrap] tool is used to
generate certificates, but traditional `openssl` tool can be used as well.
Certificates would be enough. In the example below, a https://github.com/square/certstrap[certstrap] tool is used to
generate certificates, but the traditional `openssl` tool can be used as well.
==== Setup Server certificate
@@ -54,14 +54,14 @@ tls:
openssl s_client -alpn h2 -connect 127.0.0.1:2449 -CAfile out/ca.myhost.dev.crt
----
With the configuration above the server listen using TLS and the server identity can be verified by a client against public
server certificate. Please note that a server certificate doesn't prevent from connection by an unauthorized client, it only
With the configuration above, the server listens using TLS, and the server identity can be verified by a client against a public
server certificate. Please note that a server certificate doesn't prevent a connection from an unauthorized client; it only
verifies the server and encrypts a connection.
==== Use Client Certificate Authentication
To have authentication in both ways you'll need to configure client side certificates as well, at that case the server
will also verify each incoming connection and allow to connect only by a client with a trusted certificate.
To have authentication in both ways, you'll need to configure client side certificates as well, in that case the server
also verifies each incoming connection and allow to connect only by a client with a trusted certificate.
It's possible to connect a Dshackle server to another one, and to do so you'll probably want to use TLS as well.

3
docs/99-ending.adoc
View File

@@ -8,4 +8,5 @@ image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/
== Support
Contact splix@emeraldpay.io if you want to integrate Dshackle into your project or want to sponsor the development.
Want to support the project, prioritize a specific feature, or get commercial help with using Dshackle in your project?
Please contact splix@emeraldpay.io to discuss the possibility