update docs (#348)
This commit is contained in:
a10zn8
@@ -4,7 +4,7 @@ Dshackle is an L7 Blockchain API Load Balancer with automatic discovery and heal
|
||||
It is designed to work as an edge proxy, middle proxy, or API gateway.
|
||||
|
||||
Dshackle provides a high level aggregated API on top of several underlying upstreams.
|
||||
I.e. to blockchain nodes such as Bitcoind, Geth, Parity, or providers like Infura, and so on.
|
||||
I.e. to blockchain nodes such as Bitcoind, Geth, Parity, or providers like DRPC, and so on.
|
||||
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or data successfully broadcasted to the network.
|
||||
|
||||
Example use cases:
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
|
||||
Dshackle is built using/based on:
|
||||
|
||||
- Java Virtual Machine, Java 8+
|
||||
- Java Virtual Machine, Java 20+
|
||||
- Kotlin
|
||||
- Spring Framework, Boot and Reactor
|
||||
|
||||
|
||||
@@ -33,7 +33,7 @@ proxy:
|
||||
blockchain: kovan
|
||||
cluster:
|
||||
upstreams: # <4>
|
||||
- id: infura-eth
|
||||
- id: drpc-eth
|
||||
chain: ethereum # <5>
|
||||
method-groups:
|
||||
enabled:
|
||||
@@ -41,9 +41,9 @@ cluster:
|
||||
connection:
|
||||
ethereum:
|
||||
rpc: # <6>
|
||||
url: "https://mainnet.infura.io/v3/${INFURA_USER}" # <7>
|
||||
url: "https://lb.drpc.org/ogrpc?network=ethereum&dkey=${DRPC_KEY}" # <7>
|
||||
ws: # <8>
|
||||
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
|
||||
url: "wss://lb.drpc.org/ogws?network=ethereum&dkey=${DRPC_KEY}"
|
||||
- id: local-eth # <9>
|
||||
chain: ethereum
|
||||
connection:
|
||||
@@ -68,32 +68,32 @@ cluster:
|
||||
<4> sets up 2 upstreams
|
||||
<5> one for Ethereum Mainnet, using
|
||||
<6> HTTP and
|
||||
<7> `${INFURA_USER}` value is provided through environment variable
|
||||
<8> in addition to HTTPS is uses Websocket protocol to connect to Infura, used to subscribe to updates
|
||||
<7> `${DRPC_KEY}` value is provided through environment variable
|
||||
<8> in addition to HTTPS is uses Websocket protocol to connect to DRPC, used to subscribe to updates
|
||||
<9> setup another upstream for Ethereum Mainnet
|
||||
<10> which connects to a node in the internal networks, without any authentication at this case
|
||||
<11> and another for Bitcoin
|
||||
<12> with Basic Auth for bitcoin node connection
|
||||
|
||||
The configuration above sets up the Dshackle to provide a fault-tolerant access to Bitcoin and Ethereum blockchain.
|
||||
And for Ethereum, at this particular example, it uses a Round Robin Load Balancing over the Infura and the local node.
|
||||
And for Ethereum, at this particular example, it uses a Round Robin Load Balancing over the DRPC and the local node.
|
||||
|
||||
See detailed link:reference-configuration.adoc[Configuration Reference]
|
||||
|
||||
==== Run as docker
|
||||
|
||||
Official Docker image you can find at: https://hub.docker.com/r/emeraldpay/dshackle[emeraldpay/dshackle]
|
||||
Official Docker image you can find at: https://hub.docker.com/r/p2p-org/dshackle[p2p-org/dshackle]
|
||||
|
||||
.Setup Infura username
|
||||
.Setup DRPC key
|
||||
[source,bash]
|
||||
----
|
||||
export INFURA_USER=...
|
||||
export DRPC_KEY=...
|
||||
----
|
||||
|
||||
.Run Dshackle
|
||||
[source,bash,subs="attributes"]
|
||||
----
|
||||
docker run -p 2449:2449 -p 8545:8545 -v $(pwd):/etc/dshackle -e "INFURA_USER=$INFURA_USER" emeraldpay/dshackle:{version-short}
|
||||
docker run -p 2449:2449 -p 8545:8545 -v $(pwd):/etc/dshackle -e "DRPC_KEY=$DRPC_KEY" p2p-org/dshackle:{version-short}
|
||||
----
|
||||
|
||||
==== Access using JSON RPC
|
||||
@@ -120,7 +120,7 @@ curl --request POST \
|
||||
.Connect and listen for new blocks on Ethereum Mainnet
|
||||
[source,bash]
|
||||
----
|
||||
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead
|
||||
grpcurl -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead
|
||||
----
|
||||
|
||||
`type: 100` specifies the blockchain id, and 100 means Ethereum Mainnet.
|
||||
@@ -148,35 +148,4 @@ grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plai
|
||||
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet.
|
||||
It's one of the services provided by Dshackle, in addition to standard methods provided by RPC JSON of underlying nodes.
|
||||
|
||||
.You can also subscribe to balances changes of the balance on an address:
|
||||
[source,bash]
|
||||
----
|
||||
grpcurl -import-path ./proto/ -proto blockchain.proto -d '{"asset": {"chain": "100", "code": "ether"}, "address": {"address_single": {"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}}}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeBalance
|
||||
----
|
||||
|
||||
.and see how balance of the contract responsible for Wrapped Ether is changing:
|
||||
----
|
||||
{
|
||||
"asset": {
|
||||
"chain": "CHAIN_ETHEREUM",
|
||||
"code": "ETHER"
|
||||
},
|
||||
"address": {
|
||||
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
|
||||
},
|
||||
"balance": "2410941696896999943701015"
|
||||
}
|
||||
{
|
||||
"asset": {
|
||||
"chain": "CHAIN_ETHEREUM",
|
||||
"code": "ETHER"
|
||||
},
|
||||
"address": {
|
||||
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
|
||||
},
|
||||
"balance": "2410930748488073834320430"
|
||||
}
|
||||
...
|
||||
----
|
||||
|
||||
See other enhanced methods in the link:07-methods.adoc[Documentation for Enhanced Methods]
|
||||
|
||||
@@ -12,10 +12,9 @@ Those protocols can be configures with additional security, TLS and authenticati
|
||||
|
||||
=== Notes on upstream configuration
|
||||
|
||||
==== Ethereum
|
||||
==== Generic
|
||||
|
||||
- Most of Ethereum nodes support WebSocket connection, in addition to the JSON RPC.
|
||||
If it's available on your node, it's suggested to configure both JSON RPC and WebSocket connection
|
||||
- Most common way to connect upstream. Supported for Ethereum, Starknet, Solana and Varanet upstreams. It uses rpc and ws connection to provide access via jsonprc calls.
|
||||
|
||||
==== Ethereum PoS
|
||||
|
||||
@@ -40,15 +39,6 @@ Run it with `-reindex` option, or set `txindex=1` in the config.
|
||||
version: v1
|
||||
|
||||
cluster:
|
||||
defaults:
|
||||
- chains:
|
||||
- ethereum
|
||||
options:
|
||||
min-peers: 10
|
||||
- chains:
|
||||
- kovan
|
||||
options:
|
||||
min-peers: 2
|
||||
upstreams:
|
||||
- id: us-nodes
|
||||
node-id: 1
|
||||
@@ -64,7 +54,7 @@ cluster:
|
||||
ca: ca.crt
|
||||
certificate: client.crt
|
||||
key: client.p8.key
|
||||
- id: infura-eth
|
||||
- id: drpc-eth
|
||||
node-id: 2
|
||||
chain: ethereum
|
||||
role: fallback
|
||||
@@ -76,25 +66,18 @@ cluster:
|
||||
ethereum-pos:
|
||||
execution:
|
||||
rpc:
|
||||
url: "https://mainnet.infura.io/v3/${INFURA_USER}"
|
||||
basic-auth:
|
||||
username: ${INFURA_USER}
|
||||
password: ${INFURA_PASSWD}
|
||||
url: "https://lb.drpc.org/ogrpc?network=ethereum&dkey=${DRPC_KEY}"
|
||||
ws:
|
||||
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
|
||||
basic-auth:
|
||||
username: ${INFURA_USER}
|
||||
password: ${INFURA_PASSWD}
|
||||
- id: ethereum-pos
|
||||
url: "wss://lb.drpc.org/ogws?network=ethereum&dkey=${DRPC_KEY}"
|
||||
- id: solana
|
||||
node-id: 3
|
||||
chain: ropsten
|
||||
chain: solana
|
||||
connection:
|
||||
ethereum-pos:
|
||||
execution:
|
||||
rpc:
|
||||
url: ${ROPSTEN_NODE_RPC_URL}
|
||||
ws:
|
||||
url: ${ROPSTEN_NODE_WS_URL}
|
||||
generic:
|
||||
rpc:
|
||||
url: ${SOLANA_NODE_RPC_URL}
|
||||
ws:
|
||||
url: ${SOLANA_NODE_WS_URL}
|
||||
----
|
||||
|
||||
There are two main segments for upstreams configuration:
|
||||
@@ -104,17 +87,15 @@ There are two main segments for upstreams configuration:
|
||||
|
||||
In the example above we have:
|
||||
|
||||
- default configuration for _Ethereum Mainnet_ which accepts upstream as valid when it not in fast synchronization mode and has at least 10 peers.For _Kovan Testnet_ nodes the requirements are much relieved
|
||||
- as upstreams it has 2 configurations
|
||||
* as upstreams it has 2 configurations - for ethereum and solana
|
||||
* balancer connects to another Dshackle/another machine by using gRPC protocol
|
||||
** accepts (i.e. proxies) any blockchain available on that remote
|
||||
** verifies TLS certificate of the server
|
||||
** uses client certificate for authentication, i.e. remote server is accepting only clients authenticated by a certificate
|
||||
* connects to Infura provided _Ethereum Mainnet_
|
||||
* connects to DRPC provided _Ethereum Mainnet_
|
||||
** as a _fallback_ upstream, which means that it's used only if `us-nodes` fails
|
||||
** configuration is using placeholders for `${INFURA_USER}` and `${INFURA_PASSWD}` which will be replaced with corresponding environment variables values
|
||||
** uses Basic Authentication to authenticate requests on Infura
|
||||
** label `[provider: infura]` is set for that particular upstream, which can be selected during a request.For example for some requests you may want to use nodes with that label only, i.e. _"send that tx to infura nodes only"_, or _"read only from archive node, with label [archive: true]"_
|
||||
** configuration is using placeholders for `${DRPC_KEY}` which will be replaced with corresponding environment variables values
|
||||
** label `[provider: drpc]` is set for that particular upstream, which can be selected during a request.For example for some requests you may want to use nodes with that label only, i.e. _"send that tx to drpc nodes only"_, or _"read only from archive node, with label [archive: true]"_
|
||||
** upstream validation (peers, sync status, etc) is disabled for that particular upstream
|
||||
|
||||
=== Nodes
|
||||
@@ -127,51 +108,58 @@ In the example above we have:
|
||||
|
||||
Currently, dshackle supports next chains (should be used as chain names in config):
|
||||
|
||||
- ethereum (eth)
|
||||
- ethereum-classic (etc)
|
||||
- polygon (matic)
|
||||
- polygon-mumbai
|
||||
- arbitrum (arb)
|
||||
- arbitrum-testnet
|
||||
- arbitrum-nova
|
||||
- optimism
|
||||
- optimism-testnet
|
||||
- binance (bsc, bnb-smart-chain)
|
||||
- bsc-testnet
|
||||
- zksync
|
||||
- zksync-testnet
|
||||
- polygon-zkevm
|
||||
- polygon-zkevm-testnet
|
||||
- morden
|
||||
- kovan (kovan-testnet)
|
||||
- goerli (goerli-testnet)
|
||||
- rinkeby (rinkeby-testnet)
|
||||
- ropsten (ropsten-testnet)
|
||||
- ethereum-holesky
|
||||
- bitcoin (bitcoin-testnet)
|
||||
- base
|
||||
- base-goerli
|
||||
- linea
|
||||
- linea-goerli
|
||||
- fantom
|
||||
- fantom-testnet
|
||||
- gnosis
|
||||
- gnosis-chiado
|
||||
- avalanche
|
||||
- avalanche-fuji
|
||||
- aurora
|
||||
- aurora-testnet
|
||||
- scroll-alphanet
|
||||
- scroll-sepolia
|
||||
- mantle
|
||||
- mantle-testnet
|
||||
- klaytn
|
||||
- klaytn-baobab
|
||||
- celo
|
||||
- celo-alfajores
|
||||
- moonbeam
|
||||
- moonriver
|
||||
- moonbase-alpha
|
||||
* ethereum
|
||||
** ethereum (eth)
|
||||
** ethereum-classic (etc)
|
||||
** polygon (matic)
|
||||
** polygon-mumbai
|
||||
** arbitrum (arb)
|
||||
** arbitrum-testnet
|
||||
** arbitrum-nova
|
||||
** optimism
|
||||
** optimism-testnet
|
||||
** optimism-sepolia
|
||||
** binance (bsc, bnb-smart-chain)
|
||||
** bsc-testnet
|
||||
** zksync
|
||||
** zksync-testnet
|
||||
** polygon-zkevm
|
||||
** polygon-zkevm-testnet
|
||||
** morden
|
||||
** kovan (kovan-testnet)
|
||||
** goerli (goerli-testnet)
|
||||
** rinkeby (rinkeby-testnet)
|
||||
** ropsten (ropsten-testnet)
|
||||
** ethereum-holesky
|
||||
** bitcoin (bitcoin-testnet)
|
||||
** base
|
||||
** base-goerli
|
||||
** linea
|
||||
** linea-goerli
|
||||
** fantom
|
||||
** fantom-testnet
|
||||
** gnosis
|
||||
** gnosis-chiado
|
||||
** avalanche
|
||||
** avalanche-fuji
|
||||
** aurora
|
||||
** aurora-testnet
|
||||
** scroll-alphanet
|
||||
** scroll-sepolia
|
||||
** mantle
|
||||
** mantle-testnet
|
||||
** klaytn
|
||||
** klaytn-baobab
|
||||
** celo
|
||||
** celo-alfajores
|
||||
** moonbeam
|
||||
** moonriver
|
||||
** moonbase-alpha
|
||||
* starknet
|
||||
* varanet
|
||||
* solana
|
||||
|
||||
All supported blockchains are listed link:/foundation/src/main/resources/chains.yaml[here]. Feel free to contribute new chains.
|
||||
|
||||
=== Roles and Fallback upstream
|
||||
|
||||
|
||||
@@ -8,12 +8,12 @@ Prepare configuration files `dshackle.yaml` and `upstreams.yaml` in the current
|
||||
|
||||
[source,bash,subs="attributes"]
|
||||
----
|
||||
docker run -p 2449:2449 -v $(pwd):/config -w /config emeraldpay/dshackle:{version-short}
|
||||
docker run -p 2449:2449 -v $(pwd):/config -w /config p2p-org/dshackle:{version-short}
|
||||
----
|
||||
|
||||
=== Install & Run manually
|
||||
|
||||
1. Download latest release from https://github.com/emeraldpay/dshackle/releases
|
||||
1. Download latest release from https://github.com/p2p-org/dshackle/releases
|
||||
2. Unpack `unzip dshackle-0.6.0.zip`
|
||||
3. Copy to `/opt/dshackle`
|
||||
4. Setup configuration in `/etc/dshackle`
|
||||
|
||||
@@ -146,7 +146,7 @@ Here is the example how to verify the signature with command line, and it can be
|
||||
export PUBKEY=testing/dshackle/test_key.pub
|
||||
export NONCE=10
|
||||
|
||||
export UPSTREAM=infura
|
||||
export UPSTREAM=drpc
|
||||
export PAYLOAD='["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]'
|
||||
|
||||
export SIGNATURE=3045022100be1d730e0e381e25bff64f0fc598d19e31688a01db751098d0ed21847ca785b0022002f5651a0e8d447b0815aeb7b48738cb470cf46d60ee4e2f5bc9c0dc4e072dc3
|
||||
@@ -258,67 +258,3 @@ Where:
|
||||
- `weight` - total network difficulty on that block, as raw bytes
|
||||
- `reorg` - number of reorganized blocks, if reorg happened
|
||||
|
||||
=== SubscribeBalance or GetBalance
|
||||
|
||||
Subscribes to changes (`SubscribeBalance`) or get current (`GetBalance`) balance for a single address, or a set of addresses.
|
||||
By default, it supports only main protocol coin (i.e. `bitcoin`, `ether`), but can be configured to support ERC-20 on Ethereum (see link:reference-configuration.adoc[Reference Configuration])
|
||||
|
||||
.Request
|
||||
[source,proto]
|
||||
----
|
||||
message BalanceRequest {
|
||||
Asset asset = 1;
|
||||
AnyAddress address = 2;
|
||||
}
|
||||
|
||||
message AnyAddress {
|
||||
oneof addr_type {
|
||||
SingleAddress address_single = 1;
|
||||
MultiAddress address_multi = 2;
|
||||
XpubAddress address_xpub = 3;
|
||||
ReferenceAddress address_ref = 4;
|
||||
}
|
||||
}
|
||||
----
|
||||
|
||||
.Response
|
||||
[source,proto]
|
||||
----
|
||||
message AddressBalance {
|
||||
Asset asset = 1;
|
||||
SingleAddress address = 2;
|
||||
string balance = 3;
|
||||
}
|
||||
----
|
||||
|
||||
==== SubscribeTxStatus
|
||||
|
||||
Subscribes to transaction confirmations.
|
||||
Allows to send a transactions and then listen to all changes until it gets enough confirmations.
|
||||
Changes are `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED`
|
||||
|
||||
.Request
|
||||
[source,proto]
|
||||
----
|
||||
message TxStatusRequest {
|
||||
ChainRef chain = 1;
|
||||
string tx_id = 2;
|
||||
uint32 confirmation_limit = 3;
|
||||
}
|
||||
----
|
||||
|
||||
.Response (stream of)
|
||||
[source,proto]
|
||||
----
|
||||
message TxStatus {
|
||||
string tx_id = 1;
|
||||
bool broadcasted = 2;
|
||||
bool mined = 3;
|
||||
BlockInfo block = 4;
|
||||
uint32 confirmations = 5;
|
||||
}
|
||||
----
|
||||
|
||||
=== gRPC Client Libraries
|
||||
|
||||
See link:11-client-libraries.adoc[Client Libraries] documentation.
|
||||
|
||||
@@ -11,16 +11,6 @@ immediately evicted from the caches.
|
||||
|
||||
Dshackle keeps latest blocks in memory (by default 64 blocks)
|
||||
|
||||
=== Disable all caching and cache related optimisations
|
||||
Sometimes you may need to disable all caching and optimisations and just send all requests to remote upstreams as is.
|
||||
|
||||
.Disable cache config (dshackle.yaml)
|
||||
[source, yaml]
|
||||
----
|
||||
cache:
|
||||
requests-cache-enabled: false
|
||||
----
|
||||
|
||||
=== Redis cache
|
||||
|
||||
Dshackle can optionally cache blocks and transactions in Redis cache. The values are cached up to 1 hour, but
|
||||
|
||||
@@ -1,43 +0,0 @@
|
||||
== Client Libraries using native gRPC-based protocol
|
||||
|
||||
=== Protobuf
|
||||
|
||||
Original Protobuf definitions can be found at https://github.com/emeraldpay/emerald-grpc
|
||||
|
||||
=== Javascript gRPC Client
|
||||
image:https://img.shields.io/npm/v/@emeraldpay/grpc-client.svg["npm (scoped)", link="https://www.npmjs.com/package/@emeraldpay/grpc-client"]
|
||||
|
||||
Emerald Javascript gRPC Client is a low level API and Stubs to access different services of Emerald, including Dshackle, more
|
||||
details at https://github.com/emeraldpay/emerald-js-grpc
|
||||
|
||||
[source,json]
|
||||
----
|
||||
"dependencies": {
|
||||
"@emeraldpay/grpc-client": "0.11.0-0.2",
|
||||
}
|
||||
----
|
||||
|
||||
=== Java gRPC Client
|
||||
image:https://api.bintray.com/packages/emerald/emerald-grpc/emerald-grpc/images/download.svg[link="https://bintray.com/emerald/emerald-grpc/emerald-grpc/"]
|
||||
|
||||
Emerald Java gRPC Client is a low level API and Stubs to access different services of Emerald, including Dshackle, more
|
||||
details at https://github.com/emeraldpay/emerald-java-client
|
||||
|
||||
[source,groovy]
|
||||
----
|
||||
repositories {
|
||||
maven {
|
||||
url "https://dl.bintray.com/emerald/emerald-grpc"
|
||||
}
|
||||
}
|
||||
|
||||
dependencies {
|
||||
compile "io.emeraldpay:emerald-grpc:0.6.0-0.2"
|
||||
}
|
||||
----
|
||||
|
||||
=== EtherJar Java library
|
||||
image:https://api.bintray.com/packages/infinitape/etherjar/etherjar-domain/images/download.svg[link="https://bintray.com/infinitape/etherjar"]
|
||||
|
||||
EtherJar is a framework agnostic modular Java 8+ integration library for Ethereum blockchains, it supports different protocols
|
||||
to connect to an Ethereum API, including gRPC protocol via module `etherjar-rpc-emerald`. More details at https://github.com/infinitape/etherjar
|
||||
@@ -1,10 +1,10 @@
|
||||
== Links
|
||||
|
||||
- Github: https://github.com/emeraldpay/dshackle
|
||||
- Github: https://github.com/p2p-org/dshackle
|
||||
|
||||
== Chat
|
||||
|
||||
image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge]
|
||||
Join our https://drpc.org/discord[Discord]
|
||||
|
||||
== Support
|
||||
|
||||
|
||||
@@ -6,7 +6,7 @@ Dshackle is a L7 Load Balancer for Blockchain APIs with automatic discovery, hea
|
||||
client authentication, and many other features.It can be configured as an edge proxy, middle proxy or API gateway.
|
||||
|
||||
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers,
|
||||
such as Geth, Parity, Infura, etc), automatically verifies their availability and the current status of the network,
|
||||
such as DRPC, etc), automatically verifies their availability and the current status of the network,
|
||||
it routes requests to available node, and makes sure the response is consistent and/or data successfully broadcasted to
|
||||
the networks.
|
||||
|
||||
@@ -43,7 +43,6 @@ Main goals:
|
||||
. link:08-authentication.adoc[Authentication]
|
||||
. link:09-quorum-and-selectors.adoc[Quorum and Selectors]
|
||||
. link:10-caching.adoc[Caching]
|
||||
. link:11-client-libraries.adoc[Client Libraries]
|
||||
|
||||
== Reference
|
||||
|
||||
@@ -51,12 +50,8 @@ Main goals:
|
||||
|
||||
== Chat
|
||||
|
||||
image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge]
|
||||
|
||||
== Support
|
||||
|
||||
Contact splix@emeraldpay.io if you want to integrate Dshackle into your project or want to sponsor the development.
|
||||
Join our https://drpc.org/discord[Discord]
|
||||
|
||||
== Links
|
||||
|
||||
- Github: https://github.com/emeraldpay/dshackle
|
||||
- Github: https://github.com/p2p-org/dshackle
|
||||
Reference in New Issue
Block a user