= Dshackle
:imagesdir: docs/assets
ifdef::env-github[]
:imagesdir: https://raw.githubusercontent.com/emeraldpay/dshackle/master/docs/assets
endif::[]
:version: 0.12.0
:version-short: 0.12

image:https://github.com/drpcorg/dshackle/workflows/Tests/badge.svg["Unit Tests"]
image:https://img.shields.io/github/license/drpcorg/dshackle.svg?style=flat-square&maxAge=2592000["License",link="https://github.com/emeraldpay/dshackle/blob/master/LICENSE"]

[.lead]
_Dshackle is a Fault Tolerant Load Balancer for Blockchain API used at https://drpc.org/[DRPC]_

Is is a fork of the https://github.com/emeraldpay/dshackle[EmeraldPay's dshackle], adapted to work as a provider side adapter for the DRPC network. This project is specifically designed to integrate providers into the https://drpc.org/[DRPC] network, which is distinguished by its focus on low latency and high availability. Our service facilitates providers in easily connecting to the DRPC, enabling them to be part of a network that prioritizes robust, consistent, and responsive operations. Joining this network through our adapted service allows providers to contribute towards creating a highly available and resilient decentralized system.

The goal of the Dshackle is to provide a stable routing to multiple nodes, and ensure that each request is executed on an appropriate provider.
It considers nodes locations, state, current height, RPC methods it can provide and other characteristics.

It tries to recover from connection errors, faulty nodes, invalid responses, etc.
If upstream lags behind others, lost peers below required, started to resync or went down, then Dshackle temporarily excludes it from requests and returns it when the upstream problem is fixed.

The upstreams may be blockchain nodes such as Bitcoind, Geth, Parity, or public providers like DRPC.
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or data successfully broadcast to the network.

Provides:

- Standard Bitcoin and Ethereum JSON RPC API over HTTP and WebSocket
- Enhanced gRPC-based API, with upstream selection, async execution, etc
- **Secure** TLS with optional client authentication
- Blockchain-aware edge **caching**, in memory and Redis
- Routing based on **data availability** (peers, height, sync status)
- **Data consistency**, it always gives a most actual state
- Automatic **failover** and retry
- Separate public blockchain nodes from your internal servers

=== Supported chains
List of supported chains https://drpcorg.github.io/dshackle/networks.html[here]. Feel free to contribute new chains.

=== Architecture

image::dshackle-intro.png[alt="",width=80%,align="center"]

WARNING: The project is still under development, please use with caution.

== Quick Start

=== Configuration

Create file `dshackle.yaml` with the following content:

[source,yaml]
----
version: v1
port: 2449
tls:
  enabled: false

proxy:
  host: 0.0.0.0
  port: 8545
  routes:
    - id: eth
      blockchain: ethereum
    - id: btc
      blockchain: bitcoin

cluster:
  upstreams:
    - id: drpc-eth
      chain: ethereum
      connection:
        generic:
          rpc:
            url: "https://lb.drpc.org/ogrpc?network=ethereum&dkey=${DRPC_KEY}"
          ws:
            url: "wss://lb.drpc.org/ogws?network=ethereum&dkey=${DRPC_KEY}"
    - id: solana
      chain: solana
      connection:
        generic:
          rpc:
            url: "https://localhost:8899"
          ws:
            url: "wss://localhost:8900"
    - id: bitcoin-main
      chain: bitcoin
      connection:
        bitcoin:
          rpc:
            url: "http://localhost:8332"
            basic-auth:
              username: bitcoin
              password: mypassword
----

Which sets the following:

- gRPC access through 0.0.0.0:2449
** TLS security is disabled (_please don't use in production!_)
** compression is disabled for gRPC server (enabled by default)
- JSON RPC access through 0.0.0.0:8545 (both HTTP and WebsScket)
** proxy requests to Ethereum and Bitcoin upstreams
** request path for Ethereum Mainnet is `/eth` and `/btc` for bitcoin
** i.e. call Ethereum Mainnet by `POST http://127.0.0.0:8545/eth` with JSON RPC payload
- three upstreams - ethereum, solana and bitcoin
- for Ethereum Mainnet it connects using JSON RPC and WebSocket connections,
- for Bitcoin Mainnet only JSON RPC is used
- `${DRPC_KEY}` will be provided through environment variable

Please note that you can configure many upstreams for a single blockchains.
If there is more than one upstream, then Dshackle routes requests to them as Round Robin.
If one of them becomes unavailable, Dshackle continues to use only active nodes.

I.e., you can set up a node in the local network, plus Infura with `role: fallback`.
If anything happened to your local node, you still have access to a consistent state of the Ethereum blockchain via Infura.

link:docs[See full documentations].

==== Run docker image

Official Docker image you can find at: https://hub.docker.com/r/drpcorg/dshackle[drpcorg/dshackle]

.Setup DRPC key
[source,bash]
----
export DRPC_KEY=...
----

.Run Dshackle
[source,bash,subs="attributes"]
----
docker run -p 2449:2449 -p 8545:8545 -v $(pwd):/etc/dshackle -e "DRPC_KEY=$INFURA_USER" emeraldpay/dshackle:{version-short}
----

Now it listens on port 2449 at the localhost and can be connected from any gRPC compatible client.
Tools such as https://github.com/fullstorydev/grpcurl[gRPCurl] can use protobuf definitions from proto reflection and connect to it

Alternatively you can connect to port 8545 with traditional JSON RPC requests

== Documentation

For detailed documentation see link:docs/[] directory.

== Development

WARNING: The code in `master` branch is considered a development version, which may lack proper testing and should not be used in production.

=== Setting up environment

Dshackle is JVM based project written in Kotlin.
To build and run it from sources you'll need to install https://openjdk.org/projects/jdk/20/[Java JDK] and https://gradle.org/[Gradle]

=== Tests

There is a short guide "How to write tests in dshackle"

1. First of all, all new tests must be written in Kotlin. Groovy tests are fixed only if necessary.
2. To create mocks we use `org.mockito.kotlin.mock` (instead of `Mockito` class) and all extensions from that package, so that we can write our test code in kotlin-style.
3. There are 2 main assertion libs in kotlin: the basic one `org.junit.jupiter.api.Assertions` and the more advanced `org.assertj.core.api.Assertions`.
 - junit assertion lib is super simple - there are a lot of assert functions that just compare inputs
 - assertj lib is a much more advanced lib that provides a rich set of assertion methods that read like natural language. For example, you can write assertions like `assertThat(result).isEqualTo(expected)` or `assertThat(list).containsExactly(1, 2, 3)`. This makes your tests more self-explanatory and easier to understand. Also, you can easily compare objects with ignoring some fields that sometimes can be really useful. And many other features.
4. To sum up, for assertions let's use `assertj` lib.
5. To test the reactive code we use `StepVerifier` from the reactor package. Just call `create` method and pass your `Flux` or `Mono` and add necessary assert methods which start with the `expect` prefix. Also, if your reactive pipeline is dependent on periodic operations you can use `withVirtualTime` method, with which you no longer need to work with real time.

=== Build Dshackle

==== Build everything

[source,bash]
----
gradle build
----

==== Make a Zip distribution

[source,bash]
----
gradle distZip
----

You can find a redistributable zip in `build/distributions`

==== Make a Docker distribution

[source, bash]
----
gradle jib -Pdocker=gcr.io/myproject
----

Gradle will prepare a Docker image and upload it to your custom Docker Registry at `gcr.io/myproject` (please change to address of your actual registry)

== Community

Join our https://drpc.org/discord[Discord]

== License

Copyright 2021 EmeraldPay, Inc

Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License.
You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and limitations under the License.

=== Modifications Copyright:

Modifications made by p2p.org in 2022 are licensed under the same Apache License, Version 2.0. These modifications are copyrighted by p2p.org.