== Enhanced Methods

IMPORTANT: Dshackle provides an enhanced API based on gRPC and Protobuf, in addition to JSON RPC proxy

Dshackle provides an enhanced and unified API based on HTTP2, gRPC and Protobuf.
It works in addition to JSON RPC proxy and can be used separately.
Clients can be generated for all major programming languages, and there're official libraries for Java and Javascript.

NOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.

=== gRPC definition

The source code for the Protobuf definitions could be found in link:../proto/[/proto].

.Service API
[source,proto]
----
service Blockchain {
    rpc SubscribeHead (Chain) returns (stream ChainHead) {}
    rpc SubscribeBalance (BalanceRequest) returns (stream AddressBalance) {}
    rpc SubscribeTxStatus (TxStatusRequest) returns (stream TxStatus) {}

    rpc GetBalance (BalanceRequest) returns (stream AddressBalance) {}

    rpc NativeCall (NativeCallRequest) returns (stream NativeCallReplyItem) {}
    rpc NativeSubscribe (NativeSubscribeRequest) returns (stream NativeSubscribeReplyItem) {}

    rpc Describe (DescribeRequest) returns (DescribeResponse) {}
    rpc SubscribeStatus (StatusRequest) returns (stream ChainStatus) {}
}
----

=== Wrapped JSON RPC methods

To call standard JSON RPC methods provided by Ethereum/Bitcoin you use `NativeCall` wrapping method, which provides additional flexibility and configuration for the calls.

.NativeCallRequest
[source,proto]
----
message NativeCallRequest {
    ChainRef chain = 1;
    repeated NativeCallItem items = 2;
    Selector selector = 3;
    int32 quorum = 4;
    AvailabilityEnum min_availability = 5;
}

message NativeCallItem {
    uint32 id = 1;
    string method = 3;
    bytes payload = 4;
    uint64 nonce = 5;
}
----

Where:

- `chain` target chain (see reference for ids)
- `items` as a list of independent requests, which may be executed in different nodes in parallels or in different order, with:
* `method` - a JSON RPC standard name, ex: `eth_getBlockByHash`
* `payload` - list of parameters for the methods, encoded as JSON string, ex. `["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]`
- `Selector` and `AvailabilityEnum` are described in reference, in short they allow to specify which nodes must be selected
to execute the reques (i.e. "execute only on an archive node")

.NativeCallReplyItem
[source,proto]
----

message NativeCallReplyItem {
    uint32 id = 1;
    bool succeed = 2;
    bytes payload = 3;
    bytes error = 4;
    NativeCallReplySignature signature = 5;
}

message NativeCallReplySignature {
    uint64 nonce = 1;
    bytes signature = 2;
    uint64 key_id = 3;
    string upstream_id = 4;
}
----

Where:

- `payload` is JSON response for a particular call (`result` field), encoded into a string (`succeed` is true)
- or `error` if request failed (`succeed` is false)

NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response.
It allows building non-blocking queries

[#signatures]
=== Signed JSON RPC Responses

Dshackle can sign the responses it received from an upstream.
It can be enabled on server by configuring a path to a Secp256K1 Key used for signing.
And passing a `nonce` as part of the call request.
When both of them are set Dshackle adds a Signature to the response, which can travel through multiple levels of Dshackle-Dshackle connections.

WARNING: Caching is disabled for signed requests, and you always touch an actual node even if there is a ready to use data in local cache.

The signed message looks like:
----
DSHACKLESIG/$nonce/$upstreamId/hex(sha256($response))
----

.Where
- `nonce` is a 64-bit number provided with the request
- `upstreamId` id of an upstream which produced the result
- `response` is a part of the original JSON RPC message, i.e. it's what you have in `payload` field of `NativeCallReplyItem`

.A signed response includes:
- `nonce` original nonce used for the call
- `signature` signature bytes (of the message above)
- `key_id` identifier of a key used for the signing
- `upstream_id` id of upstream which produce the response

==== How to generate a key

Here we generate a pair of Secret and Public keys using openssl.

[source, bash]
----
export KEYNAME=mykey

openssl ecparam -name secp256k1 -out $KEYNAME_param.pem
openssl ecparam -in $KEYNAME_param.pem -genkey -noout -out $KEYNAME.pem

openssl ec -in $KEYNAME.pem -text
openssl ec -in $KEYNAME.pem -out ${KEYNAME}_pub.pem -pubout

rm $KEYNAME_param.pem
cat ${KEYNAME}_pub.pem
----

As a result you get `mykey.pem` with secret key to use on server, and `mykey_pub.pem` with public key to use on client to verify signatures.

==== How to verify a signature

Here is the example how to verify the signature with command line, and it can be easily adapted for your language of choice.

.First, let's prepare all the values:
[source, bash]
----
export PUBKEY=testing/dshackle/test_key.pub
export NONCE=10

export UPSTREAM=infura
export PAYLOAD='["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]'

export SIGNATURE=3045022100be1d730e0e381e25bff64f0fc598d19e31688a01db751098d0ed21847ca785b0022002f5651a0e8d447b0815aeb7b48738cb470cf46d60ee4e2f5bc9c0dc4e072dc3
----

.Then rebuild the signed message:
[source, bash]
----
echo -n "DSHACKLESIG/$NONCE/$UPSTREAM/" > msg.txt
echo -n $PAYLOAD | shasum -a 256 - | awk '{ printf $1 }' >> msg.txt
----

.And save signature as a binary file:
[source, bash]
----
rm -f msg.sig && echo $SIGNATURE | xxd -r -p - msg.sig
----

.Now you can verify the payload with the following:
[source, bash]
----
openssl dgst -sha256 -verify $PUBKEY -signature msg.sig msg.txt
----

.Which should print:
----
Verified OK
----

==== What is the Key Identifier?

Key Id is the first 64 bits of SHA-256 hash of the x509 encoded Public Key.
It's provided with the Signed Response for a reference.

You can get it with:
[source, bash]
----
cat $PUBKEY | sed -e '$ d' | awk '(NR>1)' | base64 -d | shasum -a 256 - | head -c 16
----

=== Wrapped JSON RPC subscriptions

Most of Ethereum APIs provides _subscription_ to events usually accessed through WebSocket connection.
Dshackle gives access to same events through gRPC protocol via the `NativeSubscribe` method.

NOTE: Dshackle doesn't actually wrap existing subscription or dispatch request to an upstream.
It rather generates same events based on the available data, i.e., aggregates it from multiple upstreams.

Supported subscriptions:

- `newHeads`
- `logs`
- `syncing`

Method data:

[source,proto]
----
message NativeSubscribeRequest {
    ChainRef chain = 1;
    string method = 2;
    bytes payload = 3;
}

message NativeSubscribeReplyItem {
    bytes payload = 1;
}
----

Where:

- `method` is a subscriptions method (one of `newHeads`, `logs` or `syncing`)
- `payload` in request is optional subscription params object, which exists only for `logs` methods.
In that case it may be `address` or `topics`.
Both address and topics can be a string or array of strings.
Empty payload for `logs` accepted as subscription to _all_ events.
- `payload` in reply item is as subscription response encoded as JSON

For example to subscribe to USDC ERC-20 coin Approval events on Ethereum mainnet the request would be:

- `chain=100`
- `method=logs`
- `payload={"address": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", "topics": ["0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925"]}`

=== SubscribeHead

This methods provides subscription to the new blocks on the specified chain.
Returns stream of blocks right after it was accepted (and verified by Dshackle) by any of the upstreams.

.ChainHead
[source,proto]
----
message ChainHead {
    ChainRef chain = 1;
    uint64 height = 2;
    string block_id = 3;
    uint64 timestamp = 4;
    bytes weight = 5;
    uint64 reorg = 6;
}
----

Where:

- `chain` - chain id
- `height` - block number
- `block_id` - block hash, as a string (please note that it doesn't have `0x` prefix)
- `timestamp` - timestamp of that block
- `weight` - total network difficulty on that block, as raw bytes
- `reorg` - number of reorganized blocks, if reorg happened

=== SubscribeBalance or GetBalance

Subscribes to changes (`SubscribeBalance`) or get current (`GetBalance`) balance for a single address, or a set of addresses.
By default, it supports only main protocol coin (i.e. `bitcoin`, `ether`), but can be configured to support ERC-20 on Ethereum (see link:reference-configuration.adoc[Reference Configuration])

.Request
[source,proto]
----
message BalanceRequest {
    Asset asset = 1;
    AnyAddress address = 2;
}

message AnyAddress {
    oneof addr_type {
        SingleAddress address_single = 1;
        MultiAddress address_multi = 2;
        XpubAddress address_xpub = 3;
        ReferenceAddress address_ref = 4;
    }
}
----

.Response
[source,proto]
----
message AddressBalance {
    Asset asset = 1;
    SingleAddress address = 2;
    string balance = 3;
}
----

==== SubscribeTxStatus

Subscribes to transaction confirmations.
Allows to send a transactions and then listen to all changes until it gets enough confirmations.
Changes are `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED`

.Request
[source,proto]
----
message TxStatusRequest {
    ChainRef chain = 1;
    string tx_id = 2;
    uint32 confirmation_limit = 3;
}
----

.Response (stream of)
[source,proto]
----
message TxStatus {
    string tx_id = 1;
    bool broadcasted = 2;
    bool mined = 3;
    BlockInfo block = 4;
    uint32 confirmations = 5;
}
----

=== gRPC Client Libraries

See link:11-client-libraries.adoc[Client Libraries] documentation.