261 lines
8.1 KiB
Plaintext
261 lines
8.1 KiB
Plaintext
== Enhanced Methods
|
|
|
|
IMPORTANT: Dshackle provides an enhanced API based on gRPC and Protobuf, in addition to JSON RPC proxy
|
|
|
|
Dshackle provides an enhanced and unified API based on HTTP2, gRPC and Protobuf.
|
|
It works in addition to JSON RPC proxy and can be used separately.
|
|
Clients can be generated for all major programming languages, and there're official libraries for Java and Javascript.
|
|
|
|
NOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.
|
|
|
|
=== gRPC definition
|
|
|
|
The source code for the Protobuf definitions could be found in https://github.com/drpcorg/emerald-grpc/tree/master/proto[/proto].
|
|
|
|
.Service API
|
|
[source,proto]
|
|
----
|
|
service Blockchain {
|
|
rpc SubscribeHead (Chain) returns (stream ChainHead) {}
|
|
rpc SubscribeBalance (BalanceRequest) returns (stream AddressBalance) {}
|
|
rpc SubscribeTxStatus (TxStatusRequest) returns (stream TxStatus) {}
|
|
|
|
rpc GetBalance (BalanceRequest) returns (stream AddressBalance) {}
|
|
|
|
rpc NativeCall (NativeCallRequest) returns (stream NativeCallReplyItem) {}
|
|
rpc NativeSubscribe (NativeSubscribeRequest) returns (stream NativeSubscribeReplyItem) {}
|
|
|
|
rpc Describe (DescribeRequest) returns (DescribeResponse) {}
|
|
rpc SubscribeStatus (StatusRequest) returns (stream ChainStatus) {}
|
|
}
|
|
----
|
|
|
|
=== Wrapped JSON RPC methods
|
|
|
|
To call standard JSON RPC methods provided by Ethereum/Bitcoin you use `NativeCall` wrapping method, which provides additional flexibility and configuration for the calls.
|
|
|
|
.NativeCallRequest
|
|
[source,proto]
|
|
----
|
|
message NativeCallRequest {
|
|
ChainRef chain = 1;
|
|
repeated NativeCallItem items = 2;
|
|
Selector selector = 3;
|
|
int32 quorum = 4;
|
|
AvailabilityEnum min_availability = 5;
|
|
}
|
|
|
|
message NativeCallItem {
|
|
uint32 id = 1;
|
|
string method = 3;
|
|
bytes payload = 4;
|
|
uint64 nonce = 5;
|
|
}
|
|
----
|
|
|
|
Where:
|
|
|
|
- `chain` target chain (see reference for ids)
|
|
- `items` as a list of independent requests, which may be executed in different nodes in parallels or in different order, with:
|
|
* `method` - a JSON RPC standard name, ex: `eth_getBlockByHash`
|
|
* `payload` - list of parameters for the methods, encoded as JSON string, ex. `["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]`
|
|
- `Selector` and `AvailabilityEnum` are described in reference, in short they allow to specify which nodes must be selected
|
|
to execute the reques (i.e. "execute only on an archive node")
|
|
|
|
.NativeCallReplyItem
|
|
[source,proto]
|
|
----
|
|
|
|
message NativeCallReplyItem {
|
|
uint32 id = 1;
|
|
bool succeed = 2;
|
|
bytes payload = 3;
|
|
bytes error = 4;
|
|
NativeCallReplySignature signature = 5;
|
|
}
|
|
|
|
message NativeCallReplySignature {
|
|
uint64 nonce = 1;
|
|
bytes signature = 2;
|
|
uint64 key_id = 3;
|
|
string upstream_id = 4;
|
|
}
|
|
----
|
|
|
|
Where:
|
|
|
|
- `payload` is JSON response for a particular call (`result` field), encoded into a string (`succeed` is true)
|
|
- or `error` if request failed (`succeed` is false)
|
|
|
|
NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response.
|
|
It allows building non-blocking queries
|
|
|
|
[#signatures]
|
|
=== Signed JSON RPC Responses
|
|
|
|
Dshackle can sign the responses it received from an upstream.
|
|
It can be enabled on server by configuring a path to a Secp256K1 Key used for signing.
|
|
And passing a `nonce` as part of the call request.
|
|
When both of them are set Dshackle adds a Signature to the response, which can travel through multiple levels of Dshackle-Dshackle connections.
|
|
|
|
WARNING: Caching is disabled for signed requests, and you always touch an actual node even if there is a ready to use data in local cache.
|
|
|
|
The signed message looks like:
|
|
----
|
|
DSHACKLESIG/$nonce/$upstreamId/hex(sha256($response))
|
|
----
|
|
|
|
.Where
|
|
- `nonce` is a 64-bit number provided with the request
|
|
- `upstreamId` id of an upstream which produced the result
|
|
- `response` is a part of the original JSON RPC message, i.e. it's what you have in `payload` field of `NativeCallReplyItem`
|
|
|
|
.A signed response includes:
|
|
- `nonce` original nonce used for the call
|
|
- `signature` signature bytes (of the message above)
|
|
- `key_id` identifier of a key used for the signing
|
|
- `upstream_id` id of upstream which produce the response
|
|
|
|
==== How to generate a key
|
|
|
|
Here we generate a pair of Secret and Public keys using openssl.
|
|
|
|
[source, bash]
|
|
----
|
|
export KEYNAME=mykey
|
|
|
|
openssl ecparam -name secp256k1 -out $KEYNAME_param.pem
|
|
openssl ecparam -in $KEYNAME_param.pem -genkey -noout -out $KEYNAME.pem
|
|
|
|
openssl ec -in $KEYNAME.pem -text
|
|
openssl ec -in $KEYNAME.pem -out ${KEYNAME}_pub.pem -pubout
|
|
|
|
rm $KEYNAME_param.pem
|
|
cat ${KEYNAME}_pub.pem
|
|
----
|
|
|
|
As a result you get `mykey.pem` with secret key to use on server, and `mykey_pub.pem` with public key to use on client to verify signatures.
|
|
|
|
==== How to verify a signature
|
|
|
|
Here is the example how to verify the signature with command line, and it can be easily adapted for your language of choice.
|
|
|
|
.First, let's prepare all the values:
|
|
[source, bash]
|
|
----
|
|
export PUBKEY=testing/dshackle/test_key.pub
|
|
export NONCE=10
|
|
|
|
export UPSTREAM=drpc
|
|
export PAYLOAD='["0xe670ec64341771606e55d6b4ca35a1a6b75ee3d5145a99d05921026d1527331", true]'
|
|
|
|
export SIGNATURE=3045022100be1d730e0e381e25bff64f0fc598d19e31688a01db751098d0ed21847ca785b0022002f5651a0e8d447b0815aeb7b48738cb470cf46d60ee4e2f5bc9c0dc4e072dc3
|
|
----
|
|
|
|
.Then rebuild the signed message:
|
|
[source, bash]
|
|
----
|
|
echo -n "DSHACKLESIG/$NONCE/$UPSTREAM/" > msg.txt
|
|
echo -n $PAYLOAD | shasum -a 256 - | awk '{ printf $1 }' >> msg.txt
|
|
----
|
|
|
|
.And save signature as a binary file:
|
|
[source, bash]
|
|
----
|
|
rm -f msg.sig && echo $SIGNATURE | xxd -r -p - msg.sig
|
|
----
|
|
|
|
.Now you can verify the payload with the following:
|
|
[source, bash]
|
|
----
|
|
openssl dgst -sha256 -verify $PUBKEY -signature msg.sig msg.txt
|
|
----
|
|
|
|
.Which should print:
|
|
----
|
|
Verified OK
|
|
----
|
|
|
|
==== What is the Key Identifier?
|
|
|
|
Key Id is the first 64 bits of SHA-256 hash of the x509 encoded Public Key.
|
|
It's provided with the Signed Response for a reference.
|
|
|
|
You can get it with:
|
|
[source, bash]
|
|
----
|
|
cat $PUBKEY | sed -e '$ d' | awk '(NR>1)' | base64 -d | shasum -a 256 - | head -c 16
|
|
----
|
|
|
|
=== Wrapped JSON RPC subscriptions
|
|
|
|
Most of Ethereum APIs provides _subscription_ to events usually accessed through WebSocket connection.
|
|
Dshackle gives access to same events through gRPC protocol via the `NativeSubscribe` method.
|
|
|
|
NOTE: Dshackle doesn't actually wrap existing subscription or dispatch request to an upstream.
|
|
It rather generates same events based on the available data, i.e., aggregates it from multiple upstreams.
|
|
|
|
Supported subscriptions:
|
|
|
|
- `newHeads`
|
|
- `logs`
|
|
- `syncing`
|
|
|
|
Method data:
|
|
|
|
[source,proto]
|
|
----
|
|
message NativeSubscribeRequest {
|
|
ChainRef chain = 1;
|
|
string method = 2;
|
|
bytes payload = 3;
|
|
}
|
|
|
|
message NativeSubscribeReplyItem {
|
|
bytes payload = 1;
|
|
}
|
|
----
|
|
|
|
Where:
|
|
|
|
- `method` is a subscriptions method (one of `newHeads`, `logs` or `syncing`)
|
|
- `payload` in request is optional subscription params object, which exists only for `logs` methods.
|
|
In that case it may be `address` or `topics`.
|
|
Both address and topics can be a string or array of strings.
|
|
Empty payload for `logs` accepted as subscription to _all_ events.
|
|
- `payload` in reply item is as subscription response encoded as JSON
|
|
|
|
For example to subscribe to USDC ERC-20 coin Approval events on Ethereum mainnet the request would be:
|
|
|
|
- `chain=100`
|
|
- `method=logs`
|
|
- `payload={"address": "0xA0b86991c6218b36c1d19D4a2e9Eb0cE3606eB48", "topics": ["0x8c5be1e5ebec7d5bd14f71427d1e84f3dd0314c0f7b2291e5b200ac8c7c3b925"]}`
|
|
|
|
=== SubscribeHead
|
|
|
|
This methods provides subscription to the new blocks on the specified chain.
|
|
Returns stream of blocks right after it was accepted (and verified by Dshackle) by any of the upstreams.
|
|
|
|
.ChainHead
|
|
[source,proto]
|
|
----
|
|
message ChainHead {
|
|
ChainRef chain = 1;
|
|
uint64 height = 2;
|
|
string block_id = 3;
|
|
uint64 timestamp = 4;
|
|
bytes weight = 5;
|
|
uint64 reorg = 6;
|
|
}
|
|
----
|
|
|
|
Where:
|
|
|
|
- `chain` - chain id
|
|
- `height` - block number
|
|
- `block_id` - block hash, as a string (please note that it doesn't have `0x` prefix)
|
|
- `timestamp` - timestamp of that block
|
|
- `weight` - total network difficulty on that block, as raw bytes
|
|
- `reorg` - number of reorganized blocks, if reorg happened
|
|
|