Merge pull request #5 from emeraldpay/feat/bitcoin

Bitcoin support
This commit is contained in:
Igor Artamonov
2020-04-27 21:03:39 -04:00
committed by GitHub
77 changed files with 9098 additions and 398 deletions

View File

@@ -9,10 +9,11 @@ image:https://img.shields.io/docker/pulls/emeraldpay/dshackle?style=flat-square[
image:https://img.shields.io/github/license/emeraldpay/dshackle.svg?style=flat-square&maxAge=2592000["License",link="https://github.com/emeraldpay/dshackle/blob/master/LICENSE"] image:https://img.shields.io/github/license/emeraldpay/dshackle.svg?style=flat-square&maxAge=2592000["License",link="https://github.com/emeraldpay/dshackle/blob/master/LICENSE"]
image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge] image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge]
_Dshackle is an L7 Blockchain API Load Balancer._ It provided a high level aggregated API on top of several underlying upstreams, i.e., blockchain nodes or providers, such as Geth, Parity, Infura, etc. _Dshackle is an L7 Blockchain API Load Balancer._ It provides a high level aggregated API on top of several underlying upstreams.
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or sent data successfully broadcasted to the network. I.e. on top of blockchain nodes such as Bitcoind, Geth, Parity, or providers like Infura, and so on.
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or data successfully broadcasted to the network.
- Standard Ethereum JSON RPC API, plus advanced gRPC-based API - Standard Bitcoin and Ethereum JSON RPC API, plus enhanced gRPC-based API
- **Secure** TLS with optional client authentication - **Secure** TLS with optional client authentication
- Blockchain-aware **caching** in memory and in Redis - Blockchain-aware **caching** in memory and in Redis
- Routing based on **data availability** (peers, height, sync status) - Routing based on **data availability** (peers, height, sync status)
@@ -22,6 +23,13 @@ It automatically verifies their availability and the current status of the netwo
Dshackle allows to build a mesh network of interconnected Dshackle servers for building blockchain based services that needs to have fast, secure, stable and fail-proof access to blockchain APIs. Dshackle allows to build a mesh network of interconnected Dshackle servers for building blockchain based services that needs to have fast, secure, stable and fail-proof access to blockchain APIs.
Blockchain support:
- Ethereum, Ethereum Classic and Kovan testnet
- Bitcoin and Bitcoin Testnet
WARNING: The project is still under development, please use with caution.
Dshackle connects to several upstreams via JSON RPC, Websockets, or gRPC protocol. Dshackle connects to several upstreams via JSON RPC, Websockets, or gRPC protocol.
The server verifies if a node ("upstream") is fully synchronized (not in initial sync mode), has enough peers, and its height is not behind other nodes. The server verifies if a node ("upstream") is fully synchronized (not in initial sync mode), has enough peers, and its height is not behind other nodes.
If upstream lags behind others, lost peers below required, started to resync or went down, then Dshackle temporarily excludes it from requests and returns it when the upstream problem is fixed. If upstream lags behind others, lost peers below required, started to resync or went down, then Dshackle temporarily excludes it from requests and returns it when the upstream problem is fixed.
@@ -30,7 +38,6 @@ image::dshackle-intro.png[alt="",width=80%,align="center"]
== Roadmap == Roadmap
- [ ] Support Bitcoin RPC
- [ ] External logging - [ ] External logging
- [ ] Access to ERC-20 tokens on asset level - [ ] Access to ERC-20 tokens on asset level
- [ ] Subscription to bitcoind notification over gRPC (instead of ZeroMQ) - [ ] Subscription to bitcoind notification over gRPC (instead of ZeroMQ)
@@ -58,8 +65,8 @@ proxy:
routes: routes:
- id: eth - id: eth
blockchain: ethereum blockchain: ethereum
- id: kovan - id: btc
blockchain: kovan blockchain: bitcoin
cluster: cluster:
upstreams: upstreams:
@@ -71,12 +78,15 @@ cluster:
url: "https://mainnet.infura.io/v3/${INFURA_USER}" url: "https://mainnet.infura.io/v3/${INFURA_USER}"
ws: ws:
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}" url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
- id: infura-kovan - id: bitcoin-main
chain: kovan chain: bitcoin
connection: connection:
ethereum: bitcoin:
rpc: rpc:
url: "https://kovan.infura.io/v3/${INFURA_USER}" url: "http://localhost:8332"
basic-auth:
username: bitcoin
password: mypassword
---- ----
Which sets the following: Which sets the following:
@@ -85,11 +95,11 @@ Which sets the following:
** TLS security is disabled (_please don't use in production!_) ** TLS security is disabled (_please don't use in production!_)
- JSON RPC access through 0.0.0.0:8545 - JSON RPC access through 0.0.0.0:8545
** proxy requests to Ethereum and Kovan upstreams ** proxy requests to Ethereum and Kovan upstreams
** request path for Ethereum Mainnet is `/eth`, for Kovan is `/kovan` ** request path for Ethereum Mainnet is `/eth`, for bitcoin is `/btc`
** i.e. call Mainnet by `POST http://127.0.0.0:8545/eth` with JSON RPC payload ** i.e. call Ethereum Mainnet by `POST http://127.0.0.0:8545/eth` with JSON RPC payload
- two upstreams, one for Ethereum Mainnet and another for Kovan Testnet (both upstreams are configured to use Infura endpoint) - two upstreams, one for Ethereum Mainnet and another for Kovan Testnet (both upstreams are configured to use Infura endpoint)
- for Ethereum Mainnet it connects using JSON RPC and Websockets connections, for Kovan only JSON RPC is used - for Ethereum Mainnet it connects using JSON RPC and Websockets connections,
- Infura authentication config is omitted for this demo - for Bitcoin Mainet only JSON RPC is used
- `${INFURA_USER}` will be provided through environment variable - `${INFURA_USER}` will be provided through environment variable
@@ -137,15 +147,20 @@ curl --request POST \
==== Access using gRPC ==== Access using gRPC
NOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.
Dshackle provides a custom gRPC based API, which provides additional methods and other features such as streaming responses. Dshackle provides a custom gRPC based API, which provides additional methods and other features such as streaming responses.
Please refer to the documentation: link:docs/06-methods.adoc[gRPC Methods] Please refer to the documentation: link:docs/06-methods.adoc[gRPC Methods]
.Connect and listen for new blocks on Ethereum Mainnet .Connect and listen for new blocks on Ethereum Mainnet
[source,bash] [source,bash]
---- ----
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 io.emeraldpay.api.Blockchain/SubscribeHead grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead
---- ----
`type: 100` specifies the blockchain id, and 100 means Ethereum Mainnet. `1` is for Bitcoin Mainnet.
There we use Ethereum because it creates new blocks every 14 seconds, which works better for demo purposes, but the same request applied to Bitcoin as well.
.Output would be like .Output would be like
---- ----
{ {
@@ -166,8 +181,42 @@ grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plai
... ...
---- ----
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet. The output above is for a _streaming subscription_ to all new blocks on the Ethereum Mainnet.
It's one of services provided by Dshackle, in additional to standard methods provided by RPC JSON of underlying nodes.
It's one of the services provided by Dshackle, in addition to standard methods provided by RPC JSON of underlying nodes.
.You can also subscribe to balances changes of the balance on an address:
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d '{"asset": {"chain": "100", "code": "ether"}, "address": {"address_single": {"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}}}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeBalance
----
.and see how balance of the contract responsible for Wrapped Ether is changing:
----
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410941696896999943701015"
}
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410930748488073834320430"
}
...
----
See other enhanced methods in the link:docs/06-methods.adoc[Documentation for Enhanced Methods]
== Documentation == Documentation
@@ -214,6 +263,8 @@ See more in the documentation for link:docs/10-client-libraries.adoc[Client Libr
== Development == Development
WARNING: The code in `master` branch is considered a development version, which may lack proper testing and should not be used in production.
=== Setting up environment === Setting up environment
Dshackle is JVM based project written in Kotlin. Dshackle is JVM based project written in Kotlin.

View File

@@ -84,6 +84,7 @@ dependencies {
compile "io.infinitape:etherjar-rpc-ws:$etherjarVersion" compile "io.infinitape:etherjar-rpc-ws:$etherjarVersion"
compile "io.infinitape:etherjar-rpc-emerald:$etherjarVersion" compile "io.infinitape:etherjar-rpc-emerald:$etherjarVersion"
compile "io.infinitape:etherjar-tx:$etherjarVersion" compile "io.infinitape:etherjar-tx:$etherjarVersion"
compile 'org.bitcoinj:bitcoinj-core:0.15.8'
compile 'org.apache.httpcomponents:httpmime:4.5.8' compile 'org.apache.httpcomponents:httpmime:4.5.8'
compile 'org.apache.httpcomponents:httpclient:4.5.8' compile 'org.apache.httpcomponents:httpclient:4.5.8'
@@ -106,11 +107,12 @@ dependencies {
testCompile "org.codehaus.groovy:groovy:$groovyVersion" testCompile "org.codehaus.groovy:groovy:$groovyVersion"
testCompile 'cglib:cglib-nodep:3.2.12' testCompile 'cglib:cglib-nodep:3.3.0'
testCompile "org.spockframework:spock-core:$spockVersion" testCompile "org.spockframework:spock-core:$spockVersion"
testCompile "io.grpc:grpc-testing:${grpcVersion}" testCompile "io.grpc:grpc-testing:${grpcVersion}"
testCompile "io.projectreactor:reactor-test:$reactorVersion" testCompile "io.projectreactor:reactor-test:$reactorVersion"
testCompile 'org.objenesis:objenesis:3.0.1' testCompile 'org.objenesis:objenesis:3.1'
testCompile 'org.mock-server:mockserver-netty:5.10'
} }
compileKotlin { compileKotlin {

View File

@@ -3,14 +3,13 @@
Dshackle is an L7 Blockchain API Load Balancer with automatic discovery and health checking, authentication, and TLS termination. Dshackle is an L7 Blockchain API Load Balancer with automatic discovery and health checking, authentication, and TLS termination.
It is designed to work as an edge proxy, middle proxy, or API gateway. It is designed to work as an edge proxy, middle proxy, or API gateway.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers, such Dshackle provides a high level aggregated API on top of several underlying upstreams.
as Geth, Parity, Infura, etc), it automatically verifies their availability and the current status of the network, executes I.e. to blockchain nodes such as Bitcoind, Geth, Parity, or providers like Infura, and so on.
commands making sure that the response is consistent and/or data successfully broadcasted to the network. It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or data successfully broadcasted to the network.
Example use cases: Example use cases:
- Read a _transaction_ - Dshackle tries to find it on different nodes and retries until it's found or there is a - Read a _transaction_ - Dshackle tries to find it on different nodes and retries until it's found or there is a consistent answer from upstreams that the transaction doesn't exist
consistent answer from upstreams that the transaction doesn't exist
- Get _nonce_ to send a transaction - Dshackle makes sure it finds the highest value across several nodes - Get _nonce_ to send a transaction - Dshackle makes sure it finds the highest value across several nodes
- Send _transaction_ - Dshackle distributes it to several nodes in parallel - Send _transaction_ - Dshackle distributes it to several nodes in parallel

View File

@@ -36,15 +36,18 @@ cluster:
connection: connection:
ethereum: ethereum:
rpc: # <6> rpc: # <6>
url: "https://mainnet.infura.io/v3/${INFURA_USER}" url: "https://mainnet.infura.io/v3/${INFURA_USER}" <7>
ws: # <7> ws: # <8>
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}" url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
- id: infura-kovan - id: bitcoin-main
chain: kovan # <8> chain: bitcoin # <9>
connection: connection:
ethereum: bitcoin:
rpc: rpc:
url: "https://kovan.infura.io/v3/${INFURA_USER}" # <9> url: "http://localhost:8332"
basic-auth: # <10>
username: bitcoin
password: test
---- ----
<1> application listen for gRPC connections on 0.0.0.0:2449 <1> application listen for gRPC connections on 0.0.0.0:2449
<2> with TLS security for gRPC disabled (_never use in production!_) <2> with TLS security for gRPC disabled (_never use in production!_)
@@ -52,12 +55,12 @@ cluster:
<4> sets up 2 upstreams <4> sets up 2 upstreams
<5> one for Ethereum Mainnet, using <5> one for Ethereum Mainnet, using
<6> HTTP and <6> HTTP and
<7> Websocket conection <7> `${INFURA_USER}` value is provided through environment variable
<8> and another for Kovan Testnet <8> Websocket conection
<9> `${INFURA_USER}` value is provided through environment variable <9> and another for Bitcoin <10> with Basic Auth for bitcoin node connection
Both upstreams are configured for Infura for demo purposes, but you can use other compatible endpoints.
Infura authentication is omitted for this demo See detailed link:reference-configuration.adoc[Configuration Reference]
==== Run as docker ==== Run as docker
@@ -99,9 +102,11 @@ curl --request POST \
.Connect and listen for new blocks on Ethereum Mainnet .Connect and listen for new blocks on Ethereum Mainnet
[source,bash] [source,bash]
---- ----
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 io.emeraldpay.api.Blockchain/SubscribeHead grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead
---- ----
`type: 100` specifies the blockchain id, and 100 means Ethereum Mainnet.
.Output would be like .Output would be like
---- ----
{ {
@@ -123,4 +128,37 @@ grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plai
---- ----
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet. The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet.
It's a method provided by Dshackle, available in additional to methods provided by RPC JSON of underlying nodes. It's one of the services provided by Dshackle, in addition to standard methods provided by RPC JSON of underlying nodes.
.You can also subscribe to balances changes of the balance on an address:
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d '{"asset": {"chain": "100", "code": "ether"}, "address": {"address_single": {"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}}}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeBalance
----
.and see how balance of the contract responsible for Wrapped Ether is changing:
----
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410941696896999943701015"
}
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410930748488073834320430"
}
...
----
See other enhanced methods in the link:06-methods.adoc[Documentation for Enhanced Methods]

View File

@@ -10,7 +10,22 @@ Supported upstream protocols:
Those protocols can be configures with additional security, TLS and authentication. Those protocols can be configures with additional security, TLS and authentication.
=== Example === Notes on upstream configuration
==== Ethereum
- Most of Ethereum nodes support WebSocket connection, in addition to the JSON RPC.
If it's available on your node, it's suggested to configure both JSON RPC and WebSocket connection
==== Bitcoin
- Bitcoind needs to be configured to index/track addresses that you're going to request.
Make sure you configure it to index your addresses with `importaddress`.
If you request balance for an address that is not indexed then it returns 0 balance.
- To track all transactions you need to setup index for transactions, which is disabled by default.
Run it with `-reindex` option, or set `txindex=1` in the config.
=== Example Configuration
.upstreams.yaml .upstreams.yaml
[source,yaml] [source,yaml]
@@ -104,9 +119,28 @@ Dshackle currently supports
- `ws` websocket connection (supposed to be used in addition to `rpc` connection) - `ws` websocket connection (supposed to be used in addition to `rpc` connection)
- `grpc` connects to another Dshackle instance - `grpc` connects to another Dshackle instance
=== Methods === Bitcoin Methods
.By default an ethereum upstream supports following JSON RPC methods: .By default an ethereum upstream allows call to the following JSON RPC methods:
- `getbestblockhash`
- `getblock`
- `getblocknumber`
- `getblockcount`
- `gettransaction`
- `getrawtransaction`
- `gettxout`
- `getreceivedbyaddress`
- `listunspent`
- `sendrawtransaction`
.Plus following methods are answered directly by Dshackle
- `getmemorypool`
- `getconnectioncount`
- `getnetworkinfo`
=== Ethereum Methods
.By default an ethereum upstream allows calls to the following JSON RPC methods:
- `eth_gasPrice` - `eth_gasPrice`
- `eth_call` - `eth_call`
- `eth_estimateGas` - `eth_estimateGas`

View File

@@ -1,9 +1,12 @@
== Methods == Enhanced Methods
IMPORTANT: Dshackle provides an unified API based on gRPC and Protobuf. IMPORTANT: Dshackle provides an enhanced API based on gRPC and Protobuf, in addition to JSON RPC proxy
Dshackle provides an unified API based on gRPC and Protobuf. I.e. it's not standard JSON RPC but client libraries could Dshackle provides an enhanced and unified API based on HTTP2, gRPC and Protobuf.
be generated for all major libraries, and there're official libraries for Java and Javascript. It works in addition to JSON RPC proxy and can be used separately.
Clients can be generated for all major programming languages, and there're official libraries for Java and Javascript.
NOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.
=== gRPC definition === gRPC definition
@@ -25,8 +28,7 @@ service Blockchain {
=== Wrapped JSON RPC methods === Wrapped JSON RPC methods
Standard JSON RPC methods provided by Ethereum nodes are available wrapped into gRPC/Protobuf with additional features. To call standard JSON RPC methods provided by Ethereum/Bitcoin you use `NativeCall` wrapping method, which provides additional flexibility and configuration for the calls.
To call standard methods you use `NativeCall` method.
.NativeCallRequest .NativeCallRequest
[source,proto] [source,proto]
@@ -72,13 +74,13 @@ Where:
- `payload` is JSON response for a particular call, encoded into a string (when `succeed` is true) - `payload` is JSON response for a particular call, encoded into a string (when `succeed` is true)
- or `error` if request failed (`succeed` is false) - or `error` if request failed (`succeed` is false)
NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response. It allows to build NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response.
non-blocking queries It allows to build non-blocking queries
=== SubscribeHead === SubscribeHead
Subscribes to new blocks on the specified chain. Returns stream of blocks right after it was accepted (and verified by This methods provides subscription to the new blocks on the specified chain.
Dshackle) by any of the upstreams. Returns stream of blocks right after it was accepted (and verified by Dshackle) by any of the upstreams.
.ChainHead .ChainHead
[source,proto] [source,proto]
@@ -104,7 +106,7 @@ Where:
=== SubscribeBalance or GetBalance === SubscribeBalance or GetBalance
Subscribes to changes of the balance for a single address or a set of addresses Subscribes to changes (`SubscribeBalance`) or get current (`GetBalance`) balance for a single address or a set of addresses.
.Request .Request
[source,proto] [source,proto]
@@ -136,8 +138,9 @@ message AddressBalance {
==== SubscribeTxStatus ==== SubscribeTxStatus
Subscribes to transaction confirmations. Allows to send a transactions and then listen to all changes until it Subscribes to transaction confirmations.
gets enough confirmations. Changes is `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED` Allows to send a transactions and then listen to all changes until it gets enough confirmations.
Changes are `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED`
.Request .Request
[source,proto] [source,proto]
@@ -161,3 +164,6 @@ message TxStatus {
} }
---- ----
=== gRPC Client Libraries
See link:10-client-libraries.adoc[Client Libraries] documentation.

View File

@@ -89,6 +89,15 @@ cluster:
basic-auth: basic-auth:
username: 4fc258fe41a68149c199ad8f281f2015 username: 4fc258fe41a68149c199ad8f281f2015
password: 1a68f20154fc258fe4149c199ad8f281 password: 1a68f20154fc258fe4149c199ad8f281
- id: bitcoin
chain: bitcoin
connection:
bitcoin:
rpc:
url: "http://localhost:8332"
basic-auth:
username: bitcoin
password: e984af45bb888428207c290
- id: remote - id: remote
connection: connection:
grpc: grpc:
@@ -357,7 +366,7 @@ configuration, and may be omitted for most of the situations.
| `chain` | `chain`
| yes | yes
| Blockchain which is the provided by the upstream. `ethereum`, `ethereum_classic`, `kovan` | Blockchain which is the provided by the upstream. `bitcoin`, `bitcoin-testnet`, `ethereum`, `ethereum-classic`, or `kovan`
| `labels` | `labels`
| no | no
@@ -369,7 +378,11 @@ configuration, and may be omitted for most of the situations.
| `connection.ethereum` | `connection.ethereum`
| yes | yes
| Connection configuration for Ethereum JSON RPC API | Connection configuration for Ethereum API
| `connection.bitcoin`
| yes
| Connection configuration for Bitcoin API
|=== |===

View File

@@ -18,5 +18,5 @@ reactorVersion=3.3.3.RELEASE
etherjarVersion=0.9.1 etherjarVersion=0.9.1
# Testing # Testing
spockVersion=1.2-groovy-2.5 spockVersion=1.3-groovy-2.5

View File

@@ -96,6 +96,10 @@ class UpstreamsConfig {
var ws: WsEndpoint? = null var ws: WsEndpoint? = null
} }
class BitcoinConnection : UpstreamConnection() {
var rpc: HttpEndpoint? = null
}
class HttpEndpoint(val url: URI) { class HttpEndpoint(val url: URI) {
var basicAuth: AuthConfig.ClientBasicAuth? = null var basicAuth: AuthConfig.ClientBasicAuth? = null
var tls: AuthConfig.ClientTlsAuth? = null var tls: AuthConfig.ClientTlsAuth? = null
@@ -124,6 +128,7 @@ class UpstreamsConfig {
enum class UpstreamType private constructor(vararg code: String) { enum class UpstreamType private constructor(vararg code: String) {
ETHEREUM_JSON_RPC("ethereum"), ETHEREUM_JSON_RPC("ethereum"),
BITCOIN_JSON_RPC("bitcoin"),
DSHACKLE("dshackle", "grpc"), DSHACKLE("dshackle", "grpc"),
UNKNOWN("unknown"); UNKNOWN("unknown");

View File

@@ -87,7 +87,7 @@ class UpstreamsConfigReader(
val connConfigNode = getMapping(connNode, "ethereum")!! val connConfigNode = getMapping(connNode, "ethereum")!!
val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>() val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>()
readUpstreamCommon(upNode, upstream) readUpstreamCommon(upNode, upstream)
readUpstreamEthereum(upNode, upstream) readUpstreamStandard(upNode, upstream)
if (isValid(upstream)) { if (isValid(upstream)) {
config.upstreams.add(upstream) config.upstreams.add(upstream)
val connection = UpstreamsConfig.EthereumConnection() val connection = UpstreamsConfig.EthereumConnection()
@@ -113,6 +113,26 @@ class UpstreamsConfigReader(
} else { } else {
log.error("Upstream at #0 has invalid configuration") log.error("Upstream at #0 has invalid configuration")
} }
} else if (hasAny(connNode, "bitcoin")) {
val connConfigNode = getMapping(connNode, "bitcoin")!!
val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.BitcoinConnection>()
readUpstreamCommon(upNode, upstream)
readUpstreamStandard(upNode, upstream)
if (isValid(upstream)) {
config.upstreams.add(upstream)
val connection = UpstreamsConfig.BitcoinConnection()
upstream.connection = connection
getMapping(connConfigNode, "rpc")?.let { node ->
getValueAsString(node, "url")?.let { url ->
val http = UpstreamsConfig.HttpEndpoint(URI(url))
connection.rpc = http
http.basicAuth = authConfigReader.readClientBasicAuth(node)
http.tls = authConfigReader.readClientTls(node)
}
}
} else {
log.error("Upstream at #0 has invalid configuration")
}
} else if (hasAny(connNode, "grpc")) { } else if (hasAny(connNode, "grpc")) {
val connConfigNode = getMapping(connNode, "grpc")!! val connConfigNode = getMapping(connNode, "grpc")!!
val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.GrpcConnection>() val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.GrpcConnection>()
@@ -162,13 +182,13 @@ class UpstreamsConfigReader(
} }
} }
internal fun readUpstreamEthereum(upNode: MappingNode, upstream: UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>) { internal fun readUpstreamStandard(upNode: MappingNode, upstream: UpstreamsConfig.Upstream<*>) {
upstream.chain = getValueAsString(upNode, "chain") upstream.chain = getValueAsString(upNode, "chain")
if (hasAny(upNode, "labels")) { if (hasAny(upNode, "labels")) {
getMapping(upNode, "labels")?.let { labels -> getMapping(upNode, "labels")?.let { labels ->
labels.value.stream() labels.value.stream()
.filter { n -> n.keyNode is ScalarNode && n.valueNode is ScalarNode } .filter { n -> n.keyNode is ScalarNode && n.valueNode is ScalarNode }
.map { n -> Tuples.of((n.keyNode as ScalarNode).value, (n.valueNode as ScalarNode).value)} .map { n -> Tuples.of((n.keyNode as ScalarNode).value, (n.valueNode as ScalarNode).value) }
.map { kv -> Tuples.of(kv.t1.trim(), kv.t2.trim()) } .map { kv -> Tuples.of(kv.t1.trim(), kv.t2.trim()) }
.filter { kv -> StringUtils.isNotEmpty(kv.t1) && StringUtils.isNotEmpty(kv.t2) } .filter { kv -> StringUtils.isNotEmpty(kv.t1) && StringUtils.isNotEmpty(kv.t2) }
.forEach { kv -> .forEach { kv ->

View File

@@ -15,7 +15,9 @@
*/ */
package io.emeraldpay.dshackle.data package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.BlockHash
import io.infinitape.etherjar.rpc.json.BlockJson import io.infinitape.etherjar.rpc.json.BlockJson
import org.bouncycastle.util.encoders.Hex
class BlockId( class BlockId(
value: ByteArray value: ByteArray
@@ -23,7 +25,7 @@ class BlockId(
companion object { companion object {
@JvmStatic @JvmStatic
fun from(hash: io.infinitape.etherjar.domain.BlockHash): BlockId { fun from(hash: BlockHash): BlockId {
return BlockId(hash.bytes) return BlockId(hash.bytes)
} }
@@ -34,7 +36,13 @@ class BlockId(
@JvmStatic @JvmStatic
fun from(id: String): BlockId { fun from(id: String): BlockId {
return from(io.infinitape.etherjar.domain.BlockHash.from(id)) val clean = if (id.startsWith("0x")) {
id.substring(2)
} else {
id
}
val bytes = Hex.decode(clean)
return BlockId(bytes)
} }
} }

View File

@@ -28,11 +28,9 @@ open class HashId(
} }
fun toHex(): String { fun toHex(): String {
val hex = CharArray(value.size * 2 + 2) val hex = CharArray(value.size * 2)
hex[0] = '0'
hex[1] = 'x'
var i = 0 var i = 0
var j = 2 var j = 0
while (i < value.size) { while (i < value.size) {
hex[j++] = HEX_DIGITS[0xF0 and value[i].toInt() ushr 4] hex[j++] = HEX_DIGITS[0xF0 and value[i].toInt() ushr 4]
hex[j++] = HEX_DIGITS[0x0F and value[i].toInt()] hex[j++] = HEX_DIGITS[0x0F and value[i].toInt()]

View File

@@ -17,6 +17,8 @@ package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.TransactionId import io.infinitape.etherjar.domain.TransactionId
import io.infinitape.etherjar.rpc.json.TransactionJson import io.infinitape.etherjar.rpc.json.TransactionJson
import org.bouncycastle.util.encoders.Hex
import java.math.BigInteger
class TxId( class TxId(
value: ByteArray value: ByteArray
@@ -35,7 +37,13 @@ class TxId(
@JvmStatic @JvmStatic
fun from(id: String): TxId { fun from(id: String): TxId {
return from(TransactionId.from(id)) val clean = if (id.startsWith("0x")) {
id.substring(2)
} else {
id
}
val bytes = Hex.decode(clean)
return TxId(bytes)
} }
} }
} }

View File

@@ -19,6 +19,7 @@ import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common import io.emeraldpay.api.proto.Common
import io.emeraldpay.api.proto.ReactorBlockchainGrpc import io.emeraldpay.api.proto.ReactorBlockchainGrpc
import io.emeraldpay.dshackle.BlockchainType import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired import org.springframework.beans.factory.annotation.Autowired
@@ -30,8 +31,8 @@ import reactor.core.publisher.Mono
class BlockchainRpc( class BlockchainRpc(
@Autowired private val nativeCall: NativeCall, @Autowired private val nativeCall: NativeCall,
@Autowired private val streamHead: StreamHead, @Autowired private val streamHead: StreamHead,
@Autowired private val trackEthereumTx: TrackEthereumTx, @Autowired private val trackTx: List<TrackTx>,
@Autowired private val trackEthereumAddress: TrackEthereumAddress, @Autowired private val trackAddress: List<TrackAddress>,
@Autowired private val describe: Describe, @Autowired private val describe: Describe,
@Autowired private val subscribeStatus: SubscribeStatus @Autowired private val subscribeStatus: SubscribeStatus
): ReactorBlockchainGrpc.BlockchainImplBase() { ): ReactorBlockchainGrpc.BlockchainImplBase() {
@@ -47,15 +48,27 @@ class BlockchainRpc(
} }
override fun subscribeTxStatus(request: Mono<BlockchainOuterClass.TxStatusRequest>): Flux<BlockchainOuterClass.TxStatus> { override fun subscribeTxStatus(request: Mono<BlockchainOuterClass.TxStatusRequest>): Flux<BlockchainOuterClass.TxStatus> {
return trackEthereumTx.add(request) return request.flatMapMany { request ->
val chain = Chain.byId(request.chainValue)
trackTx.find { it.isSupported(chain) }?.subscribe(request)
?: Flux.error(SilentException.UnsupportedBlockchain(chain))
}
} }
override fun subscribeBalance(request: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> { override fun subscribeBalance(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return trackEthereumAddress.subscribe(request) return requestMono.flatMapMany { request ->
val chain = Chain.byId(request.asset.chainValue)
trackAddress.find { it.isSupported(chain) }?.subscribe(request)
?: Flux.error(SilentException.UnsupportedBlockchain(chain))
}
} }
override fun getBalance(request: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> { override fun getBalance(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return trackEthereumAddress.getBalance(request) return requestMono.flatMapMany { request ->
val chain = Chain.byId(request.asset.chainValue)
trackAddress.find { it.isSupported(chain) }?.getBalance(request)
?: Flux.error(SilentException.UnsupportedBlockchain(chain))
}
} }
override fun describe(request: Mono<BlockchainOuterClass.DescribeRequest>): Mono<BlockchainOuterClass.DescribeResponse> { override fun describe(request: Mono<BlockchainOuterClass.DescribeRequest>): Mono<BlockchainOuterClass.DescribeResponse> {

View File

@@ -19,6 +19,7 @@ import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.startup.QuorumForLabels import io.emeraldpay.dshackle.startup.QuorumForLabels
import io.emeraldpay.dshackle.upstream.* import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.dshackle.upstream.grpc.EthereumGrpcUpstream import io.emeraldpay.dshackle.upstream.grpc.EthereumGrpcUpstream
import org.springframework.beans.factory.annotation.Autowired import org.springframework.beans.factory.annotation.Autowired
@@ -47,6 +48,8 @@ class Describe(
val nodes = QuorumForLabels() val nodes = QuorumForLabels()
if (up is EthereumUpstream) { if (up is EthereumUpstream) {
nodes.add(up.node) nodes.add(up.node)
} else if (up is BitcoinUpstream) {
nodes.add(up.node)
} else if (up is EthereumGrpcUpstream) { } else if (up is EthereumGrpcUpstream) {
nodes.add(up.getNodes()) nodes.add(up.getNodes())
} }

View File

@@ -23,11 +23,8 @@ import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.dshackle.upstream.* import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.quorum.AlwaysQuorum import io.emeraldpay.dshackle.quorum.AlwaysQuorum
import io.emeraldpay.dshackle.quorum.CallQuorum import io.emeraldpay.dshackle.quorum.CallQuorum
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.RpcException import io.infinitape.etherjar.rpc.RpcException
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.apache.commons.lang3.StringUtils import org.apache.commons.lang3.StringUtils
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired import org.springframework.beans.factory.annotation.Autowired
@@ -88,17 +85,18 @@ open class NativeCall(
if (chain == Chain.UNSPECIFIED) { if (chain == Chain.UNSPECIFIED) {
return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(request.chain.number))) return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(request.chain.number)))
} }
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
if (!upstreams.isAvailable(chain)) {
return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(request.chain.number))) return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(request.chain.number)))
} }
val upstream = upstreams.getUpstream(chain) val upstream = upstreams.getUpstream(chain)
?: return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(chain))) ?: return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(chain)))
return prepareCall(request, upstream as AggregatedUpstream<EthereumApi>) return prepareCall(request, upstream)
} }
fun prepareCall(request: BlockchainOuterClass.NativeCallRequest, upstream: AggregatedUpstream<EthereumApi>): Flux<CallContext<RawCallDetails>> { fun prepareCall(request: BlockchainOuterClass.NativeCallRequest, upstream: AggregatedUpstream<*>): Flux<CallContext<RawCallDetails>> {
return request.itemsList.toFlux().map { return request.itemsList.toFlux().map {
val method = it.method val method = it.method
val params = it.payload.toStringUtf8() val params = it.payload.toStringUtf8()
@@ -137,7 +135,7 @@ open class NativeCall(
var failures = 0 var failures = 0
return Flux.from(apis) return Flux.from(apis)
.flatMap { api -> .flatMap { api ->
val upstream = api.upstream!! val upstream = ctx.upstream
api.execute(ctx.id, ctx.payload.method, ctx.payload.params) api.execute(ctx.id, ctx.payload.method, ctx.payload.params)
// on error notify quorum, it may use error message or other details // on error notify quorum, it may use error message or other details
.doOnError { err -> .doOnError { err ->
@@ -205,7 +203,7 @@ open class NativeCall(
} }
open class CallContext<T>(val id: Int, open class CallContext<T>(val id: Int,
val upstream: AggregatedUpstream<EthereumApi>, val upstream: AggregatedUpstream<*>,
val matcher: Selector.Matcher, val matcher: Selector.Matcher,
val callQuorum: CallQuorum, val callQuorum: CallQuorum,
val payload: T) { val payload: T) {
@@ -213,7 +211,7 @@ open class NativeCall(
return CallContext(id, upstream, matcher, callQuorum, payload) return CallContext(id, upstream, matcher, callQuorum, payload)
} }
fun getApis(): ApiSource<EthereumApi> { fun getApis(): ApiSource<*> {
return upstream.getApis(matcher) return upstream.getApis(matcher)
} }
} }

View File

@@ -65,7 +65,7 @@ class StreamHead(
.setHeight(block.height) .setHeight(block.height)
.setTimestamp(block.timestamp!!.toEpochMilli()) .setTimestamp(block.timestamp!!.toEpochMilli())
.setWeight(ByteString.copyFrom(block.difficulty.toByteArray())) .setWeight(ByteString.copyFrom(block.difficulty.toByteArray()))
.setBlockId(block.hash.toHex().substring(2)) .setBlockId(block.hash.toHex())
.build() .build()
} }

View File

@@ -0,0 +1,17 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
/**
* Base interface to tracking balance on a single blockchain
*/
interface TrackAddress {
fun isSupported(chain: Chain): Boolean
fun getBalance(requestMono: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance>
fun subscribe(requestMono: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance>
}

View File

@@ -0,0 +1,154 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.dshackle.upstream.Selector
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Service
import reactor.core.publisher.Flux
import reactor.util.function.Tuples
import java.math.BigDecimal
import java.math.BigInteger
import java.util.*
import kotlin.collections.HashMap
@Service
class TrackBitcoinAddress(
@Autowired private val upstreams: Upstreams
) : TrackAddress {
companion object {
private val log = LoggerFactory.getLogger(TrackBitcoinAddress::class.java)
}
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.BITCOIN && upstreams.isAvailable(chain)
}
fun allAddresses(request: BlockchainOuterClass.BalanceRequest): List<String>? {
if (!request.hasAddress()) {
return null
}
return when {
request.address.hasAddressSingle() -> {
listOf(request.address.addressSingle.address)
}
request.address.hasAddressMulti() -> {
request.address.addressMulti.addressesList
.map { addr -> addr.address }
.sorted()
}
else -> null
}
}
fun requestBalances(chain: Chain, api: DirectBitcoinApi, addresses: List<String>): Flux<AddressBalance> {
return api.executeAndResult(0, "listunspent", emptyList(), List::class.java)
.flatMapMany { unspents ->
val result = getTotal(chain, addresses, unspents)
Flux.fromIterable(result)
}
}
override fun getBalance(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
val chain = Chain.byId(request.asset.chainValue)
val upstream = upstreams.getUpstream(chain)?.castApi(DirectBitcoinApi::class.java)
?: return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
val addresses = allAddresses(request) ?: return Flux.error(SilentException("Unsupported address"))
if (addresses.isEmpty()) {
return Flux.empty()
}
val result = upstream.getApi(Selector.empty).flatMapMany { api ->
requestBalances(chain, api, addresses)
.map(this@TrackBitcoinAddress::buildResponse)
}
return result
}
fun getTotal(chain: Chain, addresses: List<String>, unspents: List<*>): List<AddressBalance> {
return unspents.asSequence()
.filterIsInstance<Map<String, Any>>()
.filter { unspent ->
unspent.containsKey("address")
&& Collections.binarySearch(addresses, unspent["address"] as String) >= 0
&& unspent.containsKey("amount")
}
.map {
Tuples.of(it["address"] as String, it["amount"] as Number)
}
.map {
AddressBalance(chain, it.t1,
//use toString because toDecimal makes rounding
BigDecimal(it.t2.toString()).multiply(BigDecimal.TEN.pow(8)).toBigInteger()
)
}
.plus(
//add default ZERO value
addresses.map {
AddressBalance(chain, it, BigInteger.ZERO)
}
)
.groupBy {
it.address
}
.map {
it.value.reduceRight { x, acc ->
x.plus(acc)
}
}.toList()
}
override fun subscribe(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
val chain = Chain.byId(request.asset.chainValue)
val upstream = upstreams.getUpstream(chain)?.castApi(DirectBitcoinApi::class.java)
?: return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
val addresses = allAddresses(request) ?: return Flux.error(SilentException("Unsupported address"))
if (addresses.isEmpty()) {
return Flux.empty()
}
val initial = upstream.getApi(Selector.empty).flatMapMany { api ->
requestBalances(chain, api, addresses)
}
val following = upstream.getHead().getFlux()
.flatMap { block ->
upstream.getApi(Selector.empty).flatMapMany { api ->
requestBalances(chain, api, addresses)
}
}
val last = HashMap<Address, BigInteger>()
val result = Flux.merge(initial, following)
.filter { curr ->
val prev = last[curr.address]
val updated = prev == null || curr.balance != prev
last[curr.address] = curr.balance
updated
}
return result.map(this@TrackBitcoinAddress::buildResponse)
}
private fun buildResponse(address: AddressBalance): BlockchainOuterClass.AddressBalance {
return BlockchainOuterClass.AddressBalance.newBuilder()
.setBalance(address.balance.toString(10))
.setAsset(Common.Asset.newBuilder()
.setChainValue(address.address.chain.id)
.setCode("BTC"))
.setAddress(Common.SingleAddress.newBuilder().setAddress(address.address.address))
.build()
}
open class AddressBalance(val address: Address, var balance: BigInteger = BigInteger.ZERO) {
constructor(chain: Chain, address: String, balance: BigInteger) : this(Address(chain, address), balance)
fun plus(other: AddressBalance) = AddressBalance(address, balance + other.balance)
}
data class Address(val chain: Chain, val address: String)
}

View File

@@ -0,0 +1,159 @@
package io.emeraldpay.dshackle.rpc
import com.google.protobuf.ByteString
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.dshackle.upstream.Selector
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.bitcoin.ExtractBlock
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Service
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import java.math.BigInteger
import java.time.Duration
import java.time.Instant
import kotlin.math.max
import kotlin.math.min
@Service
class TrackBitcoinTx(
@Autowired private val upstreams: Upstreams
) : TrackTx {
companion object {
private val log = LoggerFactory.getLogger(TrackBitcoinTx::class.java)
}
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.BITCOIN && upstreams.isAvailable(chain)
}
override fun subscribe(request: BlockchainOuterClass.TxStatusRequest): Flux<BlockchainOuterClass.TxStatus> {
val chain = Chain.byId(request.chainValue)
val upstream = upstreams.getUpstream(chain)?.cast(BitcoinUpstream::class.java, DirectBitcoinApi::class.java)
?: return Flux.error(SilentException.UnsupportedBlockchain(chain))
val txid = request.txId
val confirmations = max(min(1, request.confirmationLimit), 12)
return upstream.getApi(Selector.empty).flatMapMany { api ->
subscribe(chain, api, upstream, txid)
}.takeUntil { tx ->
tx.confirmations >= confirmations
}.map(this::asProto)
}
fun subscribe(chain: Chain, api: DirectBitcoinApi, upstream: BitcoinUpstream, txid: String): Flux<TxStatus> {
return loadExisting(api, txid)
.flatMapMany { status ->
if (status.mined) {
//Head almost always knows the current height, so it can continue with calculating confirmations
//without publishing an empty TxStatus first
continueWithMined(api, upstream, status)
} else {
loadMempool(upstream, txid)
.flatMapMany { tx ->
val next = if (tx.found) {
untilMined(upstream, tx)
} else {
untilFound(chain, api, upstream, txid)
}
//fist provide the current status, then updates
Flux.concat(Mono.just(tx), next)
}
}
}
}
fun continueWithMined(api: DirectBitcoinApi, upstream: BitcoinUpstream, status: TxStatus): Flux<TxStatus> {
return api.getBlock(status.blockHash!!)
.map { block ->
TxStatus(status.txid, true, ExtractBlock.getHeight(block), true, status.blockHash, ExtractBlock.getTime(block), ExtractBlock.getDifficulty(block))
}.flatMapMany { tx ->
withConfirmations(upstream, tx)
}
}
fun untilFound(chain: Chain, api: DirectBitcoinApi, upstream: BitcoinUpstream, txid: String): Flux<TxStatus> {
return Flux.interval(Duration.ofSeconds(1))
.take(Duration.ofMinutes(10))
.flatMap { loadMempool(upstream, txid) }
.skipUntil { it.found }
.flatMap { subscribe(chain, api, upstream, txid) }
.doOnError { t ->
log.error("Failed to wait until found", t)
}
}
fun untilMined(upstream: BitcoinUpstream, tx: TxStatus): Mono<TxStatus> {
return upstream.getHead().getFlux().flatMap {
upstream.getApi(Selector.empty).flatMap { api ->
loadExisting(api, tx.txid)
}.filter { it.mined }
}.single()
}
fun withConfirmations(upstream: BitcoinUpstream, tx: TxStatus): Flux<TxStatus> {
return upstream.getHead().getFlux().map {
tx.withHead(it.height)
}
}
fun loadExisting(api: DirectBitcoinApi, txid: String): Mono<TxStatus> {
val mined = api.getTx(txid)
return mined.map {
val block = it["blockhash"] as String?
TxStatus(txid, found = true, mined = block != null, blockHash = block, height = ExtractBlock.getHeight(it))
}
}
fun loadMempool(upstream: BitcoinUpstream, txid: String): Mono<TxStatus> {
println("access: ${upstream.getData()}")
val mempool = upstream.getData().getMempool().get()
return mempool.map {
if (it.contains(txid)) {
TxStatus(txid, found = true, mined = false)
} else {
TxStatus(txid, found = false, mined = false)
}
}
}
private fun asProto(tx: TxStatus): BlockchainOuterClass.TxStatus {
val data = BlockchainOuterClass.TxStatus.newBuilder()
.setTxId(tx.txid)
.setConfirmations(tx.confirmations.toInt())
data.broadcasted = tx.found
val isMined = tx.mined
data.mined = isMined
if (isMined) {
data.setBlock(
Common.BlockInfo.newBuilder()
.setBlockId(tx.blockHash!!.substring(2))
.setTimestamp(tx.blockTime!!.toEpochMilli())
.setWeight(ByteString.copyFrom(tx.blockTotalDifficulty!!.toByteArray()))
.setHeight(tx.height!!)
)
}
return data.build()
}
class TxStatus(
val txid: String,
val found: Boolean = false,
val height: Long? = null,
val mined: Boolean = false,
val blockHash: String? = null,
val blockTime: Instant? = null,
val blockTotalDifficulty: BigInteger? = null,
val confirmations: Long = 0) {
fun withHead(headHeight: Long) = TxStatus(txid, found, height, mined, blockHash, blockTime, blockTotalDifficulty, headHeight - height!! + 1)
}
}

View File

@@ -51,7 +51,7 @@ import javax.annotation.PostConstruct
class TrackEthereumAddress( class TrackEthereumAddress(
@Autowired private val upstreams: Upstreams, @Autowired private val upstreams: Upstreams,
@Autowired private val upstreamScheduler: Scheduler @Autowired private val upstreamScheduler: Scheduler
) { ) : TrackAddress {
private val log = LoggerFactory.getLogger(TrackEthereumAddress::class.java) private val log = LoggerFactory.getLogger(TrackEthereumAddress::class.java)
private val clients = HashMap<Chain, ConcurrentLinkedQueue<TrackedAddress>>() private val clients = HashMap<Chain, ConcurrentLinkedQueue<TrackedAddress>>()
@@ -83,6 +83,10 @@ class TrackEthereumAddress(
} }
} }
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.ETHEREUM && upstreams.isAvailable(chain)
}
private fun startTracking(client: TrackedAddress) { private fun startTracking(client: TrackedAddress) {
clients[client.chain]?.add(client) ?: log.warn("Chain ${client.chain} is not available for tracking") clients[client.chain]?.add(client) ?: log.warn("Chain ${client.chain} is not available for tracking")
} }
@@ -99,9 +103,6 @@ class TrackEthereumAddress(
private fun initializeSimple(request: BlockchainOuterClass.BalanceRequest): Flux<SimpleAddress> { private fun initializeSimple(request: BlockchainOuterClass.BalanceRequest): Flux<SimpleAddress> {
val chain = Chain.byId(request.asset.chainValue) val chain = Chain.byId(request.asset.chainValue)
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
}
if (!upstreams.isAvailable(chain)) { if (!upstreams.isAvailable(chain)) {
return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue)) return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
} }
@@ -128,43 +129,35 @@ class TrackEthereumAddress(
} }
} }
fun subscribe(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> { override fun subscribe(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
return requestMono.flatMapMany { request -> val bus = TopicProcessor.create<BlockchainOuterClass.AddressBalance>()
val chain = Chain.byId(request.asset.chainValue) return initializeSubscription(request, bus)
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) { .flatMap { tracked ->
return@flatMapMany Flux.error<BlockchainOuterClass.AddressBalance>(SilentException.UnsupportedBlockchain(request.asset.chainValue)) val current = getBalance(tracked).map {
} tracked.withBalance(it)
val bus = TopicProcessor.create<BlockchainOuterClass.AddressBalance>() }.doOnNext {
initializeSubscription(request, bus) startTracking(it)
.flatMap { tracked -> }.map {
val current = getBalance(tracked).map { buildResponse(it)
tracked.withBalance(it)
}.doOnNext {
startTracking(it)
}.map {
buildResponse(it)
}
Flux.merge(current, bus).doFinally { stopTracking(tracked) }
} }
.doOnError { t -> Flux.merge(current, bus).doFinally { stopTracking(tracked) }
if (t is SilentException) { }
if (t is SilentException.UnsupportedBlockchain) { .doOnError { t ->
log.warn("Unsupported blockchain: ${t.blockchainId}") if (t is SilentException) {
} if (t is SilentException.UnsupportedBlockchain) {
log.debug("Failed to process subscription", t) log.warn("Unsupported blockchain: ${t.blockchainId}")
} else {
log.warn("Failed to process subscription", t)
} }
log.debug("Failed to process subscription", t)
} else {
log.warn("Failed to process subscription", t)
} }
} }
} }
fun getBalance(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> { override fun getBalance(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
return requestMono.flatMapMany { request -> return initializeSimple(request)
initializeSimple(request) .flatMap { a -> getBalance(a).map { a.withBalance(it) } }
.flatMap { a -> getBalance(a).map { a.withBalance(it) } } .map { buildResponse(it) }
.map { buildResponse(it) }
}
} }
private fun simpleAddress(address: Common.SingleAddress, chain: Chain): SimpleAddress { private fun simpleAddress(address: Common.SingleAddress, chain: Chain): SimpleAddress {

View File

@@ -58,7 +58,7 @@ import kotlin.math.min
class TrackEthereumTx( class TrackEthereumTx(
@Autowired private val upstreams: Upstreams, @Autowired private val upstreams: Upstreams,
@Autowired private val upstreamScheduler: Scheduler @Autowired private val upstreamScheduler: Scheduler
) { ) : TrackTx {
companion object { companion object {
private val ZERO_BLOCK = BlockHash.from("0x0000000000000000000000000000000000000000000000000000000000000000") private val ZERO_BLOCK = BlockHash.from("0x0000000000000000000000000000000000000000000000000000000000000000")
@@ -75,6 +75,10 @@ class TrackEthereumTx(
val notFound = ConcurrentLinkedQueue<TrackedTx>() val notFound = ConcurrentLinkedQueue<TrackedTx>()
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.ETHEREUM && upstreams.isAvailable(chain)
}
@PostConstruct @PostConstruct
fun init() { fun init() {
upstreams.observeChains().subscribe { chain -> upstreams.observeChains().subscribe { chain ->
@@ -165,7 +169,7 @@ class TrackEthereumTx(
fun streamAllUpdates(tx: TxDetails): Flux<BlockchainOuterClass.TxStatus> { fun streamAllUpdates(tx: TxDetails): Flux<BlockchainOuterClass.TxStatus> {
val current = checkForUpdate(tx) val current = checkForUpdate(tx)
.doOnNext (this::onFirstUpdate) .doOnNext(this::onFirstUpdate)
.map { Notification(it, asProto(it)) } .map { Notification(it, asProto(it)) }
val updates = Flux.from(tx.bus) val updates = Flux.from(tx.bus)
@@ -175,12 +179,9 @@ class TrackEthereumTx(
.map { it.proto } .map { it.proto }
} }
fun add(requestMono: Mono<BlockchainOuterClass.TxStatusRequest>): Flux<BlockchainOuterClass.TxStatus> { override fun subscribe(request: BlockchainOuterClass.TxStatusRequest): Flux<BlockchainOuterClass.TxStatus> {
return requestMono.map { request -> val tx = prepareTracking(request)
prepareTracking(request) return streamAllUpdates(tx)
}.flatMapMany { tx ->
streamAllUpdates(tx)
}
} }
private fun verifyAll(chain: Chain) { private fun verifyAll(chain: Chain) {

View File

@@ -0,0 +1,10 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
interface TrackTx {
fun isSupported(chain: Chain): Boolean
fun subscribe(request: BlockchainOuterClass.TxStatusRequest): Flux<BlockchainOuterClass.TxStatus>
}

View File

@@ -20,6 +20,10 @@ import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.FileResolver import io.emeraldpay.dshackle.FileResolver
import io.emeraldpay.dshackle.config.UpstreamsConfig import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.CurrentUpstreams import io.emeraldpay.dshackle.upstream.CurrentUpstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinRpcClient
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.calls.ManagedCallMethods import io.emeraldpay.dshackle.upstream.calls.ManagedCallMethods
import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
@@ -32,6 +36,7 @@ import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Repository import org.springframework.stereotype.Repository
import java.net.URI import java.net.URI
import java.util.* import java.util.*
import java.util.concurrent.atomic.AtomicInteger
import javax.annotation.PostConstruct import javax.annotation.PostConstruct
import kotlin.collections.HashMap import kotlin.collections.HashMap
@@ -44,6 +49,7 @@ open class ConfiguredUpstreams(
) { ) {
private val log = LoggerFactory.getLogger(ConfiguredUpstreams::class.java) private val log = LoggerFactory.getLogger(ConfiguredUpstreams::class.java)
private var seq = AtomicInteger(0)
private val chainNames = mapOf( private val chainNames = mapOf(
"ethereum" to Chain.ETHEREUM, "ethereum" to Chain.ETHEREUM,
@@ -51,7 +57,10 @@ open class ConfiguredUpstreams(
"eth" to Chain.ETHEREUM, "eth" to Chain.ETHEREUM,
"etc" to Chain.ETHEREUM_CLASSIC, "etc" to Chain.ETHEREUM_CLASSIC,
"morden" to Chain.TESTNET_MORDEN, "morden" to Chain.TESTNET_MORDEN,
"kovan" to Chain.TESTNET_KOVAN "kovan" to Chain.TESTNET_KOVAN,
"kovan-testnet" to Chain.TESTNET_KOVAN,
"bitcoin" to Chain.BITCOIN,
"bitcoin-testnet" to Chain.TESTNET_BITCOIN
) )
@PostConstruct @PostConstruct
@@ -69,13 +78,20 @@ open class ConfiguredUpstreams(
log.error("Chain is unknown: ${up.chain}") log.error("Chain is unknown: ${up.chain}")
return@forEach return@forEach
} }
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
log.error("Chain is unsupported: ${up.chain}")
return@forEach
}
val options = (up.options ?: UpstreamsConfig.Options()) val options = (up.options ?: UpstreamsConfig.Options())
.merge(defaultOptions[chain] ?: UpstreamsConfig.Options.getDefaults()) .merge(defaultOptions[chain] ?: UpstreamsConfig.Options.getDefaults())
buildEthereumUpstream(up.cast(UpstreamsConfig.EthereumConnection::class.java), chain, options) when (BlockchainType.fromBlockchain(chain)) {
BlockchainType.ETHEREUM -> {
buildEthereumUpstream(up.cast(UpstreamsConfig.EthereumConnection::class.java), chain, options)
}
BlockchainType.BITCOIN -> {
buildBitcoinUpstream(up.cast(UpstreamsConfig.BitcoinConnection::class.java), chain, options)
}
else -> {
log.error("Chain is unsupported: ${up.chain}")
return@forEach
}
}
} }
} }
} }
@@ -101,14 +117,8 @@ open class ConfiguredUpstreams(
return defaultOptions return defaultOptions
} }
private fun buildEthereumUpstream(config: UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>, private fun buildMethods(config: UpstreamsConfig.Upstream<*>, chain: Chain): CallMethods {
chain: Chain, return if (config.methods != null) {
options: UpstreamsConfig.Options
) {
val conn = config.connection!!
var rpcApi: DirectEthereumApi? = null
val urls = ArrayList<URI>()
val methods = if (config.methods != null) {
ManagedCallMethods(currentUpstreams.getDefaultMethods(chain), ManagedCallMethods(currentUpstreams.getDefaultMethods(chain),
config.methods!!.enabled.map { it.name }.toSet(), config.methods!!.enabled.map { it.name }.toSet(),
config.methods!!.disabled.map { it.name }.toSet() config.methods!!.disabled.map { it.name }.toSet()
@@ -116,6 +126,38 @@ open class ConfiguredUpstreams(
} else { } else {
currentUpstreams.getDefaultMethods(chain) currentUpstreams.getDefaultMethods(chain)
} }
}
private fun buildBitcoinUpstream(config: UpstreamsConfig.Upstream<UpstreamsConfig.BitcoinConnection>,
chain: Chain,
options: UpstreamsConfig.Options) {
val conn = config.connection!!
var rpcApi: DirectBitcoinApi? = null
val methods = buildMethods(config, chain)
conn.rpc?.let { endpoint ->
val rpcClient = BitcoinRpcClient(endpoint.url.toString(), endpoint.basicAuth!!)
rpcApi = DirectBitcoinApi(rpcClient, objectMapper, methods)
}
rpcApi?.let { api ->
val upstream = BitcoinUpstream(config.id
?: "bitcoin-${seq.getAndIncrement()}", chain, api,
options, QuorumForLabels.QuorumItem(1, config.labels),
objectMapper, methods)
upstream.start()
currentUpstreams.update(UpstreamChange(chain, upstream, UpstreamChange.ChangeType.ADDED))
}
}
private fun buildEthereumUpstream(config: UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>,
chain: Chain,
options: UpstreamsConfig.Options) {
val conn = config.connection!!
var rpcApi: DirectEthereumApi? = null
val urls = ArrayList<URI>()
val methods = buildMethods(config, chain)
conn.rpc?.let { endpoint -> conn.rpc?.let { endpoint ->
val rpcClient = ReactorHttpRpcClient.newBuilder() val rpcClient = ReactorHttpRpcClient.newBuilder()
.connectTo(endpoint.url) .connectTo(endpoint.url)

View File

@@ -0,0 +1,53 @@
package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.data.BlockContainer
import org.slf4j.LoggerFactory
import reactor.core.Disposable
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
import java.util.concurrent.atomic.AtomicReference
abstract class AbstractHead : Head {
companion object {
private val log = LoggerFactory.getLogger(AbstractHead::class.java)
}
private val head = AtomicReference<BlockContainer>(null)
private val stream: TopicProcessor<BlockContainer> = TopicProcessor.create()
fun follow(source: Flux<BlockContainer>): Disposable {
return source.distinctUntilChanged {
it.hash
}.filter { block ->
val curr = head.get()
curr == null || curr.difficulty < block.difficulty
}
.subscribe { block ->
val prev = head.getAndUpdate { curr ->
if (curr == null || curr.difficulty < block.difficulty) {
block
} else {
curr
}
}
if (prev == null || prev.hash != block.hash) {
log.debug("New block ${block.height} ${block.hash}")
stream.onNext(block)
}
}
}
override fun getFlux(): Flux<BlockContainer> {
return Flux.merge(
Mono.justOrEmpty(head.get()),
Flux.from(stream)
).onBackpressureLatest()
}
fun getCurrent(): BlockContainer? {
return head.get()
}
}

View File

@@ -20,7 +20,6 @@ import io.emeraldpay.dshackle.cache.*
import io.emeraldpay.dshackle.config.UpstreamsConfig import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.calls.AggregatedCallMethods import io.emeraldpay.dshackle.upstream.calls.AggregatedCallMethods
import io.emeraldpay.dshackle.upstream.calls.CallMethods import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import org.springframework.context.Lifecycle import org.springframework.context.Lifecycle
import reactor.core.Disposable import reactor.core.Disposable
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
@@ -89,7 +88,7 @@ abstract class AggregatedUpstream<U : UpstreamApi>(
cacheSubscription = null cacheSubscription = null
} }
fun onHeadUpdated(head: EthereumHead) { fun onHeadUpdated(head: Head) {
reconfigLock.withLock { reconfigLock.withLock {
cacheSubscription?.dispose() cacheSubscription?.dispose()
cacheSubscription = head.getFlux().subscribe { cacheSubscription = head.getFlux().subscribe {

View File

@@ -18,9 +18,7 @@ package io.emeraldpay.dshackle.upstream
import com.fasterxml.jackson.databind.ObjectMapper import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.cache.Caches import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.data.* import io.emeraldpay.dshackle.data.*
import io.emeraldpay.dshackle.upstream.ethereum.EmptyEthereumHead
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.infinitape.etherjar.hex.HexQuantity import io.infinitape.etherjar.hex.HexQuantity
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import reactor.core.publisher.Mono import reactor.core.publisher.Mono
@@ -30,7 +28,7 @@ import java.util.function.Function
open class CachingEthereumApi( open class CachingEthereumApi(
private val objectMapper: ObjectMapper, private val objectMapper: ObjectMapper,
private val caches: Caches, private val caches: Caches,
private val head: EthereumHead private val head: Head
): EthereumApi(objectMapper) { ): EthereumApi(objectMapper) {
companion object { companion object {
@@ -41,7 +39,7 @@ open class CachingEthereumApi(
*/ */
@JvmStatic @JvmStatic
fun empty(objectMapper: ObjectMapper): CachingEthereumApi { fun empty(objectMapper: ObjectMapper): CachingEthereumApi {
return CachingEthereumApi(objectMapper, Caches.default(objectMapper), EmptyEthereumHead()) return CachingEthereumApi(objectMapper, Caches.default(objectMapper), EmptyHead())
} }
} }

View File

@@ -17,11 +17,14 @@ package io.emeraldpay.dshackle.upstream
import com.fasterxml.jackson.databind.ObjectMapper import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.cache.Caches import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.upstream.ethereum.EthereumChainUpstreams
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle import org.springframework.context.Lifecycle
import reactor.core.Disposable import reactor.core.Disposable
import reactor.core.publisher.Mono import reactor.core.publisher.Mono
import java.lang.IllegalStateException
import java.time.Duration
/** /**
* General interface to upstream(s) to a single chain * General interface to upstream(s) to a single chain
@@ -111,5 +114,22 @@ abstract class ChainUpstreams<U : UpstreamApi>(
return 0 return 0
} }
abstract fun printStatus() fun printStatus() {
var height: Long? = null
try {
height = getHead().getFlux().next().block(Duration.ofSeconds(1))?.height
} catch (e: IllegalStateException) {
//timout
} catch (e: Exception) {
log.warn("Head processing error: ${e.javaClass} ${e.message}")
}
val statuses = upstreams.map { it.getStatus() }
.groupBy { it }
.map { "${it.key.name}/${it.value.size}" }
.joinToString(",")
val lag = upstreams.map { it.getLag() }
.joinToString(", ")
log.info("State of ${chain.chainCode}: height=${height ?: '?'}, status=$statuses, lag=[$lag]")
}
} }

View File

@@ -16,17 +16,20 @@
package io.emeraldpay.dshackle.upstream package io.emeraldpay.dshackle.upstream
import com.fasterxml.jackson.databind.ObjectMapper import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.cache.CachesEnabled import io.emeraldpay.dshackle.cache.CachesEnabled
import io.emeraldpay.dshackle.cache.CachesFactory import io.emeraldpay.dshackle.cache.CachesFactory
import io.emeraldpay.dshackle.startup.UpstreamChange import io.emeraldpay.dshackle.startup.UpstreamChange
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinChainUpstreams
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.bitcoin.DefaultBitcoinMethods
import io.emeraldpay.dshackle.upstream.calls.CallMethods import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.calls.DefaultEthereumMethods import io.emeraldpay.dshackle.upstream.calls.DefaultEthereumMethods
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumChainUpstreams import io.emeraldpay.dshackle.upstream.ethereum.EthereumChainUpstreams
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired import org.springframework.beans.factory.annotation.Autowired
import org.springframework.scheduling.annotation.Scheduled import org.springframework.scheduling.annotation.Scheduled
@@ -34,6 +37,7 @@ import org.springframework.stereotype.Repository
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
import reactor.core.publisher.TopicProcessor import reactor.core.publisher.TopicProcessor
import java.util.* import java.util.*
import java.util.concurrent.Callable
import java.util.concurrent.ConcurrentHashMap import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.locks.ReentrantLock import java.util.concurrent.locks.ReentrantLock
import kotlin.concurrent.withLock import kotlin.concurrent.withLock
@@ -54,36 +58,60 @@ class CurrentUpstreams(
fun update(change: UpstreamChange) { fun update(change: UpstreamChange) {
updateLock.withLock { updateLock.withLock {
val chain = change.chain val chain = change.chain
val up = change.upstream when (BlockchainType.fromBlockchain(chain)) {
.cast(EthereumUpstream::class.java, EthereumApi::class.java) as Upstream<EthereumApi> BlockchainType.ETHEREUM -> {
val current = chainMapping[chain] as ChainUpstreams<EthereumApi>? val up = change.upstream
if (change.type == UpstreamChange.ChangeType.REMOVED) { .cast(EthereumUpstream::class.java, EthereumApi::class.java) as Upstream<EthereumApi>
current?.removeUpstream(up.getId()) val current = chainMapping[chain] as ChainUpstreams<EthereumApi>?
log.info("Upstream ${change.upstream.getId()} with chain $chain has been removed") val factory = Callable {
} else { EthereumChainUpstreams(chain, ArrayList(), cachesFactory.getCaches(chain), objectMapper) as ChainUpstreams<EthereumApi>
if (current == null) {
val created = EthereumChainUpstreams(chain, ArrayList(), cachesFactory.getCaches(chain), objectMapper)
if (up is CachesEnabled) {
up.setCaches(created.caches)
} }
created.addUpstream(up) processUpdate(change, up, current, factory)
created.start() }
chainMapping[chain] = created BlockchainType.BITCOIN -> {
chainsBus.onNext(chain) val up = change.upstream
} else { .cast(BitcoinUpstream::class.java, DirectBitcoinApi::class.java)
if (up is CachesEnabled) { val current = chainMapping[chain] as ChainUpstreams<DirectBitcoinApi>?
up.setCaches(current.caches) val factory = Callable {
BitcoinChainUpstreams(chain, ArrayList(), cachesFactory.getCaches(chain), objectMapper) as ChainUpstreams<DirectBitcoinApi>
} }
current.addUpstream(up) processUpdate(change, up, current, factory)
} }
if (!callTargets.containsKey(chain)) { else -> {
setupDefaultMethods(chain) log.error("Update for unsupported chain: $chain")
} }
log.info("Upstream ${change.upstream.getId()} with chain $chain has been added")
} }
} }
} }
fun <A : UpstreamApi> processUpdate(change: UpstreamChange, up: Upstream<A>, current: ChainUpstreams<A>?, factory: Callable<ChainUpstreams<A>>) {
val chain = change.chain
if (change.type == UpstreamChange.ChangeType.REMOVED) {
current?.removeUpstream(up.getId())
log.info("Upstream ${change.upstream.getId()} with chain $chain has been removed")
} else {
if (current == null) {
val created = factory.call()
if (up is CachesEnabled) {
up.setCaches(created.caches)
}
created.addUpstream(up)
created.start()
chainMapping[chain] = created
chainsBus.onNext(chain)
} else {
if (up is CachesEnabled) {
up.setCaches(current.caches)
}
current.addUpstream(up)
}
if (!callTargets.containsKey(chain)) {
setupDefaultMethods(chain)
}
log.info("Upstream ${change.upstream.getId()} with chain $chain has been added")
}
}
override fun getUpstream(chain: Chain): AggregatedUpstream<*>? { override fun getUpstream(chain: Chain): AggregatedUpstream<*>? {
return chainMapping[chain] return chainMapping[chain]
} }
@@ -109,7 +137,11 @@ class CurrentUpstreams(
} }
fun setupDefaultMethods(chain: Chain): CallMethods { fun setupDefaultMethods(chain: Chain): CallMethods {
val created = DefaultEthereumMethods(objectMapper, chain) val created = when (BlockchainType.fromBlockchain(chain)) {
BlockchainType.ETHEREUM -> DefaultEthereumMethods(objectMapper, chain)
BlockchainType.BITCOIN -> DefaultBitcoinMethods(objectMapper)
else -> throw IllegalStateException("Unsupported chain: $chain")
}
callTargets[chain] = created callTargets[chain] = created
return created return created
} }

View File

@@ -15,20 +15,29 @@
*/ */
package io.emeraldpay.dshackle.upstream package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
import reactor.core.publisher.TopicProcessor import reactor.core.publisher.TopicProcessor
import java.util.concurrent.atomic.AtomicReference import java.util.concurrent.atomic.AtomicReference
abstract class DefaultUpstream<U : UpstreamApi>( abstract class DefaultUpstream<U : UpstreamApi>(
private val id: String,
defaultLag: Long, defaultLag: Long,
defaultAvail: UpstreamAvailability defaultAvail: UpstreamAvailability,
private val options: UpstreamsConfig.Options,
private val targets: CallMethods?
) : Upstream<U> { ) : Upstream<U> {
constructor() : this(Long.MAX_VALUE, UpstreamAvailability.UNAVAILABLE) constructor(id: String, options: UpstreamsConfig.Options, targets: CallMethods?) : this(id, Long.MAX_VALUE, UpstreamAvailability.UNAVAILABLE, options, targets)
private val status = AtomicReference(Status(defaultLag, defaultAvail, statusByLag(defaultLag, defaultAvail))) private val status = AtomicReference(Status(defaultLag, defaultAvail, statusByLag(defaultLag, defaultAvail)))
private val statusStream: TopicProcessor<UpstreamAvailability> = TopicProcessor.create() private val statusStream: TopicProcessor<UpstreamAvailability> = TopicProcessor.create()
override fun isAvailable(): Boolean {
return getStatus() == UpstreamAvailability.OK
}
override fun getStatus(): UpstreamAvailability { override fun getStatus(): UpstreamAvailability {
return status.get().status return status.get().status
} }
@@ -67,5 +76,17 @@ abstract class DefaultUpstream<U : UpstreamApi>(
return this.status.get().lag return this.status.get().lag
} }
override fun getId(): String {
return id
}
override fun getOptions(): UpstreamsConfig.Options {
return options
}
override fun getMethods(): CallMethods {
return targets ?: throw IllegalStateException("Methods are not set")
}
class Status(val lag: Long, val avail: UpstreamAvailability, val status: UpstreamAvailability) class Status(val lag: Long, val avail: UpstreamAvailability, val status: UpstreamAvailability)
} }

View File

@@ -13,12 +13,13 @@
* See the License for the specific language governing permissions and * See the License for the specific language governing permissions and
* limitations under the License. * limitations under the License.
*/ */
package io.emeraldpay.dshackle.upstream.ethereum package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.Head
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
class EmptyEthereumHead : EthereumHead { class EmptyHead : Head {
override fun getFlux(): Flux<BlockContainer> { override fun getFlux(): Flux<BlockContainer> {
return Flux.empty() return Flux.empty()

View File

@@ -17,6 +17,7 @@ package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.data.BlockContainer
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
interface Head { interface Head {
fun getFlux(): Flux<BlockContainer> fun getFlux(): Flux<BlockContainer>

View File

@@ -13,7 +13,7 @@
* See the License for the specific language governing permissions and * See the License for the specific language governing permissions and
* limitations under the License. * limitations under the License.
*/ */
package io.emeraldpay.dshackle.upstream.ethereum package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.cache.Caches import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.cache.CachesEnabled import io.emeraldpay.dshackle.cache.CachesEnabled
@@ -21,9 +21,9 @@ import org.springframework.context.Lifecycle
import reactor.core.Disposable import reactor.core.Disposable
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
class EthereumHeadMerge( class MergedHead(
private val sources: Iterable<EthereumHead> private val sources: Iterable<Head>
): DefaultEthereumHead(), Lifecycle, CachesEnabled { ) : AbstractHead(), Lifecycle, CachesEnabled {
private var subscription: Disposable? = null private var subscription: Disposable? = null

View File

@@ -33,5 +33,6 @@ interface Upstream<out T : UpstreamApi> {
fun getMethods(): CallMethods fun getMethods(): CallMethods
fun getId(): String fun getId(): String
fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A>
fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T
} }

View File

@@ -0,0 +1,82 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
class BitcoinChainUpstreams(
chain: Chain,
val upstreams: MutableList<BitcoinUpstream>,
caches: Caches,
objectMapper: ObjectMapper
) : ChainUpstreams<DirectBitcoinApi>(chain, upstreams as MutableList<Upstream<DirectBitcoinApi>>, caches, objectMapper) {
companion object {
private val log = LoggerFactory.getLogger(BitcoinChainUpstreams::class.java)
}
private var head: Head? = null
override fun init() {
if (upstreams.size > 0) {
head = updateHead()
}
super.init()
}
override fun updateHead(): Head {
head?.let {
if (it is Lifecycle) {
it.stop()
}
}
lagObserver?.stop()
lagObserver = null
val head = if (upstreams.size == 1) {
val upstream = upstreams.first()
upstream.setLag(0)
upstream.getHead()
} else {
val newHead = MergedHead(upstreams.map { it.getHead() }).apply {
this.start()
}
// val lagObserver = TODO
// this.lagObserver = lagObserver
newHead
}
onHeadUpdated(head)
return head
}
override fun setHead(head: Head) {
this.head = head
}
override fun getHead(): Head {
return head!!
}
override fun getLabels(): Collection<UpstreamsConfig.Labels> {
return upstreams.flatMap { it.getLabels() }
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(DirectBitcoinApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
}
return this as Upstream<A>
}
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
return castApi(apiType) as T
}
}

View File

@@ -0,0 +1,33 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
open class BitcoinData(
api: DirectBitcoinApi,
head: Head
) : Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(BitcoinData::class.java)
}
private val mempool = CachingMempoolData(api, head)
open fun getMempool(): CachingMempoolData {
return mempool
}
override fun isRunning(): Boolean {
return mempool.isRunning
}
override fun start() {
mempool.start()
}
override fun stop() {
mempool.stop()
}
}

View File

@@ -0,0 +1,54 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.config.AuthConfig
import io.netty.buffer.Unpooled
import io.netty.handler.codec.http.HttpHeaderNames
import io.netty.handler.codec.http.HttpHeaders
import org.slf4j.LoggerFactory
import reactor.core.publisher.Mono
import reactor.netty.http.client.HttpClient
import java.util.*
import java.util.function.Consumer
open class BitcoinRpcClient(
private val target: String,
basicAuth: AuthConfig.ClientBasicAuth?
) {
companion object {
private val log = LoggerFactory.getLogger(BitcoinRpcClient::class.java)
}
private val httpClient: HttpClient
init {
var build = HttpClient.create()
build = build.headers { h ->
h.add(HttpHeaderNames.CONTENT_TYPE, "application/json")
}
basicAuth?.let { basicAuth ->
val authString: String = basicAuth.username + ":" + basicAuth.password
val authBase64 = Base64.getEncoder().encodeToString(authString.toByteArray())
val auth = "Basic $authBase64"
val headers = Consumer { h: HttpHeaders -> h.add(HttpHeaderNames.AUTHORIZATION, auth) }
build = build.headers(headers)
}
this.httpClient = build
}
fun execute(request: ByteArray): Mono<ByteArray> {
val response = httpClient
.post()
.uri(target)
.send(Mono.just(request).map { Unpooled.wrappedBuffer(it) })
return response.responseContent()
.aggregate()
.asByteArray()
}
}

View File

@@ -0,0 +1,62 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.Defaults
import io.emeraldpay.dshackle.upstream.AbstractHead
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import org.springframework.scheduling.concurrent.CustomizableThreadFactory
import reactor.core.Disposable
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.scheduler.Schedulers
import java.time.Duration
import java.util.concurrent.Executors
class BitcoinRpcHead(
private val api: DirectBitcoinApi,
private val extractBlock: ExtractBlock,
private val interval: Duration = Duration.ofSeconds(15)
) : Head, AbstractHead(), Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(BitcoinRpcHead::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("bitcoin-rpc-head")))
}
private var refreshSubscription: Disposable? = null
override fun isRunning(): Boolean {
return refreshSubscription != null
}
override fun start() {
if (refreshSubscription != null) {
log.warn("Called to start when running")
return
}
val base = Flux.interval(interval)
.publishOn(scheduler)
.flatMap {
api.executeAndResult(0, "getbestblockhash", emptyList(), String::class.java)
.timeout(Defaults.timeout, Mono.error(Exception("Best block hash is not received")))
}
.distinctUntilChanged()
.flatMap { hash ->
api.execute(0, "getblock", listOf(hash))
.map(extractBlock::extract)
.timeout(Defaults.timeout, Mono.error(Exception("Block data is not received")))
}
.onErrorContinue { err, _ ->
log.debug("RPC error ${err.message}")
}
refreshSubscription = super.follow(base)
}
override fun stop() {
val copy = refreshSubscription
refreshSubscription = null
copy?.dispose()
}
}

View File

@@ -0,0 +1,108 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.startup.QuorumForLabels
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Mono
open class BitcoinUpstream(
id: String,
val chain: Chain,
private val api: DirectBitcoinApi,
options: UpstreamsConfig.Options,
val node: QuorumForLabels.QuorumItem,
private val objectMapper: ObjectMapper,
callMethods: CallMethods
) : DefaultUpstream<DirectBitcoinApi>(id, options, callMethods), Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(BitcoinUpstream::class.java)
}
private val head: Head = createHead()
private var validatorSubscription: Disposable? = null
private val data = BitcoinData(api, head)
private fun createHead(): Head {
return BitcoinRpcHead(
api,
ExtractBlock(objectMapper)
)
}
open fun getData(): BitcoinData {
return data
}
override fun getHead(): Head {
return head
}
override fun getApi(matcher: Selector.Matcher): Mono<out DirectBitcoinApi> {
return Mono.just(api)
}
override fun getLabels(): Collection<UpstreamsConfig.Labels> {
return listOf(UpstreamsConfig.Labels())
}
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
return castApi(apiType) as T
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(DirectBitcoinApi::class.java)) {
throw ClassCastException("Cannot cast ${DirectBitcoinApi::class.java} to $apiType")
}
return this as Upstream<A>
}
override fun isRunning(): Boolean {
var runningAny = validatorSubscription != null
if (head is Lifecycle) {
runningAny = runningAny || head.isRunning
}
runningAny = runningAny || data.isRunning
return runningAny
}
override fun start() {
log.info("Configured for ${chain.chainName}")
if (head is Lifecycle) {
if (!head.isRunning) {
head.start()
}
}
data.start()
validatorSubscription?.dispose()
if (getOptions().disableValidation != null && getOptions().disableValidation!!) {
this.setLag(0)
this.setStatus(UpstreamAvailability.OK)
} else {
val validator = BitcoinUpstreamValidator(api, getOptions())
validatorSubscription = validator.start()
.subscribe(this::setStatus)
}
}
override fun stop() {
if (head is Lifecycle) {
head.stop()
}
data.stop()
validatorSubscription?.dispose()
}
}

View File

@@ -0,0 +1,44 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import org.slf4j.LoggerFactory
import org.springframework.scheduling.concurrent.CustomizableThreadFactory
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.scheduler.Schedulers
import java.time.Duration
import java.util.concurrent.Executors
class BitcoinUpstreamValidator(
private val api: DirectBitcoinApi,
private val options: UpstreamsConfig.Options
) {
companion object {
private val log = LoggerFactory.getLogger(BitcoinUpstreamValidator::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("bitcoin-validator")))
}
fun validate(): Mono<UpstreamAvailability> {
return api.executeAndResult(0, "getconnectioncount", emptyList(), Int::class.java)
.map { count ->
val minPeers = options.minPeers ?: 1
if (count < minPeers) {
UpstreamAvailability.IMMATURE
} else {
UpstreamAvailability.OK
}
}
.onErrorReturn(UpstreamAvailability.UNAVAILABLE)
}
fun start(): Flux<UpstreamAvailability> {
return Flux.interval(Duration.ofSeconds(15))
.subscribeOn(scheduler)
.flatMap {
validate()
}
}
}

View File

@@ -0,0 +1,71 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Mono
import java.time.Duration
import java.time.Instant
import java.util.concurrent.atomic.AtomicReference
import java.util.concurrent.locks.ReentrantLock
open class CachingMempoolData(
private val api: DirectBitcoinApi,
private val head: Head
) : Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(CachingMempoolData::class.java)
private val TTL = Duration.ofSeconds(15)
}
private val current = AtomicReference<Container>(Container.empty())
private val updateLock = ReentrantLock()
private var headListener: Disposable? = null
open fun get(): Mono<List<String>> {
val value = current.get()
return if (value.since < Instant.now().minus(TTL)) {
updateLock.lock()
fetchFromUpstream()
.timeout(Duration.ofSeconds(3), Mono.empty())
.doOnNext {
current.set(Container(Instant.now(), it))
}.doFinally {
updateLock.unlock()
}
} else {
Mono.just(value.value)
}
}
fun fetchFromUpstream(): Mono<List<String>> {
return api.executeAndResult(0, "getrawmempool", emptyList(), List::class.java) as Mono<List<String>>
}
class Container(val since: Instant, val value: List<String>) {
companion object {
fun empty(): Container {
return Container(Instant.MIN, emptyList())
}
}
}
override fun isRunning(): Boolean {
return headListener != null
}
override fun start() {
headListener?.dispose()
headListener = head.getFlux().doOnNext {
current.set(Container.empty())
}.subscribe()
}
override fun stop() {
headListener?.dispose()
headListener = null
}
}

View File

@@ -0,0 +1,71 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.quorum.*
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.infinitape.etherjar.rpc.JacksonRpcConverter
import io.infinitape.etherjar.rpc.RpcException
import java.util.*
class DefaultBitcoinMethods(
private val objectMapper: ObjectMapper
) : CallMethods {
//TODO maybe Ethereum RPC parser should not be really used for Bitcoin
private val jacksonRpcConverter = JacksonRpcConverter(objectMapper)
private val anyResponseMethods = listOf(
"getblock",
"gettransaction", "getrawtransaction", "gettxout",
"getmemorypool"
).sorted()
private val headVerifiedMethods = listOf(
"getbestblockhash", "getblocknumber", "getblockcount",
"listunspent", "getreceivedbyaddress"
).sorted()
private val hardcodedMethods = listOf(
"getconnectioncount", "getnetworkinfo"
).sorted()
private val broadcastMethods = listOf(
"sendrawtransaction"
).sorted()
private val allowedMethods = (anyResponseMethods + hardcodedMethods + headVerifiedMethods).sorted()
override fun getQuorumFor(method: String): CallQuorum {
return when {
Collections.binarySearch(hardcodedMethods, method) >= 0 -> AlwaysQuorum()
Collections.binarySearch(anyResponseMethods, method) >= 0 -> NotLaggingQuorum(2)
Collections.binarySearch(headVerifiedMethods, method) >= 0 -> NotLaggingQuorum(0)
Collections.binarySearch(broadcastMethods, method) >= 0 -> BroadcastQuorum(jacksonRpcConverter)
else -> AlwaysQuorum()
}
}
override fun isAllowed(method: String): Boolean {
return Collections.binarySearch(allowedMethods, method) >= 0
}
override fun getSupportedMethods(): Set<String> {
return allowedMethods.toSortedSet()
}
override fun isHardcoded(method: String): Boolean {
return Collections.binarySearch(hardcodedMethods, method) >= 0;
}
override fun executeHardcoded(method: String): Any {
return when (method) {
"getconnectioncount" -> 42
"getnetworkinfo" -> mapOf(
"version" to 700000,
"subversion" to "/EmeraldDshackle:v0.7/"
)
else -> throw RpcException(-32601, "Method not found")
}
}
}

View File

@@ -0,0 +1,107 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.JavaType
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.upstream.UpstreamApi
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.grpc.Status
import io.grpc.StatusRuntimeException
import io.infinitape.etherjar.rpc.RpcException
import io.infinitape.etherjar.rpc.RpcResponseError
import io.infinitape.etherjar.rpc.json.FullResponseJson
import io.infinitape.etherjar.rpc.json.RequestJson
import io.infinitape.etherjar.rpc.json.ResponseJson
import org.slf4j.LoggerFactory
import reactor.core.publisher.Mono
open class DirectBitcoinApi(
val bitcoinRpcClient: BitcoinRpcClient,
val objectMapper: ObjectMapper,
val targets: CallMethods
) : UpstreamApi {
companion object {
private val log = LoggerFactory.getLogger(DirectBitcoinApi::class.java)
}
open override fun execute(id: Int, method: String, params: List<Any>): Mono<ByteArray> {
//TODO it's almost the same code as for DirectEthereumApi; refactor
val result: Mono<out Any> = when {
targets.isHardcoded(method) -> Mono.just(method).map { targets.executeHardcoded(it) }
targets.isAllowed(method) -> executeAndResult(id, method, params, Object::class.java)
else -> Mono.error(RpcException(-32601, "Method not allowed or not found"))
}
return processResult(id, method, result)
}
public fun processResult(id: Int, method: String, result: Mono<out Any>): Mono<ByteArray> {
//TODO it's the same code as for DirectEthereumApi; refactor
return result
.doOnError { t ->
log.warn("Upstream error: [${t.message}] for $method")
}
.map {
val resp = ResponseJson<Any, Int>()
resp.id = id
resp.result = it
resp
}
.switchIfEmpty(
Mono.fromCallable {
val resp = ResponseJson<Any, Int>()
resp.id = id
resp.result = null
resp
}
)
.map {
objectMapper.writer().writeValueAsBytes(it)
}
.onErrorResume(StatusRuntimeException::class.java) { t ->
if (t.status.code == Status.Code.CANCELLED) {
Mono.empty<ByteArray>()
} else {
Mono.error(RpcException(RpcResponseError.CODE_UPSTREAM_CONNECTION_ERROR, "gRPC error ${t.status}"))
}
}
.onErrorMap { t ->
if (RpcException::class.java.isAssignableFrom(t.javaClass)) {
t
} else {
log.warn("Convert to RPC error. Exception ${t.javaClass}:${t.message}", t)
RpcException(-32020, "Error reading from upstream", null, t)
}
}
.onErrorResume(RpcException::class.java) { t ->
val resp = ResponseJson<Any, Int>()
resp.id = id
resp.error = t.error
Mono.just(objectMapper.writer().writeValueAsBytes(resp))
}
}
open fun <T> executeAndResult(id: Int, method: String, params: List<Any>, resultType: Class<T>): Mono<T> {
val rpc = RequestJson<Int>(method, params, id)
return Mono.just(rpc)
.map(objectMapper::writeValueAsBytes)
.flatMap(bitcoinRpcClient::execute)
.flatMap { json ->
val type: JavaType = objectMapper.typeFactory.constructParametricType(FullResponseJson::class.java, resultType, Int::class.java)
val resp = objectMapper.readerFor(type).readValue<FullResponseJson<T, Int>>(json)
if (resp.hasError()) {
Mono.error(resp.error.asException())
} else {
Mono.just(resp.result)
}
}
}
open fun getBlock(hash: String): Mono<Map<String, Any>> {
return executeAndResult(0, "getblock", listOf(hash), Map::class.java) as Mono<Map<String, Any>>
}
open fun getTx(txid: String): Mono<Map<String, Any>> {
return executeAndResult(0, "getrawtransaction", listOf(txid, true), Map::class.java) as Mono<Map<String, Any>>
}
}

View File

@@ -0,0 +1,56 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.data.TxId
import org.apache.commons.codec.binary.Hex
import org.slf4j.LoggerFactory
import java.math.BigInteger
import java.time.Instant
class ExtractBlock(
private val objectMapper: ObjectMapper
) {
companion object {
private val log = LoggerFactory.getLogger(ExtractBlock::class.java)
@JvmStatic
fun getHeight(data: Map<String, Any>): Long? {
val height = data["height"] as Number? ?: return null
return height.toLong()
}
@JvmStatic
fun getTime(data: Map<String, Any>): Instant? {
val time = data["time"] as Number? ?: return null
return Instant.ofEpochMilli(time.toLong() * 1000)
}
@JvmStatic
fun getDifficulty(data: Map<String, Any>): BigInteger? {
val chainwork = data["chainwork"] as String? ?: return null
return BigInteger(1, Hex.decodeHex(chainwork))
}
}
fun extract(json: ByteArray): BlockContainer {
val data = objectMapper.readValue(json, Map::class.java) as Map<String, Any>
val hash = data["hash"] as String? ?: throw IllegalArgumentException("Block JSON has no hash")
val transactions = (data["tx"] as List<String>?)?.map(TxId.Companion::from) ?: emptyList()
return BlockContainer(
getHeight(data) ?: throw IllegalArgumentException("Block JSON has no height"),
BlockId.from(hash),
getDifficulty(data) ?: throw IllegalArgumentException("Block JSON has no chainwork"),
getTime(data) ?: throw IllegalArgumentException("Block JSON has no time"),
false,
json,
transactions
)
}
}

View File

@@ -22,10 +22,10 @@ import io.emeraldpay.dshackle.quorum.CallQuorum
* Configuration that uses [AlwaysQuorum] for all available methods. The methods list itself * Configuration that uses [AlwaysQuorum] for all available methods. The methods list itself
* is provided in constructor (or empty otherwise) * is provided in constructor (or empty otherwise)
*/ */
class DirectCallMethods(private val methods: Set<String>) : CallMethods { open class DirectCallMethods(private val methods: Set<String>) : CallMethods {
constructor(): this(emptySet()) constructor() : this(emptySet())
constructor(methods: Collection<String>): this(methods.toSet()) constructor(methods: Collection<String>) : this(methods.toSet())
override fun getQuorumFor(method: String): CallQuorum { override fun getQuorumFor(method: String): CallQuorum {
return AlwaysQuorum() return AlwaysQuorum()

View File

@@ -1,49 +1,8 @@
package io.emeraldpay.dshackle.upstream.ethereum package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.upstream.AbstractHead
import org.slf4j.LoggerFactory import io.emeraldpay.dshackle.upstream.Head
import reactor.core.Disposable
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
import java.util.concurrent.atomic.AtomicReference
open class DefaultEthereumHead: EthereumHead { open class DefaultEthereumHead : Head, AbstractHead() {
private val log = LoggerFactory.getLogger(DefaultEthereumHead::class.java)
private val head = AtomicReference<BlockContainer>(null)
private val stream: TopicProcessor<BlockContainer> = TopicProcessor.create()
fun follow(source: Flux<BlockContainer>): Disposable {
return source.distinctUntilChanged {
it.hash
}.filter { block ->
val curr = head.get()
curr == null || curr.difficulty < block.difficulty
}
.subscribe { block ->
val prev = head.getAndUpdate { curr ->
if (curr == null || curr.difficulty < block.difficulty) {
block
} else {
curr
}
}
if (prev == null || prev.hash != block.hash) {
log.debug("New block ${block.height} ${block.hash}")
stream.onNext(block)
}
}
}
override fun getFlux(): Flux<BlockContainer> {
return Flux.merge(
Mono.justOrEmpty(head.get()),
Flux.from(stream)
).onBackpressureLatest()
}
fun getCurrent(): BlockContainer? {
return head.get()
}
} }

View File

@@ -20,13 +20,8 @@ import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.config.UpstreamsConfig import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.* import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle import org.springframework.context.Lifecycle
import java.lang.IllegalStateException
import java.time.Duration
class EthereumChainUpstreams( class EthereumChainUpstreams(
chain: Chain, chain: Chain,
@@ -39,7 +34,7 @@ class EthereumChainUpstreams(
private val log = LoggerFactory.getLogger(EthereumChainUpstreams::class.java) private val log = LoggerFactory.getLogger(EthereumChainUpstreams::class.java)
} }
private var head: EthereumHead? = null private var head: Head? = null
init { init {
this.init() this.init()
@@ -52,15 +47,15 @@ class EthereumChainUpstreams(
super.init() super.init()
} }
override fun getHead(): EthereumHead { override fun getHead(): Head {
return head!! return head!!
} }
override fun setHead(head: Head) { override fun setHead(head: Head) {
this.head = head as EthereumHead this.head = head
} }
override fun updateHead(): EthereumHead { override fun updateHead(): Head {
head?.let { head?.let {
if (it is Lifecycle) { if (it is Lifecycle) {
it.stop() it.stop()
@@ -73,7 +68,7 @@ class EthereumChainUpstreams(
upstream.setLag(0) upstream.setLag(0)
upstream.getHead() upstream.getHead()
} else { } else {
val newHead = EthereumHeadMerge(upstreams.map { it.getHead() }).apply { val newHead = MergedHead(upstreams.map { it.getHead() }).apply {
this.start() this.start()
} }
val lagObserver = EthereumHeadLagObserver(newHead, upstreams as Collection<Upstream<EthereumApi>>).apply { val lagObserver = EthereumHeadLagObserver(newHead, upstreams as Collection<Upstream<EthereumApi>>).apply {
@@ -90,34 +85,19 @@ class EthereumChainUpstreams(
return upstreams.flatMap { it.getLabels() } return upstreams.flatMap { it.getLabels() }
} }
override fun printStatus() {
var height: Long? = null
try {
height = getHead().getFlux().next().block(Duration.ofSeconds(1))?.height
} catch (e: IllegalStateException) {
//timout
} catch (e: Exception) {
log.warn("Head processing error: ${e.javaClass} ${e.message}")
}
val statuses = upstreams.map { it.getStatus() }
.groupBy { it }
.map { "${it.key.name}/${it.value.size}" }
.joinToString(",")
val lag = upstreams.map { it.getLag() }
.joinToString(", ")
log.info("State of ${chain.chainCode}: height=${height ?: '?'}, status=$statuses, lag=[$lag]")
}
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T { override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) { if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType") throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
} }
if (!upstreamType.isAssignableFrom(EthereumApi::class.java)) { return castApi(apiType) as T
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $upstreamType") }
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
} }
return this as T return this as Upstream<A>
} }
} }

View File

@@ -1,24 +0,0 @@
/**
* Copyright (c) 2019 ETCDEV GmbH
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.upstream.Head
import io.infinitape.etherjar.domain.TransactionId
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
interface EthereumHead : Head {
}

View File

@@ -18,12 +18,13 @@ package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.upstream.HeadLagObserver import io.emeraldpay.dshackle.upstream.HeadLagObserver
import io.emeraldpay.dshackle.upstream.Upstream import io.emeraldpay.dshackle.upstream.Upstream
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
import java.time.Duration import java.time.Duration
class EthereumHeadLagObserver( class EthereumHeadLagObserver(
master: EthereumHead, master: Head,
followers: Collection<Upstream<EthereumApi>> followers: Collection<Upstream<EthereumApi>>
) : HeadLagObserver<EthereumApi>(master, followers) { ) : HeadLagObserver<EthereumApi>(master, followers) {

View File

@@ -24,8 +24,6 @@ import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.calls.CallMethods import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.calls.DirectCallMethods import io.emeraldpay.dshackle.upstream.calls.DirectCallMethods
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle import org.springframework.context.Lifecycle
import reactor.core.Disposable import reactor.core.Disposable
@@ -33,15 +31,15 @@ import reactor.core.publisher.Mono
import java.time.Duration import java.time.Duration
open class EthereumUpstream( open class EthereumUpstream(
private val id: String, id: String,
val chain: Chain, val chain: Chain,
private val api: DirectEthereumApi, private val api: DirectEthereumApi,
private val ethereumWs: EthereumWs? = null, private val ethereumWs: EthereumWs? = null,
private val options: UpstreamsConfig.Options, options: UpstreamsConfig.Options,
val node: QuorumForLabels.QuorumItem, val node: QuorumForLabels.QuorumItem,
private val targets: CallMethods, targets: CallMethods,
private val objectMapper: ObjectMapper private val objectMapper: ObjectMapper
) : DefaultUpstream<EthereumApi>(), Upstream<EthereumApi>, CachesEnabled, Lifecycle { ) : DefaultUpstream<EthereumApi>(id, options, targets), Upstream<EthereumApi>, CachesEnabled, Lifecycle {
constructor(id: String, chain: Chain, api: DirectEthereumApi, objectMapper: ObjectMapper) : this(id, chain, api, null, constructor(id: String, chain: Chain, api: DirectEthereumApi, objectMapper: ObjectMapper) : this(id, chain, api, null,
UpstreamsConfig.Options.getDefaults(), QuorumForLabels.QuorumItem(1, UpstreamsConfig.Labels()), UpstreamsConfig.Options.getDefaults(), QuorumForLabels.QuorumItem(1, UpstreamsConfig.Labels()),
@@ -50,7 +48,7 @@ open class EthereumUpstream(
private val log = LoggerFactory.getLogger(EthereumUpstream::class.java) private val log = LoggerFactory.getLogger(EthereumUpstream::class.java)
private val head: EthereumHead = this.createHead() private val head: Head = this.createHead()
private var validatorSubscription: Disposable? = null private var validatorSubscription: Disposable? = null
init { init {
@@ -64,18 +62,14 @@ open class EthereumUpstream(
} }
} }
override fun getId(): String {
return id
}
override fun start() { override fun start() {
log.info("Configured for ${chain.chainName}") log.info("Configured for ${chain.chainName}")
if (options.disableValidation != null && options.disableValidation!!) { if (getOptions().disableValidation != null && getOptions().disableValidation!!) {
this.setLag(0) this.setLag(0)
this.setStatus(UpstreamAvailability.OK) this.setStatus(UpstreamAvailability.OK)
} else { } else {
val validator = UpstreamValidator(this, options) val validator = EthereumUpstreamValidator(this, getOptions())
validatorSubscription = validator.start() validatorSubscription = validator.start()
.subscribe(this::setStatus) .subscribe(this::setStatus)
} }
@@ -93,7 +87,7 @@ open class EthereumUpstream(
} }
} }
open fun createHead(): EthereumHead { open fun createHead(): Head {
return if (ethereumWs != null) { return if (ethereumWs != null) {
val ws = EthereumWsHead(ethereumWs).apply { val ws = EthereumWsHead(ethereumWs).apply {
this.start() this.start()
@@ -102,22 +96,18 @@ open class EthereumUpstream(
val rpc = EthereumRpcHead(api, objectMapper, Duration.ofSeconds(30)).apply { val rpc = EthereumRpcHead(api, objectMapper, Duration.ofSeconds(30)).apply {
this.start() this.start()
} }
EthereumHeadMerge(listOf(rpc, ws)).apply { MergedHead(listOf(rpc, ws)).apply {
this.start() this.start()
} }
} else { } else {
log.warn("Setting up upstream $id with RPC-only access, less effective than WS+RPC") log.warn("Setting up upstream ${this.getId()} with RPC-only access, less effective than WS+RPC")
EthereumRpcHead(api, objectMapper).apply { EthereumRpcHead(api, objectMapper).apply {
this.start() this.start()
} }
} }
} }
override fun isAvailable(): Boolean { override fun getHead(): Head {
return getStatus() == UpstreamAvailability.OK
}
override fun getHead(): EthereumHead {
return head return head
} }
@@ -125,27 +115,23 @@ open class EthereumUpstream(
return Mono.just(api) return Mono.just(api)
} }
override fun getOptions(): UpstreamsConfig.Options {
return options
}
override fun getLabels(): Collection<UpstreamsConfig.Labels> { override fun getLabels(): Collection<UpstreamsConfig.Labels> {
return listOf(node.labels) return listOf(node.labels)
} }
override fun getMethods(): CallMethods {
return targets
}
@Suppress("unchecked") @Suppress("unchecked")
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T { override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) { if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType") throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
} }
if (!upstreamType.isAssignableFrom(EthereumApi::class.java)) { return castApi(apiType) as T
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $upstreamType") }
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
} }
return this as T return this as Upstream<A>
} }
} }

View File

@@ -13,11 +13,12 @@
* See the License for the specific language governing permissions and * See the License for the specific language governing permissions and
* limitations under the License. * limitations under the License.
*/ */
package io.emeraldpay.dshackle.upstream package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.Defaults import io.emeraldpay.dshackle.Defaults
import io.emeraldpay.dshackle.config.UpstreamsConfig import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream import io.emeraldpay.dshackle.upstream.Selector
import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import io.infinitape.etherjar.rpc.* import io.infinitape.etherjar.rpc.*
import org.slf4j.LoggerFactory import org.slf4j.LoggerFactory
import org.springframework.scheduling.concurrent.CustomizableThreadFactory import org.springframework.scheduling.concurrent.CustomizableThreadFactory
@@ -27,13 +28,13 @@ import reactor.core.scheduler.Schedulers
import java.time.Duration import java.time.Duration
import java.util.concurrent.Executors import java.util.concurrent.Executors
class UpstreamValidator( class EthereumUpstreamValidator(
private val ethereumUpstream: EthereumUpstream, private val ethereumUpstream: EthereumUpstream,
private val options: UpstreamsConfig.Options private val options: UpstreamsConfig.Options
) { ) {
companion object { companion object {
private val log = LoggerFactory.getLogger(UpstreamValidator::class.java) private val log = LoggerFactory.getLogger(EthereumUpstreamValidator::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("validator"))) val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("ethereum-validator")))
} }
fun validate(): Mono<UpstreamAvailability> { fun validate(): Mono<UpstreamAvailability> {

View File

@@ -33,7 +33,6 @@ import io.emeraldpay.dshackle.upstream.calls.DirectCallMethods
import io.emeraldpay.dshackle.upstream.ethereum.DefaultEthereumHead import io.emeraldpay.dshackle.upstream.ethereum.DefaultEthereumHead
import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.domain.BlockHash import io.infinitape.etherjar.domain.BlockHash
import io.infinitape.etherjar.rpc.* import io.infinitape.etherjar.rpc.*
@@ -59,13 +58,16 @@ open class EthereumGrpcUpstream(
private val blockchainStub: ReactorBlockchainGrpc.ReactorBlockchainStub, private val blockchainStub: ReactorBlockchainGrpc.ReactorBlockchainStub,
private val objectMapper: ObjectMapper, private val objectMapper: ObjectMapper,
private val rpcClient: ReactorEmeraldClient private val rpcClient: ReactorEmeraldClient
) : DefaultUpstream<EthereumApi>(), CachesEnabled, Lifecycle { ) : DefaultUpstream<EthereumApi>(
"$parentId/${chain.chainCode}",
UpstreamsConfig.Options.getDefaults(),
null
), CachesEnabled, Lifecycle {
private var allLabels: Collection<UpstreamsConfig.Labels> = ArrayList<UpstreamsConfig.Labels>() private var allLabels: Collection<UpstreamsConfig.Labels> = ArrayList<UpstreamsConfig.Labels>()
private val log = LoggerFactory.getLogger(EthereumGrpcUpstream::class.java) private val log = LoggerFactory.getLogger(EthereumGrpcUpstream::class.java)
private var caches: Caches? = null private var caches: Caches? = null
private val options = UpstreamsConfig.Options.getDefaults()
private val nodes = AtomicReference<QuorumForLabels>(QuorumForLabels()) private val nodes = AtomicReference<QuorumForLabels>(QuorumForLabels())
private val head = DefaultEthereumHead() private val head = DefaultEthereumHead()
private var targets: CallMethods? = null private var targets: CallMethods? = null
@@ -84,10 +86,6 @@ open class EthereumGrpcUpstream(
} }
} }
override fun getId(): String {
return "$parentId/${chain.chainCode}"
}
override fun start() { override fun start() {
if (this.isRunning) return if (this.isRunning) return
val chainRef = Common.Chain.newBuilder() val chainRef = Common.Chain.newBuilder()
@@ -200,12 +198,12 @@ open class EthereumGrpcUpstream(
} }
override fun isAvailable(): Boolean { override fun isAvailable(): Boolean {
return getStatus() == UpstreamAvailability.OK && head.getCurrent() != null && nodes.get().getAll().any { return super.isAvailable() && head.getCurrent() != null && nodes.get().getAll().any {
it.quorum > 0 it.quorum > 0
} }
} }
override fun getHead(): EthereumHead { override fun getHead(): Head {
return head return head
} }
@@ -213,23 +211,22 @@ open class EthereumGrpcUpstream(
return Mono.just(createApi(matcher)) return Mono.just(createApi(matcher))
} }
override fun getOptions(): UpstreamsConfig.Options {
return options
}
override fun setCaches(caches: Caches) { override fun setCaches(caches: Caches) {
this.caches = caches this.caches = caches
} }
@SuppressWarnings("unchecked") @SuppressWarnings("unchecked")
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T { override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) { if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType") throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
} }
if (!upstreamType.isAssignableFrom(EthereumApi::class.java)) { return castApi(apiType) as T
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $upstreamType")
}
return this as T
} }
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
}
return this as Upstream<A>
}
} }

View File

@@ -77,9 +77,9 @@ class BlocksMemCacheSpec extends Specification {
def act3 = cache.read(BlockId.from(hash3)).block() def act3 = cache.read(BlockId.from(hash3)).block()
def act4 = cache.read(BlockId.from(hash4)).block() def act4 = cache.read(BlockId.from(hash4)).block()
then: then:
act2.hash.toHex() == hash2 act2.hash.toHex() == hash2.substring(2)
act3.hash.toHex() == hash3 act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4 act4.hash.toHex() == hash4.substring(2)
act1 == null act1 == null
} }

View File

@@ -57,13 +57,13 @@ class BlocksRedisCacheSpec extends Specification {
then: then:
dec.height == 100 dec.height == 100
dec.hash.toHex() == hash3 dec.hash.toHex() == hash3.substring(2)
dec.difficulty.toString() == "10515" dec.difficulty.toString() == "10515"
dec.timestamp == Instant.ofEpochSecond(10501050) dec.timestamp == Instant.ofEpochSecond(10501050)
dec.json == "test".bytes dec.json == "test".bytes
dec.transactions.size() == 2 dec.transactions.size() == 2
dec.transactions[0].toHex() == hash2 dec.transactions[0].toHex() == hash2.substring(2)
dec.transactions[1].toHex() == hash1 dec.transactions[1].toHex() == hash1.substring(2)
dec == cont dec == cont
} }

View File

@@ -38,10 +38,10 @@ class HeightCacheSpec extends Specification {
def act3 = cache.read(102).block() def act3 = cache.read(102).block()
def act4 = cache.read(103).block() def act4 = cache.read(103).block()
then: then:
act1.toHex() == hash1 act1.toHex() == hash1.substring(2)
act2.toHex() == hash2 act2.toHex() == hash2.substring(2)
act3.toHex() == hash3 act3.toHex() == hash3.substring(2)
act4.toHex() == hash4 act4.toHex() == hash4.substring(2)
} }
def "Keeps only configured amount"() { def "Keeps only configured amount"() {
@@ -65,8 +65,8 @@ class HeightCacheSpec extends Specification {
def act4 = cache.read(103).block() def act4 = cache.read(103).block()
then: then:
act1 == null act1 == null
act2.toHex() == hash2 act2.toHex() == hash2.substring(2)
act3.toHex() == hash3 act3.toHex() == hash3.substring(2)
act4.toHex() == hash4 act4.toHex() == hash4.substring(2)
} }
} }

View File

@@ -57,9 +57,9 @@ class TxMemCacheSpec extends Specification {
def act3 = cache.read(TxId.from(hash3)).block() def act3 = cache.read(TxId.from(hash3)).block()
def act4 = cache.read(TxId.from(hash4)).block() def act4 = cache.read(TxId.from(hash4)).block()
then: then:
act2.hash.toHex() == hash2 act2.hash.toHex() == hash2.substring(2)
act3.hash.toHex() == hash3 act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4 act4.hash.toHex() == hash4.substring(2)
act1 == null act1 == null
} }
@@ -93,8 +93,8 @@ class TxMemCacheSpec extends Specification {
then: then:
act1 == null act1 == null
act2 == null act2 == null
act3.hash.toHex() == hash3 act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4 act4.hash.toHex() == hash4.substring(2)
} }
def "Evict all by block data"() { def "Evict all by block data"() {
@@ -137,7 +137,7 @@ class TxMemCacheSpec extends Specification {
then: then:
act1 == null act1 == null
act2 == null act2 == null
act3.hash.toHex() == hash3 act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4 act4.hash.toHex() == hash4.substring(2)
} }
} }

View File

@@ -56,8 +56,8 @@ class TxRedisCacheSpec extends Specification {
then: then:
dec.height == 2000 dec.height == 2000
dec.hash.toHex() == hash1 dec.hash.toHex() == hash1.substring(2)
dec.blockId.toHex() == hash2 dec.blockId.toHex() == hash2.substring(2)
dec.json == "test".bytes dec.json == "test".bytes
dec == cont dec == cont
} }
@@ -170,8 +170,8 @@ class TxRedisCacheSpec extends Specification {
act1 == null act1 == null
act2 == null act2 == null
act3 != null act3 != null
act3.hash.toHex() == hash3 act3.hash.toHex() == hash3.substring(2)
act4 != null act4 != null
act4.hash.toHex() == hash4 act4.hash.toHex() == hash4.substring(2)
} }
} }

View File

@@ -0,0 +1,30 @@
package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.BlockHash
import spock.lang.Specification
class BlockIdSpec extends Specification {
def "Create from string"() {
expect:
BlockId.from(source).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "A0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xA0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
def "Create from block hash"() {
expect:
BlockId.from(BlockHash.from(source)).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
}

View File

@@ -0,0 +1,29 @@
package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.TransactionId
import spock.lang.Specification
class TxIdSpec extends Specification {
def "Create from string"() {
expect:
TxId.from(source).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "A0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xA0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
def "Create from block hash"() {
expect:
TxId.from(TransactionId.from(source)).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
}

View File

@@ -259,7 +259,7 @@ class NativeCallSpec extends Specification {
.build() .build()
}) })
.build() .build()
1 * upstreams.getUpstream(Chain.TESTNET_MORDEN) >> null 1 * upstreams.isAvailable(Chain.TESTNET_MORDEN) >> false
when: when:
def resp = nativeCall.prepareCall(req) def resp = nativeCall.prepareCall(req)
then: then:

View File

@@ -0,0 +1,251 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.AggregatedUpstream
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.Upstream
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
import java.time.Instant
class TrackBitcoinAddressSpec extends Specification {
String hash1 = "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27c22"
def "Correct sum from multiple"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-one-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], unspents)
then:
total.size() == 1
total[0].address.chain == Chain.BITCOIN
total[0].address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
total[0].balance.toString() == "32928461"
}
def "Correct sum when other addresses"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-two-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], unspents)
then:
total.size() == 1
total[0].address.chain == Chain.BITCOIN
total[0].address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
total[0].balance.toString() == "32928461"
}
def "Sum for two addresses"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-two-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", "35hK24tcLEWcgNA4JxpvbkNkoAcDGqQPsP"], unspents).sort { it.address.address }
then:
total.size() == 2
with(total[0]) {
address.chain == Chain.BITCOIN
address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
balance.toString() == "32928461"
}
with(total[1]) {
address.chain == Chain.BITCOIN
address.address == "35hK24tcLEWcgNA4JxpvbkNkoAcDGqQPsP"
balance.toString() == "25550215615737"
}
}
def "Zero for empty unspents"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], [])
then:
total.size() == 1
total[0].address.chain == Chain.BITCOIN
total[0].address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
total[0].balance.toString() == "0"
}
def "Zero for unknown address"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-two-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk", "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], unspents).sort { it.address.address }
then:
total.size() == 2
with(total[0]) {
address.address == "16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk"
balance.toString() == "0"
}
with(total[1]) {
address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
balance.toString() == "32928461"
}
}
def "One address for single provided"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def req = BlockchainOuterClass.BalanceRequest.newBuilder()
.setAddress(
Common.AnyAddress.newBuilder()
.setAddressSingle(
Common.SingleAddress.newBuilder()
.setAddress("16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk")
)
)
.build()
when:
def act = track.allAddresses(req)
then:
act == ["16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk"]
}
def "Sorted addresses for multiple provided"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def req = BlockchainOuterClass.BalanceRequest.newBuilder()
.setAddress(
Common.AnyAddress.newBuilder()
.setAddressMulti(
Common.MultiAddress.newBuilder()
.addAddresses(Common.SingleAddress.newBuilder().setAddress("16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk"))
.addAddresses(Common.SingleAddress.newBuilder().setAddress("3BMqADKWoWHPASsUdHvnUL6E1jpZkMnLZz"))
.addAddresses(Common.SingleAddress.newBuilder().setAddress("1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"))
.addAddresses(Common.SingleAddress.newBuilder().setAddress("bc1qdthqvt6cllzej7uhdddrltdfsmnt7d0gl5ue5n"))
)
)
.build()
when:
def act = track.allAddresses(req)
then:
act == ["16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk", "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", "3BMqADKWoWHPASsUdHvnUL6E1jpZkMnLZz", "bc1qdthqvt6cllzej7uhdddrltdfsmnt7d0gl5ue5n"]
}
def "Null for no address provided"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def req = BlockchainOuterClass.BalanceRequest.newBuilder()
.build()
when:
def act = track.allAddresses(req)
then:
act == null
}
def "Build proto for common balance"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def balance = new TrackBitcoinAddress.AddressBalance(Chain.BITCOIN, "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", BigInteger.valueOf(123456))
when:
def act = track.buildResponse(balance)
then:
act.address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
act.balance == "123456"
act.asset.chain.number == Chain.BITCOIN.id
act.asset.code == "BTC"
}
def "Build proto for zero balance"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def balance = new TrackBitcoinAddress.AddressBalance(Chain.BITCOIN, "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", BigInteger.ZERO)
when:
def act = track.buildResponse(balance)
then:
act.address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
act.balance == "0"
act.asset.chain.number == Chain.BITCOIN.id
act.asset.code == "BTC"
}
def "Build proto for all bitcoins"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def balance = new TrackBitcoinAddress.AddressBalance(Chain.BITCOIN, "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", BigInteger.valueOf(21_000_000).multiply(BigInteger.TEN.pow(8)))
when:
def act = track.buildResponse(balance)
then:
act.address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
act.balance == "2100000000000000"
act.asset.chain.number == Chain.BITCOIN.id
act.asset.code == "BTC"
}
def "Get update for a balance"() {
setup:
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
2 * executeAndResult(0, "listunspent", [], List) >>> [
Mono.just([]), Mono.just([[address: "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", amount: 0.0123]])
]
}
def blocks = TopicProcessor.create()
Head head = Mock(Head) {
1 * getFlux() >> Flux.from(blocks)
}
Upstream upstream
upstream = Mock(AggregatedUpstream) {
_ * getApi(_) >> Mono.just(api)
_ * getHead() >> head
_ * castApi(_) >> { return upstream }
}
Upstreams upstreams = Mock(Upstreams) {
_ * getUpstream(Chain.BITCOIN) >> upstream
}
TrackBitcoinAddress track = new TrackBitcoinAddress(upstreams)
when:
def resp = track.subscribe(BlockchainOuterClass.BalanceRequest.newBuilder()
.setAsset(Common.Asset.newBuilder().setChain(Common.ChainRef.CHAIN_BITCOIN))
.setAddress(
Common.AnyAddress.newBuilder().setAddressSingle(
Common.SingleAddress.newBuilder().setAddress("1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK")
)
)
.build()
).map {
it.balance
}
then:
StepVerifier.create(resp)
.expectNext("0")
.then {
blocks.onNext(new BlockContainer(1L, BlockId.from(hash1), BigInteger.ONE, Instant.now(), false, null, []))
}
.expectNext("1230000")
.then {
blocks.onComplete()
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
}

View File

@@ -0,0 +1,276 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.Upstream
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinData
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.bitcoin.CachingMempoolData
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
import java.time.Instant
class TrackBitcoinTxSpec extends Specification {
def "loadMempool() returns not found when not found"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
CachingMempoolData mempoolAccess = Mock(CachingMempoolData) {
1 * get() >> Mono.just([
"69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9",
"d296c6d47335a7f283574b06f1d6303b30ac75631e081ab128346a549ad93350"
])
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getData() >> Mock(BitcoinData) {
_ * getMempool() >> mempoolAccess
}
}
when:
def act = track.loadMempool(upstream, "65ce58db064bd105b14dc76a0bce0df14653cf5263d22d17e78864cf272ee367")
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == false && it.mined == false && it.blockHash == null
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "loadMempool() returns ok when found"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
CachingMempoolData mempoolAccess = Mock(CachingMempoolData) {
1 * get() >> Mono.just([
"69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9",
"d296c6d47335a7f283574b06f1d6303b30ac75631e081ab128346a549ad93350"
])
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getData() >> Mock(BitcoinData) {
_ * getMempool() >> mempoolAccess
}
}
when:
def act = track.loadMempool(upstream, "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9")
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == true && it.mined == false && it.blockHash == null
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "loadExiting() returns not found if not mined"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
1 * getTx(txid) >> Mono.just([
txid: txid
])
}
when:
def act = track.loadExisting(api, txid)
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == true && it.mined == false && it.blockHash == null
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "loadExiting() returns block if mined"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
1 * getTx(txid) >> Mono.just([
txid : txid,
blockhash: "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f",
height : 100
])
}
when:
def act = track.loadExisting(api, txid)
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == true && it.mined == true &&
it.blockHash == "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f" &&
it.height == 100
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Goes with confirmations"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
// start with the current block
def next = Flux.fromIterable([10, 12, 13, 14, 15]).map { h ->
new BlockContainer(h.longValue(), BlockId.from("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f"), BigInteger.ONE, Instant.now(), false, null, [])
}
Head head = Mock(Head) {
1 * getFlux() >> next
}
Upstream upstream = Mock(BitcoinUpstream) {
1 * getHead() >> head
}
def status = new TrackBitcoinTx.TxStatus(
txid, true, 10, true, "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f", Instant.now(), BigInteger.ONE, 0
)
when:
def act = track.withConfirmations(upstream, status)
then:
StepVerifier.create(act)
.expectNextMatches { it.confirmations == 1 }
.expectNextMatches { it.confirmations == 3 }
.expectNextMatches { it.confirmations == 4 }
.expectNextMatches { it.confirmations == 5 }
.expectNextMatches { it.confirmations == 6 }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Wait until mined"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
// start with the current block
def next = Flux.fromIterable([10, 12, 13]).map { h ->
new BlockContainer(h.longValue(), BlockId.from("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f"), BigInteger.ONE, Instant.now(), false, null, [])
}
Head head = Mock(Head) {
1 * getFlux() >> next
}
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
3 * getTx(txid) >>> [
Mono.just([
txid: txid
]),
Mono.just([
txid: txid
]),
Mono.just([
txid : txid,
blockhash: "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f",
height : 100
])
]
}
Upstream upstream = Mock(BitcoinUpstream) {
1 * getHead() >> head
_ * getApi(_) >> Mono.just(api)
}
def status = new TrackBitcoinTx.TxStatus(
txid, false, null, false, null, null, null, 0
)
when:
def act = track.untilMined(upstream, status)
then:
StepVerifier.create(act)
.expectNextMatches { it.mined && it.height == 100 && it.blockHash == "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f" }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Check mempool until found"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
1 * getTx(txid) >> Mono.just([
txid: txid
])
}
Head head = Mock(Head) {
_ * getFlux() >> Flux.empty()
}
CachingMempoolData mempoolAccess = Mock(CachingMempoolData) {
4 * get() >>> [
Mono.just([]),
Mono.just(["4523c7ac0c5c1e5628f025474529c69cd44d7c641db82e6982f5ffe64527efc9"]),
Mono.just(["4523c7ac0c5c1e5628f025474529c69cd44d7c641db82e6982f5ffe64527efc9", txid]),
Mono.just(["4523c7ac0c5c1e5628f025474529c69cd44d7c641db82e6982f5ffe64527efc9", txid]) //second call when started over
]
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getApi(_) >> Mono.just(api)
_ * getHead() >> head
_ * getData() >> Mock(BitcoinData) {
_ * getMempool() >> mempoolAccess
}
}
when:
def steps = StepVerifier.withVirtualTime {
track.untilFound(Chain.BITCOIN, api, upstream, txid).take(1)
}
then:
steps
.expectSubscription()
.expectNoEvent(Duration.ofSeconds(3))
.expectNextMatches { it.found && !it.mined }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Subscribe to an existing tx"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
_ * getTx(txid) >> Mono.just([
txid : txid,
blockhash: "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f",
height : 1
])
_ * getBlock("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f") >> Mono.just([
height : 1,
chainwork: "01",
time : 10000
])
}
def next = Flux.fromIterable([10, 11, 12]).map { h ->
new BlockContainer(h.longValue(), BlockId.from("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f"), BigInteger.ONE, Instant.now(), false, null, [])
}
Head head = Mock(Head) {
_ * getFlux() >> next
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getApi(_) >> Mono.just(api)
_ * getHead() >> head
}
when:
def act = track.subscribe(Chain.BITCOIN, api, upstream, txid)
then:
StepVerifier.create(act)
.expectNextMatches { it.found && it.mined && it.confirmations == 10 && it.blockHash == "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f" }
.expectNextMatches { it.found && it.mined && it.confirmations == 11 }
.expectNextMatches { it.found && it.mined && it.confirmations == 12 }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
}

View File

@@ -67,7 +67,7 @@ class TrackEthereumAddressSpec extends Specification {
apiMock.answer("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0x499602D2") apiMock.answer("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0x499602D2")
when: when:
def flux = trackAddress.getBalance(Mono.just(req)) def flux = trackAddress.getBalance(req)
then: then:
StepVerifier.create(flux) StepVerifier.create(flux)
.expectNext(exp) .expectNext(exp)
@@ -111,7 +111,7 @@ class TrackEthereumAddressSpec extends Specification {
apiMock.answerOnce("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0x499602D2") apiMock.answerOnce("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0x499602D2")
apiMock.answerOnce("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0xff98") apiMock.answerOnce("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0xff98")
when: when:
def flux = trackAddress.subscribe(Mono.just(req)) def flux = trackAddress.subscribe(req)
then: then:
StepVerifier.create(flux) StepVerifier.create(flux)
.expectNext(exp1) .expectNext(exp1)

View File

@@ -103,7 +103,7 @@ class TrackEthereumTxSpec extends Specification {
upstreamMock.nextBlock(BlockContainer.from(blockHeadJson, TestingCommons.objectMapper())) upstreamMock.nextBlock(BlockContainer.from(blockHeadJson, TestingCommons.objectMapper()))
when: when:
def flux = trackTx.add(Mono.just(req)) def flux = trackTx.subscribe(req)
then: then:
StepVerifier.create(flux) StepVerifier.create(flux)
.expectNext(exp1) .expectNext(exp1)
@@ -134,7 +134,7 @@ class TrackEthereumTxSpec extends Specification {
when: when:
def act = StepVerifier.withVirtualTime { def act = StepVerifier.withVirtualTime {
return trackTx.add(Mono.just(req)) return trackTx.subscribe(req)
} }
then: then:
act act
@@ -192,7 +192,7 @@ class TrackEthereumTxSpec extends Specification {
when: when:
def act = StepVerifier.withVirtualTime { def act = StepVerifier.withVirtualTime {
return trackTx.add(Mono.just(req)) return trackTx.subscribe(req)
} }
then: then:
act act
@@ -301,7 +301,7 @@ class TrackEthereumTxSpec extends Specification {
} }
when: when:
def flux = trackTx.add(Mono.just(req)) def flux = trackTx.subscribe(req)
then: then:
StepVerifier.create(flux) StepVerifier.create(flux)
.expectNext(exp1.build()) .expectNext(exp1.build())

View File

@@ -16,12 +16,12 @@
package io.emeraldpay.dshackle.test package io.emeraldpay.dshackle.test
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead import io.emeraldpay.dshackle.upstream.Head
import reactor.core.publisher.Flux import reactor.core.publisher.Flux
import reactor.core.publisher.Mono import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor import reactor.core.publisher.TopicProcessor
class EthereumHeadMock implements EthereumHead { class EthereumHeadMock implements Head {
private TopicProcessor<BlockContainer> bus = TopicProcessor.create() private TopicProcessor<BlockContainer> bus = TopicProcessor.create()
private BlockContainer latest private BlockContainer latest

View File

@@ -17,11 +17,11 @@ package io.emeraldpay.dshackle.test
import io.emeraldpay.dshackle.config.UpstreamsConfig import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.calls.CallMethods import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.startup.QuorumForLabels import io.emeraldpay.dshackle.startup.QuorumForLabels
import io.emeraldpay.dshackle.upstream.calls.DefaultEthereumMethods import io.emeraldpay.dshackle.upstream.calls.DefaultEthereumMethods
import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.dshackle.upstream.UpstreamAvailability import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
@@ -58,12 +58,12 @@ class EthereumUpstreamMock extends EthereumUpstream {
} }
@Override @Override
EthereumHead createHead() { Head createHead() {
return ethereumHeadMock return ethereumHeadMock
} }
@Override @Override
EthereumHead getHead() { Head getHead() {
return ethereumHeadMock return ethereumHeadMock
} }
} }

View File

@@ -21,15 +21,15 @@ import io.grpc.inprocess.InProcessChannelBuilder
import io.grpc.inprocess.InProcessServerBuilder import io.grpc.inprocess.InProcessServerBuilder
import io.grpc.testing.GrpcCleanupRule import io.grpc.testing.GrpcCleanupRule
class MockServer { class MockGrpcServer {
GrpcCleanupRule grpcCleanup = new GrpcCleanupRule() GrpcCleanupRule grpcCleanup = new GrpcCleanupRule()
ReactorBlockchainGrpc.ReactorBlockchainStub clientForServer(ReactorBlockchainGrpc.BlockchainImplBase impl){ ReactorBlockchainGrpc.ReactorBlockchainStub clientForServer(ReactorBlockchainGrpc.BlockchainImplBase impl) {
String serverName = InProcessServerBuilder.generateName() String serverName = InProcessServerBuilder.generateName()
grpcCleanup.register(InProcessServerBuilder grpcCleanup.register(InProcessServerBuilder
.forName(serverName).directExecutor().addService(impl).build().start()); .forName(serverName).directExecutor().addService(impl).build().start());
def channel = grpcCleanup.register(InProcessChannelBuilder.forName(serverName).directExecutor().build()) def channel = grpcCleanup.register(InProcessChannelBuilder.forName(serverName).directExecutor().build())
return ReactorBlockchainGrpc.newReactorStub(channel) return ReactorBlockchainGrpc.newReactorStub(channel)
} }

View File

@@ -10,7 +10,6 @@ import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.data.TxId import io.emeraldpay.dshackle.data.TxId
import io.emeraldpay.dshackle.test.TestingCommons import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.infinitape.etherjar.domain.BlockHash import io.infinitape.etherjar.domain.BlockHash
import io.infinitape.etherjar.domain.TransactionId import io.infinitape.etherjar.domain.TransactionId
import io.infinitape.etherjar.rpc.json.BlockJson import io.infinitape.etherjar.rpc.json.BlockJson
@@ -30,7 +29,7 @@ class CachingEthereumApiSpec extends Specification {
def "Get blockNumber from head"() { def "Get blockNumber from head"() {
setup: setup:
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.default(objectMapper), Caches.default(objectMapper),
@@ -58,7 +57,7 @@ class CachingEthereumApiSpec extends Specification {
def "Return empty if block is not cached"() { def "Return empty if block is not cached"() {
setup: setup:
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.default(objectMapper), Caches.default(objectMapper),
@@ -76,7 +75,7 @@ class CachingEthereumApiSpec extends Specification {
def "Return block by hash when cached"() { def "Return block by hash when cached"() {
setup: setup:
def cache = new BlocksMemCache(); def cache = new BlocksMemCache();
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(cache).build(), Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(cache).build(),
@@ -106,7 +105,7 @@ class CachingEthereumApiSpec extends Specification {
setup: setup:
def blocksCache = new BlocksMemCache() def blocksCache = new BlocksMemCache()
def heightCache = new HeightCache() def heightCache = new HeightCache()
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setBlockByHeight(heightCache).build(), Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setBlockByHeight(heightCache).build(),
@@ -135,7 +134,7 @@ class CachingEthereumApiSpec extends Specification {
setup: setup:
def blocksCache = Mock(BlocksMemCache) def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache) def txCache = Mock(TxMemCache)
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).build(), Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).build(),
@@ -161,7 +160,7 @@ class CachingEthereumApiSpec extends Specification {
setup: setup:
def blocksCache = Mock(BlocksMemCache) def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache) def txCache = Mock(TxMemCache)
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).build(), Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).build(),
@@ -191,7 +190,7 @@ class CachingEthereumApiSpec extends Specification {
def blocksCache = Mock(BlocksMemCache) def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache) def txCache = Mock(TxMemCache)
def heightCache = Mock(HeightCache) def heightCache = Mock(HeightCache)
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).setBlockByHeight(heightCache).build(), Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).setBlockByHeight(heightCache).build(),
@@ -219,7 +218,7 @@ class CachingEthereumApiSpec extends Specification {
def blocksCache = Mock(BlocksMemCache) def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache) def txCache = Mock(TxMemCache)
def heightCache = Mock(HeightCache) def heightCache = Mock(HeightCache)
def head = Mock(EthereumHead.class) def head = Mock(Head.class)
def api = new CachingEthereumApi( def api = new CachingEthereumApi(
objectMapper, objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).setBlockByHeight(heightCache).build(), Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).setBlockByHeight(heightCache).build(),

View File

@@ -0,0 +1,86 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.config.AuthConfig
import org.mockserver.integration.ClientAndServer
import org.mockserver.matchers.Times
import org.mockserver.model.HttpRequest
import org.mockserver.model.HttpResponse
import org.mockserver.model.MediaType
import org.mockserver.verify.VerificationTimes
import reactor.test.StepVerifier
import spock.lang.Shared
import spock.lang.Specification
import java.time.Duration
class BitcoinRpcClientSpec extends Specification {
ClientAndServer mockServer
def setup() {
mockServer = ClientAndServer.startClientAndServer(18332);
}
def cleanup() {
mockServer.stop()
}
def "Make request"() {
setup:
def client = new BitcoinRpcClient("localhost:18332", null)
mockServer.when(
HttpRequest.request()
.withMethod("POST")
.withBody("ping"),
Times.exactly(1)
).respond(
HttpResponse.response()
.withBody("pong")
)
when:
def act = client.execute("ping".bytes).map { new String(it) }
then:
StepVerifier.create(act)
.expectNext("pong")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody("ping")
.withContentType(MediaType.APPLICATION_JSON)
)
}
def "Make request with basic auth"() {
setup:
def auth = new AuthConfig.ClientBasicAuth("user", "passwd")
def client = new BitcoinRpcClient("localhost:18332", auth)
mockServer.when(
HttpRequest.request()
.withMethod("POST")
.withBody("ping")
).respond(
HttpResponse.response()
.withBody("pong")
)
when:
def act = client.execute("ping".bytes).map { new String(it) }
then:
StepVerifier.create(act)
.expectNext("pong")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody("ping")
.withContentType(MediaType.APPLICATION_JSON)
.withHeader("authorization", "Basic dXNlcjpwYXNzd2Q=")
)
}
}

View File

@@ -0,0 +1,87 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.test.TestingCommons
import reactor.core.publisher.Mono
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
class BitcoinRpcHeadSpec extends Specification {
def "Follow 2 blocks created over 3 requests"() {
setup:
String hash1 = "0000000000000000000cf5a5d4dfc4347c0c1a863ec5fdb429b02b2162e50001"
String hash2 = "0000000000000000000cf5a5d4dfc4347c0c1a863ec5fdb429b02b2162e50002"
def block1 = """
{
"hash": "${hash1}",
"confirmations": 2,
"strippedsize": 800464,
"size": 1591897,
"weight": 3993289,
"height": 626472,
"version": 549453824,
"versionHex": "20c00000",
"merkleroot": "7835a26b6c9316232f8194c6b64b0a366e26a9a14e6a7e7409c34594d98a83c2",
"tx": [],
"time": 1587171526,
"mediantime": 1587168930,
"nonce": 4111305375,
"bits": "171320bc",
"difficulty": 14715214060656.53,
"chainwork": "00000000000000000000000000000000000000000e9baec5f63190a2b0bbcf6b",
"nTx": 0,
"previousblockhash": "000000000000000000106b8cb739dd3c412613aa5d459a739794ec2572a74c10"
}
"""
def block2 = """
{
"hash": "${hash2}",
"confirmations": 2,
"strippedsize": 800464,
"size": 1591897,
"weight": 3993289,
"height": 626473,
"version": 549453824,
"versionHex": "20c00000",
"merkleroot": "7835a26b6c9316232f8194c6b64b0a366e26a9a14e6a7e7409c34594d98a83c2",
"tx": [],
"time": 1587172526,
"mediantime": 1587168930,
"nonce": 4111305375,
"bits": "171320bc",
"difficulty": 14715214060656.53,
"chainwork": "00000000000000000000000000000000000000000e9baec5f63190a2b0bbcf6c",
"nTx": 0,
"previousblockhash": "${hash1}"
}
"""
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
_ * executeAndResult(_, "getbestblockhash", _, String) >>> [
Mono.just(hash1), Mono.just(hash1), Mono.just(hash2)
]
_ * execute(_, "getblock", [hash1]) >> Mono.just(block1.bytes)
_ * execute(_, "getblock", [hash2]) >> Mono.just(block2.bytes)
}
BitcoinRpcHead head = new BitcoinRpcHead(api, new ExtractBlock(TestingCommons.objectMapper()), Duration.ofMillis(200))
when:
def act = head.flux.take(2)
head.start()
then:
StepVerifier.create(act)
.expectNextMatches { block ->
block.hash.toHex() == hash1
}.as("Block 1")
.expectNextMatches { block ->
block.hash.toHex() == hash2
}.as("Block 2")
.expectComplete()
.verify(Duration.ofSeconds(3))
}
}

View File

@@ -0,0 +1,108 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.test.TestingCommons
import io.infinitape.etherjar.rpc.RpcException
import org.mockserver.integration.ClientAndServer
import org.mockserver.model.HttpRequest
import org.mockserver.model.HttpResponse
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
class DirectBitcoinApiSpec extends Specification {
ClientAndServer mockServer
DirectBitcoinApi api
def setup() {
mockServer = ClientAndServer.startClientAndServer(18332);
api = new DirectBitcoinApi(
new BitcoinRpcClient("localhost:18332", null),
TestingCommons.objectMapper(), new DefaultBitcoinMethods(TestingCommons.objectMapper())
)
}
def cleanup() {
mockServer.stop()
}
def "Request simple"() {
setup:
def resp = '{' +
' "result": "0000000000000000000889c2e52ca5e1cecac60bce9a3754201a7a9a67791e90",' +
' "error": null,' +
' "id": 15' +
'}'
mockServer.when(
HttpRequest.request()
).respond(
HttpResponse.response(resp)
)
when:
def act = api.executeAndResult(15, "getbestblockhash", [], String)
then:
StepVerifier.create(act)
.expectNext("0000000000000000000889c2e52ca5e1cecac60bce9a3754201a7a9a67791e90")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody('{"jsonrpc":"2.0","method":"getbestblockhash","params":[],"id":15}')
)
}
def "Request with params"() {
setup:
def resp = '{' +
' "result": "something",' +
' "id": 1' +
'}'
mockServer.when(
HttpRequest.request()
).respond(
HttpResponse.response(resp)
)
when:
def act = api.executeAndResult(1, "getsomething", ["something", false], String)
then:
StepVerifier.create(act)
.expectNext("something")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody('{"jsonrpc":"2.0","method":"getsomething","params":["something",false],"id":1}')
)
}
def "Returns error"() {
setup:
def resp = '{' +
' "result": null,' +
' "error": {' +
' "code": -32601,' +
' "message": "Method not found"' +
' },' +
' "id": 1' +
'}'
mockServer.when(
HttpRequest.request()
).respond(
HttpResponse.response(resp)
)
when:
def act = api.executeAndResult(1, "geterror", [], String)
then:
StepVerifier.create(act)
.expectError(RpcException)
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody('{"jsonrpc":"2.0","method":"geterror","params":[],"id":1}')
)
}
}

View File

@@ -0,0 +1,90 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.test.TestingCommons
import spock.lang.Specification
class ExtractBlockSpec extends Specification {
ExtractBlock extractBlock = new ExtractBlock(TestingCommons.objectMapper())
def "Extract standard block"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/block-626472.json").bytes
when:
def act = extractBlock.extract(json)
then:
act.hash.toHex() == "0000000000000000000889c2e52ca5e1cecac60bce9a3754201a7a9a67791e90"
act.height == 626472
act.timestamp.toString() == "2020-04-18T00:58:46Z"
act.difficulty.toString(16) == "e9baec5f63190a2b0bbcf6b"
!act.full
act.transactions.size() == 1487
act.json == json
}
def "Shouldn't extract time from empty"() {
when:
def act = ExtractBlock.getTime([:])
then:
act == null
}
def "Shouldn't extract time from null"() {
when:
def act = ExtractBlock.getTime([time: null])
then:
act == null
}
def "Extract correct time"() {
expect:
ExtractBlock.getTime(block).toString() == time
where:
time | block
"2020-04-18T00:58:46Z" | [time: 1587171526]
"1970-01-01T00:00:00Z" | [time: 0]
}
def "Shouldn't extract height from empty"() {
when:
def act = ExtractBlock.getHeight([:])
then:
act == null
}
def "Shouldn't extract height from null"() {
when:
def act = ExtractBlock.getHeight([height: null])
then:
act == null
}
def "Should extract height"() {
when:
def act = ExtractBlock.getHeight([height: 123456])
then:
act == 123456L
}
def "Shouldn't extract difficulty from empty"() {
when:
def act = ExtractBlock.getDifficulty([:])
then:
act == null
}
def "Shouldn't extract difficulty from null"() {
when:
def act = ExtractBlock.getDifficulty([chainwork: null])
then:
act == null
}
def "Should extract difficulty"() {
when:
def act = ExtractBlock.getDifficulty([chainwork: "00000000000000000000000000000000000000000ea25fe034fa07aed9e338eb"])
then:
act.toString(16) == "ea25fe034fa07aed9e338eb"
}
}

View File

@@ -18,6 +18,7 @@ package io.emeraldpay.dshackle.upstream.ethereum
import com.fasterxml.jackson.databind.ObjectMapper import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.data.BlockContainer import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.test.TestingCommons import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.HeadLagObserver import io.emeraldpay.dshackle.upstream.HeadLagObserver
import io.emeraldpay.dshackle.upstream.Upstream import io.emeraldpay.dshackle.upstream.Upstream
import io.infinitape.etherjar.domain.BlockHash import io.infinitape.etherjar.domain.BlockHash
@@ -37,10 +38,10 @@ class EthereumHeadLagObserverSpec extends Specification {
def "Updates lag distance"() { def "Updates lag distance"() {
setup: setup:
EthereumHead master = Mock() Head master = Mock()
EthereumHead head1 = Mock() Head head1 = Mock()
EthereumHead head2 = Mock() Head head2 = Mock()
Upstream up1 = Mock { Upstream up1 = Mock {
_ * getHead() >> head1 _ * getHead() >> head1
@@ -89,7 +90,7 @@ class EthereumHeadLagObserverSpec extends Specification {
def "Probes until there is no difference"() { def "Probes until there is no difference"() {
setup: setup:
EthereumHead master = Mock() Head master = Mock()
HeadLagObserver observer = new EthereumHeadLagObserver(master, []) HeadLagObserver observer = new EthereumHeadLagObserver(master, [])
Upstream up = Mock() Upstream up = Mock()
@@ -118,7 +119,7 @@ class EthereumHeadLagObserverSpec extends Specification {
def "Correct distance"() { def "Correct distance"() {
setup: setup:
EthereumHead master = Mock() Head master = Mock()
HeadLagObserver observer = new EthereumHeadLagObserver(master, []) HeadLagObserver observer = new EthereumHeadLagObserver(master, [])
expect: expect:
def top = new BlockJson().with { def top = new BlockJson().with {

View File

@@ -21,7 +21,7 @@ import io.emeraldpay.api.proto.BlockchainGrpc
import io.emeraldpay.api.proto.BlockchainOuterClass import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.data.BlockId import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.test.MockServer import io.emeraldpay.dshackle.test.MockGrpcServer
import io.emeraldpay.dshackle.test.TestingCommons import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.UpstreamAvailability import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import io.emeraldpay.grpc.Chain import io.emeraldpay.grpc.Chain
@@ -38,7 +38,7 @@ import java.util.concurrent.CompletableFuture
class EthereumGrpcUpstreamSpec extends Specification { class EthereumGrpcUpstreamSpec extends Specification {
MockServer mockServer = new MockServer() MockGrpcServer mockServer = new MockGrpcServer()
ObjectMapper objectMapper = TestingCommons.objectMapper() ObjectMapper objectMapper = TestingCommons.objectMapper()
def "Subscribe to head"() { def "Subscribe to head"() {

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,434 @@
[
{
"txid": "e0f946c8f971b25cdffa64eed71d886019e437c0bf6a1b280584c0be5d1b5409",
"vout": 29,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01230030,
"confirmations": 2010,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "66e1e4d14ed6f454d2fda036f35cba423274ecdf5d46deb93f172c412a0f650d",
"vout": 83,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00756339,
"confirmations": 4963,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "4b30a92f5567dfbbd13b48e51c4ec1d97610daa8a1e008c428beb136e5f5670d",
"vout": 25,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00955798,
"confirmations": 111,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "507cb7191a10eaab9e2b4e223c85ef1ddd6b3aafbdfb990fc6e5e2dadf12af0e",
"vout": 10,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00289985,
"confirmations": 4840,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "7edc6cf0b680b4e127997c4b5d57320fda314d61b1ed23227cf496610e363410",
"vout": 54,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01003612,
"confirmations": 5213,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "0ae8671a15775d8eac990c92b687175a6b154fdcc4f516b0f100f813b2c9d011",
"vout": 93,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00973772,
"confirmations": 1213,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "f91b5bfd4a104baafeb0c5a62f1713580ab2d64b2c6d2fdfcd7818747645fc14",
"vout": 19,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01218703,
"confirmations": 278,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "7f9ee14424c4b52a2aa5c2b5df199fde2873a56f6ebd07d835722d2fd881b828",
"vout": 94,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00635688,
"confirmations": 3214,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6a278d265ce25c8519ad7af4b115eb8d6d5ca89957d0638f4567c2629c11d82a",
"vout": 48,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00529192,
"confirmations": 2472,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "80c673330e0c4a3b981f9acad07b5194b44c36d683cf8e0687f5faf6780a682c",
"vout": 14,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00850915,
"confirmations": 3066,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "a032d978409dbd6de66792a78147135903f1a4731781e8e5069a504cc43a8c32",
"vout": 8,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01047700,
"confirmations": 4628,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6dffd0651ae1bc29f43b864c1a097d4ecca0bd16341d13be9e32b121e6f6ca3e",
"vout": 36,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00605451,
"confirmations": 2763,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "42e10b12c32f4d6ba91b144b9c5afcccd6bcaf3f8f946d1f81a891c3d47e4346",
"vout": 78,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00982508,
"confirmations": 3363,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "70d83eda5f5024a5a7d2de7753ea0a7aa3a7cc5496ebbf9ea044efa2cd35cd47",
"vout": 55,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00625446,
"confirmations": 5605,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "253351e033465057d96e54a4fff130853782ae52dddef51486d3d7dc55e07b4c",
"vout": 74,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00720758,
"confirmations": 1539,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "39ad1738af322b6049e386a01e05fe8a6479913bd266268634eeadefdfd28c61",
"vout": 14,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01023716,
"confirmations": 4158,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "67c9e726a7acc9e7968975b150429e9d593f8e20010c03aeee58b84d2bce2b64",
"vout": 8,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01104307,
"confirmations": 889,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "2e731be5a74b5a321055a8b1ebc232c9316b275315ceaa945bad021c6e2a1865",
"vout": 47,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00679256,
"confirmations": 5081,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "88d48dae117aa8de4af3c8cde5f4d57d0e7a463d769b74379ea920474c6b527b",
"vout": 14,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00876837,
"confirmations": 730,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "c7c47da9165734c7745c13450fd61ec50c6638d1d6e58546f368c0acbe73877e",
"vout": 63,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01034200,
"confirmations": 2904,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "fe123697bc5f4ad079e81cdccbbb5834c240072f2874c8a43c68456dd8ef7a84",
"vout": 49,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00792699,
"confirmations": 4724,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6df879ac8f8f2b6fef79b71098fff03cc74085392fb2bbd4b77c14bafdefc497",
"vout": 35,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01313314,
"confirmations": 1378,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "05b2a9ae5e2d150ef042af1fe62a67652298b885d0fbe11f1be4ce850fcecfa6",
"vout": 60,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00754670,
"confirmations": 4404,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "c9590ed1299e22c2e4e1275a953258c9cb7052f87f5e7de7decb055248a06eba",
"vout": 8,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00794840,
"confirmations": 1068,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6d6dd53d70fa024011ca1a887686d6c0bbf19c739ed5017cad1adb77de7cbdbe",
"vout": 64,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01151435,
"confirmations": 3497,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "31ecd41422208033526919fee8a028a1de915da1308728ff8328a53f7f7d59cd",
"vout": 66,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00987020,
"confirmations": 2608,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "5963e589736bea2b4a438cf7cede084fbb24f9b545fa7558ad407510260805ce",
"vout": 68,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01076674,
"confirmations": 3915,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "df6a74ad3d4af01ad76937b43ac268d9775d447c97f5cc34c2b12c3b405e61cf",
"vout": 50,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00493433,
"confirmations": 4518,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "768d3e2f525d8ed24c446463f163fb9b9c7af10e07097efe0eecc8600b7083d6",
"vout": 74,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01433360,
"confirmations": 1696,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "f0eb2c2e854ecae7d3bbbdf06ef25de25c78983466be16650c030be3c558ebd6",
"vout": 58,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01098392,
"confirmations": 405,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "a73edf99d17868fd458520308febc71609e1fecd686530c525000359866a4ed9",
"vout": 35,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00902924,
"confirmations": 4037,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "aea6c1f4b231ab6900332c82fb9d48d24aac2245575e53f5fd7d6e86a7ea02dc",
"vout": 4,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00954847,
"confirmations": 563,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "366166e6a1a2d7b99d44834e2d4c2c5988fb8da2e17756f21b8e4d2c0ac972e1",
"vout": 88,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01032596,
"confirmations": 4275,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "ac53f2a2052396354333e0e500864f21ac0909c4159f46880e0f9128ac7194e2",
"vout": 19,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01480214,
"confirmations": 1848,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "63b840a8abf477305dfad93db69205ff01a58e1527a317dbcc8a360533c0e9e2",
"vout": 60,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00412783,
"confirmations": 3822,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "f14b222e652c58d11435fa9172ddea000c6f5e20e6b715eb940fc28d1c4adeef",
"vout": 58,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01105047,
"confirmations": 2139,
"spendable": false,
"solvable": false,
"safe": true
}
]

File diff suppressed because it is too large Load Diff