StakeSquid/dshackle
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #5 from emeraldpay/feat/bitcoin

Browse Source 
Bitcoin support
This commit is contained in:
Igor Artamonov
2020-04-27 21:03:39 -04:00
committed by GitHub
parent 51b25ab55d cfcb0bffff
commit 7dd90d2783
77 changed files with 9098 additions and 398 deletions
Download Patch File Download Diff File Expand all files Collapse all files

85
README.adoc
View File

@@ -9,10 +9,11 @@ image:https://img.shields.io/docker/pulls/emeraldpay/dshackle?style=flat-square[
image:https://img.shields.io/github/license/emeraldpay/dshackle.svg?style=flat-square&maxAge=2592000["License",link="https://github.com/emeraldpay/dshackle/blob/master/LICENSE"]
image:https://badges.gitter.im/emeraldpay/community.svg[link="https://gitter.im/emeraldpay/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge]
_Dshackle is an L7 Blockchain API Load Balancer._ It provided a high level aggregated API on top of several underlying upstreams, i.e., blockchain nodes or providers, such as Geth, Parity, Infura, etc.
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or sent data successfully broadcasted to the network.
_Dshackle is an L7 Blockchain API Load Balancer._ It provides a high level aggregated API on top of several underlying upstreams.
I.e. on top of blockchain nodes such as Bitcoind, Geth, Parity, or providers like Infura, and so on.
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or data successfully broadcasted to the network.
- Standard Ethereum JSON RPC API, plus advanced gRPC-based API
- Standard Bitcoin and Ethereum JSON RPC API, plus enhanced gRPC-based API
- **Secure** TLS with optional client authentication
- Blockchain-aware **caching** in memory and in Redis
- Routing based on **data availability** (peers, height, sync status)
@@ -22,6 +23,13 @@ It automatically verifies their availability and the current status of the netwo
Dshackle allows to build a mesh network of interconnected Dshackle servers for building blockchain based services that needs to have fast, secure, stable and fail-proof access to blockchain APIs.
Blockchain support:
- Ethereum, Ethereum Classic and Kovan testnet
- Bitcoin and Bitcoin Testnet
WARNING: The project is still under development, please use with caution.
Dshackle connects to several upstreams via JSON RPC, Websockets, or gRPC protocol.
The server verifies if a node ("upstream") is fully synchronized (not in initial sync mode), has enough peers, and its height is not behind other nodes.
If upstream lags behind others, lost peers below required, started to resync or went down, then Dshackle temporarily excludes it from requests and returns it when the upstream problem is fixed.
@@ -30,7 +38,6 @@ image::dshackle-intro.png[alt="",width=80%,align="center"]
== Roadmap
- [ ] Support Bitcoin RPC
- [ ] External logging
- [ ] Access to ERC-20 tokens on asset level
- [ ] Subscription to bitcoind notification over gRPC (instead of ZeroMQ)
@@ -58,8 +65,8 @@ proxy:
routes:
- id: eth
blockchain: ethereum
- id: kovan
blockchain: kovan
- id: btc
blockchain: bitcoin
cluster:
upstreams:
@@ -71,12 +78,15 @@ cluster:
url: "https://mainnet.infura.io/v3/${INFURA_USER}"
ws:
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
- id: infura-kovan
chain: kovan
- id: bitcoin-main
chain: bitcoin
connection:
ethereum:
bitcoin:
rpc:
url: "https://kovan.infura.io/v3/${INFURA_USER}"
url: "http://localhost:8332"
basic-auth:
username: bitcoin
password: mypassword
----
Which sets the following:
@@ -85,11 +95,11 @@ Which sets the following:
** TLS security is disabled (_please don't use in production!_)
- JSON RPC access through 0.0.0.0:8545
** proxy requests to Ethereum and Kovan upstreams
** request path for Ethereum Mainnet is `/eth`, for Kovan is `/kovan`
** i.e. call Mainnet by `POST http://127.0.0.0:8545/eth` with JSON RPC payload
** request path for Ethereum Mainnet is `/eth`, for bitcoin is `/btc`
** i.e. call Ethereum Mainnet by `POST http://127.0.0.0:8545/eth` with JSON RPC payload
- two upstreams, one for Ethereum Mainnet and another for Kovan Testnet (both upstreams are configured to use Infura endpoint)
- for Ethereum Mainnet it connects using JSON RPC and Websockets connections, for Kovan only JSON RPC is used
- Infura authentication config is omitted for this demo
- for Ethereum Mainnet it connects using JSON RPC and Websockets connections,
- for Bitcoin Mainet only JSON RPC is used
- `${INFURA_USER}` will be provided through environment variable
@@ -137,15 +147,20 @@ curl --request POST \
==== Access using gRPC
NOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.
Dshackle provides a custom gRPC based API, which provides additional methods and other features such as streaming responses.
Please refer to the documentation: link:docs/06-methods.adoc[gRPC Methods]
.Connect and listen for new blocks on Ethereum Mainnet
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 io.emeraldpay.api.Blockchain/SubscribeHead
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead
----
`type: 100` specifies the blockchain id, and 100 means Ethereum Mainnet. `1` is for Bitcoin Mainnet.
There we use Ethereum because it creates new blocks every 14 seconds, which works better for demo purposes, but the same request applied to Bitcoin as well.
.Output would be like
----
{
@@ -166,8 +181,42 @@ grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plai
...
----
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet.
It's one of services provided by Dshackle, in additional to standard methods provided by RPC JSON of underlying nodes.
The output above is for a _streaming subscription_ to all new blocks on the Ethereum Mainnet.
It's one of the services provided by Dshackle, in addition to standard methods provided by RPC JSON of underlying nodes.
.You can also subscribe to balances changes of the balance on an address:
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d '{"asset": {"chain": "100", "code": "ether"}, "address": {"address_single": {"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}}}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeBalance
----
.and see how balance of the contract responsible for Wrapped Ether is changing:
----
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410941696896999943701015"
}
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410930748488073834320430"
}
...
----
See other enhanced methods in the link:docs/06-methods.adoc[Documentation for Enhanced Methods]
== Documentation
@@ -214,6 +263,8 @@ See more in the documentation for link:docs/10-client-libraries.adoc[Client Libr
== Development
WARNING: The code in `master` branch is considered a development version, which may lack proper testing and should not be used in production.
=== Setting up environment
Dshackle is JVM based project written in Kotlin.

6
build.gradle
View File

@@ -84,6 +84,7 @@ dependencies {
compile "io.infinitape:etherjar-rpc-ws:$etherjarVersion"
compile "io.infinitape:etherjar-rpc-emerald:$etherjarVersion"
compile "io.infinitape:etherjar-tx:$etherjarVersion"
compile 'org.bitcoinj:bitcoinj-core:0.15.8'
compile 'org.apache.httpcomponents:httpmime:4.5.8'
compile 'org.apache.httpcomponents:httpclient:4.5.8'
@@ -106,11 +107,12 @@ dependencies {
testCompile "org.codehaus.groovy:groovy:$groovyVersion"
testCompile 'cglib:cglib-nodep:3.2.12'
testCompile 'cglib:cglib-nodep:3.3.0'
testCompile "org.spockframework:spock-core:$spockVersion"
testCompile "io.grpc:grpc-testing:${grpcVersion}"
testCompile "io.projectreactor:reactor-test:$reactorVersion"
testCompile 'org.objenesis:objenesis:3.0.1'
testCompile 'org.objenesis:objenesis:3.1'
testCompile 'org.mock-server:mockserver-netty:5.10'
}
compileKotlin {

9
docs/00-intro.adoc
View File

@@ -3,14 +3,13 @@
Dshackle is an L7 Blockchain API Load Balancer with automatic discovery and health checking, authentication, and TLS termination.
It is designed to work as an edge proxy, middle proxy, or API gateway.
Dshackle provided a high level aggregated API on top of several underlying upstreams (blockchain nodes or providers, such
as Geth, Parity, Infura, etc), it automatically verifies their availability and the current status of the network, executes
commands making sure that the response is consistent and/or data successfully broadcasted to the network.
Dshackle provides a high level aggregated API on top of several underlying upstreams.
I.e. to blockchain nodes such as Bitcoind, Geth, Parity, or providers like Infura, and so on.
It automatically verifies their availability and the current status of the network, executes commands making sure that the response is consistent and/or data successfully broadcasted to the network.
Example use cases:
- Read a _transaction_ - Dshackle tries to find it on different nodes and retries until it's found or there is a
consistent answer from upstreams that the transaction doesn't exist
- Read a _transaction_ - Dshackle tries to find it on different nodes and retries until it's found or there is a consistent answer from upstreams that the transaction doesn't exist
- Get _nonce_ to send a transaction - Dshackle makes sure it finds the highest value across several nodes
- Send _transaction_ - Dshackle distributes it to several nodes in parallel

64
docs/02-quick-start.adoc
View File

@@ -36,15 +36,18 @@ cluster:
connection:
ethereum:
rpc: # <6>
url: "https://mainnet.infura.io/v3/${INFURA_USER}"
ws: # <7>
url: "https://mainnet.infura.io/v3/${INFURA_USER}" <7>
ws: # <8>
url: "wss://mainnet.infura.io/ws/v3/${INFURA_USER}"
- id: infura-kovan
chain: kovan # <8>
- id: bitcoin-main
chain: bitcoin # <9>
connection:
ethereum:
bitcoin:
rpc:
url: "https://kovan.infura.io/v3/${INFURA_USER}" # <9>
url: "http://localhost:8332"
basic-auth: # <10>
username: bitcoin
password: test
----
<1> application listen for gRPC connections on 0.0.0.0:2449
<2> with TLS security for gRPC disabled (_never use in production!_)
@@ -52,12 +55,12 @@ cluster:
<4> sets up 2 upstreams
<5> one for Ethereum Mainnet, using
<6> HTTP and
<7> Websocket conection
<8> and another for Kovan Testnet
<9> `${INFURA_USER}` value is provided through environment variable
<7> `${INFURA_USER}` value is provided through environment variable
<8> Websocket conection
<9> and another for Bitcoin <10> with Basic Auth for bitcoin node connection
Both upstreams are configured for Infura for demo purposes, but you can use other compatible endpoints.
Infura authentication is omitted for this demo
See detailed link:reference-configuration.adoc[Configuration Reference]
==== Run as docker
@@ -99,9 +102,11 @@ curl --request POST \
.Connect and listen for new blocks on Ethereum Mainnet
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 io.emeraldpay.api.Blockchain/SubscribeHead
grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeHead
----
`type: 100` specifies the blockchain id, and 100 means Ethereum Mainnet.
.Output would be like
----
{
@@ -123,4 +128,37 @@ grpcurl -import-path ./proto/ -proto blockchain.proto -d "{\"type\": 100}" -plai
----
The output above is for a _streaming subscription_ to all new blocks on Ethereum Mainnet.
It's a method provided by Dshackle, available in additional to methods provided by RPC JSON of underlying nodes.
It's one of the services provided by Dshackle, in addition to standard methods provided by RPC JSON of underlying nodes.
.You can also subscribe to balances changes of the balance on an address:
[source,bash]
----
grpcurl -import-path ./proto/ -proto blockchain.proto -d '{"asset": {"chain": "100", "code": "ether"}, "address": {"address_single": {"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"}}}' -plaintext 127.0.0.1:2449 emerald.Blockchain/SubscribeBalance
----
.and see how balance of the contract responsible for Wrapped Ether is changing:
----
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410941696896999943701015"
}
{
"asset": {
"chain": "CHAIN_ETHEREUM",
"code": "ETHER"
},
"address": {
"address": "0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2"
},
"balance": "2410930748488073834320430"
}
...
----
See other enhanced methods in the link:06-methods.adoc[Documentation for Enhanced Methods]

40
docs/04-upstream-config.adoc
View File

@@ -10,7 +10,22 @@ Supported upstream protocols:
Those protocols can be configures with additional security, TLS and authentication.
=== Example
=== Notes on upstream configuration
==== Ethereum
- Most of Ethereum nodes support WebSocket connection, in addition to the JSON RPC.
If it's available on your node, it's suggested to configure both JSON RPC and WebSocket connection
==== Bitcoin
- Bitcoind needs to be configured to index/track addresses that you're going to request.
Make sure you configure it to index your addresses with `importaddress`.
If you request balance for an address that is not indexed then it returns 0 balance.
- To track all transactions you need to setup index for transactions, which is disabled by default.
Run it with `-reindex` option, or set `txindex=1` in the config.
=== Example Configuration
.upstreams.yaml
[source,yaml]
@@ -104,9 +119,28 @@ Dshackle currently supports
- `ws` websocket connection (supposed to be used in addition to `rpc` connection)
- `grpc` connects to another Dshackle instance
=== Methods
=== Bitcoin Methods
.By default an ethereum upstream supports following JSON RPC methods:
.By default an ethereum upstream allows call to the following JSON RPC methods:
- `getbestblockhash`
- `getblock`
- `getblocknumber`
- `getblockcount`
- `gettransaction`
- `getrawtransaction`
- `gettxout`
- `getreceivedbyaddress`
- `listunspent`
- `sendrawtransaction`
.Plus following methods are answered directly by Dshackle
- `getmemorypool`
- `getconnectioncount`
- `getnetworkinfo`
=== Ethereum Methods
.By default an ethereum upstream allows calls to the following JSON RPC methods:
- `eth_gasPrice`
- `eth_call`
- `eth_estimateGas`

32
docs/06-methods.adoc
View File

@@ -1,9 +1,12 @@
== Methods
== Enhanced Methods
IMPORTANT: Dshackle provides an unified API based on gRPC and Protobuf.
IMPORTANT: Dshackle provides an enhanced API based on gRPC and Protobuf, in addition to JSON RPC proxy
Dshackle provides an unified API based on gRPC and Protobuf. I.e. it's not standard JSON RPC but client libraries could
be generated for all major libraries, and there're official libraries for Java and Javascript.
Dshackle provides an enhanced and unified API based on HTTP2, gRPC and Protobuf.
It works in addition to JSON RPC proxy and can be used separately.
Clients can be generated for all major programming languages, and there're official libraries for Java and Javascript.
NOTE: It's not necessary to use gRPC, as Dshackle can provide standard JSON RPC proxy, but Dshackle gRPC interface improves performance and provides additional features.
=== gRPC definition
@@ -25,8 +28,7 @@ service Blockchain {
=== Wrapped JSON RPC methods
Standard JSON RPC methods provided by Ethereum nodes are available wrapped into gRPC/Protobuf with additional features.
To call standard methods you use `NativeCall` method.
To call standard JSON RPC methods provided by Ethereum/Bitcoin you use `NativeCall` wrapping method, which provides additional flexibility and configuration for the calls.
.NativeCallRequest
[source,proto]
@@ -72,13 +74,13 @@ Where:
- `payload` is JSON response for a particular call, encoded into a string (when `succeed` is true)
- or `error` if request failed (`succeed` is false)
NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response. It allows to build
non-blocking queries
NOTE: Reply Items comes right after their execution on an upstream, therefore streaming response.
It allows to build non-blocking queries
=== SubscribeHead
Subscribes to new blocks on the specified chain. Returns stream of blocks right after it was accepted (and verified by
Dshackle) by any of the upstreams.
This methods provides subscription to the new blocks on the specified chain.
Returns stream of blocks right after it was accepted (and verified by Dshackle) by any of the upstreams.
.ChainHead
[source,proto]
@@ -104,7 +106,7 @@ Where:
=== SubscribeBalance or GetBalance
Subscribes to changes of the balance for a single address or a set of addresses
Subscribes to changes (`SubscribeBalance`) or get current (`GetBalance`) balance for a single address or a set of addresses.
.Request
[source,proto]
@@ -136,8 +138,9 @@ message AddressBalance {
==== SubscribeTxStatus
Subscribes to transaction confirmations. Allows to send a transactions and then listen to all changes until it
gets enough confirmations. Changes is `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED`
Subscribes to transaction confirmations.
Allows to send a transactions and then listen to all changes until it gets enough confirmations.
Changes are `NOTFOUND -> BROADCASTED <- -> MINED <- -> CONFIRMED`
.Request
[source,proto]
@@ -161,3 +164,6 @@ message TxStatus {
}
----
=== gRPC Client Libraries
See link:10-client-libraries.adoc[Client Libraries] documentation.

17
docs/reference-configuration.adoc
View File

@@ -89,6 +89,15 @@ cluster:
basic-auth:
username: 4fc258fe41a68149c199ad8f281f2015
password: 1a68f20154fc258fe4149c199ad8f281
- id: bitcoin
chain: bitcoin
connection:
bitcoin:
rpc:
url: "http://localhost:8332"
basic-auth:
username: bitcoin
password: e984af45bb888428207c290
- id: remote
connection:
grpc:
@@ -357,7 +366,7 @@ configuration, and may be omitted for most of the situations.
| `chain`
| yes
| Blockchain which is the provided by the upstream. `ethereum`, `ethereum_classic`, `kovan`
| Blockchain which is the provided by the upstream. `bitcoin`, `bitcoin-testnet`, `ethereum`, `ethereum-classic`, or `kovan`
| `labels`
| no
@@ -369,7 +378,11 @@ configuration, and may be omitted for most of the situations.
| `connection.ethereum`
| yes
| Connection configuration for Ethereum JSON RPC API
| Connection configuration for Ethereum API
| `connection.bitcoin`
| yes
| Connection configuration for Bitcoin API
|===

2
gradle.properties
View File

@@ -18,5 +18,5 @@ reactorVersion=3.3.3.RELEASE
etherjarVersion=0.9.1
# Testing
spockVersion=1.2-groovy-2.5
spockVersion=1.3-groovy-2.5

5
src/main/kotlin/io/emeraldpay/dshackle/config/UpstreamsConfig.kt
View File

@@ -96,6 +96,10 @@ class UpstreamsConfig {
var ws: WsEndpoint? = null
}
class BitcoinConnection : UpstreamConnection() {
var rpc: HttpEndpoint? = null
}
class HttpEndpoint(val url: URI) {
var basicAuth: AuthConfig.ClientBasicAuth? = null
var tls: AuthConfig.ClientTlsAuth? = null
@@ -124,6 +128,7 @@ class UpstreamsConfig {
enum class UpstreamType private constructor(vararg code: String) {
ETHEREUM_JSON_RPC("ethereum"),
BITCOIN_JSON_RPC("bitcoin"),
DSHACKLE("dshackle", "grpc"),
UNKNOWN("unknown");

26
src/main/kotlin/io/emeraldpay/dshackle/config/UpstreamsConfigReader.kt
View File

@@ -87,7 +87,7 @@ class UpstreamsConfigReader(
val connConfigNode = getMapping(connNode, "ethereum")!!
val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>()
readUpstreamCommon(upNode, upstream)
readUpstreamEthereum(upNode, upstream)
readUpstreamStandard(upNode, upstream)
if (isValid(upstream)) {
config.upstreams.add(upstream)
val connection = UpstreamsConfig.EthereumConnection()
@@ -113,6 +113,26 @@ class UpstreamsConfigReader(
} else {
log.error("Upstream at #0 has invalid configuration")
}
} else if (hasAny(connNode, "bitcoin")) {
val connConfigNode = getMapping(connNode, "bitcoin")!!
val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.BitcoinConnection>()
readUpstreamCommon(upNode, upstream)
readUpstreamStandard(upNode, upstream)
if (isValid(upstream)) {
config.upstreams.add(upstream)
val connection = UpstreamsConfig.BitcoinConnection()
upstream.connection = connection
getMapping(connConfigNode, "rpc")?.let { node ->
getValueAsString(node, "url")?.let { url ->
val http = UpstreamsConfig.HttpEndpoint(URI(url))
connection.rpc = http
http.basicAuth = authConfigReader.readClientBasicAuth(node)
http.tls = authConfigReader.readClientTls(node)
}
}
} else {
log.error("Upstream at #0 has invalid configuration")
}
} else if (hasAny(connNode, "grpc")) {
val connConfigNode = getMapping(connNode, "grpc")!!
val upstream = UpstreamsConfig.Upstream<UpstreamsConfig.GrpcConnection>()
@@ -162,13 +182,13 @@ class UpstreamsConfigReader(
}
}
internal fun readUpstreamEthereum(upNode: MappingNode, upstream: UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>) {
internal fun readUpstreamStandard(upNode: MappingNode, upstream: UpstreamsConfig.Upstream<*>) {
upstream.chain = getValueAsString(upNode, "chain")
if (hasAny(upNode, "labels")) {
getMapping(upNode, "labels")?.let { labels ->
labels.value.stream()
.filter { n -> n.keyNode is ScalarNode && n.valueNode is ScalarNode }
.map { n -> Tuples.of((n.keyNode as ScalarNode).value, (n.valueNode as ScalarNode).value)}
.map { n -> Tuples.of((n.keyNode as ScalarNode).value, (n.valueNode as ScalarNode).value) }
.map { kv -> Tuples.of(kv.t1.trim(), kv.t2.trim()) }
.filter { kv -> StringUtils.isNotEmpty(kv.t1) && StringUtils.isNotEmpty(kv.t2) }
.forEach { kv ->

12
src/main/kotlin/io/emeraldpay/dshackle/data/BlockId.kt
View File

@@ -15,7 +15,9 @@
*/
package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.BlockHash
import io.infinitape.etherjar.rpc.json.BlockJson
import org.bouncycastle.util.encoders.Hex
class BlockId(
value: ByteArray
@@ -23,7 +25,7 @@ class BlockId(
companion object {
@JvmStatic
fun from(hash: io.infinitape.etherjar.domain.BlockHash): BlockId {
fun from(hash: BlockHash): BlockId {
return BlockId(hash.bytes)
}
@@ -34,7 +36,13 @@ class BlockId(
@JvmStatic
fun from(id: String): BlockId {
return from(io.infinitape.etherjar.domain.BlockHash.from(id))
val clean = if (id.startsWith("0x")) {
id.substring(2)
} else {
id
}
val bytes = Hex.decode(clean)
return BlockId(bytes)
}
}

6
src/main/kotlin/io/emeraldpay/dshackle/data/HashId.kt
View File

@@ -28,11 +28,9 @@ open class HashId(
}
fun toHex(): String {
val hex = CharArray(value.size * 2 + 2)
hex[0] = '0'
hex[1] = 'x'
val hex = CharArray(value.size * 2)
var i = 0
var j = 2
var j = 0
while (i < value.size) {
hex[j++] = HEX_DIGITS[0xF0 and value[i].toInt() ushr 4]
hex[j++] = HEX_DIGITS[0x0F and value[i].toInt()]

10
src/main/kotlin/io/emeraldpay/dshackle/data/TxId.kt
View File

@@ -17,6 +17,8 @@ package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.TransactionId
import io.infinitape.etherjar.rpc.json.TransactionJson
import org.bouncycastle.util.encoders.Hex
import java.math.BigInteger
class TxId(
value: ByteArray
@@ -35,7 +37,13 @@ class TxId(
@JvmStatic
fun from(id: String): TxId {
return from(TransactionId.from(id))
val clean = if (id.startsWith("0x")) {
id.substring(2)
} else {
id
}
val bytes = Hex.decode(clean)
return TxId(bytes)
}
}
}

27
src/main/kotlin/io/emeraldpay/dshackle/rpc/BlockchainRpc.kt
View File

@@ -19,6 +19,7 @@ import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.api.proto.ReactorBlockchainGrpc
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
@@ -30,8 +31,8 @@ import reactor.core.publisher.Mono
class BlockchainRpc(
@Autowired private val nativeCall: NativeCall,
@Autowired private val streamHead: StreamHead,
@Autowired private val trackEthereumTx: TrackEthereumTx,
@Autowired private val trackEthereumAddress: TrackEthereumAddress,
@Autowired private val trackTx: List<TrackTx>,
@Autowired private val trackAddress: List<TrackAddress>,
@Autowired private val describe: Describe,
@Autowired private val subscribeStatus: SubscribeStatus
): ReactorBlockchainGrpc.BlockchainImplBase() {
@@ -47,15 +48,27 @@ class BlockchainRpc(
}
override fun subscribeTxStatus(request: Mono<BlockchainOuterClass.TxStatusRequest>): Flux<BlockchainOuterClass.TxStatus> {
return trackEthereumTx.add(request)
return request.flatMapMany { request ->
val chain = Chain.byId(request.chainValue)
trackTx.find { it.isSupported(chain) }?.subscribe(request)
?: Flux.error(SilentException.UnsupportedBlockchain(chain))
}
}
override fun subscribeBalance(request: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return trackEthereumAddress.subscribe(request)
override fun subscribeBalance(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return requestMono.flatMapMany { request ->
val chain = Chain.byId(request.asset.chainValue)
trackAddress.find { it.isSupported(chain) }?.subscribe(request)
?: Flux.error(SilentException.UnsupportedBlockchain(chain))
}
}
override fun getBalance(request: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return trackEthereumAddress.getBalance(request)
override fun getBalance(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return requestMono.flatMapMany { request ->
val chain = Chain.byId(request.asset.chainValue)
trackAddress.find { it.isSupported(chain) }?.getBalance(request)
?: Flux.error(SilentException.UnsupportedBlockchain(chain))
}
}
override fun describe(request: Mono<BlockchainOuterClass.DescribeRequest>): Mono<BlockchainOuterClass.DescribeResponse> {

3
src/main/kotlin/io/emeraldpay/dshackle/rpc/Describe.kt
View File

@@ -19,6 +19,7 @@ import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.startup.QuorumForLabels
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.dshackle.upstream.grpc.EthereumGrpcUpstream
import org.springframework.beans.factory.annotation.Autowired
@@ -47,6 +48,8 @@ class Describe(
val nodes = QuorumForLabels()
if (up is EthereumUpstream) {
nodes.add(up.node)
} else if (up is BitcoinUpstream) {
nodes.add(up.node)
} else if (up is EthereumGrpcUpstream) {
nodes.add(up.getNodes())
}

16
src/main/kotlin/io/emeraldpay/dshackle/rpc/NativeCall.kt
View File

@@ -23,11 +23,8 @@ import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.quorum.AlwaysQuorum
import io.emeraldpay.dshackle.quorum.CallQuorum
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.RpcException
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.apache.commons.lang3.StringUtils
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
@@ -88,17 +85,18 @@ open class NativeCall(
if (chain == Chain.UNSPECIFIED) {
return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(request.chain.number)))
}
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
if (!upstreams.isAvailable(chain)) {
return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(request.chain.number)))
}
val upstream = upstreams.getUpstream(chain)
?: return Flux.error(CallFailure(0, SilentException.UnsupportedBlockchain(chain)))
return prepareCall(request, upstream as AggregatedUpstream<EthereumApi>)
return prepareCall(request, upstream)
}
fun prepareCall(request: BlockchainOuterClass.NativeCallRequest, upstream: AggregatedUpstream<EthereumApi>): Flux<CallContext<RawCallDetails>> {
fun prepareCall(request: BlockchainOuterClass.NativeCallRequest, upstream: AggregatedUpstream<*>): Flux<CallContext<RawCallDetails>> {
return request.itemsList.toFlux().map {
val method = it.method
val params = it.payload.toStringUtf8()
@@ -137,7 +135,7 @@ open class NativeCall(
var failures = 0
return Flux.from(apis)
.flatMap { api ->
val upstream = api.upstream!!
val upstream = ctx.upstream
api.execute(ctx.id, ctx.payload.method, ctx.payload.params)
// on error notify quorum, it may use error message or other details
.doOnError { err ->
@@ -205,7 +203,7 @@ open class NativeCall(
}
open class CallContext<T>(val id: Int,
val upstream: AggregatedUpstream<EthereumApi>,
val upstream: AggregatedUpstream<*>,
val matcher: Selector.Matcher,
val callQuorum: CallQuorum,
val payload: T) {
@@ -213,7 +211,7 @@ open class NativeCall(
return CallContext(id, upstream, matcher, callQuorum, payload)
}
fun getApis(): ApiSource<EthereumApi> {
fun getApis(): ApiSource<*> {
return upstream.getApis(matcher)
}
}

2
src/main/kotlin/io/emeraldpay/dshackle/rpc/StreamHead.kt
View File

@@ -65,7 +65,7 @@ class StreamHead(
.setHeight(block.height)
.setTimestamp(block.timestamp!!.toEpochMilli())
.setWeight(ByteString.copyFrom(block.difficulty.toByteArray()))
.setBlockId(block.hash.toHex().substring(2))
.setBlockId(block.hash.toHex())
.build()
}

17
src/main/kotlin/io/emeraldpay/dshackle/rpc/TrackAddress.kt Normal file
View File

@@ -0,0 +1,17 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
/**
* Base interface to tracking balance on a single blockchain
*/
interface TrackAddress {
fun isSupported(chain: Chain): Boolean
fun getBalance(requestMono: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance>
fun subscribe(requestMono: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance>
}

154
src/main/kotlin/io/emeraldpay/dshackle/rpc/TrackBitcoinAddress.kt Normal file
View File

@@ -0,0 +1,154 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.dshackle.upstream.Selector
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Service
import reactor.core.publisher.Flux
import reactor.util.function.Tuples
import java.math.BigDecimal
import java.math.BigInteger
import java.util.*
import kotlin.collections.HashMap
@Service
class TrackBitcoinAddress(
@Autowired private val upstreams: Upstreams
) : TrackAddress {
companion object {
private val log = LoggerFactory.getLogger(TrackBitcoinAddress::class.java)
}
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.BITCOIN && upstreams.isAvailable(chain)
}
fun allAddresses(request: BlockchainOuterClass.BalanceRequest): List<String>? {
if (!request.hasAddress()) {
return null
}
return when {
request.address.hasAddressSingle() -> {
listOf(request.address.addressSingle.address)
}
request.address.hasAddressMulti() -> {
request.address.addressMulti.addressesList
.map { addr -> addr.address }
.sorted()
}
else -> null
}
}
fun requestBalances(chain: Chain, api: DirectBitcoinApi, addresses: List<String>): Flux<AddressBalance> {
return api.executeAndResult(0, "listunspent", emptyList(), List::class.java)
.flatMapMany { unspents ->
val result = getTotal(chain, addresses, unspents)
Flux.fromIterable(result)
}
}
override fun getBalance(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
val chain = Chain.byId(request.asset.chainValue)
val upstream = upstreams.getUpstream(chain)?.castApi(DirectBitcoinApi::class.java)
?: return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
val addresses = allAddresses(request) ?: return Flux.error(SilentException("Unsupported address"))
if (addresses.isEmpty()) {
return Flux.empty()
}
val result = upstream.getApi(Selector.empty).flatMapMany { api ->
requestBalances(chain, api, addresses)
.map(this@TrackBitcoinAddress::buildResponse)
}
return result
}
fun getTotal(chain: Chain, addresses: List<String>, unspents: List<*>): List<AddressBalance> {
return unspents.asSequence()
.filterIsInstance<Map<String, Any>>()
.filter { unspent ->
unspent.containsKey("address")
&& Collections.binarySearch(addresses, unspent["address"] as String) >= 0
&& unspent.containsKey("amount")
}
.map {
Tuples.of(it["address"] as String, it["amount"] as Number)
}
.map {
AddressBalance(chain, it.t1,
//use toString because toDecimal makes rounding
BigDecimal(it.t2.toString()).multiply(BigDecimal.TEN.pow(8)).toBigInteger()
)
}
.plus(
//add default ZERO value
addresses.map {
AddressBalance(chain, it, BigInteger.ZERO)
}
)
.groupBy {
it.address
}
.map {
it.value.reduceRight { x, acc ->
x.plus(acc)
}
}.toList()
}
override fun subscribe(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
val chain = Chain.byId(request.asset.chainValue)
val upstream = upstreams.getUpstream(chain)?.castApi(DirectBitcoinApi::class.java)
?: return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
val addresses = allAddresses(request) ?: return Flux.error(SilentException("Unsupported address"))
if (addresses.isEmpty()) {
return Flux.empty()
}
val initial = upstream.getApi(Selector.empty).flatMapMany { api ->
requestBalances(chain, api, addresses)
}
val following = upstream.getHead().getFlux()
.flatMap { block ->
upstream.getApi(Selector.empty).flatMapMany { api ->
requestBalances(chain, api, addresses)
}
}
val last = HashMap<Address, BigInteger>()
val result = Flux.merge(initial, following)
.filter { curr ->
val prev = last[curr.address]
val updated = prev == null || curr.balance != prev
last[curr.address] = curr.balance
updated
}
return result.map(this@TrackBitcoinAddress::buildResponse)
}
private fun buildResponse(address: AddressBalance): BlockchainOuterClass.AddressBalance {
return BlockchainOuterClass.AddressBalance.newBuilder()
.setBalance(address.balance.toString(10))
.setAsset(Common.Asset.newBuilder()
.setChainValue(address.address.chain.id)
.setCode("BTC"))
.setAddress(Common.SingleAddress.newBuilder().setAddress(address.address.address))
.build()
}
open class AddressBalance(val address: Address, var balance: BigInteger = BigInteger.ZERO) {
constructor(chain: Chain, address: String, balance: BigInteger) : this(Address(chain, address), balance)
fun plus(other: AddressBalance) = AddressBalance(address, balance + other.balance)
}
data class Address(val chain: Chain, val address: String)
}

159
src/main/kotlin/io/emeraldpay/dshackle/rpc/TrackBitcoinTx.kt Normal file
View File

@@ -0,0 +1,159 @@
package io.emeraldpay.dshackle.rpc
import com.google.protobuf.ByteString
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.SilentException
import io.emeraldpay.dshackle.upstream.Selector
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.bitcoin.ExtractBlock
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Service
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import java.math.BigInteger
import java.time.Duration
import java.time.Instant
import kotlin.math.max
import kotlin.math.min
@Service
class TrackBitcoinTx(
@Autowired private val upstreams: Upstreams
) : TrackTx {
companion object {
private val log = LoggerFactory.getLogger(TrackBitcoinTx::class.java)
}
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.BITCOIN && upstreams.isAvailable(chain)
}
override fun subscribe(request: BlockchainOuterClass.TxStatusRequest): Flux<BlockchainOuterClass.TxStatus> {
val chain = Chain.byId(request.chainValue)
val upstream = upstreams.getUpstream(chain)?.cast(BitcoinUpstream::class.java, DirectBitcoinApi::class.java)
?: return Flux.error(SilentException.UnsupportedBlockchain(chain))
val txid = request.txId
val confirmations = max(min(1, request.confirmationLimit), 12)
return upstream.getApi(Selector.empty).flatMapMany { api ->
subscribe(chain, api, upstream, txid)
}.takeUntil { tx ->
tx.confirmations >= confirmations
}.map(this::asProto)
}
fun subscribe(chain: Chain, api: DirectBitcoinApi, upstream: BitcoinUpstream, txid: String): Flux<TxStatus> {
return loadExisting(api, txid)
.flatMapMany { status ->
if (status.mined) {
//Head almost always knows the current height, so it can continue with calculating confirmations
//without publishing an empty TxStatus first
continueWithMined(api, upstream, status)
} else {
loadMempool(upstream, txid)
.flatMapMany { tx ->
val next = if (tx.found) {
untilMined(upstream, tx)
} else {
untilFound(chain, api, upstream, txid)
}
//fist provide the current status, then updates
Flux.concat(Mono.just(tx), next)
}
}
}
}
fun continueWithMined(api: DirectBitcoinApi, upstream: BitcoinUpstream, status: TxStatus): Flux<TxStatus> {
return api.getBlock(status.blockHash!!)
.map { block ->
TxStatus(status.txid, true, ExtractBlock.getHeight(block), true, status.blockHash, ExtractBlock.getTime(block), ExtractBlock.getDifficulty(block))
}.flatMapMany { tx ->
withConfirmations(upstream, tx)
}
}
fun untilFound(chain: Chain, api: DirectBitcoinApi, upstream: BitcoinUpstream, txid: String): Flux<TxStatus> {
return Flux.interval(Duration.ofSeconds(1))
.take(Duration.ofMinutes(10))
.flatMap { loadMempool(upstream, txid) }
.skipUntil { it.found }
.flatMap { subscribe(chain, api, upstream, txid) }
.doOnError { t ->
log.error("Failed to wait until found", t)
}
}
fun untilMined(upstream: BitcoinUpstream, tx: TxStatus): Mono<TxStatus> {
return upstream.getHead().getFlux().flatMap {
upstream.getApi(Selector.empty).flatMap { api ->
loadExisting(api, tx.txid)
}.filter { it.mined }
}.single()
}
fun withConfirmations(upstream: BitcoinUpstream, tx: TxStatus): Flux<TxStatus> {
return upstream.getHead().getFlux().map {
tx.withHead(it.height)
}
}
fun loadExisting(api: DirectBitcoinApi, txid: String): Mono<TxStatus> {
val mined = api.getTx(txid)
return mined.map {
val block = it["blockhash"] as String?
TxStatus(txid, found = true, mined = block != null, blockHash = block, height = ExtractBlock.getHeight(it))
}
}
fun loadMempool(upstream: BitcoinUpstream, txid: String): Mono<TxStatus> {
println("access: ${upstream.getData()}")
val mempool = upstream.getData().getMempool().get()
return mempool.map {
if (it.contains(txid)) {
TxStatus(txid, found = true, mined = false)
} else {
TxStatus(txid, found = false, mined = false)
}
}
}
private fun asProto(tx: TxStatus): BlockchainOuterClass.TxStatus {
val data = BlockchainOuterClass.TxStatus.newBuilder()
.setTxId(tx.txid)
.setConfirmations(tx.confirmations.toInt())
data.broadcasted = tx.found
val isMined = tx.mined
data.mined = isMined
if (isMined) {
data.setBlock(
Common.BlockInfo.newBuilder()
.setBlockId(tx.blockHash!!.substring(2))
.setTimestamp(tx.blockTime!!.toEpochMilli())
.setWeight(ByteString.copyFrom(tx.blockTotalDifficulty!!.toByteArray()))
.setHeight(tx.height!!)
)
}
return data.build()
}
class TxStatus(
val txid: String,
val found: Boolean = false,
val height: Long? = null,
val mined: Boolean = false,
val blockHash: String? = null,
val blockTime: Instant? = null,
val blockTotalDifficulty: BigInteger? = null,
val confirmations: Long = 0) {
fun withHead(headHeight: Long) = TxStatus(txid, found, height, mined, blockHash, blockTime, blockTotalDifficulty, headHeight - height!! + 1)
}
}

65
src/main/kotlin/io/emeraldpay/dshackle/rpc/TrackEthereumAddress.kt
View File

@@ -51,7 +51,7 @@ import javax.annotation.PostConstruct
class TrackEthereumAddress(
@Autowired private val upstreams: Upstreams,
@Autowired private val upstreamScheduler: Scheduler
) {
) : TrackAddress {
private val log = LoggerFactory.getLogger(TrackEthereumAddress::class.java)
private val clients = HashMap<Chain, ConcurrentLinkedQueue<TrackedAddress>>()
@@ -83,6 +83,10 @@ class TrackEthereumAddress(
}
}
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.ETHEREUM && upstreams.isAvailable(chain)
}
private fun startTracking(client: TrackedAddress) {
clients[client.chain]?.add(client) ?: log.warn("Chain ${client.chain} is not available for tracking")
}
@@ -99,9 +103,6 @@ class TrackEthereumAddress(
private fun initializeSimple(request: BlockchainOuterClass.BalanceRequest): Flux<SimpleAddress> {
val chain = Chain.byId(request.asset.chainValue)
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
}
if (!upstreams.isAvailable(chain)) {
return Flux.error(SilentException.UnsupportedBlockchain(request.asset.chainValue))
}
@@ -128,43 +129,35 @@ class TrackEthereumAddress(
}
}
fun subscribe(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return requestMono.flatMapMany { request ->
val chain = Chain.byId(request.asset.chainValue)
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
return@flatMapMany Flux.error<BlockchainOuterClass.AddressBalance>(SilentException.UnsupportedBlockchain(request.asset.chainValue))
}
val bus = TopicProcessor.create<BlockchainOuterClass.AddressBalance>()
initializeSubscription(request, bus)
.flatMap { tracked ->
val current = getBalance(tracked).map {
tracked.withBalance(it)
}.doOnNext {
startTracking(it)
}.map {
buildResponse(it)
}
Flux.merge(current, bus).doFinally { stopTracking(tracked) }
override fun subscribe(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
val bus = TopicProcessor.create<BlockchainOuterClass.AddressBalance>()
return initializeSubscription(request, bus)
.flatMap { tracked ->
val current = getBalance(tracked).map {
tracked.withBalance(it)
}.doOnNext {
startTracking(it)
}.map {
buildResponse(it)
}
.doOnError { t ->
if (t is SilentException) {
if (t is SilentException.UnsupportedBlockchain) {
log.warn("Unsupported blockchain: ${t.blockchainId}")
}
log.debug("Failed to process subscription", t)
} else {
log.warn("Failed to process subscription", t)
Flux.merge(current, bus).doFinally { stopTracking(tracked) }
}
.doOnError { t ->
if (t is SilentException) {
if (t is SilentException.UnsupportedBlockchain) {
log.warn("Unsupported blockchain: ${t.blockchainId}")
}
log.debug("Failed to process subscription", t)
} else {
log.warn("Failed to process subscription", t)
}
}
}
}
fun getBalance(requestMono: Mono<BlockchainOuterClass.BalanceRequest>): Flux<BlockchainOuterClass.AddressBalance> {
return requestMono.flatMapMany { request ->
initializeSimple(request)
.flatMap { a -> getBalance(a).map { a.withBalance(it) } }
.map { buildResponse(it) }
}
override fun getBalance(request: BlockchainOuterClass.BalanceRequest): Flux<BlockchainOuterClass.AddressBalance> {
return initializeSimple(request)
.flatMap { a -> getBalance(a).map { a.withBalance(it) } }
.map { buildResponse(it) }
}
private fun simpleAddress(address: Common.SingleAddress, chain: Chain): SimpleAddress {

17
src/main/kotlin/io/emeraldpay/dshackle/rpc/TrackEthereumTx.kt
View File

@@ -58,7 +58,7 @@ import kotlin.math.min
class TrackEthereumTx(
@Autowired private val upstreams: Upstreams,
@Autowired private val upstreamScheduler: Scheduler
) {
) : TrackTx {
companion object {
private val ZERO_BLOCK = BlockHash.from("0x0000000000000000000000000000000000000000000000000000000000000000")
@@ -75,6 +75,10 @@ class TrackEthereumTx(
val notFound = ConcurrentLinkedQueue<TrackedTx>()
override fun isSupported(chain: Chain): Boolean {
return BlockchainType.fromBlockchain(chain) == BlockchainType.ETHEREUM && upstreams.isAvailable(chain)
}
@PostConstruct
fun init() {
upstreams.observeChains().subscribe { chain ->
@@ -165,7 +169,7 @@ class TrackEthereumTx(
fun streamAllUpdates(tx: TxDetails): Flux<BlockchainOuterClass.TxStatus> {
val current = checkForUpdate(tx)
.doOnNext (this::onFirstUpdate)
.doOnNext(this::onFirstUpdate)
.map { Notification(it, asProto(it)) }
val updates = Flux.from(tx.bus)
@@ -175,12 +179,9 @@ class TrackEthereumTx(
.map { it.proto }
}
fun add(requestMono: Mono<BlockchainOuterClass.TxStatusRequest>): Flux<BlockchainOuterClass.TxStatus> {
return requestMono.map { request ->
prepareTracking(request)
}.flatMapMany { tx ->
streamAllUpdates(tx)
}
override fun subscribe(request: BlockchainOuterClass.TxStatusRequest): Flux<BlockchainOuterClass.TxStatus> {
val tx = prepareTracking(request)
return streamAllUpdates(tx)
}
private fun verifyAll(chain: Chain) {

10
src/main/kotlin/io/emeraldpay/dshackle/rpc/TrackTx.kt Normal file
View File

@@ -0,0 +1,10 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
interface TrackTx {
fun isSupported(chain: Chain): Boolean
fun subscribe(request: BlockchainOuterClass.TxStatusRequest): Flux<BlockchainOuterClass.TxStatus>
}

70
src/main/kotlin/io/emeraldpay/dshackle/startup/ConfiguredUpstreams.kt
View File

@@ -20,6 +20,10 @@ import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.FileResolver
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.CurrentUpstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinRpcClient
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.calls.ManagedCallMethods
import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
@@ -32,6 +36,7 @@ import org.springframework.beans.factory.annotation.Autowired
import org.springframework.stereotype.Repository
import java.net.URI
import java.util.*
import java.util.concurrent.atomic.AtomicInteger
import javax.annotation.PostConstruct
import kotlin.collections.HashMap
@@ -44,6 +49,7 @@ open class ConfiguredUpstreams(
) {
private val log = LoggerFactory.getLogger(ConfiguredUpstreams::class.java)
private var seq = AtomicInteger(0)
private val chainNames = mapOf(
"ethereum" to Chain.ETHEREUM,
@@ -51,7 +57,10 @@ open class ConfiguredUpstreams(
"eth" to Chain.ETHEREUM,
"etc" to Chain.ETHEREUM_CLASSIC,
"morden" to Chain.TESTNET_MORDEN,
"kovan" to Chain.TESTNET_KOVAN
"kovan" to Chain.TESTNET_KOVAN,
"kovan-testnet" to Chain.TESTNET_KOVAN,
"bitcoin" to Chain.BITCOIN,
"bitcoin-testnet" to Chain.TESTNET_BITCOIN
)
@PostConstruct
@@ -69,13 +78,20 @@ open class ConfiguredUpstreams(
log.error("Chain is unknown: ${up.chain}")
return@forEach
}
if (BlockchainType.fromBlockchain(chain) != BlockchainType.ETHEREUM) {
log.error("Chain is unsupported: ${up.chain}")
return@forEach
}
val options = (up.options ?: UpstreamsConfig.Options())
.merge(defaultOptions[chain] ?: UpstreamsConfig.Options.getDefaults())
buildEthereumUpstream(up.cast(UpstreamsConfig.EthereumConnection::class.java), chain, options)
when (BlockchainType.fromBlockchain(chain)) {
BlockchainType.ETHEREUM -> {
buildEthereumUpstream(up.cast(UpstreamsConfig.EthereumConnection::class.java), chain, options)
}
BlockchainType.BITCOIN -> {
buildBitcoinUpstream(up.cast(UpstreamsConfig.BitcoinConnection::class.java), chain, options)
}
else -> {
log.error("Chain is unsupported: ${up.chain}")
return@forEach
}
}
}
}
}
@@ -101,14 +117,8 @@ open class ConfiguredUpstreams(
return defaultOptions
}
private fun buildEthereumUpstream(config: UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>,
chain: Chain,
options: UpstreamsConfig.Options
) {
val conn = config.connection!!
var rpcApi: DirectEthereumApi? = null
val urls = ArrayList<URI>()
val methods = if (config.methods != null) {
private fun buildMethods(config: UpstreamsConfig.Upstream<*>, chain: Chain): CallMethods {
return if (config.methods != null) {
ManagedCallMethods(currentUpstreams.getDefaultMethods(chain),
config.methods!!.enabled.map { it.name }.toSet(),
config.methods!!.disabled.map { it.name }.toSet()
@@ -116,6 +126,38 @@ open class ConfiguredUpstreams(
} else {
currentUpstreams.getDefaultMethods(chain)
}
}
private fun buildBitcoinUpstream(config: UpstreamsConfig.Upstream<UpstreamsConfig.BitcoinConnection>,
chain: Chain,
options: UpstreamsConfig.Options) {
val conn = config.connection!!
var rpcApi: DirectBitcoinApi? = null
val methods = buildMethods(config, chain)
conn.rpc?.let { endpoint ->
val rpcClient = BitcoinRpcClient(endpoint.url.toString(), endpoint.basicAuth!!)
rpcApi = DirectBitcoinApi(rpcClient, objectMapper, methods)
}
rpcApi?.let { api ->
val upstream = BitcoinUpstream(config.id
?: "bitcoin-${seq.getAndIncrement()}", chain, api,
options, QuorumForLabels.QuorumItem(1, config.labels),
objectMapper, methods)
upstream.start()
currentUpstreams.update(UpstreamChange(chain, upstream, UpstreamChange.ChangeType.ADDED))
}
}
private fun buildEthereumUpstream(config: UpstreamsConfig.Upstream<UpstreamsConfig.EthereumConnection>,
chain: Chain,
options: UpstreamsConfig.Options) {
val conn = config.connection!!
var rpcApi: DirectEthereumApi? = null
val urls = ArrayList<URI>()
val methods = buildMethods(config, chain)
conn.rpc?.let { endpoint ->
val rpcClient = ReactorHttpRpcClient.newBuilder()
.connectTo(endpoint.url)

53
src/main/kotlin/io/emeraldpay/dshackle/upstream/AbstractHead.kt Normal file
View File

@@ -0,0 +1,53 @@
package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.data.BlockContainer
import org.slf4j.LoggerFactory
import reactor.core.Disposable
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
import java.util.concurrent.atomic.AtomicReference
abstract class AbstractHead : Head {
companion object {
private val log = LoggerFactory.getLogger(AbstractHead::class.java)
}
private val head = AtomicReference<BlockContainer>(null)
private val stream: TopicProcessor<BlockContainer> = TopicProcessor.create()
fun follow(source: Flux<BlockContainer>): Disposable {
return source.distinctUntilChanged {
it.hash
}.filter { block ->
val curr = head.get()
curr == null || curr.difficulty < block.difficulty
}
.subscribe { block ->
val prev = head.getAndUpdate { curr ->
if (curr == null || curr.difficulty < block.difficulty) {
block
} else {
curr
}
}
if (prev == null || prev.hash != block.hash) {
log.debug("New block ${block.height} ${block.hash}")
stream.onNext(block)
}
}
}
override fun getFlux(): Flux<BlockContainer> {
return Flux.merge(
Mono.justOrEmpty(head.get()),
Flux.from(stream)
).onBackpressureLatest()
}
fun getCurrent(): BlockContainer? {
return head.get()
}
}

3
src/main/kotlin/io/emeraldpay/dshackle/upstream/AggregatedUpstream.kt
View File

@@ -20,7 +20,6 @@ import io.emeraldpay.dshackle.cache.*
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.calls.AggregatedCallMethods
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Flux
@@ -89,7 +88,7 @@ abstract class AggregatedUpstream<U : UpstreamApi>(
cacheSubscription = null
}
fun onHeadUpdated(head: EthereumHead) {
fun onHeadUpdated(head: Head) {
reconfigLock.withLock {
cacheSubscription?.dispose()
cacheSubscription = head.getFlux().subscribe {

6
src/main/kotlin/io/emeraldpay/dshackle/upstream/CachingEthereumApi.kt
View File

@@ -18,9 +18,7 @@ package io.emeraldpay.dshackle.upstream
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.data.*
import io.emeraldpay.dshackle.upstream.ethereum.EmptyEthereumHead
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.infinitape.etherjar.hex.HexQuantity
import org.slf4j.LoggerFactory
import reactor.core.publisher.Mono
@@ -30,7 +28,7 @@ import java.util.function.Function
open class CachingEthereumApi(
private val objectMapper: ObjectMapper,
private val caches: Caches,
private val head: EthereumHead
private val head: Head
): EthereumApi(objectMapper) {
companion object {
@@ -41,7 +39,7 @@ open class CachingEthereumApi(
*/
@JvmStatic
fun empty(objectMapper: ObjectMapper): CachingEthereumApi {
return CachingEthereumApi(objectMapper, Caches.default(objectMapper), EmptyEthereumHead())
return CachingEthereumApi(objectMapper, Caches.default(objectMapper), EmptyHead())
}
}

22
src/main/kotlin/io/emeraldpay/dshackle/upstream/ChainUpstreams.kt
View File

@@ -17,11 +17,14 @@ package io.emeraldpay.dshackle.upstream
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.upstream.ethereum.EthereumChainUpstreams
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Mono
import java.lang.IllegalStateException
import java.time.Duration
/**
* General interface to upstream(s) to a single chain
@@ -111,5 +114,22 @@ abstract class ChainUpstreams<U : UpstreamApi>(
return 0
}
abstract fun printStatus()
fun printStatus() {
var height: Long? = null
try {
height = getHead().getFlux().next().block(Duration.ofSeconds(1))?.height
} catch (e: IllegalStateException) {
//timout
} catch (e: Exception) {
log.warn("Head processing error: ${e.javaClass} ${e.message}")
}
val statuses = upstreams.map { it.getStatus() }
.groupBy { it }
.map { "${it.key.name}/${it.value.size}" }
.joinToString(",")
val lag = upstreams.map { it.getLag() }
.joinToString(", ")
log.info("State of ${chain.chainCode}: height=${height ?: '?'}, status=$statuses, lag=[$lag]")
}
}

82
src/main/kotlin/io/emeraldpay/dshackle/upstream/CurrentUpstreams.kt
View File

@@ -16,17 +16,20 @@
package io.emeraldpay.dshackle.upstream
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.BlockchainType
import io.emeraldpay.dshackle.cache.CachesEnabled
import io.emeraldpay.dshackle.cache.CachesFactory
import io.emeraldpay.dshackle.startup.UpstreamChange
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinChainUpstreams
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.bitcoin.DefaultBitcoinMethods
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.calls.DefaultEthereumMethods
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumChainUpstreams
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.slf4j.LoggerFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.scheduling.annotation.Scheduled
@@ -34,6 +37,7 @@ import org.springframework.stereotype.Repository
import reactor.core.publisher.Flux
import reactor.core.publisher.TopicProcessor
import java.util.*
import java.util.concurrent.Callable
import java.util.concurrent.ConcurrentHashMap
import java.util.concurrent.locks.ReentrantLock
import kotlin.concurrent.withLock
@@ -54,36 +58,60 @@ class CurrentUpstreams(
fun update(change: UpstreamChange) {
updateLock.withLock {
val chain = change.chain
val up = change.upstream
.cast(EthereumUpstream::class.java, EthereumApi::class.java) as Upstream<EthereumApi>
val current = chainMapping[chain] as ChainUpstreams<EthereumApi>?
if (change.type == UpstreamChange.ChangeType.REMOVED) {
current?.removeUpstream(up.getId())
log.info("Upstream ${change.upstream.getId()} with chain $chain has been removed")
} else {
if (current == null) {
val created = EthereumChainUpstreams(chain, ArrayList(), cachesFactory.getCaches(chain), objectMapper)
if (up is CachesEnabled) {
up.setCaches(created.caches)
when (BlockchainType.fromBlockchain(chain)) {
BlockchainType.ETHEREUM -> {
val up = change.upstream
.cast(EthereumUpstream::class.java, EthereumApi::class.java) as Upstream<EthereumApi>
val current = chainMapping[chain] as ChainUpstreams<EthereumApi>?
val factory = Callable {
EthereumChainUpstreams(chain, ArrayList(), cachesFactory.getCaches(chain), objectMapper) as ChainUpstreams<EthereumApi>
}
created.addUpstream(up)
created.start()
chainMapping[chain] = created
chainsBus.onNext(chain)
} else {
if (up is CachesEnabled) {
up.setCaches(current.caches)
processUpdate(change, up, current, factory)
}
BlockchainType.BITCOIN -> {
val up = change.upstream
.cast(BitcoinUpstream::class.java, DirectBitcoinApi::class.java)
val current = chainMapping[chain] as ChainUpstreams<DirectBitcoinApi>?
val factory = Callable {
BitcoinChainUpstreams(chain, ArrayList(), cachesFactory.getCaches(chain), objectMapper) as ChainUpstreams<DirectBitcoinApi>
}
current.addUpstream(up)
processUpdate(change, up, current, factory)
}
if (!callTargets.containsKey(chain)) {
setupDefaultMethods(chain)
else -> {
log.error("Update for unsupported chain: $chain")
}
log.info("Upstream ${change.upstream.getId()} with chain $chain has been added")
}
}
}
fun <A : UpstreamApi> processUpdate(change: UpstreamChange, up: Upstream<A>, current: ChainUpstreams<A>?, factory: Callable<ChainUpstreams<A>>) {
val chain = change.chain
if (change.type == UpstreamChange.ChangeType.REMOVED) {
current?.removeUpstream(up.getId())
log.info("Upstream ${change.upstream.getId()} with chain $chain has been removed")
} else {
if (current == null) {
val created = factory.call()
if (up is CachesEnabled) {
up.setCaches(created.caches)
}
created.addUpstream(up)
created.start()
chainMapping[chain] = created
chainsBus.onNext(chain)
} else {
if (up is CachesEnabled) {
up.setCaches(current.caches)
}
current.addUpstream(up)
}
if (!callTargets.containsKey(chain)) {
setupDefaultMethods(chain)
}
log.info("Upstream ${change.upstream.getId()} with chain $chain has been added")
}
}
override fun getUpstream(chain: Chain): AggregatedUpstream<*>? {
return chainMapping[chain]
}
@@ -109,7 +137,11 @@ class CurrentUpstreams(
}
fun setupDefaultMethods(chain: Chain): CallMethods {
val created = DefaultEthereumMethods(objectMapper, chain)
val created = when (BlockchainType.fromBlockchain(chain)) {
BlockchainType.ETHEREUM -> DefaultEthereumMethods(objectMapper, chain)
BlockchainType.BITCOIN -> DefaultBitcoinMethods(objectMapper)
else -> throw IllegalStateException("Unsupported chain: $chain")
}
callTargets[chain] = created
return created
}

25
src/main/kotlin/io/emeraldpay/dshackle/upstream/DefaultUpstream.kt
View File

@@ -15,20 +15,29 @@
*/
package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import reactor.core.publisher.Flux
import reactor.core.publisher.TopicProcessor
import java.util.concurrent.atomic.AtomicReference
abstract class DefaultUpstream<U : UpstreamApi>(
private val id: String,
defaultLag: Long,
defaultAvail: UpstreamAvailability
defaultAvail: UpstreamAvailability,
private val options: UpstreamsConfig.Options,
private val targets: CallMethods?
) : Upstream<U> {
constructor() : this(Long.MAX_VALUE, UpstreamAvailability.UNAVAILABLE)
constructor(id: String, options: UpstreamsConfig.Options, targets: CallMethods?) : this(id, Long.MAX_VALUE, UpstreamAvailability.UNAVAILABLE, options, targets)
private val status = AtomicReference(Status(defaultLag, defaultAvail, statusByLag(defaultLag, defaultAvail)))
private val statusStream: TopicProcessor<UpstreamAvailability> = TopicProcessor.create()
override fun isAvailable(): Boolean {
return getStatus() == UpstreamAvailability.OK
}
override fun getStatus(): UpstreamAvailability {
return status.get().status
}
@@ -67,5 +76,17 @@ abstract class DefaultUpstream<U : UpstreamApi>(
return this.status.get().lag
}
override fun getId(): String {
return id
}
override fun getOptions(): UpstreamsConfig.Options {
return options
}
override fun getMethods(): CallMethods {
return targets ?: throw IllegalStateException("Methods are not set")
}
class Status(val lag: Long, val avail: UpstreamAvailability, val status: UpstreamAvailability)
}

5
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EmptyEthereumHead.kt → src/main/kotlin/io/emeraldpay/dshackle/upstream/EmptyHead.kt
View File

@@ -13,12 +13,13 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.emeraldpay.dshackle.upstream.ethereum
package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.Head
import reactor.core.publisher.Flux
class EmptyEthereumHead : EthereumHead {
class EmptyHead : Head {
override fun getFlux(): Flux<BlockContainer> {
return Flux.empty()

1
src/main/kotlin/io/emeraldpay/dshackle/upstream/Head.kt
View File

@@ -17,6 +17,7 @@ package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.data.BlockContainer
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
interface Head {
fun getFlux(): Flux<BlockContainer>

8
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EthereumHeadMerge.kt → src/main/kotlin/io/emeraldpay/dshackle/upstream/MergedHead.kt
View File

@@ -13,7 +13,7 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.emeraldpay.dshackle.upstream.ethereum
package io.emeraldpay.dshackle.upstream
import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.cache.CachesEnabled
@@ -21,9 +21,9 @@ import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Flux
class EthereumHeadMerge(
private val sources: Iterable<EthereumHead>
): DefaultEthereumHead(), Lifecycle, CachesEnabled {
class MergedHead(
private val sources: Iterable<Head>
) : AbstractHead(), Lifecycle, CachesEnabled {
private var subscription: Disposable? = null

3
src/main/kotlin/io/emeraldpay/dshackle/upstream/Upstream.kt
View File

@@ -33,5 +33,6 @@ interface Upstream<out T : UpstreamApi> {
fun getMethods(): CallMethods
fun getId(): String
fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T
fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A>
fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T
}

82
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinChainUpstreams.kt Normal file
View File

@@ -0,0 +1,82 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
class BitcoinChainUpstreams(
chain: Chain,
val upstreams: MutableList<BitcoinUpstream>,
caches: Caches,
objectMapper: ObjectMapper
) : ChainUpstreams<DirectBitcoinApi>(chain, upstreams as MutableList<Upstream<DirectBitcoinApi>>, caches, objectMapper) {
companion object {
private val log = LoggerFactory.getLogger(BitcoinChainUpstreams::class.java)
}
private var head: Head? = null
override fun init() {
if (upstreams.size > 0) {
head = updateHead()
}
super.init()
}
override fun updateHead(): Head {
head?.let {
if (it is Lifecycle) {
it.stop()
}
}
lagObserver?.stop()
lagObserver = null
val head = if (upstreams.size == 1) {
val upstream = upstreams.first()
upstream.setLag(0)
upstream.getHead()
} else {
val newHead = MergedHead(upstreams.map { it.getHead() }).apply {
this.start()
}
// val lagObserver = TODO
// this.lagObserver = lagObserver
newHead
}
onHeadUpdated(head)
return head
}
override fun setHead(head: Head) {
this.head = head
}
override fun getHead(): Head {
return head!!
}
override fun getLabels(): Collection<UpstreamsConfig.Labels> {
return upstreams.flatMap { it.getLabels() }
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(DirectBitcoinApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
}
return this as Upstream<A>
}
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
return castApi(apiType) as T
}
}

33
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinData.kt Normal file
View File

@@ -0,0 +1,33 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
open class BitcoinData(
api: DirectBitcoinApi,
head: Head
) : Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(BitcoinData::class.java)
}
private val mempool = CachingMempoolData(api, head)
open fun getMempool(): CachingMempoolData {
return mempool
}
override fun isRunning(): Boolean {
return mempool.isRunning
}
override fun start() {
mempool.start()
}
override fun stop() {
mempool.stop()
}
}

54
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinRpcClient.kt Normal file
View File

@@ -0,0 +1,54 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.config.AuthConfig
import io.netty.buffer.Unpooled
import io.netty.handler.codec.http.HttpHeaderNames
import io.netty.handler.codec.http.HttpHeaders
import org.slf4j.LoggerFactory
import reactor.core.publisher.Mono
import reactor.netty.http.client.HttpClient
import java.util.*
import java.util.function.Consumer
open class BitcoinRpcClient(
private val target: String,
basicAuth: AuthConfig.ClientBasicAuth?
) {
companion object {
private val log = LoggerFactory.getLogger(BitcoinRpcClient::class.java)
}
private val httpClient: HttpClient
init {
var build = HttpClient.create()
build = build.headers { h ->
h.add(HttpHeaderNames.CONTENT_TYPE, "application/json")
}
basicAuth?.let { basicAuth ->
val authString: String = basicAuth.username + ":" + basicAuth.password
val authBase64 = Base64.getEncoder().encodeToString(authString.toByteArray())
val auth = "Basic $authBase64"
val headers = Consumer { h: HttpHeaders -> h.add(HttpHeaderNames.AUTHORIZATION, auth) }
build = build.headers(headers)
}
this.httpClient = build
}
fun execute(request: ByteArray): Mono<ByteArray> {
val response = httpClient
.post()
.uri(target)
.send(Mono.just(request).map { Unpooled.wrappedBuffer(it) })
return response.responseContent()
.aggregate()
.asByteArray()
}
}

62
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinRpcHead.kt Normal file
View File

@@ -0,0 +1,62 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.Defaults
import io.emeraldpay.dshackle.upstream.AbstractHead
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import org.springframework.scheduling.concurrent.CustomizableThreadFactory
import reactor.core.Disposable
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.scheduler.Schedulers
import java.time.Duration
import java.util.concurrent.Executors
class BitcoinRpcHead(
private val api: DirectBitcoinApi,
private val extractBlock: ExtractBlock,
private val interval: Duration = Duration.ofSeconds(15)
) : Head, AbstractHead(), Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(BitcoinRpcHead::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("bitcoin-rpc-head")))
}
private var refreshSubscription: Disposable? = null
override fun isRunning(): Boolean {
return refreshSubscription != null
}
override fun start() {
if (refreshSubscription != null) {
log.warn("Called to start when running")
return
}
val base = Flux.interval(interval)
.publishOn(scheduler)
.flatMap {
api.executeAndResult(0, "getbestblockhash", emptyList(), String::class.java)
.timeout(Defaults.timeout, Mono.error(Exception("Best block hash is not received")))
}
.distinctUntilChanged()
.flatMap { hash ->
api.execute(0, "getblock", listOf(hash))
.map(extractBlock::extract)
.timeout(Defaults.timeout, Mono.error(Exception("Block data is not received")))
}
.onErrorContinue { err, _ ->
log.debug("RPC error ${err.message}")
}
refreshSubscription = super.follow(base)
}
override fun stop() {
val copy = refreshSubscription
refreshSubscription = null
copy?.dispose()
}
}

108
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinUpstream.kt Normal file
View File

@@ -0,0 +1,108 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.startup.QuorumForLabels
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.grpc.Chain
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Mono
open class BitcoinUpstream(
id: String,
val chain: Chain,
private val api: DirectBitcoinApi,
options: UpstreamsConfig.Options,
val node: QuorumForLabels.QuorumItem,
private val objectMapper: ObjectMapper,
callMethods: CallMethods
) : DefaultUpstream<DirectBitcoinApi>(id, options, callMethods), Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(BitcoinUpstream::class.java)
}
private val head: Head = createHead()
private var validatorSubscription: Disposable? = null
private val data = BitcoinData(api, head)
private fun createHead(): Head {
return BitcoinRpcHead(
api,
ExtractBlock(objectMapper)
)
}
open fun getData(): BitcoinData {
return data
}
override fun getHead(): Head {
return head
}
override fun getApi(matcher: Selector.Matcher): Mono<out DirectBitcoinApi> {
return Mono.just(api)
}
override fun getLabels(): Collection<UpstreamsConfig.Labels> {
return listOf(UpstreamsConfig.Labels())
}
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
return castApi(apiType) as T
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(DirectBitcoinApi::class.java)) {
throw ClassCastException("Cannot cast ${DirectBitcoinApi::class.java} to $apiType")
}
return this as Upstream<A>
}
override fun isRunning(): Boolean {
var runningAny = validatorSubscription != null
if (head is Lifecycle) {
runningAny = runningAny || head.isRunning
}
runningAny = runningAny || data.isRunning
return runningAny
}
override fun start() {
log.info("Configured for ${chain.chainName}")
if (head is Lifecycle) {
if (!head.isRunning) {
head.start()
}
}
data.start()
validatorSubscription?.dispose()
if (getOptions().disableValidation != null && getOptions().disableValidation!!) {
this.setLag(0)
this.setStatus(UpstreamAvailability.OK)
} else {
val validator = BitcoinUpstreamValidator(api, getOptions())
validatorSubscription = validator.start()
.subscribe(this::setStatus)
}
}
override fun stop() {
if (head is Lifecycle) {
head.stop()
}
data.stop()
validatorSubscription?.dispose()
}
}

44
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinUpstreamValidator.kt Normal file
View File

@@ -0,0 +1,44 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import org.slf4j.LoggerFactory
import org.springframework.scheduling.concurrent.CustomizableThreadFactory
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.scheduler.Schedulers
import java.time.Duration
import java.util.concurrent.Executors
class BitcoinUpstreamValidator(
private val api: DirectBitcoinApi,
private val options: UpstreamsConfig.Options
) {
companion object {
private val log = LoggerFactory.getLogger(BitcoinUpstreamValidator::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("bitcoin-validator")))
}
fun validate(): Mono<UpstreamAvailability> {
return api.executeAndResult(0, "getconnectioncount", emptyList(), Int::class.java)
.map { count ->
val minPeers = options.minPeers ?: 1
if (count < minPeers) {
UpstreamAvailability.IMMATURE
} else {
UpstreamAvailability.OK
}
}
.onErrorReturn(UpstreamAvailability.UNAVAILABLE)
}
fun start(): Flux<UpstreamAvailability> {
return Flux.interval(Duration.ofSeconds(15))
.subscribeOn(scheduler)
.flatMap {
validate()
}
}
}

71
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/CachingMempoolData.kt Normal file
View File

@@ -0,0 +1,71 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import reactor.core.Disposable
import reactor.core.publisher.Mono
import java.time.Duration
import java.time.Instant
import java.util.concurrent.atomic.AtomicReference
import java.util.concurrent.locks.ReentrantLock
open class CachingMempoolData(
private val api: DirectBitcoinApi,
private val head: Head
) : Lifecycle {
companion object {
private val log = LoggerFactory.getLogger(CachingMempoolData::class.java)
private val TTL = Duration.ofSeconds(15)
}
private val current = AtomicReference<Container>(Container.empty())
private val updateLock = ReentrantLock()
private var headListener: Disposable? = null
open fun get(): Mono<List<String>> {
val value = current.get()
return if (value.since < Instant.now().minus(TTL)) {
updateLock.lock()
fetchFromUpstream()
.timeout(Duration.ofSeconds(3), Mono.empty())
.doOnNext {
current.set(Container(Instant.now(), it))
}.doFinally {
updateLock.unlock()
}
} else {
Mono.just(value.value)
}
}
fun fetchFromUpstream(): Mono<List<String>> {
return api.executeAndResult(0, "getrawmempool", emptyList(), List::class.java) as Mono<List<String>>
}
class Container(val since: Instant, val value: List<String>) {
companion object {
fun empty(): Container {
return Container(Instant.MIN, emptyList())
}
}
}
override fun isRunning(): Boolean {
return headListener != null
}
override fun start() {
headListener?.dispose()
headListener = head.getFlux().doOnNext {
current.set(Container.empty())
}.subscribe()
}
override fun stop() {
headListener?.dispose()
headListener = null
}
}

71
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/DefaultBitcoinMethods.kt Normal file
View File

@@ -0,0 +1,71 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.quorum.*
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.infinitape.etherjar.rpc.JacksonRpcConverter
import io.infinitape.etherjar.rpc.RpcException
import java.util.*
class DefaultBitcoinMethods(
private val objectMapper: ObjectMapper
) : CallMethods {
//TODO maybe Ethereum RPC parser should not be really used for Bitcoin
private val jacksonRpcConverter = JacksonRpcConverter(objectMapper)
private val anyResponseMethods = listOf(
"getblock",
"gettransaction", "getrawtransaction", "gettxout",
"getmemorypool"
).sorted()
private val headVerifiedMethods = listOf(
"getbestblockhash", "getblocknumber", "getblockcount",
"listunspent", "getreceivedbyaddress"
).sorted()
private val hardcodedMethods = listOf(
"getconnectioncount", "getnetworkinfo"
).sorted()
private val broadcastMethods = listOf(
"sendrawtransaction"
).sorted()
private val allowedMethods = (anyResponseMethods + hardcodedMethods + headVerifiedMethods).sorted()
override fun getQuorumFor(method: String): CallQuorum {
return when {
Collections.binarySearch(hardcodedMethods, method) >= 0 -> AlwaysQuorum()
Collections.binarySearch(anyResponseMethods, method) >= 0 -> NotLaggingQuorum(2)
Collections.binarySearch(headVerifiedMethods, method) >= 0 -> NotLaggingQuorum(0)
Collections.binarySearch(broadcastMethods, method) >= 0 -> BroadcastQuorum(jacksonRpcConverter)
else -> AlwaysQuorum()
}
}
override fun isAllowed(method: String): Boolean {
return Collections.binarySearch(allowedMethods, method) >= 0
}
override fun getSupportedMethods(): Set<String> {
return allowedMethods.toSortedSet()
}
override fun isHardcoded(method: String): Boolean {
return Collections.binarySearch(hardcodedMethods, method) >= 0;
}
override fun executeHardcoded(method: String): Any {
return when (method) {
"getconnectioncount" -> 42
"getnetworkinfo" -> mapOf(
"version" to 700000,
"subversion" to "/EmeraldDshackle:v0.7/"
)
else -> throw RpcException(-32601, "Method not found")
}
}
}

107
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/DirectBitcoinApi.kt Normal file
View File

@@ -0,0 +1,107 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.JavaType
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.upstream.UpstreamApi
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.grpc.Status
import io.grpc.StatusRuntimeException
import io.infinitape.etherjar.rpc.RpcException
import io.infinitape.etherjar.rpc.RpcResponseError
import io.infinitape.etherjar.rpc.json.FullResponseJson
import io.infinitape.etherjar.rpc.json.RequestJson
import io.infinitape.etherjar.rpc.json.ResponseJson
import org.slf4j.LoggerFactory
import reactor.core.publisher.Mono
open class DirectBitcoinApi(
val bitcoinRpcClient: BitcoinRpcClient,
val objectMapper: ObjectMapper,
val targets: CallMethods
) : UpstreamApi {
companion object {
private val log = LoggerFactory.getLogger(DirectBitcoinApi::class.java)
}
open override fun execute(id: Int, method: String, params: List<Any>): Mono<ByteArray> {
//TODO it's almost the same code as for DirectEthereumApi; refactor
val result: Mono<out Any> = when {
targets.isHardcoded(method) -> Mono.just(method).map { targets.executeHardcoded(it) }
targets.isAllowed(method) -> executeAndResult(id, method, params, Object::class.java)
else -> Mono.error(RpcException(-32601, "Method not allowed or not found"))
}
return processResult(id, method, result)
}
public fun processResult(id: Int, method: String, result: Mono<out Any>): Mono<ByteArray> {
//TODO it's the same code as for DirectEthereumApi; refactor
return result
.doOnError { t ->
log.warn("Upstream error: [${t.message}] for $method")
}
.map {
val resp = ResponseJson<Any, Int>()
resp.id = id
resp.result = it
resp
}
.switchIfEmpty(
Mono.fromCallable {
val resp = ResponseJson<Any, Int>()
resp.id = id
resp.result = null
resp
}
)
.map {
objectMapper.writer().writeValueAsBytes(it)
}
.onErrorResume(StatusRuntimeException::class.java) { t ->
if (t.status.code == Status.Code.CANCELLED) {
Mono.empty<ByteArray>()
} else {
Mono.error(RpcException(RpcResponseError.CODE_UPSTREAM_CONNECTION_ERROR, "gRPC error ${t.status}"))
}
}
.onErrorMap { t ->
if (RpcException::class.java.isAssignableFrom(t.javaClass)) {
t
} else {
log.warn("Convert to RPC error. Exception ${t.javaClass}:${t.message}", t)
RpcException(-32020, "Error reading from upstream", null, t)
}
}
.onErrorResume(RpcException::class.java) { t ->
val resp = ResponseJson<Any, Int>()
resp.id = id
resp.error = t.error
Mono.just(objectMapper.writer().writeValueAsBytes(resp))
}
}
open fun <T> executeAndResult(id: Int, method: String, params: List<Any>, resultType: Class<T>): Mono<T> {
val rpc = RequestJson<Int>(method, params, id)
return Mono.just(rpc)
.map(objectMapper::writeValueAsBytes)
.flatMap(bitcoinRpcClient::execute)
.flatMap { json ->
val type: JavaType = objectMapper.typeFactory.constructParametricType(FullResponseJson::class.java, resultType, Int::class.java)
val resp = objectMapper.readerFor(type).readValue<FullResponseJson<T, Int>>(json)
if (resp.hasError()) {
Mono.error(resp.error.asException())
} else {
Mono.just(resp.result)
}
}
}
open fun getBlock(hash: String): Mono<Map<String, Any>> {
return executeAndResult(0, "getblock", listOf(hash), Map::class.java) as Mono<Map<String, Any>>
}
open fun getTx(txid: String): Mono<Map<String, Any>> {
return executeAndResult(0, "getrawtransaction", listOf(txid, true), Map::class.java) as Mono<Map<String, Any>>
}
}

56
src/main/kotlin/io/emeraldpay/dshackle/upstream/bitcoin/ExtractBlock.kt Normal file
View File

@@ -0,0 +1,56 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.data.TxId
import org.apache.commons.codec.binary.Hex
import org.slf4j.LoggerFactory
import java.math.BigInteger
import java.time.Instant
class ExtractBlock(
private val objectMapper: ObjectMapper
) {
companion object {
private val log = LoggerFactory.getLogger(ExtractBlock::class.java)
@JvmStatic
fun getHeight(data: Map<String, Any>): Long? {
val height = data["height"] as Number? ?: return null
return height.toLong()
}
@JvmStatic
fun getTime(data: Map<String, Any>): Instant? {
val time = data["time"] as Number? ?: return null
return Instant.ofEpochMilli(time.toLong() * 1000)
}
@JvmStatic
fun getDifficulty(data: Map<String, Any>): BigInteger? {
val chainwork = data["chainwork"] as String? ?: return null
return BigInteger(1, Hex.decodeHex(chainwork))
}
}
fun extract(json: ByteArray): BlockContainer {
val data = objectMapper.readValue(json, Map::class.java) as Map<String, Any>
val hash = data["hash"] as String? ?: throw IllegalArgumentException("Block JSON has no hash")
val transactions = (data["tx"] as List<String>?)?.map(TxId.Companion::from) ?: emptyList()
return BlockContainer(
getHeight(data) ?: throw IllegalArgumentException("Block JSON has no height"),
BlockId.from(hash),
getDifficulty(data) ?: throw IllegalArgumentException("Block JSON has no chainwork"),
getTime(data) ?: throw IllegalArgumentException("Block JSON has no time"),
false,
json,
transactions
)
}
}

6
src/main/kotlin/io/emeraldpay/dshackle/upstream/calls/DirectCallMethods.kt
View File

@@ -22,10 +22,10 @@ import io.emeraldpay.dshackle.quorum.CallQuorum
* Configuration that uses [AlwaysQuorum] for all available methods. The methods list itself
* is provided in constructor (or empty otherwise)
*/
class DirectCallMethods(private val methods: Set<String>) : CallMethods {
open class DirectCallMethods(private val methods: Set<String>) : CallMethods {
constructor(): this(emptySet())
constructor(methods: Collection<String>): this(methods.toSet())
constructor() : this(emptySet())
constructor(methods: Collection<String>) : this(methods.toSet())
override fun getQuorumFor(method: String): CallQuorum {
return AlwaysQuorum()

47
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/DefaultEthereumHead.kt
View File

@@ -1,49 +1,8 @@
package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.data.BlockContainer
import org.slf4j.LoggerFactory
import reactor.core.Disposable
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
import java.util.concurrent.atomic.AtomicReference
import io.emeraldpay.dshackle.upstream.AbstractHead
import io.emeraldpay.dshackle.upstream.Head
open class DefaultEthereumHead: EthereumHead {
open class DefaultEthereumHead : Head, AbstractHead() {
private val log = LoggerFactory.getLogger(DefaultEthereumHead::class.java)
private val head = AtomicReference<BlockContainer>(null)
private val stream: TopicProcessor<BlockContainer> = TopicProcessor.create()
fun follow(source: Flux<BlockContainer>): Disposable {
return source.distinctUntilChanged {
it.hash
}.filter { block ->
val curr = head.get()
curr == null || curr.difficulty < block.difficulty
}
.subscribe { block ->
val prev = head.getAndUpdate { curr ->
if (curr == null || curr.difficulty < block.difficulty) {
block
} else {
curr
}
}
if (prev == null || prev.hash != block.hash) {
log.debug("New block ${block.height} ${block.hash}")
stream.onNext(block)
}
}
}
override fun getFlux(): Flux<BlockContainer> {
return Flux.merge(
Mono.justOrEmpty(head.get()),
Flux.from(stream)
).onBackpressureLatest()
}
fun getCurrent(): BlockContainer? {
return head.get()
}
}

46
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EthereumChainUpstreams.kt
View File

@@ -20,13 +20,8 @@ import io.emeraldpay.dshackle.cache.Caches
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import java.lang.IllegalStateException
import java.time.Duration
class EthereumChainUpstreams(
chain: Chain,
@@ -39,7 +34,7 @@ class EthereumChainUpstreams(
private val log = LoggerFactory.getLogger(EthereumChainUpstreams::class.java)
}
private var head: EthereumHead? = null
private var head: Head? = null
init {
this.init()
@@ -52,15 +47,15 @@ class EthereumChainUpstreams(
super.init()
}
override fun getHead(): EthereumHead {
override fun getHead(): Head {
return head!!
}
override fun setHead(head: Head) {
this.head = head as EthereumHead
this.head = head
}
override fun updateHead(): EthereumHead {
override fun updateHead(): Head {
head?.let {
if (it is Lifecycle) {
it.stop()
@@ -73,7 +68,7 @@ class EthereumChainUpstreams(
upstream.setLag(0)
upstream.getHead()
} else {
val newHead = EthereumHeadMerge(upstreams.map { it.getHead() }).apply {
val newHead = MergedHead(upstreams.map { it.getHead() }).apply {
this.start()
}
val lagObserver = EthereumHeadLagObserver(newHead, upstreams as Collection<Upstream<EthereumApi>>).apply {
@@ -90,34 +85,19 @@ class EthereumChainUpstreams(
return upstreams.flatMap { it.getLabels() }
}
override fun printStatus() {
var height: Long? = null
try {
height = getHead().getFlux().next().block(Duration.ofSeconds(1))?.height
} catch (e: IllegalStateException) {
//timout
} catch (e: Exception) {
log.warn("Head processing error: ${e.javaClass} ${e.message}")
}
val statuses = upstreams.map { it.getStatus() }
.groupBy { it }
.map { "${it.key.name}/${it.value.size}" }
.joinToString(",")
val lag = upstreams.map { it.getLag() }
.joinToString(", ")
log.info("State of ${chain.chainCode}: height=${height ?: '?'}, status=$statuses, lag=[$lag]")
}
@SuppressWarnings("unchecked")
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T {
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
if (!upstreamType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $upstreamType")
return castApi(apiType) as T
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
}
return this as T
return this as Upstream<A>
}
}

24
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EthereumHead.kt
View File

@@ -1,24 +0,0 @@
/**
* Copyright (c) 2019 ETCDEV GmbH
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.upstream.Head
import io.infinitape.etherjar.domain.TransactionId
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
interface EthereumHead : Head {
}

3
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EthereumHeadLagObserver.kt
View File

@@ -18,12 +18,13 @@ package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.upstream.HeadLagObserver
import io.emeraldpay.dshackle.upstream.Upstream
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.Head
import org.slf4j.LoggerFactory
import reactor.core.publisher.Flux
import java.time.Duration
class EthereumHeadLagObserver(
master: EthereumHead,
master: Head,
followers: Collection<Upstream<EthereumApi>>
) : HeadLagObserver<EthereumApi>(master, followers) {

52
src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EthereumUpstream.kt
View File

@@ -24,8 +24,6 @@ import io.emeraldpay.dshackle.upstream.*
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.upstream.calls.DirectCallMethods
import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.rpc.json.BlockJson
import io.infinitape.etherjar.rpc.json.TransactionRefJson
import org.slf4j.LoggerFactory
import org.springframework.context.Lifecycle
import reactor.core.Disposable
@@ -33,15 +31,15 @@ import reactor.core.publisher.Mono
import java.time.Duration
open class EthereumUpstream(
private val id: String,
id: String,
val chain: Chain,
private val api: DirectEthereumApi,
private val ethereumWs: EthereumWs? = null,
private val options: UpstreamsConfig.Options,
options: UpstreamsConfig.Options,
val node: QuorumForLabels.QuorumItem,
private val targets: CallMethods,
targets: CallMethods,
private val objectMapper: ObjectMapper
) : DefaultUpstream<EthereumApi>(), Upstream<EthereumApi>, CachesEnabled, Lifecycle {
) : DefaultUpstream<EthereumApi>(id, options, targets), Upstream<EthereumApi>, CachesEnabled, Lifecycle {
constructor(id: String, chain: Chain, api: DirectEthereumApi, objectMapper: ObjectMapper) : this(id, chain, api, null,
UpstreamsConfig.Options.getDefaults(), QuorumForLabels.QuorumItem(1, UpstreamsConfig.Labels()),
@@ -50,7 +48,7 @@ open class EthereumUpstream(
private val log = LoggerFactory.getLogger(EthereumUpstream::class.java)
private val head: EthereumHead = this.createHead()
private val head: Head = this.createHead()
private var validatorSubscription: Disposable? = null
init {
@@ -64,18 +62,14 @@ open class EthereumUpstream(
}
}
override fun getId(): String {
return id
}
override fun start() {
log.info("Configured for ${chain.chainName}")
if (options.disableValidation != null && options.disableValidation!!) {
if (getOptions().disableValidation != null && getOptions().disableValidation!!) {
this.setLag(0)
this.setStatus(UpstreamAvailability.OK)
} else {
val validator = UpstreamValidator(this, options)
val validator = EthereumUpstreamValidator(this, getOptions())
validatorSubscription = validator.start()
.subscribe(this::setStatus)
}
@@ -93,7 +87,7 @@ open class EthereumUpstream(
}
}
open fun createHead(): EthereumHead {
open fun createHead(): Head {
return if (ethereumWs != null) {
val ws = EthereumWsHead(ethereumWs).apply {
this.start()
@@ -102,22 +96,18 @@ open class EthereumUpstream(
val rpc = EthereumRpcHead(api, objectMapper, Duration.ofSeconds(30)).apply {
this.start()
}
EthereumHeadMerge(listOf(rpc, ws)).apply {
MergedHead(listOf(rpc, ws)).apply {
this.start()
}
} else {
log.warn("Setting up upstream $id with RPC-only access, less effective than WS+RPC")
log.warn("Setting up upstream ${this.getId()} with RPC-only access, less effective than WS+RPC")
EthereumRpcHead(api, objectMapper).apply {
this.start()
}
}
}
override fun isAvailable(): Boolean {
return getStatus() == UpstreamAvailability.OK
}
override fun getHead(): EthereumHead {
override fun getHead(): Head {
return head
}
@@ -125,27 +115,23 @@ open class EthereumUpstream(
return Mono.just(api)
}
override fun getOptions(): UpstreamsConfig.Options {
return options
}
override fun getLabels(): Collection<UpstreamsConfig.Labels> {
return listOf(node.labels)
}
override fun getMethods(): CallMethods {
return targets
}
@Suppress("unchecked")
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T {
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
if (!upstreamType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $upstreamType")
return castApi(apiType) as T
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
}
return this as T
return this as Upstream<A>
}
}

11
src/main/kotlin/io/emeraldpay/dshackle/upstream/UpstreamValidator.kt → src/main/kotlin/io/emeraldpay/dshackle/upstream/ethereum/EthereumUpstreamValidator.kt
View File

@@ -13,11 +13,12 @@
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package io.emeraldpay.dshackle.upstream
package io.emeraldpay.dshackle.upstream.ethereum
import io.emeraldpay.dshackle.Defaults
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.dshackle.upstream.Selector
import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import io.infinitape.etherjar.rpc.*
import org.slf4j.LoggerFactory
import org.springframework.scheduling.concurrent.CustomizableThreadFactory
@@ -27,13 +28,13 @@ import reactor.core.scheduler.Schedulers
import java.time.Duration
import java.util.concurrent.Executors
class UpstreamValidator(
class EthereumUpstreamValidator(
private val ethereumUpstream: EthereumUpstream,
private val options: UpstreamsConfig.Options
) {
companion object {
private val log = LoggerFactory.getLogger(UpstreamValidator::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("validator")))
private val log = LoggerFactory.getLogger(EthereumUpstreamValidator::class.java)
val scheduler = Schedulers.fromExecutor(Executors.newCachedThreadPool(CustomizableThreadFactory("ethereum-validator")))
}
fun validate(): Mono<UpstreamAvailability> {

33
src/main/kotlin/io/emeraldpay/dshackle/upstream/grpc/EthereumGrpcUpstream.kt
View File

@@ -33,7 +33,6 @@ import io.emeraldpay.dshackle.upstream.calls.DirectCallMethods
import io.emeraldpay.dshackle.upstream.ethereum.DefaultEthereumHead
import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.emeraldpay.grpc.Chain
import io.infinitape.etherjar.domain.BlockHash
import io.infinitape.etherjar.rpc.*
@@ -59,13 +58,16 @@ open class EthereumGrpcUpstream(
private val blockchainStub: ReactorBlockchainGrpc.ReactorBlockchainStub,
private val objectMapper: ObjectMapper,
private val rpcClient: ReactorEmeraldClient
) : DefaultUpstream<EthereumApi>(), CachesEnabled, Lifecycle {
) : DefaultUpstream<EthereumApi>(
"$parentId/${chain.chainCode}",
UpstreamsConfig.Options.getDefaults(),
null
), CachesEnabled, Lifecycle {
private var allLabels: Collection<UpstreamsConfig.Labels> = ArrayList<UpstreamsConfig.Labels>()
private val log = LoggerFactory.getLogger(EthereumGrpcUpstream::class.java)
private var caches: Caches? = null
private val options = UpstreamsConfig.Options.getDefaults()
private val nodes = AtomicReference<QuorumForLabels>(QuorumForLabels())
private val head = DefaultEthereumHead()
private var targets: CallMethods? = null
@@ -84,10 +86,6 @@ open class EthereumGrpcUpstream(
}
}
override fun getId(): String {
return "$parentId/${chain.chainCode}"
}
override fun start() {
if (this.isRunning) return
val chainRef = Common.Chain.newBuilder()
@@ -200,12 +198,12 @@ open class EthereumGrpcUpstream(
}
override fun isAvailable(): Boolean {
return getStatus() == UpstreamAvailability.OK && head.getCurrent() != null && nodes.get().getAll().any {
return super.isAvailable() && head.getCurrent() != null && nodes.get().getAll().any {
it.quorum > 0
}
}
override fun getHead(): EthereumHead {
override fun getHead(): Head {
return head
}
@@ -213,23 +211,22 @@ open class EthereumGrpcUpstream(
return Mono.just(createApi(matcher))
}
override fun getOptions(): UpstreamsConfig.Options {
return options
}
override fun setCaches(caches: Caches) {
this.caches = caches
}
@SuppressWarnings("unchecked")
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, upstreamType: Class<TA>): T {
override fun <T : Upstream<TA>, TA : UpstreamApi> cast(selfType: Class<T>, apiType: Class<TA>): T {
if (!selfType.isAssignableFrom(this.javaClass)) {
throw ClassCastException("Cannot cast ${this.javaClass} to $selfType")
}
if (!upstreamType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $upstreamType")
}
return this as T
return castApi(apiType) as T
}
override fun <A : UpstreamApi> castApi(apiType: Class<A>): Upstream<A> {
if (!apiType.isAssignableFrom(EthereumApi::class.java)) {
throw ClassCastException("Cannot cast ${EthereumApi::class.java} to $apiType")
}
return this as Upstream<A>
}
}

6
src/test/groovy/io/emeraldpay/dshackle/cache/BlocksMemCacheSpec.groovy vendored
View File

@@ -77,9 +77,9 @@ class BlocksMemCacheSpec extends Specification {
def act3 = cache.read(BlockId.from(hash3)).block()
def act4 = cache.read(BlockId.from(hash4)).block()
then:
act2.hash.toHex() == hash2
act3.hash.toHex() == hash3
act4.hash.toHex() == hash4
act2.hash.toHex() == hash2.substring(2)
act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4.substring(2)
act1 == null
}

6
src/test/groovy/io/emeraldpay/dshackle/cache/BlocksRedisCacheSpec.groovy vendored
View File

@@ -57,13 +57,13 @@ class BlocksRedisCacheSpec extends Specification {
then:
dec.height == 100
dec.hash.toHex() == hash3
dec.hash.toHex() == hash3.substring(2)
dec.difficulty.toString() == "10515"
dec.timestamp == Instant.ofEpochSecond(10501050)
dec.json == "test".bytes
dec.transactions.size() == 2
dec.transactions[0].toHex() == hash2
dec.transactions[1].toHex() == hash1
dec.transactions[0].toHex() == hash2.substring(2)
dec.transactions[1].toHex() == hash1.substring(2)
dec == cont
}

14
src/test/groovy/io/emeraldpay/dshackle/cache/HeightCacheSpec.groovy vendored
View File

@@ -38,10 +38,10 @@ class HeightCacheSpec extends Specification {
def act3 = cache.read(102).block()
def act4 = cache.read(103).block()
then:
act1.toHex() == hash1
act2.toHex() == hash2
act3.toHex() == hash3
act4.toHex() == hash4
act1.toHex() == hash1.substring(2)
act2.toHex() == hash2.substring(2)
act3.toHex() == hash3.substring(2)
act4.toHex() == hash4.substring(2)
}
def "Keeps only configured amount"() {
@@ -65,8 +65,8 @@ class HeightCacheSpec extends Specification {
def act4 = cache.read(103).block()
then:
act1 == null
act2.toHex() == hash2
act3.toHex() == hash3
act4.toHex() == hash4
act2.toHex() == hash2.substring(2)
act3.toHex() == hash3.substring(2)
act4.toHex() == hash4.substring(2)
}
}

14
src/test/groovy/io/emeraldpay/dshackle/cache/TxMemCacheSpec.groovy vendored
View File

@@ -57,9 +57,9 @@ class TxMemCacheSpec extends Specification {
def act3 = cache.read(TxId.from(hash3)).block()
def act4 = cache.read(TxId.from(hash4)).block()
then:
act2.hash.toHex() == hash2
act3.hash.toHex() == hash3
act4.hash.toHex() == hash4
act2.hash.toHex() == hash2.substring(2)
act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4.substring(2)
act1 == null
}
@@ -93,8 +93,8 @@ class TxMemCacheSpec extends Specification {
then:
act1 == null
act2 == null
act3.hash.toHex() == hash3
act4.hash.toHex() == hash4
act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4.substring(2)
}
def "Evict all by block data"() {
@@ -137,7 +137,7 @@ class TxMemCacheSpec extends Specification {
then:
act1 == null
act2 == null
act3.hash.toHex() == hash3
act4.hash.toHex() == hash4
act3.hash.toHex() == hash3.substring(2)
act4.hash.toHex() == hash4.substring(2)
}
}

8
src/test/groovy/io/emeraldpay/dshackle/cache/TxRedisCacheSpec.groovy vendored
View File

@@ -56,8 +56,8 @@ class TxRedisCacheSpec extends Specification {
then:
dec.height == 2000
dec.hash.toHex() == hash1
dec.blockId.toHex() == hash2
dec.hash.toHex() == hash1.substring(2)
dec.blockId.toHex() == hash2.substring(2)
dec.json == "test".bytes
dec == cont
}
@@ -170,8 +170,8 @@ class TxRedisCacheSpec extends Specification {
act1 == null
act2 == null
act3 != null
act3.hash.toHex() == hash3
act3.hash.toHex() == hash3.substring(2)
act4 != null
act4.hash.toHex() == hash4
act4.hash.toHex() == hash4.substring(2)
}
}

30
src/test/groovy/io/emeraldpay/dshackle/data/BlockIdSpec.groovy Normal file
View File

@@ -0,0 +1,30 @@
package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.BlockHash
import spock.lang.Specification
class BlockIdSpec extends Specification {
def "Create from string"() {
expect:
BlockId.from(source).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "A0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xA0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
def "Create from block hash"() {
expect:
BlockId.from(BlockHash.from(source)).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
}

29
src/test/groovy/io/emeraldpay/dshackle/data/TxIdSpec.groovy Normal file
View File

@@ -0,0 +1,29 @@
package io.emeraldpay.dshackle.data
import io.infinitape.etherjar.domain.TransactionId
import spock.lang.Specification
class TxIdSpec extends Specification {
def "Create from string"() {
expect:
TxId.from(source).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "A0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xA0E65CBC1B52A8CA60562112C6060552D882F16F34A9DBA2CCDC05C0A6A27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
def "Create from block hash"() {
expect:
TxId.from(TransactionId.from(source)).toHex() == exp
where:
exp | source
"a0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
"00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100" | "0x00e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27100"
}
}

2
src/test/groovy/io/emeraldpay/dshackle/rpc/NativeCallSpec.groovy
View File

@@ -259,7 +259,7 @@ class NativeCallSpec extends Specification {
.build()
})
.build()
1 * upstreams.getUpstream(Chain.TESTNET_MORDEN) >> null
1 * upstreams.isAvailable(Chain.TESTNET_MORDEN) >> false
when:
def resp = nativeCall.prepareCall(req)
then:

251
src/test/groovy/io/emeraldpay/dshackle/rpc/TrackBitcoinAddressSpec.groovy Normal file
View File

@@ -0,0 +1,251 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.AggregatedUpstream
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.Upstream
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
import java.time.Instant
class TrackBitcoinAddressSpec extends Specification {
String hash1 = "0xa0e65cbc1b52a8ca60562112c6060552d882f16f34a9dba2ccdc05c0a6a27c22"
def "Correct sum from multiple"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-one-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], unspents)
then:
total.size() == 1
total[0].address.chain == Chain.BITCOIN
total[0].address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
total[0].balance.toString() == "32928461"
}
def "Correct sum when other addresses"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-two-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], unspents)
then:
total.size() == 1
total[0].address.chain == Chain.BITCOIN
total[0].address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
total[0].balance.toString() == "32928461"
}
def "Sum for two addresses"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-two-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", "35hK24tcLEWcgNA4JxpvbkNkoAcDGqQPsP"], unspents).sort { it.address.address }
then:
total.size() == 2
with(total[0]) {
address.chain == Chain.BITCOIN
address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
balance.toString() == "32928461"
}
with(total[1]) {
address.chain == Chain.BITCOIN
address.address == "35hK24tcLEWcgNA4JxpvbkNkoAcDGqQPsP"
balance.toString() == "25550215615737"
}
}
def "Zero for empty unspents"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], [])
then:
total.size() == 1
total[0].address.chain == Chain.BITCOIN
total[0].address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
total[0].balance.toString() == "0"
}
def "Zero for unknown address"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/unspent-two-addr.json")
def unspents = TestingCommons.objectMapper().readValue(json, List)
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
when:
def total = track.getTotal(Chain.BITCOIN, ["16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk", "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"], unspents).sort { it.address.address }
then:
total.size() == 2
with(total[0]) {
address.address == "16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk"
balance.toString() == "0"
}
with(total[1]) {
address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
balance.toString() == "32928461"
}
}
def "One address for single provided"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def req = BlockchainOuterClass.BalanceRequest.newBuilder()
.setAddress(
Common.AnyAddress.newBuilder()
.setAddressSingle(
Common.SingleAddress.newBuilder()
.setAddress("16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk")
)
)
.build()
when:
def act = track.allAddresses(req)
then:
act == ["16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk"]
}
def "Sorted addresses for multiple provided"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def req = BlockchainOuterClass.BalanceRequest.newBuilder()
.setAddress(
Common.AnyAddress.newBuilder()
.setAddressMulti(
Common.MultiAddress.newBuilder()
.addAddresses(Common.SingleAddress.newBuilder().setAddress("16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk"))
.addAddresses(Common.SingleAddress.newBuilder().setAddress("3BMqADKWoWHPASsUdHvnUL6E1jpZkMnLZz"))
.addAddresses(Common.SingleAddress.newBuilder().setAddress("1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"))
.addAddresses(Common.SingleAddress.newBuilder().setAddress("bc1qdthqvt6cllzej7uhdddrltdfsmnt7d0gl5ue5n"))
)
)
.build()
when:
def act = track.allAddresses(req)
then:
act == ["16rCmCmbuWDhPjWTrpQGaU3EPdZF7MTdUk", "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", "3BMqADKWoWHPASsUdHvnUL6E1jpZkMnLZz", "bc1qdthqvt6cllzej7uhdddrltdfsmnt7d0gl5ue5n"]
}
def "Null for no address provided"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def req = BlockchainOuterClass.BalanceRequest.newBuilder()
.build()
when:
def act = track.allAddresses(req)
then:
act == null
}
def "Build proto for common balance"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def balance = new TrackBitcoinAddress.AddressBalance(Chain.BITCOIN, "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", BigInteger.valueOf(123456))
when:
def act = track.buildResponse(balance)
then:
act.address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
act.balance == "123456"
act.asset.chain.number == Chain.BITCOIN.id
act.asset.code == "BTC"
}
def "Build proto for zero balance"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def balance = new TrackBitcoinAddress.AddressBalance(Chain.BITCOIN, "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", BigInteger.ZERO)
when:
def act = track.buildResponse(balance)
then:
act.address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
act.balance == "0"
act.asset.chain.number == Chain.BITCOIN.id
act.asset.code == "BTC"
}
def "Build proto for all bitcoins"() {
setup:
TrackBitcoinAddress track = new TrackBitcoinAddress(Stub(Upstreams))
def balance = new TrackBitcoinAddress.AddressBalance(Chain.BITCOIN, "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", BigInteger.valueOf(21_000_000).multiply(BigInteger.TEN.pow(8)))
when:
def act = track.buildResponse(balance)
then:
act.address.address == "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK"
act.balance == "2100000000000000"
act.asset.chain.number == Chain.BITCOIN.id
act.asset.code == "BTC"
}
def "Get update for a balance"() {
setup:
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
2 * executeAndResult(0, "listunspent", [], List) >>> [
Mono.just([]), Mono.just([[address: "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK", amount: 0.0123]])
]
}
def blocks = TopicProcessor.create()
Head head = Mock(Head) {
1 * getFlux() >> Flux.from(blocks)
}
Upstream upstream
upstream = Mock(AggregatedUpstream) {
_ * getApi(_) >> Mono.just(api)
_ * getHead() >> head
_ * castApi(_) >> { return upstream }
}
Upstreams upstreams = Mock(Upstreams) {
_ * getUpstream(Chain.BITCOIN) >> upstream
}
TrackBitcoinAddress track = new TrackBitcoinAddress(upstreams)
when:
def resp = track.subscribe(BlockchainOuterClass.BalanceRequest.newBuilder()
.setAsset(Common.Asset.newBuilder().setChain(Common.ChainRef.CHAIN_BITCOIN))
.setAddress(
Common.AnyAddress.newBuilder().setAddressSingle(
Common.SingleAddress.newBuilder().setAddress("1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK")
)
)
.build()
).map {
it.balance
}
then:
StepVerifier.create(resp)
.expectNext("0")
.then {
blocks.onNext(new BlockContainer(1L, BlockId.from(hash1), BigInteger.ONE, Instant.now(), false, null, []))
}
.expectNext("1230000")
.then {
blocks.onComplete()
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
}

276
src/test/groovy/io/emeraldpay/dshackle/rpc/TrackBitcoinTxSpec.groovy Normal file
View File

@@ -0,0 +1,276 @@
package io.emeraldpay.dshackle.rpc
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.Upstream
import io.emeraldpay.dshackle.upstream.Upstreams
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinData
import io.emeraldpay.dshackle.upstream.bitcoin.DirectBitcoinApi
import io.emeraldpay.dshackle.upstream.bitcoin.BitcoinUpstream
import io.emeraldpay.dshackle.upstream.bitcoin.CachingMempoolData
import io.emeraldpay.grpc.Chain
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
import java.time.Instant
class TrackBitcoinTxSpec extends Specification {
def "loadMempool() returns not found when not found"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
CachingMempoolData mempoolAccess = Mock(CachingMempoolData) {
1 * get() >> Mono.just([
"69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9",
"d296c6d47335a7f283574b06f1d6303b30ac75631e081ab128346a549ad93350"
])
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getData() >> Mock(BitcoinData) {
_ * getMempool() >> mempoolAccess
}
}
when:
def act = track.loadMempool(upstream, "65ce58db064bd105b14dc76a0bce0df14653cf5263d22d17e78864cf272ee367")
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == false && it.mined == false && it.blockHash == null
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "loadMempool() returns ok when found"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
CachingMempoolData mempoolAccess = Mock(CachingMempoolData) {
1 * get() >> Mono.just([
"69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9",
"d296c6d47335a7f283574b06f1d6303b30ac75631e081ab128346a549ad93350"
])
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getData() >> Mock(BitcoinData) {
_ * getMempool() >> mempoolAccess
}
}
when:
def act = track.loadMempool(upstream, "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9")
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == true && it.mined == false && it.blockHash == null
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "loadExiting() returns not found if not mined"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
1 * getTx(txid) >> Mono.just([
txid: txid
])
}
when:
def act = track.loadExisting(api, txid)
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == true && it.mined == false && it.blockHash == null
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "loadExiting() returns block if mined"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
1 * getTx(txid) >> Mono.just([
txid : txid,
blockhash: "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f",
height : 100
])
}
when:
def act = track.loadExisting(api, txid)
then:
StepVerifier.create(act)
.expectNextMatches {
it.found == true && it.mined == true &&
it.blockHash == "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f" &&
it.height == 100
}
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Goes with confirmations"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
// start with the current block
def next = Flux.fromIterable([10, 12, 13, 14, 15]).map { h ->
new BlockContainer(h.longValue(), BlockId.from("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f"), BigInteger.ONE, Instant.now(), false, null, [])
}
Head head = Mock(Head) {
1 * getFlux() >> next
}
Upstream upstream = Mock(BitcoinUpstream) {
1 * getHead() >> head
}
def status = new TrackBitcoinTx.TxStatus(
txid, true, 10, true, "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f", Instant.now(), BigInteger.ONE, 0
)
when:
def act = track.withConfirmations(upstream, status)
then:
StepVerifier.create(act)
.expectNextMatches { it.confirmations == 1 }
.expectNextMatches { it.confirmations == 3 }
.expectNextMatches { it.confirmations == 4 }
.expectNextMatches { it.confirmations == 5 }
.expectNextMatches { it.confirmations == 6 }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Wait until mined"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
// start with the current block
def next = Flux.fromIterable([10, 12, 13]).map { h ->
new BlockContainer(h.longValue(), BlockId.from("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f"), BigInteger.ONE, Instant.now(), false, null, [])
}
Head head = Mock(Head) {
1 * getFlux() >> next
}
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
3 * getTx(txid) >>> [
Mono.just([
txid: txid
]),
Mono.just([
txid: txid
]),
Mono.just([
txid : txid,
blockhash: "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f",
height : 100
])
]
}
Upstream upstream = Mock(BitcoinUpstream) {
1 * getHead() >> head
_ * getApi(_) >> Mono.just(api)
}
def status = new TrackBitcoinTx.TxStatus(
txid, false, null, false, null, null, null, 0
)
when:
def act = track.untilMined(upstream, status)
then:
StepVerifier.create(act)
.expectNextMatches { it.mined && it.height == 100 && it.blockHash == "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f" }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Check mempool until found"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
1 * getTx(txid) >> Mono.just([
txid: txid
])
}
Head head = Mock(Head) {
_ * getFlux() >> Flux.empty()
}
CachingMempoolData mempoolAccess = Mock(CachingMempoolData) {
4 * get() >>> [
Mono.just([]),
Mono.just(["4523c7ac0c5c1e5628f025474529c69cd44d7c641db82e6982f5ffe64527efc9"]),
Mono.just(["4523c7ac0c5c1e5628f025474529c69cd44d7c641db82e6982f5ffe64527efc9", txid]),
Mono.just(["4523c7ac0c5c1e5628f025474529c69cd44d7c641db82e6982f5ffe64527efc9", txid]) //second call when started over
]
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getApi(_) >> Mono.just(api)
_ * getHead() >> head
_ * getData() >> Mock(BitcoinData) {
_ * getMempool() >> mempoolAccess
}
}
when:
def steps = StepVerifier.withVirtualTime {
track.untilFound(Chain.BITCOIN, api, upstream, txid).take(1)
}
then:
steps
.expectSubscription()
.expectNoEvent(Duration.ofSeconds(3))
.expectNextMatches { it.found && !it.mined }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
def "Subscribe to an existing tx"() {
setup:
TrackBitcoinTx track = new TrackBitcoinTx(Stub(Upstreams))
def txid = "69cd44d7c641db82e69824523c7ac0c5c1e5628f025474529cf5ffe64527efc9"
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
_ * getTx(txid) >> Mono.just([
txid : txid,
blockhash: "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f",
height : 1
])
_ * getBlock("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f") >> Mono.just([
height : 1,
chainwork: "01",
time : 10000
])
}
def next = Flux.fromIterable([10, 11, 12]).map { h ->
new BlockContainer(h.longValue(), BlockId.from("0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f"), BigInteger.ONE, Instant.now(), false, null, [])
}
Head head = Mock(Head) {
_ * getFlux() >> next
}
BitcoinUpstream upstream = Mock(BitcoinUpstream) {
_ * getApi(_) >> Mono.just(api)
_ * getHead() >> head
}
when:
def act = track.subscribe(Chain.BITCOIN, api, upstream, txid)
then:
StepVerifier.create(act)
.expectNextMatches { it.found && it.mined && it.confirmations == 10 && it.blockHash == "0000000000000000000895d1b9d3898700e1deecc3b0e69f439aa77875e6042f" }
.expectNextMatches { it.found && it.mined && it.confirmations == 11 }
.expectNextMatches { it.found && it.mined && it.confirmations == 12 }
.expectComplete()
.verify(Duration.ofSeconds(1))
}
}

4
src/test/groovy/io/emeraldpay/dshackle/rpc/TrackEthereumAddressSpec.groovy
View File

@@ -67,7 +67,7 @@ class TrackEthereumAddressSpec extends Specification {
apiMock.answer("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0x499602D2")
when:
def flux = trackAddress.getBalance(Mono.just(req))
def flux = trackAddress.getBalance(req)
then:
StepVerifier.create(flux)
.expectNext(exp)
@@ -111,7 +111,7 @@ class TrackEthereumAddressSpec extends Specification {
apiMock.answerOnce("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0x499602D2")
apiMock.answerOnce("eth_getBalance", ["0xe2c8fa8120d813cd0b5e6add120295bf20cfa09f", "latest"], "0xff98")
when:
def flux = trackAddress.subscribe(Mono.just(req))
def flux = trackAddress.subscribe(req)
then:
StepVerifier.create(flux)
.expectNext(exp1)

8
src/test/groovy/io/emeraldpay/dshackle/rpc/TrackEthereumTxSpec.groovy
View File

@@ -103,7 +103,7 @@ class TrackEthereumTxSpec extends Specification {
upstreamMock.nextBlock(BlockContainer.from(blockHeadJson, TestingCommons.objectMapper()))
when:
def flux = trackTx.add(Mono.just(req))
def flux = trackTx.subscribe(req)
then:
StepVerifier.create(flux)
.expectNext(exp1)
@@ -134,7 +134,7 @@ class TrackEthereumTxSpec extends Specification {
when:
def act = StepVerifier.withVirtualTime {
return trackTx.add(Mono.just(req))
return trackTx.subscribe(req)
}
then:
act
@@ -192,7 +192,7 @@ class TrackEthereumTxSpec extends Specification {
when:
def act = StepVerifier.withVirtualTime {
return trackTx.add(Mono.just(req))
return trackTx.subscribe(req)
}
then:
act
@@ -301,7 +301,7 @@ class TrackEthereumTxSpec extends Specification {
}
when:
def flux = trackTx.add(Mono.just(req))
def flux = trackTx.subscribe(req)
then:
StepVerifier.create(flux)
.expectNext(exp1.build())

4
src/test/groovy/io/emeraldpay/dshackle/test/EthereumHeadMock.groovy
View File

@@ -16,12 +16,12 @@
package io.emeraldpay.dshackle.test
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.emeraldpay.dshackle.upstream.Head
import reactor.core.publisher.Flux
import reactor.core.publisher.Mono
import reactor.core.publisher.TopicProcessor
class EthereumHeadMock implements EthereumHead {
class EthereumHeadMock implements Head {
private TopicProcessor<BlockContainer> bus = TopicProcessor.create()
private BlockContainer latest

6
src/test/groovy/io/emeraldpay/dshackle/test/EthereumUpstreamMock.groovy
View File

@@ -17,11 +17,11 @@ package io.emeraldpay.dshackle.test
import io.emeraldpay.dshackle.config.UpstreamsConfig
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.calls.CallMethods
import io.emeraldpay.dshackle.startup.QuorumForLabels
import io.emeraldpay.dshackle.upstream.calls.DefaultEthereumMethods
import io.emeraldpay.dshackle.upstream.ethereum.DirectEthereumApi
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.emeraldpay.dshackle.upstream.ethereum.EthereumUpstream
import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import io.emeraldpay.grpc.Chain
@@ -58,12 +58,12 @@ class EthereumUpstreamMock extends EthereumUpstream {
}
@Override
EthereumHead createHead() {
Head createHead() {
return ethereumHeadMock
}
@Override
EthereumHead getHead() {
Head getHead() {
return ethereumHeadMock
}
}

6
src/test/groovy/io/emeraldpay/dshackle/test/MockServer.groovy → src/test/groovy/io/emeraldpay/dshackle/test/MockGrpcServer.groovy
View File

@@ -21,15 +21,15 @@ import io.grpc.inprocess.InProcessChannelBuilder
import io.grpc.inprocess.InProcessServerBuilder
import io.grpc.testing.GrpcCleanupRule
class MockServer {
class MockGrpcServer {
GrpcCleanupRule grpcCleanup = new GrpcCleanupRule()
ReactorBlockchainGrpc.ReactorBlockchainStub clientForServer(ReactorBlockchainGrpc.BlockchainImplBase impl){
ReactorBlockchainGrpc.ReactorBlockchainStub clientForServer(ReactorBlockchainGrpc.BlockchainImplBase impl) {
String serverName = InProcessServerBuilder.generateName()
grpcCleanup.register(InProcessServerBuilder
.forName(serverName).directExecutor().addService(impl).build().start());
def channel = grpcCleanup.register(InProcessChannelBuilder.forName(serverName).directExecutor().build())
def channel = grpcCleanup.register(InProcessChannelBuilder.forName(serverName).directExecutor().build())
return ReactorBlockchainGrpc.newReactorStub(channel)
}

17
src/test/groovy/io/emeraldpay/dshackle/upstream/CachingEthereumApiSpec.groovy
View File

@@ -10,7 +10,6 @@ import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.data.TxId
import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.ethereum.EthereumHead
import io.infinitape.etherjar.domain.BlockHash
import io.infinitape.etherjar.domain.TransactionId
import io.infinitape.etherjar.rpc.json.BlockJson
@@ -30,7 +29,7 @@ class CachingEthereumApiSpec extends Specification {
def "Get blockNumber from head"() {
setup:
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.default(objectMapper),
@@ -58,7 +57,7 @@ class CachingEthereumApiSpec extends Specification {
def "Return empty if block is not cached"() {
setup:
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.default(objectMapper),
@@ -76,7 +75,7 @@ class CachingEthereumApiSpec extends Specification {
def "Return block by hash when cached"() {
setup:
def cache = new BlocksMemCache();
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(cache).build(),
@@ -106,7 +105,7 @@ class CachingEthereumApiSpec extends Specification {
setup:
def blocksCache = new BlocksMemCache()
def heightCache = new HeightCache()
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setBlockByHeight(heightCache).build(),
@@ -135,7 +134,7 @@ class CachingEthereumApiSpec extends Specification {
setup:
def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache)
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).build(),
@@ -161,7 +160,7 @@ class CachingEthereumApiSpec extends Specification {
setup:
def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache)
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).build(),
@@ -191,7 +190,7 @@ class CachingEthereumApiSpec extends Specification {
def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache)
def heightCache = Mock(HeightCache)
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).setBlockByHeight(heightCache).build(),
@@ -219,7 +218,7 @@ class CachingEthereumApiSpec extends Specification {
def blocksCache = Mock(BlocksMemCache)
def txCache = Mock(TxMemCache)
def heightCache = Mock(HeightCache)
def head = Mock(EthereumHead.class)
def head = Mock(Head.class)
def api = new CachingEthereumApi(
objectMapper,
Caches.newBuilder().setObjectMapper(objectMapper).setBlockByHash(blocksCache).setTxByHash(txCache).setBlockByHeight(heightCache).build(),

86
src/test/groovy/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinRpcClientSpec.groovy Normal file
View File

@@ -0,0 +1,86 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.config.AuthConfig
import org.mockserver.integration.ClientAndServer
import org.mockserver.matchers.Times
import org.mockserver.model.HttpRequest
import org.mockserver.model.HttpResponse
import org.mockserver.model.MediaType
import org.mockserver.verify.VerificationTimes
import reactor.test.StepVerifier
import spock.lang.Shared
import spock.lang.Specification
import java.time.Duration
class BitcoinRpcClientSpec extends Specification {
ClientAndServer mockServer
def setup() {
mockServer = ClientAndServer.startClientAndServer(18332);
}
def cleanup() {
mockServer.stop()
}
def "Make request"() {
setup:
def client = new BitcoinRpcClient("localhost:18332", null)
mockServer.when(
HttpRequest.request()
.withMethod("POST")
.withBody("ping"),
Times.exactly(1)
).respond(
HttpResponse.response()
.withBody("pong")
)
when:
def act = client.execute("ping".bytes).map { new String(it) }
then:
StepVerifier.create(act)
.expectNext("pong")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody("ping")
.withContentType(MediaType.APPLICATION_JSON)
)
}
def "Make request with basic auth"() {
setup:
def auth = new AuthConfig.ClientBasicAuth("user", "passwd")
def client = new BitcoinRpcClient("localhost:18332", auth)
mockServer.when(
HttpRequest.request()
.withMethod("POST")
.withBody("ping")
).respond(
HttpResponse.response()
.withBody("pong")
)
when:
def act = client.execute("ping".bytes).map { new String(it) }
then:
StepVerifier.create(act)
.expectNext("pong")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody("ping")
.withContentType(MediaType.APPLICATION_JSON)
.withHeader("authorization", "Basic dXNlcjpwYXNzd2Q=")
)
}
}

87
src/test/groovy/io/emeraldpay/dshackle/upstream/bitcoin/BitcoinRpcHeadSpec.groovy Normal file
View File

@@ -0,0 +1,87 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.test.TestingCommons
import reactor.core.publisher.Mono
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
class BitcoinRpcHeadSpec extends Specification {
def "Follow 2 blocks created over 3 requests"() {
setup:
String hash1 = "0000000000000000000cf5a5d4dfc4347c0c1a863ec5fdb429b02b2162e50001"
String hash2 = "0000000000000000000cf5a5d4dfc4347c0c1a863ec5fdb429b02b2162e50002"
def block1 = """
{
"hash": "${hash1}",
"confirmations": 2,
"strippedsize": 800464,
"size": 1591897,
"weight": 3993289,
"height": 626472,
"version": 549453824,
"versionHex": "20c00000",
"merkleroot": "7835a26b6c9316232f8194c6b64b0a366e26a9a14e6a7e7409c34594d98a83c2",
"tx": [],
"time": 1587171526,
"mediantime": 1587168930,
"nonce": 4111305375,
"bits": "171320bc",
"difficulty": 14715214060656.53,
"chainwork": "00000000000000000000000000000000000000000e9baec5f63190a2b0bbcf6b",
"nTx": 0,
"previousblockhash": "000000000000000000106b8cb739dd3c412613aa5d459a739794ec2572a74c10"
}
"""
def block2 = """
{
"hash": "${hash2}",
"confirmations": 2,
"strippedsize": 800464,
"size": 1591897,
"weight": 3993289,
"height": 626473,
"version": 549453824,
"versionHex": "20c00000",
"merkleroot": "7835a26b6c9316232f8194c6b64b0a366e26a9a14e6a7e7409c34594d98a83c2",
"tx": [],
"time": 1587172526,
"mediantime": 1587168930,
"nonce": 4111305375,
"bits": "171320bc",
"difficulty": 14715214060656.53,
"chainwork": "00000000000000000000000000000000000000000e9baec5f63190a2b0bbcf6c",
"nTx": 0,
"previousblockhash": "${hash1}"
}
"""
DirectBitcoinApi api = Mock(DirectBitcoinApi) {
_ * executeAndResult(_, "getbestblockhash", _, String) >>> [
Mono.just(hash1), Mono.just(hash1), Mono.just(hash2)
]
_ * execute(_, "getblock", [hash1]) >> Mono.just(block1.bytes)
_ * execute(_, "getblock", [hash2]) >> Mono.just(block2.bytes)
}
BitcoinRpcHead head = new BitcoinRpcHead(api, new ExtractBlock(TestingCommons.objectMapper()), Duration.ofMillis(200))
when:
def act = head.flux.take(2)
head.start()
then:
StepVerifier.create(act)
.expectNextMatches { block ->
block.hash.toHex() == hash1
}.as("Block 1")
.expectNextMatches { block ->
block.hash.toHex() == hash2
}.as("Block 2")
.expectComplete()
.verify(Duration.ofSeconds(3))
}
}

108
src/test/groovy/io/emeraldpay/dshackle/upstream/bitcoin/DirectBitcoinApiSpec.groovy Normal file
View File

@@ -0,0 +1,108 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.test.TestingCommons
import io.infinitape.etherjar.rpc.RpcException
import org.mockserver.integration.ClientAndServer
import org.mockserver.model.HttpRequest
import org.mockserver.model.HttpResponse
import reactor.test.StepVerifier
import spock.lang.Specification
import java.time.Duration
class DirectBitcoinApiSpec extends Specification {
ClientAndServer mockServer
DirectBitcoinApi api
def setup() {
mockServer = ClientAndServer.startClientAndServer(18332);
api = new DirectBitcoinApi(
new BitcoinRpcClient("localhost:18332", null),
TestingCommons.objectMapper(), new DefaultBitcoinMethods(TestingCommons.objectMapper())
)
}
def cleanup() {
mockServer.stop()
}
def "Request simple"() {
setup:
def resp = '{' +
' "result": "0000000000000000000889c2e52ca5e1cecac60bce9a3754201a7a9a67791e90",' +
' "error": null,' +
' "id": 15' +
'}'
mockServer.when(
HttpRequest.request()
).respond(
HttpResponse.response(resp)
)
when:
def act = api.executeAndResult(15, "getbestblockhash", [], String)
then:
StepVerifier.create(act)
.expectNext("0000000000000000000889c2e52ca5e1cecac60bce9a3754201a7a9a67791e90")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody('{"jsonrpc":"2.0","method":"getbestblockhash","params":[],"id":15}')
)
}
def "Request with params"() {
setup:
def resp = '{' +
' "result": "something",' +
' "id": 1' +
'}'
mockServer.when(
HttpRequest.request()
).respond(
HttpResponse.response(resp)
)
when:
def act = api.executeAndResult(1, "getsomething", ["something", false], String)
then:
StepVerifier.create(act)
.expectNext("something")
.expectComplete()
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody('{"jsonrpc":"2.0","method":"getsomething","params":["something",false],"id":1}')
)
}
def "Returns error"() {
setup:
def resp = '{' +
' "result": null,' +
' "error": {' +
' "code": -32601,' +
' "message": "Method not found"' +
' },' +
' "id": 1' +
'}'
mockServer.when(
HttpRequest.request()
).respond(
HttpResponse.response(resp)
)
when:
def act = api.executeAndResult(1, "geterror", [], String)
then:
StepVerifier.create(act)
.expectError(RpcException)
.verify(Duration.ofSeconds(1))
mockServer.verify(
HttpRequest.request()
.withMethod("POST")
.withBody('{"jsonrpc":"2.0","method":"geterror","params":[],"id":1}')
)
}
}

90
src/test/groovy/io/emeraldpay/dshackle/upstream/bitcoin/ExtractBlockSpec.groovy Normal file
View File

@@ -0,0 +1,90 @@
package io.emeraldpay.dshackle.upstream.bitcoin
import io.emeraldpay.dshackle.test.TestingCommons
import spock.lang.Specification
class ExtractBlockSpec extends Specification {
ExtractBlock extractBlock = new ExtractBlock(TestingCommons.objectMapper())
def "Extract standard block"() {
setup:
def json = this.class.getClassLoader().getResourceAsStream("bitcoin/block-626472.json").bytes
when:
def act = extractBlock.extract(json)
then:
act.hash.toHex() == "0000000000000000000889c2e52ca5e1cecac60bce9a3754201a7a9a67791e90"
act.height == 626472
act.timestamp.toString() == "2020-04-18T00:58:46Z"
act.difficulty.toString(16) == "e9baec5f63190a2b0bbcf6b"
!act.full
act.transactions.size() == 1487
act.json == json
}
def "Shouldn't extract time from empty"() {
when:
def act = ExtractBlock.getTime([:])
then:
act == null
}
def "Shouldn't extract time from null"() {
when:
def act = ExtractBlock.getTime([time: null])
then:
act == null
}
def "Extract correct time"() {
expect:
ExtractBlock.getTime(block).toString() == time
where:
time | block
"2020-04-18T00:58:46Z" | [time: 1587171526]
"1970-01-01T00:00:00Z" | [time: 0]
}
def "Shouldn't extract height from empty"() {
when:
def act = ExtractBlock.getHeight([:])
then:
act == null
}
def "Shouldn't extract height from null"() {
when:
def act = ExtractBlock.getHeight([height: null])
then:
act == null
}
def "Should extract height"() {
when:
def act = ExtractBlock.getHeight([height: 123456])
then:
act == 123456L
}
def "Shouldn't extract difficulty from empty"() {
when:
def act = ExtractBlock.getDifficulty([:])
then:
act == null
}
def "Shouldn't extract difficulty from null"() {
when:
def act = ExtractBlock.getDifficulty([chainwork: null])
then:
act == null
}
def "Should extract difficulty"() {
when:
def act = ExtractBlock.getDifficulty([chainwork: "00000000000000000000000000000000000000000ea25fe034fa07aed9e338eb"])
then:
act.toString(16) == "ea25fe034fa07aed9e338eb"
}
}

11
src/test/groovy/io/emeraldpay/dshackle/upstream/ethereum/EthereumHeadLagObserverSpec.groovy
View File

@@ -18,6 +18,7 @@ package io.emeraldpay.dshackle.upstream.ethereum
import com.fasterxml.jackson.databind.ObjectMapper
import io.emeraldpay.dshackle.data.BlockContainer
import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.Head
import io.emeraldpay.dshackle.upstream.HeadLagObserver
import io.emeraldpay.dshackle.upstream.Upstream
import io.infinitape.etherjar.domain.BlockHash
@@ -37,10 +38,10 @@ class EthereumHeadLagObserverSpec extends Specification {
def "Updates lag distance"() {
setup:
EthereumHead master = Mock()
Head master = Mock()
EthereumHead head1 = Mock()
EthereumHead head2 = Mock()
Head head1 = Mock()
Head head2 = Mock()
Upstream up1 = Mock {
_ * getHead() >> head1
@@ -89,7 +90,7 @@ class EthereumHeadLagObserverSpec extends Specification {
def "Probes until there is no difference"() {
setup:
EthereumHead master = Mock()
Head master = Mock()
HeadLagObserver observer = new EthereumHeadLagObserver(master, [])
Upstream up = Mock()
@@ -118,7 +119,7 @@ class EthereumHeadLagObserverSpec extends Specification {
def "Correct distance"() {
setup:
EthereumHead master = Mock()
Head master = Mock()
HeadLagObserver observer = new EthereumHeadLagObserver(master, [])
expect:
def top = new BlockJson().with {

4
src/test/groovy/io/emeraldpay/dshackle/upstream/grpc/EthereumGrpcUpstreamSpec.groovy
View File

@@ -21,7 +21,7 @@ import io.emeraldpay.api.proto.BlockchainGrpc
import io.emeraldpay.api.proto.BlockchainOuterClass
import io.emeraldpay.api.proto.Common
import io.emeraldpay.dshackle.data.BlockId
import io.emeraldpay.dshackle.test.MockServer
import io.emeraldpay.dshackle.test.MockGrpcServer
import io.emeraldpay.dshackle.test.TestingCommons
import io.emeraldpay.dshackle.upstream.UpstreamAvailability
import io.emeraldpay.grpc.Chain
@@ -38,7 +38,7 @@ import java.util.concurrent.CompletableFuture
class EthereumGrpcUpstreamSpec extends Specification {
MockServer mockServer = new MockServer()
MockGrpcServer mockServer = new MockGrpcServer()
ObjectMapper objectMapper = TestingCommons.objectMapper()
def "Subscribe to head"() {

1509
src/test/resources/bitcoin/block-626472.json Normal file
View File

File diff suppressed because it is too large Load Diff

434
src/test/resources/bitcoin/unspent-one-addr.json Normal file
View File

@@ -0,0 +1,434 @@
[
{
"txid": "e0f946c8f971b25cdffa64eed71d886019e437c0bf6a1b280584c0be5d1b5409",
"vout": 29,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01230030,
"confirmations": 2010,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "66e1e4d14ed6f454d2fda036f35cba423274ecdf5d46deb93f172c412a0f650d",
"vout": 83,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00756339,
"confirmations": 4963,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "4b30a92f5567dfbbd13b48e51c4ec1d97610daa8a1e008c428beb136e5f5670d",
"vout": 25,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00955798,
"confirmations": 111,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "507cb7191a10eaab9e2b4e223c85ef1ddd6b3aafbdfb990fc6e5e2dadf12af0e",
"vout": 10,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00289985,
"confirmations": 4840,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "7edc6cf0b680b4e127997c4b5d57320fda314d61b1ed23227cf496610e363410",
"vout": 54,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01003612,
"confirmations": 5213,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "0ae8671a15775d8eac990c92b687175a6b154fdcc4f516b0f100f813b2c9d011",
"vout": 93,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00973772,
"confirmations": 1213,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "f91b5bfd4a104baafeb0c5a62f1713580ab2d64b2c6d2fdfcd7818747645fc14",
"vout": 19,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01218703,
"confirmations": 278,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "7f9ee14424c4b52a2aa5c2b5df199fde2873a56f6ebd07d835722d2fd881b828",
"vout": 94,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00635688,
"confirmations": 3214,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6a278d265ce25c8519ad7af4b115eb8d6d5ca89957d0638f4567c2629c11d82a",
"vout": 48,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00529192,
"confirmations": 2472,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "80c673330e0c4a3b981f9acad07b5194b44c36d683cf8e0687f5faf6780a682c",
"vout": 14,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00850915,
"confirmations": 3066,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "a032d978409dbd6de66792a78147135903f1a4731781e8e5069a504cc43a8c32",
"vout": 8,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01047700,
"confirmations": 4628,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6dffd0651ae1bc29f43b864c1a097d4ecca0bd16341d13be9e32b121e6f6ca3e",
"vout": 36,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00605451,
"confirmations": 2763,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "42e10b12c32f4d6ba91b144b9c5afcccd6bcaf3f8f946d1f81a891c3d47e4346",
"vout": 78,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00982508,
"confirmations": 3363,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "70d83eda5f5024a5a7d2de7753ea0a7aa3a7cc5496ebbf9ea044efa2cd35cd47",
"vout": 55,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00625446,
"confirmations": 5605,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "253351e033465057d96e54a4fff130853782ae52dddef51486d3d7dc55e07b4c",
"vout": 74,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00720758,
"confirmations": 1539,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "39ad1738af322b6049e386a01e05fe8a6479913bd266268634eeadefdfd28c61",
"vout": 14,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01023716,
"confirmations": 4158,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "67c9e726a7acc9e7968975b150429e9d593f8e20010c03aeee58b84d2bce2b64",
"vout": 8,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01104307,
"confirmations": 889,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "2e731be5a74b5a321055a8b1ebc232c9316b275315ceaa945bad021c6e2a1865",
"vout": 47,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00679256,
"confirmations": 5081,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "88d48dae117aa8de4af3c8cde5f4d57d0e7a463d769b74379ea920474c6b527b",
"vout": 14,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00876837,
"confirmations": 730,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "c7c47da9165734c7745c13450fd61ec50c6638d1d6e58546f368c0acbe73877e",
"vout": 63,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01034200,
"confirmations": 2904,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "fe123697bc5f4ad079e81cdccbbb5834c240072f2874c8a43c68456dd8ef7a84",
"vout": 49,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00792699,
"confirmations": 4724,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6df879ac8f8f2b6fef79b71098fff03cc74085392fb2bbd4b77c14bafdefc497",
"vout": 35,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01313314,
"confirmations": 1378,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "05b2a9ae5e2d150ef042af1fe62a67652298b885d0fbe11f1be4ce850fcecfa6",
"vout": 60,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00754670,
"confirmations": 4404,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "c9590ed1299e22c2e4e1275a953258c9cb7052f87f5e7de7decb055248a06eba",
"vout": 8,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00794840,
"confirmations": 1068,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "6d6dd53d70fa024011ca1a887686d6c0bbf19c739ed5017cad1adb77de7cbdbe",
"vout": 64,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01151435,
"confirmations": 3497,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "31ecd41422208033526919fee8a028a1de915da1308728ff8328a53f7f7d59cd",
"vout": 66,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00987020,
"confirmations": 2608,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "5963e589736bea2b4a438cf7cede084fbb24f9b545fa7558ad407510260805ce",
"vout": 68,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01076674,
"confirmations": 3915,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "df6a74ad3d4af01ad76937b43ac268d9775d447c97f5cc34c2b12c3b405e61cf",
"vout": 50,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00493433,
"confirmations": 4518,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "768d3e2f525d8ed24c446463f163fb9b9c7af10e07097efe0eecc8600b7083d6",
"vout": 74,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01433360,
"confirmations": 1696,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "f0eb2c2e854ecae7d3bbbdf06ef25de25c78983466be16650c030be3c558ebd6",
"vout": 58,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01098392,
"confirmations": 405,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "a73edf99d17868fd458520308febc71609e1fecd686530c525000359866a4ed9",
"vout": 35,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00902924,
"confirmations": 4037,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "aea6c1f4b231ab6900332c82fb9d48d24aac2245575e53f5fd7d6e86a7ea02dc",
"vout": 4,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00954847,
"confirmations": 563,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "366166e6a1a2d7b99d44834e2d4c2c5988fb8da2e17756f21b8e4d2c0ac972e1",
"vout": 88,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01032596,
"confirmations": 4275,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "ac53f2a2052396354333e0e500864f21ac0909c4159f46880e0f9128ac7194e2",
"vout": 19,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01480214,
"confirmations": 1848,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "63b840a8abf477305dfad93db69205ff01a58e1527a317dbcc8a360533c0e9e2",
"vout": 60,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.00412783,
"confirmations": 3822,
"spendable": false,
"solvable": false,
"safe": true
},
{
"txid": "f14b222e652c58d11435fa9172ddea000c6f5e20e6b715eb940fc28d1c4adeef",
"vout": 58,
"address": "1K7xkspJg7DDKNwzXgoRSDCUxiFsRegsSK",
"label": "",
"scriptPubKey": "76a914c6c34d94969bccf7faaa8e5e5bb739e4d68ee9f888ac",
"amount": 0.01105047,
"confirmations": 2139,
"spendable": false,
"solvable": false,
"safe": true
}
]

4514
src/test/resources/bitcoin/unspent-two-addr.json Normal file
View File

File diff suppressed because it is too large Load Diff