Using the zmq notifications to avoid excessive mempool polling can be difficult
given the current notifications available. It announces all transactions
being added to mempool or included in blocks, but announces no evictions
and gives no indication if the transaction is in the mempool or a block.
Block notifications for zmq are also substandard, in that it only announces
block tips, while all block transactions are still announced.
This commit adds a unified stream which can be used to closely track mempool:
1) getrawmempool to fill out mempool knowledge
2) if txhash is announced, add or remove from set
based on add/remove flag
3) if blockhash is announced, get block txn list,
remove from those transactions local view of mempool
4) if we drop a sequence number, go to (1)
The mempool sequence number starts at the value 1, and
increments each time a transaction enters the mempool,
or is evicted from the mempool for any reason, including
block inclusion. The mempool sequence number is published
via ZMQ for any transaction-related notification.
These features allow for ZMQ/RPC consumer to track mempool
state in a more exacting way, without unnecesarily polling
getrawmempool. See interface_zmq.py::test_mempool_sync for
example usage.
This is a pure refactoring of zmqnotificationinterface to make the
code easier to read and maintain. It replaces explicit iterators
with C++11 for-each loops where appropriate and uses std::unique_ptr
to make memory ownership more explicit.
To do so we update CValidationInterface::BlockDisconnect to take a
CBlockIndex pointing to the block being disconnected.
This new parameter will be use in the following commit to establish
wallet height.
This adds a new RPC method "getzmqnotifications", which returns
information about all active ZMQ notification endpoints. This is useful
for software that layers on top of bitcoind, so it can verify that
ZeroMQ is enabled and also figure out where it should listen.
See https://github.com/bitcoin/bitcoin/issues/13526.
This moves the used instance of CZMQNotificationInterface from a static
variable in init.cpp to a globally-accessible one declared in
zmq/zmqnotificationinterface.h. The variable is also renamed to
g_zmq_notification_interface, to be consistent with other globals.
We need this to implement a new RPC method "getzmqnotifications" (see
https://github.com/bitcoin/bitcoin/issues/13526) in a follow up.
This simplifies fixing the wallet-returns-stale-info issue as we
can now hold cs_wallet across an entire block instead of only
per-tx (though we only actually do so in the next commit).
This change also removes the NOT_IN_BLOCK constant in favor of only
passing the CBlockIndex* parameter to SyncTransactions when a new
block is being connected, instead of also when a block is being
disconnected.
This change adds a parameter to BlockConnectedDisconnected which
lists the transactions which were removed from mempool due to
confliction as a result of this operation. While its somewhat of a
shame to make block-validation-logic generate a list of mempool
changes to be included in its generated callbacks, fixing this isnt
too hard.
Further in this change-set, CValidationInterface starts listening
to mempool directly, placing it in the middle and giving it a bit
of logic to know how to route notifications from block-validation,
mempool, etc (though not listening for conflicted-removals yet).
In anticipation of making all the callbacks out of block processing
flow through it. Note that vHashes will always have something in it
since pindexFork != pindexNewTip.
Moves the call Initialize() from init.cpp to CreateWithArguments() and handles the
return value. Moves the call Shutdown() from init.cpp to destructor.
Changes Initialize() and Shutdown() to protected members.
- removes mapBlockIndex find operation
- theoretically allows removing the cs_main lock during zqm notification while introducing a new file position lock
Continues Johnathan Corgan's work.
Publishing multipart messages
Bugfix: Add missing zmq header includes
Bugfix: Adjust build system to link ZeroMQ code for Qt binaries
