phantom/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-01-10 03:47:29 -03:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge bitcoin/bitcoin#25290: [kernel 3a/n] Decouple CTxMemPool from ArgsManager

Browse source 
d1684beabe fees: Pass in a filepath instead of referencing gArgs (Carl Dong)
9a3d825c30 init: Remove redundant -*mempool*, -limit* queries (Carl Dong)
6c5c60c412 mempool: Use m_limit for UpdateTransactionsFromBlock (Carl Dong)
9e93b10301 node/ifaces: Use existing MemPoolLimits (Carl Dong)
38af2bcf35 mempoolaccept: Use limits from mempool in constructor (Carl Dong)
9333427014 mempool: Introduce (still-unused) MemPoolLimits (Carl Dong)
716bb5fbd3 scripted-diff: Rename anc/desc size limit vars to indicate SI unit (Carl Dong)
1ecc77321d scripted-diff: Rename DEFAULT_MEMPOOL_EXPIRY to indicate time unit (Carl Dong)
aa9141cd81 mempool: Pass in -mempoolexpiry instead of referencing gArgs (Carl Dong)
51c7a41a5e init: Only determine maxmempool once (Carl Dong)
386c9472c8 mempool: Make GetMinFee() with custom size protected (Carl Dong)
82f00de7a6 mempool: Pass in -maxmempool instead of referencing gArgs (Carl Dong)
f1941e8bfd pool: Add and use MemPoolOptions, ApplyArgsManOptions (Carl Dong)
0199bd35bb fuzz/rbf: Add missing TestingSetup (Carl Dong)
ccbaf546a6 scripted-diff: Rename DEFAULT_MAX_MEMPOOL_SIZE to indicate SI unit (Carl Dong)
fc02f77ca6 ArgsMan: Add Get*Arg functions returning optional (Carl Dong)

Pull request description:

  This is part of the `libbitcoinkernel` project: #24303, https://github.com/bitcoin/bitcoin/projects/18

  -----

  As mentioned in the Stage 1 Step 2 description of [the `libbitcoinkernel` project](https://github.com/bitcoin/bitcoin/issues/24303), `ArgsManager` will not be part of `libbitcoinkernel`. Therefore, it is important that we remove any dependence on `ArgsManager` by code that will be part of `libbitcoinkernel`. This is the first in a series of PRs aiming to achieve this.

  This PR removes `CTxMemPool+MempoolAccept`'s dependency on `ArgsManager` by introducing a `CTxMemPool::Options` struct, which is used to specify `CTxMemPool`'s various options at construction time.

  These options are:
  - `-maxmempool` -> `CTxMemPool::Options::max_size`
  - `-mempoolexpiry` -> `CTxMemPool::Options::expiry`
  - `-limitancestorcount` -> `CTxMemPool::Options::limits::ancestor_count`
  - `-limitancestorsize` -> `CTxMemPool::Options::limits::ancestor_size`
  - `-limitdescendantcount` -> `CTxMemPool::Options::limits::descendant_count`
  - `-limitdescendantsize` -> `CTxMemPool::Options::limits::descendant_size`

  More context can be gleaned from the commit messages. The important commits are:

  - 56eb479ded8bfb2ef635bb6f3b484f9d5952c70d "pool: Add and use MemPoolOptions, ApplyArgsManOptions"
  - a1e08b70f3068f4e8def1c630d8f50cd54da7832 "mempool: Pass in -maxmempool instead of referencing gArgs"
  - 6f4bf3ede5812b374828f08fc728ceded2f10024 "mempool: Pass in -mempoolexpiry instead of referencing gArgs"
  - 5958a7fe4806599fc620ee8c1a881ca10fa2dd16 "mempool: Introduce (still-unused) MemPoolLimits"

  Reviewers: Help needed in the following commits (see commit messages):
  - a1e08b70f3068f4e8def1c630d8f50cd54da7832 "mempool: Pass in -maxmempool instead of referencing gArgs"
  - 0695081a797e9a5d7787b78b0f8289dafcc6bff7 "node/ifaces: Use existing MemPoolLimits"

  Note to Reviewers: There are perhaps an infinite number of ways to architect `CTxMemPool::Options`, the current one tries to keep it simple, usable, and flexible. I hope we don't spend too much time arguing over the design here since that's not the point. In the case that you're 100% certain that a different design is strictly better than this one in every regard, please show us a fully-implemented branch.

  -----

  TODO:
  - [x] Use the more ergonomic `CTxMemPool::Options` where appropriate
  - [x] Doxygen comments for `ApplyArgsManOptions`, `MemPoolOptions`

  -----

  Questions for Reviewers:
  1. Should we use `std::chrono::seconds` for `CTxMemPool::Options::expiry` and `CTxMemPool::m_expiry` instead of an `int64_t`? Something else? (`std::chrono::hours`?)
  2. Should I merge `CTxMemPool::Limits` inside `CTxMemPool::Options`?

ACKs for top commit:
  MarcoFalke:
    ACK d1684beabe 🍜
  ryanofsky:
    Code review ACK d1684beabe. Just minor cleanups since last review, mostly switching to brace initialization

Tree-SHA512: 2c138e52d69f61c263f1c3648f01c801338a8f576762c815f478ef5148b8b2f51e91ded5c1be915e678c0b14f6cfba894b82afec58d999d39a7bb7c914736e0b
This commit is contained in:
MacroFake 2022-06-29 09:13:18 +02:00
commit e4e201dfd9
No known key found for this signature in database
GPG key ID: CE2B75697E69A548
31 changed files with 398 additions and 118 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

6
src/Makefile.am
View file

@ -173,11 +173,14 @@ BITCOIN_CORE_H = \
kernel/checks.h \
kernel/coinstats.h \
kernel/context.h \
kernel/mempool_limits.h \
kernel/mempool_options.h \
key.h \
key_io.h \
logging.h \
logging/timer.h \
mapport.h \
mempool_args.h \
memusage.h \
merkleblock.h \
net.h \
@ -203,6 +206,7 @@ BITCOIN_CORE_H = \
outputtype.h \
policy/feerate.h \
policy/fees.h \
policy/fees_args.h \
policy/packages.h \
policy/policy.h \
policy/rbf.h \
@ -361,6 +365,7 @@ libbitcoin_node_a_SOURCES = \
kernel/coinstats.cpp \
kernel/context.cpp \
mapport.cpp \
mempool_args.cpp \
net.cpp \
netgroup.cpp \
net_processing.cpp \
@ -377,6 +382,7 @@ libbitcoin_node_a_SOURCES = \
node/interface_ui.cpp \
noui.cpp \
policy/fees.cpp \
policy/fees_args.cpp \
policy/packages.cpp \
policy/rbf.cpp \
policy/settings.cpp \

38
src/init.cpp
View file

@ -30,6 +30,7 @@
#include <interfaces/init.h>
#include <interfaces/node.h>
#include <mapport.h>
#include <mempool_args.h>
#include <net.h>
#include <net_permissions.h>
#include <net_processing.h>
@ -39,10 +40,11 @@
#include <node/caches.h>
#include <node/chainstate.h>
#include <node/context.h>
#include <node/miner.h>
#include <node/interface_ui.h>
#include <node/miner.h>
#include <policy/feerate.h>
#include <policy/fees.h>
#include <policy/fees_args.h>
#include <policy/policy.h>
#include <policy/settings.h>
#include <protocol.h>
@ -62,6 +64,7 @@
#include <txorphanage.h>
#include <util/asmap.h>
#include <util/check.h>
#include <util/designator.h>
#include <util/moneystr.h>
#include <util/strencodings.h>
#include <util/string.h>
@ -413,9 +416,9 @@ void SetupServerArgs(ArgsManager& argsman)
argsman.AddArg("-dbcache=<n>", strprintf("Maximum database cache size <n> MiB (%d to %d, default: %d). In addition, unused mempool memory is shared for this cache (see -maxmempool).", nMinDbCache, nMaxDbCache, nDefaultDbCache), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-includeconf=<file>", "Specify additional configuration file, relative to the -datadir path (only useable from configuration file, not command line)", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-loadblock=<file>", "Imports blocks from external file on startup", ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-maxmempool=<n>", strprintf("Keep the transaction memory pool below <n> megabytes (default: %u)", DEFAULT_MAX_MEMPOOL_SIZE), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-maxmempool=<n>", strprintf("Keep the transaction memory pool below <n> megabytes (default: %u)", DEFAULT_MAX_MEMPOOL_SIZE_MB), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-maxorphantx=<n>", strprintf("Keep at most <n> unconnectable transactions in memory (default: %u)", DEFAULT_MAX_ORPHAN_TRANSACTIONS), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-mempoolexpiry=<n>", strprintf("Do not keep transactions in the mempool longer than <n> hours (default: %u)", DEFAULT_MEMPOOL_EXPIRY), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-mempoolexpiry=<n>", strprintf("Do not keep transactions in the mempool longer than <n> hours (default: %u)", DEFAULT_MEMPOOL_EXPIRY_HOURS), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
argsman.AddArg("-minimumchainwork=<hex>", strprintf("Minimum work assumed to exist on a valid chain in hex (default: %s, testnet: %s, signet: %s)", defaultChainParams->GetConsensus().nMinimumChainWork.GetHex(), testnetChainParams->GetConsensus().nMinimumChainWork.GetHex(), signetChainParams->GetConsensus().nMinimumChainWork.GetHex()), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::OPTIONS);
argsman.AddArg("-par=<n>", strprintf("Set the number of script verification threads (%u to %d, 0 = auto, <0 = leave that many cores free, default: %d)",
-GetNumCores(), MAX_SCRIPTCHECK_THREADS, DEFAULT_SCRIPTCHECK_THREADS), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS);
@ -536,9 +539,9 @@ void SetupServerArgs(ArgsManager& argsman)
argsman.AddArg("-stopafterblockimport", strprintf("Stop running after importing blocks from disk (default: %u)", DEFAULT_STOPAFTERBLOCKIMPORT), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-stopatheight", strprintf("Stop running after reaching the given height in the main chain (default: %u)", DEFAULT_STOPATHEIGHT), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-limitancestorcount=<n>", strprintf("Do not accept transactions if number of in-mempool ancestors is <n> or more (default: %u)", DEFAULT_ANCESTOR_LIMIT), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-limitancestorsize=<n>", strprintf("Do not accept transactions whose size with all in-mempool ancestors exceeds <n> kilobytes (default: %u)", DEFAULT_ANCESTOR_SIZE_LIMIT), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-limitancestorsize=<n>", strprintf("Do not accept transactions whose size with all in-mempool ancestors exceeds <n> kilobytes (default: %u)", DEFAULT_ANCESTOR_SIZE_LIMIT_KVB), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-limitdescendantcount=<n>", strprintf("Do not accept transactions if any ancestor would have <n> or more in-mempool descendants (default: %u)", DEFAULT_DESCENDANT_LIMIT), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-limitdescendantsize=<n>", strprintf("Do not accept transactions if any ancestor would have more than <n> kilobytes of in-mempool descendants (default: %u).", DEFAULT_DESCENDANT_SIZE_LIMIT), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-limitdescendantsize=<n>", strprintf("Do not accept transactions if any ancestor would have more than <n> kilobytes of in-mempool descendants (default: %u).", DEFAULT_DESCENDANT_SIZE_LIMIT_KVB), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-addrmantest", "Allows to test address relay on localhost", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-capturemessages", "Capture all P2P messages to disk", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
argsman.AddArg("-mocktime=<n>", "Replace actual time with " + UNIX_EPOCH_TIME + " (default: 0)", ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::DEBUG_TEST);
@ -928,11 +931,6 @@ bool AppInitParameterInteraction(const ArgsManager& args, bool use_syscall_sandb
LogPrintf("Warning: nMinimumChainWork set below default value of %s\n", chainparams.GetConsensus().nMinimumChainWork.GetHex());
}
// mempool limits
int64_t nMempoolSizeMax = args.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000;
int64_t nMempoolSizeMin = args.GetIntArg("-limitdescendantsize", DEFAULT_DESCENDANT_SIZE_LIMIT) * 1000 * 40;
if (nMempoolSizeMax < 0 || nMempoolSizeMax < nMempoolSizeMin)
return InitError(strprintf(_("-maxmempool must be at least %d MB"), std::ceil(nMempoolSizeMin / 1000000.0)));
// incremental relay fee sets the minimum feerate increase necessary for BIP 125 replacement in the mempool
// and the amount the mempool min fee increases above the feerate of txs evicted due to mempool limiting.
if (args.IsArgSet("-incrementalrelayfee")) {
@ -1294,7 +1292,7 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
assert(!node.fee_estimator);
// Don't initialize fee estimation with old data if we don't relay transactions,
// as they would never get updated.
if (!ignores_incoming_txs) node.fee_estimator = std::make_unique<CBlockPolicyEstimator>();
if (!ignores_incoming_txs) node.fee_estimator = std::make_unique<CBlockPolicyEstimator>(FeeestPath(args));
// sanitize comments per BIP-0014, format user agent and check total size
std::vector<std::string> uacomments;
@ -1411,7 +1409,6 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
// cache size calculations
CacheSizes cache_sizes = CalculateCacheSizes(args, g_enabled_filter_types.size());
int64_t nMempoolSizeMax = args.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000;
LogPrintf("Cache configuration:\n");
LogPrintf("* Using %.1f MiB for block index database\n", cache_sizes.block_tree_db * (1.0 / 1024 / 1024));
if (args.GetBoolArg("-txindex", DEFAULT_TXINDEX)) {
@ -1422,14 +1419,25 @@ bool AppInitMain(NodeContext& node, interfaces::BlockAndHeaderTipInfo* tip_info)
cache_sizes.filter_index * (1.0 / 1024 / 1024), BlockFilterTypeName(filter_type));
}
LogPrintf("* Using %.1f MiB for chain state database\n", cache_sizes.coins_db * (1.0 / 1024 / 1024));
LogPrintf("* Using %.1f MiB for in-memory UTXO set (plus up to %.1f MiB of unused mempool space)\n", cache_sizes.coins * (1.0 / 1024 / 1024), nMempoolSizeMax * (1.0 / 1024 / 1024));
assert(!node.mempool);
assert(!node.chainman);
const int mempool_check_ratio = std::clamp<int>(args.GetIntArg("-checkmempool", chainparams.DefaultConsistencyChecks() ? 1 : 0), 0, 1000000);
CTxMemPool::Options mempool_opts{
Desig(estimator) node.fee_estimator.get(),
Desig(check_ratio) chainparams.DefaultConsistencyChecks() ? 1 : 0,
};
ApplyArgsManOptions(args, mempool_opts);
mempool_opts.check_ratio = std::clamp<int>(mempool_opts.check_ratio, 0, 1'000'000);
int64_t descendant_limit_bytes = mempool_opts.limits.descendant_size_vbytes * 40;
if (mempool_opts.max_size_bytes < 0 || mempool_opts.max_size_bytes < descendant_limit_bytes) {
return InitError(strprintf(_("-maxmempool must be at least %d MB"), std::ceil(descendant_limit_bytes / 1'000'000.0)));
}
LogPrintf("* Using %.1f MiB for in-memory UTXO set (plus up to %.1f MiB of unused mempool space)\n", cache_sizes.coins * (1.0 / 1024 / 1024), mempool_opts.max_size_bytes * (1.0 / 1024 / 1024));
for (bool fLoaded = false; !fLoaded && !ShutdownRequested();) {
node.mempool = std::make_unique<CTxMemPool>(node.fee_estimator.get(), mempool_check_ratio);
node.mempool = std::make_unique<CTxMemPool>(mempool_opts);
const ChainstateManager::Options chainman_opts{
chainparams,

30
src/kernel/mempool_limits.h Normal file
View file

@ -0,0 +1,30 @@
// Copyright (c) 2022 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_KERNEL_MEMPOOL_LIMITS_H
#define BITCOIN_KERNEL_MEMPOOL_LIMITS_H
#include <policy/policy.h>
#include <cstdint>
namespace kernel {
/**
* Options struct containing limit options for a CTxMemPool. Default constructor
* populates the struct with sane default values which can be modified.
*
* Most of the time, this struct should be referenced as CTxMemPool::Limits.
*/
struct MemPoolLimits {
//! The maximum allowed number of transactions in a package including the entry and its ancestors.
int64_t ancestor_count{DEFAULT_ANCESTOR_LIMIT};
//! The maximum allowed size in virtual bytes of an entry and its ancestors within a package.
int64_t ancestor_size_vbytes{DEFAULT_ANCESTOR_SIZE_LIMIT_KVB * 1'000};
//! The maximum allowed number of transactions in a package including the entry and its descendants.
int64_t descendant_count{DEFAULT_DESCENDANT_LIMIT};
//! The maximum allowed size in virtual bytes of an entry and its descendants within a package.
int64_t descendant_size_vbytes{DEFAULT_DESCENDANT_SIZE_LIMIT_KVB * 1'000};
};
} // namespace kernel
#endif // BITCOIN_KERNEL_MEMPOOL_LIMITS_H

38
src/kernel/mempool_options.h Normal file
View file

@ -0,0 +1,38 @@
// Copyright (c) 2022 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_KERNEL_MEMPOOL_OPTIONS_H
#define BITCOIN_KERNEL_MEMPOOL_OPTIONS_H
#include <kernel/mempool_limits.h>
#include <chrono>
#include <cstdint>
class CBlockPolicyEstimator;
/** Default for -maxmempool, maximum megabytes of mempool memory usage */
static constexpr unsigned int DEFAULT_MAX_MEMPOOL_SIZE_MB{300};
/** Default for -mempoolexpiry, expiration time for mempool transactions in hours */
static constexpr unsigned int DEFAULT_MEMPOOL_EXPIRY_HOURS{336};
namespace kernel {
/**
* Options struct containing options for constructing a CTxMemPool. Default
* constructor populates the struct with sane default values which can be
* modified.
*
* Most of the time, this struct should be referenced as CTxMemPool::Options.
*/
struct MemPoolOptions {
/* Used to estimate appropriate transaction fees. */
CBlockPolicyEstimator* estimator{nullptr};
/* The ratio used to determine how often sanity checks will run. */
int check_ratio{0};
int64_t max_size_bytes{DEFAULT_MAX_MEMPOOL_SIZE_MB * 1'000'000};
std::chrono::seconds expiry{std::chrono::hours{DEFAULT_MEMPOOL_EXPIRY_HOURS}};
MemPoolLimits limits{};
};
} // namespace kernel
#endif // BITCOIN_KERNEL_MEMPOOL_OPTIONS_H

37
src/mempool_args.cpp Normal file
View file

@ -0,0 +1,37 @@
// Copyright (c) 2022 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <mempool_args.h>
#include <kernel/mempool_limits.h>
#include <kernel/mempool_options.h>
#include <util/system.h>
using kernel::MemPoolLimits;
using kernel::MemPoolOptions;
namespace {
void ApplyArgsManOptions(const ArgsManager& argsman, MemPoolLimits& mempool_limits)
{
mempool_limits.ancestor_count = argsman.GetIntArg("-limitancestorcount", mempool_limits.ancestor_count);
if (auto vkb = argsman.GetIntArg("-limitancestorsize")) mempool_limits.ancestor_size_vbytes = *vkb * 1'000;
mempool_limits.descendant_count = argsman.GetIntArg("-limitdescendantcount", mempool_limits.descendant_count);
if (auto vkb = argsman.GetIntArg("-limitdescendantsize")) mempool_limits.descendant_size_vbytes = *vkb * 1'000;
}
}
void ApplyArgsManOptions(const ArgsManager& argsman, MemPoolOptions& mempool_opts)
{
mempool_opts.check_ratio = argsman.GetIntArg("-checkmempool", mempool_opts.check_ratio);
if (auto mb = argsman.GetIntArg("-maxmempool")) mempool_opts.max_size_bytes = *mb * 1'000'000;
if (auto hours = argsman.GetIntArg("-mempoolexpiry")) mempool_opts.expiry = std::chrono::hours{*hours};
ApplyArgsManOptions(argsman, mempool_opts.limits);
}

22
src/mempool_args.h Normal file
View file

@ -0,0 +1,22 @@
// Copyright (c) 2022 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_MEMPOOL_ARGS_H
#define BITCOIN_MEMPOOL_ARGS_H
class ArgsManager;
namespace kernel {
struct MemPoolOptions;
};
/**
* Overlay the options set in \p argsman on top of corresponding members in \p mempool_opts.
*
* @param[in] argsman The ArgsManager in which to check set options.
* @param[in,out] mempool_opts The MemPoolOptions to modify according to \p argsman.
*/
void ApplyArgsManOptions(const ArgsManager& argsman, kernel::MemPoolOptions& mempool_opts);
#endif // BITCOIN_MEMPOOL_ARGS_H

2
src/net_processing.cpp
View file

@ -4604,7 +4604,7 @@ void PeerManagerImpl::MaybeSendFeefilter(CNode& pto, Peer& peer, std::chrono::mi
// transactions to us, regardless of feefilter state.
if (pto.IsBlockOnlyConn()) return;
CAmount currentFilter = m_mempool.GetMinFee(gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000).GetFeePerK();
CAmount currentFilter = m_mempool.GetMinFee().GetFeePerK();
static FeeFilterRounder g_filter_rounder{CFeeRate{DEFAULT_MIN_RELAY_TX_FEE}};
if (m_chainman.ActiveChainstate().IsInitialBlockDownload()) {

19
src/node/interfaces.cpp
View file

@ -653,8 +653,12 @@ public:
}
void getPackageLimits(unsigned int& limit_ancestor_count, unsigned int& limit_descendant_count) override
{
limit_ancestor_count = gArgs.GetIntArg("-limitancestorcount", DEFAULT_ANCESTOR_LIMIT);
limit_descendant_count = gArgs.GetIntArg("-limitdescendantcount", DEFAULT_DESCENDANT_LIMIT);
const CTxMemPool::Limits default_limits{};
const CTxMemPool::Limits& limits{m_node.mempool ? m_node.mempool->m_limits : default_limits};
limit_ancestor_count = limits.ancestor_count;
limit_descendant_count = limits.descendant_count;
}
bool checkChainLimits(const CTransactionRef& tx) override
{
@ -662,15 +666,12 @@ public:
LockPoints lp;
CTxMemPoolEntry entry(tx, 0, 0, 0, false, 0, lp);
CTxMemPool::setEntries ancestors;
auto limit_ancestor_count = gArgs.GetIntArg("-limitancestorcount", DEFAULT_ANCESTOR_LIMIT);
auto limit_ancestor_size = gArgs.GetIntArg("-limitancestorsize", DEFAULT_ANCESTOR_SIZE_LIMIT) * 1000;
auto limit_descendant_count = gArgs.GetIntArg("-limitdescendantcount", DEFAULT_DESCENDANT_LIMIT);
auto limit_descendant_size = gArgs.GetIntArg("-limitdescendantsize", DEFAULT_DESCENDANT_SIZE_LIMIT) * 1000;
const CTxMemPool::Limits& limits{m_node.mempool->m_limits};
std::string unused_error_string;
LOCK(m_node.mempool->cs);
return m_node.mempool->CalculateMemPoolAncestors(
entry, ancestors, limit_ancestor_count, limit_ancestor_size,
limit_descendant_count, limit_descendant_size, unused_error_string);
entry, ancestors, limits.ancestor_count, limits.ancestor_size_vbytes,
limits.descendant_count, limits.descendant_size_vbytes, unused_error_string);
}
CFeeRate estimateSmartFee(int num_blocks, bool conservative, FeeCalculation* calc) override
{
@ -685,7 +686,7 @@ public:
CFeeRate mempoolMinFee() override
{
if (!m_node.mempool) return {};
return m_node.mempool->GetMinFee(gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000);
return m_node.mempool->GetMinFee();
}
CFeeRate relayMinFee() override { return ::minRelayTxFee; }
CFeeRate relayIncrementalFee() override { return ::incrementalRelayFee; }

16
src/policy/fees.cpp
View file

@ -31,8 +31,6 @@
#include <stdexcept>
#include <utility>
static const char* FEE_ESTIMATES_FILENAME = "fee_estimates.dat";
static constexpr double INF_FEERATE = 1e99;
std::string StringForFeeEstimateHorizon(FeeEstimateHorizon horizon)
@ -529,8 +527,8 @@ bool CBlockPolicyEstimator::_removeTx(const uint256& hash, bool inBlock)
}
}
CBlockPolicyEstimator::CBlockPolicyEstimator()
: nBestSeenHeight(0), firstRecordedHeight(0), historicalFirst(0), historicalBest(0), trackedTxs(0), untrackedTxs(0)
CBlockPolicyEstimator::CBlockPolicyEstimator(const fs::path& estimation_filepath)
: m_estimation_filepath{estimation_filepath}, nBestSeenHeight{0}, firstRecordedHeight{0}, historicalFirst{0}, historicalBest{0}, trackedTxs{0}, untrackedTxs{0}
{
static_assert(MIN_BUCKET_FEERATE > 0, "Min feerate must be nonzero");
size_t bucketIndex = 0;
@ -548,10 +546,9 @@ CBlockPolicyEstimator::CBlockPolicyEstimator()
longStats = std::unique_ptr<TxConfirmStats>(new TxConfirmStats(buckets, bucketMap, LONG_BLOCK_PERIODS, LONG_DECAY, LONG_SCALE));
// If the fee estimation file is present, read recorded estimations
fs::path est_filepath = gArgs.GetDataDirNet() / FEE_ESTIMATES_FILENAME;
CAutoFile est_file(fsbridge::fopen(est_filepath, "rb"), SER_DISK, CLIENT_VERSION);
CAutoFile est_file(fsbridge::fopen(m_estimation_filepath, "rb"), SER_DISK, CLIENT_VERSION);
if (est_file.IsNull() || !Read(est_file)) {
LogPrintf("Failed to read fee estimates from %s. Continue anyway.\n", fs::PathToString(est_filepath));
LogPrintf("Failed to read fee estimates from %s. Continue anyway.\n", fs::PathToString(m_estimation_filepath));
}
}
@ -907,10 +904,9 @@ CFeeRate CBlockPolicyEstimator::estimateSmartFee(int confTarget, FeeCalculation
void CBlockPolicyEstimator::Flush() {
FlushUnconfirmed();
fs::path est_filepath = gArgs.GetDataDirNet() / FEE_ESTIMATES_FILENAME;
CAutoFile est_file(fsbridge::fopen(est_filepath, "wb"), SER_DISK, CLIENT_VERSION);
CAutoFile est_file(fsbridge::fopen(m_estimation_filepath, "wb"), SER_DISK, CLIENT_VERSION);
if (est_file.IsNull() || !Write(est_file)) {
LogPrintf("Failed to write fee estimates to %s. Continue anyway.\n", fs::PathToString(est_filepath));
LogPrintf("Failed to write fee estimates to %s. Continue anyway.\n", fs::PathToString(m_estimation_filepath));
}
}

4
src/policy/fees.h
View file

@ -6,6 +6,7 @@
#define BITCOIN_POLICY_FEES_H
#include <consensus/amount.h>
#include <fs.h>
#include <policy/feerate.h>
#include <random.h>
#include <sync.h>
@ -179,9 +180,10 @@ private:
*/
static constexpr double FEE_SPACING = 1.05;
const fs::path m_estimation_filepath;
public:
/** Create new BlockPolicyEstimator and initialize stats tracking classes with default values */
CBlockPolicyEstimator();
CBlockPolicyEstimator(const fs::path& estimation_filepath);
~CBlockPolicyEstimator();
/** Process all the transactions that have been included in a block */

12
src/policy/fees_args.cpp Normal file
View file

@ -0,0 +1,12 @@
#include <policy/fees_args.h>
#include <util/system.h>
namespace {
const char* FEE_ESTIMATES_FILENAME = "fee_estimates.dat";
} // namespace
fs::path FeeestPath(const ArgsManager& argsman)
{
return argsman.GetDataDirNet() / FEE_ESTIMATES_FILENAME;
}

15
src/policy/fees_args.h Normal file
View file

@ -0,0 +1,15 @@
// Copyright (c) 2022 The Bitcoin Core developers
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#ifndef BITCOIN_POLICY_FEES_ARGS_H
#define BITCOIN_POLICY_FEES_ARGS_H
#include <fs.h>
class ArgsManager;
/** @return The fee estimates data file path. */
fs::path FeeestPath(const ArgsManager& argsman);
#endif // BITCOIN_POLICY_FEES_ARGS_H

4
src/policy/packages.h
View file

@ -25,8 +25,8 @@ static_assert(MAX_PACKAGE_SIZE * WITNESS_SCALE_FACTOR * 1000 >= MAX_STANDARD_TX_
// defaults reflect this constraint.
static_assert(DEFAULT_DESCENDANT_LIMIT >= MAX_PACKAGE_COUNT);
static_assert(DEFAULT_ANCESTOR_LIMIT >= MAX_PACKAGE_COUNT);
static_assert(DEFAULT_ANCESTOR_SIZE_LIMIT >= MAX_PACKAGE_SIZE);
static_assert(DEFAULT_DESCENDANT_SIZE_LIMIT >= MAX_PACKAGE_SIZE);
static_assert(DEFAULT_ANCESTOR_SIZE_LIMIT_KVB >= MAX_PACKAGE_SIZE);
static_assert(DEFAULT_DESCENDANT_SIZE_LIMIT_KVB >= MAX_PACKAGE_SIZE);
/** A "reason" why a package was invalid. It may be that one or more of the included
* transactions is invalid or the package itself violates our rules.

6
src/policy/policy.h
View file

@ -31,8 +31,6 @@ static constexpr unsigned int MIN_STANDARD_TX_NONWITNESS_SIZE{82};
static constexpr unsigned int MAX_P2SH_SIGOPS{15};
/** The maximum number of sigops we're willing to relay/mine in a single tx */
static constexpr unsigned int MAX_STANDARD_TX_SIGOPS_COST{MAX_BLOCK_SIGOPS_COST/5};
/** Default for -maxmempool, maximum megabytes of mempool memory usage */
static constexpr unsigned int DEFAULT_MAX_MEMPOOL_SIZE{300};
/** Default for -incrementalrelayfee, which sets the minimum feerate increase for mempool limiting or BIP 125 replacement **/
static constexpr unsigned int DEFAULT_INCREMENTAL_RELAY_FEE{1000};
/** Default for -bytespersigop */
@ -60,11 +58,11 @@ static constexpr unsigned int DEFAULT_MIN_RELAY_TX_FEE{1000};
/** Default for -limitancestorcount, max number of in-mempool ancestors */
static constexpr unsigned int DEFAULT_ANCESTOR_LIMIT{25};
/** Default for -limitancestorsize, maximum kilobytes of tx + all in-mempool ancestors */
static constexpr unsigned int DEFAULT_ANCESTOR_SIZE_LIMIT{101};
static constexpr unsigned int DEFAULT_ANCESTOR_SIZE_LIMIT_KVB{101};
/** Default for -limitdescendantcount, max number of in-mempool descendants */
static constexpr unsigned int DEFAULT_DESCENDANT_LIMIT{25};
/** Default for -limitdescendantsize, maximum kilobytes of in-mempool descendants */
static constexpr unsigned int DEFAULT_DESCENDANT_SIZE_LIMIT{101};
static constexpr unsigned int DEFAULT_DESCENDANT_SIZE_LIMIT_KVB{101};
/**
* An extra transaction can be added to a package, as long as it only has one
* ancestor and is no larger than this. Not really any reason to make this

2
src/rpc/fees.cpp
View file

@ -89,7 +89,7 @@ static RPCHelpMan estimatesmartfee()
FeeCalculation feeCalc;
CFeeRate feeRate{fee_estimator.estimateSmartFee(conf_target, &feeCalc, conservative)};
if (feeRate != CFeeRate(0)) {
CFeeRate min_mempool_feerate{mempool.GetMinFee(gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000)};
CFeeRate min_mempool_feerate{mempool.GetMinFee()};
CFeeRate min_relay_feerate{::minRelayTxFee};
feeRate = std::max({feeRate, min_mempool_feerate, min_relay_feerate});
result.pushKV("feerate", ValueFromAmount(feeRate.GetFeePerK()));

5
src/rpc/mempool.cpp
View file

@ -657,9 +657,8 @@ UniValue MempoolInfoToJSON(const CTxMemPool& pool)
ret.pushKV("bytes", (int64_t)pool.GetTotalTxSize());
ret.pushKV("usage", (int64_t)pool.DynamicMemoryUsage());
ret.pushKV("total_fee", ValueFromAmount(pool.GetTotalFee()));
int64_t maxmempool{gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000};
ret.pushKV("maxmempool", maxmempool);
ret.pushKV("mempoolminfee", ValueFromAmount(std::max(pool.GetMinFee(maxmempool), ::minRelayTxFee).GetFeePerK()));
ret.pushKV("maxmempool", pool.m_max_size_bytes);
ret.pushKV("mempoolminfee", ValueFromAmount(std::max(pool.GetMinFee(), ::minRelayTxFee).GetFeePerK()));
ret.pushKV("minrelaytxfee", ValueFromAmount(::minRelayTxFee.GetFeePerK()));
ret.pushKV("incrementalrelayfee", ValueFromAmount(::incrementalRelayFee.GetFeePerK()));
ret.pushKV("unbroadcastcount", uint64_t{pool.GetUnbroadcastTxs().size()});

8
src/test/fuzz/policy_estimator.cpp
View file

@ -3,6 +3,7 @@
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <policy/fees.h>
#include <policy/fees_args.h>
#include <primitives/transaction.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
@ -15,15 +16,20 @@
#include <string>
#include <vector>
namespace {
const BasicTestingSetup* g_setup;
} // namespace
void initialize_policy_estimator()
{
static const auto testing_setup = MakeNoLogFileContext<>();
g_setup = testing_setup.get();
}
FUZZ_TARGET_INIT(policy_estimator, initialize_policy_estimator)
{
FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size());
CBlockPolicyEstimator block_policy_estimator;
CBlockPolicyEstimator block_policy_estimator{FeeestPath(*g_setup->m_node.args)};
LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 10000) {
CallOneOf(
fuzzed_data_provider,

8
src/test/fuzz/policy_estimator_io.cpp
View file

@ -3,6 +3,7 @@
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <policy/fees.h>
#include <policy/fees_args.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
#include <test/fuzz/util.h>
@ -11,9 +12,14 @@
#include <cstdint>
#include <vector>
namespace {
const BasicTestingSetup* g_setup;
} // namespace
void initialize_policy_estimator_io()
{
static const auto testing_setup = MakeNoLogFileContext<>();
g_setup = testing_setup.get();
}
FUZZ_TARGET_INIT(policy_estimator_io, initialize_policy_estimator_io)
@ -22,7 +28,7 @@ FUZZ_TARGET_INIT(policy_estimator_io, initialize_policy_estimator_io)
FuzzedAutoFileProvider fuzzed_auto_file_provider = ConsumeAutoFile(fuzzed_data_provider);
CAutoFile fuzzed_auto_file = fuzzed_auto_file_provider.open();
// Re-using block_policy_estimator across runs to avoid costly creation of CBlockPolicyEstimator object.
static CBlockPolicyEstimator block_policy_estimator;
static CBlockPolicyEstimator block_policy_estimator{FeeestPath(*g_setup->m_node.args)};
if (block_policy_estimator.Read(fuzzed_auto_file)) {
block_policy_estimator.Write(fuzzed_auto_file);
}

21
src/test/fuzz/rbf.cpp
View file

@ -2,12 +2,14 @@
// Distributed under the MIT software license, see the accompanying
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <mempool_args.h>
#include <policy/rbf.h>
#include <primitives/transaction.h>
#include <sync.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
#include <test/fuzz/util.h>
#include <test/util/setup_common.h>
#include <txmempool.h>
#include <cstdint>
@ -15,7 +17,17 @@
#include <string>
#include <vector>
FUZZ_TARGET(rbf)
namespace {
const BasicTestingSetup* g_setup;
} // namespace
void initialize_rbf()
{
static const auto testing_setup = MakeNoLogFileContext<>();
g_setup = testing_setup.get();
}
FUZZ_TARGET_INIT(rbf, initialize_rbf)
{
FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size());
SetMockTime(ConsumeTime(fuzzed_data_provider));
@ -23,8 +35,11 @@ FUZZ_TARGET(rbf)
if (!mtx) {
return;
}
CTxMemPool pool;
LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 10000) {
CTxMemPool pool{MemPoolOptionsForTest(g_setup->m_node)};
LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 10000)
{
const std::optional<CMutableTransaction> another_mtx = ConsumeDeserializable<CMutableTransaction>(fuzzed_data_provider);
if (!another_mtx) {
break;

20
src/test/fuzz/tx_pool.cpp
View file

@ -3,6 +3,8 @@
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <consensus/validation.h>
#include <mempool_args.h>
#include <node/context.h>
#include <node/miner.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
@ -15,6 +17,7 @@
#include <validationinterface.h>
using node::BlockAssembler;
using node::NodeContext;
namespace {
@ -121,6 +124,19 @@ void MockTime(FuzzedDataProvider& fuzzed_data_provider, const CChainState& chain
SetMockTime(time);
}
CTxMemPool MakeMempool(const NodeContext& node)
{
// Take the default options for tests...
CTxMemPool::Options mempool_opts{MemPoolOptionsForTest(node)};
// ...override specific options for this specific fuzz suite
mempool_opts.estimator = nullptr;
mempool_opts.check_ratio = 1;
// ...and construct a CTxMemPool from it
return CTxMemPool{mempool_opts};
}
FUZZ_TARGET_INIT(tx_pool_standard, initialize_tx_pool)
{
FuzzedDataProvider fuzzed_data_provider(buffer.data(), buffer.size());
@ -142,7 +158,7 @@ FUZZ_TARGET_INIT(tx_pool_standard, initialize_tx_pool)
// The sum of the values of all spendable outpoints
constexpr CAmount SUPPLY_TOTAL{COINBASE_MATURITY * 50 * COIN};
CTxMemPool tx_pool_{/*estimator=*/nullptr, /*check_ratio=*/1};
CTxMemPool tx_pool_{MakeMempool(node)};
MockedTxPool& tx_pool = *static_cast<MockedTxPool*>(&tx_pool_);
chainstate.SetMempool(&tx_pool);
@ -320,7 +336,7 @@ FUZZ_TARGET_INIT(tx_pool, initialize_tx_pool)
txids.push_back(ConsumeUInt256(fuzzed_data_provider));
}
CTxMemPool tx_pool_{/*estimator=*/nullptr, /*check_ratio=*/1};
CTxMemPool tx_pool_{MakeMempool(node)};
MockedTxPool& tx_pool = *static_cast<MockedTxPool*>(&tx_pool_);
LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 300)

4
src/test/fuzz/validation_load_mempool.cpp
View file

@ -3,6 +3,7 @@
// file COPYING or http://www.opensource.org/licenses/mit-license.php.
#include <chainparamsbase.h>
#include <mempool_args.h>
#include <test/fuzz/FuzzedDataProvider.h>
#include <test/fuzz/fuzz.h>
#include <test/fuzz/util.h>
@ -30,7 +31,8 @@ FUZZ_TARGET_INIT(validation_load_mempool, initialize_validation_load_mempool)
SetMockTime(ConsumeTime(fuzzed_data_provider));
FuzzedFileProvider fuzzed_file_provider = ConsumeFile(fuzzed_data_provider);
CTxMemPool pool{};
CTxMemPool pool{MemPoolOptionsForTest(g_setup->m_node)};
auto fuzzed_fopen = [&](const fs::path&, const char*) {
return fuzzed_file_provider.open();
};

8
src/test/mempool_tests.cpp
View file

@ -16,6 +16,12 @@ BOOST_FIXTURE_TEST_SUITE(mempool_tests, TestingSetup)
static constexpr auto REMOVAL_REASON_DUMMY = MemPoolRemovalReason::REPLACED;
class MemPoolTest final : public CTxMemPool
{
public:
using CTxMemPool::GetMinFee;
};
BOOST_AUTO_TEST_CASE(MempoolRemoveTest)
{
// Test CTxMemPool::remove functionality
@ -423,7 +429,7 @@ BOOST_AUTO_TEST_CASE(MempoolAncestorIndexingTest)
BOOST_AUTO_TEST_CASE(MempoolSizeLimitTest)
{
CTxMemPool& pool = *Assert(m_node.mempool);
auto& pool = static_cast<MemPoolTest&>(*Assert(m_node.mempool));
LOCK2(cs_main, pool.cs);
TestMemPoolEntryHelper entry;

28
src/test/util/setup_common.cpp
View file

@ -14,13 +14,16 @@
#include <init.h>
#include <init/common.h>
#include <interfaces/chain.h>
#include <mempool_args.h>
#include <net.h>
#include <net_processing.h>
#include <node/blockstorage.h>
#include <node/chainstate.h>
#include <node/context.h>
#include <node/miner.h>
#include <noui.h>
#include <policy/fees.h>
#include <policy/fees_args.h>
#include <pow.h>
#include <rpc/blockchain.h>
#include <rpc/register.h>
@ -32,6 +35,8 @@
#include <test/util/net.h>
#include <timedata.h>
#include <txdb.h>
#include <txmempool.h>
#include <util/designator.h>
#include <util/strencodings.h>
#include <util/string.h>
#include <util/thread.h>
@ -50,11 +55,12 @@
using node::BlockAssembler;
using node::CalculateCacheSizes;
using node::LoadChainstate;
using node::RegenerateCommitments;
using node::VerifyLoadedChainstate;
using node::fPruneMode;
using node::fReindex;
using node::LoadChainstate;
using node::NodeContext;
using node::RegenerateCommitments;
using node::VerifyLoadedChainstate;
const std::function<std::string(const char*)> G_TRANSLATION_FUN = nullptr;
UrlDecodeFn* const URL_DECODE = nullptr;
@ -149,6 +155,18 @@ BasicTestingSetup::~BasicTestingSetup()
gArgs.ClearArgs();
}
CTxMemPool::Options MemPoolOptionsForTest(const NodeContext& node)
{
CTxMemPool::Options mempool_opts{
Desig(estimator) node.fee_estimator.get(),
// Default to always checking mempool regardless of
// chainparams.DefaultConsistencyChecks for tests
Desig(check_ratio) 1,
};
ApplyArgsManOptions(*node.args, mempool_opts);
return mempool_opts;
}
ChainTestingSetup::ChainTestingSetup(const std::string& chainName, const std::vector<const char*>& extra_args)
: BasicTestingSetup(chainName, extra_args)
{
@ -160,8 +178,8 @@ ChainTestingSetup::ChainTestingSetup(const std::string& chainName, const std::ve
m_node.scheduler->m_service_thread = std::thread(util::TraceThread, "scheduler", [&] { m_node.scheduler->serviceQueue(); });
GetMainSignals().RegisterBackgroundSignalScheduler(*m_node.scheduler);
m_node.fee_estimator = std::make_unique<CBlockPolicyEstimator>();
m_node.mempool = std::make_unique<CTxMemPool>(m_node.fee_estimator.get(), m_node.args->GetIntArg("-checkmempool", 1));
m_node.fee_estimator = std::make_unique<CBlockPolicyEstimator>(FeeestPath(*m_node.args));
m_node.mempool = std::make_unique<CTxMemPool>(MemPoolOptionsForTest(m_node));
m_cache_sizes = CalculateCacheSizes(m_args);

3
src/test/util/setup_common.h
View file

@ -90,6 +90,9 @@ struct BasicTestingSetup {
ArgsManager m_args;
};
CTxMemPool::Options MemPoolOptionsForTest(const node::NodeContext& node);
/** Testing setup that performs all steps up until right before
* ChainstateManager gets initialized. Meant for testing ChainstateManager
* initialization behaviour.

17
src/txmempool.cpp
View file

@ -110,8 +110,7 @@ size_t CTxMemPoolEntry::GetTxSize() const
}
void CTxMemPool::UpdateForDescendants(txiter updateIt, cacheMap& cachedDescendants,
const std::set<uint256>& setExclude, std::set<uint256>& descendants_to_remove,
uint64_t ancestor_size_limit, uint64_t ancestor_count_limit)
const std::set<uint256>& setExclude, std::set<uint256>& descendants_to_remove)
{
CTxMemPoolEntry::Children stageEntries, descendants;
stageEntries = updateIt->GetMemPoolChildrenConst();
@ -151,7 +150,7 @@ void CTxMemPool::UpdateForDescendants(txiter updateIt, cacheMap& cachedDescendan
// Don't directly remove the transaction here -- doing so would
// invalidate iterators in cachedDescendants. Mark it for removal
// by inserting into descendants_to_remove.
if (descendant.GetCountWithAncestors() > ancestor_count_limit || descendant.GetSizeWithAncestors() > ancestor_size_limit) {
if (descendant.GetCountWithAncestors() > uint64_t(m_limits.ancestor_count) || descendant.GetSizeWithAncestors() > uint64_t(m_limits.ancestor_size_vbytes)) {
descendants_to_remove.insert(descendant.GetTx().GetHash());
}
}
@ -159,7 +158,7 @@ void CTxMemPool::UpdateForDescendants(txiter updateIt, cacheMap& cachedDescendan
mapTx.modify(updateIt, update_descendant_state(modifySize, modifyFee, modifyCount));
}
void CTxMemPool::UpdateTransactionsFromBlock(const std::vector<uint256> &vHashesToUpdate, uint64_t ancestor_size_limit, uint64_t ancestor_count_limit)
void CTxMemPool::UpdateTransactionsFromBlock(const std::vector<uint256>& vHashesToUpdate)
{
AssertLockHeld(cs);
// For each entry in vHashesToUpdate, store the set of in-mempool, but not
@ -202,7 +201,7 @@ void CTxMemPool::UpdateTransactionsFromBlock(const std::vector<uint256> &vHashes
}
}
} // release epoch guard for UpdateForDescendants
UpdateForDescendants(it, mapMemPoolDescendantsToUpdate, setAlreadyIncluded, descendants_to_remove, ancestor_size_limit, ancestor_count_limit);
UpdateForDescendants(it, mapMemPoolDescendantsToUpdate, setAlreadyIncluded, descendants_to_remove);
}
for (const auto& txid : descendants_to_remove) {
@ -454,8 +453,12 @@ void CTxMemPoolEntry::UpdateAncestorState(int64_t modifySize, CAmount modifyFee,
assert(int(nSigOpCostWithAncestors) >= 0);
}
CTxMemPool::CTxMemPool(CBlockPolicyEstimator* estimator, int check_ratio)
: m_check_ratio(check_ratio), minerPolicyEstimator(estimator)
CTxMemPool::CTxMemPool(const Options& opts)
: m_check_ratio{opts.check_ratio},
minerPolicyEstimator{opts.estimator},
m_max_size_bytes{opts.max_size_bytes},
m_expiry{opts.expiry},
m_limits{opts.limits}
{
_clear(); //lock free clear
}

37
src/txmempool.h
View file

@ -14,6 +14,9 @@
#include <utility>
#include <vector>
#include <kernel/mempool_limits.h>
#include <kernel/mempool_options.h>
#include <coins.h>
#include <consensus/amount.h>
#include <indirectmap.h>
@ -450,6 +453,8 @@ protected:
bool m_is_loaded GUARDED_BY(cs){false};
CFeeRate GetMinFee(size_t sizelimit) const;
public:
static const int ROLLING_FEE_HALFLIFE = 60 * 60 * 12; // public only for testing
@ -559,15 +564,21 @@ public:
indirectmap<COutPoint, const CTransaction*> mapNextTx GUARDED_BY(cs);
std::map<uint256, CAmount> mapDeltas GUARDED_BY(cs);
using Options = kernel::MemPoolOptions;
const int64_t m_max_size_bytes;
const std::chrono::seconds m_expiry;
using Limits = kernel::MemPoolLimits;
const Limits m_limits;
/** Create a new CTxMemPool.
* Sanity checks will be off by default for performance, because otherwise
* accepting transactions becomes O(N^2) where N is the number of transactions
* in the pool.
*
* @param[in] estimator is used to estimate appropriate transaction fees.
* @param[in] check_ratio is the ratio used to determine how often sanity checks will run.
*/
explicit CTxMemPool(CBlockPolicyEstimator* estimator = nullptr, int check_ratio = 0);
explicit CTxMemPool(const Options& opts);
/**
* If sanity-checking is turned on, check makes sure the pool is
@ -647,13 +658,8 @@ public:
*
* @param[in] vHashesToUpdate The set of txids from the
* disconnected block that have been accepted back into the mempool.
* @param[in] ancestor_size_limit The maximum allowed size in virtual
* bytes of an entry and its ancestors
* @param[in] ancestor_count_limit The maximum allowed number of
* transactions including the entry and its ancestors.
*/
void UpdateTransactionsFromBlock(const std::vector<uint256>& vHashesToUpdate,
uint64_t ancestor_size_limit, uint64_t ancestor_count_limit) EXCLUSIVE_LOCKS_REQUIRED(cs, cs_main) LOCKS_EXCLUDED(m_epoch);
void UpdateTransactionsFromBlock(const std::vector<uint256>& vHashesToUpdate) EXCLUSIVE_LOCKS_REQUIRED(cs, cs_main) LOCKS_EXCLUDED(m_epoch);
/** Try to calculate all in-mempool ancestors of entry.
* (these are all calculated including the tx itself)
@ -700,7 +706,9 @@ public:
* takes the fee rate to go back down all the way to 0. When the feerate
* would otherwise be half of this, it is set to 0 instead.
*/
CFeeRate GetMinFee(size_t sizelimit) const;
CFeeRate GetMinFee() const {
return GetMinFee(m_max_size_bytes);
}
/** Remove transactions from the mempool until its dynamic size is <= sizelimit.
* pvNoSpendsRemaining, if set, will be populated with the list of outpoints
@ -826,14 +834,9 @@ private:
* @param[out] descendants_to_remove Populated with the txids of entries that
* exceed ancestor limits. It's the responsibility of the caller to
* removeRecursive them.
* @param[in] ancestor_size_limit the max number of ancestral bytes allowed
* for any descendant
* @param[in] ancestor_count_limit the max number of ancestor transactions
* allowed for any descendant
*/
void UpdateForDescendants(txiter updateIt, cacheMap& cachedDescendants,
const std::set<uint256>& setExclude, std::set<uint256>& descendants_to_remove,
uint64_t ancestor_size_limit, uint64_t ancestor_count_limit) EXCLUSIVE_LOCKS_REQUIRED(cs);
const std::set<uint256>& setExclude, std::set<uint256>& descendants_to_remove) EXCLUSIVE_LOCKS_REQUIRED(cs);
/** Update ancestors of hash to add/remove it as a descendant transaction. */
void UpdateAncestorsOf(bool add, txiter hash, setEntries &setAncestors) EXCLUSIVE_LOCKS_REQUIRED(cs);
/** Set ancestor state for an entry */

52
src/util/system.cpp
View file

@ -609,36 +609,76 @@ bool ArgsManager::IsArgNegated(const std::string& strArg) const
}
std::string ArgsManager::GetArg(const std::string& strArg, const std::string& strDefault) const
{
return GetArg(strArg).value_or(strDefault);
}
std::optional<std::string> ArgsManager::GetArg(const std::string& strArg) const
{
const util::SettingsValue value = GetSetting(strArg);
return SettingToString(value, strDefault);
return SettingToString(value);
}
std::optional<std::string> SettingToString(const util::SettingsValue& value)
{
if (value.isNull()) return std::nullopt;
if (value.isFalse()) return "0";
if (value.isTrue()) return "1";
if (value.isNum()) return value.getValStr();
return value.get_str();
}
std::string SettingToString(const util::SettingsValue& value, const std::string& strDefault)
{
return value.isNull() ? strDefault : value.isFalse() ? "0" : value.isTrue() ? "1" : value.isNum() ? value.getValStr() : value.get_str();
return SettingToString(value).value_or(strDefault);
}
int64_t ArgsManager::GetIntArg(const std::string& strArg, int64_t nDefault) const
{
return GetIntArg(strArg).value_or(nDefault);
}
std::optional<int64_t> ArgsManager::GetIntArg(const std::string& strArg) const
{
const util::SettingsValue value = GetSetting(strArg);
return SettingToInt(value, nDefault);
return SettingToInt(value);
}
std::optional<int64_t> SettingToInt(const util::SettingsValue& value)
{
if (value.isNull()) return std::nullopt;
if (value.isFalse()) return 0;
if (value.isTrue()) return 1;
if (value.isNum()) return value.getInt<int64_t>();
return LocaleIndependentAtoi<int64_t>(value.get_str());
}
int64_t SettingToInt(const util::SettingsValue& value, int64_t nDefault)
{
return value.isNull() ? nDefault : value.isFalse() ? 0 : value.isTrue() ? 1 : value.isNum() ? value.getInt<int64_t>() : LocaleIndependentAtoi<int64_t>(value.get_str());
return SettingToInt(value).value_or(nDefault);
}
bool ArgsManager::GetBoolArg(const std::string& strArg, bool fDefault) const
{
return GetBoolArg(strArg).value_or(fDefault);
}
std::optional<bool> ArgsManager::GetBoolArg(const std::string& strArg) const
{
const util::SettingsValue value = GetSetting(strArg);
return SettingToBool(value, fDefault);
return SettingToBool(value);
}
std::optional<bool> SettingToBool(const util::SettingsValue& value)
{
if (value.isNull()) return std::nullopt;
if (value.isBool()) return value.get_bool();
return InterpretBool(value.get_str());
}
bool SettingToBool(const util::SettingsValue& value, bool fDefault)
{
return value.isNull() ? fDefault : value.isBool() ? value.get_bool() : InterpretBool(value.get_str());
return SettingToBool(value).value_or(fDefault);
}
bool ArgsManager::SoftSetArg(const std::string& strArg, const std::string& strValue)

8
src/util/system.h
View file

@ -161,8 +161,13 @@ struct SectionInfo
};
std::string SettingToString(const util::SettingsValue&, const std::string&);
std::optional<std::string> SettingToString(const util::SettingsValue&);
int64_t SettingToInt(const util::SettingsValue&, int64_t);
std::optional<int64_t> SettingToInt(const util::SettingsValue&);
bool SettingToBool(const util::SettingsValue&, bool);
std::optional<bool> SettingToBool(const util::SettingsValue&);
class ArgsManager
{
@ -335,6 +340,7 @@ protected:
* @return command-line argument or default value
*/
std::string GetArg(const std::string& strArg, const std::string& strDefault) const;
std::optional<std::string> GetArg(const std::string& strArg) const;
/**
* Return path argument or default value
@ -356,6 +362,7 @@ protected:
* @return command-line argument (0 if invalid number) or default value
*/
int64_t GetIntArg(const std::string& strArg, int64_t nDefault) const;
std::optional<int64_t> GetIntArg(const std::string& strArg) const;
/**
* Return boolean argument or default value
@ -365,6 +372,7 @@ protected:
* @return command-line argument or default value
*/
bool GetBoolArg(const std::string& strArg, bool fDefault) const;
std::optional<bool> GetBoolArg(const std::string& strArg) const;
/**
* Set an argument if it doesn't already have a value

36
src/validation.cpp
View file

@ -255,18 +255,18 @@ bool CheckSequenceLocksAtTip(CBlockIndex* tip,
// Returns the script flags which should be checked for a given block
static unsigned int GetBlockScriptFlags(const CBlockIndex& block_index, const ChainstateManager& chainman);
static void LimitMempoolSize(CTxMemPool& pool, CCoinsViewCache& coins_cache, size_t limit, std::chrono::seconds age)
static void LimitMempoolSize(CTxMemPool& pool, CCoinsViewCache& coins_cache)
EXCLUSIVE_LOCKS_REQUIRED(::cs_main, pool.cs)
{
AssertLockHeld(::cs_main);
AssertLockHeld(pool.cs);
int expired = pool.Expire(GetTime<std::chrono::seconds>() - age);
int expired = pool.Expire(GetTime<std::chrono::seconds>() - pool.m_expiry);
if (expired != 0) {
LogPrint(BCLog::MEMPOOL, "Expired %i transactions from the memory pool\n", expired);
}
std::vector<COutPoint> vNoSpendsRemaining;
pool.TrimToSize(limit, &vNoSpendsRemaining);
pool.TrimToSize(pool.m_max_size_bytes, &vNoSpendsRemaining);
for (const COutPoint& removed : vNoSpendsRemaining)
coins_cache.Uncache(removed);
}
@ -320,9 +320,7 @@ void CChainState::MaybeUpdateMempoolForReorg(
// previously-confirmed transactions back to the mempool.
// UpdateTransactionsFromBlock finds descendants of any transactions in
// the disconnectpool that were added back and cleans up the mempool state.
const uint64_t ancestor_count_limit = gArgs.GetIntArg("-limitancestorcount", DEFAULT_ANCESTOR_LIMIT);
const uint64_t ancestor_size_limit = gArgs.GetIntArg("-limitancestorsize", DEFAULT_ANCESTOR_SIZE_LIMIT) * 1000;
m_mempool->UpdateTransactionsFromBlock(vHashUpdate, ancestor_size_limit, ancestor_count_limit);
m_mempool->UpdateTransactionsFromBlock(vHashUpdate);
// Predicate to use for filtering transactions in removeForReorg.
// Checks whether the transaction is still final and, if it spends a coinbase output, mature.
@ -374,11 +372,7 @@ void CChainState::MaybeUpdateMempoolForReorg(
// We also need to remove any now-immature transactions
m_mempool->removeForReorg(m_chain, filter_final_and_mature);
// Re-limit mempool size, in case we added any transactions
LimitMempoolSize(
*m_mempool,
this->CoinsTip(),
gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000,
std::chrono::hours{gArgs.GetIntArg("-mempoolexpiry", DEFAULT_MEMPOOL_EXPIRY)});
LimitMempoolSize(*m_mempool, this->CoinsTip());
}
/**
@ -429,10 +423,10 @@ class MemPoolAccept
{
public:
explicit MemPoolAccept(CTxMemPool& mempool, CChainState& active_chainstate) : m_pool(mempool), m_view(&m_dummy), m_viewmempool(&active_chainstate.CoinsTip(), m_pool), m_active_chainstate(active_chainstate),
m_limit_ancestors(gArgs.GetIntArg("-limitancestorcount", DEFAULT_ANCESTOR_LIMIT)),
m_limit_ancestor_size(gArgs.GetIntArg("-limitancestorsize", DEFAULT_ANCESTOR_SIZE_LIMIT)*1000),
m_limit_descendants(gArgs.GetIntArg("-limitdescendantcount", DEFAULT_DESCENDANT_LIMIT)),
m_limit_descendant_size(gArgs.GetIntArg("-limitdescendantsize", DEFAULT_DESCENDANT_SIZE_LIMIT)*1000) {
m_limit_ancestors(m_pool.m_limits.ancestor_count),
m_limit_ancestor_size(m_pool.m_limits.ancestor_size_vbytes),
m_limit_descendants(m_pool.m_limits.descendant_count),
m_limit_descendant_size(m_pool.m_limits.descendant_size_vbytes) {
}
// We put the arguments we're handed into a struct, so we can pass them
@ -644,7 +638,7 @@ private:
{
AssertLockHeld(::cs_main);
AssertLockHeld(m_pool.cs);
CAmount mempoolRejectFee = m_pool.GetMinFee(gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000).GetFee(package_size);
CAmount mempoolRejectFee = m_pool.GetMinFee().GetFee(package_size);
if (mempoolRejectFee > 0 && package_fee < mempoolRejectFee) {
return state.Invalid(TxValidationResult::TX_MEMPOOL_POLICY, "mempool min fee not met", strprintf("%d < %d", package_fee, mempoolRejectFee));
}
@ -1082,7 +1076,7 @@ bool MemPoolAccept::Finalize(const ATMPArgs& args, Workspace& ws)
// in the package. LimitMempoolSize() should be called at the very end to make sure the mempool
// is still within limits and package submission happens atomically.
if (!args.m_package_submission && !bypass_limits) {
LimitMempoolSize(m_pool, m_active_chainstate.CoinsTip(), gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000, std::chrono::hours{gArgs.GetIntArg("-mempoolexpiry", DEFAULT_MEMPOOL_EXPIRY)});
LimitMempoolSize(m_pool, m_active_chainstate.CoinsTip());
if (!m_pool.exists(GenTxid::Txid(hash)))
return state.Invalid(TxValidationResult::TX_MEMPOOL_POLICY, "mempool full");
}
@ -1147,9 +1141,7 @@ bool MemPoolAccept::SubmitPackage(const ATMPArgs& args, std::vector<Workspace>&
// It may or may not be the case that all the transactions made it into the mempool. Regardless,
// make sure we haven't exceeded max mempool size.
LimitMempoolSize(m_pool, m_active_chainstate.CoinsTip(),
gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000,
std::chrono::hours{gArgs.GetIntArg("-mempoolexpiry", DEFAULT_MEMPOOL_EXPIRY)});
LimitMempoolSize(m_pool, m_active_chainstate.CoinsTip());
// Find the wtxids of the transactions that made it into the mempool. Allow partial submission,
// but don't report success unless they all made it into the mempool.
@ -2292,7 +2284,7 @@ CoinsCacheSizeState CChainState::GetCoinsCacheSizeState()
AssertLockHeld(::cs_main);
return this->GetCoinsCacheSizeState(
m_coinstip_cache_size_bytes,
gArgs.GetIntArg("-maxmempool", DEFAULT_MAX_MEMPOOL_SIZE) * 1000000);
m_mempool ? m_mempool->m_max_size_bytes : 0);
}
CoinsCacheSizeState CChainState::GetCoinsCacheSizeState(
@ -4647,7 +4639,7 @@ static const uint64_t MEMPOOL_DUMP_VERSION = 1;
bool LoadMempool(CTxMemPool& pool, CChainState& active_chainstate, FopenFn mockable_fopen_function)
{
int64_t nExpiryTimeout = gArgs.GetIntArg("-mempoolexpiry", DEFAULT_MEMPOOL_EXPIRY) * 60 * 60;
int64_t nExpiryTimeout = std::chrono::seconds{pool.m_expiry}.count();
FILE* filestr{mockable_fopen_function(gArgs.GetDataDirNet() / "mempool.dat", "rb")};
CAutoFile file(filestr, SER_DISK, CLIENT_VERSION);
if (file.IsNull()) {

2
src/validation.h
View file

@ -59,8 +59,6 @@ namespace Consensus {
struct Params;
} // namespace Consensus
/** Default for -mempoolexpiry, expiration time for mempool transactions in hours */
static const unsigned int DEFAULT_MEMPOOL_EXPIRY = 336;
/** Maximum number of dedicated script-checking threads allowed */
static const int MAX_SCRIPTCHECK_THREADS = 15;
/** -par default (number of script-checking threads, 0 = auto) */

8
test/functional/mempool_expiry.py
View file

@ -5,7 +5,7 @@
"""Tests that a mempool transaction expires after a given timeout and that its
children are removed as well.
Both the default expiry timeout defined by DEFAULT_MEMPOOL_EXPIRY and a user
Both the default expiry timeout defined by DEFAULT_MEMPOOL_EXPIRY_HOURS and a user
definable expiry timeout via the '-mempoolexpiry=<n>' command line argument
(<n> is the timeout in hours) are tested.
"""
@ -20,7 +20,7 @@ from test_framework.util import (
)
from test_framework.wallet import MiniWallet
DEFAULT_MEMPOOL_EXPIRY = 336 # hours
DEFAULT_MEMPOOL_EXPIRY_HOURS = 336 # hours
CUSTOM_MEMPOOL_EXPIRY = 10 # hours
@ -98,8 +98,8 @@ class MempoolExpiryTest(BitcoinTestFramework):
def run_test(self):
self.log.info('Test default mempool expiry timeout of %d hours.' %
DEFAULT_MEMPOOL_EXPIRY)
self.test_transaction_expiry(DEFAULT_MEMPOOL_EXPIRY)
DEFAULT_MEMPOOL_EXPIRY_HOURS)
self.test_transaction_expiry(DEFAULT_MEMPOOL_EXPIRY_HOURS)
self.log.info('Test custom mempool expiry timeout of %d hours.' %
CUSTOM_MEMPOOL_EXPIRY)