8c277b19c8 refactor: Make m_cs_fee_estimator non-recursive (Hennadii Stepanov)
5ee5b696b5 refactor: Add non-thread-safe CBlockPolicyEstimator::_removeTx helper (Hennadii Stepanov)
5c3033d45e Add thread safety annotations to CBlockPolicyEstimator public functions (Hennadii Stepanov)
Pull request description:
This PR eliminates the only place that `m_cs_fee_estimator` is recursively locked by refactoring out `_removeTx` member function.
Related to #19303.
ACKs for top commit:
theStack:
Code-review ACK 8c277b19c8
amadeuszpawlik:
ACK 8c277b19c8 reviewed, built and ran tests
Tree-SHA512: 65b0b59460d3d5fadf7e75e916b2898b0dcfafdf5b278ef8c3975660f67c9f88ae4b937944313bd36d7513a7a53e1e5859aaf4a6deb4a1aea089936b101635a1
3d71d16d1e test: listtranscations with externally generated addresses (S3RK)
d04566415e Add to spends only transcations from me (S3RK)
9f3a622b1c Automatically add labels to detected receiving addresses (S3RK)
c1b99c088c Return used destinations from ScriptPubKeyMan::MarkUnusedAddresses (S3RK)
03840c2064 Add CWallet::IsInternalScriptPubKeyMan (S3RK)
456e350926 wallet: resolve ambiguity of two ScriptPubKey managers providing same script (S3RK)
Pull request description:
This PR fixes certain use-cases when **send-to-self** transactions are missing from `listtransactions` output.
1. When a receiving address is generated externally to the wallet
(e.g. same wallet running on two nodes, or by 3rd party from xpub)
2. When restoring backup with lost metadata, but keypool gap is not exceeded yet
When the block is connected or tx added to mempool we already mark used keys. This PR extends this logic to determine whether the destination is a receiving one and if yes add it to the address book with empty label.
Works both for legacy and descriptors wallets.
- For legacy it uses the internal flag from the keypool entry. Caveat: because we don't know which script type would be used we add all possible destinations for such keys.
- For descriptor wallets it uses internal flag for the script pub key manager. Caveat: it only works for active descriptors.
fixes#19856fixes#20293
ACKs for top commit:
laanwj:
Code review ACK 3d71d16d1e
Tree-SHA512: 03fafd5548ead0c4ffe9ebcc9eb2849f1d2fa7270fda4166419b86877d4e57dcf04460e465fbb9c90b42031f3c05d1b83f1b67a9f82c2a42980825ed1e7b52e6
fa551b3bdd Remove GetAdjustedTime from init.cpp (MarcoFalke)
fa815f8473 Replace addrman.h include with forward decl in net.h (MarcoFalke)
Pull request description:
It seems confusing to call `GetAdjustedTime` there, because no offset could have been retrieved from the network at this point. Even if connman was started, `timedata` needs at least 5 peer connections to calculate an offset.
Fix the confusion by replacing `GetAdjustedTime` with `GetTime`, which does not change behavior.
Also:
* Replace magic number with `MAX_FUTURE_BLOCK_TIME` to clarify the context
* Add test, which passes both on current master and this pull request
* An unrelated refactoring commit, happy to drop
ACKs for top commit:
dongcarl:
Code Review ACK fa551b3bdd, noticed the exact same thing here: e073634c37
mzumsande:
Code Review ACK fa551b3bdd
jnewbery:
Code review ACK fa551b3bdd
shaavan:
ACK fa551b3bdd
theStack:
Code-review ACK fa551b3bdd
Tree-SHA512: 15807a0e943e3e8d8c5250c8f6d7b56afb26002b1e290bf93636a2c747f27e78f01f1de04ce1a83d6339e27284c69c43e077a8467545c4078746f4c1ecb1164d
3333070208 refactor: Call type-solver earlier in decodescript (MarcoFalke)
fab0d998f4 style: Remove whitespace (MarcoFalke)
Pull request description:
The current logic is a bit confusing. First creating the `UniValue` return dict, then parsing it again to get the type as `std::string`.
Clean this up by using a strong type `TxoutType`. Also, remove whitespace.
ACKs for top commit:
shaavan:
ACK 3333070208
theStack:
Code-review ACK 3333070208
Tree-SHA512: 49db7bc614d2491cd3ec0177d21ad1e9924dbece1eb5635290cd7fd18cb30adf4711b891daf522e7c4f6baab3033b66393bbfcd1d4726f24f90a433124f925d6
ffd09281fe rpc: various fixups for dumptxoutset (James O'Beirne)
Pull request description:
This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: #15606)
---
A few fixes to make this RPC actually useful when generating snapshots.
- Generate an assumeutxo hash and display it (sort of a bugfix)
- Add nchaintx to output (necessary for use in chainparams entry)
- Add path of serialized UTXO file to output
ACKs for top commit:
laanwj:
Code review ACK ffd09281fe
Tree-SHA512: b0b5fd5138dea0e21258b1b18ab75bf3fd1628522cc1dbafa81af9cb9fa96562a1c39124fdb31057f256bfc560f462f907e9fe5e209b577b3f57afae2b7be826
fa7da227da refactor: Fix implicit-signed-integer-truncation in cuckoocache.h (MarcoFalke)
Pull request description:
Using a file-wide suppression for this implicit truncation has several issues:
* It is file-wide, thus suppressing any other (newly introduced) issues
* The file doesn't compile with `-Wimplicit-int-conversion`
Fix both issues by making the truncation explicit.
ACKs for top commit:
fanquake:
ACK fa7da227da
Tree-SHA512: bf2076ed94c4e80d0d29ff883080edc7a73144c73d6d3e872ec87966177ee3160f4760fc4c774aaa6fb591f4acee450a24b0f7c82291e0bef96582a6d134046e
ff945e553a MOVEONLY: Move utility functions from rpcwallet to wallet/rpc/util (Samuel Dobson)
7b04a064f6 Introduce wallet/rpc/util (Samuel Dobson)
Pull request description:
This is part one of multiple to split up rpcwallet.cpp into smaller, more logical units.
See #23622 for context and overall plan. I'll open PRs in stages to hopefully minimise conflicts.
Can be reviewed with `--color-moved=dimmed-zebra`
The end goal can be seen here: https://github.com/meshcollider/bitcoin/tree/202111_split_walletrpc
ACKs for top commit:
MarcoFalke:
nice, ACK ff945e553a🐰
shaavan:
ACK ff945e553a
Tree-SHA512: 6e3d1de6db770fe2fca540c8c4f30183ab8258c26e3a1fb46937714d28818a52933eafbfcafe2995f6a6e2551a3f3dd3ec93363b81de7912c0d81f5749d1c86d
fa46ac4d9d miner: Remove uncompiled MTP code (MarcoFalke)
fa6b7adf96 style: Add {} to if-bodies in node/miner (MarcoFalke)
Pull request description:
This removes uncompiled code.
Can be checked by inserting `static_assert(STANDARD_LOCKTIME_VERIFY_FLAGS & LOCKTIME_MEDIAN_TIME_PAST)` and compiling or by reading the source code.
Even if the code was compiled, it would be unsafe to execute, since it is not allowed to include transactions that are locked until some time after the current MTP.
Also, rename the member to cause explicit merge conflicts in case there is a patch out there referencing the variable.
ACKs for top commit:
shaavan:
ACK fa46ac4d9d
theStack:
Code-review ACK fa46ac4d9d
Tree-SHA512: 0288f45918996b58d0c0060773aa3cb15c828a649439f3d589c5d6b4854d6da1d8c2ea11d5ca06c654532453ab5ce1892de7ca820e284e96e78b959ef87cac5c
a64078e385 Break validation <-> txmempool circular dependency (glozow)
64e4963c63 [mempool] always assert coin spent (glozow)
bb9078ed51 [refactor] put finality and maturity checking into a lambda (glozow)
bedf246f1e [mempool] only update lockpoints for non-removed entries (glozow)
1b3a11e126 MOVEONLY: TestLockPointValidity to txmempool (glozow)
Pull request description:
Remove 2 circular dependencies: validation - txmempool and validation - policy/rbf - txmempool
Validation should depend on txmempool (e.g. `CChainstateManager` has a mempool and we often need to know what's in our mempool to validate transactions), but txmempool is a data structure that shouldn't really need to know about chain state.
- Changes `removeForReorg()` to be parameterized by a callable that returns true/false (i.e. whether the transaction should be removed due to being now immature or nonfinal) instead of a `CChainState`. The mempool really shouldn't need to know about coinbase maturity or lockpoints, it just needs to know which entries to remove.
ACKs for top commit:
laanwj:
Code review ACK a64078e385
mjdietzx:
reACK a64078e385
theStack:
re-ACK a64078e385
Tree-SHA512: f75995200569c09dfb8ddc09729da66ddb32167ff1e8a7e72f105ec062d2d6a9a390e6b4a2a115e7ad8ad3525f891ee1503f3cd2bed11773abcaf7c3230b1136
123f5de826 Remove calls to global Params() in tx_pool test (lsilva01)
9360778d6e Remove AcceptToMemoryPoolWithTime (lsilva01)
Pull request description:
This PR refactors AcceptToMemoryPool.
. Remove `AcceptToMemoryPoolWithTime` (after #23173, this function is no longer needed).
. Remove the `CChainParams chainparams` parameter from ATMP as they can be inferred from the current chain state.
. Update the `tx_pool` test with new function signature.
ACKs for top commit:
jnewbery:
reACK 123f5de826
glozow:
light code review ACK 123f5de826
theStack:
re-ACK 123f5de826
Tree-SHA512: b672d9f091f5fcb4d163c0a443aa469b2ce6f37201136803201c620c3fa329ffb41abd50c5b528787fd1c47e8e09af5ec20ddaa0a4235352c9c619e5691dddb6
c771ee8571 doc: use BIP125-replaceable (fanquake)
36dc5bb8cb doc: Extract CreateTxDoc in rawtransaction (fanquake)
Pull request description:
For the fields: inputs, outputs, locktime, replaceable. Similar to #23172.
Can be reviewed with `--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space`.
ACKs for top commit:
MarcoFalke:
ACK c771ee8571😸
Tree-SHA512: e6e4211b89bedec472f8381b3cbea5670f82b728955888c794f694164b8d8bdd51a99c64762b625357ac2171005712b82f81ee7c1b8f5c5620bdedeeefa2b9da
fa00447442 scripted-diff: Use clang-tidy syntax for C++ named arguments (MarcoFalke)
fae13c3989 doc: Use clang-tidy comments in crypto_tests (MarcoFalke)
Pull request description:
Incorrect named args are source of bugs, like #22979.
To allow them being checked by `clang-tidy`, use a format it can understand.
ACKs for top commit:
shaavan:
ACK fa00447442
rajarshimaitra:
ACK fa00447442
jonatack:
ACK fa00447442
fanquake:
ACK fa00447442
Tree-SHA512: 4d23a8363da81dfea21a4cd8516ab5e0dc70119e4d503f3f240f38573218b2c2e84083b97e956c62942d78b2f17490f8b3b2e8077d257644fda1d901e2b80507
6c9ee92ffe net: don't check if the listening socket is valid (Vasil Dimov)
Pull request description:
_This is a piece of #21878, chopped off to ease review._
Listening sockets in `CConnman::vhListenSocket` are always valid
(underlying file descriptor is not `INVALID_SOCKET`).
ACKs for top commit:
theStack:
Code-review ACK 6c9ee92ffe🔌
Tree-SHA512: b2e29711c6a0c7c85467ca61cfd7fb734eb06bd83a41f88735901caf90aec095ca80707ce5bb897d39c80fdec16819dbf5a84979c9b1ab3dc3fb8b08cebe7c61
faad05c6d2 Crash debug builds when mempool ConsensusScriptChecks fails (MarcoFalke)
Pull request description:
Currently a bug in the function might sneak around our testing infrastructure.
Fix that by turning bugs into crashes during tests.
ACKs for top commit:
glozow:
utACK faad05c6d2, there's something seriously wrong with the code if this returns false, good to throw in debug mode
Tree-SHA512: dfea1cd9ce3f1c303f49cca1417cd5c77c6ed12849aaff7b6ab1b6060f2f0c9cf5d4689017355d11f66639bab35823f65f848e6979042fa875181509dfd5d3d7
d9803f7a0a test: add stress tests for initialization (James O'Beirne)
23f85616a8 test: add node.chain_path and node.debug_log_path (James O'Beirne)
Pull request description:
In the course of coming up with a test plan for #23280, I thought it would be neat to include a Python snippet showing how I tested the initialization process. I quickly realized I was reinventing the functional test framework... so here's a new test.
This change bangs init around like the Fonz hitting a jukebox. It adds some interesting (read: lazy and random) coverage for the initialization process by
- interrupting init with SIGTERM after certain log statements,
- interrupting init at random points, and
- starting init with some essential data missing (block files, block indices, etc.) to test init error paths.
As far as I can tell, some of these code paths are uncovered otherwise (namely the startup errors).
---
Incidentally, I think I may have uncovered some kind of a bug or race condition with indexing initialization based on an intermittent failure in this testcase. This test sometimes fails after shutting down immediately after `loadblk` thread start:
```
2021-10-15T21:14:51.295000Z TestFramework (INFO): Starting node and will exit after line 'loadblk thread start'
36 │ 2021-10-15T21:14:51.296000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up
37 │ 2021-10-15T21:14:51.493000Z TestFramework (INFO): terminating node after 110 log lines seen
38 │ 2021-10-15T21:14:51.625000Z TestFramework (INFO): Starting node and will exit after line 'txindex thread start'
39 │ 2021-10-15T21:14:51.625000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up
------> [[ FAILURE HERE ]] 2021-10-15T21:15:21.626000Z TestFramework (WARNING): missed line {bail_line}; bailing now after {num_lines} lines
```
and then fails to start up afterwards. Combined logs showing `Error: txindex best block of the index goes beyond pruned data`, when the node under test is not pruned:
```
node0 2021-10-15T21:16:51.848439Z [shutoff] [validationinterface.cpp:244] [ChainStateFlushed] Enqueuing ChainStateFlushed: block hash=1014bc4ff4917602ae53d10e9dfe230af4b7d52a6cdaa8a47798b9c288180907
node0 2021-10-15T21:16:51.848954Z [shutoff] [init.cpp:302] [Shutdown] Shutdown: done
test 2021-10-15T21:16:51.882000Z TestFramework (ERROR): Unexpected exception caught during testing
Traceback (most recent call last):
File "/home/james/src/bitcoin/test/functional/test_framework/test_framework.py", line 132, in main
self.run_test()
File "/home/james/src/bitcoin/./test/functional/stress_init.py", line 87, in run_test
check_clean_start()
File "/home/james/src/bitcoin/./test/functional/stress_init.py", line 60, in check_clean_start
node.wait_for_rpc_connection()
File "/home/james/src/bitcoin/test/functional/test_framework/test_node.py", line 224, in wait_for_rpc_connection
raise FailedToStartError(self._node_msg(
test_framework.test_node.FailedToStartError: [node 0] bitcoind exited with status 1 during initialization
test 2021-10-15T21:16:51.882000Z TestFramework (DEBUG): Closing down network thread
test 2021-10-15T21:16:51.933000Z TestFramework (INFO): Stopping nodes
test 2021-10-15T21:16:51.933000Z TestFramework.node0 (DEBUG): Stopping node
node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
```
ACKs for top commit:
laanwj:
Code review ACK d9803f7a0a
Tree-SHA512: 4d80dc399daf199a6222e81e47d12d830dc7af07355eddbb7f52479a676a645b8d3d45093ff54a9295f01a163b2f4fe0e038e83fc269969e03d4cfda69eaf111
a3f61676e8 test: Make more shell scripts verifiable by the `shellcheck` tool (Hennadii Stepanov)
Pull request description:
Some shell scripts from `contrib/guix` and `contrib/shell` are not verifiable by the `shellcheck` tool for the following reasons:
- they have no extension (see 4eccf063b2 from bitcoin/bitcoin#21375)
- they have the `.bash` extension while `.sh` is expected
This PR adds these scripts to the input for the `shellcheck` tool, and it fixes discovered `shellcheck` warnings.
ACKs for top commit:
dongcarl:
Code Review ACK a3f61676e8, this is a good robustness improvement for our shell scripts.
jamesob:
crACK a3f61676e8
Tree-SHA512: 6703f5369d9c04c1a174491f381afa5ec2cc4d37321c1b93615abcdde4dfd3caae82868b699c25b72132d8c8c6f2e9cf24d38eb180ed4d0f0584d8c282e58935
6aed8b7e9b [test] tx processing before and after ibd (glozow)
b9e105b664 [net_processing] ignore all transactions during ibd (glozow)
Pull request description:
This is basically a mini, IBD-only version of #21224
Incoming transactions aren't really relevant until we're caught up. That's why we send a giant feefilter and don't send tx getdatas, but we also shouldn't process them if peers send them anyway. Simply ignore them.
ACKs for top commit:
jnewbery:
reACK 6aed8b7e9b
laanwj:
Code review ACK 6aed8b7e9b
Tree-SHA512: 8e1616bf355f9d0b180bdbc5461f24c757dc5d7bc7bf651470f3b0bffcca5d5e68287106255b5cede2d96b42bce448a0f8c0649de35a530c5e079f7c89c70a35
- Actually generate an assumeutxo hash and display it
- Add nchaintx to output (necessary for use in chainparams entry)
- Add path of serialized UTXO file to output
No behavior change.
Parameterize removeForReorg using a CChain and callable that
encapsulates validation logic. The mempool shouldn't need to know a
bunch of details about coinbase maturity and lock finality. Instead,
just pass in a callable function that says true/false. Breaks circular
dependency by removing txmempool's dependency on validation.
Each entry's lockpoints are independent of one another, so there isn't
any reason to update lockpoints for entries that will be removed.
Separating the loops also allows us to move validation logic out and
leave lockpoints in txmempool.
