Commit graph

36051 commits

Author SHA1 Message Date
Andrew Chow
1647a11f39 tests: Reorder longer running tests in test_runner
The logest running tests should be at the front of the list in
test_runner.py. Since compiling with --enable-debug can have a
significant effect on test runtime, the order is based on the runtime
with that option configured.
2022-12-09 13:57:55 -05:00
Andrew Chow
ff6c9fe027 tests: Whitelist test p2p connection in rpc_packages
test_submit_child_with_parents creates a p2p connection which waits for
the node to announce transactions to it. By whitelisting this
connection, we can reduce the amount of time spent waiting for this
announcement which improves the test runtime and runtime variance.
2022-12-09 13:57:01 -05:00
Andrew Chow
8c20796aac tests: Use waitfornewblock for work queue test in interface_rpc
The work queue exceeded test in interface_rpc.py would repeatedly call
an RPC until the error was achieved. However hitting this error is
dependent on the processing speed of the computer and the optimization
level of the binary. Configurations that result in slower processing
would result in the RPC used being processed before the error could be
hit, resulting the test's runtime having a high variance.

Switching the RPC to waitfornewblock allows it to run in a much more
consistent time that is still fairly fast. waitfornewblock forces
the RPC server to allocate a thread and wait, occupying a spot in the
work queue. This is perfect for this test because the slower the RPC,
the more likely we will achieve the race condition necessary to pass the
test. Using a timeout of 500 ms appears to work reliably without causing
the test to take too long.
2022-12-09 13:57:01 -05:00
Andrew Chow
6c872d5e65 tests: Initialize sigops draining script with bytes in feature_taproot
The sigops draining script in feature_taproot's block_submit was
initialized with a list that would end up always being iterated by
CScript's constructor. Since this list is very large, a lot of time
would be wasted. By creating and passing a bytes object initialized from
that list, we can avoid this iteration and dramatically improve the
runtime of feature_taproot.
2022-12-09 13:57:01 -05:00
Andrew Chow
544cbf776c tests: Use batched RPC in feature_fee_estimation
feature_fee_estimation has a lot of loops that hit the RPC many times in
succession in order to setup scenarios. Using batched requests for these
can reduce the test's runtime without effecting the test's behavior.
2022-12-09 13:57:01 -05:00
Andrew Chow
4ad7272f8b tests: reduce number of generated blocks for wallet_import_rescan
Generating blocks is slow, especially when --enable-debug. There is no
need to generate a new block for each transaction, so avoid doing that
to improve this test's runtime.
2022-12-09 13:57:01 -05:00
MarcoFalke
9e229a542f
Merge bitcoin/bitcoin#26601: test: Move wallet tests to wallet_*.py
fa7d71accc test: Move rpc_fundrawtransaction.py to wallet_fundrawtransaction.py (MarcoFalke)
fa933d6985 test: Move feature_backwards_compatibility.py to wallet_backwards_compatibility.py (MarcoFalke)

Pull request description:

  The tests only tests the wallet and it doesn't make sense to extend it for other stuff, so clarify that.

ACKs for top commit:
  fanquake:
    ACK fa7d71accc
  pablomartin4btc:
    re-ACK fa7d71accc

Tree-SHA512: 9dc131ed8ff119bd6d43d04ecc5c96d02f2d6fdc3d0805492774a414c0fbe6a984da7631154122a080c54ddd25fc5a5bdd52b282c918fce4932057ba72c2bf71
2022-12-09 16:34:48 +01:00
MarcoFalke
fa7d71accc
test: Move rpc_fundrawtransaction.py to wallet_fundrawtransaction.py 2022-12-09 11:54:29 +01:00
MarcoFalke
fa933d6985
test: Move feature_backwards_compatibility.py to wallet_backwards_compatibility.py 2022-12-09 11:54:17 +01:00
MarcoFalke
16624e6ff3
Merge bitcoin/bitcoin#26660: test: Use last release in compatibility tests
fabb24cbef test: Use last release in compatibility tests (MarcoFalke)

Pull request description:

  In compatibility tests it makes sense to always use the last release without the new feature, as it is likely more in use than any even older previous release.

ACKs for top commit:
  Sjors:
    utACK fabb24c

Tree-SHA512: beb854f4d28ba313282e1e0303abb0e09377828b138bde5a3e209337210b6b4c24855ab90a68f8789387001e4ca33b15cc37dbc9b7809929f4e7d1b69833a527
2022-12-09 09:25:52 +01:00
MarcoFalke
6d11f19cf5
Merge bitcoin/bitcoin#26658: test: Fix backwards compatibility intermittent failure
c29bff5b91 test: Fix backwards compatibility intermittent failure (Aurèle Oulès)

Pull request description:

  Fixes #24400.

  See https://github.com/bitcoin/bitcoin/issues/24400#issuecomment-1341531696 to reproduce the failure.

  As MarcoFalke suggested in https://github.com/bitcoin/bitcoin/issues/24400#issuecomment-1342282165, it can happen that the wallet is not fully flushed before being copied over to the other node. Fixed by unloading the wallet before copying the file.

ACKs for top commit:
  Sjors:
    utACK c29bff5

Tree-SHA512: ea93b859878873d07fa44ccb1c5b4c1b1014be8568dc2b9bbef02bb7a9230fefa7a71b149eb553870d3f4e986263342ea4f996fa0221098a2244f38952100471
2022-12-09 09:21:54 +01:00
fanquake
3eaf7be6ad
Merge bitcoin/bitcoin#24279: build: Make $(package)_*_env available to all $(package)_*_cmds
affbf58a1e build: Move environment variables into `$(package)_config_env` (Hennadii Stepanov)
d44fcd3c97 build: Make $(package)_*_env available to all $(package)_*_cmds (Hennadii Stepanov)

Pull request description:

  On master (1e7564eca8) the depends build system, which is based on pure GNU Make, works, but it lacks robustness, and in some corner cases it fails. For example, see bitcoin/bitcoin#22552.

  Another [bug](https://github.com/bitcoin/bitcoin/issues/22719) in the depends build system has already become a problem at least two times in the past (https://github.com/bitcoin/bitcoin/pull/16883#issuecomment-683817472 and https://github.com/bitcoin/bitcoin/pull/24134). Each time the problem was solved with other means.

  The initial [solution](https://github.com/bitcoin/bitcoin/pull/19882) had some discussion. Also it was discussed on the IRC meeting in #bitcoin-core-builds channel. This PR, actually, is a resurrection of it, as the bug silently struck pretty [recently](https://github.com/bitcoin/bitcoin/pull/24134).

  The bug is well described in bitcoin/bitcoin#22719.

  Here is another, a bit simpler description, which requires only basic shell (bash, dash etc) experience.
  After creating targets by this code:1e7564eca8/depends/funcs.mk (L280) a "draft" line of recipe like `$($(1)_config_env) $(call $(1)_config_cmds, $(1))` becomes a shell command sequence `VAR1=foo VAR2=bar command1 && command2` which is supposed to be executed in a [new sub-shell](https://www.gnu.org/software/make/manual/html_node/Execution.html#Execution).

  Please note that `VAR1=foo VAR2=bar` part is visible for the first `command1` only (for details see shell docs). Example:
  ```sh
  $ VAR1="foo" VAR2="bar" echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
  begin
  $ echo $?
  1
  ```

  Using the `export` command is a trivial solution:
  ```sh
  $ export VAR1="foo" VAR2="bar"; echo "begin" && printenv VAR1 && printenv VAR2 && echo "end"
  begin
  foo
  bar
  end
  $ echo $?
  0
  ```

  As a [new sub-shell](https://www.gnu.org/software/make/manual/html_node/Execution.html#Execution) is invoked for each line of the recipe, there are no side effects of using `export`. It means this solution should not be considered invasive.

  Fixes bitcoin/bitcoin#22719.

  ---

  Also this PR removes no longer needed crutch from `qt.mk`.

ACKs for top commit:
  fanquake:
    ACK affbf58a1e

Tree-SHA512: 0ce2cf82870a7774bdf1592fac50857126ae47da902e349f1092d50109223be9d6a8efd5e92ec08c2ca775b17516482aabaf232378950ade36484a883acc177b
2022-12-08 16:41:40 +00:00
MarcoFalke
5126e625cb
Merge bitcoin/bitcoin#26378: refactor: Pass reference to last header, not pointer
fa579f3063 refactor: Pass reference to last header, not pointer (MacroFake)

Pull request description:

  It is never a nullptr, otherwise an assertion would fire in UpdatePeerStateForReceivedHeaders.

  Passing a reference makes the code easier to read and less brittle.

ACKs for top commit:
  john-moffett:
    ACK fa579f3
  aureleoules:
    ACK fa579f3063

Tree-SHA512: 9725195663a31df57ae46bb7b11211cc4963a8f3d100f60332bfd4a3f3327a73ac978b3172e3007793cfc508dfc7c3a81aab57a275a6963a5ab662ce85743fd0
2022-12-08 17:04:05 +01:00
fanquake
07ac7a2dbf
Merge bitcoin/bitcoin#26513: Make static nLastFlush and nLastWrite Chainstate members
07dfbb5bb8 Make static nLastFlush and nLastWrite Chainstate members (Aurèle Oulès)

Pull request description:

  Fixes #22189.

  The `static std::multimap<uint256, FlatFilePos> mapBlocksUnknownParent; ` referenced in the issue was already fixed by #25571. I don't believe Chainstate references any other static variables.

ACKs for top commit:
  jamesob:
    ACK 07dfbb5bb8 ([`jamesob/ackr/26513.1.aureleoules.make_static_nlastflush_a`](https://github.com/jamesob/bitcoin/tree/ackr/26513.1.aureleoules.make_static_nlastflush_a))
  theStack:
    Concept and code-review ACK 07dfbb5bb8

Tree-SHA512: 0f26463c079bbc5e0e62707d4ca4c8c9bbb99edfa3391d48d4915d24e2a1190873ecd4f9f11da25b44527671cdc82c41fd8234d56a4592a246989448d34406b0
2022-12-08 15:35:28 +00:00
MarcoFalke
fabb24cbef
test: Use last release in compatibility tests 2022-12-08 14:57:25 +01:00
Aurèle Oulès
c29bff5b91
test: Fix backwards compatibility intermittent failure 2022-12-08 11:13:34 +01:00
MarcoFalke
1801d8c3c9
Merge bitcoin/bitcoin#26308: rpc/rest/zmq: reduce LOCK(cs_main) scope: ~6 times as many requests per second
d7f61e7d59 rpc: reduce LOCK(cs_main) scope in gettxoutproof (Andrew Toth)
4d92b5aaba rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats (Andrew Toth)
efd82aec8a rpc: reduce LOCK(cs_main) scope in blockToJSON (Andrew Toth)
f00808e932 rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock (Andrew Toth)
7d253c943f zmq: remove LOCK(cs_main) from NotifyBlock (Andrew Toth)
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block (Andrew Toth)

Pull request description:

  Picking up from #21006.

  After commit ccd8ef65f9 it is no longer required to hold `cs_main` when calling `ReadBlockFromDisk`. This can be verified in `master` at https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L755. Same can be seen for `UndoReadFromDisk` https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L485.

  The first commit moves `ReadBlockFromDisk` outside the lock scope in `rest_block`, where we can see a huge performance improvement when fetching blocks with multiple threads.

  My test setup, on an Intel i7 with 8 cores (16 threads):

  1. Start a fully synced bitcoind, with this `bitcoin.conf`:
  ```
      rest=1
      rpcthreads=16
      rpcworkqueue=64
      rpcuser=user
      rpcpassword=password
  ```
  2. Run ApacheBench: 10000 requests, 16 parallel threads, fetching block nr. 750000 in binary:
  ```
      ab -n 10000 -c 16 "http://127.0.0.1:8332/rest/block/0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e.bin"
  ```

  Time per request (mean)
      183 ms on master
      30 ms this branch

  So this can process 6.1 times as many requests, and saturates all the cores instead of keeping them partly idle waiting in the lock. With 8 threads the mean times were 90 ms on master and 19 ms on this branch, a speedup of 4.7x.

  Big thanks to martinus for finding this and the original PR.

  The second commit is from a suggestion on the original PR by jonatack to remove the unnecessary `LOCK(cs_main)` in the zmq notifier's `NotifyBlock`.

  I also found that this approach could be applied to rpcs `getblock` (including `verbosity=3`), `getblockstats`, and `gettxoutproof` with similar very good results. The above benchmarks steps need to be modified slightly for RPC. Run the following ApacheBench command with different request data in a file named `data.json`:
  ```
  ab -p data.json -n 10000 -c 16 -A user:password "http://127.0.0.1:8332/"
  ```
  For `getblock`, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e"]}
  ```
  master - 184 ms mean request time
  branch - 28 ms mean request time

  For `getblock` with verbosity level 3, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", 3]}
  ```
  This verbosity level fetches an undo file from disk, so it benefits from this approach as well. However, a lot of time is spent serializing to JSON so the performance gain is not as severe.
  master - 818 ms mean request time
  branch - 505 ms mean request time

  For `getblockstats`, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblockstats", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", ["minfeerate","avgfeerate"]]}
  ```
  This request used a lock on reading both a block and undo file, so the results are very good.
  master - 244 ms mean request time
  branch - 28 ms mean request time

ACKs for top commit:
  MarcoFalke:
    re-ACK d7f61e7d59 💫
  hebasto:
    ACK d7f61e7d59, I have reviewed the code and it looks OK. Did not make benchmarking though.

Tree-SHA512: 305ac945b4571c5f47646d4f0e78180d7a3d40b2f70ee43e4b3e00c96a465f6d0b9c750b8e85c89ed833e557e2cdb5896743f07ef90e4e53d4ad85452b545886
2022-12-08 10:48:02 +01:00
Andrew Chow
a653f4bb1f
Merge bitcoin/bitcoin#25934: wallet, rpc: add label to listsinceblock
4e362c2b72 doc: add release note for 25934 (brunoerg)
fe488b4c4b test: add coverage for `label` in `listsinceblock` (brunoerg)
722e9a418d wallet, rpc: add `label` to `listsinceblock` (brunoerg)
852891ff98 refactor, wallet: use optional for `label` in `ListTransactions` (brunoerg)

Pull request description:

  This PR adds `label` parameter to `listsinceblock` to be able to fetch all incoming transactions having the specified label since a specific block.

  It's possible to use it in `listtransactions`, however, it's only possible to set the number of transactions to return, not a specific block to fetch from. `getreceivedbylabel` only returns the total amount received, not the txs info. `listreceivedbylabel` doesn't list all the informations about the transactions and it's not possible to fetch since a block.

ACKs for top commit:
  achow101:
    ACK 4e362c2b72
  w0xlt:
    ACK 4e362c2b72
  aureleoules:
    ACK 4e362c2b72

Tree-SHA512: fbde5db8cebf7a27804154fa61997b5155ad512e978cebb78c17acab9efcb624ea5f39d649899d12e5e675f80d4d0064cae8132b864de0d93a8d1e6fbcb9a737
2022-12-07 18:42:41 -05:00
Andrew Chow
bbfcbcfa0c
Merge bitcoin/bitcoin#24611: Add fish completions
ccba4fe7e3 doc: Add completion subdir to contrib/README.md (willcl-ark)
7075848f96 script: Add fish completions (willcl-ark)
a27a445b71 refactor: Sub-folder bash completions (willcl-ark)

Pull request description:

  The completions are dynamically generated from the respective binary
  help pages.

  Completions should be sourced into the shell or added to
  `$XDG_CONFIG/fish/completions`. See [where to put completions](https://fishshell.com/docs/current/completions.html#where-to-put-completions) for more information.

  As the completions are auto-generated they should only require as much maintenance as the bash equivalents, which is to say very little!

ACKs for top commit:
  achow101:
    ACK ccba4fe7e3
  josibake:
    ACK ccba4fe7e3

Tree-SHA512: fe6ed899ea1fe90f82970bde7739db11dd0c845ccd70b65f28ad5212f75b57d9105a3a7f70ccdff552d5b21fa3fe9c697d128fb10740bae31fe1854e716b4b8b
2022-12-07 18:30:14 -05:00
Hennadii Stepanov
affbf58a1e
build: Move environment variables into $(package)_config_env 2022-12-07 16:51:48 +00:00
Hennadii Stepanov
d44fcd3c97
build: Make $(package)_*_env available to all $(package)_*_cmds 2022-12-07 16:51:35 +00:00
MarcoFalke
9052d869c9
Merge bitcoin/bitcoin#26517: test: Changed small_txpuzzle_randfee to return the virtual size instead of the transaction hex for feerate calculation.
6fb102c9f3 test: Changed small_txpuzzle_randfee to return the virtual size instead of the transaction hex for feerate calculation. (Randall Naar)

Pull request description:

  The fee rates used in feature_fee_estimation.py are calculated using the raw transaction size instead of the virtual transaction size (which is used in 'CBlockPolicyEstimator::processBlockTx' and 'CBlockPolicyEstimator::processBlock'). This leads to inconsistencies as the fee rates used in check_raw_estimates are incorrect and can cause assertions to fail.

  refs #25179

ACKs for top commit:
  MarcoFalke:
    ACK 6fb102c9f3

Tree-SHA512: b2bca85fffa605aeb17574f050736d4556506d782dd7fd969e165e48e108fd95ef4c4e2abbae83cce05ca777a00f4459cabfa0932694fa8bb93ddfba09d84357
2022-12-07 17:33:37 +01:00
fanquake
7d51560003
Merge bitcoin/bitcoin#26298: refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a
b19c4124b3 refactor: Rename ambiguous interfaces::MakeHandler functions (Ryan Ofsky)
dd6e8bd71c build: remove BOOST_CPPFLAGS from libbitcoin_util (fanquake)
82e272a109 refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a (Ryan Ofsky)

Pull request description:

  These belong in `libbitcoin_common.a`, not `libbitcoin_util.a`, because they aren't general-purpose utilities, they just contain some common glue code that is used by both the node and the wallet. Another reason not to include these in `libbitcoin_util.a` is to prevent them from being used by the kernel library.

  Also rename ambiguous `MakeHandler` functions to `MakeCleanupHandler` and `MakeSignalHandler`. Cleanup function handler was introduced after boost signals handler, so original naming didn't make much sense.

  This just contains a move-only commit, and a rename commit. There are no actual code or behavior changes.

  This PR is an alternative to #26293, and solves the same issue of removing a boost dependency from the _util_ library. The advantages of this PR compared to #26293 are that it keeps the source directory structure more flat, and it avoids having to change #includes all over the codebase.

ACKs for top commit:
  hebasto:
    ACK b19c4124b3

Tree-SHA512: b3a1d33eedceda7ad852c6d6f35700159d156d96071e59acae2bc325467fef81476f860a8855ea39cf3ea706a1df2a341f34fb2dcb032c31a3b0e9cf14103b6a
2022-12-07 14:54:23 +00:00
MarcoFalke
272fb0a5cf
Merge bitcoin/bitcoin#26645: util: Include full version id in bug reports
fa825bd227 util: Include full version id in bug reports (MarcoFalke)

Pull request description:

  This will show the unique id of the full source code when the bug occurred, which can help debugging

ACKs for top commit:
  1440000bytes:
    utACK fa825bd227
  theStack:
    ACK fa825bd227
  john-moffett:
    ACK fa825bd227

Tree-SHA512: a7a775718f5f9796b5cffafbb3ace8adb5c163414ec584a57143157fc9dfb86f799e3b9c8365fcb831ee1e9eafc59d699d1653d772c68392de421b3de74dcd61
2022-12-07 08:47:32 +01:00
Andrew Toth
d7f61e7d59 rpc: reduce LOCK(cs_main) scope in gettxoutproof 2022-12-06 15:07:04 -05:00
Andrew Toth
4d92b5aaba rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats 2022-12-06 15:07:04 -05:00
Andrew Toth
efd82aec8a rpc: reduce LOCK(cs_main) scope in blockToJSON 2022-12-06 15:07:04 -05:00
Andrew Toth
f00808e932 rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock 2022-12-06 15:07:04 -05:00
Andrew Toth
7d253c943f zmq: remove LOCK(cs_main) from NotifyBlock 2022-12-06 15:07:04 -05:00
Andrew Toth
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block 2022-12-06 15:07:04 -05:00
Hennadii Stepanov
0596aa40f7
Merge bitcoin-core/gui#683: doc: Drop no longer relevant comment
5d332da2cf doc: Drop no longer relevant comment (Hennadii Stepanov)

Pull request description:

  The comment was introduced in 4cf3411056, and since 7e4bd19785 it has been no longer relevant.

ACKs for top commit:
  jarolrod:
    ACK 5d332da2cf

Tree-SHA512: 6d32561336993b1ff7d7c524d090ac52aefb40078ed706ca4c6d5026cc3f63244c49c0e00e45ff192ba0e9f1527faf63249aa18bc8aa677b9e053d387e0f4027
2022-12-06 18:58:08 +00:00
brunoerg
4e362c2b72 doc: add release note for 25934 2022-12-06 15:27:50 -03:00
brunoerg
fe488b4c4b test: add coverage for label in listsinceblock 2022-12-06 15:27:50 -03:00
brunoerg
722e9a418d wallet, rpc: add label to listsinceblock 2022-12-06 15:27:50 -03:00
brunoerg
852891ff98 refactor, wallet: use optional for label in ListTransactions 2022-12-06 15:27:50 -03:00
fanquake
0abb5b2dd6
Merge bitcoin/bitcoin#26648: doc: move release notes to 24.0.1 and add notice
896fca16a3 doc: move release notes to 24.0.1 and add notice (fanquake)

Pull request description:

  This mirrors what was done with 0.19.0.1.

ACKs for top commit:
  instagibbs:
    ACK 896fca16a3
  gruve-p:
    ACK 896fca16a3
  w0xlt:
    ACK 896fca16a3

Tree-SHA512: 590462555422c0f96895152d1a2f9f9cf0ebf2c61dc342094d747f4d48b878e5d91840b9c2ac6825bb7214239f035789f1765a907fb614017205377ed89631fd
2022-12-06 18:14:52 +00:00
MarcoFalke
1ff79292e3
Merge bitcoin/bitcoin#26609: refactor: Move txmempool_entry.h --> kernel/mempool_entry.h
38941a703e refactor: Move `txmempool_entry.h` --> `kernel/mempool_entry.h` (Hennadii Stepanov)

Pull request description:

  This PR addresses the https://github.com/bitcoin/bitcoin/pull/17786#discussion_r1027818360:
  > why not move it to the right place, that is to `kernel/txmempool_entry.h`?

ACKs for top commit:
  MarcoFalke:
    review ACK 38941a703e 📊

Tree-SHA512: 0145974b63b67ca1d9d89af2dd9d4438beca480c16a563f330da05fec49b8394d7ba20ed83cf7d50b2e19454e006978ebed42b0e07887b98d00210f3201ce9ba
2022-12-06 19:04:31 +01:00
Andrew Chow
ef744c03e5
Merge bitcoin/bitcoin#25729: wallet: Check max transaction weight in CoinSelection
c7c7ee9d0b test: Check max transaction weight in CoinSelection (Aurèle Oulès)
6b563cae92 wallet: Check max tx weight in coin selector (Aurèle Oulès)

Pull request description:

  This PR is an attempt to fix #5782.

  I have added 4 test scenarios, 3 of them provided here https://github.com/bitcoin/bitcoin/issues/5782#issuecomment-73819058 (slightly modified to use a segwit wallet).

  Here are my benchmarks :
  ## PR
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        1,466,341.00 |              681.97 |    0.6% |   11,176,762.00 |    3,358,752.00 |  3.328 |   1,897,839.00 |    0.3% |      0.02 | `CoinSelection`

  ## Master

  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        1,526,029.00 |              655.30 |    0.5% |   11,142,188.00 |    3,499,200.00 |  3.184 |   1,994,156.00 |    0.2% |      0.02 | `CoinSelection`

ACKs for top commit:
  achow101:
    reACK c7c7ee9d0b
  w0xlt:
    ACK c7c7ee9d0b
  furszy:
    diff ACK c7c7ee9d

Tree-SHA512: ef0b28576ff845174651ba494aa9adee234c96e6f886d0e032eceb7050296e45b099dda0039d1dfb9944469f067627b2101f3ff855c70353cf39d1fc7ee81828
2022-12-06 12:08:58 -05:00
fanquake
896fca16a3
doc: move release notes to 24.0.1 and add notice
This mirrors what was done with 0.19.0.1.
2022-12-06 14:10:45 +00:00
MarcoFalke
8ccab65f28
Merge bitcoin/bitcoin#26238: clang-tidy: fixup named argument comments
203886c443 Fixup clang-tidy named argument comments (fanquake)

Pull request description:

  Fix comments so they are checked/consistent.
  Fix incorrect comments.

ACKs for top commit:
  hebasto:
    ACK 203886c443, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: e1257840f91fe39842e2b19299c1633604697b8584fe44b1977ada33cdde5433c877ed0b669fa334e20b04971dc89cd47d58b2783b6f7004521f01d05a1245da
2022-12-06 12:05:09 +01:00
MarcoFalke
fa825bd227
util: Include full version id in bug reports 2022-12-06 11:14:47 +01:00
MarcoFalke
edbe4f808a
Merge bitcoin/bitcoin#26611: wallet: Change coin selection fee assert to error
3eb041f014 wallet: Change coin selection fee assert to error (Andrew Chow)
c6e7f224c1 util: Add StrFormatInternalBug and STR_INTERNAL_BUG (MarcoFalke)

Pull request description:

  Returning an error instead of asserting for the low fee check will be better as it does not crash the node and instructs users to report the bug.

ACKs for top commit:
  S3RK:
    ACK 3eb041f014
  aureleoules:
    ACK 3eb041f014
  furszy:
    ACK 3eb041f0

Tree-SHA512: 118c13d7cdfce492080edd4cb12e6d960695377b978c7573f9c58b6d918664afd0e8e591eed0605d08ac756fa8eceed456349de5f3a025174069abf369bb5a5f
2022-12-06 10:31:02 +01:00
Andrew Chow
5d9b5305af
Merge bitcoin/bitcoin#19888: rpc, test: Improve getblockstats for unspendables
d885bb2f6e test: Test exclusion of OP_RETURN from getblockstats (Fabian Jahr)
ba9d288b24 test: Fix getblockstats test data generator (Fabian Jahr)
2ca5a496c2 rpc: Improve getblockstats (Fabian Jahr)
cb94db119f validation, index: Add unspendable coinbase helper functions (Fabian Jahr)

Pull request description:

  Fixes #19885

  The genesis block does not have undo data saved to disk so the RPC errored because of that.

ACKs for top commit:
  achow101:
    ACK d885bb2f6e
  aureleoules:
    ACK d885bb2f6e
  stickies-v:
    ACK d885bb2f6

Tree-SHA512: f37bda736ed605b7a41a81eeb4bfbb5d2b8518f847819e5d6a090548a61caf1455623e15165d72589ab3f4478252b00e7b624f9313ad6708cac06dd5edb62e9a
2022-12-05 17:46:54 -05:00
Andrew Chow
2ce3d26757
Merge bitcoin/bitcoin#26462: wallet: fix crash on loading descriptor wallet containing legacy key type entries
3198e4239e test: check that loading descriptor wallet with legacy entries throws error (Sebastian Falbesoner)
349ed2a0ee wallet: throw error if legacy entries are present on loading descriptor wallets (Sebastian Falbesoner)

Pull request description:

  Loading a descriptor wallet currently leads to a segfault if a legacy key type entry is present that can be deserialized successfully and needs SPKman-interaction. To reproduce with a "cscript" entry (see second commit for details):

  ```
  $ ./src/bitcoin-cli createwallet crashme
  $ ./src/bitcoin-cli unloadwallet crashme
  $ sqlite3 ~/.bitcoin/wallets/crashme/wallet.dat
  SQLite version 3.38.2 2022-03-26 13:51:10
  Enter ".help" for usage hints.
  sqlite> INSERT INTO main VALUES(x'07637363726970740000000000000000000000000000000000000000', x'00');
  $ ./src/bitcoin-cli loadwallet crashme

  --- bitcoind output: ---
  2022-11-06T13:51:01Z Using SQLite Version 3.38.2
  2022-11-06T13:51:01Z Using wallet /home/honey/.bitcoin/wallets/crashme
  2022-11-06T13:51:01Z init message: Loading wallet…
  2022-11-06T13:51:01Z [crashme] Wallet file version = 10500, last client version = 249900

  Segmentation fault (core dumped)
  ```

  Background: In the wallet key-value-loading routine, most legacy type entries require a `LegacyScriptPubKeyMan` instance after successful deserialization. On a descriptor wallet, creating that (via method `GetOrCreateLegacyScriptPubKeyMan`) fails and then leads to a null-pointer dereference crash. E.g. for CSCRIPT: 50422b770a/src/wallet/walletdb.cpp (L589-L594)

  ~~This PR fixes this by simply ignoring legacy entries if the wallet flags indicate that we have a descriptor wallet. The second commits adds a regression test to the descriptor wallet's functional test (fortunately Python includes sqlite3 support in the standard library).~~

  ~~Probably it would be even better to throw a warning to the user if unexpected legacy entries are found in descriptor wallets, but I think as a first mitigation everything is obvisouly better than crashing. As far as I'm aware, descriptor wallets created/migrated by Bitcoin Core should never end up in a state containing legacy type entries though.~~

  This PR fixes this by throwing an error if legacy entries are found in descriptor wallets on loading.

ACKs for top commit:
  achow101:
    ACK 3198e4239e
  aureleoules:
    ACK 3198e4239e

Tree-SHA512: ee43da3f61248e0fde55d9a705869202cb83df678ebf4816f0e77263f0beac0d7bae9490465d1753159efb093ee37182931d76b2e2b6e8c6f8761285700ace1c
2022-12-05 17:37:48 -05:00
Aurèle Oulès
c7c7ee9d0b
test: Check max transaction weight in CoinSelection
Co-authored-by: Andrew Chow <github@achow101.com>
2022-12-05 19:32:11 +01:00
Aurèle Oulès
6b563cae92
wallet: Check max tx weight in coin selector
Co-authored-by: Andrew Chow <github@achow101.com>
2022-12-05 19:32:11 +01:00
Andrew Chow
3eb041f014 wallet: Change coin selection fee assert to error
Returning an error instead of asserting for the low fee check will be
better as it does not crash the node and instructs users to report the
bug.
2022-12-05 12:59:22 -05:00
Andrew Chow
7734a0160d
Merge bitcoin/bitcoin#26640: test: Run mempool_compatibility.py with MiniWallet
fa43f60a0c test: Run mempool_compatibility.py with MiniWallet (MarcoFalke)

Pull request description:

  By using the already existing miniwallet, the test can be run even when no wallet is compiled.

ACKs for top commit:
  glozow:
    ACK fa43f60a0c
  achow101:
    ACK fa43f60a0c

Tree-SHA512: 6877b3f2f364663f04c28ab9f3d69780de6d1b77cc862379bba8c8242bbcfb0d26eb84c56cf721141407c393f1f3b49f667ae4fb32b3566108d71250e8b5d7bc
2022-12-05 12:39:15 -05:00
Andrew Chow
f0c4807a6a
Merge bitcoin/bitcoin#26560: wallet: bugfix, invalid CoinsResult cached total amount
7362f8e5e2 refactor: make CoinsResult total amounts members private (furszy)
3282fad599 wallet: add assert to SelectionResult::Merge for safety (S3RK)
c4e3b7d6a1 wallet: SelectCoins, return early if wallet's UTXOs cannot cover the target (furszy)
cac2725fd0 test: bugfix, coinselector_test, use 'CoinsResult::Erase/Add' instead of direct member access (furszy)
cf79384697 test: Coin Selection, duplicated preset inputs selection (furszy)
341ba7ffd8 test: wallet, coverage for CoinsResult::Erase function (furszy)
f930aefff9 wallet: bugfix, 'CoinsResult::Erase' is erasing only one output of the set (furszy)

Pull request description:

  This comes with #26559.

  Solving few bugs inside the wallet's transaction creation
  process and adding test coverage for them.
  Plus, making use of the `CoinsResult::total_amount` cached value
  inside the Coin Selection process to return early if we don't have
  enough funds to cover the target amount.

  ### Bugs

  1) The `CoinsResult::Erase` method removes only one
  output from the available coins vector (there is a [loop break](c1061be14a/src/wallet/spend.cpp (L112))
  that should have never been there) and not all the preset inputs.

     Which on master is not a problem, because since [#25685](https://github.com/bitcoin/bitcoin/pull/25685)
     we are no longer using the method. But, it's a bug on v24
     (check [#26559](https://github.com/bitcoin/bitcoin/pull/26559)).

     This method it's being fixed and not removed because I'm later using it to solve
     another bug inside this PR.

  2) As we update the total cached amount of the `CoinsResult` object inside
     `AvailableCoins` and we don't use such function inside the coin selection
     tests (we manually load up the `CoinsResult` object), there is a discrepancy
     between the outputs that we add/erase and the total amount cached value.

  ### Improvements

  * This makes use of the `CoinsResult` total amount field to early return
    with an "Insufficient funds" error inside Coin Selection if the tx target
    amount is greater than the sum of all the wallet available coins plus the
    preset inputs amounts (we don't need to perform the entire coin selection
    process if we already know that there aren't enough funds inside our wallet).

  ### Test Coverage

  1) Adds test coverage for the duplicated preset input selection bug that we have in v24.
    Where the wallet invalidly selects the preset inputs twice during the Coin Selection
    process. Which ends up with a "good" Coin Selection result that does not cover the
    total tx target amount. Which, alone, crashes the wallet due an insane fee.
    But.. to make it worst, adding the subtract fee from output functionality
    to this mix ends up with the wallet by-passing the "insane" fee assertion,
    decreasing the output amount to fulfill the insane fee, and.. sadly,
    broadcasting the tx to the network.

  2) Adds test coverage for the `CoinsResult::Erase` method.

  ------------------------------------

  TO DO:
  * [ ] Update [#26559 ](https://github.com/bitcoin/bitcoin/pull/26559) description.

ACKs for top commit:
  achow101:
    ACK 7362f8e5e2
  glozow:
    ACK 7362f8e5e2, I assume there will be a followup PR to add coin selection sanity checks and we can discuss the best way to do that there.
  josibake:
    ACK [7362f8e](7362f8e5e2)

Tree-SHA512: 37a6828ea10d8d36c8d5873ceede7c8bef72ae4c34bef21721fa9dad83ad6dba93711c3170a26ab6e05bdbc267bb17433da08ccb83b82956d05fb16090328cba
2022-12-05 12:00:45 -05:00
fanquake
203886c443
Fixup clang-tidy named argument comments
Fix comments so they are checked/consistent.
Fix incorrect arguments.
2022-12-05 15:51:46 +00:00