Commit graph

23212 commits

Author SHA1 Message Date
MarcoFalke
6061eb6564
Merge bitcoin/bitcoin#26199: p2p: Don't self-advertise during version processing
956c67059c refactor, doc: Improve SetupAddressRelay call in version processing (Martin Zumsande)
3c43d9db1e p2p: Don't self-advertise during VERSION processing (Gleb Naumenko)

Pull request description:

  This picks up the last commit from #19843.

  Previously, we would prepare to self-announce to a new peer while parsing a `version` message from that peer.
  This is redundant, because we do something very similar in `MaybeSendAddr()`, which is called from `SendMessages()` after
  the version handshake is finished.

  There are a couple of differences:

  1) `MaybeSendAddr()` self-advertises to all peers we do address relay with, not just outbound ones.
  2) `GetLocalAddrForPeer()` called from `MaybeSendAddr()` makes a probabilistic decision to either advertise what they think we are or what we think we are, while `PushAddress()` on `version` deterministically only does the former if the address from the latter is unroutable.
  3) During `version` processing, we haven't received a potential sendaddrv2 message from our peer yet, so self-advertisements with addresses from addrV2-only networks would always be dropped in `PushAddress()`.

  Since it's confusing to have two slightly different mechanisms for self-advertising, and the one in `MaybeSendAddr()` is better, remove the one in `version`.

ACKs for top commit:
  stratospher:
    ACK  956c670
  naumenkogs:
    ACK 956c67059c
  amitiuttarwar:
    reACK 956c67059c

Tree-SHA512: 933d40615289f055c022170dde7bad0ac0a1d4be377538bfe9ba64375cfeb03bcd803901591f0739ac4850c880e8475a68fd1ab0330800030ab7f19e38c00274
2022-12-12 10:12:09 +01:00
MarcoFalke
1ea02791f3
Merge bitcoin/bitcoin#26666: refactor: Deleted unreachable code in httpserver.cpp
8f5c560e11 refactor: Refactored RequestMethodString function to follow developer notes (JoaoAJMatos)
7fd3b9491b refactor: Deleted unreachable code in httpserver.cpp (JoaoAJMatos)

Pull request description:

  Some of the code in httpserver.cpp was unreachable, and didn't follow the developer notes.
  Continuation of [#26570 ](https://github.com/bitcoin/bitcoin/pull/26570)

ACKs for top commit:
  stickies-v:
    re-ACK [8f5c560](8f5c560e11)

Tree-SHA512: ba8cf4c6dde9e2bb0ca9d63a0de86dfa37b070803dde71ac8384c261045835697a2335652cf5894511b3af8fd99f30e1cbda4e4234815b8b39538ade90fab3f9
2022-12-10 13:03:22 +01:00
fanquake
a28fb36c47
Merge bitcoin/bitcoin#26673: univalue: Remove confusing getBool method
293849a260 univalue: Remove confusing getBool method (Ryan Ofsky)

Pull request description:

  Drop `UniValue::getBool` method because it is easy to confuse with the `UniValue::get_bool` method, and could potentially cause bugs. Unlike `get_bool`, `getBool` doesn't ensure that the value is a boolean and returns false for all integer, string, array, and object values instead of throwing an exception.

  The `getBool` method is also redundant because it is an alias for `isTrue`. There were only 5 `getBool()` calls in the codebase, so this commit replaces them with `isTrue()` or `get_bool()` calls as appropriate.

  These changes were originally made by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/26213 but were dropped to limit the scope of that PR.

ACKs for top commit:
  justinpickering:
    ACK 293849a260
  sipa:
    utACK 293849a260
  w0xlt:
    ACK 293849a260
  hebasto:
    ACK 293849a260, also verified that the removed `getBool` method is not mentioned in any docs:
  furszy:
    ACK 293849a2

Tree-SHA512: 9fbfe5e2083410f123b18703a0cc0161ecbbb4958f331c9ff808dcfcc6ad499b0e896abd16fb8ea200c53ba29878db9812ce141e59cc5e0fd174741b0bcb192d
2022-12-10 10:18:18 +00:00
fanquake
3b5fb6e77a
Merge bitcoin/bitcoin#26213: rpc: Strict type checking for RPC boolean parameters
fa0153e609 refactor: Replace isTrue with get_bool (MarcoFalke)
fa2cc5d1d6 bugfix: Strict type checking for RPC boolean parameters (MarcoFalke)

Pull request description:

ACKs for top commit:
  ryanofsky:
    Code review ACK fa0153e609
  furszy:
    Code ACK fa0153e6

Tree-SHA512: b221f823c69d90c94447fd491071ff3659cfd512872b495ebc3e711f50633351974102c9ef7e50fa4a393c4131d349adea8fd41cc9d66f1f31e1f5e7a5f78757
2022-12-10 09:58:33 +00:00
JoaoAJMatos
8f5c560e11 refactor: Refactored RequestMethodString function to follow developer notes
Removed the default case in the switch statement in order to comply with the Developer Notes
2022-12-09 16:14:27 +00:00
JoaoAJMatos
7fd3b9491b refactor: Deleted unreachable code in httpserver.cpp
Removed all break statements from both RequestMethodString and GetRequestMethod functions as they were unreachable
2022-12-09 16:13:57 +00:00
Ryan Ofsky
293849a260 univalue: Remove confusing getBool method
Drop UniValue::getBool method because it is easy to confuse with the
UniValue::get_bool method, and could potentially cause bugs. Unlike get_bool,
getBool doesn't ensure that the value is a boolean and returns false for all
integer, string, array, and object values instead of throwing an exceptions.

The getBool method is also redundant because it is an alias for isTrue. There
were only 5 getBool() calls in the codebase, so this commit replaces them with
isTrue() or get_bool() calls as appropriate.

These changes were originally made by MarcoFalke in
https://github.com/bitcoin/bitcoin/pull/26213 but were dropped to limit the
scope of that PR.

Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2022-12-09 10:03:26 -05:00
MarcoFalke
5126e625cb
Merge bitcoin/bitcoin#26378: refactor: Pass reference to last header, not pointer
fa579f3063 refactor: Pass reference to last header, not pointer (MacroFake)

Pull request description:

  It is never a nullptr, otherwise an assertion would fire in UpdatePeerStateForReceivedHeaders.

  Passing a reference makes the code easier to read and less brittle.

ACKs for top commit:
  john-moffett:
    ACK fa579f3
  aureleoules:
    ACK fa579f3063

Tree-SHA512: 9725195663a31df57ae46bb7b11211cc4963a8f3d100f60332bfd4a3f3327a73ac978b3172e3007793cfc508dfc7c3a81aab57a275a6963a5ab662ce85743fd0
2022-12-08 17:04:05 +01:00
fanquake
07ac7a2dbf
Merge bitcoin/bitcoin#26513: Make static nLastFlush and nLastWrite Chainstate members
07dfbb5bb8 Make static nLastFlush and nLastWrite Chainstate members (Aurèle Oulès)

Pull request description:

  Fixes #22189.

  The `static std::multimap<uint256, FlatFilePos> mapBlocksUnknownParent; ` referenced in the issue was already fixed by #25571. I don't believe Chainstate references any other static variables.

ACKs for top commit:
  jamesob:
    ACK 07dfbb5bb8 ([`jamesob/ackr/26513.1.aureleoules.make_static_nlastflush_a`](https://github.com/jamesob/bitcoin/tree/ackr/26513.1.aureleoules.make_static_nlastflush_a))
  theStack:
    Concept and code-review ACK 07dfbb5bb8

Tree-SHA512: 0f26463c079bbc5e0e62707d4ca4c8c9bbb99edfa3391d48d4915d24e2a1190873ecd4f9f11da25b44527671cdc82c41fd8234d56a4592a246989448d34406b0
2022-12-08 15:35:28 +00:00
MarcoFalke
1801d8c3c9
Merge bitcoin/bitcoin#26308: rpc/rest/zmq: reduce LOCK(cs_main) scope: ~6 times as many requests per second
d7f61e7d59 rpc: reduce LOCK(cs_main) scope in gettxoutproof (Andrew Toth)
4d92b5aaba rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats (Andrew Toth)
efd82aec8a rpc: reduce LOCK(cs_main) scope in blockToJSON (Andrew Toth)
f00808e932 rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock (Andrew Toth)
7d253c943f zmq: remove LOCK(cs_main) from NotifyBlock (Andrew Toth)
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block (Andrew Toth)

Pull request description:

  Picking up from #21006.

  After commit ccd8ef65f9 it is no longer required to hold `cs_main` when calling `ReadBlockFromDisk`. This can be verified in `master` at https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L755. Same can be seen for `UndoReadFromDisk` https://github.com/bitcoin/bitcoin/blob/master/src/node/blockstorage.cpp#L485.

  The first commit moves `ReadBlockFromDisk` outside the lock scope in `rest_block`, where we can see a huge performance improvement when fetching blocks with multiple threads.

  My test setup, on an Intel i7 with 8 cores (16 threads):

  1. Start a fully synced bitcoind, with this `bitcoin.conf`:
  ```
      rest=1
      rpcthreads=16
      rpcworkqueue=64
      rpcuser=user
      rpcpassword=password
  ```
  2. Run ApacheBench: 10000 requests, 16 parallel threads, fetching block nr. 750000 in binary:
  ```
      ab -n 10000 -c 16 "http://127.0.0.1:8332/rest/block/0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e.bin"
  ```

  Time per request (mean)
      183 ms on master
      30 ms this branch

  So this can process 6.1 times as many requests, and saturates all the cores instead of keeping them partly idle waiting in the lock. With 8 threads the mean times were 90 ms on master and 19 ms on this branch, a speedup of 4.7x.

  Big thanks to martinus for finding this and the original PR.

  The second commit is from a suggestion on the original PR by jonatack to remove the unnecessary `LOCK(cs_main)` in the zmq notifier's `NotifyBlock`.

  I also found that this approach could be applied to rpcs `getblock` (including `verbosity=3`), `getblockstats`, and `gettxoutproof` with similar very good results. The above benchmarks steps need to be modified slightly for RPC. Run the following ApacheBench command with different request data in a file named `data.json`:
  ```
  ab -p data.json -n 10000 -c 16 -A user:password "http://127.0.0.1:8332/"
  ```
  For `getblock`, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e"]}
  ```
  master - 184 ms mean request time
  branch - 28 ms mean request time

  For `getblock` with verbosity level 3, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblock", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", 3]}
  ```
  This verbosity level fetches an undo file from disk, so it benefits from this approach as well. However, a lot of time is spent serializing to JSON so the performance gain is not as severe.
  master - 818 ms mean request time
  branch - 505 ms mean request time

  For `getblockstats`, use the following in `data.json`:
  ```
  {"jsonrpc": "1.0", "id": "curltest", "method": "getblockstats", "params": ["0000000000000000000592a974b1b9f087cb77628bb4a097d5c2c11b3476a58e", ["minfeerate","avgfeerate"]]}
  ```
  This request used a lock on reading both a block and undo file, so the results are very good.
  master - 244 ms mean request time
  branch - 28 ms mean request time

ACKs for top commit:
  MarcoFalke:
    re-ACK d7f61e7d59 💫
  hebasto:
    ACK d7f61e7d59, I have reviewed the code and it looks OK. Did not make benchmarking though.

Tree-SHA512: 305ac945b4571c5f47646d4f0e78180d7a3d40b2f70ee43e4b3e00c96a465f6d0b9c750b8e85c89ed833e557e2cdb5896743f07ef90e4e53d4ad85452b545886
2022-12-08 10:48:02 +01:00
Andrew Chow
a653f4bb1f
Merge bitcoin/bitcoin#25934: wallet, rpc: add label to listsinceblock
4e362c2b72 doc: add release note for 25934 (brunoerg)
fe488b4c4b test: add coverage for `label` in `listsinceblock` (brunoerg)
722e9a418d wallet, rpc: add `label` to `listsinceblock` (brunoerg)
852891ff98 refactor, wallet: use optional for `label` in `ListTransactions` (brunoerg)

Pull request description:

  This PR adds `label` parameter to `listsinceblock` to be able to fetch all incoming transactions having the specified label since a specific block.

  It's possible to use it in `listtransactions`, however, it's only possible to set the number of transactions to return, not a specific block to fetch from. `getreceivedbylabel` only returns the total amount received, not the txs info. `listreceivedbylabel` doesn't list all the informations about the transactions and it's not possible to fetch since a block.

ACKs for top commit:
  achow101:
    ACK 4e362c2b72
  w0xlt:
    ACK 4e362c2b72
  aureleoules:
    ACK 4e362c2b72

Tree-SHA512: fbde5db8cebf7a27804154fa61997b5155ad512e978cebb78c17acab9efcb624ea5f39d649899d12e5e675f80d4d0064cae8132b864de0d93a8d1e6fbcb9a737
2022-12-07 18:42:41 -05:00
MarcoFalke
fa0153e609
refactor: Replace isTrue with get_bool
This makes the code more robust, see previous commit.

In general replacing isTrue with get_bool is not equivalent because
get_bool can throw exceptions, but in this case, exceptions won't happen
because of RPCTypeCheck() and isNull() checks in the preceding code.
2022-12-07 17:56:49 +01:00
MarcoFalke
fa2cc5d1d6
bugfix: Strict type checking for RPC boolean parameters 2022-12-07 17:55:58 +01:00
fanquake
7d51560003
Merge bitcoin/bitcoin#26298: refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a
b19c4124b3 refactor: Rename ambiguous interfaces::MakeHandler functions (Ryan Ofsky)
dd6e8bd71c build: remove BOOST_CPPFLAGS from libbitcoin_util (fanquake)
82e272a109 refactor: Move src/interfaces/*.cpp files to libbitcoin_common.a (Ryan Ofsky)

Pull request description:

  These belong in `libbitcoin_common.a`, not `libbitcoin_util.a`, because they aren't general-purpose utilities, they just contain some common glue code that is used by both the node and the wallet. Another reason not to include these in `libbitcoin_util.a` is to prevent them from being used by the kernel library.

  Also rename ambiguous `MakeHandler` functions to `MakeCleanupHandler` and `MakeSignalHandler`. Cleanup function handler was introduced after boost signals handler, so original naming didn't make much sense.

  This just contains a move-only commit, and a rename commit. There are no actual code or behavior changes.

  This PR is an alternative to #26293, and solves the same issue of removing a boost dependency from the _util_ library. The advantages of this PR compared to #26293 are that it keeps the source directory structure more flat, and it avoids having to change #includes all over the codebase.

ACKs for top commit:
  hebasto:
    ACK b19c4124b3

Tree-SHA512: b3a1d33eedceda7ad852c6d6f35700159d156d96071e59acae2bc325467fef81476f860a8855ea39cf3ea706a1df2a341f34fb2dcb032c31a3b0e9cf14103b6a
2022-12-07 14:54:23 +00:00
MarcoFalke
272fb0a5cf
Merge bitcoin/bitcoin#26645: util: Include full version id in bug reports
fa825bd227 util: Include full version id in bug reports (MarcoFalke)

Pull request description:

  This will show the unique id of the full source code when the bug occurred, which can help debugging

ACKs for top commit:
  1440000bytes:
    utACK fa825bd227
  theStack:
    ACK fa825bd227
  john-moffett:
    ACK fa825bd227

Tree-SHA512: a7a775718f5f9796b5cffafbb3ace8adb5c163414ec584a57143157fc9dfb86f799e3b9c8365fcb831ee1e9eafc59d699d1653d772c68392de421b3de74dcd61
2022-12-07 08:47:32 +01:00
Andrew Toth
d7f61e7d59 rpc: reduce LOCK(cs_main) scope in gettxoutproof 2022-12-06 15:07:04 -05:00
Andrew Toth
4d92b5aaba rpc: reduce LOCK(cs_main) scope in GetUndoChecked and getblockstats 2022-12-06 15:07:04 -05:00
Andrew Toth
efd82aec8a rpc: reduce LOCK(cs_main) scope in blockToJSON 2022-12-06 15:07:04 -05:00
Andrew Toth
f00808e932 rpc: reduce LOCK(cs_main) scope in GetBlockChecked and getblock 2022-12-06 15:07:04 -05:00
Andrew Toth
7d253c943f zmq: remove LOCK(cs_main) from NotifyBlock 2022-12-06 15:07:04 -05:00
Andrew Toth
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block 2022-12-06 15:07:04 -05:00
Hennadii Stepanov
0596aa40f7
Merge bitcoin-core/gui#683: doc: Drop no longer relevant comment
5d332da2cf doc: Drop no longer relevant comment (Hennadii Stepanov)

Pull request description:

  The comment was introduced in 4cf3411056, and since 7e4bd19785 it has been no longer relevant.

ACKs for top commit:
  jarolrod:
    ACK 5d332da2cf

Tree-SHA512: 6d32561336993b1ff7d7c524d090ac52aefb40078ed706ca4c6d5026cc3f63244c49c0e00e45ff192ba0e9f1527faf63249aa18bc8aa677b9e053d387e0f4027
2022-12-06 18:58:08 +00:00
brunoerg
722e9a418d wallet, rpc: add label to listsinceblock 2022-12-06 15:27:50 -03:00
brunoerg
852891ff98 refactor, wallet: use optional for label in ListTransactions 2022-12-06 15:27:50 -03:00
MarcoFalke
1ff79292e3
Merge bitcoin/bitcoin#26609: refactor: Move txmempool_entry.h --> kernel/mempool_entry.h
38941a703e refactor: Move `txmempool_entry.h` --> `kernel/mempool_entry.h` (Hennadii Stepanov)

Pull request description:

  This PR addresses the https://github.com/bitcoin/bitcoin/pull/17786#discussion_r1027818360:
  > why not move it to the right place, that is to `kernel/txmempool_entry.h`?

ACKs for top commit:
  MarcoFalke:
    review ACK 38941a703e 📊

Tree-SHA512: 0145974b63b67ca1d9d89af2dd9d4438beca480c16a563f330da05fec49b8394d7ba20ed83cf7d50b2e19454e006978ebed42b0e07887b98d00210f3201ce9ba
2022-12-06 19:04:31 +01:00
Andrew Chow
ef744c03e5
Merge bitcoin/bitcoin#25729: wallet: Check max transaction weight in CoinSelection
c7c7ee9d0b test: Check max transaction weight in CoinSelection (Aurèle Oulès)
6b563cae92 wallet: Check max tx weight in coin selector (Aurèle Oulès)

Pull request description:

  This PR is an attempt to fix #5782.

  I have added 4 test scenarios, 3 of them provided here https://github.com/bitcoin/bitcoin/issues/5782#issuecomment-73819058 (slightly modified to use a segwit wallet).

  Here are my benchmarks :
  ## PR
  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        1,466,341.00 |              681.97 |    0.6% |   11,176,762.00 |    3,358,752.00 |  3.328 |   1,897,839.00 |    0.3% |      0.02 | `CoinSelection`

  ## Master

  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |        1,526,029.00 |              655.30 |    0.5% |   11,142,188.00 |    3,499,200.00 |  3.184 |   1,994,156.00 |    0.2% |      0.02 | `CoinSelection`

ACKs for top commit:
  achow101:
    reACK c7c7ee9d0b
  w0xlt:
    ACK c7c7ee9d0b
  furszy:
    diff ACK c7c7ee9d

Tree-SHA512: ef0b28576ff845174651ba494aa9adee234c96e6f886d0e032eceb7050296e45b099dda0039d1dfb9944469f067627b2101f3ff855c70353cf39d1fc7ee81828
2022-12-06 12:08:58 -05:00
MarcoFalke
8ccab65f28
Merge bitcoin/bitcoin#26238: clang-tidy: fixup named argument comments
203886c443 Fixup clang-tidy named argument comments (fanquake)

Pull request description:

  Fix comments so they are checked/consistent.
  Fix incorrect comments.

ACKs for top commit:
  hebasto:
    ACK 203886c443, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: e1257840f91fe39842e2b19299c1633604697b8584fe44b1977ada33cdde5433c877ed0b669fa334e20b04971dc89cd47d58b2783b6f7004521f01d05a1245da
2022-12-06 12:05:09 +01:00
MarcoFalke
fa825bd227
util: Include full version id in bug reports 2022-12-06 11:14:47 +01:00
MarcoFalke
edbe4f808a
Merge bitcoin/bitcoin#26611: wallet: Change coin selection fee assert to error
3eb041f014 wallet: Change coin selection fee assert to error (Andrew Chow)
c6e7f224c1 util: Add StrFormatInternalBug and STR_INTERNAL_BUG (MarcoFalke)

Pull request description:

  Returning an error instead of asserting for the low fee check will be better as it does not crash the node and instructs users to report the bug.

ACKs for top commit:
  S3RK:
    ACK 3eb041f014
  aureleoules:
    ACK 3eb041f014
  furszy:
    ACK 3eb041f0

Tree-SHA512: 118c13d7cdfce492080edd4cb12e6d960695377b978c7573f9c58b6d918664afd0e8e591eed0605d08ac756fa8eceed456349de5f3a025174069abf369bb5a5f
2022-12-06 10:31:02 +01:00
Andrew Chow
5d9b5305af
Merge bitcoin/bitcoin#19888: rpc, test: Improve getblockstats for unspendables
d885bb2f6e test: Test exclusion of OP_RETURN from getblockstats (Fabian Jahr)
ba9d288b24 test: Fix getblockstats test data generator (Fabian Jahr)
2ca5a496c2 rpc: Improve getblockstats (Fabian Jahr)
cb94db119f validation, index: Add unspendable coinbase helper functions (Fabian Jahr)

Pull request description:

  Fixes #19885

  The genesis block does not have undo data saved to disk so the RPC errored because of that.

ACKs for top commit:
  achow101:
    ACK d885bb2f6e
  aureleoules:
    ACK d885bb2f6e
  stickies-v:
    ACK d885bb2f6

Tree-SHA512: f37bda736ed605b7a41a81eeb4bfbb5d2b8518f847819e5d6a090548a61caf1455623e15165d72589ab3f4478252b00e7b624f9313ad6708cac06dd5edb62e9a
2022-12-05 17:46:54 -05:00
Andrew Chow
2ce3d26757
Merge bitcoin/bitcoin#26462: wallet: fix crash on loading descriptor wallet containing legacy key type entries
3198e4239e test: check that loading descriptor wallet with legacy entries throws error (Sebastian Falbesoner)
349ed2a0ee wallet: throw error if legacy entries are present on loading descriptor wallets (Sebastian Falbesoner)

Pull request description:

  Loading a descriptor wallet currently leads to a segfault if a legacy key type entry is present that can be deserialized successfully and needs SPKman-interaction. To reproduce with a "cscript" entry (see second commit for details):

  ```
  $ ./src/bitcoin-cli createwallet crashme
  $ ./src/bitcoin-cli unloadwallet crashme
  $ sqlite3 ~/.bitcoin/wallets/crashme/wallet.dat
  SQLite version 3.38.2 2022-03-26 13:51:10
  Enter ".help" for usage hints.
  sqlite> INSERT INTO main VALUES(x'07637363726970740000000000000000000000000000000000000000', x'00');
  $ ./src/bitcoin-cli loadwallet crashme

  --- bitcoind output: ---
  2022-11-06T13:51:01Z Using SQLite Version 3.38.2
  2022-11-06T13:51:01Z Using wallet /home/honey/.bitcoin/wallets/crashme
  2022-11-06T13:51:01Z init message: Loading wallet…
  2022-11-06T13:51:01Z [crashme] Wallet file version = 10500, last client version = 249900

  Segmentation fault (core dumped)
  ```

  Background: In the wallet key-value-loading routine, most legacy type entries require a `LegacyScriptPubKeyMan` instance after successful deserialization. On a descriptor wallet, creating that (via method `GetOrCreateLegacyScriptPubKeyMan`) fails and then leads to a null-pointer dereference crash. E.g. for CSCRIPT: 50422b770a/src/wallet/walletdb.cpp (L589-L594)

  ~~This PR fixes this by simply ignoring legacy entries if the wallet flags indicate that we have a descriptor wallet. The second commits adds a regression test to the descriptor wallet's functional test (fortunately Python includes sqlite3 support in the standard library).~~

  ~~Probably it would be even better to throw a warning to the user if unexpected legacy entries are found in descriptor wallets, but I think as a first mitigation everything is obvisouly better than crashing. As far as I'm aware, descriptor wallets created/migrated by Bitcoin Core should never end up in a state containing legacy type entries though.~~

  This PR fixes this by throwing an error if legacy entries are found in descriptor wallets on loading.

ACKs for top commit:
  achow101:
    ACK 3198e4239e
  aureleoules:
    ACK 3198e4239e

Tree-SHA512: ee43da3f61248e0fde55d9a705869202cb83df678ebf4816f0e77263f0beac0d7bae9490465d1753159efb093ee37182931d76b2e2b6e8c6f8761285700ace1c
2022-12-05 17:37:48 -05:00
Aurèle Oulès
c7c7ee9d0b
test: Check max transaction weight in CoinSelection
Co-authored-by: Andrew Chow <github@achow101.com>
2022-12-05 19:32:11 +01:00
Aurèle Oulès
6b563cae92
wallet: Check max tx weight in coin selector
Co-authored-by: Andrew Chow <github@achow101.com>
2022-12-05 19:32:11 +01:00
Andrew Chow
3eb041f014 wallet: Change coin selection fee assert to error
Returning an error instead of asserting for the low fee check will be
better as it does not crash the node and instructs users to report the
bug.
2022-12-05 12:59:22 -05:00
Andrew Chow
f0c4807a6a
Merge bitcoin/bitcoin#26560: wallet: bugfix, invalid CoinsResult cached total amount
7362f8e5e2 refactor: make CoinsResult total amounts members private (furszy)
3282fad599 wallet: add assert to SelectionResult::Merge for safety (S3RK)
c4e3b7d6a1 wallet: SelectCoins, return early if wallet's UTXOs cannot cover the target (furszy)
cac2725fd0 test: bugfix, coinselector_test, use 'CoinsResult::Erase/Add' instead of direct member access (furszy)
cf79384697 test: Coin Selection, duplicated preset inputs selection (furszy)
341ba7ffd8 test: wallet, coverage for CoinsResult::Erase function (furszy)
f930aefff9 wallet: bugfix, 'CoinsResult::Erase' is erasing only one output of the set (furszy)

Pull request description:

  This comes with #26559.

  Solving few bugs inside the wallet's transaction creation
  process and adding test coverage for them.
  Plus, making use of the `CoinsResult::total_amount` cached value
  inside the Coin Selection process to return early if we don't have
  enough funds to cover the target amount.

  ### Bugs

  1) The `CoinsResult::Erase` method removes only one
  output from the available coins vector (there is a [loop break](c1061be14a/src/wallet/spend.cpp (L112))
  that should have never been there) and not all the preset inputs.

     Which on master is not a problem, because since [#25685](https://github.com/bitcoin/bitcoin/pull/25685)
     we are no longer using the method. But, it's a bug on v24
     (check [#26559](https://github.com/bitcoin/bitcoin/pull/26559)).

     This method it's being fixed and not removed because I'm later using it to solve
     another bug inside this PR.

  2) As we update the total cached amount of the `CoinsResult` object inside
     `AvailableCoins` and we don't use such function inside the coin selection
     tests (we manually load up the `CoinsResult` object), there is a discrepancy
     between the outputs that we add/erase and the total amount cached value.

  ### Improvements

  * This makes use of the `CoinsResult` total amount field to early return
    with an "Insufficient funds" error inside Coin Selection if the tx target
    amount is greater than the sum of all the wallet available coins plus the
    preset inputs amounts (we don't need to perform the entire coin selection
    process if we already know that there aren't enough funds inside our wallet).

  ### Test Coverage

  1) Adds test coverage for the duplicated preset input selection bug that we have in v24.
    Where the wallet invalidly selects the preset inputs twice during the Coin Selection
    process. Which ends up with a "good" Coin Selection result that does not cover the
    total tx target amount. Which, alone, crashes the wallet due an insane fee.
    But.. to make it worst, adding the subtract fee from output functionality
    to this mix ends up with the wallet by-passing the "insane" fee assertion,
    decreasing the output amount to fulfill the insane fee, and.. sadly,
    broadcasting the tx to the network.

  2) Adds test coverage for the `CoinsResult::Erase` method.

  ------------------------------------

  TO DO:
  * [ ] Update [#26559 ](https://github.com/bitcoin/bitcoin/pull/26559) description.

ACKs for top commit:
  achow101:
    ACK 7362f8e5e2
  glozow:
    ACK 7362f8e5e2, I assume there will be a followup PR to add coin selection sanity checks and we can discuss the best way to do that there.
  josibake:
    ACK [7362f8e](7362f8e5e2)

Tree-SHA512: 37a6828ea10d8d36c8d5873ceede7c8bef72ae4c34bef21721fa9dad83ad6dba93711c3170a26ab6e05bdbc267bb17433da08ccb83b82956d05fb16090328cba
2022-12-05 12:00:45 -05:00
fanquake
203886c443
Fixup clang-tidy named argument comments
Fix comments so they are checked/consistent.
Fix incorrect arguments.
2022-12-05 15:51:46 +00:00
MarcoFalke
5b3f05b7eb
Merge bitcoin/bitcoin#24226: rpc: warn that nodes ignore requests for old stale blocks
f39d9269eb rpc: warn that nodes ignore requests for old stale blocks (Sjors Provoost)

Pull request description:

  Adds warning to RPC help that `getblockfrompeer` is of little use for stale blocks that are more than a month old.

  This is an anti-fingerprinting measure. See `BlockRequestAllowed` in `net_processing`.

  It's been in Bitcoin Core since 2014, introduced in #2910 and later improved to not rely on checkpoints.
  Older and alternative clients might still serve these blocks, so not throwing an error.

  Allowing whitelisted nodes to fetch these blocks anyway might be nice.

ACKs for top commit:
  fjahr:
    Code review ACK f39d9269eb

Tree-SHA512: db88f9f7521289640c5e629c840dda1c2c3ab70d458e9e7136c60fbaeb02acfb36dc093502d83d4c098c331e22aab81bf8f4c4961d805e3bde0f8f3cfe68d968
2022-12-05 14:01:59 +01:00
glozow
8b796866b4
Merge bitcoin/bitcoin#26624: refactor: Rename local variable to distinguish it from type alias
1984db1d50 refactor: Rename local variable to distinguish it from type alias (Hennadii Stepanov)

Pull request description:

  The `txiter` type alias is declared in the `txmempool.h`: 9e59d21fbe/src/txmempool.h (L406)

ACKs for top commit:
  stickies-v:
    ACK 1984db1d5
  vasild:
    ACK 1984db1d50
  jarolrod:
    ACK 1984db1d50

Tree-SHA512: 127bfb62627e2d79d8cdb0bd0ac11b3737568c3631b54b2d1e37984f673a1f60edf7bc102a269f7eb40e4bb124b910b924a89475c6a6ea978b2171219fa30685
2022-12-05 11:15:41 +00:00
Hennadii Stepanov
5d332da2cf
doc: Drop no longer relevant comment
The comment was introduced in 4cf3411056,
and since 7e4bd19785 it has been no longer
relevant.
2022-12-02 15:44:34 +00:00
furszy
7362f8e5e2
refactor: make CoinsResult total amounts members private 2022-12-02 12:39:16 -03:00
S3RK
3282fad599
wallet: add assert to SelectionResult::Merge for safety 2022-12-02 12:39:15 -03:00
furszy
c4e3b7d6a1
wallet: SelectCoins, return early if wallet's UTXOs cannot cover the target
The CoinsResult class will now count the raw total amount and the effective
total amount internally (inside the 'CoinsResult::Add' and 'CoinsResult::Erase'
methods).
So there is no discrepancy between what we add/erase and the total values.
(which is what was happening on the coinselector_test because the 'CoinsResult'
object is manually created there, and we were not keeping the total amount
in sync with the outputs being added/removed).
2022-12-02 12:39:15 -03:00
furszy
cac2725fd0
test: bugfix, coinselector_test, use 'CoinsResult::Erase/Add' instead of direct member access
Aside from the cleanup, this solves a bug in the following-up commit. Because, in these
tests, we are manually adding/erasing outputs from the CoinsResult object but never
updating the internal total amount field.
2022-12-02 12:39:15 -03:00
furszy
cf79384697
test: Coin Selection, duplicated preset inputs selection
This exercises the bug inside CoinsResult::Erase that
ends up on (1) a wallet crash or (2) a created and
broadcasted tx that contains a reduced recipient's amount.

This is covered by making the wallet selects the preset
inputs twice during the coin selection process.

Making the wallet think that the selection process result covers
the entire tx target when it does not. It's actually creating
a tx that sends more coins than what inputs are covering for.

Which, combined with the SFFO option, makes the wallet
incorrectly reduce the recipient's amount by the difference
between the original target and the wrongly counted inputs.
Which means, a created and relayed tx sending less coins to
the destination than what the user inputted.
2022-12-02 12:39:15 -03:00
furszy
341ba7ffd8
test: wallet, coverage for CoinsResult::Erase function 2022-12-02 12:39:15 -03:00
fanquake
78aee0fe2c
Merge bitcoin/bitcoin#26569: p2p: Ensure transaction announcements are only queued for fully connected peers
8f2dac5409 [test] Add p2p_tx_privacy.py (dergoegge)
ce63fca13e [net processing] Assume that TxRelay::m_tx_inventory_to_send is empty pre-verack (dergoegge)
845e3a34c4 [net processing] Ensure transaction announcements are only queued for fully connected peers (dergoegge)

Pull request description:

  `TxRelay::m_next_inv_send_time` is initialized to 0, which means that any txids in `TxRelay::m_tx_inventory_to_send` will be announced on the first call to `PeerManagerImpl::SendMessages` for a fully connected peer (i.e. it completed the version handshake).

  Prior to #21160, `TxRelay::m_tx_inventory_to_send` was guaranteed to be empty on the first `SendMessages` call, as transaction announcements were only queued for fully connected peers. #21160 replaced a `CConnman::ForEachNode` call with a loop over `PeerManagerImpl::m_peer_map`, in which the txid for a transaction to be relayed is added to `TxRelay::m_tx_inventory_to_send` for all peers. Even for those peers that have not completed the version handshake. Prior to the PR this was not the case as `ForEachNode` has a "fully connected check" before calling a function for each node.

ACKs for top commit:
  MarcoFalke:
    ACK 8f2dac5409 🔝
  jnewbery:
    utACK 8f2dac5409

Tree-SHA512: e9eaccf7e00633ee0806fff1068b0e413a69a5e389d96c9659f68079915a6381ad5040c61f716cfcde77931d1b563b1049da97a232a95c6cd8355bd3d13404b9
2022-12-02 15:13:31 +00:00
Hennadii Stepanov
1984db1d50
refactor: Rename local variable to distinguish it from type alias
The `txiter` type alias is declared in the `txmempool.h`.
2022-12-02 11:25:35 +00:00
MarcoFalke
c6e7f224c1 util: Add StrFormatInternalBug and STR_INTERNAL_BUG 2022-12-01 12:22:47 -05:00
fanquake
e334f7a545
Merge bitcoin/bitcoin#26594: wallet: Avoid a segfault in migratewallet failure cleanup
5e65a216d1 wallet: Explicitly say migratewallet on encrypted wallets is unsupported (Andrew Chow)
88afc73ae0 tests: Test for migrating encrypted wallets (Andrew Chow)
86ef7b3c7b wallet: Avoid null pointer deref when cleaning up migratewallet (Andrew Chow)

Pull request description:

  When `migratewallet` fails, we do an automatic cleanup in order to reset everything so that the user does not experience any interruptions. However, this apparently has a segfault in it, caused by the the pointers to the watchonly and solvables wallets being nullptr. If those wallets are not created (either not needed, or failed early on), we will accidentally attempt to dereference these nullptrs, which causes a segfault.

  This failure can be easily reached by trying to migrate an encrypted wallet. Currently, we can't migrate encrypted wallets because of how we unload wallets before migrating, and therefore forget the encryption key if the wallet was unlocked. So any encrypted wallets will fail, entering the cleanup, and because watchonly and solvables wallets don't exist yet, the segfault is reached.

  This PR fixes this by not putting those nullptrs in a place that we will end up dereferencing them later. It also adds a test that uses the encrypted wallet issue.

ACKs for top commit:
  S3RK:
    reACK 5e65a216d1
  stickies-v:
    ACK [5e65a21](5e65a216d1)
  furszy:
    diff ACK 5e65a21

Tree-SHA512: f75643797220d4232ad3ab8cb4b46d0f3667f00486e910ca748c9b6d174d446968f1ec4dd7f907da1be9566088849da7edcd8cd8f12de671c3241b513deb8e80
2022-12-01 10:17:09 +00:00
Andrew Chow
e2bfd41f83
Merge bitcoin/bitcoin#25942: test: add ismine test for descriptor ScriptPubKeyMan
1b77db2653 test: add `ismine` test for descriptor scriptpubkeyman (w0xlt)

Pull request description:

  Currently `src/wallet/test/ismine_tests.cpp` has tests for the legacy ScriptPubKeyMan only.
  This PR adds tests for the descriptor ScriptPubKeyMan.

ACKs for top commit:
  ishaanam:
    ACK 1b77db2653
  achow101:
    ACK 1b77db2653
  furszy:
    ACK 1b77db26 with a non-blocking comment.

Tree-SHA512: 977b5d1e71f9468331aeb4ebaf3708dd651f9f3018d4544a395b87ca6d7fb8bfa6d20acc1a4f6e096e240e81d30fb7a6e8add190e52536e7a3cb5a80f392883f
2022-11-30 11:28:32 -05:00