Adds test coverage for the wallet's crypted key loading from db process.
The following scenarios are covered:
1) "All ckeys checksums valid" test:
Loads an encrypted wallet with all the crypted keys with a valid checksum and
verifies that 'CWallet::Unlock' doesn't force an entire crypted keys re-write.
(we force a complete ckeys re-write if we find any missing crypted key checksum
during the wallet loading process)
2) "Missing checksum in one ckey" test:
Verifies that loading up a wallet with, at least one, 'ckey' with no checksum
triggers a complete re-write of the crypted keys.
3) "Invalid ckey checksum error" test:
Verifies that loading up a ckey with an invalid checksum stops the wallet loading
process with a corruption error.
4) "Invalid ckey pubkey error" test:
Verifies that loading up a ckey with an invalid pubkey stops the wallet loading
process with a corruption error.
files share the same purpose, and we shouldn't have wallet code
inside the test directory.
This later is needed to use wallet util functions in the bench
and test binaries without be forced to duplicate them.
0eeb9b0442 [fuzz] Move ConsumeNetAddr to fuzz/util/net.h (dergoegge)
291c8697d4 [fuzz] Make ConsumeNetAddr produce valid onion addresses (dergoegge)
c9ba3f836e [netaddress] Make OnionToString public (dergoegge)
Pull request description:
The chance that the fuzzer is able to guess a valid onion address is probably slim, as they are Base32 encoded and include a checksum. Right now, any target using `ConsumeNetAddr` would have a hard time uncovering bugs that require valid onion addresses as input.
This PR makes `ConsumeNetAddr` produce valid onion addresses by using the 32 bytes given by the fuzzer as the pubkey for the onion address and forming a valid address according to the torv3 spec.
ACKs for top commit:
vasild:
ACK 0eeb9b0442
brunoerg:
ACK 0eeb9b0442
Tree-SHA512: 7c687a4d12f9659559be8f0c3cd4265167d1261d419cfd3d503fd7c7f207cc0db745220f02fb1737e4a5700ea7429311cfc0b42e6c15968ce6a85f8813c7e1d8
cc597bd56d src/bitcoin-cli.cpp: -getinfo help - grammar correction (@RandyMcMillan)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: a5321968d0d377e1481170b4220a1319bf9040ec198b27c011609a5b7a81e9193500b750980c7de423b8b99655ed0f7772a9621e0b230aa6cc5d7b48167ed4f9
c8dc0e3eaa refactor: Inline `CTxMemPoolEntry` class's functions (Hennadii Stepanov)
75bbe594e5 refactor: Move `CTxMemPoolEntry` class to its own module (Hennadii Stepanov)
Pull request description:
This PR:
- gets rid of the `policy/fees` -> `txmempool` -> `policy/fees` circular dependency
- is an alternative to #13949, which nukes only one circular dependency
ACKs for top commit:
ryanofsky:
Code review ACK c8dc0e3eaa. Just include and whitespace changes since last review, and there's a moveonly commit now so it's very easy to review
theStack:
Code-review ACK c8dc0e3eaa
glozow:
utACK c8dc0e3eaa, agree these changes are an improvement.
Tree-SHA512: 36ece824e6ed3ab1a1e198b30a906c8ac12de24545f840eb046958a17315ac9260c7de26e11e2fbab7208adc3d74918db7a7e389444130f8810548ca2e81af41
6630a1e844 Add warning on first startup if free disk space is less than necessary (Ben Woosley)
Pull request description:
This reworks/revives https://github.com/bitcoin/bitcoin/pull/15848 to add a check for low disk space on first startup and issue a warning if disk space is below the expected space required to accommodate the blocks.
This PR was fashioned by a team of developers at the [bitcoin++](https://www.btcplusplus.dev/) conference workshop: "[Let's contribute to Bitcoin Core](https://sched.co/12P6Z)"
Fixes#15813
ACKs for top commit:
achow101:
ACK 6630a1e844
willcl-ark:
tACK 6630a1e844 rebased on master. Warning shows on first start but not on restart after some blocks have been downloaded.
aureleoules:
ACK 6630a1e844
pablomartin4btc:
re-ACK 6630a1e844
hernanmarino:
ReACK 6630a1e844
Tree-SHA512: 0f18acabdf2b514e96e2eea8f304960b952226b83dc91334cf7d1f6355ea2f257aaec0ee38d43ac36435385ecd918333d20657c35a8a7407e7cf2680ccb643bb
At wallet load time, we set the crypted key "checksum_valid" variable always to false.
Which, on every wallet decryption call, forces the process to re-write the entire ckeys to db when
it's not needed.
ac410e6fc0 log: improve some validation log messages to include hashPrevBlock (Skuli Dulfari)
Pull request description:
When there is an issue with a previous block the current log messages do not indicate hashPrevBlock. Adding it makes debugging easier.
ACKs for top commit:
stickies-v:
ACK ac410e6fc0
aureleoules:
reACK ac410e6fc0
theStack:
ACK ac410e6fc0
Tree-SHA512: d91481321f4474bb4fdf6ad55d1c897437b631b0a12308815c4ac5b053c8a76726e2d93f2aa0701e8cfd48fba7fad19ef5ffca3c67d3aa973dc593df806f1757
8a5014cd8a Fixes bitcoin#26490 by preventing notifications (John Moffett)
Pull request description:
This is a PR to address https://github.com/bitcoin/bitcoin/issues/26490
The menu bar currently subscribes to window focus change notifications to enable or disable certain menu options in response to the window status.
Notifications are automatically unsubscribed (disconnected in Qt parlance) if the sender is deleted -- in this case, the sender is the QTApplication object (`qApp`). However, MacOS 13 sends a window focus change notification *after* the main window has been destroyed but *before* `qApp` has been fully destroyed.
Since the menu bar is deleted in the main window's destructor, it no longer exists when it receives these notifications (in two different places via lambda expressions). The solution is to pass the main window (`this`) as context when subscribing to the notifications. In this [overloaded version](https://doc.qt.io/qt-5/qobject.html#connect-1) of `connect`, Qt automatically unsubscribes to notifications if the sender OR context (here the main window object) is destroyed. Since the spurious notifications are sent after the main window object is destroyed, this change prevents them from being sent.
Tested on Mac OS 13 and 12 only.
ACKs for top commit:
hebasto:
ACK 8a5014cd8a
Tree-SHA512: 3dff0a252fe0e93dd68cf5503135ecf6a72bcf385ba38407d6021ab77cca323f8bbe58aeca90ec124aa2a22ab9d35b706946179ac3b5d171c96a7010de51a090
2222ec71fd util: Move error message formatting of NonFatalCheckError to cpp (MacroFake)
Pull request description:
This allows to strip down the header file.
ACKs for top commit:
hebasto:
re-ACK 2222ec71fd, only rebased and suggested changes since my recent [review](https://github.com/bitcoin/bitcoin/pull/25112#pullrequestreview-1182361605).
aureleoules:
ACK 2222ec71fd
Tree-SHA512: 313b3c891bb000cf606df1793b068f93df99915a254fbd67a45f003d440cce7355cdcc6b196f35757cc02d3697970d30e9de0d675f2aa8eb74107c13d663927a
fa84df1f03 scripted-diff: wallet: rename AvailableCoinsParams members to snake_case (furszy)
61c2265629 wallet: group AvailableCoins filtering parameters in a single struct (furszy)
f0f6a3577b RPC: listunspent, add "include immature coinbase" flag (furszy)
Pull request description:
Simple PR; adds a "include_immature_coinbase" flag to `listunspent` to include the immature coinbase UTXOs on the response. Requested by #25728.
ACKs for top commit:
danielabrozzoni:
reACK fa84df1f03
achow101:
ACK fa84df1f03
aureleoules:
reACK fa84df1f03
kouloumos:
reACK fa84df1f03
theStack:
Code-review ACK fa84df1f03
Tree-SHA512: 0f3544cb8cfd0378a5c74594480f78e9e919c6cfb73a83e0f3112f8a0132a9147cf846f999eab522cea9ef5bd3ffd60690ea2ca367dde457b0554d7f38aec792
db929893ef Faster -reindex by initially deserializing only headers (Larry Ruane)
c72de9990a util: add CBufferedFile::SkipTo() to move ahead in the stream (Larry Ruane)
48a68908ba Add LoadExternalBlockFile() benchmark (Larry Ruane)
Pull request description:
### Background
During the first part of reindexing, `LoadExternalBlockFile()` sequentially reads raw blocks from the `blocks/blk00nnn.dat` files (rather than receiving them from peers, as with initial block download) and eventually adds all of them to the block index. When an individual block is initially read, it can't be immediately added unless all its ancestors have been added, which is rare (only about 8% of the time), because the blocks are not sorted by height. When the block can't be immediately added to the block index, its disk location is saved in a map so it can be added later. When its parent is later added to the block index, `LoadExternalBlockFile()` reads and deserializes the block from disk a second time and adds it to the block index. Most blocks (92%) get deserialized twice.
### This PR
During the initial read, it's rarely useful to deserialize the entire block; only the header is needed to determine if the block can be added to the block index immediately. This change to `LoadExternalBlockFile()` initially deserializes only a block's header, then deserializes the entire block only if it can be added immediately. This reduces reindex time on mainnet by 7 hours on a Raspberry Pi, which translates to around a 25% reduction in the first part of reindexing (adding blocks to the index), and about a 6% reduction in overall reindex time.
Summary: The performance gain is the result of deserializing each block only once, except its header which is deserialized twice, but the header is only 80 bytes.
ACKs for top commit:
andrewtoth:
ACK db929893ef
achow101:
ACK db929893ef
aureleoules:
ACK db929893ef - minor changes and new benchmark since last review
theStack:
re-ACK db929893ef
stickies-v:
re-ACK db929893e
Tree-SHA512: 5a5377192c11edb5b662e18f511c9beb8f250bc88aeadf2f404c92c3232a7617bade50477ebf16c0602b9bd3b68306d3ee7615de58acfd8cae664d28bb7b0136
MacOS 13 sends a window focus change notification after the main
window has been destroyed but before the QTApplication has been
destroyed. This results in the menu bar receiving a notification
despite it no longer existing. The solution is to pass the main
window as context when subscribing to the notifications. Qt
automatically unsubscribes to notifications if the sender OR
context is destroyed.
Since faf44876db, the maxtipage comparison
in IsInitialBlockDownload() has been broken, since the NodeClock::now()
time_point is in the system's native denomination (micrcoseconds).
Without this patch, specifying the maximum allowable -maxtipage
(9223372036854775807) results in a SIGABRT crash.
Co-authored-by: MacroFake <falke.marco@gmail.com>
2dede9f675 Adjust RPCTypeCheckObj error string (Leonardo Araujo)
Pull request description:
Unifies the JSON type error strings as mentioned in #26214. Also refer to #25737.
ACKs for top commit:
furszy:
ACK 2dede9f6
Tree-SHA512: c918889e347ba32cb6d0e33c0de5956c2077dd40c996151e16741b0c4983ff098c60258206ded76ad7bbec4876c780c6abb494a97e4f1e05717d28a59b9167a6
fa09525751 univalue: string_view test (MacroFake)
1111c7e3f1 univalue: Avoid std::string copies (MacroFake)
Pull request description:
This shouldn't matter too much, unless a really large string is pushed into a json struct, but I think it also clarifies the code.
ACKs for top commit:
martinus:
Code review ACK fa09525751
aureleoules:
reACK fa09525751
ryanofsky:
Code review ACK fa09525751
Tree-SHA512: 74c441912bd0b00cdb9ea7890121f71ae5d62a7594e7d29aa402c9e3f033710c5d3afb27a37c552e6513804b249aa37e375ce013a3db853a25d1fd7b6e6cd3a8
In the wallet key-value-loading routine, most legacy type entries
require a LegacyScriptPubKeyMan instance after successful
deserialization. On a descriptor wallet, creating that (via method
`GetOrCreateLegacyScriptPubKeyMan`) fails and then leads to a
null-pointer dereference crash. Fix this by throwing an error if
if the wallet flags indicate that we have a descriptor wallet and there
is a legacy entry found.
It's nice to be able to use named options and positional arguments together.
Most shell tools accept both, and python functions combine options and
arguments allowing them to be passed with even more flexibility. This change
adds support for python's approach so as a motivating example:
bitcoin-cli -named createwallet wallet_name=mywallet load_on_startup=1
Can be shortened to:
bitcoin-cli -named createwallet mywallet load_on_startup=1
JSON-RPC standard doesn't have a convention for passing named and positional
parameters together, so this implementation makes one up and interprets any
unused "args" named parameter as a positional parameter array.
25ef049d60 log: mempool: log removal reason in validation interface (James O'Beirne)
Pull request description:
Currently the exact reason a transaction is removed from the mempool isn't logged. It is sometimes detectable from context, but adding the `reason` to the validation interface logs (where it is already passed) seems like an easy way to disambiguate.
For example in the case of mempool expiry, the logs look like this:
```
[validationinterface.cpp:220] [TransactionRemovedFromMempool] [validation] Enqueuing TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
[txmempool.cpp:1050] [RemoveUnbroadcastTx] [mempool] Removed <txid> from set of unbroadcast txns before confirmation that txn was sent out
[validationinterface.cpp:220] [operator()] [validation] TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
[validation.cpp:267] [LimitMempoolSize] [mempool] Expired 1 transactions from the memory pool
```
There is no context-free way to know $txid was evicted on the basis of expiry. This change will make that case (and probably others) clear.
ACKs for top commit:
0xB10C:
ACK 25ef049d60
Tree-SHA512: 9890f9fa16f66c8a9296798d8c28993e1b81da17cf592946f2abc22041f0b30b0911ab86a0c48d4aa46b9a8b3f7f5de67778649ac48c97740b0a09aa6816e0af
c3b1fe59db rpc: doc: add missing option "bech32m" for `change_type` parameters (Sebastian Falbesoner)
Pull request description:
Affects the help of the `fundrawtransaction`, `send` and `walletcreatefundedpsbt` RPCs.
This was found by manually inspecting the results of `$ git grep p2sh-segwit.*bech32`.
ACKs for top commit:
achow101:
ACK c3b1fe59db
Tree-SHA512: a3f1f8fde5905c80e1b95bd042ca0bc73d08c1c0e79c52ab0d6d12d7afdd4aa288afb41e12279fcea328a396f3d0a5564018170c0a11c5aa26dc6d44d2a62b1c
0de30ed509 tests: Test Taproot PSBT signing with keys in other descriptor (Andrew Chow)
6efcdf6b7f tests: Use new wallets for each test in wallet_taproot.py (Andrew Chow)
8781a1b6bb psbt: Include output pubkey in additional pubkeys to sign (Andrew Chow)
323890d0d7 sign: Fill in taproot pubkey info for all script path sigs (Andrew Chow)
Pull request description:
A user reported on [stackexchange](https://bitcoin.stackexchange.com/q/115742/48884) that they were unable to sign for a `multi_a` script using a wallet that only had the corresponding keys (i.e. it did not have the `multi_a()` descriptor). This PR fixes this issue.
Additionally, `wallet_taproot.py` is modified to test for this scenario by having another wallet in `do_test_psbt` which contains descriptors that only have the keys involved in the descriptor being tested. `wallet_taproot.py` was also modified to create new wallets for each test case rather than sharing wallets throughout as the sharing could result in the signing wallet having the keys in a different descriptor and accidentally result in failing to detect a test failure.
The changes to the test also revealed a similar issue with `rawtr()` descriptors, which has also been fixed by checking if a descriptor can produce a `SigningProvider` for the Taproot output pubkey.
ACKs for top commit:
instagibbs:
crACK 0de30ed509
darosior:
ACK 0de30ed509
Tree-SHA512: 12e131dd8afd93da7b1288c9054de2415a228d4477b97102da3ee4e82ce9de20b186260c3085a4b7b067bd8b74400751dcadf153f113db83abc59e7466e69f14
Currently the exact reason a transaction is removed from the mempool isn't
logged. It is sometimes detectable from context, but adding the `reason` to
the validation interface logs (where it is already passed) seems like an easy
way to disambiguate.
For example, in the case of mempool expiry, the logs look like this:
```
[validationinterface.cpp:220] [TransactionRemovedFromMempool] [validation] Enqueuing TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
[txmempool.cpp:1050] [RemoveUnbroadcastTx] [mempool] Removed <txid> from set of unbroadcast txns before confirmation that txn was sent out
[validationinterface.cpp:220] [operator()] [validation] TransactionRemovedFromMempool: txid=<txid> wtxid=<wtxid>
[validation.cpp:267] [LimitMempoolSize] [mempool] Expired 1 transactions from the memory pool
```
There is no context-free way to know $txid was evicted on the basis of expiry.
This change will make that case (and probably others) clear.
fa3ea81c3e refactor: Add LIFETIMEBOUND / -Wdangling-gsl to Assert() (MacroFake)
Pull request description:
Currently compiles clean, but I think it may still be useful.
Can be tested by adding an `&`:
```diff
diff --git a/src/test/util_tests.cpp b/src/test/util_tests.cpp
index 5766fff92d..300c1ec60f 100644
--- a/src/test/util_tests.cpp
+++ b/src/test/util_tests.cpp
@@ -125,7 +125,7 @@ BOOST_AUTO_TEST_CASE(util_check)
// Check -Wdangling-gsl does not trigger when copying the int. (It would
// trigger on "const int&")
- const int nine{*Assert(std::optional<int>{9})};
+ const int& nine{*Assert(std::optional<int>{9})};
BOOST_CHECK_EQUAL(9, nine);
}
```
Output:
```
test/util_tests.cpp:128:29: warning: object backing the pointer will be destroyed at the end of the full-expression [-Wdangling-gsl]
const int& nine{*Assert(std::optional<int>{9})};
^~~~~~~~~~~~~~~~~~~~~
./util/check.h:75:50: note: expanded from macro 'Assert'
#define Assert(val) inline_assertion_check<true>(val, __FILE__, __LINE__, __func__, #val)
^~~
1 warning generated.
ACKs for top commit:
jonatack:
ACK fa3ea81c3e
theuni:
ACK fa3ea81c3e
Tree-SHA512: 17dea4d75f2ee2bf6e1b6a6f6d8f439711c777df0390574e8d8edb6ac9ee807a135341e4439050bd6a15ecc4097a1ba9a7ab15d27541ebf70a4e081fa6871877
fa24239a1c net: Avoid SetTxRelay for feeler connections (MacroFake)
Pull request description:
Seems odd to reserve memory for the struct (the heaviest member being `m_tx_inventory_known_filter`) when it is never used.
This also avoids sending out `msg_sendtxrcncl` before disconnecting. This shouldn't matter, as other messages, such as `msg_wtxidrelay`, `msg_sendaddrv2`, `msg_verack` or `msg_getaddr` are still sent. Though, it allows to test the changes here as a side-effect.
ACKs for top commit:
naumenkogs:
ACK fa24239a1c
vasild:
ACK fa24239a1c
jonatack:
ACK fa24239a1c
mzumsande:
ACK fa24239a1c
Tree-SHA512: d7604c7eb4df8f2de811e600bdd312440ee03e508d3a0f09ae79f7f2d3eeec663bfd47a2d079fa50b756d61e35dfa998de068a7b9afaf35378fa0e62a538263d
e049fd76f0 Bugfix: Check for readlink buffer overflow and handle gracefully (Luke Dashjr)
Pull request description:
If readlink returns the size of the buffer, an overflow may have (safely) occurred.
Pass a buffer size of MAX_PATH+1 (the size of the actual buffer) to detect this scenario.
ACKs for top commit:
hebasto:
ACK e049fd76f0.
Tree-SHA512: 188bace79cbe556efe7782e46b870c02729b07b104a9316b0f7d50013504972e85baf507403d2d6060bb2bf3e13f40d735bddd18255d97a60810208c3de87691
