The rpc command verifychain now fails if the dbcache was not sufficient
to complete the verification at the specified level and depth.
In the same situation, the VerifyDB check during Init will now fail (and lead to
an early shutdown) if the user has explicitly specified -checkblocks or
-checklevel but the check couldn't be executed because of the limited
cache. If the user didn't change any of the two and is using the defaults, log a warning
but don't prevent the node from starting up.
This does not change behavior. It is in preparation for
special handling of the case where VerifyDB doesn't finish
for various reasons, but doesn't fail.
4c8ecccdcd test: add tests for `outputs` argument to `bumpfee`/`psbtbumpfee` (Seibart Nedor)
c0ebb98382 wallet: add `outputs` arguments to `bumpfee` and `psbtbumpfee` (Seibart Nedor)
a804f3cfc0 wallet: extract and reuse RPC argument format definition for outputs (Seibart Nedor)
Pull request description:
This implements a modification of the proposal in #22007: instead of **adding** outputs to the set of outputs in the original transaction, the outputs given by `outputs` argument **completely replace** the outputs in the original transaction.
As noted below, this makes it easier to "cancel" a transaction or to reduce the amounts in the outputs, which is not the case with the original proposal in #22007, but it seems from the discussion in this PR that the **replace** behavior is more desirable than **add** one.
ACKs for top commit:
achow101:
ACK 4c8ecccdcd
1440000bytes:
Code Review ACK 4c8ecccdcd
ishaanam:
reACK 4c8ecccdcd
Tree-SHA512: 31361f4a9b79c162bda7929583b0a3fd200e09f4c1a5378b12007576d6b14e02e9e4f0bab8aa209f08f75ac25a1f4805ad16ebff4a0334b07ad2378cc0090103
Since migration reloads the wallet, the wallet will always be locked
unless the passphrase is given. migratewallet can now take the
passphrase in order to unlock the wallet for migration.
An overload of MigrateLegacyToDescriptor is added which takes the wallet
name. The original that took a wallet pointer is still available, it
just gets the name, closes the wallet, and calls the new overload.
Instead of having gpg.sh check against the trusted keys for a valid
signature, do it inside of verify-commits itself.
This also allows us to use the same trusted-keys throughout the
verify-commits.py check rather than it possibly being modified during
the clean merge check.
Using a FillableSigningProvider results in decodescript being unable to
infer descriptors for scripts larger than 520 bytes. Using a
FlatSigningProvider resolves this.
c572eae989 update the freebsd build doc to reflect recent changes to DB4 install process (Murray Nesbitt)
Pull request description:
This PR introduces documentation changes needed to keep up with #26834.
ACKs for top commit:
fanquake:
ACK c572eae989 - have not tested, but looks ok.
Tree-SHA512: 42a79e7b45834916b1b738db524b51b9ff4fde8348ba66fc331ff6603532dd9fce73ea392eef97d31112326c6d60ec2c5c7c29e66aab33aaf846aab8aea1d1aa
7a83aa0982 test: add coverage for unparsable `-maxuploadtarget` (brunoerg)
Pull request description:
This PR adds test coverage for the following error:
7386da7a0b/src/init.cpp (L1096-L1099)
Top commit has no ACKs.
Tree-SHA512: c115b2b4d2d0eb2316bf9fafd7e0046aa18c9650062779b3a82d6145d188765bff5317f4ca5f79607732fde6d83e1f67756ac20a12c98d060ee68d8acc20c76e
304ae6dc8e doc: remove mention of "proper signing key" (fanquake)
Pull request description:
This key is no-longer in use: https://lists.linuxfoundation.org/pipermail/bitcoin-core-dev/2023-February/000115.html
> Please remove it from verification pipelines.
ACKs for top commit:
hebasto:
ACK 304ae6dc8e
Tree-SHA512: 3dfd221a48f69ac56b4568db06b5d5b5d6a60b7d027a26157912219a2073589a0a3934cb30e11a161d48db55d3a637338f96617e3f3b92cb9e60e0d1d1dd372a
741908afc1 test: previous releases: add v24.0.1 (Sebastian Falbesoner)
Pull request description:
The same procedure as every release (see dba1231672 [v23.0] and d8b705f1ca [v22.0]), only a little simpler now: thanks to #25650, the previous release fetch script defaults to downloading/building the necessary tags, i.e. we don't need to extend the tag list in the CI scripts and test/README.md anymore.
ACKs for top commit:
Sjors:
tACK 741908afc1
Tree-SHA512: a5426e989bd0bba42aa13e7d4cf60f792bf36bd9a6cdb6ef5799f7574d9a8a20979244627bbd0c6219630367e7fd73bac9e677814bc50233f64592ad035e713e
5669afb80e fs: drop old WSL1 hack. (sinetek)
Pull request description:
Following discussion, the WSL1 patch will be removed, as WSL1 is no longer being developed by Microsoft. Instead, please upgrade to a mainstream WSL2 version. More information can be found on [the official website](https://docs.microsoft.com/en-us/windows/wsl/).
ACKs for top commit:
1440000bytes:
ACK 5669afb80e
fanquake:
ACK 5669afb80e - seems ok as-is.
Tree-SHA512: 256c13985f6dd3453caf39c7ef1c951dbdfa8457a18cd05e4624db36d8ed8a4f809bb78a7b3c82c72997e9ed3823d5566a5c2d0812d2501aba2e54bc5e6eec79
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction (Antoine Poinsot)
840a396029 qa: add a "smart" Miniscript fuzz target (Antoine Poinsot)
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding (Antoine Poinsot)
611e12502a qa: functional test Miniscript signing with key and timelocks (Antoine Poinsot)
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable (Antoine Poinsot)
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins (Antoine Poinsot)
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges (Antoine Poinsot)
a2f81b6a8f script/sign: signing support for Miniscript with timelocks (Antoine Poinsot)
61c6d1a844 script/sign: basic signing support for Miniscript descriptors (Antoine Poinsot)
4242c1c521 Align 'e' property of or_d and andor with website spec (Pieter Wuille)
f5deb41780 Various additional explanations of the satisfaction logic from Pieter (Pieter Wuille)
22c5b00345 miniscript: satisfaction support (Antoine Poinsot)
Pull request description:
This makes the Miniscript descriptors solvable.
Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.
ACKs for top commit:
achow101:
ACK 6c7a17a8e0
sipa:
utACK 6c7a17a8e0 (to the extent that it's not my own code).
Tree-SHA512: a71ec002aaf66bd429012caa338fc58384067bcd2f453a46e21d381ed1bacc8e57afb9db57c0fb4bf40de43b30808815e9ebc0ae1fbd9e61df0e7b91a17771cc
906631450d s/transcation/transaction/ (Greg Sanders)
Pull request description:
ACKs for top commit:
fanquake:
ACK 906631450d - looks like other comments are being addressed elsewhere.
Tree-SHA512: c835a14db2e0cf5e0317c95c8c7441df1f7c6cb14be7809fd947e07ea9d23f1f171f111429aabd0509b7f17601bc742041316b18e1135e547a966961f2c65038
9fa43b5af6 refactor: Disable unused special members functions in `UnlockContext` (Hennadii Stepanov)
Pull request description:
Also `UnlockContext::valid` and `UnlockContext::relock` are `const` now.
ACKs for top commit:
achow101:
ACK 9fa43b5af6
john-moffett:
ACK 9fa43b5af6
furszy:
ACK 9fa43b5a
Tree-SHA512: 6d9fa8208676b9bd5d85b73cb2d3136e7f28ef59e68ee34915ec598458868e302a80b9ef1384c0bf7a4c42f936830c3add9662ca0bae73860a55a25cc374b699
that was resolved in PR27036 "test: Remove last uses of snprintf and simplify"
and while here, fix up 2 words in docs to make the spelling linter green again.
Since we no longer store a ref to the global `ArgsManager`
inside the wallet, we can move the util/system.h
include to the cpp.
This dependency removal opened a can of worms, as few
other places were, invalidly, depending on the wallet's
header including it.
691eaf8873 Pass MSG_MORE flag when sending non-final network messages (Matt Whitlock)
Pull request description:
**N.B.:** This is my second attempt at introducing this optimization. #12519 (2018) was closed in deference to switching to doing gathering socket writes using `sendmsg(2)`, which I agree would have superior performance due to fewer syscalls, but that work was apparently abandoned in late 2018. Ever since, Bitcoin Core has continued writing tons of runt packets to the wire. Can we proceed with my halfway solution for now?
----
Since Nagle's algorithm is disabled, each and every call to `send(2)` can potentially generate a separate TCP segment on the wire. This is especially inefficient when sending the tiny header preceding each message payload.
Linux implements a `MSG_MORE` flag that tells the kernel not to push the passed data immediately to the connected peer but rather to collect it in the socket's internal transmit buffer where it can be combined with data from successive calls to `send(2)`. Where available, specify this flag when calling `send(2)` in `CConnman::SocketSendData(CNode &)` if the data buffer being sent is not the last one in `node.vSendMsg`.
ACKs for top commit:
sipa:
ACK 691eaf8873
vasild:
ACK 691eaf8873
Tree-SHA512: 9a7f46bc12edbf78d488f05d1c46760110a24c95af74b627d2604fcd198fa3f511c5956bac36d0034e88c632d432f7d394147e667a11b027af0a30f70a546d70
dee8549be3 test: simplify and speedup mempool_updatefromblock.py by using MiniWallet (Sebastian Falbesoner)
Pull request description:
This PR simplifies the functional test mempool_updatefromblock.py by using MiniWallet in order to avoid manual low-level tx creation (signing, outputs selection, fee calculation). Most of the tedious work is done by the method `MiniWallet.send_self_transfer_multi` (calling `create_self_transfer_multi` internally) which supports spending a given set of UTXOs and creating a certain number of outputs.
As a nice side-effect, the test's performance increases significantly (~3.5x on my system):
```
master
1m56.80s real 1m50.10s user 0m06.36s system
PR
0m32.34s real 0m30.26s user 0m01.41s system
```
The arguments `start_input_txid` and `end_address` have been removed from the `transaction_graph_test` method, as they are currently unused and I don't see them being needed for future tests.
ACKs for top commit:
brunoerg:
crACK dee8549be3
MarcoFalke:
lgtm ACK dee8549be3🚏
Tree-SHA512: 9f6da634bdc8c272f9a2af1cddaa364ee371d4e95554463a066249eecebb668d8c6cb123ec8a5404c41b3291010c0c8806a8a01dd227733cec03e73aa93b0103
511aa4f1c7 Add unit test for ChaCha20's new caching (Pieter Wuille)
fb243d25f7 Improve test vectors for ChaCha20 (Pieter Wuille)
93aee8bbda Inline ChaCha20 32-byte specific constants (Pieter Wuille)
62ec713961 Only support 32-byte keys in ChaCha20{,Aligned} (Pieter Wuille)
f21994a02e Use ChaCha20Aligned in MuHash3072 code (Pieter Wuille)
5d16f75763 Use ChaCha20 caching in FastRandomContext (Pieter Wuille)
38eaece67b Add fuzz test for testing that ChaCha20 works as a stream (Pieter Wuille)
5f05b27841 Add xoroshiro128++ PRNG (Martin Leitner-Ankerl)
12ff72476a Make unrestricted ChaCha20 cipher not waste keystream bytes (Pieter Wuille)
6babf40213 Rename ChaCha20::Seek -> Seek64 to clarify multiple of 64 (Pieter Wuille)
e37bcaa0a6 Split ChaCha20 into aligned/unaligned variants (Pieter Wuille)
Pull request description:
This is an alternative to #25354 (by my benchmarking, somewhat faster), subsumes #25712, and adds additional test vectors.
It separates the multiple-of-64-bytes-only "core" logic (which becomes simpler) from a layer around which performs caching/slicing to support arbitrary byte amounts. Both have their uses (in particular, the MuHash3072 code can benefit from multiple-of-64-bytes assumptions), plus the separation results in more readable code. Also, since FastRandomContext effectively had its own (more naive) caching on top of ChaCha20, that can be dropped in favor of ChaCha20's new built-in caching.
I thought about rebasing #25712 on top of this, but the changes before are fairly extensive, so redid it instead.
ACKs for top commit:
ajtowns:
ut reACK 511aa4f1c7
dhruv:
tACK crACK 511aa4f1c7
Tree-SHA512: 3aa80971322a93e780c75a8d35bd39da3a9ea570fbae4491eaf0c45242f5f670a24a592c50ad870d5fd09b9f88ec06e274e8aa3cefd9561d623c63f7198cf2c7
6ada37d44c verify-commits: Bump trusted git root to after most recent laanwj merge (Andrew Chow)
Pull request description:
To prepare for the removal of laanwj's key from trusted key (#27054), the trusted git root needs to be newer than the most recent merge commit signed by his key.
This can be tested by removing the laanwj's key from trusted keys (e.g. by merging with #27054) and running `verify-commits.py` with `--clean-merge 0`: `./contrib/verify-commits/verify-commits.py --clean-merge 0 HEAD~`. (`--clean-merge 0` disables the clean merge check which will checkout some commits, which results in the `trusted-keys` used in checking of subsequent commits to be different than expected).
ACKs for top commit:
fanquake:
ACK 6ada37d44c
hebasto:
ACK 6ada37d44c, I've verified the history of laanwj's merge commits.
Tree-SHA512: 55cafeddd54aa2b62d7b7cd41c542f4fd974b322a0405de546600d88658575714ebc893b087eb31f28c205559a7b213f88d9038de431271fca00be866610df74
9d3127b11e Add settings.json prune-prev, proxy-prev, onion-prev settings (Ryan Ofsky)
Pull request description:
With #602, if proxy and pruning settings are disabled in the GUI and the GUI is restarted, proxy and prune values are not stored anywhere. So if these settings are enabled in the future, default values will be shown, not previous values.
This PR stores previous values so they will preserved across restarts. I'm not sure I like this behavior because showing default values seems simpler and safer to me. Previous values may just have been set temporarily and may have never actually worked, and it adds some code complexity to store them.
This PR is one way of resolving #596. Other solutions are possible and could be implemented as alternatives.
ACKs for top commit:
hebasto:
ACK 9d3127b11e, tested on Ubuntu 22.04.
vasild:
ACK 9d3127b11e
jarolrod:
tACK 9d3127b11e
Tree-SHA512: 1778d1819443490c880cfd5c1711d9c5ac75ea3ee8440e2f0ced81d293247163a78ae8aba6027215110aec6533bd7dc6472aeead6796bfbd51bf2354e28f24a9
77192c9598 cli: include local ("unreachable") peers in -netinfo table (Matthew Zipkin)
Pull request description:
Closes https://github.com/bitcoin/bitcoin/issues/26579
The `-netinfo` dashboard did not list peers that were connected via "unroutable" networks. This included local peers including local-network peers. Personally, I run one bitcoind instance on my network that is used by other services like Wasabi Wallet and LND running on other machines.
This PR adds an "npr" (not publicly routable) column to the table of networks (ipv4, ipv6, onion, etc) so that every connection to the node is listed, and the totals are accurate as they relate to max inbound and max outbound limits.
Example connecting in regtest mode to one local and one remote peer:
```
Bitcoin Core client v24.99.0-151ce099ea8f-dirty regtest - server 70016/Satoshi:24.99.0/
<-> type net mping ping send recv txn blk hb addrp addrl age id address version
in npr 0 0 90 90 1 1 127.0.0.1:59180 70016/Satoshi:24.99.0/
out manual ipv4 63 63 84 84 3 3 0 143.244.175.41 70016/Satoshi:24.0.1/
ms ms sec sec min min min
ipv4 ipv6 npr total block manual
in 0 0 1 1
out 1 0 0 1 0 1
total 1 0 1 2
Local addresses: n/a
```
ACKs for top commit:
jonatack:
Re-tested ACK 77192c9598
Tree-SHA512: 78aa68bcff0dbaadb5f0604bf023fe8fd921313bd8276d12581f7655c089466a48765f9e123cb31d7f1d294d5ca45fdefdf8aa220466ff738f32414f41099c06
`m_relock_mutex` is introduced so that the passphrase is not
deleted from memory when the timeout provided in
`walletpassphrase` is up, but the wallet is still rescanning.
Wallet passphrases are needed to top up the keypool during a
rescan. The following RPCs need the passphrase when rescanning:
- `importdescriptors`
- `rescanblockchain`
The following RPCs use the information about whether or not the
passphrase is being used to ensure that full rescans are able to
take place:
- `walletlock`
- `encryptwallet`
- `walletpassphrasechange`
772671245d test: p2p: check that headers message with invalid proof-of-work disconnects peer (Sebastian Falbesoner)
Pull request description:
One of the earliest anti-DoS checks done after receiving and deserializing a `headers` message from a peer is verifying whether the proof-of-work is valid (called in method `PeerManagerImpl::ProcessHeadersMessage`):
f227e153e8/src/net_processing.cpp (L2752-L2762)
The called method `PeerManagerImpl::CheckHeadersPoW` calls `Misbehaving` with a score of 100, i.e. leading to an immediate disconnect of the peer:
f227e153e8/src/net_processing.cpp (L2368-L2372)
This PR adds a simple test for both the misbehaving log and the resulting disconnect. For creating a block header with invalid proof-of-work, we first create one that is accepted by the node (the difficulty field `nBits` is copied from the genesis block) and based on that the nonce is modified until we have block header hash prefix that is too high to fulfill even the minimum difficulty.
ACKs for top commit:
Sjors:
ACK 772671245d
achow101:
ACK 772671245d
brunoerg:
crACK 772671245d
furszy:
Code review ACK 77267124 with a non-blocking speedup.
Tree-SHA512: 680aa7939158d1dc672b90aa6554ba2b3a92584b6d3bcb0227776035858429feb8bc66eed18b47de0fe56df7d9b3ddaee231aaeaa360136603b9ad4b19e6ac11
588fad868d descriptors: fix docstring (param [in] vs [out]) (SomberNight)
Pull request description:
As in title, these docstrings look incorrect.
ACKs for top commit:
john-moffett:
ACK 588fad868d
Tree-SHA512: 1ab343a1b1fc57a7d6bd8363b84db9d96e8ea11a4cec85bcf79885c9df53da889fe2fb10b1fa92d824ddf0dee800c07353f46f1fea9887d2ad518bed0afebe3d
e4e17907b6 Modernize rpcauth.py and its tests (Pieter Wuille)
Pull request description:
Use Python3 constructions, and f-strings.
ACKs for top commit:
jamesob:
Github ACK e4e17907b6
Tree-SHA512: 005573d967e04400fec727f45739f138879be703e692745c0a639272d37d221d230f388de23f2615cb954bb47179fb46e53da0410ae9f0865319b91bb2dc01f4
3a11adc700 Zero out wallet master key upon lock (John Moffett)
Pull request description:
When an encrypted wallet is locked (for instance via the RPC `walletlock`), the documentation indicates that the key is removed from memory:
b92d609fb2/src/wallet/rpc/encrypt.cpp (L157-L158)
However, the vector (a `std::vector<unsigned char, secure_allocator<unsigned char>>`) is merely _cleared_. As it is a member variable, it also stays in scope as long as the wallet is loaded, preventing the secure allocator from deallocating. This allows the key to persist indefinitely in memory. I confirmed this behavior on my macOS machine by using an open-source third party memory inspector ("Bit Slicer"). I was able to find my wallet's master key in Bit Slicer after unlocking and re-locking my encrypted wallet. I then confirmed the key data was at the address in LLDB.
This PR manually fills the bytes with zeroes before calling `clear()` by using our `memory_cleanse` function, which is designed to prevent the compiler from optimizing it away. I confirmed that it does remove the data from memory on my machine upon locking.
Note: An alternative approach could be to call `vMasterKey.shrink_to_fit()` after the `clear()`, which would trigger the secure allocator's deallocation. However, `shrink_to_fit()` is not _guaranteed_ to actually change the vector's capacity, so I think it's unwise to rely on it.
## Edit: A little more clarity on why this is an improvement.
Since `mlock`ed memory is guaranteed not to be swapped to disk and our threat model doesn't consider a super-user monitoring the memory in realtime, why is this an improvement? Most importantly, consider hibernation. Even `mlock`ed memory may get written to disk. From the `mlock` [manpage](https://man7.org/linux/man-pages/man2/mlock.2.html):
> (But be aware that the suspend mode on laptops and some desktop computers will save a copy of the system's RAM to disk, regardless of memory locks.)
As far as I can tell, this is true of [Windows](https://web.archive.org/web/20190127110059/https://blogs.msdn.microsoft.com/oldnewthing/20140207-00/?p=1833#:~:text=%5BThere%20does%20not%20appear%20to%20be%20any%20guarantee%20that%20the%20memory%20won%27t%20be%20written%20to%20disk%20while%20locked.%20As%20you%20noted%2C%20the%20machine%20may%20be%20hibernated%2C%20or%20it%20may%20be%20running%20in%20a%20VM%20that%20gets%20snapshotted.%20%2DRaymond%5D) and macOS as well.
Therefore, a user with a strong OS password and a strong wallet passphrase could still have their keys stolen if a thief takes their (hibernated) machine and reads the permanent storage.
ACKs for top commit:
S3RK:
Code review ACK 3a11adc700
achow101:
ACK 3a11adc700
Tree-SHA512: c4e3dab452ad051da74855a13aa711892c9b34c43cc43a45a3b1688ab044e75d715b42843c229219761913b4861abccbcc8d5cb6ac54957d74f6e357f04e8730
