e4b0dabb21 test: add functional test for tagged MiniWallet instances (Sebastian Falbesoner)
3162c917e9 test: fix MiniWallet internal key derivation for tagged instances (Sebastian Falbesoner)
c9f7364ab2 test: fix MiniWallet script-path spend (missing parity bit in leaf version) (Sebastian Falbesoner)
7774c314fb test: refactor: return TaprootInfo from P2TR address creation routine (Sebastian Falbesoner)
Pull request description:
This PR fixes a dormant bug in MiniWallet that exists since support for P2TR was initially added in #23371 (see commit 041abfebe4).
In the course of spending the output, the leaf version byte of the control block in the witness stack doesn't set the parity bit, i.e. we were so far just lucky that the used combinations of relevant data (internal pubkey, leaf script / version) didn't result in a tweaked pubkey with odd y-parity. If that was the case, we'd get the following validation error:
`mandatory-script-verify-flag-failed (Witness program hash mismatch) (-26)`
Since MiniWallets can now optionally be tagged (#29939), resulting in different internal pubkeys, the issue is more prevalent now. Fix it by passing the parity bit, as specified in BIP341.
Can be tested with the following patch (fails on master, succeeds on PR):
```diff
diff --git a/test/functional/test_framework/mempool_util.py b/test/functional/test_framework/mempool_util.py
index 148cc935ed..7ebe858681 100644
--- a/test/functional/test_framework/mempool_util.py
+++ b/test/functional/test_framework/mempool_util.py
@@ -42,7 +42,7 @@ def fill_mempool(test_framework, node):
# Generate UTXOs to flood the mempool
# 1 to create a tx initially that will be evicted from the mempool later
# 75 transactions each with a fee rate higher than the previous one
- ephemeral_miniwallet = MiniWallet(node, tag_name="fill_mempool_ephemeral_wallet")
+ ephemeral_miniwallet = MiniWallet(node, tag_name="fill_mempool_ephemeral_wallet3")
test_framework.generate(ephemeral_miniwallet, 1 + num_of_batches * tx_batch_size)
# Mine enough blocks so that the UTXOs are allowed to be spent
```
In addition to that, another bug is fixed where the internal key derivation failed, as not every pseudorandom hash results in a valid x-only pubkey. Fix this by treating the hash result as private key and calculate the x-only public key out of that, to be used then as internal key.
Fixes#30528.
ACKs for top commit:
glozow:
ACK e4b0dabb21
rkrux:
reACK [e4b0dab](e4b0dabb21)
hodlinator:
ACK e4b0dabb21
Tree-SHA512: a16f33f76bcb1012857cc3129438a9f6badf28aa2b1d25696da0d385ba5866b46de0f1f93ba777ed9263fe6952f98d7d9c44ea0c0170a2bcc86cbef90bf6ac58
fac0c3d4bf doc: Add release notes for two pull requests (MarcoFalke)
fa7b57e5f5 refactor: Replace ParseHashStr with FromHex (MarcoFalke)
fa90777245 rest: Reject truncated hex txid early in getutxos parsing (MarcoFalke)
fab6ddbee6 refactor: Expose FromHex in transaction_identifier (MarcoFalke)
fad2991ba0 refactor: Implement strict uint256::FromHex() (MarcoFalke)
fa103db2bb scripted-diff: Rename SetHex to SetHexDeprecated (MarcoFalke)
fafe4b8051 test: refactor: Replace SetHex with uint256 constructor directly (MarcoFalke)
Pull request description:
In `rest_getutxos` truncated txids such as `aa` or `ff` are accepted. This is brittle at best.
Fix it by rejecting any truncated (or overlarge) input.
----
Review note: This also starts a major refactor to rework hex parsing in Bitcoin Core, meaning that a few refactor commits are included as well. They are explained individually in the commit message and the work will be continued in the future.
ACKs for top commit:
stickies-v:
re-ACK fac0c3d4bf - only doc and test updates to address review comments, thanks!
hodlinator:
ACK fac0c3d4bf
Tree-SHA512: 473feb3fcf6118443435d1dd321006135b0b54689bfbbcb1697bb5811a449bef51f475c715de6911ff3c4ea3bdb75f601861ff93347bc4414d6b9e5298105dd7
25bf86a225 [test]: ensure `estimatesmartfee` default mode is `economical` (ismaelsadeeq)
41a2545046 [fees]: change `estimatesmartfee` default mode to `economical` (ismaelsadeeq)
Pull request description:
Fixes#30009
This PR changes the `estimatesmartfee` default mode to `economical`.
This was also suggested on IRC https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2024-04-26#1021609
- `conservative` mode: This is the `estimatesmartfee` RPC mode which considers a longer history of blocks. It potentially returns a higher fee rate and is more likely to be sufficient for the desired target, but it is not as responsive to short-term drops in the prevailing fee market.
- `economical` mode: This is the `estimatesmartfee` RPC mode where estimates are potentially lower and more responsive to short-term drops in the prevailing fee market.
Since users are likely to use the default mode, this change will reduce overestimation for many users. The conservative mode remains available for those who wish to opt-in.
For an in-depth analysis of how significantly the `conservative` mode overestimates, see
https://delvingbitcoin.org/t/bitcoind-policy-estimator-modes-analysis/964.
ACKs for top commit:
instagibbs:
reACK 25bf86a225
glozow:
ACK 25bf86a225
willcl-ark:
ACK 25bf86a225
Tree-SHA512: 78ebda667eb9c8f87dcc2f0e6c14968bd1de30358dc77a13611b186fb8427ad97d9f537bad6e32e0a1aa477ccd8c64fee4d41e19308ef3cb184ff1664e6ba8a6
d63ef73800 test: Add loadtxoutset test with tip on snapshot block (Fabian Jahr)
c2f86d4bcb test: Remove already resolved assumeutxo todo comments (Fabian Jahr)
Pull request description:
The first commit removes three Todos that have been addressed previously (see commit message for details).
The second message resolves another todo by adding the missing test case. This is a special case of "the tip has more work than the snapshot" where the tip is the same block as the snapshot base block.
Related to #28648.
ACKs for top commit:
jrakibi:
ACK [d63ef73](d63ef73800)
achow101:
ACK d63ef73800
maflcko:
ACK d63ef73800
alfonsoromanz:
Re ACK d63ef73800
Tree-SHA512: 8d5a25fc0b26531db3a9740132694138f2103b7b42eeb1d4a64095bfc901c1372e23601c0855c7def84c8a4e185d10611e4e830c4e479f1b663ae6ed53abb130
a8e3af1a82 qa: Do not assume running `feature_asmap.py` from source directory (Hennadii Stepanov)
9bf7ca6cad qa: Consider `cache` and `config.ini` relative to invocation directory (Hennadii Stepanov)
a0473442d1 scripted-diff: Add `__file__` argument to `BitcoinTestFramework.init()` (Hennadii Stepanov)
Pull request description:
This PR includes changes split from https://github.com/bitcoin/bitcoin/pull/30454. They improve the functional test framework, allowing users to [run individual functional tests](https://github.com/hebasto/bitcoin/issues/146) from the build directory in the new CMake-based build system.
This functionality is not available for out-of-source builds using the current Autotools-based build system, which always requires write permissions for the source directory. Nevertheless, this PR can be tested as suggested in https://github.com/bitcoin/bitcoin/pull/30463#issuecomment-2232618421:
1. Make an out-of-source build:
```
$ ./autogen.sh
$ mkdir ../build && cd ../build
$ ../bitcoin/configure
$ make
```
2. Create a symlink in the build directory to a functional test:
```
$ ln --symbolic ../../../bitcoin/test/functional/wallet_disable.py ./test/functional/
```
3. Run this symlink:
```
$ ./test/functional/wallet_disable.py
```
The last command fails on the master branch:
```
Traceback (most recent call last):
File "/home/hebasto/git/build/./test/functional/wallet_disable.py", line 31, in <module>
DisableWalletTest().main()
^^^^^^^^^^^^^^^^^^^
File "/home/hebasto/git/bitcoin/test/functional/test_framework/test_framework.py", line 106, in __init__
self.parse_args()
File "/home/hebasto/git/bitcoin/test/functional/test_framework/test_framework.py", line 210, in parse_args
config.read_file(open(self.options.configfile))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '/home/hebasto/git/bitcoin/test/config.ini'
```
and succeeds with this PR.
ACKs for top commit:
maflcko:
tested ACK a8e3af1a82🎨
glozow:
ACK a8e3af1a82, tested with the steps in op
stickies-v:
ACK a8e3af1a82
Tree-SHA512: 899e4efc09edec13ea3f5b47825d03173fb21d3569c360deda7fa6a56b99b4d24e09ad4f0883bad1ee926b1c706e47ba07c6a6160c63c07c82b3cf4ae5816e91
Also pulls out the guarding assert and calls it explicitly before the test function is called. This is already done before the existing call of the test function so it was not needed there.
- "Valid snapshot file, but referencing a snapshot block that turns out
to be invalid, or has an invalid parent" has been addressed in #30267
- "An ancestor of snapshot block" - If chain tip refers to blocks in this context then any successful load is addressing this because if we have synced past the snapshot base block we fail because we don't need assumeutxo anymore. And if this is about headers then this is the `test_headers_not_synced()` case.
- "A descendant of the snapshot block" - If this refers to blocks the
`test_snapshot_with_less_work()` addressed this and if it is just headers in this case again it would be represented in all of the successful loads in the test.
Co-authored-by: Alfonso Roman Zubeldia <alfonsoromanz24@gmail.com>
55b6d7be68 validation: Don't load a snapshot if it's not in the best header chain. (Martin Zumsande)
Pull request description:
This was suggested by me in the discussion of #30288, which has more context.
If the snapshot is not an ancestor of the most-work header (`m_best_header`), syncing from that alternative chain leading to `m_best_header` should be prioritised. Therefore it's not useful loading the snapshot in this situation.
If the other chain turns out to be invalid or the chain with the snapshot retrieves additional headers so that it's the most-work one again (see functional test), `m_best_header` will change and loading the snapshot will be possible again.
Because of the work required to generate a conflicting headers chain, a situation with two conflicting chains should only be possible under extreme circumstances, such as major forks.
ACKs for top commit:
fjahr:
re-ACK 55b6d7be68
achow101:
ACK 55b6d7be68
alfonsoromanz:
Re ACK 55b6d7be68
Tree-SHA512: 4fbea5ab1038ae353fc949a186041cf9b397e7ce4ac59ff36f881c9437b4f22ada922490ead5b2661389eb1ca0f3d1e7e7e6a4261057678643e71594a691ac36
fac932bf93 refactor: Use util::Split to avoid a harmless unsigned-integer-overflow (MarcoFalke)
fab54db9f1 rest: Reject negative outpoint index in getutxos parsing (MarcoFalke)
Pull request description:
In `rest_getutxos` outpoint indexes such as `+N` or `-N` are accepted. This should be harmless, because any index out of range should be treated as a non-existent utxo. However, a negative index can't exist ever, so it seems better to reject all signs, whether `+` or `-`.
ACKs for top commit:
achow101:
ACK fac932bf93
hodlinator:
ut-ACK fac932bf93
tdb3:
re ACK fac932bf93
danielabrozzoni:
ACK fac932bf93
brunoerg:
reACK fac932bf93
Tree-SHA512: 8f1a75248cb61e1c4beceded6ed170db83b07f30fbcf93a26acfffc00ec4546572366eff87907a7e1423d7d3a2a9e57a0a7a9bacb787c86463f842d7161c16bc
faed5d3870 test: Non-Shy version sender (MarcoFalke)
Pull request description:
After `add_outbound_p2p_connection`, the test framework normally sends a version message only in reply to a received version. This is fine, but the protocol does not require this and tolerates a version to be sent earlier.
However, this is untested, and the missing test coverage leads to bugs being missed. For example https://github.com/bitcoin/bitcoin/pull/30394#pullrequestreview-2166824948
Fix it by adding a test.
ACKs for top commit:
brunoerg:
ACK faed5d3870
tdb3:
ACK faed5d3870
theStack:
tACK faed5d3870
glozow:
ACK faed5d3870
Tree-SHA512: dbf527a39c932e994a1e8248ba78058000811a4bf69275278f1fd1e545716ac4d2d3be5dcf362976bbafa2a49f91d13e3601daf71d29e9c556179b01af62c03c
734076c6de [wallet, rpc]: add `max_tx_weight` to tx funding options (ismaelsadeeq)
b6fc5043c1 [wallet]: update the data type of `change_output_size`, `change_spend_size` and `tx_noinputs_size` to `int` (ismaelsadeeq)
baab0d2d43 [doc]: update reason for deducting change output weight (ismaelsadeeq)
7f61d31a5c [refactor]: update coin selection algorithms input parameter `max_weight` name (ismaelsadeeq)
Pull request description:
This PR taken over from #29264
The PR added an option `max_tx_weight` to transaction funding RPC's that ensures the resulting transaction weight does not exceed the specified `max_tx_weight` limit.
If `max_tx_weight` is not given `MAX_STANDARD_TX_WEIGHT` is used as the max threshold.
This PR addressed outstanding review comments in #29264
For more context and rationale behind this PR see https://delvingbitcoin.org/t/lightning-transactions-with-v3-and-ephemeral-anchors/418/11?u=instagibbs
ACKs for top commit:
achow101:
ACK 734076c6de
furszy:
utACK 734076c6de
rkrux:
reACK [734076c](734076c6de)
Tree-SHA512: 013501aa443d239ee2ac01bccfc5296490c27b4edebe5cfca6b96c842375e895e5cfeb5424e82e359be581460f8be92095855763a62779a18ccd5bdfdd7ddce7
c322bddd08 test: bump mocktime after node has received and sent bytes (stratospher)
Pull request description:
Fixes an intermittent failure for `p2p_v2_misbehaving.py` reported in https://github.com/bitcoin/bitcoin/pull/30420#discussion_r1680462164.
A [different error message](262260ce1e/src/net.cpp (L1970)) `"socket no message in first %i seconds"` will be displayed if `m_last_send=0` or if `m_last_recv is 0`. Fix this by:
1. mocktime bump is done after all the bytes are received. (`m_last_recv is not 0 now`)
2. wait until bytes are sent by `TestNode`/`bitcoind` (`m_last_send is not 0 now`)
See https://cirrus-ci.com/task/5359619151757312?logs=ci#L3935 for an example failure (I wasn't able to reproduce the intermittent failure locally but I think the fix is logical)
ACKs for top commit:
maflcko:
reACK c322bddd08
Tree-SHA512: 1c05524c2819041eb2001c2baf2c912d4f812a39347f784f212634e8c53131357a73116a46b4b7542bc7fc8c1370c4d36fc9898a2cbdb40bcee61105123c4a35
fa6390df20 doc: getaddressinfo[isscript] is optional (MarcoFalke)
Pull request description:
`isscript` is unknown for unknown witness versions, so it should be marked optional in the docs
Fixes https://github.com/bitcoin/bitcoin/issues/30456
ACKs for top commit:
stickies-v:
ACK fa6390df20
tdb3:
ACK fa6390df20
Tree-SHA512: f728f18e0871923225e0bf29594f8095997456cf55409f42087b5f70f95bef10f984323b48d2b484b6705f23b04e9e8a3fe42446830638fdd70453c18fd7f189
a different error message "socket no message in first %i seconds"
will be displayed if m_last_send=0 or if m_last_recv is 0. make
the test robust by ensuring that they will not be 0 before
bumping mocktime.
7e36dca657 test: add test for modififed walletprocesspsbt calls (willcl-ark)
39cea21ec5 wallet: fix FillPSBT errantly showing as complete (willcl-ark)
Pull request description:
Fixes: #30077
Fix cases of calls to `FillPSBT` returning `complete=true` when it's not
the case.
This can happen when some inputs have been signed but the transaction is
subsequently modified, e.g. in the context of PayJoins.
Also fixes a related bug where a finalized hex string is attempted to be
added during `walletprocesspsbt` but a CHECK_NONFATAL causes an abort.
ACKs for top commit:
achow101:
ACK 7e36dca657
ismaelsadeeq:
Tested ACK 7e36dca657
pinheadmz:
re-ACK 7e36dca657
Tree-SHA512: e35d19789899c543866d86d513506494d672e4bed9aa36a995dbec4e72f0a8ec5536b57c4a940a18002ae4a8efd0b007c77ba64e57cd52af98e4ac0e7bf650d6
In CMake-based build system (1) `config.ini` is created in the build
directory, and (2) `cache` must also be created in the same directory.
This change enables running individual functional tests from the build
directory.
bca346a970 net: require P2P binds to succeed (Vasil Dimov)
af552534ab net: report an error if unable to bind on the Tor listening addr:port (Vasil Dimov)
9a7e5f4d68 net: don't extra bind for Tor if binds are restricted (Vasil Dimov)
Pull request description:
Make it possible to disable the Tor binding on `127.0.0.1:8334` and stop startup if any P2P bind fails instead of "if all P2P binds fail".
Fixes https://github.com/bitcoin/bitcoin/issues/22726
Fixes https://github.com/bitcoin/bitcoin/issues/22727
ACKs for top commit:
achow101:
ACK bca346a970
cbergqvist:
ACK bca346a970
pinheadmz:
ACK bca346a970
Tree-SHA512: fabef89a957191eea4f3e3b6109d2b8389f27ecc74440a920b0c10f31fff00a85bcfd1eb3c91826c7169c618f4de8a8d0a260e2caf40fd854f07ea9a980d8603
fa6270737e rpc: Use CHECK_NONFATAL over Assert (MarcoFalke)
Pull request description:
Any RPC method should not abort the whole node when an internal logic error happens.
Fix it by just aborting this single RPC method call when an error happens.
Also, fix the linter to find the fixed cases.
ACKs for top commit:
achow101:
ACK fa6270737e
stickies-v:
ACK fa6270737e
tdb3:
ACK fa6270737e
hodlinator:
ACK fa6270737e
Tree-SHA512: dad2f31b01a66578949009499e4385fb4d72f0f897419f2a6e0ea02e799b9a31e6ecb5a67fa5d27fcbc7939fe8acd62dc04e877b35831493b7f2c604dec7dc64
16bd283b3a Reapply "test: p2p: check that connecting to ourself leads to disconnect" (Sebastian Falbesoner)
0dbcd4c148 net: prevent sending messages in `NetEventsInterface::InitializeNode` (Sebastian Falbesoner)
66673f1c13 net: fix race condition in self-connect detection (Sebastian Falbesoner)
Pull request description:
This PR fixes a recently discovered race condition in the self-connect detection (see #30362 and #30368).
Initiating an outbound network connection currently involves the following steps after the socket connection is established (see [`CConnman::OpenNetworkConnection`](bd5d1688b4/src/net.cpp (L2923-L2930)) method):
1. set up node state
2. queue VERSION message (both steps 1 and 2 happen in [`InitializeNode`](bd5d1688b4/src/net_processing.cpp (L1662-L1683)))
3. add new node to vector `m_nodes`
If we connect to ourself, it can happen that the sent VERSION message (step 2) is received and processed locally *before* the node object is added to the connection manager's `m_nodes` vector (step 3). In this case, the self-connect remains undiscovered, as the detection doesn't find the outbound peer in `m_nodes` yet (see `CConnman::CheckIncomingNonce`).
Fix this by swapping the order of 2. and 3., by taking the `PushNodeVersion` call out of `InitializeNode` and doing that in the `SendMessages` method instead, which is only called for `CNode` instances in `m_nodes`.
The temporarily reverted test introduced in #30362 is readded. Fixes#30368.
Thanks go to vasild, mzumsande and dergoegge for suggestions on how to fix this (see https://github.com/bitcoin/bitcoin/issues/30368#issuecomment-2200625017 ff. and https://github.com/bitcoin/bitcoin/pull/30394#discussion_r1668290789).
ACKs for top commit:
naiyoma:
tested ACK [16bd283b3a), built and tested locally, test passes successfully.
mzumsande:
ACK 16bd283b3a
tdb3:
ACK 16bd283b3a
glozow:
ACK 16bd283b3a
dergoegge:
ACK 16bd283b3a
Tree-SHA512: 5b8aced6cda8deb38d4cd3fe4980b8af505d37ffa0925afaa734c5d81efe9d490dc48a42e1d0d45dd2961c0e1172a3d5b6582ae9a2d642f2592a17fbdc184445
c6d43367a1 test: Fix intermittent failure in p2p_v2_misbehaving.py (stratospher)
Pull request description:
Fixes#30419.
Make sure that ellswift computation is complete in the `NetworkThread` in `test/functional/p2p_v2_misbehaving.py` before sending the ellswift in the `MainThread`.
One way to reproduce this failure on master would be:
```diff
diff --git a/test/functional/test_framework/v2_p2p.py b/test/functional/test_framework/v2_p2p.py
index 87600c36de..ea0615ef3b 100644
--- a/test/functional/test_framework/v2_p2p.py
+++ b/test/functional/test_framework/v2_p2p.py
@@ -111,6 +111,7 @@ class EncryptedP2PState:
def generate_keypair_and_garbage(self, garbage_len=None):
"""Generates ellswift keypair and 4095 bytes garbage at max"""
+ import time; time.sleep(3)
self.privkey_ours, self.ellswift_ours = ellswift_create()
if garbage_len is None:
garbage_len = random.randrange(MAX_GARBAGE_LEN + 1)
```
ACKs for top commit:
maflcko:
ACK c6d43367a1
mzumsande:
Code Review ACK c6d43367a1
tdb3:
cr and t ACK c6d43367a1
Tree-SHA512: dfc3a6afa09773b7e84d35aff0aa14e0b8a4475860e0b31ab5c1a8d54911c814f07138f624fea651fba90cc5c526c0d05c3fe33d2ce0ad833b2be3a3caa9f522
The ellswift bytes are computed in the NetworkThread and sent in
the MainThread. Add a `wait_until()` to make sure that ellswift
computation is completed in NetworkThread before sending it in
the MainThread. Also wait until bytes sent are actually received
and use mocktime for more robust disconnection checking.
00b8e26bd6 test: fix inconsistency in fundrawtransaction weight limits test (furszy)
Pull request description:
Fix https://github.com/bitcoin/bitcoin/pull/30309#discussion_r1657628378 inconsistency.
Currently, the test is passing due to a mistake in the test inputs
selection process. We are selecting the parent transaction change
output as one of the inputs of the transaction to fund, which
helps to surpass the target amount when it shouldn't due to the
fee reduction.
The failure arises when the test behaves as intended by its coder;
that is, when it does not select the change output. In this case,
the pre-selected inputs aren't enough to cover the target amount.
Fix this by excluding the parent transaction's change output from
the inputs selection and including an extra input to cover the tx
fee.
The CI failure can be replicated with the following patch in master:
```diff
diff --git a/test/functional/wallet_fundrawtransaction.py b/test/functional/wallet_fundrawtransaction.py
--- a/test/functional/wallet_fundrawtransaction.py(revision 9b480f7a25)
+++ b/test/functional/wallet_fundrawtransaction.py(date 1720652934739)
@@ -1322,7 +1322,7 @@
outputs = []
for _ in range(1472):
outputs.append({wallet.getnewaddress(address_type="legacy"): 0.1})
- txid = self.nodes[0].send(outputs=outputs)["txid"]
+ txid = self.nodes[0].send(outputs=outputs, change_position=0)["txid"]
self.generate(self.nodes[0], 1)
# 272 WU per input (273 when high-s); picking 1471 inputs will exceed the max standard tx weight.
@@ -1330,7 +1330,7 @@
# 1) Try to fund transaction only using the preset inputs
input_weights = []
- for i in range(1471):
+ for i in range(1, 1472): # skip first output as it is the parent tx change output
input_weights.append({"txid": txid, "vout": i, "weight": 273})
assert_raises_rpc_error(-4, "Transaction too large", wallet.fundrawtransaction, hexstring=rawtx, input_weights=input_weights)
```
ACKs for top commit:
achow101:
ACK 00b8e26bd6
ismaelsadeeq:
Code review and Tested ACK 00b8e26bd6
Tree-SHA512: 5ef792961b7fad4999fc30aa03366432103ddf672ca5cbb366c9eab4c2e46d5ae1ab0c073dfc4fbb2b4e63203653bc0e54463c731c5f8655140207ba5f8e542e
If the snapshot is not an ancestor of the most-work header (m_best_header),
syncing from that alternative chain should be prioritised.
Therefore don't accept loading a snapshot in this situation.
If that other chain turns out to be invalid, m_best_header
would be reset and loading the snapshot should be possible again.
Because of the work required to generate a conflicting headers chain,
this should only be possible under extreme circumstances, such as major forks.
8ce3739edb test: verify wallet is still active post-migration failure (furszy)
771bc60f13 wallet: Use LegacyDataSPKM when loading (Ava Chow)
61d872f1b3 wallet: Move MigrateToDescriptor and DeleteRecords to LegacyDataSPKM (Ava Chow)
b231f4d556 wallet: Move LegacyScriptPubKeyMan::IsMine to LegacyDataSPKM (Ava Chow)
7461d0c006 wallet: Move LegacySPKM data storage and handling to LegacyDataSPKM (Ava Chow)
517e204bac Change MigrateLegacyToDescriptor to reopen wallet as BERKELEY_RO (Ava Chow)
Pull request description:
#26606 introduced `BerkeleyRODatabase` which is an independent parser for BDB files. This PR uses this in legacy wallet migration so that migration will continue to work once the legacy wallet and BDB are removed. `LegacyDataSPKM` is introduced to have the minimum data and functions necessary for a legacy wallet to be loaded for migration.
ACKs for top commit:
cbergqvist:
ACK 8ce3739edb
theStack:
Code-review ACK 8ce3739edb
furszy:
Code review ACK 8ce3739edb
Tree-SHA512: dccea12d6c597de15e3e42f97ab483cfd069e103611200279a177e021e8e9c4e74387c4f45d2e58b3a1e7e2bdb32a1d2d2060b1f8086c03eeaa0c68579d9d54e
Currently, the test is passing due to a mistake in the test inputs
selection process. We are selecting the parent transaction change
output as one of the inputs of the transaction to fund, which
helps to surpass the target amount when it shouldn't due to the
fee reduction.
The failure arises when the test behaves as intended by its coder;
that is, when it does not select the change output. In this case,
the pre-selected inputs aren't enough to cover the target amount.
Fix this by excluding the parent transaction's change output from
the inputs selection and including an extra input to cover the tx
fee.
8789dc8f31 doc: Add note to getblockfrompeer on missing undo data (Fabian Jahr)
4a1975008b rpc: Make pruneheight also reflect undo data presence (Fabian Jahr)
96b4facc91 refactor, blockstorage: Generalize GetFirstStoredBlock (Fabian Jahr)
Pull request description:
The function `GetFirstStoredBlock()` helps us find the first block for which we have data. So far this function only looked for a block with `BLOCK_HAVE_DATA`. However, this doesn't mean that we also have the undo data of that block, and undo data might be required for what a user would like to do with those blocks. One example of how this might happen is if some blocks were fetched using the `getblockfrompeer` RPC. Blocks fetched from a peer will have data but no undo data.
The first commit here allows `GetFirstStoredBlock()` to check for undo data as well by passing a parameter. This alone is useful for #29553 and I would use it there.
In the second commit I am applying the undo check to the RPCs that report `pruneheight` to the user. I find this much more intuitive because I think the user expects to be able to do all operations on blocks up until the `pruneheight` but that is not the case if undo data is missing. I personally ran into this once before and now again when testing for assumeutxo when I had used `getblockfrompeer`. The following commit adds test coverage for this change of behavior.
The last commit adds a note in the docs of `getblockfrompeer` that undo data will not be available.
ACKs for top commit:
achow101:
ACK 8789dc8f31
furszy:
Code review ACK 8789dc8f31.
stickies-v:
ACK 8789dc8f31
Tree-SHA512: 90ae8bdd07a496ade579aa25240609c61c9ed173ad38d30533f6c631fe674e5a41727478ade69ca4b71a571ad94c9da4b33ebba6b5d8821109313c2de3bdfb3d
5b7f70ba26 test: loadtxoutset in divergent chain with less work (Alfonso Roman Zubeldia)
d35efe1efc p2p: Start downloading historical blocks from common ancestor (Martin Zumsande)
Pull request description:
This PR adds a test to cover the scenario of loading an assumeutxo snapshot when the current chain tip is not an ancestor of the snapshot block but has less work.
During the review process, a bug was discovered where blocks between the last common ancestor and the background tip were not being requested if the background tip was not an ancestor of the snapshot block. mzumsande suggested a fix (65343ec49a6b73c4197dfc38e1c2f433b0a3838a) to start downloading historical blocks from the last common ancestor to address this issue. This fix has been incorporated into the PR with a slight modification.
Related to https://github.com/bitcoin/bitcoin/issues/28648
ACKs for top commit:
fjahr:
tACK 5b7f70ba26
achow101:
ACK 5b7f70ba26
mzumsande:
Code Review ACK 5b7f70ba26
Tree-SHA512: f8957349686a6a1292165ea9e0fd8c912d21466072632a10f8ef9d852a5f430bc6b2a531e6884a4dbf2e3adb28b3d512b25919e78f5804a67320ef54c3b1aaf6
d93b794709 tests: improve wallet multisig descriptor test and docs (Michael Dietz)
Pull request description:
It is best to store all key origin information
(master key fingerprint and all derivation steps)
in the multisig descriptor. Being explicit with
this information should be beneficial if this approach is used with other wallets/signers (whether hardware or software). There is no harm including all of this with xpubs (if anything it simplifies the test code) and makes this example/docs more complete and safer incase it is referenced by others.
ACKs for top commit:
S3RK:
Code Review ACK d93b794709
achow101:
ACK d93b794709
Tree-SHA512: 0e5c4d13f060489405e6cf50c8a09911f5a0cee71023649235afd80a5e3aae38d52c6e12ad4660205b9357b09f45596941391bdcf6fceccbe07c4e5a1592a482
c9dacd958d test: Check that non empty version packet is ignored and no disconnection happens (stratospher)
997cc00b95 test: Check that disconnection happens when AAD isn't filled (stratospher)
b5e6238fdb test: Check that disconnection happens when garbage sent/received are different (stratospher)
ad1482d5a2 test: Check that disconnection happens when wrong garbage terminator is sent (stratospher)
e351576862 test: Check that disconnection happens when >4095 garbage bytes is sent (stratospher)
e075fd131d test: Introduce test types and modify v2 handshake function accordingly (stratospher)
7d07daa623 log: Add V2 handshake timeout (stratospher)
d4a1da8543 test: Make global TRANSPORT_VERSION variable an instance variable (stratospher)
c642b08c4e test: Log when the garbage is actually sent to transport layer (stratospher)
86cca2cba2 test: Support disconnect waiting for add_p2p_connection (stratospher)
bf9669af9c test: Rename early key response test and move random_bitflip to util (stratospher)
Pull request description:
Add tests for the following v2 handshake scenarios:
1. Disconnection happens when > `MAX_GARBAGE_LEN` bytes garbage is sent
2. Disconnection happens when incorrect garbage terminator is sent
3. Disconnection happens when garbage bytes are tampered with
4. Disconnection happens when AAD of first encrypted packet after the garbage terminator is not filled
5. bitcoind ignores non-empty version packet and no disconnection happens
All these tests require a modified v2 P2P class (different from `EncryptedP2PState` used in `v2_p2p.py`) to implement our custom handshake behaviour based on different scenarios and have been kept in a single test file (`test/functional/p2p_v2_misbehaving.py`). Shifted the test in `test/functional/p2p_v2_earlykeyresponse.py` which is of the same pattern to this file too.
ACKs for top commit:
achow101:
ACK c9dacd958d
mzumsande:
ACK c9dacd958d
theStack:
Code-review ACK c9dacd958d
Tree-SHA512: 90df81f0c7f4ecf0a47762d290a618ded92cde9f83d3ef3cc70e1b005ecb16125ec39a9d80ce95f99e695d29abd63443240cb5490aa57c5bc8fa2e52149a0672
de71d4dece fuzz: improve utxo_snapshot target (Martin Zumsande)
Pull request description:
Add the possibility of giving more guidance to the creation of the metadata and/or coins, so that the fuzzer gets the chance
to reach more error conditions in ActivateSnapshot and sometimes successfully creates a valid snapshot.
This also changes the asserts for the success case that were outdated (after #29370) and only didn't result in a crash because the fuzzer wasn't able to reach this code before.
ACKs for top commit:
maflcko:
re-ACK de71d4dece🎆
fjahr:
utACK de71d4dece
TheCharlatan:
ACK de71d4dece
Tree-SHA512: 346974d594164544d8cd3df7d8362c905fd93116215e9f5df308dfdac55bab04d727bfd7fd001cf11318682d11ee329b4b4a43308124c04d64b67840ab8a58a0
9e13ccc50e psbt: Check non witness utxo outpoint early (Ava Chow)
Pull request description:
A common issue that our fuzzers keep finding is that outpoints don't exist in the non witness utxos. Instead of trying to track this down and checking in various individual places, do the check early during deserialization. This also unifies the error message returned for this class of problems.
ACKs for top commit:
maflcko:
lgtm ACK 9e13ccc50e
S3RK:
tACK 9e13ccc50e
dergoegge:
utACK 9e13ccc50e
Tree-SHA512: 81b8055b146c6358052226578ddfec0ae5bd877968c7f4f62dc3d6a684545ea568f37c7f1bd619918441af9e453ba8b26531a2280d218da37fa15480f1b45d0e
Add the possibility of giving more guidance to the creation of the
metadata and/or coins, so that the fuzzer gets the chance
to reach more error conditions in ActivateSnapshot and sometimes
successfully creates a valid snapshot.
This also changes the asserts for the success case that were outdated,
and only didn't result in a crash because the fuzzer wasn't able
to reach this code before.
ce8094246e random: replace construct/assign with explicit Reseed() (Pieter Wuille)
2ae392d561 random: use LogError for init failure (Pieter Wuille)
97e16f5704 tests: make fuzz tests (mostly) deterministic with fixed seed (Pieter Wuille)
2c91330dd6 random: cleanup order, comments, static (Pieter Wuille)
8e31cf9c9b net, net_processing: use existing RNG objects more (Pieter Wuille)
d5fcbe966b random: improve precision of MakeExponentiallyDistributed (Pieter Wuille)
cfb0dfe2cf random: convert GetExponentialRand into rand_exp_duration (Pieter Wuille)
4eaa239dc3 random: convert GetRand{Micros,Millis} into randrange (Pieter Wuille)
82de1b80d9 net: use GetRandMicros for cache expiration (Pieter Wuille)
ddc184d999 random: get rid of GetRand by inlining (Pieter Wuille)
e2d1f84858 random: make GetRand() support entire range (incl. max) (Pieter Wuille)
810cdf6b4e tests: overhaul deterministic test randomness (Pieter Wuille)
6cfdc5b104 random: convert XoRoShiRo128PlusPlus into full RNG (Pieter Wuille)
8cc2f45065 random: move XoRoShiRo128PlusPlus into random module (Pieter Wuille)
8f5ac0d0b6 xoroshiro128plusplus: drop comment about nonexisting copy() (Pieter Wuille)
8924f5120f random: modernize XoRoShiRo128PlusPlus a bit (Pieter Wuille)
ddb7d26cfd random: add RandomMixin::randbits with compile-known bits (Pieter Wuille)
21ce9d8658 random: Improve RandomMixin::randbits (Pieter Wuille)
9b14d3d2da random: refactor: move rand* utilities to RandomMixin (Pieter Wuille)
40dd86fc3b random: use BasicByte concept in randbytes (Pieter Wuille)
27cefc7fd6 random: add a few noexcepts to FastRandomContext (Pieter Wuille)
b3b382dde2 random: move rand256() and randbytes() to .h file (Pieter Wuille)
493a2e024e random: write rand256() in function of fillrand() (Pieter Wuille)
Pull request description:
This PR contains a number of vaguely-related improvements to the random module.
The specific changes and more detailed rationale is in the commit messages, but the highlights are:
* `XoRoShiRo128PlusPlus` (previously a test-only RNG) moves to random.h and becomes `InsecureRandomContext`, which is even faster than `FastRandomContext` but non-cryptographic. It also gets all helper randomness functions (`randrange`, `fillrand`, ...), making it a lot more succinct to use.
* During tests, **all** randomness is made deterministic (except for `GetStrongRandBytes`) but non-repeating (like `GetRand()` used to be when `g_mock_deterministic_tests` was used), either fixed, or from a random seed (overridden by env var).
* Several infrequently used top-level functions (`GetRandMillis`, `GetRandMicros`, `GetExponentialRand`) are converted into member functions of `FastRandomContext` (and `InsecureRandomContext`).
* `GetRand<T>()` (without argument) can now return the maximum value of the type (previously e.g. `GetRand<uint32_t>()` would never return 0xffffffff).
ACKs for top commit:
achow101:
ACK ce8094246e
maflcko:
re-ACK ce8094246e🐈
hodlinator:
ACK ce8094246e
dergoegge:
utACK ce8094246e
Tree-SHA512: 79bc0cbafaf27e95012c1ce2947a8ca6f9a3c78af5f1f16e69354b6fc9b987a28858adf4cd356dc5baf21163e9af8dcc24e70f8d7173be870e8a3ddcdd47c02c
Updating to MLC v0.18.0 includes a new feature which will ignore all
files ignored by git: `--gitignore`.
This helps avoid false-positives flagged by this linter in non-project
files, such as a developer might expect to have in their directory (e.g.
guix-builds, python venvs, etc.)
This test checks that we can successfully process PSBTs and opt out of
finalization.
Previously trying to call `walletprocesspsbt` would attempt to
auto-finalize (as a convenience), and would not permit opt-out of
finalization, instead aborting via `CHECK_NONFATAL`.
