5493e92501 Check descriptors returned by external signers (sstone)
Pull request description:
Check that descriptors returned by external signers have been parsed properly when creating a new wallet.
See https://github.com/bitcoin/bitcoin/issues/23627 for context.
The problem is that parsing an invalid descriptor will return `null` which is not checked for in `CWallet::SetupDescriptorScriptPubKeyMans()`.
I'm not completely sure what the best fix is since there several strategies for dealing with errors in the current codebase but the proposed fix is very simple and consistent with other validation checks in `CWallet::SetupDescriptorScriptPubKeyMans()`.
ACKs for top commit:
jamesob:
Code review ACK 5493e92501
achow101:
ACK 5493e92501
Tree-SHA512: 63259f4aa519405a86c554b6813efdb741314bdaa18bf005b70ea8bb92a27abc6e2b65f7c584641dc257fc78a6499f42b51b5310c243e611c4663430dccf3d04
The section `Committing Patches` contains an example commit subject line
that violates rule seven of the linked guide to writing commit
logs (Chris Beams famous blog post).
We should practice what we preach, especially in examples :)
Use the imperative mood in example commit message subject line.
61fb410c0d test: add feature_coinstatsindex.py --descriptors to test_runner.py (Sebastian Falbesoner)
50b044a88e test: fix test feature_coinstatsindex.py for descriptor wallets (Sebastian Falbesoner)
Pull request description:
The functional test feature_coinstatsindex.py currently fails on master branch, if descriptor wallets are used (argument `--descriptors`; or if BDB is not compiled, see https://github.com/bitcoin/bitcoin/pull/23682#issuecomment-989827592). This is due to the fact that different change output types are used for created transactions (P2WPKH for legacy wallets, P2TR for descriptor wallets; the former doesn't have a ScriptPubKeyMan for bech32m), resulting in different tx sizes and hence also fees. Fix this by explicitely setting the output type via passing both `-addresstype=bech32` and `-changetype=bech32` as argument. The former would not be needed by now, but makes the test more deterministic and avoids a failure if bech32m becomes the default address type.
Should further pave the way for #23682.
ACKs for top commit:
MarcoFalke:
cr ACK 61fb410c0d
Tree-SHA512: 300a53f539c0b874da5fc1dd1e4e41b9408dc5526c5858c79f0aabf2ab07e57df4c9cc627fafe25246206752754a91a2977a3df8f8b2d98fb98e51c7e4d81633
05300c1439 Use SelectionResult in SelectCoins (Andrew Chow)
9d9b101d20 Use SelectionResult in AttemptSelection (Andrew Chow)
bb50850a44 Use SelectionResult for waste calculation (Andrew Chow)
e8f7ae5eb3 Make an OutputGroup for preset inputs (Andrew Chow)
51a9c00b4d Return SelectionResult from SelectCoinsSRD (Andrew Chow)
0ef6184575 Return SelectionResult from KnapsackSolver (Andrew Chow)
60d2ca72e3 Return SelectionResult from SelectCoinsBnB (Andrew Chow)
a339add471 Make member variables of SelectionResult private (Andrew Chow)
cbf0b9f4ff scripted-diff: Use SelectionResult in coin selector tests (Andrew Chow)
9d1d86da04 Introduce SelectionResult struct (Andrew Chow)
94d851d28c Fix bnb_search_test to use set equivalence for (Andrew Chow)
Pull request description:
Instead of returning a set of selected coins and their total value as separate items, encapsulate both of these, and other variables, into a new `SelectionResult` struct. This allows us to have all of the things relevant to a coin selection solution be in a single object. `SelectionResult` enables us to implement the waste calculation in a cleaner way.
All of the coin selection functions (`SelectCoinsBnB`, `KnapsackSolver`, `AttemptSelection`, and `SelectCoins`) are changed to use a `SelectionResult` as the output parameter.
Based on #22009
ACKs for top commit:
laanwj:
Code review ACK 05300c1439
Tree-SHA512: e4dbb4d78a6cda9c237d230b19e7265591efac5a101a64e6970f0654e2c4f93d13bb5d07b98e8c7b8d37321753dbfc94c28c3a7810cb1c59b5bc29b08a8493ef
fa1b63c018 test: Replace hashlib.new with named constructor (MarcoFalke)
Pull request description:
A small refactor that doesn't matter too much, but it using the named constructor is nice because:
* It clarifies that it is a built-in function
* It is (trivially) faster and less code.
ACKs for top commit:
Zero-1729:
ACK fa1b63c018
w0xlt:
ACK fa1b63c
Tree-SHA512: d23dc4552c1e6fc1f90f8272e47e4efcbe727f0b66a6f6a264db8a50ee6cb6d57a2809befcb95fda6725136672268633817a03dd1859f2298d20e3f9e0ca4a7f
This simplifies the code, and slightly speeds up the test.
Running `./test/functional/test_runner.py -j15 $(printf 'feature_fee_estimation %.0s' {1..15})`
on master 3 times gives:
- Before:
ALL | ✓ Passed | 788 s (accumulated)
ALL | ✓ Passed | 818 s (accumulated)
ALL | ✓ Passed | 873 s (accumulated)
- After:
ALL | ✓ Passed | 763 s (accumulated)
ALL | ✓ Passed | 798 s (accumulated)
ALL | ✓ Passed | 731 s (accumulated)
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
b062da0090 contrib: add check for wget command in install_db4.sh (Florian Baumgartl)
Pull request description:
This PR is motivated by 7bb8eb0bc3 commit (see also https://github.com/bitcoin/bitcoin/pull/23579) and ensures that `install_db4.sh` will check for `curl` and `wget` utilities. Currently, the conditional statement in the `http_get()` function assumes that `wget` is always available but we actually do not know it since there is no check or validation for the `wget` command. So let's make sure that we check for both commands and print an error message if they are missing.
ACKs for top commit:
jamesob:
ACK b062da0090
laanwj:
Tested ACK b062da0090
shaavan:
ACK b062da0090
Tree-SHA512: bfc1ccad9a5b99764b759e02dde1976616c2af4747b7d5af8e71d33624c2cb21d93a09a60d244756e86bbd5fd7541331c62d7eb84d3458b6a059f1d9cb2a5f42
19f3896c9a build: Fix x86_64 <-> arm64 cross-compiling for macOS (Hennadii Stepanov)
Pull request description:
Currently, on master (111c3e06b3), dependencies are built for the build system architecture, not the provided host.
On Intel-based macOS Big Sur 11.6.1 (20G224):
```
% make -C depends HOST=arm64-apple-darwin20
% lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a
Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: x86_64
```
On M1-based macOS Monterey 12.0.1 (21A559) the `boost` package building fails with multiple errors like that:
```
% make -C depends boost HOST=x86_64-apple-darwin19
...
error: option 'cf-protection=return' cannot be specified on this target
error: option 'cf-protection=branch' cannot be specified on this target
2 errors generated.
```
This PR allows to cross-compile as follows:
- on Intel-based macOS Big Sur 11.6.1 (20G224):
```
% make -C depends HOST=arm64-apple-darwin20
% lipo -info depends/arm64-apple-darwin20/lib/libsqlite3.a
Non-fat file: depends/arm64-apple-darwin20/lib/libsqlite3.a is architecture: arm64
% CONFIG_SITE=$PWD/depends/arm64-apple-darwin20/share/config.site ./configure
% make
% lipo -info src/qt/bitcoin-qt
Non-fat file: src/qt/bitcoin-qt is architecture: arm64
```
- on M1-based macOS Monterey 12.0.1 (21A559):
```
% make -C depends HOST=x86_64-apple-darwin19
% CONFIG_SITE=$PWD/depends/x86_64-apple-darwin19/share/config.site ./configure
% make
% lipo -info src/qt/bitcoin-qt
Non-fat file: src/qt/bitcoin-qt is architecture: x86_64
```
No behavior change for other builder-host pairs.
This is an alternative to bitcoin/bitcoin#22506.
ACKs for top commit:
fanquake:
ACK 19f3896c9a - this is definitely cleaner than the approach in #22506. I've tested the x86_64 -> arm64 and arm64 -> x86_64 cross-compiles. Going to go-ahead and merge this given it's scoped to building for darwin targets, on darwin. Not sure if this is something we'll actually see used very often.
Tree-SHA512: 67a7fed1db79bc43d41aabb5a676f90417a879e5dcffec1384c288b56ee32ba06ae3819abdc7aca85b81af6e782bcfff5c83a67a455560249fa095a807fc1187
6575d354c8 build: Bump Fonconfig version up to 2.12.6 (Hennadii Stepanov)
Pull request description:
This PR gets rid of `remove_char_width_usage.patch`.
Some additional observations:
1. Newer Fontconfig versions (2.13.0 and 2.13.1) introduce a new dependency, `uuid`, in the [`7b48fd3dd406b926f0e5240b211f72197ed538a9`](7b48fd3dd4) commit
2. In Fonconfig 2.13.1 (the current stable) excludes the `fcobjshash.h` from the distributive archive (see [`31269e3589e0e6432d12f55db316f4c720a090b5`](31269e3589)), that makes our `gperf_header_regen.patch` unusable, and requires `gperf` as a dependency.
ACKs for top commit:
fanquake:
ACK 6575d354c8 - from the best I can determine this doesn't have any versioning / ABI implications. The ABI difference between 2.12.1 and 2.12.6 is two symbol additions, neither of which are used by Qt. Fontconfig seems to be better at maintaining backwards compatibility compared to a library like Freetype.
Tree-SHA512: 36780a0c5a658469697e524d682ebab56c320cb04f8297bc215f4552f183d4f560501fb0a869982fd9053d4a2d571c7fd971d8f5e96c9da9a9d142c485e3baa4
3431839c33 util, refactor: Improve headers for bitcoin-wallet tool (Hennadii Stepanov)
Pull request description:
This PR:
- removes unneeded `#include <wallet/wallet.h>` from `<wallet/wallettool.h>`
- introduces class forward declaration in `<wallet/wallettool.h>`
- added `#include <config/bitcoin-config.h>` to `wallet/wallettool.cpp` where the `USE_BDB` macro is used
Top commit has no ACKs.
Tree-SHA512: a0de560d821f8b570ae806a1165b9b382c9e0b339687d932052fa4c38ab2ba493e7e050f19adc02ad7db40c42cf88ac1d37209f9071494a0ab268ed33ff22b9f
fa9aaf8694 scripted-diff: Use named args in RPC docs (MarcoFalke)
Pull request description:
Incorrect named args are source of bugs, like #22979.
To allow them being checked by `clang-tidy`, use a format it can understand.
ACKs for top commit:
fanquake:
ACK fa9aaf8694 - checked `clang-tidy` and it's fine here, (but throwing errors in other files. i.e `wallet/test/wallet_tests.cpp`).
Tree-SHA512: e09dae8ee999a5c4819e6f848c12139593ca0e915e645c8fabeb97c379188fb9104d286c02c71f590abc64cdec125f78026735f83e016111976baa49d588a9bc
84bc35d7a5 test: feature_rbf.py: check specified wallet type availability (Sebastian Falbesoner)
Pull request description:
The test currently leads to a failure if in general wallet support is compiled, but the library for the specified type (BDB/SQLite) is not, i.e. if started with the `--legacy-wallet` parameter, but bitcoind is compiled without BDB support, see e.g. https://github.com/bitcoin/bitcoin/pull/23682#issuecomment-989044207
Fix this by checking if the specified wallet type (BDB for legacy wallet, SQLite for descriptor wallet) is available.
Also move the helper `is_specified_wallet_compiled()` to the
test framework's class BitcoinTestFramework first, so it can be reused.
Should further pave the way for #23682. On my local instance without BDB compiled, all targets in test_runner pass now.
ACKs for top commit:
mzumsande:
ACK 84bc35d7a5, changes loook good for me and I confirmed that this fixes the error encountered on master when running the test `--without-bdb`.
Tree-SHA512: 1575c03c793c8e0ac195d0914eff75d02604301c8fb77d0fdb7c0b245561569c0c7db387ef4de499044b68ab6e14b4b78b955f6e74c84197bcaed95f439c9824
5b559dc7ec Swap out hashlib.ripemd160 for own implementation (Pieter Wuille)
ad3e9e1f21 Add pure Python RIPEMD-160 (Pieter Wuille)
Pull request description:
Closes#23710.
ACKs for top commit:
jamesob:
ACK 5b559dc7ec, pending CI
Tree-SHA512: dcd4ea2027eac572f7ab0da434b081b9a5d6b78675e559258a446b4d254b29d93c4d2cc12da4a28303543d6d99f5f2246fde4052e84af81d18e04399b137b39e
2f356a0ca8 scripted-diff: Drop Darwin version for better maintainability (Hennadii Stepanov)
Pull request description:
After this PR, any macOS tools version bumping in the future will touch fewer files in the repo.
Pointing a Darwin version for the `--host` system does not matter for the following reasons:
- in terms of the resulted binaries, we should only care about the minimum supported macOS version which is a separated parameter in our build system.
- in terms of the build system itself, the usage of the `$(host)` variable is self-consistent enough. Btw `$(host_os)` value already has the version dropped:
```
$ make -C depends --no-print-directory print-host_os HOST=x86_64-apple-darwin19
host_os=darwin
```
ACKs for top commit:
gruve-p:
ACK 2f356a0ca8
promag:
ACK 2f356a0ca8.
fanquake:
ACK 2f356a0ca8
Tree-SHA512: 374896ab0ba02b0d8b4b21431fe963bd213b0d09586e0898c13a4c5fa294c1b693f1b2c92880c245c4157c14217b4825b36522f461930477f4d2a727086ebb2a
faa0833c43 doc: Normalize RPC description whitespace (MarcoFalke)
Pull request description:
It is tedious to manually add trailing newlines after the description so that there is an empty new line before the `Arguments` section.
Fix that by adding it with C++ code.
ACKs for top commit:
fanquake:
ACK faa0833c43
Tree-SHA512: 061786c7f19d767f2b7a0362b948e34d181f4cc740a60211756da29ece7554e95be39a9beec3e201eddc8da3ea7e22ac917479eae04b230bb7b0db7a9647af8c
Often when we're updating mempool entries we update entries that we
ultimately end up removing the updated entries shortly thereafter. This
patch makes it so that we filter for such entries a bit earlier in
processing, which yields a mild improvement for these cases, and is
negligible overhead otherwise.
The test currently leads to a failure if in general wallet
support is compiled, but the library for the specified type
(BDB/SQLite) is not, i.e. if started with the
`--legacy-wallet` parameter, but bitcoind is compiled
without BDB support.
Fix this by checking if the specified wallet type (BDB for
legacy wallet, SQLite for descriptor wallet) is available.
Also move the helper `is_specified_wallet_compiled()` to the
test framework's class BitcoinTestFramework first, so it can
be reused.
b57bf25cfe test: interface_bitcoin_cli.py: check specified wallet type availability (Sebastian Falbesoner)
Pull request description:
Currently the test `interface_bitcoin_cli.py` performs the wallet-relevant parts if _any_ wallet type support is compiled in, independently of
whether the test is run with legacy or descriptor wallet specified. This leads to a failure if the test is started with the `--legacy-wallet` parameter, but bitcoind is compiled without BDB support, see e.g
https://github.com/bitcoin/bitcoin/pull/23686#issuecomment-987705540
Fix this by checking if the specified wallet type (BDB for legacy wallet, SQLite for descriptor wallet) is available.
Should further pave the way for #23682.
ACKs for top commit:
achow101:
ACK b57bf25cfe
Tree-SHA512: ddb5a94ba61133eff8de79d4946b3b9d476232b26e83bf768894cac4691e72602f88b6c02c72b992e12c2feb9bff1f0a2e0a265948a00954311104add1347184
Currently the test performs the wallet-relevant parts if
_any_ wallet type support is compiled in, independently of
whether the test is run with legacy or descriptor wallet
specified. This leads to a failure if the test is started
with the `--legacy-wallet` parameter, but bitcoind is
compiled without BDB support.
Fix this by checking if the specified wallet type (BDB for
legacy wallet, SQLite for descriptor wallet) is available.
When topping up an inactive HD chain, either key_origin will be
available and we can use the path given there, or we need to figure out
the path from the string hdKeypath.
fa77f95c2f fuzz: Fix RPC internal bug detection (MarcoFalke)
Pull request description:
Previously the fuzz test considered any exception which contains the string `Internal bug detected` (magic string) as a bug. This is not true when the user (fuzzer) passes in the magic string from outside.
Fix that by:
1. Changing the format the string in `NonFatalCheckError` to start with the magic string.
2. Only treat exceptions that start with the magic string as internal bugs.
This should fix the bug because any other exception shouldn't start with the magic string.
To test:
```
echo 'bG9nZ2luZ1y+bUludGVybmFsIGJ1ZyBkZXRlY3RlZAAXCqNcjqNcjuYjeg==' | base64 --decode > /tmp/a
FUZZ=rpc ./src/test/fuzz/fuzz /tmp/a
```
Before:
```
fuzz: test/fuzz/rpc.cpp:365: void rpc_fuzz_target(FuzzBufferType): Assertion `error_msg.find("trigger_internal_bug") != std::string::npos' failed.
```
After:
```
Executed /tmp/a in 0 ms
ACKs for top commit:
shaavan:
crACK fa77f95c2f
Tree-SHA512: 079bc97b6ce0cbad8603c7b577cc1ac0fd19e884ccbaba317588b91d98b36afeaa8cb398344b52bf12c9fd1737b3fdd8452b4e833a3b06cb3c789651955f78b8
aaaa34e34d doc: Add missing optional to getblockfrompeer (MarcoFalke)
Pull request description:
Can be reviewed with `--word-diff-regex=. --ignore-all-space`
ACKs for top commit:
Sjors:
utACK aaaa34e34d
Tree-SHA512: 7f46c82a46b8cc19f7eb549b9aa13be8cd6849a8ef8a2ddda6d1eee6978d099fccadd29a2bf817f44d601b905f5d5f6b5d8f4f54be5ee8b914b520359c058e68
ffd11ea876 Fix typo and grammar (Heebs)
Pull request description:
Fix typo and grammar in the coin selection algorithm's description.
ACKs for top commit:
meshcollider:
ACK ffd11ea876
Tree-SHA512: bba07c2efd5140fb3e021618739d70aaa761bbc274afb8158809492b0606773c217e42e58e58b18a2454b9c45ebc883ebece17cdc467ac60e3d3140d7a979db7
dce8c4c381 rpc: getblockfrompeer (Sjors Provoost)
b884ababc2 rpc: move Ensure* helpers to server_util.h (Sjors Provoost)
Pull request description:
This adds an RPC method to fetch a block directly from a peer. This can used to fetch stale blocks with lower proof of work that are normally ignored by the node (`headers-only` in `getchaintips`).
Usage:
```
bitcoin-cli getblockfrompeer HASH peer_n
```
Closes#20155
Limitations:
* you have to specify which peer to fetch the block from
* the node must already have the header
ACKs for top commit:
jnewbery:
ACK dce8c4c381
fjahr:
re-ACK dce8c4c381
Tree-SHA512: 843ba2b7a308f640770d624d0aa3265fdc5c6ea48e8db32269b96a082b7420f7953d1d8d1ef2e6529392c7172dded9d15639fbc9c24e7bfa5cfb79e13a5498c8
f1f10c0514 Remove CTxMemPool params from ATMP (lsilva01)
Pull request description:
Remove `CTxMemPool` parameter from `AcceptToMemoryPool` function, as suggested in https://github.com/bitcoin/bitcoin/pull/23437#issuecomment-962536149 .
This requires that `CChainState` has access to `MockedTxPool` in `tx_pool.cpp` as mentioned https://github.com/bitcoin/bitcoin/pull/23173#discussion_r731895386. So the `MockedTxPool` is attributed to `CChainState::m_mempool` before calling `AcceptToMemoryPool`.
Requires #23437.
ACKs for top commit:
jnewbery:
utACK f1f10c0514
MarcoFalke:
review ACK f1f10c0514🔙
Tree-SHA512: 2a4885f4645014fc1fa98bb1090f13721c1a0796bc0021b9cb43bc8cc13920b6eaf057d1f5ed796e0a110e7813e41fe0196334ce7c80d1231fc057a9a3bdf349
d796091b04 build: Bump AX_PTHREAD macro to the latest version (Hennadii Stepanov)
Pull request description:
This PR silents autoconf >2.69 (this [one](https://formulae.brew.sh/formula/autoconf), for instance) warnings about the obsolete `$as_echo`:
```
% ./autogen.sh
...
configure.ac:847: warning: $as_echo is obsolete; use AS_ECHO(["message"]) instead
lib/m4sugar/m4sh.m4:692: _AS_IF_ELSE is expanded from...
lib/m4sugar/m4sh.m4:699: AS_IF is expanded from...
./lib/autoconf/general.m4:2249: AC_CACHE_VAL is expanded from...
./lib/autoconf/general.m4:2270: AC_CACHE_CHECK is expanded from...
build-aux/m4/ax_pthread.m4:89: AX_PTHREAD is expanded from...
configure.ac:847: the top level
...
```
No other behavior changes.
ACKs for top commit:
fanquake:
ACK d796091b04 - matches upstream at serial 31.
Tree-SHA512: aa9b60698f453427221444a5a63420d833c4c5dd23f8b0c74e5bd4639daec9c6cff0907a5281c00103ccb030e394998cf05653be750d4a3bf0f37ca41ff6fbe1
