561848aaf2 Exercise non-DIRTY spent coins in caches in fuzz test (Pieter Wuille)
59e6828bb5 Add deterministic mode to CCoinsViewCache (Pieter Wuille)
b0ff310840 Add CCoinsViewCache::SanityCheck() and use it in fuzz test (Pieter Wuille)
3c9cea1340 Add simulation-based CCoinsViewCache fuzzer (Pieter Wuille)
Pull request description:
The fuzzer goes through a sequence of operations that get applied to both a real stack of `CCoinsViewCache` objects, and to simulation data, comparing the two at the end.
ACKs for top commit:
jamesob:
re-ACK 561848aaf2
dergoegge:
Code review ACK 561848aaf2
Tree-SHA512: 68634f251fdb39436b128ecba093f651bff12ac11508dc9885253e57fd21efd44edf3b22b0f821c228175ec507df7d46c7f9f5404fc1eb8187fdbd136a5d5ee2
fa8e92c022 doc: Update ci docs (721217.xyz)
5fffff54e9 ci: Cache stuff in volumes, not host folders (MarcoFalke)
Pull request description:
Storing cached stuff in host system folders may lead to unexpected issues when the ci-built stuff is used for a non-ci build or a ci task leaks into another ci task.
ACKs for top commit:
john-moffett:
ACK fa8e92c022
Tree-SHA512: 8b0c9019452fbe507a272c1037c3dce3c178c21f85ab1096ed3372ad9d4b3c7aa27d89e5bf80c9a6260ea652e0268be0cbe61d6a4fcb3add569fa38076d32287
ab4efad51b test: fix immediate tx relay in wallet_groups.py (Sebastian Falbesoner)
Pull request description:
In the functional test wallet_groups.py we whitelist peers on all nodes (`-whitelist=noban@127.0.0.1`) to enable immediate tx relay for fast mempool synchronization. However, considering that this setting only applies to inbound peers and the default test topology looks like this:
```
node0 <--- node1 <---- node2 <--- ... <-- nodeN
```
txs propagate fast only from lower- to higher-numbered nodes (i.e. "left to right" in the above diagram) and take long from higher- to lower-numbered nodes ("right to left") since in the latter direction we only have outbound peers, where the trickle relay is still active. As a consequence, if a tx is submitted from any node other than node0, the mempool synchronization can take quite long.
This PR fixes this by simply adding another connection from node0 to the last node, leading to a ~2-3x speedup (5 runs measured via `time ./test/functional/wallet_groups.py` are shown):
```
master:
0m53.31s real 0m08.22s user 0m05.60s system
0m32.85s real 0m07.44s user 0m04.08s system
0m46.40s real 0m09.18s user 0m04.23s system
0m46.96s real 0m11.10s user 0m05.74s system
0m57.23s real 0m10.53s user 0m05.59s system
PR:
0m19.64s real 0m09.58s user 0m05.50s system
0m18.05s real 0m07.77s user 0m04.03s system
0m18.99s real 0m07.90s user 0m04.25s system
0m17.49s real 0m07.56s user 0m03.92s system
0m18.11s real 0m07.74s user 0m03.88s system
```
Note that in most tests this is not a problem since txs very often originate from node0.
ACKs for top commit:
brunoerg:
utACK ab4efad51b
Tree-SHA512: 12675357e6eb5a18383f2bfe719a184c0790863b37a98749d8e757dd5dc3a36212e16a81f0a192340c11b793eda00db359c7011f46f7c27e3a093af4f5b62147
This is a "dumb" way of randomly generating a Miniscript node from
fuzzer input. It defines a strict binary encoding and will always generate
a node defined from the encoding without "helping" to create valid nodes.
It will cut through as soon as it encounters an invalid fragment so
hopefully the fuzzer can tend to learn the encoding and generate valid
nodes with a higher probability.
On a valid generated node a number of invariants are checked, especially
around the satisfactions and testing them against the Script
interpreter.
The node generation and testing is modular in order to later introduce
other ways to generate nodes from fuzzer inputs with minimal code.
Co-Authored-By: Pieter Wuille <pieter@wuille.net>
We'll need a better integration of the hash preimages PSBT fields to
satisfy Miniscript with such challenges from the RPC.
Thanks to Greg Sanders for his examples and suggestions to improve this
test.
This is a workaround for Miniscript descriptors containing hash
challenges. For those we can't mock the signature creator without making
OP_EQUAL mockable in the interpreter, so CalculateMaximumInputSize will
always return -1 and outputs for these descriptors would appear
unsolvable while they actually are.
Try to solve a script using the Miniscript satisfier if the legacy
solver fails under P2WSH context. Only solve public key and public key
hash challenges for now.
We don't entirely replace the raw solver and especially rule out trying to
solve CHECKMULTISIG-based multisigs with the Miniscript satisfier since
some features, such as the transaction input combiner, rely on the
specific behaviour of the former.
Cherry-picked and squashed from
https://github.com/sipa/bitcoin/commits/202302_miniscript_improve.
- Explain thresh() and multi() satisfaction algorithms
- Comment on and_v dissatisfaction
- Mark overcomplete thresh() dissats as malleable and explain
- Add comment on unnecessity of Malleable() in and_b dissat
When an encrypted wallet is locked (for instance via the
RPC `walletlock`), the docs indicate that the key is
removed from memory. However, the vector (with a secure
allocator) is merely cleared. This allows the key to persist
indefinitely in memory. Instead, manually fill the bytes with
zeroes before clearing.
To prepare for the removal of laanwj's key from trusted key, the trusted
git root needs to be newer than the most recent merge commit signed by
his key.
2d955ff006 net: add `Ensure{any}Banman` (brunoerg)
Pull request description:
This PR adds `Ensure{any}Banman` functions to avoid code repetition and make it cleaner. Same approach as done with argsman, chainman, connman and others.
ACKs for top commit:
davidgumberg:
ACK [2d955ff](2d955ff006)
Tree-SHA512: 0beb7125312168a3df130c1793a1412ab423ef0f46023bfe2a121630c79df7e55d3d143fcf053bd09e2d96e9385a7a04594635da3e5c6be0c5d3a9cafbe3b631
Use DBParams struct to remove ArgsManager uses from txdb.
To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in chainstate.cpp. But these moves are temporary. The
gArgs references in chainstate.cpp are moved out to calling code in init.cpp in
later commits.
This commit does not change behavior.
Add CoinsViewOptions struct to remove ArgsManager uses from txdb.
To reduce size of this commit, this moves references to gArgs variable out of
txdb.cpp to calling code in validation.cpp. But these moves are temporary. The
gArgs references in validation.cpp are moved out to calling code in init.cpp in
later commits.
This commit does not change behavior.
Add DBParams and DBOptions structs to remove ArgsManager uses from dbwrapper.
To reduce size of this commit, this moves references to gArgs variable out of
dbwrapper.cpp to calling code in txdb.cpp. But these moves are temporary. The
gArgs references in txdb.cpp are moved out to calling code in init.cpp in later
commits.
This commit does not change behavior.
fa6f67837b test: Fix intermittent sync issue in wallet_pruning (721217.xyz)
Pull request description:
Setting the mocktime on each loop iteration will make net processing racy and cause a disconnect due to timeout.
Fix that by setting the mocktime only once.
Fixes https://github.com/bitcoin/bitcoin/issues/27065
ACKs for top commit:
brunoerg:
crACK fa6f67837b
Tree-SHA512: 128b962c05a6fa3caf3ce392e870fff6609ce2206a43bbae6661ecb45291df93bed77fe362a514d4472056f83fb6631df39a5170fa34e41a7577b9685dd26b1f
it adds `Ensure{any}Banman` functions to avoid
code repetition and make it cleaner. Similar
approach as done with argsman, chainman, connman
and others.
4de02def84 qt: Persist Mask Values option (Andrew Chow)
Pull request description:
The mask values option is memory only. If a user has enabled this option, it's reasonable to expect that they would want to have it enabled on the next start.
ACKs for top commit:
RandyMcMillan:
tACK 4de02def84
jarolrod:
tACK 4de02def84
pablomartin4btc:
> tACK [4de02de](4de02def84)
john-moffett:
tACK 4de02def84
Tree-SHA512: 247deb78df4911516625bf8b25d752feb480ce30eb31335cf9baeb07b7c6c225fcc37d5c45de62d6e6895ec10c7eefabb15527e3c9723a3b8ddda1e12ebbf46b
73a3b161b7 ci: Inline `MACOS_NATIVE_TASK_TEMPLATE` (Hennadii Stepanov)
8a61527cf6 ci: Re-introduce `depends_built` cache back in macOS and Android tasks (Hennadii Stepanov)
Pull request description:
This PR brings a `depends_built` cache back to the "macOS 10.15" and "ARM64 Android APK" CI tasks.
Fixes#27031.
ACKs for top commit:
MarcoFalke:
reACK 73a3b161b7🌻
Tree-SHA512: 2eb845f865ee2ee453c1fd284d5eeddbebb2653586b17946822fec03d46e73c5eb483499761a0de6c3c466b06623957664e22dee01f7312ad18e212f1c9c6439
887bb53b67 ci: Use the latest Ubuntu LTS for "ARM64 Android APK" task (Hennadii Stepanov)
Pull request description:
Suggested in https://github.com/bitcoin/bitcoin/pull/25797#discussion_r1100172227:
> I don't expect that anyone is building for android, and if they did, it should be fine to just require the latest Ubuntu LTS, which is Jammy
ACKs for top commit:
fanquake:
ACK 887bb53b67 - but I'd also suggest we remove this task entirely, and either replacing it with another task, maybe a *BSD, or delegating the resources to other jobs.
Tree-SHA512: 1f4b6155e5bbb8ca3580809c5999e3abf6b15b409d164a719b0a89205ca48c178aa6401039a22151ce464009adc48ba272a5a2ff05dc3ca06d3b2d64c99e3e22
b03a98291b build: set boost cppflags with --enable-fuzz (fanquake)
Pull request description:
Even though all other targets are disabled, we still need Boost CPPFLAGS (`use_boost`) to compile. This currently works everywhere, except on arm macOS (where the include path is non-standard), because generally, the Boost include path is generic, i.e `/usr/include`.
ACKs for top commit:
hebasto:
ACK b03a98291b
Tree-SHA512: 7544a903ce641fd4b994ae51423a7007de85628ae29be36362a5cbdd62f9b16ac0a62e9edadaaa998ad4c1e82c0fde0d8c53aba41f94ad30ffa9f10ba0984521
Even though all other targets are disabled, we still need Boost CPPFLAGS
(use_boost) to compile. This currently works everywhere, except on arm
macOS (where the include path is pretty non-standard), because
generally, the Boost include path is generic, i.e `/usr/include`.
0e02f72548 depends: define `__BSD_VISIBLE` for FreeBSD bdb build (fanquake)
Pull request description:
Required for additional definitions (`IPC_R` & friends), to be available, when compiling under C11, which would otherwise cause compile fails.
See: https://github.com/MarcoFalke/btc_nightly/pull/4.
ACKs for top commit:
hebasto:
ACK 0e02f72548, tested on FreeBSD 13.1:
Tree-SHA512: 885d4aa341d9668da360cf6dfafb97ce816803c54e76c0a06e448db39a723666d42cd14b3e713d17ecbe33163f5af69924567cf449d679a2db95b36357005d43
fa83005a26 doc: Document affected gcc versions for -fstack-reuse=none workaround (MarcoFalke)
Pull request description:
gcc version(s) 11 and prior won't be fixed, looking at the activity in the bug report. So it seems best to just document gcc 12.1+ as fixed, so that in the future the workaround can be removed once the minimum compiler is gcc12.1.
ACKs for top commit:
fanquake:
ACK fa83005a26
hebasto:
re-ACK fa83005a26
Tree-SHA512: a19723457eb1925196828a5fafd4e7f75a04f86ffae63cb86679d732c662fd1a22e17fe3c69195a97438ff189ba3ff681be3650cf99aa195d7a3e89cd8ee137c