e9d60af988 refactor: Replace init retry for loop with if statement (TheCharlatan)
c1d8870ea4 refactor: Move most of init retry for loop to a function (TheCharlatan)
781c01f580 init: Check mempool arguments in AppInitParameterInteractions (TheCharlatan)
Pull request description:
The for loop around the chain loading logic in `init.cpp` allows users of the GUI to retry once on failure with reindexing without having to manually set the reindex flag on startup. However this current mechanism has problems:
* It is badly documented and has led to confusion among developers and bugs making it into master. Examples:
* https://github.com/bitcoin/bitcoin/pull/28830/files#r1598392660
* https://github.com/bitcoin/bitcoin/pull/30132#issuecomment-2120741121
* It can only ever iterate once, making the choice of a for loop questionable.
* With its large scope it is easy for re-entry bugs to sneak in. Example:
* https://github.com/bitcoin/bitcoin/pull/28830#discussion_r1601589963
Attempt to fix this by moving the bulk of the logic into a separate function and replacing the for loop with a simpler `if` statement.
The diff's in this pull request are best reviewed with `--color-moved-ws=ignore-all-space --color-moved=dimmed-zebra`. The error behaviour can be tested by either manually making `LoadChainstate` return a failure, or deleting some of the block index database files.
ACKs for top commit:
maflcko:
review ACK e9d60af988🚸
josibake:
crACK e9d60af988
achow101:
ACK e9d60af988
ryanofsky:
Code review ACK e9d60af988. Nice change to make AppInitMain shorter and more understandable.
Tree-SHA512: 5e5c0a5fd1b32225346450f8482f0ae8792e1557cdab1518112c1a3ec3a4400b64f5796692245cc5bf2f9010bb97b3a9558f07626a285ccd6ae525dd671ead13
5c7cacf649 ci: Remove natpmp build option and libnatpmp dependency (laanwj)
7e7ec984da doc: Remove mention of natpmp build options (laanwj)
061c3e32a2 depends: Drop natpmp and associated option from depends (laanwj)
20a18bf6aa build: Drop libnatpmp from build system (laanwj)
7b04709862 qt: Changes for built-in PCP+NAT-PMP (laanwj)
52f8ef66c6 net: Replace libnatpmp with built-in NATPMP+PCP implementation in mapport (laanwj)
97c97177cd net: Add PCP and NATPMP implementation (laanwj)
d72df63d16 net: Use GetLocalAddresses in Discover (laanwj)
e02030432b net: Add netif utility (laanwj)
754e425438 crypto: Add missing WriteBE16 function (laanwj)
Pull request description:
Continues #30005. Closes #17012..
This PR adds PCP (Port Control Protocol) from [RFC6887](https://datatracker.ietf.org/doc/html/rfc6887). This adds, in addition to the existing IPv4 port mapping (which now uses PCP, with fallback to NAT-PMP), support for IPv6 pinholing-that is, opening a port on the firewall to make it reachable.
PCP, like NAT-PMP is a simple UDP-based protocol, and the implementation is self-contained, so this gets rid of lthe libnatpnp dependency without adding a new one. It should otherwise be a drop-in replacement. NAT-PMP fallback is implemented so this will not make router support worse.
For now it is disabled by default, though in the future (not in this PR) we could consider enable it by default to increase the number of connectable nodes without adding significant attack surface.
To test:
```bash
bitcoind -regtest -natpmp=1 -debug=net
```
(most of the changes in this PR are, ironically, removing the libnatpmp dependency and associated build system and build docs)
## TODO
- [x] Default gateway discovery on Linux / FreeBSD
- [x] Default gateway discovery on Windows
- [x] Default gateway discovery on MacOS
- [x] Either solve FreeBSD compile issue (probably upstream issue) or remove FreeBSD support
## Things to consider for follow-up PRs
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658764974 avoid unreachable nets (not given to -onlynet=)
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658949236 could announce an addr:port where we do not listen (no -bind)
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1684368824 could announce the wrong port because it uses GetListenPort()
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1679709347 if we requested one port but another was assigned, then which one to use in the renewal?
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1772017020 Use `GetAdapterAddresses` to discover local addresses for Windows
ACKs for top commit:
Sjors:
ACK 5c7cacf649
achow101:
ACK 5c7cacf649
vasild:
ACK 5c7cacf649
Tree-SHA512: e35b69e56d5f5449a3d48a799f9b7b65107c65eeb3e245c2c1e9d42221e469ca5ead90afae423160601cd664dd553a51c859e04f4492f335b064aae3bf23e3bc
f1daa80521 guix: Drop no longer needed `PATH` modification (Hennadii Stepanov)
Pull request description:
I don't see any reason why this would be necessary in the master branch @ d812cf1189.
Additionally, from https://github.com/bitcoin/bitcoin/pull/30940#pullrequestreview-2322355196:
> 2. I don't understand why "In the Guix environment, `${BASEPREFIX}/${HOST}/native/bin` is added to the `PATH` environment variable," according to the description. Setting this seems indiscriminate, like a sledgehammer approach, something that would cause the guix build to behave differently from normal depends builds and lead to confusing issues like this one.
My Guix build:
```
aarch64
015b853d60c742120b88f1501ce241c8b7b3e874eca9ab150ba2ec282ecb9572 guix-build-f1daa80521ec/output/aarch64-linux-gnu/SHA256SUMS.part
2a8ed51f02046a73dc9a391b8939528c2e506d545274c934202a5643f26b102b guix-build-f1daa80521ec/output/aarch64-linux-gnu/bitcoin-f1daa80521ec-aarch64-linux-gnu-debug.tar.gz
0ce7a6c81b657cfcbd2edf1e18cca8f66bd7bbe15a12b90dd60ddb1218b72254 guix-build-f1daa80521ec/output/aarch64-linux-gnu/bitcoin-f1daa80521ec-aarch64-linux-gnu.tar.gz
de6cb71e37a1c2e9a9a9952d4456a7fde407b38f95a1447928ded3f592b2e47f guix-build-f1daa80521ec/output/arm-linux-gnueabihf/SHA256SUMS.part
c91be594ad4d02a2cb4cea2f57e91ebeae9a1cda66ec49e05ecc3a793e767f24 guix-build-f1daa80521ec/output/arm-linux-gnueabihf/bitcoin-f1daa80521ec-arm-linux-gnueabihf-debug.tar.gz
eb8ea448df1734009129d88cdf28a1ae5918bff19a58fa9525c0b1dde0dfd987 guix-build-f1daa80521ec/output/arm-linux-gnueabihf/bitcoin-f1daa80521ec-arm-linux-gnueabihf.tar.gz
6d558c036b66c81fb5843b1918f24fec6bd901098a0dfb15100b497e12e8fdc3 guix-build-f1daa80521ec/output/arm64-apple-darwin/SHA256SUMS.part
23691ecaf5d23c72f06fe81054a84e2549d8e89582317b6d3e14276aeba0b07f guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin-unsigned.tar.gz
8965a32937894d6dd75e6b04809bdc925187967c2547a795dec2e11a75262624 guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin-unsigned.zip
ec0b2f35f498537ca6eb8b306a1e26cf97b7f1bdf140f3c4ca8b18c643fc4599 guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin.tar.gz
d46d8117efdbfe90be13bcf36ba2ddcfa7c53ba01762a53c72a1b48f2cac569c guix-build-f1daa80521ec/output/dist-archive/bitcoin-f1daa80521ec.tar.gz
facf7bbec0e9324e9ed58b8da07c5b1df2f120bd9090f7d124613ed62092dd46 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/SHA256SUMS.part
c065b222f60ec19b7585daf197dadcb529fa588de1b26e767e4ebd43d6345562 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/bitcoin-f1daa80521ec-powerpc64-linux-gnu-debug.tar.gz
4e837a86ce6adbd595dc31d2b584c3322acd30b6f18b57144f18fc09289fec65 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/bitcoin-f1daa80521ec-powerpc64-linux-gnu.tar.gz
f4362984a846e97c6a388366bb2922294c66bb3f78adf71064f97ab5a346e4ed guix-build-f1daa80521ec/output/riscv64-linux-gnu/SHA256SUMS.part
427fc7fdac244c6dd4fdf0312486e3bcf8372c68fd3570bdb815734544b8369e guix-build-f1daa80521ec/output/riscv64-linux-gnu/bitcoin-f1daa80521ec-riscv64-linux-gnu-debug.tar.gz
ae9a07f7e2e656efbba99246be5767798028c13fcf5d172a595b734f5e1241c4 guix-build-f1daa80521ec/output/riscv64-linux-gnu/bitcoin-f1daa80521ec-riscv64-linux-gnu.tar.gz
8ff7494e648fe5744efd4522a003d94b531dcab28cb8c2fea05a09897be111ce guix-build-f1daa80521ec/output/x86_64-apple-darwin/SHA256SUMS.part
9845e894fc6b0dd339dc4f62f3bc4e37f76935f309887798ca488fb5465b2b6c guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin-unsigned.tar.gz
fa0e07573ae977ef6bb3ecaa07b1e434c52041865e2def9de6a041fb3749d27d guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin-unsigned.zip
cd99cda53a8fbcc5380333058426055977cd39d3bdc0da571b3f64d293787719 guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin.tar.gz
a6beac93eb8f9516a13ab7451b0c45b2898fd56315a066cc6470ba84226bba27 guix-build-f1daa80521ec/output/x86_64-linux-gnu/SHA256SUMS.part
f50e03971274371ef0ec5710de4879670f75cb29a8eacd5c02f0d622740d026a guix-build-f1daa80521ec/output/x86_64-linux-gnu/bitcoin-f1daa80521ec-x86_64-linux-gnu-debug.tar.gz
97dcd833014cccaac1b228f438ac49aec94603f0317c606e9344d9709302dbbd guix-build-f1daa80521ec/output/x86_64-linux-gnu/bitcoin-f1daa80521ec-x86_64-linux-gnu.tar.gz
7c2ea5572f9f137523b88f6a0f1ac711abd6a7ef8aa361ceea35d01e700a3778 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/SHA256SUMS.part
c64d33e04dfc8adfe5a48d6ed17579a69e0b8938e2973bd1810bcaefe5dc9506 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-debug.zip
87d81e11510ffef0082e3be80ce3f8f5e7d9f5c3cdb1dae887d4341cf678af31 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-setup-unsigned.exe
d46887ef5d23fe19ce23dd356dc3a1c03a1164778f78466b4ef415038b42e3eb guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-unsigned.tar.gz
c1a54433d0849548734e8962590e3a33b529665cd610f2ed5acbb1a52c02ae23 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64.zip
```
ACKs for top commit:
theuni:
utACK f1daa80521 since guix is happy.
fanquake:
ACK f1daa80521
Tree-SHA512: 50fd8fb01727a462e3935ad840de465acee9520eb5e9cfd972476960e6f738a8fd7e9cb62f27cdad643d013e5b487df1671c45f46af2476aaeeec21cfa60e6c1
The `QWARN()` macro internally uses `QTest::qWarn()`, which has been
deprecated since Qt 6.3. Replacing it with `qWarning()` ensures
compatibility across all Qt versions.
The weight unit is merely a consensus rule detail and is largely
irrelevant for fee-rate calculations and mempool policy rules (e.g. for
package relay and TRUC limits), so there doesn't seem to be any value of
using a granularity that we can't even guarantee to reach exactly
anyway.
Switch to the more natural unit of vsize instead, which simplifies both
the padding implementation and the current tests that take use of this
padding. The rather annoying multiplications by `WITNESS_SCALE_FACTOR`
can then be removed and weird-looking magic numbers like `4004` can be
replaced by numbers that are more connected to actual policy limit
constants from the codebase, e.g. `1001` for exceeding
`TRUC_CHILD_MAX_VSIZE` by one.
f5a2000579 test: re-bucket long-running tests (willcl-ark)
Pull request description:
Re-bucket:
- `p2p_node_network_limited -v*transport`
- `feature_assume_utxo`
On CI runners these tests are taking longer than their current bucket suggests, often being among the last to finish.
Re-bucket them to improve CI efficiency.
ACKs for top commit:
maflcko:
review ACK f5a2000579
Tree-SHA512: 3da5c888db64a311276338270ba1dcad3eb2a24e205f6bb86fc92f767ecfa63682f13fafffff569fa0cfaea607ccb538f31e3934a086d482c3fe1be5d39f8791
6c3c619b35 test: generalize HasReason and use it in FailFmtWithError (Lőrinc)
Pull request description:
Standardized boost exception checking in recent tests introduced in https://github.com/bitcoin/bitcoin/pull/30546#discussion_r1756493521 by extending `HasReason` to accept `const char*` through `string_view` in `operator()`.
Note that `HasReason` only checks partial matches - but since we're specifying the whole error string, it doesn't affect us in this case.
ACKs for top commit:
maflcko:
review ACK 6c3c619b35
hodlinator:
ACK 6c3c619b35
Tree-SHA512: 740fb18b8fea78e4eb9740ceb0fe75d37246c28cfa2638b9d093e9514dd6d7926cc5be9ec57f8027cca3aa9d616e8c54322d2401cfa67fd25282f7816e63532d
fafd1a0f64 ci: Inline PACKAGE_MANAGER_INSTALL (MarcoFalke)
Pull request description:
The fallback `bash -c "$PACKAGE_MANAGER_INSTALL git"` is only needed by the `lint` task, so simplify it and inline `PACKAGE_MANAGER_INSTALL` once. Also, fixup the docs to add some other packages which are needed by podman in user-mode.
ACKs for top commit:
fanquake:
ACK fafd1a0f64
Tree-SHA512: e1665446d0fb5a2e8f2cb912117c7a42838c840199e7504a859b9155a13a2ff2e8606ac56689446f973fce02b00240041f071ebf00520778ed81eb1a01de6663
f158993fd5 contrib: Update asmap link in seeds readme (Fabian Jahr)
Pull request description:
I am moving all my ASMap related repositories to an asmap org. While there is a redirect in place that works for now, GitHub doesn't guarantee that it will keep working in the long term. So we should still fix the links.
ACKs for top commit:
laanwj:
ACK f158993fd5
jarolrod:
ACK f158993fd5
brunoerg:
ACK f158993fd5
Tree-SHA512: c6f831b6ed1b520a38623a7d29c01677a45bf06fea77f4da6b421dd16e630836c9abf8191330e3887232fd688edca534067142d37d7e504a1a785ae451b11721
f951f1fab2 ci: add timestamps to cirrus jobs (willcl-ark)
Pull request description:
Currently, debugging where time is spent in the cirrus jobs feels annoying, e.g. trying to see where time may be spent in https://github.com/bitcoin/bitcoin/issues/30969
Enable timestamps in the logs for more information.
ACKs for top commit:
maflcko:
review ACK f951f1fab2
tdb3:
Code Review ACK f951f1fab2
hebasto:
ACK f951f1fab2.
Tree-SHA512: 4ca6c3722013323e794339c39af3041a519e3ecf75216093c4ca99668c0932fe41cc547819f742a3e8a47709ec6baec976c542eb3848c0248f439841110a4e85
faf801515f test: Add missing sync_mempools() to fill_mempool() (MarcoFalke)
fa48be6f02 test: Refactor fill_mempool to extract send_batch helper (MarcoFalke)
Pull request description:
Not doing the sync will lead to (intermittent) issues, as explained in https://github.com/bitcoin/bitcoin/issues/30922#issuecomment-2364529013.
Fix all issues by doing the sync by default and disable it in places that do not need the sync.
Fixes#30922
ACKs for top commit:
mzumsande:
Tested ACK faf801515f
ismaelsadeeq:
Tested ACK faf801515f
marcofleon:
Tested ACK faf801515f
Tree-SHA512: 2de62d168cbb6857a9fb8bc12c42a9093fedf5e9beb6f83a32b3fa72a5ba3cf03631055fd25ef553399a27a6fe0d71c44cfe60660a4d31986debd13b8ab00228
4cf84b344d depends, doc: No need to specify general requirement (Hennadii Stepanov)
Pull request description:
CMake is no longer required solely for `libmultiprocess`.
ACKs for top commit:
maflcko:
review ACK 4cf84b344d
Tree-SHA512: 014808ef523d6f0101aaa52a1858bebc16da9eb44664beb204e271b033c4dfa48115ce85c4ce5f151e82f90f83c64aaa25af73ef5a9b9b64953d2e07ff19633f
The current order is incorrect:
```bash
./build/src/bitcoin-cli loadtxoutset -rpcclienttimeout=0 utxo-840000.dat
error code: -1
error message:
loadtxoutset "path"
```
The Mining interface uses this function in the next commit
to calculate the coinbase merkle path. Stratum v2 uses
this to send a compact work template.
This partially undoes the change in 4defdfab94,
but is not a revert, because the implementation changed in the meantime.
This commit also documents the function.
1a33281766 doc: multiprocess documentation improvements (Ryan Ofsky)
d043950ba2 multiprocess: Add serialization code for BlockValidationState (Ryan Ofsky)
33c2eee285 multiprocess: Add IPC wrapper for Mining interface (Ryan Ofsky)
06882f8401 multiprocess: Add serialization code for vector<char> (Russell Yanofsky)
095286f790 multiprocess: Add serialization code for CTransaction (Russell Yanofsky)
69dfeb1876 multiprocess: update common-types.h to use C++20 concepts (Ryan Ofsky)
206c6e78ee build: Make bitcoin_ipc_test depend on bitcoin_ipc (Ryan Ofsky)
070e6a32d5 depends: Update libmultiprocess library for cmake headers target (Ryan Ofsky)
Pull request description:
Add Cap'n Proto wrapper for the Mining interface introduced in #30200, and its associated types.
This PR combined with #30509 will allow a separate mining process, like the one being implemented in https://github.com/Sjors/bitcoin/pull/48, to connect to the node over IPC, and create, manage, and submit block templates. (#30437 shows another simpler demo of a process using the Mining interface.)
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).
ACKs for top commit:
achow101:
ACK 1a33281766
TheCharlatan:
ACK 1a33281766
itornaza:
ACK 1a33281766
Tree-SHA512: 0791078dd6885dbd81e3d14c75fffff3da8d1277873af379ea6f9633e910c11485bb324e4cde3d936d50d343b16a10b0e8fc1e0fc6d7bdca7f522211da50c01e
c0a0c72b4d validation: Disable CheckForkWarningConditions for background chainstate (Martin Zumsande)
Pull request description:
The comparison of `m_best_invalid` with the tip of the chainstate makes no sense for the background chainstate and can lead to incorrect log messages.
Fixes#30958
ACKs for top commit:
fjahr:
utACK c0a0c72b4d
achow101:
ACK c0a0c72b4d
TheCharlatan:
ACK c0a0c72b4d
tdb3:
CR ACK c0a0c72b4d
Tree-SHA512: 316811b1ade8521681f3a105c7db59dfdb27b1e13efeba95cd6183d750db2e07f48968e8fc54fe3834d3311519cf8c6b77d7e0fd61facbf2b07598e79110ff85
The for loop has been a long standing source of confusion and bugs, both
because its purpose is not clearly documented and because the body of
the for loop contains a lot of logic.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
This makes it clearer which state is being mutated by the function and
facilitates getting rid of the for loop in the following commit. Move
creation of the required options into the function too, such that the
function takes fewer arguments and is more self-contained.
Co-Authored-By: Ryan Ofsky <ryan@ofsky.org>
06e7e83632 doc: correct the zmq automatic build info (tdb3)
Pull request description:
The current documentation states that the ZMQ feature is automatically included when building.
Noticed this was no longer the case when building to review PR #30942.
ACKs for top commit:
l0rinc:
ACK 06e7e83632
hebasto:
re-ACK 06e7e83632.
Tree-SHA512: 24269c29415e54815b105e3f830f31f055e9bdcb3bf4a7cede068651875190adfc4e72ad5448b3c30a23b0db024f743d9f18576a6fda46d87deb5ea39352e626
fa6c1946d2 doc: Adjust links in OSS-Fuzz section (MarcoFalke)
Pull request description:
Adjust the links after the google issue tracker migration and replace the remaining paragraph with a link to https://bitcoincore.org/en/security-advisories/
ACKs for top commit:
dergoegge:
ACK fa6c1946d2
Tree-SHA512: 4c3aba8bb5641756be5c66c2ee21b927a675622ddac021303f8196a1b2d8eb891406f6dedab0f04e12312339953c87302891df4ad5f5956c0b2ddd453452e1c9
The comparison of m_best_invalid with the tip of the respective chainstate
makes no sense for the background chainstate, and can lead to incorrect
error messages.
7bd3ee62f6 test: Use shell builtins in run_command test case (Ava Chow)
Pull request description:
Uses the [suggested command](https://github.com/bitcoin/bitcoin/issues/30938#issuecomment-2363906135)
Fixes#30938
ACKs for top commit:
maflcko:
review ACK 7bd3ee62f6
hebasto:
ACK 7bd3ee62f6.
Tree-SHA512: 683b15cafaf0103eeadf872ea6ce9a7d884b2605d3dcf4e66b0173cdb149c24965e7c5fa62aaddf2ac55df3f449aeb787176992c96cfee5d0b86621259e1dfe9
