fa2b7d8d6b Remove redundant unterminated-logprintf tidy check (MarcoFalke)
bbbb2e43ee log: Enforce trailing newline, Remove redundant m_started_new_line (MarcoFalke)
Pull request description:
There are many problems around missing a trailing newline while logging:
* All log lines are currently terminated by a trailing newline. This means any runtime code trying to handle a "missing" newline is currently dead code.
* Leaving a line unterminated is racy and can cause content corruption by mixing log lines from different sources.
* It requires extra code like `m_started_new_line` to keep track of, which is annoying and pointless to maintain, because it is currently dead code, see https://github.com/bitcoin/bitcoin/pull/30386#discussion_r1684380835.
* It requires a standalone `unterminated-logprintf` clang-tidy plugin, which is unmaintained (no one updated it for the new log function names), probably harder to maintain than normal C++ code (because it requires clang AST matcher knowledge), brittle (it can fail to detect issues at any time, if it goes out-of-sync, or be explicitly disabled via `NOLINT`), and annoying for devs (it is slow and intricate to run locally and thus only effectively run on CI or via the CI scripts).
Fix all issues by enforcing the trailing newline in logs directly in the code. Then remove all the other stuff.
This refactor does not change behavior.
ACKs for top commit:
stickies-v:
re-ACK fa2b7d8d6b
achow101:
ACK fa2b7d8d6b
ryanofsky:
Code review ACK fa2b7d8d6b. Just comment and test cleanup since last review
Tree-SHA512: 10ed420f6c2fdb0f491d6c880be8dd2e8beef628f510adebadf4c3849d9f5e28906519d5cbaeb295f4c7c1b07c4c88a9905b3cfe30fee3a2c91ac9fd24ae6755
fd38711217 ci: make CI job fail when check-deps.sh script fails (Ryan Ofsky)
d51edecddc common: move pcp.cpp and netif.cpp files from util to common library since they depend on netaddress.cpp (Ryan Ofsky)
Pull request description:
Move util/pcp.cpp and util/netif.cpp to common/ because they depend on netaddress.cpp which is part of the common library. This was causing check-deps.sh script to fail as reported by _fanquake_ in https://github.com/bitcoin/bitcoin/pull/30415#issuecomment-2385475097.
Also make CI fail when the `check-deps.sh` script fails. Previously it would output errors but not cause the job to fail (which was not intended).
ACKs for top commit:
Sjors:
utACK fd38711217
laanwj:
Untested ACK fd38711217
achow101:
ACK fd38711217
tdb3:
ACK fd38711217
Tree-SHA512: 06316e68617ded7d96d540c9934b08cf9fbba5ff5e7f54d7a0c0a9087a26bf8adc97e9e8c39a2bfd3da34e27f3652b1531ec6136a2c69393ae0b26585abadb6b
a1576edab3 test: add missing sync to feature_fee_estimation.py (Martin Zumsande)
Pull request description:
This fixes a race:
- In the `test_estimate_dat_is_flushed_periodically` subtest, node 0 is isolated and creates 10 blocks (no sync).
- In `clear_estimates` the nodes are reconnected (but we don't wait for them to sync!)
- In the `sanity_check_rbf_estimates` subtest, node 1 generates another block and syncs with the other nodes. The sync fails if the generated block is at the same height as the tip of node 0.
Fix this by adding a sync to `clear_estimates`.
Fixes#30990Fixes#30640
ACKs for top commit:
maflcko:
lgtm ACK a1576edab3
tdb3:
code review ACK a1576edab3
Tree-SHA512: 608ba619cacb4ff3a1ea934e03286f18c96afeebf06439334d40bff72025bd7bcc2c1093dae1824b30a37d3ac3ea569bc3118c33c0ca51610592aa1b4f420840
CMake parses some paths from the spec of the C compiler, assuming it
will be the linker, resulting in the link to end up with
`-L/usr/lib/gcc/x86_64-w64-mingw32/12-win32` on debian bookworm if both
-win32 and -posix variants are installed, and -win32 is the default
alternative.
This results in the wrong C++ library being linked, missing
std::threads::hardware_concurrency and other threading functions.
To fix this, use the -posix variant of gcc as well when available. This
fixes a regression compared to autotools, where this scenario worked.
Previously the check-deps.sh would write information about unexpected
dependencies to stderr, but return exit code 0, so the error would be ignored
by CI. Now it will return code 1 and cause CI to fail if unexpected
dependencies are detected.
Instead of having a single NodeContext::shutdown member that is used both to
request shutdowns and check if they have been requested, use separate members
for each. Benefits of this change:
1. Should make code a little clearer and easier to search because it is easier
to see which parts of code are triggering shutdowns and which parts are just
checking to see if they were triggered.
2. Makes it possible for init.cpp to specify additional code to run when a
shutdown is requested, like signalling the m_tip_block_cv condition variable.
Motivation for this change was to remove hacky NodeContext argument and
m_tip_block_cv access from the StopRPC function, so StopRPC can just be
concerned with RPC functionality, not other node functionality.
This is not strictly required, but all places using m_tip_block_cv
(except shutdown) already take the lock. The annotation makes it easier
to catch potential deadlocks before review.
Adding the missing lock to the shutdown sequence is a bugfix.
An alternative would be to take the lock and release it before
notifying, see
https://github.com/bitcoin/bitcoin/pull/30967#discussion_r1778899716
It should be obvious that a wait is not needed if the tip does not
match.
Also, remove a comment that the blockTip notification was only meant for
the "UI". (It is used by other stuff for a long time)
940edd6ac2 test: refactor: introduce and use `TRUC_CHILD_MAX_VSIZE` constant (Sebastian Falbesoner)
c16ae71768 test: switch MiniWallet padding unit from weight to vsize (Sebastian Falbesoner)
Pull request description:
This PR is a late follow-up for #30162, where I retrospectively consider the padding unit of choice as a mistake. The weight unit is merely a consensus rule detail and is largely irrelevant from a user's perspective w.r.t. fee-rate calculations and mempool policy rules (e.g. for package relay and TRUC limits), so there doesn't seem to be any value of using a granularity that we can't even guarantee to reach exactly anyway.
Switch to the more natural unit of vsize instead, which simplifies both the padding implementation (no "round up to the next multiple of 4" anymore) and the current tests that take use of this padding. The rather annoying multiplications by `WITNESS_SCALE_FACTOR` can then be removed and weird-looking magic numbers like `4004` can be replaced by numbers that are more connected to actual policy limit constants from the codebase, e.g. `1001` for exceeding `TRUC_CHILD_MAX_VSIZE` by one. The second commits introduces a constant for that.
ACKs for top commit:
glozow:
reACK 940edd6 via range-diff
instagibbs:
reACK 940edd6ac2
maflcko:
re-ACK 940edd6ac2🍷
achow101:
ACK 940edd6ac2
Tree-SHA512: 35325f22bbe548664273051b705059b8f2f4316215be116c71b8c21dc87d190b3e8fcc4a48f04deaba2f3632a9c809d272b0bae654cf74d7492759554c0f0d14
e9d60af988 refactor: Replace init retry for loop with if statement (TheCharlatan)
c1d8870ea4 refactor: Move most of init retry for loop to a function (TheCharlatan)
781c01f580 init: Check mempool arguments in AppInitParameterInteractions (TheCharlatan)
Pull request description:
The for loop around the chain loading logic in `init.cpp` allows users of the GUI to retry once on failure with reindexing without having to manually set the reindex flag on startup. However this current mechanism has problems:
* It is badly documented and has led to confusion among developers and bugs making it into master. Examples:
* https://github.com/bitcoin/bitcoin/pull/28830/files#r1598392660
* https://github.com/bitcoin/bitcoin/pull/30132#issuecomment-2120741121
* It can only ever iterate once, making the choice of a for loop questionable.
* With its large scope it is easy for re-entry bugs to sneak in. Example:
* https://github.com/bitcoin/bitcoin/pull/28830#discussion_r1601589963
Attempt to fix this by moving the bulk of the logic into a separate function and replacing the for loop with a simpler `if` statement.
The diff's in this pull request are best reviewed with `--color-moved-ws=ignore-all-space --color-moved=dimmed-zebra`. The error behaviour can be tested by either manually making `LoadChainstate` return a failure, or deleting some of the block index database files.
ACKs for top commit:
maflcko:
review ACK e9d60af988🚸
josibake:
crACK e9d60af988
achow101:
ACK e9d60af988
ryanofsky:
Code review ACK e9d60af988. Nice change to make AppInitMain shorter and more understandable.
Tree-SHA512: 5e5c0a5fd1b32225346450f8482f0ae8792e1557cdab1518112c1a3ec3a4400b64f5796692245cc5bf2f9010bb97b3a9558f07626a285ccd6ae525dd671ead13
5c7cacf649 ci: Remove natpmp build option and libnatpmp dependency (laanwj)
7e7ec984da doc: Remove mention of natpmp build options (laanwj)
061c3e32a2 depends: Drop natpmp and associated option from depends (laanwj)
20a18bf6aa build: Drop libnatpmp from build system (laanwj)
7b04709862 qt: Changes for built-in PCP+NAT-PMP (laanwj)
52f8ef66c6 net: Replace libnatpmp with built-in NATPMP+PCP implementation in mapport (laanwj)
97c97177cd net: Add PCP and NATPMP implementation (laanwj)
d72df63d16 net: Use GetLocalAddresses in Discover (laanwj)
e02030432b net: Add netif utility (laanwj)
754e425438 crypto: Add missing WriteBE16 function (laanwj)
Pull request description:
Continues #30005. Closes #17012..
This PR adds PCP (Port Control Protocol) from [RFC6887](https://datatracker.ietf.org/doc/html/rfc6887). This adds, in addition to the existing IPv4 port mapping (which now uses PCP, with fallback to NAT-PMP), support for IPv6 pinholing-that is, opening a port on the firewall to make it reachable.
PCP, like NAT-PMP is a simple UDP-based protocol, and the implementation is self-contained, so this gets rid of lthe libnatpnp dependency without adding a new one. It should otherwise be a drop-in replacement. NAT-PMP fallback is implemented so this will not make router support worse.
For now it is disabled by default, though in the future (not in this PR) we could consider enable it by default to increase the number of connectable nodes without adding significant attack surface.
To test:
```bash
bitcoind -regtest -natpmp=1 -debug=net
```
(most of the changes in this PR are, ironically, removing the libnatpmp dependency and associated build system and build docs)
## TODO
- [x] Default gateway discovery on Linux / FreeBSD
- [x] Default gateway discovery on Windows
- [x] Default gateway discovery on MacOS
- [x] Either solve FreeBSD compile issue (probably upstream issue) or remove FreeBSD support
## Things to consider for follow-up PRs
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658764974 avoid unreachable nets (not given to -onlynet=)
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658949236 could announce an addr:port where we do not listen (no -bind)
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1684368824 could announce the wrong port because it uses GetListenPort()
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1679709347 if we requested one port but another was assigned, then which one to use in the renewal?
- https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1772017020 Use `GetAdapterAddresses` to discover local addresses for Windows
ACKs for top commit:
Sjors:
ACK 5c7cacf649
achow101:
ACK 5c7cacf649
vasild:
ACK 5c7cacf649
Tree-SHA512: e35b69e56d5f5449a3d48a799f9b7b65107c65eeb3e245c2c1e9d42221e469ca5ead90afae423160601cd664dd553a51c859e04f4492f335b064aae3bf23e3bc
f1daa80521 guix: Drop no longer needed `PATH` modification (Hennadii Stepanov)
Pull request description:
I don't see any reason why this would be necessary in the master branch @ d812cf1189.
Additionally, from https://github.com/bitcoin/bitcoin/pull/30940#pullrequestreview-2322355196:
> 2. I don't understand why "In the Guix environment, `${BASEPREFIX}/${HOST}/native/bin` is added to the `PATH` environment variable," according to the description. Setting this seems indiscriminate, like a sledgehammer approach, something that would cause the guix build to behave differently from normal depends builds and lead to confusing issues like this one.
My Guix build:
```
aarch64
015b853d60c742120b88f1501ce241c8b7b3e874eca9ab150ba2ec282ecb9572 guix-build-f1daa80521ec/output/aarch64-linux-gnu/SHA256SUMS.part
2a8ed51f02046a73dc9a391b8939528c2e506d545274c934202a5643f26b102b guix-build-f1daa80521ec/output/aarch64-linux-gnu/bitcoin-f1daa80521ec-aarch64-linux-gnu-debug.tar.gz
0ce7a6c81b657cfcbd2edf1e18cca8f66bd7bbe15a12b90dd60ddb1218b72254 guix-build-f1daa80521ec/output/aarch64-linux-gnu/bitcoin-f1daa80521ec-aarch64-linux-gnu.tar.gz
de6cb71e37a1c2e9a9a9952d4456a7fde407b38f95a1447928ded3f592b2e47f guix-build-f1daa80521ec/output/arm-linux-gnueabihf/SHA256SUMS.part
c91be594ad4d02a2cb4cea2f57e91ebeae9a1cda66ec49e05ecc3a793e767f24 guix-build-f1daa80521ec/output/arm-linux-gnueabihf/bitcoin-f1daa80521ec-arm-linux-gnueabihf-debug.tar.gz
eb8ea448df1734009129d88cdf28a1ae5918bff19a58fa9525c0b1dde0dfd987 guix-build-f1daa80521ec/output/arm-linux-gnueabihf/bitcoin-f1daa80521ec-arm-linux-gnueabihf.tar.gz
6d558c036b66c81fb5843b1918f24fec6bd901098a0dfb15100b497e12e8fdc3 guix-build-f1daa80521ec/output/arm64-apple-darwin/SHA256SUMS.part
23691ecaf5d23c72f06fe81054a84e2549d8e89582317b6d3e14276aeba0b07f guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin-unsigned.tar.gz
8965a32937894d6dd75e6b04809bdc925187967c2547a795dec2e11a75262624 guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin-unsigned.zip
ec0b2f35f498537ca6eb8b306a1e26cf97b7f1bdf140f3c4ca8b18c643fc4599 guix-build-f1daa80521ec/output/arm64-apple-darwin/bitcoin-f1daa80521ec-arm64-apple-darwin.tar.gz
d46d8117efdbfe90be13bcf36ba2ddcfa7c53ba01762a53c72a1b48f2cac569c guix-build-f1daa80521ec/output/dist-archive/bitcoin-f1daa80521ec.tar.gz
facf7bbec0e9324e9ed58b8da07c5b1df2f120bd9090f7d124613ed62092dd46 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/SHA256SUMS.part
c065b222f60ec19b7585daf197dadcb529fa588de1b26e767e4ebd43d6345562 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/bitcoin-f1daa80521ec-powerpc64-linux-gnu-debug.tar.gz
4e837a86ce6adbd595dc31d2b584c3322acd30b6f18b57144f18fc09289fec65 guix-build-f1daa80521ec/output/powerpc64-linux-gnu/bitcoin-f1daa80521ec-powerpc64-linux-gnu.tar.gz
f4362984a846e97c6a388366bb2922294c66bb3f78adf71064f97ab5a346e4ed guix-build-f1daa80521ec/output/riscv64-linux-gnu/SHA256SUMS.part
427fc7fdac244c6dd4fdf0312486e3bcf8372c68fd3570bdb815734544b8369e guix-build-f1daa80521ec/output/riscv64-linux-gnu/bitcoin-f1daa80521ec-riscv64-linux-gnu-debug.tar.gz
ae9a07f7e2e656efbba99246be5767798028c13fcf5d172a595b734f5e1241c4 guix-build-f1daa80521ec/output/riscv64-linux-gnu/bitcoin-f1daa80521ec-riscv64-linux-gnu.tar.gz
8ff7494e648fe5744efd4522a003d94b531dcab28cb8c2fea05a09897be111ce guix-build-f1daa80521ec/output/x86_64-apple-darwin/SHA256SUMS.part
9845e894fc6b0dd339dc4f62f3bc4e37f76935f309887798ca488fb5465b2b6c guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin-unsigned.tar.gz
fa0e07573ae977ef6bb3ecaa07b1e434c52041865e2def9de6a041fb3749d27d guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin-unsigned.zip
cd99cda53a8fbcc5380333058426055977cd39d3bdc0da571b3f64d293787719 guix-build-f1daa80521ec/output/x86_64-apple-darwin/bitcoin-f1daa80521ec-x86_64-apple-darwin.tar.gz
a6beac93eb8f9516a13ab7451b0c45b2898fd56315a066cc6470ba84226bba27 guix-build-f1daa80521ec/output/x86_64-linux-gnu/SHA256SUMS.part
f50e03971274371ef0ec5710de4879670f75cb29a8eacd5c02f0d622740d026a guix-build-f1daa80521ec/output/x86_64-linux-gnu/bitcoin-f1daa80521ec-x86_64-linux-gnu-debug.tar.gz
97dcd833014cccaac1b228f438ac49aec94603f0317c606e9344d9709302dbbd guix-build-f1daa80521ec/output/x86_64-linux-gnu/bitcoin-f1daa80521ec-x86_64-linux-gnu.tar.gz
7c2ea5572f9f137523b88f6a0f1ac711abd6a7ef8aa361ceea35d01e700a3778 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/SHA256SUMS.part
c64d33e04dfc8adfe5a48d6ed17579a69e0b8938e2973bd1810bcaefe5dc9506 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-debug.zip
87d81e11510ffef0082e3be80ce3f8f5e7d9f5c3cdb1dae887d4341cf678af31 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-setup-unsigned.exe
d46887ef5d23fe19ce23dd356dc3a1c03a1164778f78466b4ef415038b42e3eb guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64-unsigned.tar.gz
c1a54433d0849548734e8962590e3a33b529665cd610f2ed5acbb1a52c02ae23 guix-build-f1daa80521ec/output/x86_64-w64-mingw32/bitcoin-f1daa80521ec-win64.zip
```
ACKs for top commit:
theuni:
utACK f1daa80521 since guix is happy.
fanquake:
ACK f1daa80521
Tree-SHA512: 50fd8fb01727a462e3935ad840de465acee9520eb5e9cfd972476960e6f738a8fd7e9cb62f27cdad643d013e5b487df1671c45f46af2476aaeeec21cfa60e6c1
The `QWARN()` macro internally uses `QTest::qWarn()`, which has been
deprecated since Qt 6.3. Replacing it with `qWarning()` ensures
compatibility across all Qt versions.
The weight unit is merely a consensus rule detail and is largely
irrelevant for fee-rate calculations and mempool policy rules (e.g. for
package relay and TRUC limits), so there doesn't seem to be any value of
using a granularity that we can't even guarantee to reach exactly
anyway.
Switch to the more natural unit of vsize instead, which simplifies both
the padding implementation and the current tests that take use of this
padding. The rather annoying multiplications by `WITNESS_SCALE_FACTOR`
can then be removed and weird-looking magic numbers like `4004` can be
replaced by numbers that are more connected to actual policy limit
constants from the codebase, e.g. `1001` for exceeding
`TRUC_CHILD_MAX_VSIZE` by one.
f5a2000579 test: re-bucket long-running tests (willcl-ark)
Pull request description:
Re-bucket:
- `p2p_node_network_limited -v*transport`
- `feature_assume_utxo`
On CI runners these tests are taking longer than their current bucket suggests, often being among the last to finish.
Re-bucket them to improve CI efficiency.
ACKs for top commit:
maflcko:
review ACK f5a2000579
Tree-SHA512: 3da5c888db64a311276338270ba1dcad3eb2a24e205f6bb86fc92f767ecfa63682f13fafffff569fa0cfaea607ccb538f31e3934a086d482c3fe1be5d39f8791
6c3c619b35 test: generalize HasReason and use it in FailFmtWithError (Lőrinc)
Pull request description:
Standardized boost exception checking in recent tests introduced in https://github.com/bitcoin/bitcoin/pull/30546#discussion_r1756493521 by extending `HasReason` to accept `const char*` through `string_view` in `operator()`.
Note that `HasReason` only checks partial matches - but since we're specifying the whole error string, it doesn't affect us in this case.
ACKs for top commit:
maflcko:
review ACK 6c3c619b35
hodlinator:
ACK 6c3c619b35
Tree-SHA512: 740fb18b8fea78e4eb9740ceb0fe75d37246c28cfa2638b9d093e9514dd6d7926cc5be9ec57f8027cca3aa9d616e8c54322d2401cfa67fd25282f7816e63532d
fafd1a0f64 ci: Inline PACKAGE_MANAGER_INSTALL (MarcoFalke)
Pull request description:
The fallback `bash -c "$PACKAGE_MANAGER_INSTALL git"` is only needed by the `lint` task, so simplify it and inline `PACKAGE_MANAGER_INSTALL` once. Also, fixup the docs to add some other packages which are needed by podman in user-mode.
ACKs for top commit:
fanquake:
ACK fafd1a0f64
Tree-SHA512: e1665446d0fb5a2e8f2cb912117c7a42838c840199e7504a859b9155a13a2ff2e8606ac56689446f973fce02b00240041f071ebf00520778ed81eb1a01de6663