Commit graph

6776 commits

Author SHA1 Message Date
Hennadii Stepanov
70713303b6
scripted-diff: Rename PACKAGE_* variables to CLIENT_*
This change ensures consistent use of the `CLIENT_` namespace everywhere
in the repository.

-BEGIN VERIFY SCRIPT-

ren() { sed -i "s/\<$1\>/$2/g" $( git grep -l "$1" ./cmake ./src :\(exclude\)./src/secp256k1 ./test ) ; }

ren PACKAGE_NAME      CLIENT_NAME
ren PACKAGE_VERSION   CLIENT_VERSION_STRING
ren PACKAGE_URL       CLIENT_URL
ren PACKAGE_BUGREPORT CLIENT_BUGREPORT

-END VERIFY SCRIPT-
2024-10-28 12:36:19 +00:00
Hennadii Stepanov
332655cb52
build: Rename PACKAGE_* variables to CLIENT_*
The use of `PACKAGE_NAME` for the project's variable name is
problematic, as this name is commonly used in CMake's interface
variables. If third-party CMake code handles with scopes improperly,
our `PACKAGE_NAME` variable could end up with an unexpected value.

This change avoids such conflicts by renaming all `PACKAGE_*` variables
to `CLIENT_*`.
2024-10-28 12:35:55 +00:00
merge-script
6e21dedbf2
Merge bitcoin/bitcoin#31130: Drop miniupnp dependency
40e5f26a3f mapport: remove dead code in DispatchMapPort (Antoine Poinsot)
38fdf7c1fb mapport: drop outdated comments (Antoine Poinsot)
b7b2435290 doc: add release note for #31130 (Antoine Poinsot)
1b6dec98da depends: drop miniupnpc (Antoine Poinsot)
953533d021 doc: remove mentions of UPnP (Antoine Poinsot)
94ad614482 ci: remove UPnP options (Antoine Poinsot)
a9598e5eaa build: drop miniupnpc dependency (Antoine Poinsot)
a5fcfb7385 interfaces: remove now unused 'use_upnp' arg from 'mapPort' (Antoine Poinsot)
038bbe7b20 daemon: remove UPnP support (Antoine Poinsot)
844770b05e qt: remove UPnP settings (Antoine Poinsot)

Pull request description:

  This PR removes UPnP IGD support and drops our [miniupnp](https://github.com/miniupnp/miniupnp) dependency.

  Miniupnpc is a C library (somewhat) maintained by a single person which had several vulnerabilities in the past (a couple dozens are listed [here](https://cve.mitre.org/cgi-bin/cvekey.cgi?keyword=miniupnp)), some of which directly affected our software ([RCE in 2015](https://bitcoincore.org/en/2024/07/03/disclose_upnp_rce/), [OOM in 2020](https://bitcoincore.org/en/2024/07/31/disclose-upnp-oom/)).

  The main purpose of this functionality is to have more (non-data-center) reachable nodes on the network. For a non-technical user running Bitcoin Core at home, the software would automatically open a port on their router to receive incoming connections. This way, users not able to manually open a port on their router would still provide the network with more resources and enhance its diversity.

  However, due to past vulnerabilities (and a worry about unknown future ones) in miniupnpc this feature was disabled by default in https://github.com/bitcoin/bitcoin/pull/6795. Having it disabled by default kills (most of?) the purpose of having this functionality in the first place: someone technical enough to understand the `-upnp` startup option or the "enable UPnP" setting is most likely able to open a port on his box in the first place.

  In addition, laanwj implemented PCP with a NAT-PMP fallback directly in Bitcoin Core in https://github.com/bitcoin/bitcoin/pull/30043. If we ever want to re-enable automatic NAT traversal by default in Bitcoin Core, this is the best option (and in my opinion the only sane one). The NAT-PMP fallback makes it so compatibility shouldn't be (much of) an issue.

  On balance, i believe that keeping this functionality and this barely maintained C dependency has higher costs than benefits. Therefore i propose that we get rid of it.

ACKs for top commit:
  jarolrod:
    ACK 40e5f26a3f
  1440000bytes:
    Code Review ACK 40e5f26a3f
  laanwj:
    Code review ACK 40e5f26a3f
  i-am-yuvi:
    Tested ACK 40e5f26a3f

Tree-SHA512: 9ea48662775510f5ec6de7af65790f7c8d211603398e9d8c634a86387be81b28081419a95b4d6680d3d7fe6a9f16cec99f16516548201dc7e49781909899a657
2024-10-28 10:47:34 +00:00
kevkevinpal
d7fd766feb
test: added test to assert TX decode rpc error on submitpackage rpc 2024-10-27 15:17:23 -04:00
glozow
2a52718d73
Merge bitcoin/bitcoin#31152: functional test: Additional package evaluation coverage
Some checks failed
CI / test each commit (push) Has been cancelled
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Has been cancelled
CI / Win64 native, VS 2022 (push) Has been cancelled
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Has been cancelled
f32c34d0c3 functional test: Additional package evaluation coverage (Greg Sanders)

Pull request description:

  Current test coverage doesn't ensure that mempool trimming doesn't appear prior to the entire package, and not just the subpackage, is finished being submitted.

  Add a scenario that covers this case, where package ancestors can make it in individually, but would be immadiately evicted if not for the package CPFP.

  in response to https://github.com/bitcoin/bitcoin/pull/31122#discussion_r1813272637 where if applied onto that PR's old commit, the test fails due to package failure.

ACKs for top commit:
  sdaftuar:
    re-ACK f32c34d0c3
  rkrux:
    tACK f32c34d0c3
  glozow:
    reACK f32c34d0c3

Tree-SHA512: 739fcc5e66878b3def9b25dc588d8cb5349aaaa0901b11475879a413a03f6ea0e87d19de5bc4fb44ddd0436fdc052cdc3ed564f7e2ad510269aab9732d5c24eb
2024-10-26 09:37:20 -04:00
tdb3
9de9c858d5
test: enhance p2p_orphan_handling
Increases test robustness by adding
checks for orphanage size and presence
of orphans in the orphanage
2024-10-25 20:46:00 -04:00
tdb3
33af14b62e
test: reduce assert_debug_log reliance
p2p_orphan_handling now uses tx_in_orphanage
to more directly check for inclusion/exclusion
in the orphanage.
2024-10-25 18:52:39 -04:00
tdb3
0ea84bc362
test: explicitly check boolean verbosity is disallowed 2024-10-25 17:54:05 -04:00
tdb3
63f5e6ec79
test: add entry and expiration time checks 2024-10-25 17:11:27 -04:00
tdb3
56bf302714
refactor: rename rpc_getorphantxs to rpc_orphans
Generalizes the test to accommodate additional
orphan-related RPCs
2024-10-25 17:11:20 -04:00
tdb3
7824f6b077
test: check that getorphantxs is hidden 2024-10-25 17:11:12 -04:00
tdb3
ac68fcca70
rpc: disallow undefined verbosity in getorphantxs 2024-10-25 17:06:12 -04:00
Ava Chow
25dacae9c7
Merge bitcoin/bitcoin#31040: test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
5c299ecafe test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped (kevkevinpal)

Pull request description:

  After joining the bitcoin pr review club about https://github.com/bitcoin/bitcoin/pull/30793

  I learned about [`CVE-2012-3789`](https://github.com/bitcoin/bitcoin/blob/master/src/net_processing.cpp#L4693)

  So I was motivated to write a functional test that covers this part of the code,

  This test should add the max number of orphans to a nodes orphanage and then attempt to add another, then asserts that the number of orphans is still at the max amount

ACKs for top commit:
  achow101:
    ACK 5c299ecafe
  rkrux:
    ACK 5c299ecafe
  instagibbs:
    ACK 5c299ecafe
  tdb3:
    ACK 5c299ecafe

Tree-SHA512: 687bba337978e0945e94af71632998221e5565a5d83cf5a59ecf2ee52c7262d8ff907b94dceea3b80bed441dd19b24790b2904e88e1da14d30827c5469fcb4d3
2024-10-25 16:35:18 -04:00
Greg Sanders
f32c34d0c3 functional test: Additional package evaluation coverage
Current test coverage doesn't ensure that mempool trimming
doesn't appear prior to the entire package, and not just
the subpackage, is finished being submitted.

Add a scenario that covers this case, where package
ancestors can make it in individually, but would be
immadiately evicted if not for the package CPFP.
2024-10-25 09:22:57 -04:00
kevkevinpal
5c299ecafe
test: Assert that when we add the max orphan amount that we cannot add anymore and that a random orphan gets dropped 2024-10-24 21:48:39 -04:00
Ava Chow
7640cfdd62
Merge bitcoin/bitcoin#31118: doc: replace -? with -h and -help
33a28e252a Change default help arg to `-help` and mention `-h` and `-?` as alternatives (Lőrinc)
f0130ab1a1 doc: replace `-?` with `-h` for bench_bitcoin help (Lőrinc)

Pull request description:

  The question mark is interpreted as a wildcard for any single character in Zsh (see https://www.techrepublic.com/article/globbing-wildcard-characters-with-zsh), so `bench_bitcoin -?` will not show the help message on systems using Zsh, such as macOS.

  Since `-h` provides equivalent help functionality (as defined in https://github.com/bitcoin/bitcoin/blob/master/src/common/args.cpp#L684-L693), the `benchmarking.md` documentation has been updated to ensure compatibility with macOS.

  ----

  ### -?
  > % cmake -B build -DBUILD_BENCH=ON && cmake --build build && build/src/bench/bench_bitcoin -?
  zsh: no matches found: -?

  ### -h
  > % cmake -B build -DBUILD_BENCH=ON && cmake --build build && build/src/bench/bench_bitcoin -h
  Usage:  bench_bitcoin [options]
  Options:
  ...

  ----

  Based on the comments the args help default was also changed to `-help`, mentioning `-h` and `-?` (instead of `-?` being the default)

ACKs for top commit:
  edilmedeiros:
    tACK 33a28e252a
  maflcko:
    lgtm ACK 33a28e252a
  achow101:
    ACK 33a28e252a
  rkrux:
    tACK 33a28e252a
  laanwj:
    Code review ACK 33a28e252a

Tree-SHA512: 8c6e27488462be9ba9186b34abe6249c1d93026b3963acc0f42c75496f39407563766ae518cf1839156039cc0047e29d91f70d191cfb97e0fbde85665e88c71e
2024-10-24 18:01:41 -04:00
Antoine Poinsot
038bbe7b20
daemon: remove UPnP support
Keep the "-upnp" option as a hidden arg for one major version in order
to show a more user friendly error to people who had this option set in
their config file.
2024-10-24 18:23:30 +02:00
Ava Chow
e9b95665ee
Merge bitcoin/bitcoin#31046: init: Some small chainstate load improvements
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
31cc5006c3 init: Return fatal failure on snapshot validation failure (Martin Zumsande)
8f1246e833 init: Improve chainstate init db error messages (TheCharlatan)
cd093049dd init: Remove incorrect comment about shutdown condition (MarcoFalke)
635e9f85d7 init: Remove misleading log line when user chooses not to retry (TheCharlatan)
720ce880a3 init: Improve comment describing chainstate load retry behaviour (Martin Zumsande)
baea842ff1 init: Remove unneeded argument for mempool_opts checks (stickies-v)

Pull request description:

  These are mostly followups from #30968, making the code, log lines, error messages, and comments more consistent.

  The last commit is an attempt at improving the error reporting when loading the chainstate. It aims to more cleanly distinguish between errors arising from a specific database, and errors where the culprit may be less clear.

ACKs for top commit:
  achow101:
    ACK 31cc5006c3
  mzumsande:
    Code Review / lightly tested ACK 31cc5006c3
  BrandonOdiwuor:
    Code Review ACK 31cc5006c3.
  stickies-v:
    ACK 31cc5006c3

Tree-SHA512: 59fba4845ee45a3d91bf55807ae6b1c81458463b96bf664c8b1badfac503f6b01efd52a915fc399294e68a3f69985362a5a10a3844fa23f7707145ebe9ad349b
2024-10-23 18:33:31 -04:00
Ava Chow
b8c821cc1e
Merge bitcoin/bitcoin#30724: test: add test for specifying custom pidfile via -pid
04e4d52420 test: add test for specifying custom pidfile via `-pid` (Sebastian Falbesoner)
b832ffe044 refactor: introduce default pid file name constant in tests (tdb3)

Pull request description:

  This small PR adds test coverage for the `-pid` command line option, which allows to overrule the pid filename (`bitcoind.pid` by default). One can specify either a relative path (within the datadir) or an absolute one; the latter is tested using `self.options.tmpdir`. Note that the functional test file `feature_init.py` so far only contained a stress test; with this new sub-test added, both the description and the test name are adapted to be more generic.

ACKs for top commit:
  achow101:
    ACK 04e4d52420
  tdb3:
    ACK 04e4d52420
  ryanofsky:
    Code review ACK 04e4d52420
  naiyoma:
    Tested ACK [04e4d52420)

Tree-SHA512: b2bc8a790e5d187e2c84345f344f65a176b62caecd9797c3b9edf10294c741c33a24e535be640b56444b91dcf9c65c7dd152cdffd8b1c1d9ca68e5e3c6ad1e99
2024-10-23 17:39:30 -04:00
Martin Zumsande
6c9fe7b73e test: Prevent connection attempts to random IPs in p2p_seednodes.py
These addrs aren't unreachable as the test claims.
Specify a (non-working) proxy to make sure the connections fails
even if the addr was reachable.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2024-10-23 15:39:48 -04:00
Martin Zumsande
bb97b1ffa9 test: fix intermittent timeout in p2p_seednodes.py
On some CI runs, the timer in ThreadOpenConnection was only started *after*
the mocktime was set. Fix this by waiting for the first connection attempt,
which happens after the timer was started.

Also convert some comments into log messages/add a log, so that the test
isn't completely silent.
2024-10-23 15:39:15 -04:00
Vasil Dimov
57529ac4db
test: set P2PConnection.p2p_connected_to_node in peer_connect_helper()
Set `P2PConnection.p2p_connected_to_node` in
`P2PConnection.peer_connect_helper()` instead of
`TestNode.add_p2p_connection()` and
`TestNode.add_outbound_p2p_connection()`.

This way tests can create an instance of `P2PConnection` and use
`P2PConnection.peer_connect_helper()` directly.
2024-10-22 13:03:11 +02:00
Vasil Dimov
22cd0e888c
test: support WTX INVs from P2PDataStore and fix a comment 2024-10-22 13:03:11 +02:00
Vasil Dimov
ebe42c00aa
test: extend the SOCKS5 Python proxy to actually connect to a destination
If requested, make the SOCKS5 Python proxy redirect each connection to a
given destination. Actually act as a real proxy, connecting the
client to a destination, except that the destination is not what the
client asked for.

This would enable us to "connect" to Tor addresses from the functional
tests.
2024-10-22 13:03:02 +02:00
merge-script
d9f8dc6453
Merge bitcoin/bitcoin#31097: validation: Improve input script check error reporting
86e2a6b749 [test] A non-standard transaction which is also consensus-invalid should return the consensus error (Antoine Poinsot)
f859ff8a4e [validation] Improve script check error reporting (dergoegge)

Pull request description:

  An input script might be invalid for multiple reasons. For example, it might fail both a standardness check and a consensus check, which can lead to a `mandatory-script-verify-flag-failed` error being reported that includes the script error string from the standardness failure (e.g. `mandatory-script-verify-flag-failed (Using OP_CODESEPARATOR in non-witness script)`), which is confusing.

ACKs for top commit:
  darosior:
    re-ACK 86e2a6b749
  ariard:
    Re-Code Review ACK 86e2a6b7
  instagibbs:
    ACK 86e2a6b749

Tree-SHA512: 053939107c0bcd6643e9006b2518ddc3a6de47d2c6c66af71a04e8af5cf9ec207f19e54583b7a056efd77571edf5fd4f36c31ebe80d1f0777219c756c055eb42
2024-10-21 14:58:44 +01:00
merge-script
0e9f20625a
Merge bitcoin/bitcoin#31063: lint: commit-script-check.sh: echo to stderr
fac6cfe5ac lint: commit-script-check.sh: echo to stderr (MarcoFalke)

Pull request description:

  This makes it easier to redirect the produced `git diff` on failure. On success, it shouldn't hurt, because the same output is still present, just on stderr.

  Can be tested by introducing a fault in any scripted diff and then calling `commit-script-check.sh HEAD~..HEAD > any_file.txt`. Previously the file contained the full output, now it contains just the diff.

ACKs for top commit:
  TheCharlatan:
    ACK fac6cfe5ac

Tree-SHA512: b4dfad10a4a902729a7ad7533ed0ef86b9e79761083f2ec623d448a551462b268fe04bdba387ca62160dae9ef7b1781e005dec60f18b111d9bfa6b97357108e6
2024-10-21 10:46:46 +01:00
Lőrinc
33a28e252a Change default help arg to -help and mention -h and -? as alternatives
% build/src/bench/bench_bitcoin -h
[...]
  -help
       Print this help message and exit (also -h or -?)
2024-10-21 11:08:51 +02:00
merge-script
e8f72aefd2
Merge bitcoin/bitcoin#29877: tracing: explicitly cast block_connected duration to nanoseconds
Some checks failed
CI / test each commit (push) Has been cancelled
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Has been cancelled
CI / Win64 native, VS 2022 (push) Has been cancelled
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Has been cancelled
cd0edf26c0 tracing: cast block_connected duration to nanoseconds (0xb10c)

Pull request description:

  When the `validation:block_connected` tracepoint was introduced in 8f37f5c2a5, the connect block duration was passed in microseconds `µs`. By starting to use steady clock in fabf1cdb20 this changed to nanoseconds `ns`. As the test only checked if the duration value is `> 0` as a plausibility check, this went unnoticed. This was detected this when setting up monitoring for block validation time as part of the Great Consensus Cleanup Revival discussion.

  This change casts the duration explicitly to nanoseconds, updates the documentation, and adds a check for an upper bound to the tracepoint interface tests. The upper bound is quite lax as mining the block takes much longer than connecting the empty test block. It's however able to detect a duration passed in an incorrect unit (1000x off).

  A previous version of this PR casted the duration to microseconds `µs` - however, as the last three major releases have had the duration as nanoseconds (and this went unnoticed), we assume that this is the API now and changeing it back to microseconds would break the API again. See also https://github.com/bitcoin/bitcoin/pull/29877#issuecomment-2067867597

ACKs for top commit:
  maflcko:
    re-lgtm ACK cd0edf26c0
  laanwj:
    re-ACK cd0edf26c0

Tree-SHA512: 54a1eea0297e01c07c2d071ffafbf97dbd080f763e1dc0014ff086a913b739637c1634b1cf87c90b94a3c2f66006acfaada0414a15769cac761e03bc4aab2a77
2024-10-17 16:30:12 +01:00
Antoine Poinsot
86e2a6b749 [test] A non-standard transaction which is also consensus-invalid should return the consensus error 2024-10-17 10:58:42 +01:00
merge-script
21e2f06a1c
Merge bitcoin/bitcoin#31067: test: Print CompletedProcess object on error
fa43c4f93c test: Print CompletedProcess object on error (MarcoFalke)

Pull request description:

  It would be good to know the output on `Error parsing command output`. Otherwise test failures are meaningless: https://github.com/bitcoin/bitcoin/issues/30792#issuecomment-2325911157

  Fix it by just printing the full `CompletedProcess` object.

  Also, use the modern `subprocess.run` to simplify the code.

ACKs for top commit:
  BrandonOdiwuor:
    Code Review ACK fa43c4f93c
  laanwj:
    This contains some useful information, so ACK fa43c4f93c

Tree-SHA512: ae7c1cb1f48af2a6feae6d1a5a967c0720f6c6675c1ce20ace7cac18c00f3d4069b8abcc58204855e92ff5303158b9a942bab3b71acae0737768d941a5773c91
2024-10-16 14:50:49 +01:00
merge-script
15563d3388
Merge bitcoin/bitcoin#30859: doc: cmake: prepend "build" to functional/test_runner.py
e64b2f1a16 doc: cmake: prepend and explain "build/" where needed (Larry Ruane)

Pull request description:

  This is a small follow-on to #30741, prepend `build/` to the path for `test_runner.py`.

ACKs for top commit:
  jonatack:
    ACK e64b2f1a16
  maflcko:
    lgtm ACK e64b2f1a16
  tdb3:
    re ACK e64b2f1a16

Tree-SHA512: 80943d4f342987bf060adacb1c7db2e9ff8de5a6da592846ba23f230281d3a5b306162c4c86e61739a29323eaa4abf09f69f41302996d5809f448e5788a74a87
2024-10-15 11:10:20 +01:00
Larry Ruane
e64b2f1a16 doc: cmake: prepend and explain "build/" where needed 2024-10-11 11:24:21 -06:00
MarcoFalke
fa43c4f93c
test: Print CompletedProcess object on error 2024-10-10 21:16:52 +02:00
Sebastian Falbesoner
882f736d0a doc: lint: correct outdated comment (s/Makefile.am/CMakeLists.txt/) 2024-10-10 12:25:14 +02:00
Sebastian Falbesoner
1786be7b4a scripted-diff: drop config/ subdir for bitcoin-config.h, rename to bitcoin-build-config.h
Follow-up for PR #30856, commit 0dd66251.

-BEGIN VERIFY SCRIPT-
sed -i "s|config/bitcoin-config\.h|bitcoin-build-config.h|g" $(git grep -l config/bitcoin-config\.h)
sed -i "s|bitcoin-config\.h|bitcoin-build-config.h|g" $(git grep -l "bitcoin-config\.h" ./src ./test ./cmake)
git mv ./cmake/bitcoin-config.h.in ./cmake/bitcoin-build-config.h.in
-END VERIFY SCRIPT-
2024-10-10 12:22:12 +02:00
MarcoFalke
fac6cfe5ac
lint: commit-script-check.sh: echo to stderr 2024-10-09 18:00:42 +02:00
TheCharlatan
8f1246e833
init: Improve chainstate init db error messages
They should name the correct source of an error, or be generic if no
clear source can be ascertained.
2024-10-08 22:45:02 +02:00
merge-script
a9f6a57b69
Merge bitcoin/bitcoin#30920: test: Remove 0.16.3 test from wallet_backwards_compatibility.py
fae44c83da test: Remove 0.16.3 test from wallet_backwards_compatibility.py (MarcoFalke)

Pull request description:

  The test checks that any wallet created with current master can not be loaded with `v0.16.3`. This is interesting documentation, however it is probably not something to keep as a test, because:

  * It seems like an extremely unlikely (and unsupported) edge case that someone creates a wallet with master and then goes ahead to open it with a long EOL software version.
  * A better test would be the inverse: Create a wallet with `v0.16.3` and open it with current master. This is already tested in `wallet_upgradewallet.py`, where I've added an additional balance check before upgrading the `v0.16.3` wallet.
  * The test is intermittently failing when shutting down the `v0.16.3` node, for example in https://github.com/bitcoin/bitcoin/pull/30875#issuecomment-2357565564. The exact cause is unclear, but given that the test isn't worthy to keep, removing it will ensure that the error disappears.

ACKs for top commit:
  Sjors:
    utACK fae44c83da
  fanquake:
    ACK fae44c83da - I agree that test seems to have past it's usefulness, and the fact that it otherwise causes intemittent issues is further reason to remove it.

Tree-SHA512: 85bf428e616e0880198c1a7529936520505d7fa87c2eeb87a0457f13b50a163accaf5f80f9364dea978f6bd14b0b5350cda88f49aa7584682c8b5e0b0b117724
2024-10-08 16:10:04 +01:00
merge-script
56093565bb
Merge bitcoin/bitcoin#31018: test: Treat exclude list warning as failure in CI
fa6d14eacb test: Treat exclude list warning as failure in CI (MarcoFalke)

Pull request description:

  An outdated exclude list or otherwise an error in the exclude list handling is usually a bug.

  So make it fatal in the CI, instead of silently ignoring it.

  Fixes https://github.com/bitcoin/bitcoin/pull/30872/files#r1757015334

  Can be tested with something like (with and without `--ci`):

  ```
  ./bld-cmake/test/functional/test_runner.py wallet_disable -x wallet_disablee

ACKs for top commit:
  tdb3:
    ACK fa6d14eacb
  ismaelsadeeq:
    utACK fa6d14eacb

Tree-SHA512: 03a70dff9d1272d982591d60ab764f9233d4802488bc1bad305a2755e2d7ed86e691ee94767a3bc5f68321b63214aba44e6f9edd1543dfad7a20f9397cf78734
2024-10-08 15:29:33 +01:00
glozow
5ea335a97f
Merge bitcoin/bitcoin#30793: rpc: add getorphantxs
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
98c1536852 test: add getorphantxs tests (tdb3)
93f48fceb7 test: add tx_in_orphanage() (tdb3)
34a9c10e8c rpc: add getorphantxs (tdb3)
f511ff3654 refactor: move verbosity parsing to rpc/util (tdb3)
532491faf1 net: add GetOrphanTransactions() to PeerManager (tdb3)
91b65adff2 refactor: add OrphanTxBase for external use (tdb3)

Pull request description:

  This PR adds a new hidden rpc, `getorphantxs`, that provides the caller with a list of orphan transactions.  This rpc may be helpful when checking orphan behavior/scenarios (e.g. in tests like `p2p_orphan_handling`) or providing additional data for statistics/visualization.

  ```
  getorphantxs ( verbosity )

  Shows transactions in the tx orphanage.

  EXPERIMENTAL warning: this call may be changed in future releases.

  Arguments:
  1. verbosity    (numeric, optional, default=0) 0 for an array of txids (may contain duplicates), 1 for an array of objects with tx details, and 2 for details from (1) and tx hex

  Result (for verbose = 0):
  [           (json array)
    "hex",    (string) The transaction hash in hex
    ...
  ]

  Result (for verbose = 1):
  [                          (json array)
    {                        (json object)
      "txid" : "hex",        (string) The transaction hash in hex
      "wtxid" : "hex",       (string) The transaction witness hash in hex
      "bytes" : n,           (numeric) The serialized transaction size in bytes
      "vsize" : n,           (numeric) The virtual transaction size as defined in BIP 141. This is different from actual serialized size for witness transactions as witness data is discounted.
      "weight" : n,          (numeric) The transaction weight as defined in BIP 141.
      "expiration" : xxx,    (numeric) The orphan expiration time expressed in UNIX epoch time
      "from" : [             (json array)
        n,                   (numeric) Peer ID
        ...
      ]
    },
    ...
  ]

  Result (for verbose = 2):
  [                          (json array)
    {                        (json object)
      "txid" : "hex",        (string) The transaction hash in hex
      "wtxid" : "hex",       (string) The transaction witness hash in hex
      "bytes" : n,           (numeric) The serialized transaction size in bytes
      "vsize" : n,           (numeric) The virtual transaction size as defined in BIP 141. This is different from actual serialized size for witness transactions as witness data is discounted.
      "weight" : n,          (numeric) The transaction weight as defined in BIP 141.
      "expiration" : xxx,    (numeric) The orphan expiration time expressed in UNIX epoch time
      "from" : [             (json array)
        n,                   (numeric) Peer ID
        ...
      ],
      "hex" : "hex"          (string) The serialized, hex-encoded transaction data
    },
    ...
  ]

  Examples:
  > bitcoin-cli getorphantxs 2
  > curl --user myusername --data-binary '{"jsonrpc": "2.0", "id": "curltest", "method": "getorphantxs", "params": [2]}' -H 'content-type: application/json' http://127.0.0.1:8332/
  ```
  ```
  $ build/src/bitcoin-cli getorphantxs 2
  [
    {
      "txid": "50128aac5deab548228d74d846675ad4def91cd92453d81a2daa778df12a63f2",
      "wtxid": "bb61659336f59fcf23acb47c05dc4bbea63ab533a98c412f3a12cb813308d52c",
      "bytes": 133,
      "vsize": 104,
      "weight": 415,
      "expiration": 1725663854,
      "from": [
        1
      ],
      "hex": "020000000001010b992959eaa2018bbf31a4a3f9aa30896a8144dbd5cfaf263bf07c0845a3a6620000000000000000000140fe042a010000002251202913b252fe537830f843bfdc5fa7d20ba48639a87c86ff837b92d083c55ad7c102015121c0000000000000000000000000000000000000000000000000000000000000000100000000"
    },
    {
      "txid": "330bb7f701604a40ade20aa129e9a3eb8a7bf024e599084ca1026d3222b9f8a1",
      "wtxid": "b7651f7d4c1a40c4d01f6a1e43a121967091fa0f56bb460146c1c5c068e824f6",
      "bytes": 133,
      "vsize": 104,
      "weight": 415,
      "expiration": 1725663854,
      "from": [
        2
      ],
      "hex": "020000000001013600adfe41e0ebd2454838963d270916d2b47239c9eebb93a992b720d3589a080000000000000000000140fe042a010000002251202913b252fe537830f843bfdc5fa7d20ba48639a87c86ff837b92d083c55ad7c102015121c0000000000000000000000000000000000000000000000000000000000000000100000000"
    }
  ]
  ```

ACKs for top commit:
  glozow:
    reACK 98c1536852
  hodlinator:
    re-ACK 98c1536852
  danielabrozzoni:
    ACK 98c1536852
  pablomartin4btc:
    tACK 98c1536852
  itornaza:
    reACK 98c1536852

Tree-SHA512: 66075f9faa83748350b87397302100d08af92cbef5fadb27f2b4903f028c08020bf34a23e17262b41abb3f379ca9f46cf6cd5459b8681f2b83bffbbaf3c03ff9
2024-10-05 11:20:06 -04:00
Ava Chow
dda2613239
Merge bitcoin/bitcoin#30929: log: Enforce trailing newline
fa2b7d8d6b Remove redundant unterminated-logprintf tidy check (MarcoFalke)
bbbb2e43ee log: Enforce trailing newline, Remove redundant m_started_new_line (MarcoFalke)

Pull request description:

  There are many problems around missing a trailing newline while logging:

  * All log lines are currently terminated by a trailing newline. This means any runtime code trying to handle a "missing" newline is currently dead code.
  * Leaving a line unterminated is racy and can cause content corruption by mixing log lines from different sources.
  * It requires extra code like `m_started_new_line` to keep track of, which is annoying and pointless to maintain, because it is currently dead code, see https://github.com/bitcoin/bitcoin/pull/30386#discussion_r1684380835.
  * It requires a standalone `unterminated-logprintf` clang-tidy plugin, which is unmaintained (no one updated it for the new log function names), probably harder to maintain than normal C++ code (because it requires clang AST matcher knowledge), brittle (it can fail to detect issues at any time, if it goes out-of-sync, or be explicitly disabled via `NOLINT`), and annoying for devs (it is slow and intricate to run locally and thus only effectively run on CI or via the CI scripts).

  Fix all issues by enforcing the trailing newline in logs directly in the code. Then remove all the other stuff.

  This refactor does not change behavior.

ACKs for top commit:
  stickies-v:
    re-ACK fa2b7d8d6b
  achow101:
    ACK fa2b7d8d6b
  ryanofsky:
    Code review ACK fa2b7d8d6b. Just comment and test cleanup since last review

Tree-SHA512: 10ed420f6c2fdb0f491d6c880be8dd2e8beef628f510adebadf4c3849d9f5e28906519d5cbaeb295f4c7c1b07c4c88a9905b3cfe30fee3a2c91ac9fd24ae6755
2024-10-02 19:05:34 -04:00
tdb3
98c1536852
test: add getorphantxs tests
Adds functional tests for getorphantxs
2024-10-02 18:28:54 -04:00
tdb3
93f48fceb7
test: add tx_in_orphanage()
Allows tests to check if a transaction
is contained within the orphanage
2024-10-02 18:23:27 -04:00
MarcoFalke
fa6d14eacb
test: Treat exclude list warning as failure in CI 2024-10-02 15:10:39 +02:00
Martin Zumsande
a1576edab3 test: add missing sync to feature_fee_estimation.py
Fixes a race between node 1 catching up with the chain and mining a
new block in the sanity_check_rbf_estimates subtest.
2024-10-01 18:10:31 -04:00
MarcoFalke
fa2b7d8d6b
Remove redundant unterminated-logprintf tidy check 2024-10-01 11:34:22 +02:00
MarcoFalke
bbbbaa0d9a
Fix unsigned integer overflows in interpreter 2024-10-01 10:45:44 +02:00
Fabian Jahr
c4dc81f9c6
test: Remove dead code from interface_zmq 2024-09-29 21:46:15 +02:00
Sebastian Falbesoner
940edd6ac2 test: refactor: introduce and use TRUC_CHILD_MAX_VSIZE constant 2024-09-28 22:49:41 +02:00
Sebastian Falbesoner
c16ae71768 test: switch MiniWallet padding unit from weight to vsize
The weight unit is merely a consensus rule detail and is largely
irrelevant for fee-rate calculations and mempool policy rules (e.g. for
package relay and TRUC limits), so there doesn't seem to be any value of
using a granularity that we can't even guarantee to reach exactly
anyway.

Switch to the more natural unit of vsize instead, which simplifies both
the padding implementation and the current tests that take use of this
padding. The rather annoying multiplications by `WITNESS_SCALE_FACTOR`
can then be removed and weird-looking magic numbers like `4004` can be
replaced by numbers that are more connected to actual policy limit
constants from the codebase, e.g. `1001` for exceeding
`TRUC_CHILD_MAX_VSIZE` by one.
2024-09-28 22:49:41 +02:00
merge-script
d812cf1189
Merge bitcoin/bitcoin#30879: test: re-bucket long-running tests
Some checks failed
CI / test each commit (push) Has been cancelled
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Has been cancelled
CI / Win64 native, VS 2022 (push) Has been cancelled
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Has been cancelled
f5a2000579 test: re-bucket long-running tests (willcl-ark)

Pull request description:

  Re-bucket:

  - `p2p_node_network_limited -v*transport`
  - `feature_assume_utxo`

  On CI runners these tests are taking longer than their current bucket suggests, often being among the last to finish.

  Re-bucket them to improve CI efficiency.

ACKs for top commit:
  maflcko:
    review ACK f5a2000579

Tree-SHA512: 3da5c888db64a311276338270ba1dcad3eb2a24e205f6bb86fc92f767ecfa63682f13fafffff569fa0cfaea607ccb538f31e3934a086d482c3fe1be5d39f8791
2024-09-27 11:17:43 +01:00
merge-script
fa7c2838a5
Merge bitcoin/bitcoin#30948: test: Add missing sync_mempools() to fill_mempool()
faf801515f test: Add missing sync_mempools() to fill_mempool() (MarcoFalke)
fa48be6f02 test: Refactor fill_mempool to extract send_batch helper (MarcoFalke)

Pull request description:

  Not doing the sync will lead to (intermittent) issues, as explained in https://github.com/bitcoin/bitcoin/issues/30922#issuecomment-2364529013.

  Fix all issues by doing the sync by default and disable it in places that do not need the sync.

  Fixes #30922

ACKs for top commit:
  mzumsande:
    Tested ACK faf801515f
  ismaelsadeeq:
    Tested ACK faf801515f
  marcofleon:
    Tested ACK faf801515f

Tree-SHA512: 2de62d168cbb6857a9fb8bc12c42a9093fedf5e9beb6f83a32b3fa72a5ba3cf03631055fd25ef553399a27a6fe0d71c44cfe60660a4d31986debd13b8ab00228
2024-09-26 16:37:02 +01:00
MarcoFalke
faf801515f
test: Add missing sync_mempools() to fill_mempool()
Also disable the function, when it is not needed.
2024-09-24 10:13:21 +02:00
MarcoFalke
fa48be6f02
test: Refactor fill_mempool to extract send_batch helper
This is needed for the next commit
2024-09-24 09:59:36 +02:00
Ava Chow
90a5786bba
Merge bitcoin/bitcoin#30678: wallet: Write best block to disk before backup
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
f20fe33e94 test: Add basic balance coverage to wallet_assumeutxo.py (Fabian Jahr)
037b101e80 test: Add coverage for best block locator write in wallet_backup (Fabian Jahr)
31c0df0389 wallet: migration, write best locator before unloading wallet (furszy)
7e3dbe4180 wallet: Write best block to disk before backup (Fabian Jahr)

Pull request description:

  I discovered that we don't write the best block to disk when trying to explain the behavior described here: https://github.com/bitcoin/bitcoin/pull/30455#discussion_r1719951882

  In the context of that test, the behavior is confusing and I think it also shows that one of the already existing tests in `wallet_assumeutxo.py` doesn't actually test what it says. It only fails because the best block isn't written and actually, the height of the backup that is loaded is at the snapshot height during backup. So it really shouldn't fail since it's past the background sync blocks already.

  I'm not sure if this is super relevant in practice though so I am first looking for concept ACKs on the `BackupWallet` code change. Either way, I think this behavior should be documented better if it is left as is and the test should be changed.

ACKs for top commit:
  achow101:
    ACK f20fe33e94
  furszy:
    ACK f20fe33

Tree-SHA512: bb384a940df5c942fffe2eb06314ade4fc5d9b924012bfef3b1c456c4182a30825d1e137d8ae561d93d3a8a2f4d1c1ffe568132d20fa7d04844f1e289ab4a28b
2024-09-23 16:03:04 -04:00
Ava Chow
dabc74e86c
Merge bitcoin/bitcoin#30409: Introduce waitTipChanged() mining interface, replace RPCNotifyBlockChange, drop CRPCSignals & g_best_block
7942951e3f Remove unused g_best_block (Ryan Ofsky)
e3a560ca68 rpc: use waitTipChanged for longpoll (Ryan Ofsky)
460687a09c Remove unused CRPCSignals (Sjors Provoost)
dca923150e Replace RPCNotifyBlockChange with waitTipChanged() (Sjors Provoost)
2a40ee1121 rpc: check for negative timeout arg in waitfor* (Sjors Provoost)
de7c855b3a rpc: recommend -rpcclienttimeout=0 for waitfor* (Sjors Provoost)
77ec072925 rpc: fix waitfornewblock description (Sjors Provoost)
285fe9fb51 rpc: add test for waitforblock and waitfornewblock (Sjors Provoost)
b94b27cf05 Add waitTipChanged to Mining interface (Sjors Provoost)
7eccdaf160 node: Track last block that received a blockTip notification (Sjors Provoost)
ebb8215f23 Rename getTipHash() to getTip() and return BlockRef (Sjors Provoost)
89a8f74bbb refactor: rename BlockKey to BlockRef (Sjors Provoost)

Pull request description:

  This continues the work in #30200 so that a future Stratum v2 Template Provider (see #29432) can avoid accessing node internals. It needs to know when a new block arrives in order to push new templates to connected clients.

  `waitTipChanged()` uses a new kernel notification `notifications().m_tip_block_mutex`, which this PR also introduces (a previous version used `g_best_block`).

  In order to ensure the new method works as intended, the `waitfornewblock`, `waitforblock` and `waitforblockheight` RPC methods are refactored to use it. This allows removing `RPCNotifyBlockChange`.

  There's a commit to add (direct) tests for the methods that are about to be refactored:
  - `waitfornewblock` was already implicitly tested by `feature_shutdown.py`.
  - `waitforblockheight` by `feature_coinstatsindex.py` and `example_test.py`

  This PR renames `getTipHash()` to `getTip()` and returns a `BlockRef` (renamed from `BlockKey`) so that callers can use either the height or hash.

  The later commits make trivial improvements to the `waitfor*` RPC calls (not needed for this PR).

  The `waitTipChanged()` method could probably also be used for the longpoll functionality in `getblocktemplate`, but I'm a bit reluctant to touch that.

  `RPCServer::OnStarted` no longer does anything and `RPCServer::OnStopped` merely prints a log statement. They were added in #5711 as a refactor. This PR drops them entirely.

  Finally `g_best_block` is also dropped.

ACKs for top commit:
  achow101:
    ACK 7942951e3f
  ryanofsky:
    Code review ACK 7942951e3f. Just rebased since last review
  TheCharlatan:
    Re-ACK 7942951e3f

Tree-SHA512: a5559446b4000c95e07aad33284b7ee2e57aafd87e1ae778b3825d59689566d047a8047e47a10f76e6e341e7dc72fd265a65afbc0a9c011d17c4cafd55031837
2024-09-23 15:40:33 -04:00
willcl-ark
f5a2000579
test: re-bucket long-running tests
- p2p_node_network_limited -v*transport
- feature_assume_utxo

On CI runners these tests are taking longer than their current bucket
suggests, often being among the last to finish.

Re-bucket them to improve CI efficiency.
2024-09-23 15:23:20 +01:00
Sebastian Falbesoner
04e4d52420 test: add test for specifying custom pidfile via -pid 2024-09-23 12:41:10 +02:00
tdb3
b832ffe044 refactor: introduce default pid file name constant in tests 2024-09-23 12:41:10 +02:00
Fabian Jahr
f20fe33e94
test: Add basic balance coverage to wallet_assumeutxo.py 2024-09-22 19:19:12 +02:00
Ava Chow
4148e60909
Merge bitcoin/bitcoin#30679: fix: handle invalid -rpcbind port earlier
e6994efe08 fix: increase rpcbind check robustness (tdb3)
d38e3aed89 fix: handle invalid rpcbind port earlier (tdb3)
83b67f2e6d refactor: move host/port checking (tdb3)
73c243965a test: add tests for invalid rpcbind ports (tdb3)

Pull request description:

  Previously, when an invalid port was specified in `-rpcbind`, the `SplitHostPort()` return value in `HTTPBindAddresses()` was ignored and attempt would be made to bind to the default rpcbind port (with the host/port treated as a host).

  This rearranges port checking code in `AppInitMain()` to handle the invalid port before reaching `HTTPBindAddresses()`.  Also adds a check in `HTTPBindAddresses()` as a defensive measure for future changes.

  Adds then updates associated functional tests as well.

ACKs for top commit:
  achow101:
    ACK e6994efe08
  ryanofsky:
    Code review ACK e6994efe08
  zaidmstrr:
    Code review ACK [e6994ef](e6994efe08)

Tree-SHA512: bcc3e5ceef21963821cd16ce6ecb83d5c5657755882c05872a7cfe661a1492b1d631f54de22f41fdd173512d62dd15dc37e394fe1a7abe4de484b82cd2438b92
2024-09-20 13:34:24 -04:00
Ava Chow
c985a34b9c
Merge bitcoin/bitcoin#26990: cli: Improve error message on multiwallet cli-side commands
54227e681a rpc, cli: improve error message on multiwallet mode (pablomartin4btc)

Pull request description:

  Running a CLI command when multiple wallets are loaded and `-rpcwallet` is not specified, should return a clearer error.

  Currently in `master`:

  ```
  $ bitcoin-cli -regtest -generate 1
  error code: -19
  error message:
  Wallet file not specified (must request wallet RPC through /wallet/<filename> uri-path).
  Try adding "-rpcwallet=<filename>" option to bitcoin-cli command line.
  ```

  With this change:

  ```
  $ bitcoin-cli -regtest -generate 1
  error code: -19
  error message:
  Multiple wallets are loaded. Please select which wallet to use by requesting the RPC through the /wallet/<walletname> URI path. Or for the CLI, specify the "-rpcwallet=<walletname>" option before the command (run "bitcoin-cli -h" for help or "bitcoin-cli listwallets" to see which wallets are currently loaded).
  ```

ACKs for top commit:
  maflcko:
    review ACK 54227e681a
  achow101:
    ACK 54227e681a
  furszy:
    utACK 54227e681a
  mzumsande:
    Code Review ACK 54227e681a
  jonatack:
    ACK 54227e681a

Tree-SHA512: 51ff24f64858aa6be6adf6f20105c9f076ebea743780bf2a4399f7fe8b5239cbb1ea06d32b2ef5e850da2369abb0ef7a52c50c2b8f31f4ca90d3a486abc9b77e
2024-09-20 12:00:27 -04:00
Fabian Jahr
037b101e80
test: Add coverage for best block locator write in wallet_backup 2024-09-20 17:20:15 +02:00
Fabian Jahr
7e3dbe4180
wallet: Write best block to disk before backup
This ensures that the best block is included in the backup which leads to a more consistent behavior when loading the backup.
2024-09-20 17:16:35 +02:00
merge-script
2db926f49c
Merge bitcoin/bitcoin#30889: log: Use ConstevalFormatString
facbcd4cef log: Use ConstevalFormatString (MarcoFalke)
fae9b60c4f test: Use LogPrintStr to test m_log_sourcelocations (MarcoFalke)
fa39b1ca63 doc: move-only logging warning (MarcoFalke)

Pull request description:

  This changes all logging (including the wallet logging) to produce a
  `ConstevalFormatString` at compile time, so that the format string can be
  validated at compile-time.

  I tested with `clang` and found that the compiler will use less than 1% more of time and memory.

  When an error is found, the compile-time error depends on the compiler, but it may look similar to:

  ```
  src/util/string.h: In function ‘int main(int, char**)’:
  src/bitcoind.cpp:265:5:   in ‘constexpr’ expansion of ‘util::ConstevalFormatString<1>(((const char*)"Hi %s %s"))’
  src/util/string.h:38:98:   in ‘constexpr’ expansion of ‘util::ConstevalFormatString<1>::Detail_CheckNumFormatSpecifiers(std::basic_string_view<char>(((const char*)((util::ConstevalFormatString<1>*)this)->util::ConstevalFormatString<1>::fmt)))’
  src/util/string.h:78:34: error: expression ‘<throw-expression>’ is not a constant expression
     78 |         if (num_params != count) throw "Format specifier count must match the argument count!";
        |                                  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  ```

  This refactor does not change behavior of the compiled executables.

ACKs for top commit:
  hodlinator:
    re-ACK facbcd4cef
  l0rinc:
    ACK facbcd4cef
  ryanofsky:
    Code review ACK facbcd4cef
  pablomartin4btc:
    re-ACK facbcd4cef
  stickies-v:
    Approach ACK and code LGTM facbcd4cef modulo a `tinyformat::format_error` concern.

Tree-SHA512: 852f74d360897020f0d0f6e5064edc5e7f7dacc2bec1d5feff22c634a2fcd2eb535aa75be0b7191d9053728be6108484c737154b02d68ad3186a2e5544ba0db8
2024-09-19 12:17:14 +01:00
merge-script
ab0b5706b2
Merge bitcoin/bitcoin#30875: doc: fixed inconsistencies in documentation between autotools to cmake change
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
a9964c0444 doc: Updating docs from autotools to cmake (kevkevinpal)

Pull request description:

  A bit of a followup from https://github.com/bitcoin/bitcoin/pull/30840

  - In this change the documentation where we refer to the `./configure` script which is now gone and have converted the configure params to use the `cmake` equivalent.

ACKs for top commit:
  maflcko:
    ACK a9964c0444
  jonatack:
    utACK a9964c0444
  jarolrod:
    ACK a9964c0444
  tdb3:
    ACK a9964c0444
  pablomartin4btc:
    re-ACK a9964c0444

Tree-SHA512: f7ed20b8ad61f028c0d242b9cc70650d8da63057d4a8f7da88f0117a8d3241c5fe8fcf19d56ec82088160b9fee9b175fe9f64e5a845260d3696dc7e94bfdd0bd
2024-09-18 18:44:02 +01:00
kevkevinpal
a9964c0444
doc: Updating docs from autotools to cmake
replaced --enable-debug with -DCMAKE_BUILD_TYPE=Debug in developer-notes
replaced --enable-multiprocess with -DWITH_MULTIPROCESS=ON
replaced --disable-zmq with -DWITH_ZMQ=OFF
2024-09-18 11:04:52 -04:00
MarcoFalke
fae44c83da
test: Remove 0.16.3 test from wallet_backwards_compatibility.py 2024-09-18 12:08:28 +02:00
tdb3
d38e3aed89
fix: handle invalid rpcbind port earlier
Previously, when an invalid port was specified
in `-rpcbind`, the `SplitHostPort()` return value
in `HTTPBindAddresses()` was ignored and attempt
would be made to bind to the default rpcbind port
(with the host/port treated as a host).

This rearranges port checking code in
`AppInitMain()` to handle the invalid
port before reaching `HTTPBindAddresses()`.

Also adjusts associated functional tests.
2024-09-17 21:47:29 -04:00
tdb3
73c243965a
test: add tests for invalid rpcbind ports 2024-09-17 20:03:00 -04:00
pablomartin4btc
54227e681a rpc, cli: improve error message on multiwallet mode
The primary objective is to provide users with clearer
and more informative error messages when encountering
the RPC_WALLET_NOT_SPECIFIED error, which occurs when
multiple wallets are loadad.

This commit also rectifies the error message consistency
by bringing the error message in line with the definition
established in protocol.h ("error when there are multiple
wallets loaded").
2024-09-17 16:22:12 -03:00
MarcoFalke
facbcd4cef
log: Use ConstevalFormatString
This changes all logging (including the wallet logging) to produce a
ConstevalFormatString at compile time, so that the format string can be
validated at compile-time.

Also, while touching the wallet logging, avoid a copy of the template
Params by using const Params&.
2024-09-17 18:21:23 +02:00
Martin Zumsande
0bd53d913c test: add test for getchaintips behavior with invalid chains
This test would fail to mark the chain tip as "invalid" instead
of "headers-only" without the previous commit marking the headers
as BLOCK_FAILED_CHILD.
2024-09-17 11:40:36 -04:00
Martin Zumsande
ccd98ea4c8 test: cleanup rpc_getchaintips.py
Remove whitespace that doesn't conform with pep8 and turn some
comments into log messages.
2024-09-17 11:40:36 -04:00
Martin Zumsande
f5149ddb9b validation: mark blocks building on an invalid block as BLOCK_FAILED_CHILD
Without doing so, header-only chains building on a chain that
will be marked as invalid would still be eligible for m_best_header.
This improves both getblockchaininfo and getchaintips behavior.

While this adds an iteration over the entire block index, it can only be
triggered by the user (invalidateblock) or by others at a cost (the
header needs to be accepted in the first place, so it needs valid PoW).

Co-authored-by: TheCharlatan <seb.kung@gmail.com>
2024-09-17 11:40:36 -04:00
Martin Zumsande
783cb7337f validation: call RecalculateBestHeader in InvalidChainFound
This means that it is being called in two situations:
1.) As part of the invalidateblock rpc
2.) When we receive a block for which we have a valid
header in our block index, but the block turns out to be invalid
2024-09-17 11:39:21 -04:00
Martin Zumsande
9275e9689a rpc: call RecalculateBestHeader as part of reconsiderblock
Co-authored-by: Fabian Jahr <fjahr@protonmail.com>
2024-09-17 11:39:21 -04:00
Sjors Provoost
285fe9fb51
rpc: add test for waitforblock and waitfornewblock 2024-09-17 09:27:43 +02:00
Ava Chow
e983ed41d9
Merge bitcoin/bitcoin#30410: rpc, rest: Improve block rpc error handling, check header before attempting to read block data.
6a1aa510e3 rpc: check block index before reading block / undo data (Martin Zumsande)
6cbf2e5f81 rpc: Improve gettxoutproof error when only header is available. (Martin Zumsande)
69fc867ea1 test: add coverage to getblock and getblockstats (Martin Zumsande)
5290cbd585 rpc: Improve getblock / getblockstats error when only header is available. (Martin Zumsande)
e5b537bbdf rest: improve error when only header of a block is available. (Martin Zumsande)

Pull request description:

  Fixes #20978

  If a block was pruned, `getblock` already returns a specific error: "Block not available (pruned data)".
  But if we haven't received the full block yet (e.g. in a race with block downloading after a new block was received headers-first, or during IBD) we just return an unspecific "Block not found on disk" error and log
  `ERROR: ReadBlockFromDisk: OpenBlockFile failed for FlatFilePos(nFile=-1, nPos=0) `
  which suggest something went wrong even though this is a completely normal and expected situation.

  This PR improves the error message and stops calling `ReadRawBlockFromDisk()`, when we already know from the header that the block is not available on disk.
  Similarly, it prevents all other rpcs from calling blockstorage read functions unless we expect the data to be there, so that `LogError()` will only be thrown when there is an actual file system problem.

  I'm not  completely sure if the cause is important enough to change the wording of the rpc error, that some scripts may rely on.
  If reviewers prefer it, an alternative solution would be to keep returning the current "Block not found on disk" error, but return it immediately instead of calling `ReadRawBlockFromDisk`, which would at least prevent the log error and also be an improvement in my opinion.

ACKs for top commit:
  fjahr:
    re-ACK 6a1aa510e3
  achow101:
    ACK 6a1aa510e3
  andrewtoth:
    re-ACK 6a1aa510e3

Tree-SHA512: 491aef880e8298a05841c4bf8eb913ef84820d1ad5415fd17d9b441bff181959ebfdd432b5eb8347dc9c568433f9a2384ca9d84cd72c79d8a58323ca117538fe
2024-09-16 16:17:59 -04:00
Ava Chow
87d54500bf
Merge bitcoin/bitcoin#30892: test: Check already deactivated network stays suspended after dumptxoutset
72c9a1fe94 test: Check that network stays suspended after dumptxoutset if it was off before (Fabian Jahr)

Pull request description:

  Follow-up to #30817 which covered the robustness of `dumptxoutset`: network is deactivated during the run but re-activated even when an issue was encountered. But it did not cover the case if the user had deactivated the network themselves before. In that case the user may want the network to stay off so the network is not reactivated after `dumptxoutset` finishes. A test for this behavior is added here.

ACKs for top commit:
  achow101:
    ACK 72c9a1fe94
  pablomartin4btc:
    ACK 72c9a1fe94
  theStack:
    utACK 72c9a1fe94
  tdb3:
    tested ACK 72c9a1fe94

Tree-SHA512: 18a57c5782e99a018414db0597e88debeb32666712c2a75dddbb55135d8f1ddd1eeacba8bbbd35fc03b6c4ab0522fe074ec08edea729560b018f51efabf00e89
2024-09-13 16:12:19 -04:00
Martin Zumsande
6a1aa510e3 rpc: check block index before reading block / undo data
This avoids low-level log errors that are supposed to only occur when
there is an actual problem with the block on disk missing unexpectedly,
but not in the case where the block and/or undo data are expected not to be there.

It changes behavior such that in the first case (block index indicates
data is available but retrieving it fails) an error is thrown.

It also adjusts a functional tests that tried to simulate not
having undo data (but having block data) by deleting the undo file.
This situation should occur reality because block and undo data are pruned together.
Instead, test this situation with a block that hasn't been connected.
2024-09-13 10:50:49 -04:00
Martin Zumsande
6cbf2e5f81 rpc: Improve gettxoutproof error when only header is available. 2024-09-13 10:50:49 -04:00
Martin Zumsande
69fc867ea1 test: add coverage to getblock and getblockstats
also removes an unnecessary newline.

Co-authored-by: tdb3 <106488469+tdb3@users.noreply.github.com>
2024-09-13 10:50:49 -04:00
Martin Zumsande
5290cbd585 rpc: Improve getblock / getblockstats error when only header is available.
This improves the error message of the getblock and getblockstats rpc and prevents calls to
ReadRawBlockFromDisk(), which are unnecessary if we know
from the header nStatus field that the block is not available.
2024-09-13 10:50:49 -04:00
Fabian Jahr
72c9a1fe94
test: Check that network stays suspended after dumptxoutset if it was off before 2024-09-12 23:21:58 +02:00
Ava Chow
cf0120ff02
Merge bitcoin/bitcoin#30880: test: Wait for local services to update in feature_assumeutxo
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
19f4a7c95a test: Wait for local services to update in feature_assumeutxo (Fabian Jahr)

Pull request description:

  Closes #30878

  It seems like there is a race where the block is stored locally and `getblock` does not error anymore, but `ActivateBestChain` has not finished yet, so the local services are not updated yet either. Fix this by waiting for the local services to update.

  Can be reproduced locally by adding the sleep here:

  ```cpp
  ──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
  src/validation.cpp:3567: bool Chainstate::ActivateBestChain(BlockValidationState& state, std::shared_ptr< │
  ──────────────────────────────────────────────────────────────────────────────────────────────────────────┘
          }

          if (WITH_LOCK(::cs_main, return m_disabled)) {
              std::this_thread::sleep_for(std::chrono::seconds(10));
              // Background chainstate has reached the snapshot base block, so exit.

              // Restart indexes to resume indexing for all blocks unique to the snapshot
  ```

ACKs for top commit:
  maflcko:
    review-only ACK 19f4a7c95a
  achow101:
    ACK 19f4a7c95a
  pablomartin4btc:
    tACK 19f4a7c95a
  furszy:
    Code review ACK [19f4a7c](19f4a7c95a).

Tree-SHA512: 70dad3795988956c5e20f2d2d895fb56c5e3ce257c7547d3fd729c88949f0e24cb34594da1537bce8794ad02b2db44e7e46e995aa32539cd4dd84c4f1d4bb1c4
2024-09-12 14:52:14 -04:00
Ryan Ofsky
e46bebb444
Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure
fa5bc450d5 util: Use compile-time check for LogConnectFailure (MarcoFalke)
fa7087b896 util: Use compile-time check for FatalErrorf (MarcoFalke)
faa62c0112 util: Add ConstevalFormatString (MarcoFalke)
fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py (MarcoFalke)

Pull request description:

  The `test/lint/lint-format-strings.py` was designed to count the number of format specifiers and assert that they are equal to the number of parameters passed to the format function. The goal seems reasonable, but the implementation has many problems:

  * It is written in Python, meaning that C++ code can not be parsed correctly. Currently it relies on brittle regex and string parsing.
  * Apart from the parsing errors, there are also many logic errors. For example, `count_format_specifiers` allows a mix of positional specifiers and non-positional specifiers, which can lead to runtime format bugs. Also, `count_format_specifiers` silently skipped over "special" format specifiers, which are valid in tinyformat, which again can lead to runtime format bugs being undetected.
  * The brittle logic has a history of breaking in pull requests that are otherwise fine. This causes the CI to fail and the pull request being blocked from progress until the bug in the linter is fixed, or the code is rewritten to work around the bug.
  * It is only run in the CI, or when the developer invokes the script. It would be better if the developer got the error message at compile-time, directly when writing the code.

  Fix all issues by using a `consteval` checked format string in `FatalErrorf` and `LogConnectFailure`.

  This is the first step toward https://github.com/bitcoin/bitcoin/issues/30530 and a follow-up will apply the approach to the other places.

ACKs for top commit:
  stickies-v:
    re-ACK fa5bc450d5
  l0rinc:
    ACK fa5bc450d5
  hodlinator:
    ACK fa5bc450d5
  ryanofsky:
    Code review ACK fa5bc450d5

Tree-SHA512: d6189096b16083143687ed1b1559cf4f92f97dd87bc5d00673e44f4fb9fce7bb7b215cfdfc39b6e6a24f0b75a79a03ededce966639e554f7172e1fc22cf015ae
2024-09-12 13:21:53 -04:00
Fabian Jahr
19f4a7c95a
test: Wait for local services to update in feature_assumeutxo 2024-09-12 16:30:50 +02:00
merge-script
7d43bca052
Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional runner
72b46f28bf test: fix exclude parsing for functional runner (Max Edwards)

Pull request description:

  This restores previous behaviour of being able to exclude a test by name without having to specify .py extension.

  It was noticed in https://github.com/bitcoin/bitcoin/issues/30851 that tests were no longer being excluded.

  PR https://github.com/bitcoin/bitcoin/pull/30244 introduced being able to exclude a specific tests based on args (such as `--exclude "rpc_bind.py --ipv6`) but it made the wrong assumption that test names intended to be excluded would include the .py extension.

  The following https://github.com/bitcoin/bitcoin/pull/30244#issuecomment-2344009687 shows that this is not how the `--exclude` flag was used in CI.

  https://github.com/bitcoin/bitcoin/pull/30244#issuecomment-2344009687 gave three examples of `--exclude` being used in CI so I compared the number of tests that the runner would run for these three examples in three situations, before #30244 was introduced, in master today and with this PR applied.

  Example:

  `--previous-releases --coverage --extended --exclude feature_dbcrash`

  Test count:
  Before #30244 introduced: 314
  Master: 315
  With this PR: 314

  Example:

  `--exclude feature_init,rpc_bind,feature_bind_extra`

  Test count:
  Before #30244 introduced: 306
  Master 311
  With this PR: 306

  Example:

  `--exclude rpc_bind,feature_bind_extra`

  Before #30244 introduced:  307
  Master 311
  With this PR: 307

  I've also tested that the functionality introduced with #30244 remains and we can still exclude specific tests by argument.

ACKs for top commit:
  maflcko:
    review ACK 72b46f28bf
  willcl-ark:
    ACK 72b46f28bf

Tree-SHA512: 37c0e3115f4e3efdf9705f4ff8cd86a5cc906aacc1ab26b0f767f5fb6a953034332b29b0667073f8382a48a2fe9d649b7e60493daf04061260adaa421419d8c8
2024-09-12 15:30:34 +01:00
MarcoFalke
fa5bc450d5
util: Use compile-time check for LogConnectFailure 2024-09-12 15:01:35 +02:00
MarcoFalke
fa7087b896
util: Use compile-time check for FatalErrorf 2024-09-12 15:01:20 +02:00
Max Edwards
72b46f28bf test: fix exclude parsing for functional runner
This restores previous behaviour of being able to exclude a test by name without having to specify .py extension.
2024-09-12 13:42:34 +01:00
merge-script
a5e99669cc
Merge bitcoin/bitcoin#30733: test: remove unused src_dir param from run_tests after CMake migration
Some checks are pending
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
2ad560139b Remove unused src_dir param from run_tests (Lőrinc)

Pull request description:

  The `src_dir` usage was removed in  a8a2e364ac (diff-437d7f6e9f2229879b60aae574a8217f14c643bbf3cfa9225d8011d6d52df00cL598), making the parameter unused.

Top commit has no ACKs.

Tree-SHA512: 1fd8b93811b4ab467ba5a160a4fe204e9606e1bf237c7595ed6f8b7821cf59d2a776c0e1e154852a45b2a35e5bdbd8996314e4f63a9c750f21b9a17875cb636a
2024-09-12 12:17:06 +01:00
Ava Chow
349632e022
Merge bitcoin/bitcoin#30807: Fix peers abruptly disconnecting from AssumeUTXO nodes during IBD
Some checks are pending
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
992f83bb6f test: add coverage for assumeUTXO honest peers disconnection (furszy)
6d5812e5c8 assumeUTXO: fix peers disconnection during sync (furszy)

Pull request description:

  Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
  address through the network before completing the background chain sync.
  This, combined with the advertising of full-node service (`NODE_NETWORK`), can
  result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
  and requesting an historical block the node does not have. This behavior leads to
  an abrupt disconnection due to perceived unresponsiveness from the AssumeUTXO
  node.

  This lack of response occurs because nodes ignore `getdata` requests when they do
  not have the block data available (further discussion can be found in #30385).

  Fix this by refraining from signaling full-node service support while the
  background chain is being synced. During this period, the node will only
  signal `NODE_NETWORK_LIMITED` support. Then, full-node (`NODE_NETWORK`)
  support will be re-enabled once the background chain sync is completed.

  Thanks mzumsande for a post-#30385 convo too.

  Testing notes:
  Just cherry-pick the second commit (bb08c22) on master.
  It will fail there, due to the IBD node requesting historical blocks to the snapshot
  node - which is bad because the snapshot node will ignore the requests and
  stall + disconnect after some time.

ACKs for top commit:
  achow101:
    ACK 992f83bb6f
  naumenkogs:
    ACK 992f83bb6f
  mzumsande:
    ACK 992f83bb6f

Tree-SHA512: fef525d1cf3200c2dd89a346be9c82d77f2e28ddaaea1f490a435e180d1a47a371cadea508349777d740ab56e94be536ad8f7d61cc81f6550c58b609b3779ed3
2024-09-11 13:37:40 -04:00
MarcoFalke
fae7b83eb5
lint: Remove forbidden functions from lint-format-strings.py
Given that all of them are forbidden by the
test/lint/lint-locale-dependence.py check, they can be removed.
2024-09-11 16:52:57 +02:00
glozow
0725a37494
Merge bitcoin/bitcoin#30805: test: Add explicit onion bind to p2p_permissions
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
082779d606 test: Add explicit onion bind to p2p_permissions (Ava Chow)

Pull request description:

  When the bind option is replaced in the bitcoin.conf, bitcoind will attempd to bind to the default tor listening port. If another bitcoind is running that is already bound to that port, the bind will fail which, since #22729, causes the test to fail.

  This failure can be avoided by explicitly binding the tor port when the bind is removed.

ACKs for top commit:
  tdb3:
    ACK 082779d606
  theStack:
    re-ACK 082779d606
  glozow:
    ACK 082779d606

Tree-SHA512: 4acb69ea2e00aeacf9e7c9ab9595ceaf0e0d2adbd795602034b2184197d9bad54c7bc9f3da43ef9c52a71869fe96ba8c87fc5b7c37880f258f5a2aaab2b4046c
2024-09-10 21:49:47 -04:00
furszy
992f83bb6f
test: add coverage for assumeUTXO honest peers disconnection
Exercising and verifying the following points:

1. An IBD node can sync headers from an AssumeUTXO node at
   any time.

2. IBD nodes do not request historical blocks from AssumeUTXO
   nodes while they are syncing the background-chain.

3. The assumeUTXO node dynamically adjusts the network services
   it offers according to its state.

4. IBD nodes can fully sync from AssumeUTXO nodes after they
   finish the background-chain sync.
2024-09-10 18:08:33 -03:00
furszy
6d5812e5c8
assumeUTXO: fix peers disconnection during sync
Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
address through the network before completing the background chain sync.
This, combined with the advertising of full-node service (NODE_NETWORK), can
result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
and requesting an historical block the node does not have. This behavior leads to
an abrupt disconnection due to perceived unresponsiveness (lack of response)
from the AssumeUTXO node.

This lack of response occurs because nodes ignore getdata requests when they do
not have the block data available (further discussion can be found in PR 30385).

Fix this by refraining from signaling full-node service support while the
background chain is being synced. During this period, the node will only
signal 'NODE_NETWORK_LIMITED' support. Then, full-node ('NODE_NETWORK')
support will be re-enabled once the background chain sync is completed.
2024-09-10 18:08:32 -03:00
Ava Chow
082779d606 test: Add explicit onion bind to p2p_permissions
When the bind option is replaced in the bitcoin.conf, bitcoind will
attempd to bind to the default tor listening port. If another bitcoind
is running that is already bound to that port, the bind will fail which,
since #22729, causes the test to fail.

This failure can be avoided by explicitly binding the tor port when the
bind is removed.
2024-09-10 16:32:08 -04:00
Ava Chow
712a2b5453
Merge bitcoin/bitcoin#30817: test: Add coverage for dumptxoutset failure robustness
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
c2b779da4e refactor: Manage dumptxoutset RAII classes with std::optional (Fabian Jahr)
4b5bf335ad test: Add coverage for failing dumptxoutset behavior (Fabian Jahr)

Pull request description:

  This adds a test that checks that network activity is not suspended if dumptxoutset fails in the middle of its process which is implemented with the `NetworkDisable` RAII class. I would have liked to add coverage for the `TemporaryRollback` RAII class but that seems a lot more tricky since the failure needs to happen at some point after the rollback and on the scale of our test chain here I couldn't find a way to do it yet. This was requested by pablomartin4btc here: https://github.com/bitcoin/bitcoin/pull/30808#pullrequestreview-2280450117. To test the test you can comment out the content of the destructor of `NetworkDisable`.

  It also addresses the feedback by ryanofsky to use `std::optional` instead of `std::unique_ptr` for the management of the RAII object: https://github.com/bitcoin/bitcoin/pull/30808#discussion_r1744149228

ACKs for top commit:
  achow101:
    ACK c2b779da4e
  pablomartin4btc:
    cr & tACK c2b779da4e
  tdb3:
    ACK c2b779da4e
  BrandonOdiwuor:
    Code Review ACK c2b779da4e
  theStack:
    ACK c2b779da4e

Tree-SHA512: 9556e75014a2599bb870b70faf887608b332f2312626333f771d4ec11c04f863a2cf17e223ec473d4e8b0c9e8008394a4e0c321561f7ef3a2eec713dcfaea58a
2024-09-09 13:02:51 -04:00