Commit graph

42743 commits

Author SHA1 Message Date
MarcoFalke
fa39b1ca63
doc: move-only logging warning
Put the warning closer to where it is relevant. That is, put it close to
the functions that actually do unconditional logging.

Also, remove a stray empty line.
2024-09-12 19:33:46 +02:00
Ryan Ofsky
e46bebb444
Merge bitcoin/bitcoin#30546: util: Use consteval checked format string in FatalErrorf, LogConnectFailure
fa5bc450d5 util: Use compile-time check for LogConnectFailure (MarcoFalke)
fa7087b896 util: Use compile-time check for FatalErrorf (MarcoFalke)
faa62c0112 util: Add ConstevalFormatString (MarcoFalke)
fae7b83eb5 lint: Remove forbidden functions from lint-format-strings.py (MarcoFalke)

Pull request description:

  The `test/lint/lint-format-strings.py` was designed to count the number of format specifiers and assert that they are equal to the number of parameters passed to the format function. The goal seems reasonable, but the implementation has many problems:

  * It is written in Python, meaning that C++ code can not be parsed correctly. Currently it relies on brittle regex and string parsing.
  * Apart from the parsing errors, there are also many logic errors. For example, `count_format_specifiers` allows a mix of positional specifiers and non-positional specifiers, which can lead to runtime format bugs. Also, `count_format_specifiers` silently skipped over "special" format specifiers, which are valid in tinyformat, which again can lead to runtime format bugs being undetected.
  * The brittle logic has a history of breaking in pull requests that are otherwise fine. This causes the CI to fail and the pull request being blocked from progress until the bug in the linter is fixed, or the code is rewritten to work around the bug.
  * It is only run in the CI, or when the developer invokes the script. It would be better if the developer got the error message at compile-time, directly when writing the code.

  Fix all issues by using a `consteval` checked format string in `FatalErrorf` and `LogConnectFailure`.

  This is the first step toward https://github.com/bitcoin/bitcoin/issues/30530 and a follow-up will apply the approach to the other places.

ACKs for top commit:
  stickies-v:
    re-ACK fa5bc450d5
  l0rinc:
    ACK fa5bc450d5
  hodlinator:
    ACK fa5bc450d5
  ryanofsky:
    Code review ACK fa5bc450d5

Tree-SHA512: d6189096b16083143687ed1b1559cf4f92f97dd87bc5d00673e44f4fb9fce7bb7b215cfdfc39b6e6a24f0b75a79a03ededce966639e554f7172e1fc22cf015ae
2024-09-12 13:21:53 -04:00
Ryan Ofsky
be768dbd18
Merge bitcoin/bitcoin#30618: test: support std::optional in BOOST_CHECK_* and increase FromUserHex fuzz feature coverage
1eac96a503 Compare FromUserHex result against other hex validators and parsers (Lőrinc)
19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160 (Lőrinc)
743ac30e34 Add std::optional support to Boost's equality check (Lőrinc)

Pull request description:

  Enhanced `FromUserHex` coverage by:

  * Added `std::optional` support to `BOOST_CHECK_EQUAL`, allowing direct comparisons of `std::optional<T>` with other `T` expected values.
  * Increased fuzz testing for hex parsing to validate against other hex validators and parsers.

  ----

  * Use BOOST_CHECK_EQUAL for https://github.com/bitcoin/bitcoin/pull/30569#discussion_r1706637780 arith_uint256, uint256, uint160

  Example error before:
  > unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
  test/validation_chainstatemanager_tests.cpp:781: last checkpoint

  after:
  > test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]

ACKs for top commit:
  stickies-v:
    re-ACK 1eac96a503
  ryanofsky:
    Code review ACK 1eac96a503. Only changes since last review were auto type and fuzz test tweaks.
  hodlinator:
    ACK 1eac96a503

Tree-SHA512: f1d2c65f0ee4e97830700be5b330189207b11ed0c89a8cebf0f97d43308402a6b3732e10130c79a0c044f7d2eeabfb5359990825aadf02c4ec19428dcd982b00
2024-09-12 12:36:37 -04:00
merge-script
07c7c96022
Merge bitcoin/bitcoin#30883: build: Revert "Minimize I/O operations in GenerateHeaderFrom{Json,Raw}.cmake"
fdeb717e78 Revert "build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake`" (Hennadii Stepanov)

Pull request description:

  This reverts commit b07fe666f2 from https://github.com/bitcoin/bitcoin/pull/30842.

  Fixes https://github.com/bitcoin/bitcoin/issues/30881.

  Apparently, the `string(APPEND ...)` command isn't optimized for large strings.

ACKs for top commit:
  maflcko:
    review ACK fdeb717e78

Tree-SHA512: ad5c3d49d3395ab318edcd7c9a98090838bec0cd3c1f1cc6ebc6f4262df2494f605458b523251bf5e590bbcfda15ed963f0a814678135ce4cc2dca9a108d20c7
2024-09-12 16:53:31 +01:00
merge-script
24817e8b15
Merge bitcoin/bitcoin#30814: kernel: Create usable static kernel library
0dd16d7118 build: Add a pkg-config file for libbitcoinkernel (TheCharlatan)
45be32f838 build: Produce a usable static kernel library (TheCharlatan)

Pull request description:

  Since the move to cmake, the kernel static library that is installed after a cmake --install build is unusable. It lacks symbols for the internal libraries, besides those defined in the kernel library target.

  Fix this by explicitly installing all the required internal static libraries. To make usage of these installed libraries easy, add a pkg-config file that can be used during linking.

  This patch can be tested with:

  ```
  cmake -B build -DBUILD_SHARED_LIBS=OFF -DBUILD_KERNEL_LIB=ON
  cmake --build build
  cmake --install build
  g++ -std=c++20 -o test_chainstate src/bitcoin-chainstate.cpp -I/home/drgrid/bitcoin/src $(pkg-config --libs --static libbitcoinkernel)
  ```

  Attempts to solve #30801

ACKs for top commit:
  hebasto:
    ACK 0dd16d7118.
  fanquake:
    ACK 0dd16d7118 - this looks like a good place to start.
  ryanofsky:
    Code review ACK 0dd16d7118

Tree-SHA512: 92f7bc959584bdc595f4aa6d0ab133355481075fe8564224fd7ac122fd7bdd75f98cf26ef0a6a7d84fd552d2258ddca1b674eca91122469a58bacc5f0a0ec2ef
2024-09-12 16:39:34 +01:00
Hennadii Stepanov
fdeb717e78
Revert "build: Minimize I/O operations in GenerateHeaderFrom{Json,Raw}.cmake"
This reverts commit b07fe666f2.
2024-09-12 16:34:57 +01:00
Sergi Delgado Segura
07f4cebe57 refactor: move m_is_inbound out of CNodeState
`m_is_inbound` cannot be changed throughout the life of a `Peer`. However, we
are currently storing it in `CNodeState`, which requires locking `cs_main` in
order to access it. This can be moved to the outside scope and only require
`m_peer_mutex`.

This is a refactor in preparation for Erlay reworks.
2024-09-12 11:20:44 -04:00
Fabian Jahr
19f4a7c95a
test: Wait for local services to update in feature_assumeutxo 2024-09-12 16:30:50 +02:00
merge-script
7d43bca052
Merge bitcoin/bitcoin#30872: test: fix exclude parsing for functional runner
72b46f28bf test: fix exclude parsing for functional runner (Max Edwards)

Pull request description:

  This restores previous behaviour of being able to exclude a test by name without having to specify .py extension.

  It was noticed in https://github.com/bitcoin/bitcoin/issues/30851 that tests were no longer being excluded.

  PR https://github.com/bitcoin/bitcoin/pull/30244 introduced being able to exclude a specific tests based on args (such as `--exclude "rpc_bind.py --ipv6`) but it made the wrong assumption that test names intended to be excluded would include the .py extension.

  The following https://github.com/bitcoin/bitcoin/pull/30244#issuecomment-2344009687 shows that this is not how the `--exclude` flag was used in CI.

  https://github.com/bitcoin/bitcoin/pull/30244#issuecomment-2344009687 gave three examples of `--exclude` being used in CI so I compared the number of tests that the runner would run for these three examples in three situations, before #30244 was introduced, in master today and with this PR applied.

  Example:

  `--previous-releases --coverage --extended --exclude feature_dbcrash`

  Test count:
  Before #30244 introduced: 314
  Master: 315
  With this PR: 314

  Example:

  `--exclude feature_init,rpc_bind,feature_bind_extra`

  Test count:
  Before #30244 introduced: 306
  Master 311
  With this PR: 306

  Example:

  `--exclude rpc_bind,feature_bind_extra`

  Before #30244 introduced:  307
  Master 311
  With this PR: 307

  I've also tested that the functionality introduced with #30244 remains and we can still exclude specific tests by argument.

ACKs for top commit:
  maflcko:
    review ACK 72b46f28bf
  willcl-ark:
    ACK 72b46f28bf

Tree-SHA512: 37c0e3115f4e3efdf9705f4ff8cd86a5cc906aacc1ab26b0f767f5fb6a953034332b29b0667073f8382a48a2fe9d649b7e60493daf04061260adaa421419d8c8
2024-09-12 15:30:34 +01:00
merge-script
cf786eccd7
Merge bitcoin/bitcoin#30865: build: Skip secp256k1 ctime tests when tests are not being built
23eedc5d1e build: Skip secp256k1 ctime tests when tests are not being built (Hennadii Stepanov)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/pull/30791#issuecomment-2340860619:
  > Building with a fuzz engine fails, because the ctime tests are auto-detected in cmake, based on whether or not valgrind-devel is installed or not.

ACKs for top commit:
  maflcko:
    re-review ACK 23eedc5d1e
  fanquake:
    ACK 23eedc5d1e

Tree-SHA512: bfc0f2798acd36be9c52073d578b42c002606c60ef3fe8ef633eaea4f5382a3e9765d31637e4c25d8b71fd70473b29c24af4732e55e5183f27b48725b61fa15b
2024-09-12 15:29:33 +01:00
Hennadii Stepanov
23eedc5d1e
build: Skip secp256k1 ctime tests when tests are not being built
Co-authored-by: fanquake <fanquake@gmail.com>
2024-09-12 14:24:26 +01:00
MarcoFalke
fa5bc450d5
util: Use compile-time check for LogConnectFailure 2024-09-12 15:01:35 +02:00
MarcoFalke
fa7087b896
util: Use compile-time check for FatalErrorf 2024-09-12 15:01:20 +02:00
MarcoFalke
faa62c0112
util: Add ConstevalFormatString
The type is used to wrap a format string once it has been compile-time
checked to contain the right number of format specifiers.
2024-09-12 15:00:53 +02:00
Max Edwards
72b46f28bf test: fix exclude parsing for functional runner
This restores previous behaviour of being able to exclude a test by name without having to specify .py extension.
2024-09-12 13:42:34 +01:00
merge-script
a5e99669cc
Merge bitcoin/bitcoin#30733: test: remove unused src_dir param from run_tests after CMake migration
Some checks are pending
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
2ad560139b Remove unused src_dir param from run_tests (Lőrinc)

Pull request description:

  The `src_dir` usage was removed in  a8a2e364ac (diff-437d7f6e9f2229879b60aae574a8217f14c643bbf3cfa9225d8011d6d52df00cL598), making the parameter unused.

Top commit has no ACKs.

Tree-SHA512: 1fd8b93811b4ab467ba5a160a4fe204e9606e1bf237c7595ed6f8b7821cf59d2a776c0e1e154852a45b2a35e5bdbd8996314e4f63a9c750f21b9a17875cb636a
2024-09-12 12:17:06 +01:00
MarcoFalke
fa252da0b9
ci: Remove hardcoded CCACHE_DIR in cirrus
This makes it easier to overwrite the value.

Also, drop the dot in the CCACHE_DIR fallback value, because the folder
in the scratch dir does not need and probably should not be hidden.
2024-09-12 12:34:15 +02:00
MarcoFalke
fa146904e1
ci: Bump default CCACHE_MAXSIZE to 500M
This also allows to drop individually hardcoded values, which are
impossible to overwrite and hard to maintain.
2024-09-12 12:34:02 +02:00
MarcoFalke
aaaa7cf8ba
cirrus: Drop CCACHE_NOHASHDIR
Now that the build path is constant again after commit fa193f5dfc
normalized all folders, this can be dropped.
2024-09-12 12:33:36 +02:00
merge-script
0c1e507278
Merge bitcoin/bitcoin#30871: build: Add more cmake presets
f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan) (Pieter Wuille)

Pull request description:

  Add three more cmake presets to the project-wide `CMakePresets.json` file:
  * `dev-mode`: enables all features and dependencies
  * `libfuzzer`: builds for fuzzing with libfuzzer and the typical sanitizers (but not the optional ones that require suppressions) enabled.
  * `libfuzzer-nosan`: builds for fuzzing with libfuzzer and no (other) sanitizers

  ... and then uses these in some documentation.

ACKs for top commit:
  ryanofsky:
    Code review ACK f15e817811. This change is much needed to simplify my command line.
  TheCharlatan:
    ACK f15e817811

Tree-SHA512: a5f67bb7119fd36832ca5eb7189db04bfaf88f954aa461bfb2aeb866469057b0d0272835c418bc3a264c30dd8fba6d2e2cc8a6741a889f28f52c1c09b3ba9704
2024-09-12 11:33:15 +01:00
merge-script
fcb61bbc8d
Merge bitcoin/bitcoin#27038: security-check: test for _FORTIFY_SOURCE usage in release binaries
be4f78275f contrib: test for FORTIFY_SOURCE in security-check.py (fanquake)

Pull request description:

  Test for the existence of fortified functions in the ELF release binaries.
  Currently skips `bitcoin-util` and checks for RISC-V.

ACKs for top commit:
  TheCharlatan:
    ACK be4f78275f

Tree-SHA512: decea5f359f1e673aa0119916f674f409a13b69db7da366cd95c1540201e117ff5a979da67bc2517fe786c2ac23d1006a9aaf662d7eadeec35da6aae4998c065
2024-09-12 11:32:31 +01:00
merge-script
85833cf05f
Merge bitcoin/bitcoin#30847: test: Drop no longer needed workarounds
5c80192ff6 test: Drop no longer needed workarounds (Hennadii Stepanov)

Pull request description:

  This PR deletes the workarounds introduced in https://github.com/bitcoin/bitcoin/pull/16564 and https://github.com/bitcoin/bitcoin/pull/15382, as `ctest` skips these cases gracefully: 5c80192ff6/src/test/CMakeLists.txt (L201-L203)

ACKs for top commit:
  kevkevinpal:
    ACK [5c80192](5c80192ff6)
  fanquake:
    ACK 5c80192ff6. Looks correct:

Tree-SHA512: c47c606ecf7d64016b3c6353c3d4898350edc2caeac494dfd44484417f500a73f0c88c39f0f24651f3a02ef31ed9ca5c70d938bb9a8ca1eea54927e4d6a8fcd2
2024-09-12 11:28:27 +01:00
MarcoFalke
fa7ca182a9
ci: Print inner env 2024-09-12 12:28:23 +02:00
merge-script
11e2f9fff4
Merge bitcoin/bitcoin#30835: build: Introduce "Kernel" installation component
7b04fabe2d build: Introduce "Kernel" installation component (Hennadii Stepanov)

Pull request description:

  This PR enables building and installing only `libbitcoinkernel`, without the need to disable other targets during the project build system generation:

  ```
  $ rm -rf build && cmake -B build -DBUILD_KERNEL_LIB=ON
  $ cmake --build build --target bitcoinkernel
  $ cmake --install build --component Kernel --prefix /home/hebasto/INSTALL
  -- Install configuration: "RelWithDebInfo"
  -- Installing: /home/hebasto/INSTALL/lib/libbitcoinkernel.so
  ```

  Please note, that only the `bitcoinkernel` target is being built.

  Related to https://github.com/bitcoin/bitcoin/issues/30801 and https://github.com/bitcoin/bitcoin/pull/30814.

ACKs for top commit:
  TheCharlatan:
    ACK 7b04fabe2d
  ryanofsky:
    Code review ACK 7b04fabe2d

Tree-SHA512: eac114dde059e47c91938a4a9108fc0fc693b5342ed3b6ecb971615be8ad3225b9985aae12d6ad18e673edf1bd39a5ecf259c1b61734f221669091bf2ce93a67
2024-09-12 10:58:52 +01:00
merge-script
db8350b0e3
Merge bitcoin/bitcoin#30803: build: Minor build system fixes and amendments
1cc93fe7b4 build: Delete dead code that implements `IF_CHECK_FAILED` option (Hennadii Stepanov)
341ad23809 build: Delete MSVC special case for `BUILD_FOR_FUZZING` option (Hennadii Stepanov)
fdad128b52 build: Stop enabling CMake's CMP0141 policy (Hennadii Stepanov)
b2a6f545b4 doc: Drop `ctest` command from Windows cross-compiling instructions (Hennadii Stepanov)
73b618582d build: Print `CMAKE_CXX_COMPILER_ARG1` in summary (Hennadii Stepanov)
f03c942095 build, test: Add missed log options (Hennadii Stepanov)
6f2cb0eafd doc: Amend comment about ZeroMQ config files (Hennadii Stepanov)

Pull request description:

  This PR addresses the following comments:
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742342524
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1728692369
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1736110362
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742931121
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1747723657
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1742328675
  - https://github.com/bitcoin/bitcoin/pull/30454#discussion_r1723106474

ACKs for top commit:
  sipsorcery:
    tACK 1cc93fe7b4 (win11 msvc).
  maflcko:
    re-ACK 1cc93fe7b4

Tree-SHA512: a390797bb4d3b7eb9163653b6c9c324e7a01090f6cdda74df7349a24a5c4a2084e5912878747f56561315afc70cae9adb1c363f47ceb0af96004ea591d25171b
2024-09-12 10:30:06 +01:00
merge-script
a86e7a476d
Merge bitcoin/bitcoin#30838: build: Use CMake's default permissions in macOS deploy target
5ba03e7d35 build: Use CMake's default permissions in macOS `deploy` target (Hennadii Stepanov)

Pull request description:

  This PR ensures that the file permissions in macOS `zip` archives are independent of the user's `umask` value.

  Fixes https://github.com/bitcoin/bitcoin/issues/30815.

ACKs for top commit:
  fanquake:
    ACK 5ba03e7d35 - I'm going to merge this now so we return to usable (comparable) guix builds.

Tree-SHA512: 78f724cd3ffd5c1fd5fc1b4832f1e8154c62723f3de5ac9599f44715cbd08a3dfbb806801411c55069773d2e34c9f8cab25585dbad2f032c36b68dd83cb51847
2024-09-12 10:17:08 +01:00
merge-script
f0eb63399a
Merge bitcoin/bitcoin#30841: ci: Post CMake-migration fixes and amendments
c45186ca54 ci: Switch from `make` to `cmake --build` (Hennadii Stepanov)
6e5f33af58 ci: Handle log files regardless of CMake's version (Hennadii Stepanov)

Pull request description:

  This PR addresses the change in logging that [happened](https://cmake.org/cmake/help/latest/release/3.26.html#configure-log) in CMake 3.26.

  Additionally, the `make` invocation replaced with `cmake --build`.

  Here are examples of the CI logs:
  - for a an error during the build system generation: https://cirrus-ci.com/build/5210987156996096
  - for a compiler error: https://cirrus-ci.com/build/4617660913156096

ACKs for top commit:
  maflcko:
    review ACK c45186ca54
  fanquake:
    ACK c45186ca54

Tree-SHA512: 2096f08c482ab9e10056cd4ec694ce40996243e2a1af2212dfff8cccbf0f51391d9a3dc396f7bba4f2877072a13a42bf667a02a44eab44e917aafb14d04e8e39
2024-09-12 09:54:20 +01:00
merge-script
155963768a
Merge bitcoin/bitcoin#30842: build: Minimize I/O operations in GenerateHeaderFrom{Json,Raw}.cmake
b07fe666f2 build: Minimize I/O operations in `GenerateHeaderFrom{Json,Raw}.cmake` (Hennadii Stepanov)

Pull request description:

  This PR aims to reduce build time by replacing multiple `file(WRITE|APPEND ...)` commands with a single `file(WRITE ...)` command.

  Due to differences in implementation (e.g., filesystem design, system calls, caching), a noticeable improvement in build time is observed only on Windows.

  Additionally, the code has been refactored to remove the `remainder` local variables.

ACKs for top commit:
  sipsorcery:
    tACK b07fe666f2
  maflcko:
    review ACK b07fe666f2
  TheCharlatan:
    ACK b07fe666f2

Tree-SHA512: 6ed3ae8fe7d8859af38d83918eddf7cb318607787863b95589f4a7a45a36f8c4bd1c01e366078d0515115c121bc857dc63471e52ff26fc49edbc8bb69875e947
2024-09-12 09:37:42 +01:00
merge-script
c773618886
Merge bitcoin/bitcoin#30867: build: Fix ENABLE_WALLET option
0037d53d1a build: Fix `ENABLE_WALLET` option (Hennadii Stepanov)

Pull request description:

  The removed commands were left over from the transition from autodetection to explicit options in the CMake staging branch. These commands prevented the `-DENABLE_WALLET=OFF` option from being work properly when building with depends.

  How to test:
  ```
  $ make -C depends NO_QT=1
  ```

  On the master branch @ c66c68345e:
  ```
  $ rm -rf build && cmake -B build --toolchain depends/x86_64-pc-linux-gnu/toolchain.cmake -DENABLE_WALLET=OFF
  < snip >
  Optional features:
    wallet support ...................... ON
     - descriptor wallets (SQLite) ...... ON
     - legacy wallets (Berkeley DB) ..... ON
    external signer ..................... ON
  < snip >
  ```

  With this PR:
  ```
  $ rm -rf build && cmake -B build --toolchain depends/x86_64-pc-linux-gnu/toolchain.cmake -DENABLE_WALLET=OFF
  < snip >
  Optional features:
    wallet support ...................... OFF
    external signer ..................... ON
  < snip >

ACKs for top commit:
  maflcko:
    review ACK 0037d53d1a
  kevkevinpal:
    ACK [0037d53](0037d53d1a)
  pablomartin4btc:
    tACK 0037d53d1a

Tree-SHA512: 0eb14ef104f12a4205172d646c2af820e04514286b5b9a4ceb59c248ce880198dd4051d669098c46c0c0dce069bb60899d90509bbcae65cbeb958e52564fe920
2024-09-12 09:28:20 +01:00
Ava Chow
349632e022
Merge bitcoin/bitcoin#30807: Fix peers abruptly disconnecting from AssumeUTXO nodes during IBD
Some checks are pending
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
992f83bb6f test: add coverage for assumeUTXO honest peers disconnection (furszy)
6d5812e5c8 assumeUTXO: fix peers disconnection during sync (furszy)

Pull request description:

  Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
  address through the network before completing the background chain sync.
  This, combined with the advertising of full-node service (`NODE_NETWORK`), can
  result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
  and requesting an historical block the node does not have. This behavior leads to
  an abrupt disconnection due to perceived unresponsiveness from the AssumeUTXO
  node.

  This lack of response occurs because nodes ignore `getdata` requests when they do
  not have the block data available (further discussion can be found in #30385).

  Fix this by refraining from signaling full-node service support while the
  background chain is being synced. During this period, the node will only
  signal `NODE_NETWORK_LIMITED` support. Then, full-node (`NODE_NETWORK`)
  support will be re-enabled once the background chain sync is completed.

  Thanks mzumsande for a post-#30385 convo too.

  Testing notes:
  Just cherry-pick the second commit (bb08c22) on master.
  It will fail there, due to the IBD node requesting historical blocks to the snapshot
  node - which is bad because the snapshot node will ignore the requests and
  stall + disconnect after some time.

ACKs for top commit:
  achow101:
    ACK 992f83bb6f
  naumenkogs:
    ACK 992f83bb6f
  mzumsande:
    ACK 992f83bb6f

Tree-SHA512: fef525d1cf3200c2dd89a346be9c82d77f2e28ddaaea1f490a435e180d1a47a371cadea508349777d740ab56e94be536ad8f7d61cc81f6550c58b609b3779ed3
2024-09-11 13:37:40 -04:00
Ava Chow
f6298a878f
Merge bitcoin/bitcoin#30840: docs: Updated debug build instructions for cmake
0b003e1ff7 docs: Updated debug build instructions for cmake (ion-)

Pull request description:

  In the [developer notes](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md) the section on [compiling for debug](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#compiling-for-debugging) displays outdated instructions that were applicable before the move to cmake build system.

  This PR just gives instructions on how to build for debugging in the context of cmake.

ACKs for top commit:
  kevkevinpal:
    ACK [0b003e1](0b003e1ff7)
  achow101:
    ACK 0b003e1ff7
  tdb3:
    ACK 0b003e1ff7

Tree-SHA512: fc8b4824d68e47b3e64320b4ce728df4242d00c2e8c75dcf5daa60af3050c62d94fa9741e447c7d81e72194470ed68a4e19bbee8cf6a1412d6566c7b699cf242
2024-09-11 13:20:19 -04:00
Ava Chow
a8809aeb6e
Merge bitcoin/bitcoin#30870: docs: updated developer notes for --with-sanitizers to -DSANITIZERS
4b1ce3cac8 docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags (kevkevinpal)

Pull request description:

  In the developer notes we are incorrectly using the Autotools `--with-sanitizers` configure flag which we should now be using `cmake -B build -DSANITIZERS=<values>` instead now

ACKs for top commit:
  maflcko:
    review ACK 4b1ce3cac8
  achow101:
    ACK 4b1ce3cac8
  pablomartin4btc:
    ACK 4b1ce3cac8

Tree-SHA512: 029d55d802f6b4a85f3541f9a23e9ac85e6c590e91081204bfa737169138f61877883db51ad99cd8b802b0737eec35df5a33a5351b3e6b2f180f3ad0282d3616
2024-09-11 13:18:10 -04:00
Pieter Wuille
f15e817811 build: add more CMake presets (dev-mode, libfuzzer, libfuzzer-nosan) 2024-09-11 12:51:34 -04:00
ismaelsadeeq
8466329127
chain: simplify deleteRwSettings code and improve it's doc
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-09-11 17:04:29 +01:00
ismaelsadeeq
f8d91f49c7
chain: dont check for null settings value in overwriteRwSetting
- Just call updateRwSetting it will erase the settings when the new
  value is null.
2024-09-11 17:04:28 +01:00
ismaelsadeeq
df601993f2
chain: ensure updateRwSetting doesn't update to a null settings
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-09-11 16:58:40 +01:00
Lőrinc
5e190cd11f Replace CScript _hex_v_u8 appends with _hex
This will skip vector conversion before serializing to the prevector in CScript.
2024-09-11 17:41:27 +02:00
Lőrinc
cac846c2fb Allow CScript's operator<< to accept spans, not just vectors
Extracted existing serialization to append size & data in separate private methods to clarify that it does more than just a simple data insertion.

* the C style casts were changed to static_cast
* `unsigned char` and `uint8_t` were changed to value_type for forward compatibility
* `data + sizeof(data)` was changed to `std::cend`
* data insertion (in AppendData) relies on pointer arithmetic now to enable both `std::span<const value_type>` and `std::span<const std::byte>` operators
* use uint32_t for data size instead of size_t
* used span instead of raw pointers in the new methods

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
2024-09-11 17:41:27 +02:00
Lőrinc
c78d8ff4cb prevector: avoid GCC bogus warnings in insert method
When compiling with GCC 12.2, both `-Warray-bounds` and `-Wstringop-overflow` warnings were triggered in the `prevector::insert` method during CScript prevector operations.

GCC incorrectly assumed that operator new could modify the state of class members, leading to false positives during the memmove operation.

Following the approach in https://gcc.gnu.org/git/gitweb.cgi?p=gcc.git;h=cca06f0d6d76b0, we introduced local copies for the destination pointer in memmove operations. This prevents GCC from misinterpreting memory manipulation as unsafe.

A minimal reproducer triggering this issue in GCC 12.2 and passing in GCC 12.3 can be found at https://godbolt.org/z/8r9TKKoxv.

-------

Full error (with changes from the next commit as well):
```
In file included from /ci_container_base/src/script/script.h:11,
                 from /ci_container_base/src/primitives/transaction.h:11,
                 from /ci_container_base/src/primitives/block.h:9,
                 from /ci_container_base/src/kernel/chainparams.h:11,
                 from /ci_container_base/src/kernel/chainparams.cpp:6:
In member function ‘void prevector<N, T, Size, Diff>::fill(T*, InputIterator, InputIterator) [with InputIterator = const unsigned char*; unsigned int N = 28; T = unsigned char; Size = unsigned int; Diff = int]’,
    inlined from ‘void prevector<N, T, Size, Diff>::insert(iterator, InputIterator, InputIterator) [with InputIterator = const unsigned char*; unsigned int N = 28; T = unsigned char; Size = unsigned int; Diff = int]’ at /ci_container_base/src/prevector.h:395:13,
    inlined from ‘void CScript::AppendData(const prevector<28, unsigned char>::value_type*, size_t)’ at /ci_container_base/src/script/script.h:439:15,
    inlined from ‘CScript& CScript::operator<<(std::span<const std::byte>)’ at /ci_container_base/src/script/script.h:496:17,
    inlined from ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’ at /ci_container_base/src/kernel/chainparams.cpp:76:54:
/ci_container_base/src/prevector.h:216:13: error: writing 65 bytes into a region of size 32 [-Werror=stringop-overflow=]
  216 |             new(static_cast<void*>(dst)) T(*first);
      |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/ci_container_base/src/kernel/chainparams.cpp: In function ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’:
/ci_container_base/src/kernel/chainparams.cpp:76:49: note: destination object ‘<anonymous>’ of size 32
   76 |     const CScript genesisOutputScript = CScript() << "04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5f"_hex << OP_CHECKSIG;
      |                                                 ^
In file included from /usr/lib/gcc/x86_64-w64-mingw32/12-posix/include/c++/cstring:42,
                 from /ci_container_base/src/crypto/common.h:11,
                 from /ci_container_base/src/uint256.h:9,
                 from /ci_container_base/src/consensus/params.h:9,
                 from /ci_container_base/src/kernel/chainparams.h:9:
In function ‘void* memmove(void*, const void*, size_t)’,
    inlined from ‘void prevector<N, T, Size, Diff>::insert(iterator, InputIterator, InputIterator) [with InputIterator = const unsigned char*; unsigned int N = 28; T = unsigned char; Size = unsigned int; Diff = int]’ at /ci_container_base/src/prevector.h:393:16,
    inlined from ‘void CScript::AppendData(const prevector<28, unsigned char>::value_type*, size_t)’ at /ci_container_base/src/script/script.h:439:15,
    inlined from ‘CScript& CScript::operator<<(std::span<const std::byte>)’ at /ci_container_base/src/script/script.h:496:17,
    inlined from ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’ at /ci_container_base/src/kernel/chainparams.cpp:76:54:
/usr/share/mingw-w64/include/string.h:214:33: warning: ‘void* __builtin_memmove(void*, const void*, long long unsigned int)’ offset [65, 35] is out of the bounds [0, 32] of object ‘<anonymous>’ with type ‘CScript’ [-Warray-bounds]
  214 |   return __builtin___memmove_chk(__dst, __src, __n, __mingw_bos(__dst, 0));
      |          ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/ci_container_base/src/kernel/chainparams.cpp: In function ‘CBlock CreateGenesisBlock(uint32_t, uint32_t, uint32_t, int32_t, const CAmount&)’:
/ci_container_base/src/kernel/chainparams.cpp:76:49: note: ‘<anonymous>’ declared here
   76 |     const CScript genesisOutputScript = CScript() << "04678afdb0fe5548271967f1a67130b7105cd6a828e03909a67962e0ea1f61deb649f6bc3f4cef38c4f35504e51ec112de5c384df7ba0b8d578a4c702b6bf11d5f"_hex << OP_CHECKSIG;
      |                                                 ^
```

Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-09-11 17:41:26 +02:00
MarcoFalke
fae7b83eb5
lint: Remove forbidden functions from lint-format-strings.py
Given that all of them are forbidden by the
test/lint/lint-locale-dependence.py check, they can be removed.
2024-09-11 16:52:57 +02:00
kevkevinpal
4b1ce3cac8
docs: updated developer notes for --with-sanitizers to -DSANITIZERS and removed resource for -fsanitze flags 2024-09-11 10:26:58 -04:00
Lőrinc
1eac96a503 Compare FromUserHex result against other hex validators and parsers 2024-09-11 15:41:15 +02:00
Lőrinc
19947863e1 Use BOOST_CHECK_EQUAL for optional, arith_uint256, uint256, uint160
Example error before:
> unknown location:0: fatal error: in "validation_chainstatemanager_tests/chainstatemanager_args": std::bad_optional_access: bad_optional_access
test/validation_chainstatemanager_tests.cpp:781: last checkpoint

after:
> test/validation_chainstatemanager_tests.cpp:801: error: in "validation_chainstatemanager_tests/chainstatemanager_args": check set_opts({"-assumevalid=0"}).assumed_valid_block == uint256::ZERO has failed [std::nullopt != 0000000000000000000000000000000000000000000000000000000000000000]

Also added extra minimum_chainwork test to make it symmetric with assumevalid

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
Co-authored-by: Hodlinator <172445034+hodlinator@users.noreply.github.com>
2024-09-11 15:41:15 +02:00
glozow
0725a37494
Merge bitcoin/bitcoin#30805: test: Add explicit onion bind to p2p_permissions
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
082779d606 test: Add explicit onion bind to p2p_permissions (Ava Chow)

Pull request description:

  When the bind option is replaced in the bitcoin.conf, bitcoind will attempd to bind to the default tor listening port. If another bitcoind is running that is already bound to that port, the bind will fail which, since #22729, causes the test to fail.

  This failure can be avoided by explicitly binding the tor port when the bind is removed.

ACKs for top commit:
  tdb3:
    ACK 082779d606
  theStack:
    re-ACK 082779d606
  glozow:
    ACK 082779d606

Tree-SHA512: 4acb69ea2e00aeacf9e7c9ab9595ceaf0e0d2adbd795602034b2184197d9bad54c7bc9f3da43ef9c52a71869fe96ba8c87fc5b7c37880f258f5a2aaab2b4046c
2024-09-10 21:49:47 -04:00
Hennadii Stepanov
0037d53d1a
build: Fix ENABLE_WALLET option
The removed commands were left over from the transition from
autodetection to explicit options. These commands prevented the
`-DENABLE_WALLET=OFF` option from being work properly when building with
depends.
2024-09-10 22:13:09 +01:00
furszy
992f83bb6f
test: add coverage for assumeUTXO honest peers disconnection
Exercising and verifying the following points:

1. An IBD node can sync headers from an AssumeUTXO node at
   any time.

2. IBD nodes do not request historical blocks from AssumeUTXO
   nodes while they are syncing the background-chain.

3. The assumeUTXO node dynamically adjusts the network services
   it offers according to its state.

4. IBD nodes can fully sync from AssumeUTXO nodes after they
   finish the background-chain sync.
2024-09-10 18:08:33 -03:00
furszy
6d5812e5c8
assumeUTXO: fix peers disconnection during sync
Because AssumeUTXO nodes prioritize tip synchronization, they relay their local
address through the network before completing the background chain sync.
This, combined with the advertising of full-node service (NODE_NETWORK), can
result in an honest peer in IBD connecting to the AssumeUTXO node (while syncing)
and requesting an historical block the node does not have. This behavior leads to
an abrupt disconnection due to perceived unresponsiveness (lack of response)
from the AssumeUTXO node.

This lack of response occurs because nodes ignore getdata requests when they do
not have the block data available (further discussion can be found in PR 30385).

Fix this by refraining from signaling full-node service support while the
background chain is being synced. During this period, the node will only
signal 'NODE_NETWORK_LIMITED' support. Then, full-node ('NODE_NETWORK')
support will be re-enabled once the background chain sync is completed.
2024-09-10 18:08:32 -03:00
Ava Chow
082779d606 test: Add explicit onion bind to p2p_permissions
When the bind option is replaced in the bitcoin.conf, bitcoind will
attempd to bind to the default tor listening port. If another bitcoind
is running that is already bound to that port, the bind will fail which,
since #22729, causes the test to fail.

This failure can be avoided by explicitly binding the tor port when the
bind is removed.
2024-09-10 16:32:08 -04:00
Ryan Ofsky
c66c68345e
Merge bitcoin/bitcoin#30773: Remove unsafe uint256S() and test-only uint160S()
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 13 native, x86_64, no depends, sqlite only, gui (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
43cd83b0c7 test: move uint256_tests/operator_with_self to arith_uint256_tests (stickies-v)
c6c994cb2b test: remove test-only uint160S (stickies-v)
62cc4656e2 test: remove test-only uint256S (stickies-v)
adc00ad728 test: remove test-only arith_uint256S (stickies-v)
f51b237723 refactor: rpc: use uint256::FromHex for ParseHashV (stickies-v)

Pull request description:

  _Continuation of #30569._

  Since fad2991ba0, `uint256S()` has been [deprecated](fad2991ba0 (diff-800776e2dda39116e889839f69409571a5d397de048a141da7e4003bc099e3e2R138)) because it is less robust than the `base_blob::FromHex()` introduced in https://github.com/bitcoin/bitcoin/pull/30482. Specifically, it tries to recover from length-mismatches, recover from untrimmed whitespace, 0x-prefix and garbage at the end, instead of simply requiring exactly 64 hex-only characters. (see also https://github.com/bitcoin/bitcoin/pull/30532)

  This PR removes `uint256S()` (and `uint160S()`) completely, with no non-test behaviour change.

  Specifically, the main changes in this PR are:
  - the (minimal) last non-test usage of `uint256S()` in `ParseHashV()` is removed without behaviour change, which can partially be verified by cherry-picking and/or modifying [this test commit](1f2b0fa86d)).
  - the test usage of `uint{160,256}S()` is removed, largely replacing it with `uint{160,256}::FromHex()` where applicable, potentially modifying the test by removing non-hex characters or dropping the test entirely if removing non-hex characters makes it redundant
  - the now unused `uint{160,256}S()` functions are removed completely.
  - unit test coverage on converting `uint256` <-> `arith_uint256` through `UintToArith256()` and `ArithToUint256()` is beefed up, and `arith_uint256` tests are moved to `arith_uint256_tests.cpp`, removing the `uint256_tests.cpp` dependency on `uint256h`, mirroring how the code is structured.

  _Note:  `uint256::FromUserHex()` exists to more leniently construct uint256 from user input, allowing "0x" prefixes and too-short-input, as safer alternative to `uint256S()` where necessary._

ACKs for top commit:
  l0rinc:
    reACK 43cd83b0c7
  hodlinator:
    re-ACK 43cd83b0c7
  ryanofsky:
    Code review ACK 43cd83b0c7. Only code change is a small refactoring which looks good. The rest of the PR is all test changes, which I only lightly reviewed, but seem to be positive and do what's described

Tree-SHA512: 48147a4c6af671597df0f72c1b477ae4631cd2cae4645ec54d0e327611ff302c9899e344518c81242cdde82930f6ad23a3a7e6e0b80671816e9f457b9de90a5c
2024-09-10 15:41:35 -04:00
Ryan Ofsky
2756797eca
Merge bitcoin/bitcoin#30065: init: fixes file descriptor accounting
d4c7c4009d init: error out if -maxconnections is negative (Sergi Delgado Segura)
c773649481 init: improves file descriptors accounting and docs (Sergi Delgado Segura)
29008a7ff4 init: fixes fd accounting regarding poll/select (Sergi Delgado Segura)

Pull request description:

  The current logic for file descriptor accounting is pretty convoluted and hard to follow. This is partially caused by the lack of documentation plus non-intuitive variable naming (which was more intuitive when fewer things were accounted for, but
  hasn't aged well). This has led to this accounting being error-prone and hard to maintain (as shown in the first commit of this PR).

  Redefine some of the constants, plus document what are we accounting for so this can be extended more easily

  Fixes #18911

ACKs for top commit:
  sr-gi:
    > ACK [d4c7c40](d4c7c4009d)
  naumenkogs:
    ACK d4c7c4009d
  vasild:
    ACK d4c7c4009d
  TheCharlatan:
    ACK d4c7c4009d

Tree-SHA512: 1524d10c8ad8f354f6ab9c244699adbcdae2dd7aba37de5b8f9e177c629e8a2cce0f6e8117e076dde3a592f5283bd30a4201db96a3c011e335c02d1fde7414bc
2024-09-10 13:19:01 -04:00