Commit graph

43632 commits

Author SHA1 Message Date
Sjors Provoost
a4df12323c
doc: add release notes
Co-Authored-By: tdb3 <106488469+tdb3@users.noreply.github.com>
2025-01-22 12:31:46 +01:00
tdb3
c75872ffdd
test: use DIFF_1_N_BITS in tool_signet_miner 2025-01-22 12:31:46 +01:00
Sjors Provoost
4131f322ac
test: check difficulty adjustment using alternate mainnet 2025-01-22 12:31:46 +01:00
Sjors Provoost
c4f68c12e2
Use OP_0 for BIP34 padding in signet and tests
For blocks 1 through 15 the script_BIP34_coinbase_height appends OP_1
to comply with BIP34 and avoid bad-cb-length.

This is inconsistent with BlockAssembler::CreateNewBlock() which adds
OP_0 instead.

The utxo_total_supply fuzzer and MinerTestingSetup::Block also use OP_0.

Changing it is required to import the test vectors in the next commit.

It also ensures the test vectors can be regenerated using the CPU miner
at https://github.com/pooler/cpuminer without patches (it uses OP_0).

The same helper is used by the signet miner, so this will impact newly
bootstrapped signets.
2025-01-22 12:29:16 +01:00
Sjors Provoost
cf0a62878b
rpc: add next to getmininginfo
Obtain difficulty and target for the next block without having to call
getblocktemplate.
2025-01-22 12:28:45 +01:00
Sjors Provoost
2d18a078a2
rpc: add target and bits to getchainstates 2025-01-22 12:28:42 +01:00
Sjors Provoost
f153f57acc
rpc: add target and bits to getblockchaininfo 2025-01-22 12:28:38 +01:00
Sjors Provoost
baa504fdfa
rpc: add target to getmininginfo result 2025-01-22 12:04:02 +01:00
Sjors Provoost
2a7bfebd5e
Add target to getblock(header) in RPC and REST 2025-01-22 12:04:02 +01:00
Sjors Provoost
341f932516
rpc: add GetTarget helper 2025-01-22 12:04:02 +01:00
tdb3
d20d96fa41
test: use REGTEST_N_BITS in feature_block 2025-01-22 11:29:06 +01:00
Sjors Provoost
7ddbed4f9f
rpc: add nBits to getmininginfo
Also expands nBits test coverage.
2025-01-22 11:29:06 +01:00
Sjors Provoost
ba7b9f3d7b
build: move pow and chain to bitcoin_common
The next commit needs pow.cpp in rpc/util.cpp.
2025-01-22 11:29:05 +01:00
Sjors Provoost
c4cc9e3e9d
consensus: add DeriveTarget() to pow.h
Split CheckProofOfWorkImpl() to introduce a helper function
DeriveTarget() which converts the nBits value to the target.

The function takes pow_limit as an argument so later commits can
avoid having to pass ChainstateManager through the call stack.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2025-01-22 11:29:05 +01:00
merge-script
5691fa93c4
Merge bitcoin/bitcoin#31661: depends: Override default build type for libevent
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / macOS 14 native, arm64, fuzz (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / Win64 native fuzz, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
d44626a9c2 depends: Override default build type for `libevent` (Hennadii Stepanov)

Pull request description:

  This PR fixes a regression for the `libevent` package introduced in https://github.com/bitcoin/bitcoin/pull/29835.

  The `libevent` package defaults to the "Release" build type, which overrides our per-build-type optimization flags with `-O3`.

  To prevent this behavior, set `CMAKE_BUILD_TYPE` to "None", consistent with how other packages are handled.

ACKs for top commit:
  fanquake:
    ACK d44626a9c2

Tree-SHA512: 77abd2e28ad8dda86eb0548d8e49ecf23bac08a2e07dc35c71db62539aa659d471c863d361534c3cf693f9945c1b4f12de7e04eef05d11f8cc5e86d6eff5242d
2025-01-21 17:04:38 +00:00
merge-script
8fc7140846
Merge bitcoin/bitcoin#31671: Update leveldb subtree to latest upstream
910a11fa66 build: remove LEVELDB_IS_BIG_ENDIAN (fanquake)
d336b7ab85 Squashed 'src/leveldb/' changes from 688561cba8..04b5790928 (fanquake)

Pull request description:

  Includes:
  * https://github.com/bitcoin-core/leveldb-subtree/pull/40 (used in #29852)
  * https://github.com/bitcoin-core/leveldb-subtree/pull/45
  * https://github.com/bitcoin-core/leveldb-subtree/pull/46

ACKs for top commit:
  kevkevinpal:
    Concept ACK [910a11f](910a11fa66)
  l0rinc:
    ACK 910a11fa66
  hebasto:
    ACK 910a11fa66, I've performed a subtree update locally and got the same changes.
  theuni:
    utACK 910a11fa66

Tree-SHA512: c5a2224c67d3fd598bc682589b805c324abf91003032a85764766048030285f56154779f29d3f0b3673c8f7f497ae62de5fc6b95ef0b022c873750053c7d27d5
2025-01-21 17:01:10 +00:00
merge-script
a9edec9419
Merge bitcoin/bitcoin#31701: test: Bump sync_mempools timeout in p2p_1p1c_network.py
fa80a7dac4 test: Bump sync_mempools timeout in p2p_1p1c_network.py (MarcoFalke)
1111b0ac19 ci: Add missing --combinedlogslen to test-each-commit task (MarcoFalke)

Pull request description:

  This should address the two issues that happened in https://github.com/bitcoin/bitcoin/actions/runs/12885576442/job/35924329657?pr=25832#step:6:7601:

  * The combined log isn't printed on a test failure.
  * The timeout is too strict for the GHA virtual machines.

  For reference, the output was:

  ```
  ...
  149/315 - rpc_blockchain.py --v2transport passed, Duration: 10 s
  150/315 - p2p_addrfetch.py passed, Duration: 1 s
  151/315 - p2p_1p1c_network.py failed, Duration: 31 s

  stdout:
  2025-01-21T12:05:49.465000Z TestFramework (INFO): PRNG seed is: 6581340712385622842
  2025-01-21T12:05:49.466000Z TestFramework (INFO): Initializing test directory /tmp/test_runner_₿_🏃_20250121_120233/p2p_1p1c_network_207
  2025-01-21T12:05:52.408000Z TestFramework (INFO): Fill mempools with large transactions to raise mempool minimum feerates
  2025-01-21T12:05:52.408000Z TestFramework (INFO): Fill the mempool until eviction is triggered and the mempoolminfee rises
  2025-01-21T12:05:59.692000Z TestFramework (INFO): Pre-send some transactions to nodes
  2025-01-21T12:06:00.203000Z TestFramework (INFO): Submit full packages to node0
  2025-01-21T12:06:00.220000Z TestFramework (INFO): Wait for mempools to sync
  2025-01-21T12:06:20.384000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/runner/work/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 135, in main
      self.run_test()
    File "/home/runner/work/bitcoin/bitcoin/build/test/functional/p2p_1p1c_network.py", line 153, in run_test
      self.sync_mempools(timeout=20)
    File "/home/runner/work/bitcoin/bitcoin/test/functional/test_framework/test_framework.py", line 803, in sync_mempools
      raise AssertionError("Mempool sync timed out after {}s:{}".format(
  AssertionError: Mempool sync timed out after 20s:
  ...

ACKs for top commit:
  l0rinc:
    utACK fa80a7dac4
  glozow:
    ACK fa80a7dac4

Tree-SHA512: b326b7906b184fb47abc50d0d7ec91a6c90d324997f2abc40f156f588090e8d89bd8486bb8950cac604e77b1b336142a47b53ad463b2670d81222814eeb313d4
2025-01-21 16:49:42 +00:00
MarcoFalke
fa80a7dac4
test: Bump sync_mempools timeout in p2p_1p1c_network.py 2025-01-21 15:34:51 +01:00
MarcoFalke
1111b0ac19
ci: Add missing --combinedlogslen to test-each-commit task 2025-01-21 15:34:39 +01:00
Hennadii Stepanov
d44626a9c2
depends: Override default build type for libevent
The `libevent` package defaults to the "Release" build type, which
overrides our per-build-type optimization flags with `-O3`.

To prevent this behavior, set `CMAKE_BUILD_TYPE` to "None", consistent
with how other packages are handled.
2025-01-21 10:39:11 +00:00
merge-script
d7f56cc5d9
Merge bitcoin/bitcoin#31590: descriptors: Try pubkeys of both parities when retrieving the private keys for an xonly pubkey in a descriptor
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / macOS 14 native, arm64, fuzz (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / Win64 native fuzz, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
c0045e6cee Add test for multipath miniscript expression (David Gumberg)
b4ac48090f descriptor: Use InferXOnlyPubkey for miniscript XOnly pubkey from script (Ava Chow)
4c50c21f6b tests: Check ExpandPrivate matches for both parsed descriptors (Ava Chow)
092569e858 descriptor: Try the other parity in ConstPubkeyProvider::GetPrivKey() (Ava Chow)

Pull request description:

  When a `ConstPubkeyProvider` is xonly, the stored pubkey does not necessarily have the correct parity bit. `ToPrivateString()` is correctly handling this by looking up the keys for both parity bits, but `GetPrivKey` does not. This results in not finding the private key when it is actually available if its pubkey has the other parity bit value.

  To fix this, this key finding is refactored into `GetPrivKey()` so that its behavior is corrected, and `ToPrivateString()` is changed to use `GetPrivKey()` as well.

  Additionally, the descriptor test checks are updated to include a check for `ExpandPrivate()` to verify that both the parsed public and private descriptors produce `SigningProvider`s with the same contents.

  Fixes #31589

ACKs for top commit:
  Pttn:
    ACK c0045e6cee
  davidgumberg:
    utACK c0045e6cee
  kevkevinpal:
    Concept and Code review ACK [c0045e6](c0045e6cee)
  furszy:
    ACK c0045e6cee
  theStack:
    re-ACK c0045e6cee
  rkrux:
    Concept ACK c0045e6cee

Tree-SHA512: 3dcf2a802b996e0680a3f819075e5a689eb22e484c81ea79b40ec04197ee4ba3f6b9c87c45dfe8a847c9b805b2fd0fad77ffb92a93e65dc3aad74d69d9e3d97f
2025-01-21 10:20:13 +00:00
merge-script
4601b7ca61
Merge bitcoin/bitcoin#31125: depends: add *FLAGS to gen_id
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / macOS 14 native, arm64, fuzz (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / Win64 native fuzz, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
01df180bfb depends: add mold & ld.lld to gen_id (fanquake)
d032ac8063 depends: add *FLAGS to gen_id (fanquake)

Pull request description:

  The depends cache should be busted when flags change, the same as any other tooling change. I'd also like to start passing `*FLAGS` into depends inside the Guix env, which, without this change, doesn't bust the cache.

ACKs for top commit:
  hebasto:
    ACK 01df180bfb.

Tree-SHA512: 3809359fe763af9dde484e0c6bd3e262c4c09fcbe2f96ccf64194f5f9f840f5476b9c9929cf7bda7b8c14efeffd369cdb8c233625b79a944e1380df20698246f
2025-01-20 16:25:39 +00:00
merge-script
eb243ff06c
Merge bitcoin/bitcoin#31593: ci: Bump centos stream 10
faaabfaea7 ci: Bump centos stream 10 (MarcoFalke)

Pull request description:

  This is a follow-up to fa47baa03b, which bumped the gcc version to avoid a warning bloat in the CI log. However, it is also required to bump python3, see https://github.com/bitcoin/bitcoin/issues/31476#issue-2735206340

  > This will uncover an issue in the centos task that the correct python version is missing. I guess this should be fixed by installing and activating an acceptable python version.

  Instead of bumping the packages individually in centos stream 9, just bump to stream 10.

ACKs for top commit:
  fanquake:
    ACK faaabfaea7

Tree-SHA512: a564ff3a2a0dc4d39874e87540e67072f293bbed82c8eca22266fcadc16c5571e0e41d38576a63e466b64d13f7e3acbd95be10cf2420de33127aa420eca3b928
2025-01-20 16:11:58 +00:00
merge-script
6dc6012670
Merge bitcoin/bitcoin#31657: ci: Supply --platform argument to docker commands.
6e29de2101 ci: Supply `platform` argument to docker commands. (David Gumberg)

Pull request description:

  I ran into this issue when following the instructions in `ci/README.md` for running CI locally.

  Newer versions of docker require a `--platform` argument when building from a platform-specific image that differs from the host platform, I'm not sure when this change took place, but trying to build any of the cross-platform CI images on Docker 27.5.0 fails in the following manner:
  ```console
  $ # From ci/README.md
  $ env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
  WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v4) and no specific platform was requested
  Creating docker.io/arm64v8/debian:bookworm container to run in
  + docker build --file $BITCOIN_SRC/ci/test_imagefile --build-arg CI_IMAGE_NAME_TAG=docker.io/arm64v8/debian:bookworm --build-arg FILE_ENV=./ci/test/00_setup_env_arm.sh --label=bitcoin-ci-test --tag=ci_arm_linux $BITCOIN_SRC
  [+] Building 0.6s (2/2) FINISHED                                                                                                                                                 docker:default
   => [internal] load build definition from test_imagefile                                                                                                                                   0.0s
   => => transferring dockerfile: 600B                                                                                                                                                       0.0s
   => WARN: InvalidDefaultArgInFrom: Default value for ARG ${CI_IMAGE_NAME_TAG} results in empty or invalid base image name (line 8)                                                         0.0s
   => ERROR [internal] load metadata for docker.io/arm64v8/debian:bookworm                                                                                                                   0.5s
  ------
   > [internal] load metadata for docker.io/arm64v8/debian:bookworm:
  ------

   1 warning found (use docker --debug to expand):
   - InvalidDefaultArgInFrom: Default value for ARG ${CI_IMAGE_NAME_TAG} results in empty or invalid base image name (line 8)
  test_imagefile:8
  --------------------
     6 |
     7 |     ARG CI_IMAGE_NAME_TAG
     8 | >>> FROM ${CI_IMAGE_NAME_TAG}
     9 |
    10 |     ARG FILE_ENV
  --------------------
  ERROR: failed to solve: docker.io/arm64v8/debian:bookworm: failed to resolve source metadata for docker.io/arm64v8/debian:bookworm: no match for platform in manifest: not found
  ```

  This branch fixes this by setting the `--platform` argument of `docker build` and `docker run` with an environment variable `CI_IMAGE_PLATFORM` for each platform specific job, and `linux/{$cpuarch}` for any native jobs.

  Thi

  ## Steps to reproduce
  1. Install relevant dependencies, on Ubuntu:
  ```bash
  sudo apt install bash docker.io python3 qemu-user-static
  ```
  2. Run one of the platform-specific CI images, e.g.:
  ```bash
  env -i HOME="$HOME" PATH="$PATH" USER="$USER" bash -c 'FILE_ENV="./ci/test/00_setup_env_arm.sh" ./ci/test_run_all.sh'
  ```

ACKs for top commit:
  maflcko:
    lgtm ACK 6e29de2101
  hebasto:
    ACK 6e29de2101

Tree-SHA512: 81b9fa8ec1f3d21619d37d864047c8d7917ef2c8536851f80facf7f1973dfe14628d7755f12d2a9c6edebb6cb16877c582d4d41cdab52b73b23c44f08c6e6b30
2025-01-20 16:10:40 +00:00
merge-script
4e52a63443
Merge bitcoin/bitcoin#31691: util: fix compiler warning about deprecated space before _MiB
d3339a7cd5 util: fix compiler warning about deprecated space before _MiB (Vasil Dimov)

Pull request description:

  ```
  src/util/byte_units.h:13:29: error: identifier '_MiB' preceded by whitespace in a literal operator declaration is deprecated [-Werror,-Wdeprecated-literal-operator]
     13 | constexpr size_t operator"" _MiB(unsigned long long mebibytes)
        |                  ~~~~~~~~~~~^~~~
        |                  operator""_MiB
  1 error generated.
  ```

  Clang 20.0.0

ACKs for top commit:
  maflcko:
    lgtm ACK d3339a7cd5
  TheCharlatan:
    ACK d3339a7cd5

Tree-SHA512: 534537eefae2b6a9cef1ba3aa9ef16c491c84c0e55fb827f4ea0bae7ab61a5cc266a71499a79da97a9e68953a2c7f9cd1864ddde41fda173c829ed2b7bd73a46
2025-01-20 14:09:01 +00:00
merge-script
2e839dd641
Merge bitcoin/bitcoin#30774: depends: Qt 5.15.16
31a0e5f090 depends: Qt 5.15.16 (fanquake)

Pull request description:

  Contains a handful of miscellaneous bug fixes.
  We can drop a few of our patches.

  See https://github.com/qt/qtbase/compare/v5.15.14-lts-lgpl...v5.15.16-lts-lgpl.

ACKs for top commit:
  hebasto:
    ACK 31a0e5f090.
  TheCharlatan:
    ACK 31a0e5f090

Tree-SHA512: dd7b3332dd6ecb95189bc72364883425fb8869e03850791d2ee92555a37046c7abaaee16575a0396f1ce9674856b894563dbd36868c2cf46f9fee48028fd967b
2025-01-20 14:03:54 +00:00
Vasil Dimov
d3339a7cd5
util: fix compiler warning about deprecated space before _MiB
```
src/util/byte_units.h:13:29: error: identifier '_MiB' preceded by whitespace in a literal operator declaration is deprecated [-Werror,-Wdeprecated-literal-operator]
   13 | constexpr size_t operator"" _MiB(unsigned long long mebibytes)
      |                  ~~~~~~~~~~~^~~~
      |                  operator""_MiB
1 error generated.
```

Clang 20.0.0
2025-01-20 14:32:20 +01:00
David Gumberg
6e29de2101 ci: Supply platform argument to docker commands.
Later versions of docker require a `--platform` argument when building
and running from a platform-specific image that differs from the host
platform.
2025-01-17 10:36:41 -08:00
MarcoFalke
faaabfaea7
ci: Bump centos stream 10 2025-01-17 15:34:11 +01:00
merge-script
89720b7a1b
Merge bitcoin/bitcoin#31543: cmake: Always provide RPATH on NetBSD
Some checks failed
CI / Win64 native fuzz, VS 2022 (push) Has been cancelled
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Has been cancelled
CI / test each commit (push) Has been cancelled
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Has been cancelled
CI / macOS 14 native, arm64, fuzz (push) Has been cancelled
CI / Win64 native, VS 2022 (push) Has been cancelled
11115e9aa8 cmake: Always provide `RPATH` on NetBSD (Hennadii Stepanov)

Pull request description:

  Apparently, runtime paths cannot be skipped on NetBSD, even for system-wide packages.

  On NetBSD 10.0:
  - on the master branch @ bb57017b29:
  ```
  $ cmake -B build -DCMAKE_C_COMPILER="/usr/pkg/gcc14/bin/gcc" -DCMAKE_CXX_COMPILER="/usr/pkg/gcc14/bin/g++"
  $ cmake --build build
  $ ./build/src/bitcoin-wallet -version
  ./build/src/bitcoin-wallet: Shared object "libsqlite3.so.0" not found
  $ cmake --install build --prefix /home/hebasto/INSTALL
  $ /home/hebasto/INSTALL/bin/bitcoin-wallet -version
  /home/hebasto/INSTALL/bin/bitcoin-wallet: Shared object "libsqlite3.so.0" not found
  ```
  - with this PR:
  ```
  $ cmake -B build -DCMAKE_C_COMPILER="/usr/pkg/gcc14/bin/gcc" -DCMAKE_CXX_COMPILER="/usr/pkg/gcc14/bin/g++"
  $ cmake --build build
  $ ./build/src/bitcoin-wallet -version | head -1
  Bitcoin Core bitcoin-wallet utility version v28.99.0-11115e9aa845
  $ cmake --install build --prefix /home/hebasto/INSTALL
  $ /home/hebasto/INSTALL/bin/bitcoin-wallet -version | head -1
  Bitcoin Core bitcoin-wallet utility version v28.99.0-11115e9aa845
  ```

ACKs for top commit:
  theuni:
    utACK 11115e9aa8

Tree-SHA512: c9cd5d8c65fcf12677c381060dd53794396b7cd3e61ec39d3c5dadd7cdc08ab9790c59aa346402d53f8f9f317830919edf7f8135a6c0d5703c3bd9e519b157a4
2025-01-17 14:11:33 +00:00
merge-script
5b3a81f44d
Merge bitcoin/bitcoin#31626: depends: Use base system's sha256sum utility on FreeBSD
727c542769 depends: Use base system's `sha256sum` utility (Hennadii Stepanov)

Pull request description:

  On FreeBSD, the `shasum` utility is provided by the [`perl5`](https://ports.freebsd.org/cgi/ports.cgi?query=%5Eperl5&stype=all&sektion=all) port, which is not part of the base system and must be [installed](0e3518579a/.github/workflows/freebsd.yml (L104)) separately. Note that this requirement is currently not documented in [`depends/README.md`](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md).

  This PR switches to using the [`sha256sum`](https://man.freebsd.org/cgi/man.cgi?query=sha256sum&apropos=0&sektion=0&manpath=FreeBSD+14.2-RELEASE+and+Ports&arch=default&format=html) utility, which is included in the base system.

ACKs for top commit:
  davidgumberg:
    ACK 727c542769

Tree-SHA512: 41f11b80c2dc32514116400aa8ae57ef809edcbff5883dd62042d12ff14a54557864e2f1979c8705914242a2334880463da3e376f32c5b57a68fe41a90db1f85
2025-01-17 14:02:03 +00:00
merge-script
832d573073
Merge bitcoin/bitcoin#31651: ci: Turn CentOS task into native one
fabefd9915 ci: Turn CentOS task into native one (MarcoFalke)

Pull request description:

  Cross-compiling to `i686-pc-linux-gnu` on CentOS in the CI is mostly redundant with the `ci/test/00_setup_env_i686_multiprocess.sh` task (albeit it using clang):

  35bf426e02/ci/test/00_setup_env_i686_multiprocess.sh (L9-L12)

  One task seems sufficient as a sanity check, given that there seems to be no real demand for this architecture anyway.

  Turning the task into a native one makes it possible to run the task natively on aarch64 or any other supported architecture.

  Also, remove the install of the `lbzip2` package, which is unused since commit a46065e36c

  Also, remove the `CONFIG_SHELL` env var, which is unused since the cmake migration. (`CONFIG_SHELL` in depends is still kept).

ACKs for top commit:
  davidgumberg:
    ACK fabefd9915
  hebasto:
    ACK fabefd9915, tested locally on Ubuntu 24.10.

Tree-SHA512: 5a7b3131b379d11ef602e5821165861e9bdf61d605014bf8fcb33b8e12d8823450798af2d3289b96f7559dfa47b839bf939ddc0b3725efecfeac7ae570a981e7
2025-01-17 11:42:48 +00:00
merge-script
5ac1e0814a
Merge bitcoin/bitcoin#31621: doc: Update dependency installation for Debian/Ubuntu
160c27ec07 doc: Update dependency installation for Debian/Ubuntu and CI (Adlai Chandrasekhar)

Pull request description:

  This is similar to the recently-pushed 8d20348 and results in slightly cleaner systems for future Debian/Ubuntu builds.

  According to the description for pkg-config, "pkgconf is a replacement for pkg-config, providing additional functionality while also maintaining compatibility. This package only provides a dependency link to the pkgconf package to help with package upgrades. It can be safely removed."

  Thus the relevant sections of `doc/build-unix.md` and `depends/README.md` are updated.

ACKs for top commit:
  maflcko:
    weak ACK 160c27ec07
  fanquake:
    ACK 160c27ec07 - seems correct for modern distro versions, and using pkgconf on older ones also seems to work fine.

Tree-SHA512: fadeffe464073df91b706e30f560bfe332ce676521cc5d2044d3bf499f08d986ccaab0a10dd1178f626a90bbac3a4f8c445fe4f8e3a63960721664a247b758f7
2025-01-17 11:03:11 +00:00
merge-script
370b9c1a0b
Merge bitcoin/bitcoin#31675: [test] fix p2p_orphan_handling.py empty orphanage check
Some checks are pending
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
CI / macOS 14 native, arm64, fuzz (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / Win64 native fuzz, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
2e75ebb616 [test] fix p2p_orphan_handling.py empty orphanage check (glozow)

Pull request description:

  Fix CI: https://cirrus-ci.com/task/4584274617171968?logs=ci#L2784

ACKs for top commit:
  maflcko:
    lgtm ACK 2e75ebb616
  instagibbs:
    ACK 2e75ebb616
  theStack:
    ACK 2e75ebb616

Tree-SHA512: 7c5c4d1eb816368b2fdd8e090769ff586f69a9800a99ab99d42beda96c933b5c449dfece91b659d48e4c2d7dcaae0dde6441a294d33c991db3d85c15df0123b6
2025-01-17 09:37:21 +00:00
glozow
2e75ebb616 [test] fix p2p_orphan_handling.py empty orphanage check
It's possible getorphantxs isn't empty immediately. Prevent intermittent errors.
2025-01-16 13:56:12 -05:00
merge-script
f9032a4abb
Merge bitcoin/bitcoin#31242: wallet, desc spkm: Return SigningProvider only if we have the privkey
Some checks are pending
CI / macOS 14 native, arm64, fuzz (push) Waiting to run
CI / Win64 native, VS 2022 (push) Waiting to run
CI / Win64 native fuzz, VS 2022 (push) Waiting to run
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Waiting to run
CI / test each commit (push) Waiting to run
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Waiting to run
f6a6d91205 test: add check for getting SigningProvider for a CPubKey (Sebastian Falbesoner)
62a95f5af9 test: refactor: move `CreateDescriptor` helper to wallet test util module (Sebastian Falbesoner)
493656763f desc spkm: Return SigningProvider only if we have the privkey (Ava Chow)

Pull request description:

  If we know about a pubkey that's in our descriptor, but we don't have the private key, don't return a SigningProvider for that pubkey.

  This is specifically an issue for Taproot outputs that use the H point as the resulting PSBTs may end up containing irrelevant information because the H point was detected as a pubkey each unrelated descriptor knew about.

  Split from #29675

ACKs for top commit:
  fjahr:
    ACK f6a6d91205
  theStack:
    re-ACK f6a6d91205
  furszy:
    utACK f6a6d91205. Only reviewed the actual change in detail, not the test commit.

Tree-SHA512: 30a196e611a0c5d9ebe5baf6d896caaa6af66f1615463dbb0c31e52604d53cf342922bb9967b3c697b47083d76b0485c77a5f545bd6381247c8bc44321c70f97
2025-01-16 17:30:36 +00:00
merge-script
9dc4eedb67
Merge bitcoin/bitcoin#31673: doc: fix minor typos in comments
b30cc71e85 doc: fix typos (Adlai Chandrasekhar)

Pull request description:

  In the unrelated PR #31621 the linter reported a few typos, that are fixed in this commit. I used the "doc" prefix as it only modifies comments, so none of the more significant prefixes seem appropriate.

ACKs for top commit:
  maflcko:
    lgtm ACK b30cc71e85

Tree-SHA512: 7bba2d928fc0b98f62f96d9abf6dba98f699b386b75730271fa3e7b57a8a220df2265b699007f066e585e1db2ee3cbe5a272b74a8c153f6f8814c01e6de7a3ee
2025-01-16 16:21:14 +00:00
Adlai Chandrasekhar
b30cc71e85 doc: fix typos 2025-01-16 17:36:16 +02:00
Adlai Chandrasekhar
160c27ec07 doc: Update dependency installation for Debian/Ubuntu and CI
According to the description for pkg-config, "pkgconf is a
replacement for pkg-config, providing additional functionality
while also maintaining compatibility. This package only provides
a dependency link to the pkgconf package to help with package
upgrades. It can be safely removed."

Thus several scripts and markdown files are updated.
2025-01-16 17:31:13 +02:00
merge-script
df8bf65745
Merge bitcoin/bitcoin#31483: kernel: Move kernel-related cache constants to kernel cache
2a92702baf init: Use size_t consistently for cache sizes (TheCharlatan)
65cde3621d kernel: Move default cache constants to caches (TheCharlatan)
8826cae285 kernel: Move non-kernel db cache size constants (TheCharlatan)
e758b26b85 kernel: Move kernel-specific cache size options to kernel (TheCharlatan)
d5e2c4a409 fuzz: Add fuzz test for checked and saturating add and left shift (TheCharlatan)
c03a2795a8 util: Add integer left shift helpers (TheCharlatan)
8bd5f8a38c [refactor] init: Simplify coinsdb cache calculation (TheCharlatan)
5db7d4d3d2 doc: Correct docstring describing max block tree db cache (TheCharlatan)

Pull request description:

  Carrying non-kernel related fields in the cache sizes for the indexes is confusing for kernel library users. The cache sizes are set currently with magic numbers in bitcoin-chainstate. The comments for the cache size calculations are not completely clear. The constants for the cache sizes are also currently in `txdb.h`, which is not an ideal place for holding all cache size related constants.

  Solve these things by moving the kernel-specific cache size fields to their own struct and moving the constants to either the node or the kernel cache sizes.

  This slightly changes the way the cache is allocated if (and only if) the txindex and/or blockfilterindex is used. Since they are now given precedence over the block tree db cache, this results in a bit less cache being allocated to the block tree db, coinsdb and coins caches. The effect is negligible though, i.e. cache sizes with default dbcache reported through the logs are:

  master:
  ```
  Cache configuration:
  * Using 2.0 MiB for block index database
  * Using 56.0 MiB for transaction index database
  * Using 49.0 MiB for basic block filter index database
  * Using 8.0 MiB for chain state database
  * Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
  ```

  this PR:
  ```
  Cache configuration:
  * Using 2.0 MiB for block index database
  * Using 56.2 MiB for transaction index database
  * Using 49.2 MiB for basic block filter index database
  * Using 8.0 MiB for chain state database
  * Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
  ```

  ---
  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  stickies-v:
    re-ACK 2a92702baf
  ryanofsky:
    Code review ACK 2a92702baf. Changes since last review are fixing size options to use size_t instead of int64_t again, simplifying CheckedLeftShift more, and making other minor suggested cleanups
  hodlinator:
    re-ACK 2a92702baf

Tree-SHA512: 98376eaa0660b1b8c096a5ce1f3e7c8c30e7cd6644de36856c2d3e573108cfc9473c93ebb3952b7881047b5ae6c85c5b096e6726f30f35be58b98eca07c8c785
2025-01-16 15:04:58 +00:00
merge-script
335798c496
Merge bitcoin/bitcoin#31397: p2p: track and use all potential peers for orphan resolution
86d7135e36 [p2p] only attempt 1p1c when both txns provided by the same peer (glozow)
f7658d9b14 [cleanup] remove p2p_inv from AddTxAnnouncement (glozow)
063c1324c1 [functional test] getorphantxs reflects multiple announcers (glozow)
0da693f7e1 [functional test] orphan handling with multiple announcers (glozow)
b6ea4a9afe [p2p] try multiple peers for orphan resolution (glozow)
1d2e1d709c [refactor] move creation of unique_parents to helper function (glozow)
c6893b0f0b [txdownload] remove unique_parents that we already have (glozow)
163aaf285a [fuzz] orphanage multiple announcer functions (glozow)
22b023b09d [unit test] multiple orphan announcers (glozow)
96c1a822a2 [unit test] TxOrphanage EraseForBlock (glozow)
04448ce32a [txorphanage] add GetTx so that orphan vin can be read (glozow)
e810842acd [txorphanage] support multiple announcers (glozow)
62a9ff1870 [refactor] change type of unique_parents to Txid (glozow)
6951ddcefd [txrequest] GetCandidatePeers (glozow)

Pull request description:

  Part of #27463.

  (Transaction) **orphan resolution** is a process that kicks off when we are missing UTXOs to validate an unconfirmed transaction. We currently request missing parents by txid; BIP 331 also defines a way to [explicitly request ancestors](https://github.com/bitcoin/bips/blob/master/bip-0331.mediawiki#handle-orphans-better).

  Currently, when we find that a transaction is an orphan, we only try to resolve it with the peer who provided the `tx`. If this doesn't work out (e.g. they send a `notfound` or don't respond), we do not try again. We actually can't, because we've already forgotten who else could resolve this orphan (i.e. all the other peers who announced the transaction).

  What is wrong with this? It makes transaction download less reliable, particularly for 1p1c packages which must go through orphan resolution in order to be downloaded.

  Can we fix this with BIP 331 / is this "duct tape" before the real solution?
  BIP 331 (receiver-initiated ancestor package relay) is also based on the idea that there is an orphan that needs resolution, but it's just a new way of communicating information. It's not inherently more honest; you can request ancestor package information and get a `notfound`. So ancestor package relay still requires some kind of procedure for retrying when an orphan resolution attempt fails. See the #27742 implementation which builds on this orphan resolution tracker to keep track of what packages to download (it just isn't rebased on this exact branch). The difference when using BIP 331 is that we request `ancpkginfo` and then `pkgtxns` instead of the parent txids.

  Zooming out, we'd like orphan handling to be:
  - Bandwidth-efficient: don't have too many requests out at once. As already implemented today, transaction requests for orphan parents and regular download both go through the `TxRequestTracker` so that we don't have duplicate requests out.
  - Not vulnerable to censorship: don't give up too easily, use all candidate peers. See e.g. https://bitcoincore.org/en/2024/07/03/disclose_already_asked_for/
  - Load-balance between peers: don't overload peers; use all peers available. This is also useful for when we introduce per-peer orphan protection, since each peer will have limited slots.

  The approach taken in this PR is to think of each peer who announces an orphan as a potential "orphan resolution candidate." These candidates include:
  - the peer who sent us the orphan tx
  - any peers who announced the orphan prior to us downloading it
  - any peers who subsequently announce the orphan after we have started trying to resolve it
  For each orphan resolution candidate, we treat them as having "announced" all of the missing parents to us at the time of receipt of this orphan transaction (or at the time they announced the tx if they do so after we've already started tracking it as an orphan). We add the missing parents as entries to `m_txrequest`, incorporating the logic of typical txrequest processing, which means we prefer outbounds, try not to have duplicate requests in flight, don't overload peers, etc.

ACKs for top commit:
  marcofleon:
    Code review ACK 86d7135e36
  instagibbs:
    reACK 86d7135e36
  dergoegge:
    Code review ACK 86d7135e36
  mzumsande:
    ACK 86d7135e36

Tree-SHA512: 618d523b86e60c3ea039e88326d50db4e55e8e18309c6a20e8f2b10ed9e076f1de0315c335fd3b8abdabcc8b53cbceb66fb59147d05470ea25b83a2b4bd9c877
2025-01-16 13:42:26 +00:00
merge-script
98939ce7b7
Merge bitcoin/bitcoin#31655: refactor: Avoid UB in SHA3_256::Write
fabeca3458 refactor: Avoid UB in SHA3_256::Write (MarcoFalke)
fad4032b21 refactor: Drop unused UCharCast (MarcoFalke)

Pull request description:

  It is UB to apply a distance to a pointer or iterator further than the
  end itself, even if the distance is (partially) revoked later on.

  Fix the issue by advancing the data pointer at most to the end.

  This fix is required to adopt C++ safe buffers https://github.com/bitcoin/bitcoin/issues/31272.

  Also included is a somewhat unrelated commit.

ACKs for top commit:
  sipa:
    utACK fabeca3458
  theuni:
    utACK fabeca3458
  hebasto:
    ACK fabeca3458.

Tree-SHA512: 78c53691322b72c3ba9c25ec94eec275dbbbc3049b0ad45e7d9fb2df0afbbaa905b0d8fa7106a3582f937bb1dc15a7592c4ad2d80fe4cff1062a3acfd3638f08
2025-01-16 12:20:05 +00:00
fanquake
910a11fa66
build: remove LEVELDB_IS_BIG_ENDIAN 2025-01-16 11:10:23 +00:00
fanquake
9ec64253ab
Update leveldb subtree to latest upstream 2025-01-16 11:09:56 +00:00
fanquake
d336b7ab85 Squashed 'src/leveldb/' changes from 688561cba8..04b5790928
04b5790928 Merge bitcoin-core/leveldb-subtree#46: Fix invalid pointer arithmetic in Hash (#1222)
59669817c5 Merge bitcoin-core/leveldb-subtree#40: cherry-pick: Remove leveldb::port::kLittleEndian.
73013d1a37 Merge bitcoin-core/leveldb-subtree#45: [jumbo] Add begin()/end() to Slice.
a8844b23ab Fix invalid pointer arithmetic in Hash (#1222)
be4dfc94b3 [jumbo] Add begin()/end() to Slice.
2e3c0131d3 Remove leveldb::port::kLittleEndian.

git-subtree-dir: src/leveldb
git-subtree-split: 04b57909285c7335c1908d53bcde9b90fe0439be
2025-01-16 11:09:56 +00:00
merge-script
712cab3a8f
Merge bitcoin/bitcoin#31061: refactor: Check translatable format strings at compile-time
Some checks failed
CI / test each commit (push) Has been cancelled
CI / macOS 14 native, arm64, no depends, sqlite only, gui (push) Has been cancelled
CI / macOS 14 native, arm64, fuzz (push) Has been cancelled
CI / Win64 native, VS 2022 (push) Has been cancelled
CI / Win64 native fuzz, VS 2022 (push) Has been cancelled
CI / ASan + LSan + UBSan + integer, no depends, USDT (push) Has been cancelled
fa3efb5729 refactor: Introduce struct to hold a runtime format string (MarcoFalke)
fa6adb0134 lint: Remove unused and broken format string linter (MarcoFalke)
fadc6b9bac refactor: Check translatable format strings at compile-time (MarcoFalke)
fa1d5acb8d refactor: Use TranslateFn type consistently (MarcoFalke)
eeee6cf2ff refactor: Delay translation of _() literals (MarcoFalke)

Pull request description:

  All translatable format strings are fixed. This change surfaces errors in them at compile-time.

  The implementation achieves this by allowing to delay the translation (or `std::string` construction) that previously happened in `_()` by returning a new type from this function. The new type can be converted to `bilingual_str` where needed.

  This can be tested by adding a format string error in an original string literal and observing a new compile-time failure.

  Fixes https://github.com/bitcoin/bitcoin/issues/30530

ACKs for top commit:
  stickies-v:
    re-ACK fa3efb5729
  ryanofsky:
    Code review ACK fa3efb5729. Since last review added TranslateFn commit, clarified FormatStringCheck documentation, dropped redundant `inline` keyword

Tree-SHA512: 28fa1db11e85935d998031347bd519675d75c171c8323b0ed6cdd0b628c95250bb86b30876946cc48840ded541e95b8a152696f9f2b13a5f28f5673228ee0509
2025-01-15 15:47:00 +00:00
TheCharlatan
2a92702baf
init: Use size_t consistently for cache sizes
This avoids having to rely on implicit casts when passing them to the
various functions allocating the caches.

This also ensures that if the requested amount of db_cache does not fit
in a size_t, it is clamped to the maximum value of a size_t.

Also take this opportunity to make the total amounts of cache in the
chainstate manager a size_t too.
2025-01-15 15:44:56 +01:00
TheCharlatan
65cde3621d
kernel: Move default cache constants to caches
They are not related to the txdb, so a better place for them is the
new kernel and node cache file. Re-use the default amount of kernel
cache for the default node cache.
2025-01-15 15:44:55 +01:00
TheCharlatan
8826cae285
kernel: Move non-kernel db cache size constants
These have nothing to do with the txdb, so move them out and into the
node caches.
2025-01-15 15:44:44 +01:00
TheCharlatan
e758b26b85
kernel: Move kernel-specific cache size options to kernel
Carrying non-kernel related fields in the cache sizes for the indexes is
confusing for kernel library users. The cache sizes also are set
currently with magic numbers in bitcoin-chainstate. The comments for the
cache size calculations are also not completely clear.

Solve these things by moving the kernel-specific cache size fields to
their own struct.

This slightly changes the way the cache is allocated if the txindex
and/or blockfilterindex is used. Since they are now given precedence
over the block tree db cache, this results in a bit less cache being
allocated to the block tree db, coinsdb and coins caches. The effect is
negligible though, i.e. cache sizes with default dbcache reported
through the logs are:

master:
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.0 MiB for transaction index database
* Using 49.0 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 335.0 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)

this branch:
Cache configuration:
* Using 2.0 MiB for block index database
* Using 56.2 MiB for transaction index database
* Using 49.2 MiB for basic block filter index database
* Using 8.0 MiB for chain state database
* Using 334.5 MiB for in-memory UTXO set (plus up to 286.1 MiB of unused mempool space)
2025-01-15 15:44:16 +01:00