Merge bitcoin/bitcoin#29780: [27.x] More backports and finalize

910e3e8728 doc: update manual pages for v27.0 (fanquake)
2d2a0a369e build: bump version to v27.0 final (fanquake)
9442ea82da doc: import release notes from devwiki (fanquake)
fe51aceeca Update the developer mailing list address. (Edil Medeiros)
bde54b7627 ci: Print tsan errors to stderr (MarcoFalke)
753c68dc0f ci: Pull in qtbase5-dev instead of seperate low-level libraries (laanwj)
c4da61b323 doc: Suggest installing dev packages for debian/ubuntu qt5 build (laanwj)

Pull request description:

  Currently backports:
  * https://github.com/bitcoin/bitcoin/pull/29740
  * https://github.com/bitcoin/bitcoin/pull/29764
  * https://github.com/bitcoin/bitcoin/pull/29782

  Pulls release notes:  https://github.com/bitcoin-core/bitcoin-devwiki/wiki/27.0-Release-Notes-Draft.
  Finalizes `v27.0`.

ACKs for top commit:
  achow101:
    ACK 910e3e8728
  stickies-v:
    ACK 910e3e8728 - no changes except addressing nits

Tree-SHA512: 5dd8c6adaab154699f8ad9a5bda26de54901d3d1d09034a5a6775a37f12b44274d2d5ce6e1a527a6c211b56a75fe154f3ccb37bcbc39882be8299a4f127bfe30

.github/workflows/ci.yml

@@ -62,7 +62,7 @@ jobs:
           echo "TEST_BASE=$(git rev-list -n$((${{ env.MAX_COUNT }} + 1)) --reverse HEAD ^$(git rev-list -n1 --merges HEAD)^@ | head -1)" >> "$GITHUB_ENV"
       - run: |
           sudo apt-get update
-          sudo apt-get install clang-15 ccache build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
+          sudo apt-get install clang-15 ccache build-essential libtool autotools-dev automake pkg-config bsdmainutils python3-zmq libevent-dev libboost-dev libsqlite3-dev libdb++-dev systemtap-sdt-dev libminiupnpc-dev libnatpmp-dev qtbase5-dev qttools5-dev qttools5-dev-tools qtwayland5 libqrencode-dev -y
       - name: Compile and run tests
         run: |
           # Run tests on commits after the last merge commit and before the PR head commit

CONTRIBUTING.md

@@ -66,9 +66,10 @@ Discussion about codebase improvements happens in GitHub issues and pull
 requests.
 
 The developer
-[mailing list](https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev)
+[mailing list](https://groups.google.com/g/bitcoindev)
 should be used to discuss complicated or controversial consensus or P2P protocol changes before working on
 a patch set.
+Archives can be found on [https://gnusha.org/pi/bitcoindev/](https://gnusha.org/pi/bitcoindev/).
 
 
 Contributor Workflow

ci/test/00_setup_env_native_tidy.sh

@@ -9,7 +9,7 @@ export LC_ALL=C.UTF-8
 export CI_IMAGE_NAME_TAG="docker.io/ubuntu:24.04"
 export CONTAINER_NAME=ci_native_tidy
 export TIDY_LLVM_V="17"
-export PACKAGES="clang-${TIDY_LLVM_V} libclang-${TIDY_LLVM_V}-dev llvm-${TIDY_LLVM_V}-dev libomp-${TIDY_LLVM_V}-dev clang-tidy-${TIDY_LLVM_V} jq bear libevent-dev libboost-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev systemtap-sdt-dev libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools libqrencode-dev libsqlite3-dev libdb++-dev"
+export PACKAGES="clang-${TIDY_LLVM_V} libclang-${TIDY_LLVM_V}-dev llvm-${TIDY_LLVM_V}-dev libomp-${TIDY_LLVM_V}-dev clang-tidy-${TIDY_LLVM_V} jq bear libevent-dev libboost-dev libminiupnpc-dev libnatpmp-dev libzmq3-dev systemtap-sdt-dev qtbase5-dev qttools5-dev qttools5-dev-tools libqrencode-dev libsqlite3-dev libdb++-dev"
 export NO_DEPENDS=1
 export RUN_UNIT_TESTS=false
 export RUN_FUNCTIONAL_TESTS=false

ci/test/03_test_script.sh

@@ -10,7 +10,7 @@ set -ex
 
 export ASAN_OPTIONS="detect_stack_use_after_return=1:check_initialization_order=1:strict_init_order=1"
 export LSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/lsan"
-export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:halt_on_error=1:log_path=${BASE_SCRATCH_DIR}/sanitizer-output/tsan"
+export TSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/tsan:halt_on_error=1"
 export UBSAN_OPTIONS="suppressions=${BASE_ROOT_DIR}/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1"
 
 if [ "$CI_OS_NAME" == "macos" ]; then
@@ -71,8 +71,6 @@ elif [ "$RUN_UNIT_TESTS" = "true" ] || [ "$RUN_UNIT_TESTS_SEQUENTIAL" = "true" ]
   fi
 fi
 
-mkdir -p "${BASE_SCRATCH_DIR}/sanitizer-output/"
-
 if [ "$USE_BUSY_BOX" = "true" ]; then
   echo "Setup to use BusyBox utils"
   # tar excluded for now because it requires passing in the exact archive type in ./depends (fixed in later BusyBox version)

configure.ac

@@ -2,7 +2,7 @@ AC_PREREQ([2.69])
 define(_CLIENT_VERSION_MAJOR, 27)
 define(_CLIENT_VERSION_MINOR, 0)
 define(_CLIENT_VERSION_BUILD, 0)
-define(_CLIENT_VERSION_RC, 1)
+define(_CLIENT_VERSION_RC, 0)
 define(_CLIENT_VERSION_IS_RELEASE, true)
 define(_COPYRIGHT_YEAR, 2024)
 define(_COPYRIGHT_HOLDERS,[The %s developers])

doc/build-unix.md

@@ -81,7 +81,7 @@ To build without GUI pass `--without-gui`.
 
 To build with Qt 5 you need the following:
 
-    sudo apt-get install libqt5gui5 libqt5core5a libqt5dbus5 qttools5-dev qttools5-dev-tools
+    sudo apt-get install qtbase5-dev qttools5-dev qttools5-dev-tools
 
 Additionally, to support Wayland protocol for modern desktop environments:
 

doc/dnsseed-policy.md

@@ -44,7 +44,7 @@ related to the DNS seed operation.
 If these expectations cannot be satisfied the operator should
 discontinue providing services and contact the active Bitcoin
 Core development team as well as posting on
-[bitcoin-dev](https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev).
+[bitcoin-dev](https://groups.google.com/g/bitcoindev).
 
 Behavior outside of these expectations may be reasonable in some
 situations but should be discussed in public in advance.

doc/man/bitcoin-cli.1

@@ -1,7 +1,7 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.49.3.
-.TH BITCOIN-CLI "1" "March 2024" "bitcoin-cli v27.0.0rc1" "User Commands"
+.TH BITCOIN-CLI "1" "April 2024" "bitcoin-cli v27.0.0" "User Commands"
 .SH NAME
-bitcoin-cli \- manual page for bitcoin-cli v27.0.0rc1
+bitcoin-cli \- manual page for bitcoin-cli v27.0.0
 .SH SYNOPSIS
 .B bitcoin-cli
 [\fI\,options\/\fR] \fI\,<command> \/\fR[\fI\,params\/\fR]  \fI\,Send command to Bitcoin Core\/\fR
@@ -15,7 +15,7 @@ bitcoin-cli \- manual page for bitcoin-cli v27.0.0rc1
 .B bitcoin-cli
 [\fI\,options\/\fR] \fI\,help <command>      Get help for a command\/\fR
 .SH DESCRIPTION
-Bitcoin Core RPC client version v27.0.0rc1
+Bitcoin Core RPC client version v27.0.0
 .SH OPTIONS
 .HP
 \-?

doc/man/bitcoin-qt.1

@@ -1,12 +1,12 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.49.3.
-.TH BITCOIN-QT "1" "March 2024" "bitcoin-qt v27.0.0rc1" "User Commands"
+.TH BITCOIN-QT "1" "April 2024" "bitcoin-qt v27.0.0" "User Commands"
 .SH NAME
-bitcoin-qt \- manual page for bitcoin-qt v27.0.0rc1
+bitcoin-qt \- manual page for bitcoin-qt v27.0.0
 .SH SYNOPSIS
 .B bitcoin-qt
 [\fI\,command-line options\/\fR] [\fI\,URI\/\fR]
 .SH DESCRIPTION
-Bitcoin Core version v27.0.0rc1
+Bitcoin Core version v27.0.0
 .PP
 Optional URI is a Bitcoin address in BIP21 URI format.
 .SH OPTIONS

doc/man/bitcoin-tx.1

@@ -1,7 +1,7 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.49.3.
-.TH BITCOIN-TX "1" "March 2024" "bitcoin-tx v27.0.0rc1" "User Commands"
+.TH BITCOIN-TX "1" "April 2024" "bitcoin-tx v27.0.0" "User Commands"
 .SH NAME
-bitcoin-tx \- manual page for bitcoin-tx v27.0.0rc1
+bitcoin-tx \- manual page for bitcoin-tx v27.0.0
 .SH SYNOPSIS
 .B bitcoin-tx
 [\fI\,options\/\fR] \fI\,<hex-tx> \/\fR[\fI\,commands\/\fR]  \fI\,Update hex-encoded bitcoin transaction\/\fR
@@ -9,7 +9,7 @@ bitcoin-tx \- manual page for bitcoin-tx v27.0.0rc1
 .B bitcoin-tx
 [\fI\,options\/\fR] \fI\,-create \/\fR[\fI\,commands\/\fR]   \fI\,Create hex-encoded bitcoin transaction\/\fR
 .SH DESCRIPTION
-Bitcoin Core bitcoin\-tx utility version v27.0.0rc1
+Bitcoin Core bitcoin\-tx utility version v27.0.0
 .SH OPTIONS
 .HP
 \-?

doc/man/bitcoin-util.1

@@ -1,12 +1,12 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.49.3.
-.TH BITCOIN-UTIL "1" "March 2024" "bitcoin-util v27.0.0rc1" "User Commands"
+.TH BITCOIN-UTIL "1" "April 2024" "bitcoin-util v27.0.0" "User Commands"
 .SH NAME
-bitcoin-util \- manual page for bitcoin-util v27.0.0rc1
+bitcoin-util \- manual page for bitcoin-util v27.0.0
 .SH SYNOPSIS
 .B bitcoin-util
 [\fI\,options\/\fR] [\fI\,commands\/\fR]  \fI\,Do stuff\/\fR
 .SH DESCRIPTION
-Bitcoin Core bitcoin\-util utility version v27.0.0rc1
+Bitcoin Core bitcoin\-util utility version v27.0.0
 .SH OPTIONS
 .HP
 \-?

doc/man/bitcoin-wallet.1

@@ -1,9 +1,9 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.49.3.
-.TH BITCOIN-WALLET "1" "March 2024" "bitcoin-wallet v27.0.0rc1" "User Commands"
+.TH BITCOIN-WALLET "1" "April 2024" "bitcoin-wallet v27.0.0" "User Commands"
 .SH NAME
-bitcoin-wallet \- manual page for bitcoin-wallet v27.0.0rc1
+bitcoin-wallet \- manual page for bitcoin-wallet v27.0.0
 .SH DESCRIPTION
-Bitcoin Core bitcoin\-wallet version v27.0.0rc1
+Bitcoin Core bitcoin\-wallet version v27.0.0
 .PP
 bitcoin\-wallet is an offline tool for creating and interacting with Bitcoin Core wallet files.
 By default bitcoin\-wallet will act on wallets in the default mainnet wallet directory in the datadir.

doc/man/bitcoind.1

@@ -1,12 +1,12 @@
 .\" DO NOT MODIFY THIS FILE!  It was generated by help2man 1.49.3.
-.TH BITCOIND "1" "March 2024" "bitcoind v27.0.0rc1" "User Commands"
+.TH BITCOIND "1" "April 2024" "bitcoind v27.0.0" "User Commands"
 .SH NAME
-bitcoind \- manual page for bitcoind v27.0.0rc1
+bitcoind \- manual page for bitcoind v27.0.0
 .SH SYNOPSIS
 .B bitcoind
 [\fI\,options\/\fR]                     \fI\,Start Bitcoin Core\/\fR
 .SH DESCRIPTION
-Bitcoin Core version v27.0.0rc1
+Bitcoin Core version v27.0.0
 .SH OPTIONS
 .HP
 \-?

doc/release-notes.md

@@ -1 +1,217 @@
-See https://github.com/bitcoin-core/bitcoin-devwiki/wiki/27.0-Release-Notes-Draft
+Bitcoin Core version 27.0 is now available from:
+
+  <https://bitcoincore.org/bin/bitcoin-core-27.0/>
+
+This release includes new features, various bug fixes and performance
+improvements, as well as updated translations.
+
+Please report bugs using the issue tracker at GitHub:
+
+  <https://github.com/bitcoin/bitcoin/issues>
+
+To receive security and update notifications, please subscribe to:
+
+  <https://bitcoincore.org/en/list/announcements/join/>
+
+How to Upgrade
+==============
+
+If you are running an older version, shut it down. Wait until it has completely
+shut down (which might take a few minutes in some cases), then run the
+installer (on Windows) or just copy over `/Applications/Bitcoin-Qt` (on macOS)
+or `bitcoind`/`bitcoin-qt` (on Linux).
+
+Upgrading directly from a version of Bitcoin Core that has reached its EOL is
+possible, but it might take some time if the data directory needs to be migrated. Old
+wallet versions of Bitcoin Core are generally supported.
+
+Compatibility
+==============
+
+Bitcoin Core is supported and extensively tested on operating systems
+using the Linux Kernel 3.17+, macOS 11.0+, and Windows 7 and newer. Bitcoin
+Core should also work on most other Unix-like systems but is not as
+frequently tested on them. It is not recommended to use Bitcoin Core on
+unsupported systems.
+
+Notable changes
+===============
+
+libbitcoinconsensus
+-------------------
+
+- libbitcoinconsensus is deprecated and will be removed for v28. This library has
+  existed for nearly 10 years with very little known uptake or impact. It has
+  become a maintenance burden.
+
+  The underlying functionality does not change between versions, so any users of
+  the library can continue to use the final release indefinitely, with the
+  understanding that Taproot is its final consensus update.
+
+  In the future, libbitcoinkernel will provide a much more useful API that is
+  aware of the UTXO set, and therefore be able to fully validate transactions and
+  blocks. (#29189)
+
+mempool.dat compatibility
+-------------------------
+
+- The `mempool.dat` file created by -persistmempool or the savemempool RPC will
+  be written in a new format. This new format includes the XOR'ing of transaction
+  contents to mitigate issues where external programs (such as anti-virus) attempt
+  to interpret and potentially modify the file.
+
+  This new format can not be read by previous software releases. To allow for a
+  downgrade, a temporary setting `-persistmempoolv1` has been added to fall back
+  to the legacy format. (#28207)
+
+P2P and network changes
+-----------------------
+
+- BIP324 v2 transport is now enabled by default. It remains possible to disable v2
+  by running with `-v2transport=0`. (#29347)
+- Manual connection options (`-connect`, `-addnode` and `-seednode`) will
+  now follow `-v2transport` to connect with v2 by default. They will retry with
+  v1 on failure. (#29058)
+
+- Network-adjusted time has been removed from consensus code. It is replaced
+  with (unadjusted) system time. The warning for a large median time offset
+  (70 minutes or more) is kept. This removes the implicit security assumption of
+  requiring an honest majority of outbound peers, and increases the importance
+  of the node operator ensuring their system time is (and stays) correct to not
+  fall out of consensus with the network. (#28956)
+
+Mempool Policy Changes
+----------------------
+
+- Opt-in Topologically Restricted Until Confirmation (TRUC) Transactions policy
+  (aka v3 transaction policy) is available for use on test networks when
+  `-acceptnonstdtxn=1` is set. By setting the transaction version number to 3, TRUC transactions
+  request the application of limits on spending of their unconfirmed outputs. These
+  restrictions simplify the assessment of incentive compatibility of accepting or
+  replacing TRUC transactions, thus ensuring any replacements are more profitable for
+  the node and making fee-bumping more reliable. TRUC transactions are currently
+  nonstandard and can only be used on test networks where the standardness rules are
+  relaxed or disabled (e.g. with `-acceptnonstdtxn=1`). (#28948)
+
+External Signing
+----------------
+
+- Support for external signing on Windows has been disabled. It will be re-enabled
+  once the underlying dependency (Boost Process), has been replaced with a different
+  library. (#28967)
+
+Updated RPCs
+------------
+
+- The addnode RPC now follows the `-v2transport` option (now on by default, see above) for making connections.
+  It remains possible to specify the transport type manually with the v2transport argument of addnode. (#29239)
+
+Build System
+------------
+
+- A C++20 capable compiler is now required to build Bitcoin Core. (#28349)
+- MacOS releases are configured to use the hardened runtime libraries (#29127)
+
+Wallet
+------
+
+- The CoinGrinder coin selection algorithm has been introduced to mitigate unnecessary
+  large input sets and lower transaction costs at high feerates. CoinGrinder
+  searches for the input set with minimal weight. Solutions found by
+  CoinGrinder will produce a change output. CoinGrinder is only active at
+  elevated feerates (default: 30+ sat/vB, based on `-consolidatefeerate`×3). (#27877)
+- The Branch And Bound coin selection algorithm will be disabled when the subtract fee
+  from outputs feature is used. (#28994)
+- If the birth time of a descriptor is detected to be later than the first transaction
+  involving that descriptor, the birth time will be reset to the earlier time. (#28920)
+
+Low-level changes
+=================
+
+Pruning
+-------
+
+- When pruning during initial block download, more blocks will be pruned at each
+  flush in order to speed up the syncing of such nodes. (#20827)
+
+Init
+----
+
+- Various fixes to prevent issues where subsequent instances of Bitcoin Core would
+  result in deletion of files in use by an existing instance. (#28784, #28946)
+- Improved handling of empty `settings.json` files. (#29144)
+
+Credits
+=======
+
+Thanks to everyone who directly contributed to this release:
+
+- 22388o⚡️
+- Aaron Clauson
+- Amiti Uttarwar
+- Andrew Toth
+- Anthony Towns
+- Antoine Poinsot
+- Ava Chow
+- Brandon Odiwuor
+- brunoerg
+- Chris Stewart
+- Cory Fields
+- dergoegge
+- djschnei21
+- Fabian Jahr
+- fanquake
+- furszy
+- Gloria Zhao
+- Greg Sanders
+- Hennadii Stepanov
+- Hernan Marino
+- iamcarlos94
+- ismaelsadeeq
+- Jameson Lopp
+- Jesse Barton
+- John Moffett
+- Jon Atack
+- josibake
+- jrakibi
+- Justin Dhillon
+- Kashif Smith
+- kevkevin
+- Kristaps Kaupe
+- L0la L33tz
+- Luke Dashjr
+- Lőrinc
+- marco
+- MarcoFalke
+- Mark Friedenbach
+- Marnix
+- Martin Leitner-Ankerl
+- Martin Zumsande
+- Max Edwards
+- Murch
+- muxator
+- naiyoma
+- Nikodemas Tuckus
+- ns-xvrn
+- pablomartin4btc
+- Peter Todd
+- Pieter Wuille
+- Richard Myers
+- Roman Zeyde
+- Russell Yanofsky
+- Ryan Ofsky
+- Sebastian Falbesoner
+- Sergi Delgado Segura
+- Sjors Provoost
+- stickies-v
+- stratospher
+- Supachai Kheawjuy
+- TheCharlatan
+- UdjinM6
+- Vasil Dimov
+- w0xlt
+- willcl-ark
+
+
+As well as to everyone that helped with translations on
+[Transifex](https://www.transifex.com/bitcoin/bitcoin/).