bitcoin/depends
Ava Chow c33eb2360e
Merge bitcoin/bitcoin#30043: net: Replace libnatpmp with built-in PCP+NATPMP implementation
5c7cacf649 ci: Remove natpmp build option and libnatpmp dependency (laanwj)
7e7ec984da doc: Remove mention of natpmp build options (laanwj)
061c3e32a2 depends: Drop natpmp and associated option from depends (laanwj)
20a18bf6aa build: Drop libnatpmp from build system (laanwj)
7b04709862 qt: Changes for built-in PCP+NAT-PMP (laanwj)
52f8ef66c6 net: Replace libnatpmp with built-in NATPMP+PCP implementation in mapport (laanwj)
97c97177cd net: Add PCP and NATPMP implementation (laanwj)
d72df63d16 net: Use GetLocalAddresses in Discover (laanwj)
e02030432b net: Add netif utility (laanwj)
754e425438 crypto: Add missing WriteBE16 function (laanwj)

Pull request description:

  Continues #30005. Closes #17012..

  This PR adds PCP (Port Control Protocol) from [RFC6887](https://datatracker.ietf.org/doc/html/rfc6887).  This adds, in addition to the existing IPv4 port mapping (which now uses PCP, with fallback to NAT-PMP), support for IPv6 pinholing-that is, opening a port on the firewall to make it reachable.

  PCP, like NAT-PMP is a simple UDP-based protocol, and the implementation is self-contained, so this gets rid of lthe libnatpnp dependency without adding a new one. It should otherwise be a drop-in replacement. NAT-PMP fallback is implemented so this will not make router support worse.

  For now it is disabled by default, though in the future (not in this PR) we could consider enable it by default to increase the number of connectable nodes without adding significant attack surface.

  To test:
  ```bash
  bitcoind -regtest -natpmp=1 -debug=net
  ```

  (most of the changes in this PR are, ironically, removing the libnatpmp dependency and associated build system and build docs)

  ## TODO

  - [x] Default gateway discovery on Linux / FreeBSD
  - [x] Default gateway discovery on Windows
  - [x] Default gateway discovery on MacOS
  - [x] Either solve FreeBSD compile issue (probably upstream issue) or remove FreeBSD support

  ## Things to consider for follow-up PRs

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658764974 avoid unreachable nets (not given to -onlynet=)

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658949236 could announce an addr:port where we do not listen (no -bind)

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1684368824 could announce the wrong port because it uses GetListenPort()

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1679709347 if we requested one port but another was assigned, then which one to use in the renewal?

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1772017020 Use `GetAdapterAddresses` to discover local addresses for Windows

ACKs for top commit:
  Sjors:
    ACK 5c7cacf649
  achow101:
    ACK 5c7cacf649
  vasild:
    ACK 5c7cacf649

Tree-SHA512: e35b69e56d5f5449a3d48a799f9b7b65107c65eeb3e245c2c1e9d42221e469ca5ead90afae423160601cd664dd553a51c859e04f4492f335b064aae3bf23e3bc
2024-09-30 16:27:47 -04:00
..
builders depends: remove FORCE_USE_SYSTEM_CLANG 2024-06-10 13:15:23 +01:00
hosts depends: Add host-specific cmake_system_version variables 2024-08-16 21:19:10 +01:00
packages Merge bitcoin/bitcoin#30043: net: Replace libnatpmp with built-in PCP+NATPMP implementation 2024-09-30 16:27:47 -04:00
patches depends: Fix CMake-generated libzmq.pc file 2024-08-28 12:00:49 +01:00
.gitignore build: Remove Autotools-based build system 2024-08-30 21:31:39 +01:00
config.guess build: latest config.guess in depends 2023-11-03 11:04:09 +00:00
config.sub build: latest config.sub in depends 2023-11-03 11:04:37 +00:00
description.md Fix typos in description.md and wallet_util.py 2024-04-25 16:14:10 +08:00
funcs.mk build: Generate toolchain.cmake in depends 2024-08-16 21:19:10 +01:00
gen_id guix: remove ZERO_AR_DATE export 2024-05-22 08:51:33 +01:00
Makefile depends: Drop natpmp and associated option from depends 2024-09-30 11:37:55 +02:00
packages.md chore: fix some typos in comments 2024-04-15 20:12:54 +08:00
README.md Merge bitcoin/bitcoin#30043: net: Replace libnatpmp with built-in PCP+NATPMP implementation 2024-09-30 16:27:47 -04:00
toolchain.cmake.in Merge bitcoin/bitcoin#30043: net: Replace libnatpmp with built-in PCP+NATPMP implementation 2024-09-30 16:27:47 -04:00

Usage

To build dependencies for the current arch+OS:

make

To build for another arch/OS:

make HOST=host-platform-triplet

For example:

make HOST=x86_64-w64-mingw32 -j4

When configuring Bitcoin Core, CMake by default will ignore the depends output. In order for it to pick up libraries, tools, and settings from the depends build, you must specify the toolchain file. In the above example, a file named depends/x86_64-w64-mingw32/toolchain.cmake will be created. To use it during configuring Bitcoin Core:

cmake -B build --toolchain depends/x86_64-w64-mingw32/toolchain.cmake

Common host-platform-triplets for cross compilation are:

  • i686-pc-linux-gnu for Linux 32 bit
  • x86_64-pc-linux-gnu for x86 Linux
  • x86_64-w64-mingw32 for Win64
  • x86_64-apple-darwin for macOS
  • arm64-apple-darwin for ARM macOS
  • arm-linux-gnueabihf for Linux ARM 32 bit
  • aarch64-linux-gnu for Linux ARM 64 bit
  • powerpc64-linux-gnu for Linux POWER 64-bit (big endian)
  • powerpc64le-linux-gnu for Linux POWER 64-bit (little endian)
  • riscv32-linux-gnu for Linux RISC-V 32 bit
  • riscv64-linux-gnu for Linux RISC-V 64 bit
  • s390x-linux-gnu for Linux S390X

The paths are automatically configured and no other options are needed.

Install the required dependencies: Ubuntu & Debian

Common

apt install bison cmake curl make patch pkg-config python3 xz-utils

For macOS cross compilation

apt install clang lld llvm g++ zip

Clang 18 or later is required. You must also obtain the macOS SDK before proceeding with a cross-compile. Under the depends directory, create a subdirectory named SDKs. Then, place the extracted SDK under this new directory. For more information, see SDK Extraction.

For Win64 cross compilation

For linux (including i386, ARM) cross compilation

Common linux dependencies:

sudo apt-get install g++-multilib binutils

For linux ARM cross compilation:

sudo apt-get install g++-arm-linux-gnueabihf binutils-arm-linux-gnueabihf

For linux AARCH64 cross compilation:

sudo apt-get install g++-aarch64-linux-gnu binutils-aarch64-linux-gnu

For linux POWER 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-powerpc64-linux-gnu binutils-powerpc64-linux-gnu g++-powerpc64le-linux-gnu binutils-powerpc64le-linux-gnu

For linux RISC-V 64-bit cross compilation (there are no packages for 32-bit):

sudo apt-get install g++-riscv64-linux-gnu binutils-riscv64-linux-gnu

For linux S390X cross compilation:

sudo apt-get install g++-s390x-linux-gnu binutils-s390x-linux-gnu

Install the required dependencies: FreeBSD

pkg install bash

Install the required dependencies: NetBSD

pkgin install bash gmake

Install the required dependencies: OpenBSD

pkg_add bash gmake gtar

Dependency Options

The following can be set when running make: make FOO=bar

  • SOURCES_PATH: Downloaded sources will be placed here
  • BASE_CACHE: Built packages will be placed here
  • SDK_PATH: Path where SDKs can be found (used by macOS)
  • FALLBACK_DOWNLOAD_PATH: If a source file can't be fetched, try here before giving up
  • C_STANDARD: Set the C standard version used. Defaults to c11.
  • CXX_STANDARD: Set the C++ standard version used. Defaults to c++20.
  • NO_BOOST: Don't download/build/cache Boost
  • NO_LIBEVENT: Don't download/build/cache Libevent
  • NO_QT: Don't download/build/cache Qt and its dependencies
  • NO_QR: Don't download/build/cache packages needed for enabling qrencode
  • NO_ZMQ: Don't download/build/cache packages needed for enabling ZeroMQ
  • NO_WALLET: Don't download/build/cache libs needed to enable the wallet
  • NO_BDB: Don't download/build/cache BerkeleyDB
  • NO_SQLITE: Don't download/build/cache SQLite
  • NO_UPNP: Don't download/build/cache packages needed for enabling UPnP
  • NO_USDT: Don't download/build/cache packages needed for enabling USDT tracepoints
  • MULTIPROCESS: Build libmultiprocess (experimental)
  • DEBUG: Disable some optimizations and enable more runtime checking
  • HOST_ID_SALT: Optional salt to use when generating host package ids
  • BUILD_ID_SALT: Optional salt to use when generating build package ids
  • LOG: Use file-based logging for individual packages. During a package build its log file resides in the depends directory, and the log file is printed out automatically in case of build error. After successful build log files are moved along with package archives
  • LTO: Enable options needed for LTO. Does not add -flto related options to *FLAGS.
  • NO_HARDEN=1: Don't use hardening options when building packages

If some packages are not built, for example make NO_WALLET=1, the appropriate CMake cache variables will be set when generating the Bitcoin Core buildsystem. In this case, -DENABLE_WALLET=OFF.

Additional targets

download: run 'make download' to fetch all sources without building them
download-osx: run 'make download-osx' to fetch all sources needed for macOS builds
download-win: run 'make download-win' to fetch all sources needed for win builds
download-linux: run 'make download-linux' to fetch all sources needed for linux builds

Other documentation