Commit graph

1012 commits

Author SHA1 Message Date
Ava Chow
c33eb2360e
Merge bitcoin/bitcoin#30043: net: Replace libnatpmp with built-in PCP+NATPMP implementation
5c7cacf649 ci: Remove natpmp build option and libnatpmp dependency (laanwj)
7e7ec984da doc: Remove mention of natpmp build options (laanwj)
061c3e32a2 depends: Drop natpmp and associated option from depends (laanwj)
20a18bf6aa build: Drop libnatpmp from build system (laanwj)
7b04709862 qt: Changes for built-in PCP+NAT-PMP (laanwj)
52f8ef66c6 net: Replace libnatpmp with built-in NATPMP+PCP implementation in mapport (laanwj)
97c97177cd net: Add PCP and NATPMP implementation (laanwj)
d72df63d16 net: Use GetLocalAddresses in Discover (laanwj)
e02030432b net: Add netif utility (laanwj)
754e425438 crypto: Add missing WriteBE16 function (laanwj)

Pull request description:

  Continues #30005. Closes #17012..

  This PR adds PCP (Port Control Protocol) from [RFC6887](https://datatracker.ietf.org/doc/html/rfc6887).  This adds, in addition to the existing IPv4 port mapping (which now uses PCP, with fallback to NAT-PMP), support for IPv6 pinholing-that is, opening a port on the firewall to make it reachable.

  PCP, like NAT-PMP is a simple UDP-based protocol, and the implementation is self-contained, so this gets rid of lthe libnatpnp dependency without adding a new one. It should otherwise be a drop-in replacement. NAT-PMP fallback is implemented so this will not make router support worse.

  For now it is disabled by default, though in the future (not in this PR) we could consider enable it by default to increase the number of connectable nodes without adding significant attack surface.

  To test:
  ```bash
  bitcoind -regtest -natpmp=1 -debug=net
  ```

  (most of the changes in this PR are, ironically, removing the libnatpmp dependency and associated build system and build docs)

  ## TODO

  - [x] Default gateway discovery on Linux / FreeBSD
  - [x] Default gateway discovery on Windows
  - [x] Default gateway discovery on MacOS
  - [x] Either solve FreeBSD compile issue (probably upstream issue) or remove FreeBSD support

  ## Things to consider for follow-up PRs

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658764974 avoid unreachable nets (not given to -onlynet=)

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1658949236 could announce an addr:port where we do not listen (no -bind)

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1684368824 could announce the wrong port because it uses GetListenPort()

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1679709347 if we requested one port but another was assigned, then which one to use in the renewal?

  - https://github.com/bitcoin/bitcoin/pull/30043#discussion_r1772017020 Use `GetAdapterAddresses` to discover local addresses for Windows

ACKs for top commit:
  Sjors:
    ACK 5c7cacf649
  achow101:
    ACK 5c7cacf649
  vasild:
    ACK 5c7cacf649

Tree-SHA512: e35b69e56d5f5449a3d48a799f9b7b65107c65eeb3e245c2c1e9d42221e469ca5ead90afae423160601cd664dd553a51c859e04f4492f335b064aae3bf23e3bc
2024-09-30 16:27:47 -04:00
laanwj
061c3e32a2 depends: Drop natpmp and associated option from depends 2024-09-30 11:37:55 +02:00
merge-script
29d00a1cee
Merge bitcoin/bitcoin#30940: depends: Fix build with MULTIPROCESS=1 in Guix environment
06b4c339e8 depends: Fix reproducibility when building with `MULTIPROCESS=1` (Hennadii Stepanov)
d8e3afc335 depends: Fix build with `MULTIPROCESS=1` in Guix environment (Hennadii Stepanov)

Pull request description:

  In the Guix environment, `${BASEPREFIX}/${HOST}/native/bin` is added to the `PATH` environment variable:33adc7521c/contrib/guix/libexec/build.sh (L233-L234)

  This causes CMake to search for package configurations in the `native` subdirectory first.

  Explicitly specifying the top-priority search prefixes for the `Libmultiprocess` and `LibmultiprocessNative` packages resolves https://github.com/bitcoin/bitcoin/issues/30931.

  Can be tested on this [branch](https://github.com/hebasto/bitcoin/commits/240921-guix-mp.DEMO/) with an additional [commit](d8ec933456):
  ```
  $ uname -m
  x86_64
  $ env MULTIPROCESS=1 HOSTS=x86_64-linux-gnu ./contrib/guix/guix-build
  $ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  70d482a09099931345888a918df5ccc78a0c742b1d18bb8a5d442af9b2717867  guix-build-d8ec933456bc/output/dist-archive/bitcoin-d8ec933456bc.tar.gz
  ea9465517c0e1cc559a6a353f404b2bf47447332c918b69410dc51b6c4fbdedc  guix-build-d8ec933456bc/output/x86_64-linux-gnu/SHA256SUMS.part
  f83a296139d0757abc47d053b8fddebc6d2ccc36dabea9b2d52883d23ba923bb  guix-build-d8ec933456bc/output/x86_64-linux-gnu/bitcoin-d8ec933456bc-x86_64-linux-gnu-debug.tar.gz
  81aed48bece8d9bf5327530763c17d73dd66d5a3c666cbaf9381c03d4a660845  guix-build-d8ec933456bc/output/x86_64-linux-gnu/bitcoin-d8ec933456bc-x86_64-linux-gnu.tar.gz
  ```
  ```
  $ uname -m
  aarch64
  $ env MULTIPROCESS=1 HOSTS=x86_64-linux-gnu ./contrib/guix/guix-build
  $ find guix-build-$(git rev-parse --short=12 HEAD)/output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
  70d482a09099931345888a918df5ccc78a0c742b1d18bb8a5d442af9b2717867  guix-build-d8ec933456bc/output/dist-archive/bitcoin-d8ec933456bc.tar.gz
  ea9465517c0e1cc559a6a353f404b2bf47447332c918b69410dc51b6c4fbdedc  guix-build-d8ec933456bc/output/x86_64-linux-gnu/SHA256SUMS.part
  f83a296139d0757abc47d053b8fddebc6d2ccc36dabea9b2d52883d23ba923bb  guix-build-d8ec933456bc/output/x86_64-linux-gnu/bitcoin-d8ec933456bc-x86_64-linux-gnu-debug.tar.gz
  81aed48bece8d9bf5327530763c17d73dd66d5a3c666cbaf9381c03d4a660845  guix-build-d8ec933456bc/output/x86_64-linux-gnu/bitcoin-d8ec933456bc-x86_64-linux-gnu.tar.gz
  ```

ACKs for top commit:
  ryanofsky:
    Code review ACK 06b4c339e8
  fanquake:
    ACK 06b4c339e8

Tree-SHA512: b219ff29723bf571784af11b8d41fdc0f3fe7e21a87493909f9fcc695e37f9dbf2f6eb0fda172a1e0060dec4d2aeaa7017b1078c0ea0040cabfbbe3e8a352c6b
2024-09-27 10:45:32 +01:00
Hennadii Stepanov
4cf84b344d
depends, doc: No need to specify general requirement
CMake is no longer required solely for `libmultiprocess`.
2024-09-26 11:56:15 +01:00
Ryan Ofsky
070e6a32d5 depends: Update libmultiprocess library for cmake headers target
This update brings in the following changes:

https://github.com/chaincodelabs/libmultiprocess/pull/107 example: Remove manual client adding
https://github.com/chaincodelabs/libmultiprocess/pull/108 doc: Add comments for socket descriptor handling when forking
https://github.com/chaincodelabs/libmultiprocess/pull/109 example: Add missing thread.join() call so example can exit cleanly
https://github.com/chaincodelabs/libmultiprocess/pull/110 cmake: add target_capnp_sources headers target
2024-09-23 16:03:04 -04:00
Hennadii Stepanov
06b4c339e8
depends: Fix reproducibility when building with MULTIPROCESS=1 2024-09-23 13:24:08 +01:00
Hennadii Stepanov
d8e3afc335
depends: Fix build with MULTIPROCESS=1 in Guix environment
In the Guix environment, `${BASEPREFIX}/${HOST}/native/bin` is added to
the `PATH` environment variable, causing CMake to search for package
configurations in the `native` subdirectory first.

Explicitly specifying the top-priority search prefixes for the
`Libmultiprocess` and `LibmultiprocessNative` packages resolves the
issue.
2024-09-23 10:58:49 +01:00
Hennadii Stepanov
712d105e09
depends, doc: Do not install Autotools packages 2024-09-14 13:38:33 +01:00
Ryan Ofsky
4da20434d4 depends: Update libmultiprocess library for CustomMessage function and ThreadContext bugfix
The CustomMessage functions allow simplifying custom IPC type code, and the
bugfix is needed to prevent in a crash in a new test which creates and destroys
connections in a loop. Upstream PRs are:

https://github.com/chaincodelabs/libmultiprocess/pull/105 types: Add Custom{Build,Read,Pass}Message hooks
https://github.com/chaincodelabs/libmultiprocess/pull/106 Bugfix: Clean up ThreadContext pointers when Connection is destroyed
2024-09-06 09:08:10 -04:00
fanquake
5567754087
depends: build libevent with -D_GNU_SOURCE
Currently, builds of libevent in depends, using CMake, fail on some
systems, like Alpine, with the following:
```bash
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c: In function 'evmap_signal_add_':
/bitcoin/depends/work/build/aarch64-unknown-linux-musl/libevent/2.1.12-stable-1516ed47ea8/evmap.c:456:31: error: 'NSIG' undeclared (first use in this function)
  456 |         if (sig < 0 || sig >= NSIG)
```

From what I can tell the `_GNU_SOURCE` "detection" in libevents CMake build
system, never? really worked, and it's not clear what a nice fix is.

For now, always use `_GNU_SOURCE` when building libevent in depends.
2024-09-03 10:16:52 +01:00
Hennadii Stepanov
d71ac76842
build: Remove Autotools-based build system 2024-08-30 21:31:39 +01:00
Hennadii Stepanov
371910a768
depends: Fix CMake-generated libzmq.pc file
See https://github.com/zeromq/libzmq/pull/4706.
2024-08-28 12:00:49 +01:00
Hennadii Stepanov
6ce50fd9d0
doc: Update for CMake-based build system
Co-authored-by: Lőrinc <pap.lorinc@gmail.com>
Co-authored-by: pablomartin4btc <pablomartin4btc@gmail.com>
2024-08-16 21:24:08 +01:00
Hennadii Stepanov
84ac35cfd4
cmake: Add cross-compiling support
To configure CMake for cross-compiling, use
`--toolchain depends/${HOST}/toolchain.cmake` command-line option.
2024-08-16 21:19:11 +01:00
Hennadii Stepanov
0d01c228a7
build: Generate toolchain.cmake in depends 2024-08-16 21:19:10 +01:00
Hennadii Stepanov
91a799247d
depends: Add host-specific cmake_system_version variables 2024-08-16 21:19:10 +01:00
Hennadii Stepanov
9b31209b4c
depends: Rename cmake_system -> cmake_system_name 2024-08-16 21:19:10 +01:00
Hennadii Stepanov
4a5208a81d
Revert "build, qt: Do not install *.prl files"
This reverts commit 1155978d8f.
2024-08-16 21:19:10 +01:00
Hennadii Stepanov
6522af62af
depends: Amend handling flags environment variables
If any of {C,CXX,CPP,LD}FLAGS is specified it should be assigned to
a non-type-specific variable.
2024-08-16 21:19:10 +01:00
sunerok
4c2cc63d84 Revert "depends: Fetch miniupnpc sources from an alternative website"
This reverts commit 21b8a14d37.
2024-08-07 15:31:45 -04:00
fanquake
a0a9a11642
depends: fix ZMQ CMake getcachesize check
Fixes #30587.
2024-08-05 14:17:58 +01:00
Hennadii Stepanov
89b1d5c818
depends: Fix zeromq build on OpenBSD 2024-07-31 23:37:50 +01:00
fanquake
bda537f7c4
depends: remove ENV unsetting for darwin
Now that we use the native compiler, and have fixed Qt, and these vars
are unset it Guix, we can remove the unsetting from our compiler command
here.

Fixes #21552.
2024-07-25 14:18:25 +01:00
fanquake
0b2aeee21d
depends: patch explicit -lm usage out of Qt tools 2024-07-25 14:14:23 +01:00
fanquake
a0314c1516
depends: cleanup after qrencode build 2024-07-25 12:02:48 +01:00
fanquake
745bf0fa7e
depends: cleanup after miniupnpc build 2024-07-25 12:01:22 +01:00
Hennadii Stepanov
06d4aab77a
depends: Cleanup postprocess commands after switching to CMake 2024-07-25 11:59:18 +01:00
merge-script
1518c086fd
Merge bitcoin/bitcoin#30513: depends: Bump libmultiprocess for CMake fixes
ec0e805d11 depends: Bump `libmultiprocess` for CMake fixes (Hennadii Stepanov)

Pull request description:

  This PR amends https://github.com/bitcoin/bitcoin/pull/30490 and bumps the upstream branch, which now includes a required CMake [fix](https://github.com/chaincodelabs/libmultiprocess/pull/103).

  Addresses https://github.com/bitcoin/bitcoin/pull/30490#issuecomment-2241153244.

  The CI logs are available in https://github.com/bitcoin/bitcoin/pull/29790 where the recent [push](https://github.com/hebasto/bitcoin/tree/pr29790-0723.2.mp) uses this PR implementation.

ACKs for top commit:
  ryanofsky:
    Code review ACK ec0e805d11
  theuni:
    utACK ec0e805d11.

Tree-SHA512: e300a27bcab80a63a518719e9af8e10a938294fc07289cadbf4a7744627c10b0e9541a36971d08b65152f3f7d0eb434e427274d9c9d9f0bdd216afd914027a0f
2024-07-24 09:11:00 +01:00
Hennadii Stepanov
ec0e805d11
depends: Bump libmultiprocess for CMake fixes 2024-07-23 20:04:53 +01:00
fanquake
a517029646
depends: switch to building expat with CMake
Add a patch to set the minimum CMake to 3.16.
2024-07-23 15:37:01 +01:00
merge-script
b927a39c63
Merge bitcoin/bitcoin#30488: depends: Fix CMake-generated libevent*.pc files
8c935e625e depends: Fix CMake-generated `libevent*.pc` files (Hennadii Stepanov)

Pull request description:

  Broken out of #30454. This is a backport of the merged upstream PR: https://github.com/libevent/libevent/pull/1622.

  Note that after #29835 we might end up dropping pkg-config and using the installed CMake files directly, but that depends on whether or not enough distros actually ship those files.

  Either way, having fixed up .pc files won't hurt.

ACKs for top commit:
  hebasto:
    ACK 8c935e625e.
  fanquake:
    ACK 8c935e625e

Tree-SHA512: 259c2ad78fb9e90370a7205dc71c40acda1a872f6509435133bc1c4c2c3de57366e80679aa083e13ed85e7966883dc470c0147ee171a2ed0171a18cd5ffc99b3
2024-07-22 17:59:47 +01:00
merge-script
c69ba20bce
Merge bitcoin/bitcoin#29723: depends: build zeromq with CMake
0388ad0d65 depends: switch zmq to CMake (Cory Fields)
fefb3bbe5b depends: add zeromq no librt patch (fanquake)
a522ef1542 depends: add zeromq cmake minimum patch (fanquake)
cbbc229adf depends: add zeromq windows usage patch (fanquake)
2de68d6d38 depends: add zeromq builtin sha1 patch (fanquake)
0c8605253a depends: add zeromq mktemp macos patch (fanquake)

Pull request description:

  This picks up a change, which is a switch to building zeromq with CMake. It includes a number of patches, some which have already been upstreamed (see each patch for details).

ACKs for top commit:
  hebasto:
    ACK 0388ad0d65.

Tree-SHA512: 5567e432b4e4e0446c41d502bd61810a80b329dea2399b5d9d9f6e79acc450d1c6ba861c8238ba895de98338cfc5dc44ad2bf86ee8c222ecb3fbf47d6eb60da4
2024-07-22 17:49:27 +01:00
Cory Fields
d318c4ef56 depends: bump libmultiprocess for CMake fixes 2024-07-19 18:32:56 +00:00
Hennadii Stepanov
8c935e625e depends: Fix CMake-generated libevent*.pc files
See https://github.com/libevent/libevent/pull/1622.
2024-07-19 17:45:53 +00:00
Cory Fields
0388ad0d65
depends: switch zmq to CMake
The CMake WIN32_WINNT autodetection is broken, and must be set
manually. We may want to set is explicitly in any case, but the
brokenness should also be fixed upstream.

Also patch out depends paths, that would cause non-determinism.

Co-authored-by: fanquake <fanquake@gmail.com>
2024-07-19 13:38:21 +01:00
fanquake
fefb3bbe5b
depends: add zeromq no librt patch 2024-07-19 10:00:24 +01:00
fanquake
a522ef1542
depends: add zeromq cmake minimum patch 2024-07-19 10:00:24 +01:00
fanquake
cbbc229adf
depends: add zeromq windows usage patch 2024-07-19 10:00:24 +01:00
fanquake
2de68d6d38
depends: add zeromq builtin sha1 patch 2024-07-19 10:00:24 +01:00
fanquake
0c8605253a
depends: add zeromq mktemp macos patch 2024-07-19 10:00:24 +01:00
fanquake
ff4f3deb7b
depends: use CMake to build FreeType 2024-07-18 14:22:20 +01:00
merge-script
66114cd45b
Merge bitcoin/bitcoin#30336: depends: update doc in Qt pwd patch
f170fe04ca depends: update doc in Qt pwd patch (fanquake)

Pull request description:

  Now that upstream has gotten around to fixing this. We don't need any more of the patch, and it likely wont apply to our version of Qt in any case. See: 3388de698b.

ACKs for top commit:
  theuni:
    ACK f170fe04ca

Tree-SHA512: f6db8ccad591b1bf144ce71f873f42a115d394c432a95b6b855e3e32751e6331145e0d9676657599b25fd369af8c72c1bd34e192a7a1062c15f152421422a9ed
2024-07-12 09:40:32 +01:00
fanquake
f170fe04ca
depends: update doc in Qt pwd patch
Now that upstream has gotten around to fixing this. We don't need any
more of the patch, and it likely wont apply to our version of Qt in any
case. See:
3388de698b.
2024-06-25 16:58:00 +01:00
Cory Fields
f59e9057e2
depends: switch libevent to CMake 2024-06-25 16:05:40 +01:00
fanquake
d042230f7a
depends: swap mmacosx-version-min for mmacos-version-min
Whilst these remain aliases for each other, the later is preferred,
and I assume the former will be removed at some point in the future;
see: https://github.com/llvm/llvm-project/pull/95374.
2024-06-14 09:43:36 +01:00
fanquake
7cbfd7a7ce
refactor: rename (macho) ld64 to lld
Change some references to the macho ld64 to lld, which is now what is
used.
2024-06-10 13:20:54 +01:00
fanquake
9ebdd5e9e0
depends: update install docs for macOS cross compilation 2024-06-10 13:15:23 +01:00
fanquake
fb74fd66cb
depends: remove no-longer used llvm_* vars from macOS build 2024-06-10 13:15:23 +01:00
fanquake
9043f12425
depends: no-longer pass -B to clang in macOS cross-compile 2024-06-10 13:15:23 +01:00
fanquake
f9994b025e
depends: remove native LLVM package 2024-06-10 13:15:23 +01:00