9796dcacdc doc: Install only "-posix" MinGW compiler when possible (Hennadii Stepanov)
0bbae237a8 ci: Drop no longer needed `update-alternatives` (Hennadii Stepanov)
01d1845a80 build, qt: Specify QMAKE_CXX explicitly (Hennadii Stepanov)
Pull request description:
While changes introduced in bitcoin/bitcoin#22093 worked fine with Qt 5.12, after bumping Qt up to 5.15 the cross-compiling of `qt` package for Windows fails with `error: ‘mutex’ in namespace ‘std’ does not name a type`.
The first commit fixes this bug.
The second commit cleans up a related CI script.
The third commit improves related docs (see https://github.com/bitcoin/bitcoin/pull/22093#discussion_r680911586).
ACKs for top commit:
prusnak:
ACK 9796dca
Tree-SHA512: 0dc46c5dfab85bd6d2901052cd630e86f9b4e09c08ef87136b44ddecb1783cdf3cd0a6e67b95ac7a78da24cd7adedc88745f61f9a8d9993fbff26d33bf88d874
01e121d290 depends: fix capnp's descriptor for make download (Cory Fields)
Pull request description:
The non-native capnp was trying to fetch the wrong file.
Without this, "make -C depends MULTIPROCESS=1 download" is broken.
Presumably it breaks with the download target because the dependency graph is flattened. It manages to work if native_capnp is encountered first because it will then be found in the cache.
ACKs for top commit:
gruve-p:
tACK 01e121d290
hebasto:
ACK 01e121d290, tested on Linux Mint 20.2 (x86_64).
Tree-SHA512: 2605d895f3799be5a311f6f7d36a5c13cdb715dc148915ad818f4afc7d5de92cd6b8ecd34ff2b21cef6743b090819bba1e3353096cfb5659c55f76113ce5adf3
ae9175f608 build: add FreeBSD support to depends (fanquake)
Pull request description:
Setup to use the system Clang (11.0.1 as of FreeBSD 13.0).
Doesn't build the Qt package; that requires a few additional changes. The current issue is that FreeBSDs `byacc` seems to have an issue parsing something in libxkbcommon. Work in progress branch here: https://github.com/fanquake/bitcoin/tree/depends_support_freebsd_qt. In any case, I don't think building the Qt libs on FreeBSD is a super high priority (I'd also have no way to test the GUI).
ACKs for top commit:
laanwj:
ACK ae9175f608
Tree-SHA512: 1ecc7855f0195f68c96e534bd77ce17c3975db1dfb3caa35302c2b46643c8c30f75b1c2e1735cf69f1eddb70447d11e67e7f339ef5497336cdff7a59b32be961
The non-native capnp was trying to fetch the wrong file.
Without this, "make -C depends MULTIPROCESS=1 download" is broken.
Presumably it breaks with the download target because the dependency graph is
flattened. It manages to work if native_capnp is encountered first because it
will then be found in the cache.
f13e642c83 build: Disable valgrind when building zeromq package in depends (Hennadii Stepanov)
b970f03bea build: Disable libbsd when building zeromq package in depends (Hennadii Stepanov)
77899991b1 build: Update netbsd_kevent_void.patch (Hennadii Stepanov)
Pull request description:
Since v4.3.3 (068385c951) `libzmq` uses `libbsd` by default.
This PR disables `libbsd` explicitly, as it's not a part of our depends. Zeromq will fallback to its internal `strlcpy` implementation.
Otherwise, on systems with installed `libbsd-dev` package the `zeromq` package build system erroneously detects `libbsd` package from the host system:
```diff
--- a/libzmq.pc
+++ b/libzmq.pc
@@ -8,5 +8,5 @@
Version: 4.3.4
Libs: -L${libdir} -lzmq
Libs.private: -lpthread
-Requires.private:
+Requires.private: libbsd
Cflags: -I${includedir}
```
This causes the `configure` fails to detect the `zeromq` package:
```
configure: WARNING: libzmq version 4.x or greater not found, disabling
```
---
Other minor improvements:
- fixed `netbsd_kevent_void.patch` offset
- disabled valgrind as it's used in unit tests which we do not run:
```diff
--- a/zmq-configure-output
+++ b/zmq-configure-output
@@ -119,11 +119,6 @@
checking whether the g++ -m64 linker (/usr/bin/ld -m elf_x86_64) supports shared libraries... yes
checking dynamic linker characteristics... (cached) GNU/Linux ld.so
checking how to hardcode library paths into programs... immediate
-checking for valgrind... valgrind
-checking for Valgrind tool memcheck... memcheck
-checking for Valgrind tool helgrind... helgrind
-checking for Valgrind tool drd... drd
-checking for Valgrind tool exp-sgcheck... exp-sgcheck
checking linker version script flag... --version-script
checking if version scripts can use complex wildcards... yes
checking for working posix_memalign... yes
```
ACKs for top commit:
fanquake:
ACK f13e642c83
Tree-SHA512: d4c86d4a841eb6e7c32157e84972243072f905496c2a4c14ec6f6ab4216df6695cbf29baa2233ce27eaede35d1e250ad2f9975b16f570d01509f0c5da4597cad
Since v4.3.3 (068385c951c0608edec6264d55ba9c4c923acccc) libbsd is used
by default. As we have no libbsd package in our depends, disable it
explicitly. Zeromq will fallback to its internal strlcpy implementation.
6897c4bdf5 build: patch depends zeromq to fix building on NetBSD Current (fanquake)
ce6dd2f1a2 zeromq 4.3.4 (fanquake)
Pull request description:
This is a prerequisite for #23955. It updates zeromq to the latest available version, and adds a patch, [that I've sent upstream](https://github.com/zeromq/libzmq/pull/4326), to fix building on NetBSD Current (10).
ACKs for top commit:
hebasto:
ACK 6897c4bdf5, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: d05d9753630faebe842e1ca70c8c4af660a38e7331a9d95e84df3a3b14564c5118ca41c4fc49fb71dfee563b63e1014e5a3f8874d652e26de59e8e188a12970e
Similar to how we specify the OS's we build Qt for, specify which OS's
we will build qrencode for (a qt dependency). This commit alone doesn't
change anything, but when we start supporting other OS's, i.e #23948,
where we wont support qt (or at least initially), it'll skip building
the qrencode package, which would be unused.
6200fbf54f build: rename --enable-ebpf to --enable-usdt (0xb10c)
e158a2a7aa build: add systemtap's sys/sdt.h as depends (0xb10c)
Pull request description:
There has been light conceptual agreement on including the Userspace, Statically Defined Tracing tracepoints in Bitcoin Core release builds. This, for example, enables user to hook into production deployments, if they need to. Binaries don't have to be switched out. This is possible because we don't do [expensive computations](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#no-expensive-computations-for-tracepoints) only needed for the tracepoints. The tracepoints are NOPs when not used.
Systemtap's `sys/sdt.h` header is required to build Bitcoin Core with USDT support. The header file defines the `DTRACE_PROBE` macros used in [`src/util/trace.h`](https://github.com/bitcoin/bitcoin/blob/master/src/util/trace.h). This PR adds Systemtap 4.5 (May 2021) as dependency. GUIX builds for Linux hosts now include the tracepoints.
Closes https://github.com/bitcoin/bitcoin/issues/23297.
ACKs for top commit:
fanquake:
ACK 6200fbf54f - tested enabling / disabling and with/without SDT from depends. We can follow up with #23819, #23907 and #23296, and if any serious issues arise before feature freeze, it is easy for us to flip depends such that USDT becomes opt-in, rather than opt-out, and thus, releases would be tracepoint free.
Tree-SHA512: 0263f44892bf8450e8a593e4de7a498243687f8d81269e1c3283fa8354922c7cf93fddef4b92cf5192d33798424aa5812e03e68ef8de31af078a32dd34021382
This is required so that we can test for control flow in our ELF
security checks. Otherwise test_bitcoin will fail, as it has a main that
is provided by Boost, and wont have been built with
-fcf-protection=full.
While here simplify to using -fcf-protection for all x86_64 builds.
The sys/sdt.h header is required to build Bitcoin Core with Userspace
Statically Defined Tracing support. Systemtap version 4.5 (May 2021)
is used as the most recent version 4.6 (Nov 2021) fails to build.
See e.g. https://sourceware.org/git/?p=systemtap.git;a=commit;h=1d3653936fc1fd13135a723a27e6c7e959793ad0
As Systemtap itself is not needed, the build steps (configure and
make) are skipped. We require fewer build dependecies and don't
waste time building depends we don't end up using. However, the
configure step would normally processes sys/sdt-config.h.in. The
resulting sdt-config.h defines _SDT_ASM_SECTION_AUTOGROUP_SUPPORT
(either 0 or 1 to indicate whether the assembler supports "?" in
.pushsection directives). For now, we assume all currently used
assemblers supports this feature and remove the check from the
sys/sdt.h header file in a patch.
Co-authored-by: Michael Ford <fanquake@gmail.com>
33796a964a build: Add ability to build qt in depends with -stdlib=libc++ (Hennadii Stepanov)
Pull request description:
This PR makes possible to build the `qt` package in depends against `libc++` for x86_64 platform.
Fixes#22344.
Required for #22815.
Also this PR [fixes](https://github.com/bitcoin/bitcoin/pull/23060#discussion_r716077050) the `[no wallet] [bionic]` task on CI:
- on master (a8bbd4cc81), https://api.cirrus-ci.com/v1/task/5558609250615296/logs/ci.log:
```
Options used to compile and link:
external signer = yes
multiprocess = no
with libs = yes
with wallet = no
with gui / qt = no
```
- this PR, https://api.cirrus-ci.com/v1/task/5502605561430016/logs/ci.log:
```
Options used to compile and link:
external signer = yes
multiprocess = no
with libs = yes
with wallet = no
with gui / qt = yes
```
ACKs for top commit:
fanquake:
ACK 33796a964a - While this sort of string matching is fragile, I think the risk of this causing any actual issues is low.
Tree-SHA512: 586dde2e9864cec7a49aeb4f2b77fb8c4ae96bd10b51f9c6de0cfe8512ad61db15bb7f8d1b0eb6a5a66fd2deee52ac52218f01eb6be107ac12f1a956190de54b
66a20a54a2 build, qt: Drop support for `i686-linux-android` host (Hennadii Stepanov)
Pull request description:
There are no reasons to keep support for `i686-linux-android` host, which is actually broken in master (50c502f54a), and this fact has been unnoticed for months :)
https://github.com/bitcoin/bitcoin/pull/23675#issuecomment-986206434:
> I'm surprised `i686-linux-android` ABI is still supported. I would love to drop it...
https://github.com/bitcoin/bitcoin/pull/23675#issuecomment-991340132
> What is `i686-linux-android`? 32-bit x86 android? is that really a thing?
ACKs for top commit:
prusnak:
utACK 66a20a54a2
Tree-SHA512: 211f794de2fc569f0ade2a4da805b8bfd4ce2ab0930c5d427acc4f5d015fcdc4911f02fc64f6401197f7641aed79944a9594be80c817547be3269cdd721cf79b
Since Qt 5.14.0 the CROSS_COMPILE variable always being overridden by
the Qt build system internally.
See upstream commit: c28b881c98fadcd3415370fad2525b558f6b03e4
6575d354c8 build: Bump Fonconfig version up to 2.12.6 (Hennadii Stepanov)
Pull request description:
This PR gets rid of `remove_char_width_usage.patch`.
Some additional observations:
1. Newer Fontconfig versions (2.13.0 and 2.13.1) introduce a new dependency, `uuid`, in the [`7b48fd3dd406b926f0e5240b211f72197ed538a9`](7b48fd3dd4) commit
2. In Fonconfig 2.13.1 (the current stable) excludes the `fcobjshash.h` from the distributive archive (see [`31269e3589e0e6432d12f55db316f4c720a090b5`](31269e3589)), that makes our `gperf_header_regen.patch` unusable, and requires `gperf` as a dependency.
ACKs for top commit:
fanquake:
ACK 6575d354c8 - from the best I can determine this doesn't have any versioning / ABI implications. The ABI difference between 2.12.1 and 2.12.6 is two symbol additions, neither of which are used by Qt. Fontconfig seems to be better at maintaining backwards compatibility compared to a library like Freetype.
Tree-SHA512: 36780a0c5a658469697e524d682ebab56c320cb04f8297bc215f4552f183d4f560501fb0a869982fd9053d4a2d571c7fd971d8f5e96c9da9a9d142c485e3baa4
We need to append-to rather than set CXXFLAGS, otherwise we loose -O2 &
-pipe. Currently this results in zeromq being built without optimizations
at all (or whatever the compiler would default too, essentially always -O0).
Bdb is the same, for the CXX portion of its code. C code has been built
with -O2.
Boost has actually been uneffected because it receives -O3 from it's own
build flags.
acaac6e86a ci: Bump Android NDK to r22 which supports std::filesystem (Hennadii Stepanov)
cac7890386 build: Add support for Android NDK r22+ (Hennadii Stepanov)
Pull request description:
This is required to support [`std::filesystem`](https://github.com/android/ndk/wiki/Changelog-r22#changes) on Android (see #20744).
Fixes#22074.
ACKs for top commit:
icota:
re-tACK acaac6e86a
Tree-SHA512: ecbec374ee590c4cb30012210f1422d469e7e8b68989f9eb53d36b5feee150d31e6bd10e1fc4a2056fbf4f8f8513e435b446e5feaf21a3a4d09dfc561fb22e73
54c7754f31 build: use aarch64 Clang if cross-compiling for darwin on aarch64 (fanquake)
Pull request description:
If we're cross-compiling for darwin on aarch64 hardware, we need to
use a Clang that will run on that hardware.
Only tested in a Linux Docker container (aarch64-unknown-linux-gnu),
running on an Apple M1 mac-mini (aarch64-apple-darwin20.5.0).
ACKs for top commit:
hebasto:
ACK 54c7754f31, I agree it can be merged (fix in #22448 is orthogonal to this one).
Tree-SHA512: 66c530097a5dc072a0a00dc22eb3d4a7d923dfa8ab8160f7c3e395cbe58da324f367548d673c0510606f5225d5d37bb5607a76b1703b8b03ac7d2cceeccbd542
If we're cross-compiling for darwin on aarch64 hardware, we need need to
use a Clang that will run on that hardware.
Only tested in a Linux Docker container (aarch64-unknown-linux-gnu),
running on an Apple M1 mac-mini (aarch64-apple-darwin20.5.0).
a58868d201 build: Makes rcc output always deterministic (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler ([rcc](https://doc.qt.io/qt-5/rcc.html)) has a command-line option `--format-version` which has the [default value](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/main.cpp?h=5.12.10#n172) 2.
The only difference from `--format-version 1` is adding a [last modified timestamp](https://code.qt.io/cgit/qt/qtbase.git/tree/src/tools/rcc/rcc.cpp?h=5.12.10#n207) to the output file ([credits](https://github.com/bitcoin/bitcoin/pull/21654#issuecomment-819198228) to **fanquake**). That, in turn, forces us to use `QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds (#13732).
This change makes rcc output always deterministic by using `--format-version 1` option that makes usage of the
`QT_RCC_SOURCE_DATE_OVERRIDE` needless.
---
Also it improves interaction with ccache:
On master (f6c44e999b):
```
$ make && make clean && ccache --zero-stats && make && ccache --show-stats
...
cache directory /home/hebasto/.ccache
primary config /home/hebasto/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Sun Apr 11 15:45:43 2021
stats zeroed Sun Apr 11 15:45:05 2021
cache hit (direct) 638
cache hit (preprocessed) 0
cache miss 1
cache hit rate 99.84 %
called for link 10
cleanups performed 0
files in cache 20023
cache size 13.2 GB
max cache size 15.0 GB
```
The missed file is always `qt/libbitcoinqt_a-qrc_bitcoin_locale.o`.
With this PR:
```
$ make && make clean && ccache --zero-stats && make && ccache --show-stats
...
cache directory /home/hebasto/.ccache
primary config /home/hebasto/.ccache/ccache.conf
secondary config (readonly) /etc/ccache.conf
stats updated Sun Apr 11 15:28:46 2021
stats zeroed Sun Apr 11 15:28:21 2021
cache hit (direct) 639
cache hit (preprocessed) 0
cache miss 0
cache hit rate 100.00 %
called for link 10
cleanups performed 0
files in cache 20012
cache size 13.2 GB
max cache size 15.0 GB
```
ACKs for top commit:
fanquake:
ACK a58868d201
Tree-SHA512: 52f4a3267f41883d13025c0de79b6da22e92d60c729e01b986935c6812bbfe7fadc40b742bd715bfdf09df94af6838d4fbbe8208c6123f366108e38c8e1121c5
After two reports on IRC of issues building depends on an Apple M1
machine, it turns out that this option can't be used when targeting
arm-apple-darwin. For now, just use it for x86_64-apple-darwin.
```bash
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.4.0
error: option 'cf-protection=return' cannot be specified on this target
error: option 'cf-protection=branch' cannot be specified on this target
2 errors generated.
```
42b589d18f scripts: test for MACHO control flow instrumentation (fanquake)
469a5bc4fa build: build Boost with -fcf-protection when targeting Darwin (fanquake)
Pull request description:
Addresses the macOS portion of #21888.
Build Boost with `-fcf-protection` when targeting Darwin. This should be ok, because our cross-compiler (Clang 10) supports the option, and I'd expect all versions of Apple Clang being used to compile Core would also support it. Building Boost with this option is required so that the `main` provided to `test_bitcoin` has instrumentation.
Note that the presence of instrumentation does not mean it will be used, as that is determined at runtime by the CPU.
From the Intel control flow enforcement documentation:
> The ENDBR32 and ENDBR64 instructions will have the same effect as the NOP instruction on Intel 64 processors that do not support CET. On processors supporting CET, these instructions do not change register or flag state. This allows CET instrumented programs to execute on processors that do not support CET. Even when CET is supported and enabled, these NOP–like instructions do not affect the execution state of the program, do not cause any additional register pressure, and are minimally intrusive from power and performance perspectives.
Follow up from #21135.
Guix builds:
```bash
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 guix-build-42b589d18fed/output/dist-archive/bitcoin-42b589d18fed.tar.gz
45e841661e1659a634468b6f8c9fb0a7956c31ba296f1fd0c02cd880736d6127 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.dmg
0ea85c99fef35429a5048fa14850bce6b900eaa887aeea419b019852f8d2be78 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx-unsigned.tar.gz
85857a5a4a5d4d3a172d6c361c12c4a94f6505fc12b527ea63b75bfe54ee1001 guix-build-42b589d18fed/output/x86_64-apple-darwin18/bitcoin-42b589d18fed-osx64.tar.gz
```
Gitian builds:
```bash
# macOS:
bdfd677a6b88273a741b433e1e7f554af50cc76b3342d44ab0c441e2b40efc96 bitcoin-42b589d18fed-osx-unsigned.dmg
f3b2d09f3bea7a5cc489b02e8e53dd76a9922338500fae79cad0506655af56f9 bitcoin-42b589d18fed-osx-unsigned.tar.gz
29d5ad5e46bc9fb0056922a8b47c026e5e9f71e6cf447203b74644587d6fb6f7 bitcoin-42b589d18fed-osx64.tar.gz
663df8471400f06d4da739e39a886aa17f56a36d66e0ff7cc290686294ef39c9 src/bitcoin-42b589d18fed.tar.gz
366f8d7a2fc1f3e22cb1018043099126a71ce65380cc27b1c3280cce42d06c98 bitcoin-core-osx-22-res.yml
```
ACKs for top commit:
laanwj:
Code review ACK 42b589d18f
Tree-SHA512: 12cb8d462d64d845b9fe48c5c6978892adff8bf5b5572bb29f35df1f6176e47b32a68bcb6e4883c7d9454e76e8868851005a7325916852a2d0d32659ac7dae3f
b95f7f8ac0 build, qt, refactor: Drop sed commands for win32-g++/qmake.conf (Hennadii Stepanov)
Pull request description:
Such possibility is [available](https://codereview.qt-project.org/c/qt/qtbase/+/165348) since Qt 5.8.0.
ACKs for top commit:
fanquake:
ACK b95f7f8ac0
Tree-SHA512: e56a3d208a6bd5d42c722f8b344010fe7d1b6f7a28486613dfcb03f0403a47cee8476e2366eeaac401a19836cd09f782e8741a1e781ab4d78f72c500a30e4929
The Qt Resource Compiler (rcc) has a command-line option
`--format-version` which has the default value 2.
The only difference from `--format-version 1` is adding a last modified
timestamp to the output file. That, in turn, forces us to use
`QT_RCC_SOURCE_DATE_OVERRIDE=1` to get deterministic builds.
This change makes rcc output always deterministic by using
`--format-version 1` option that makes usage of the
`QT_RCC_SOURCE_DATE_OVERRIDE` needless. Also it improves interaction
with ccache.
Co-authored-by: fanquake <fanquake@gmail.com>
The LLVM Clang we use for cross-compilation supports this option, and it's expected
that any builders on macOS will also be using an Apple Clang that supports it.
c799a19b4b build, qt: No longer need to set QT_RCC_TEST=1 for determinism (Hennadii Stepanov)
Pull request description:
The Qt Resource Compiler (rcc) output order relies on [`QHash`](https://doc.qt.io/qt-5/qhash.html):
> This randomization of `QHash` is enabled by default. Even though programs should never depend on a particular `QHash` ordering, there may be situations where you temporarily need deterministic behavior, for example for debugging or regression testing. To disable the randomization, define the environment variable `QT_HASH_SEED` to have the value 0.
Since #3620 we use `QT_RCC_TEST=1` to achieve a deterministic output.
Since Qt 5.3.1 hash seeding is disabled for all of the bootstrapped tools, including rcc. Therefore, `QT_RCC_TEST=1` is no longer needed.
See commit [5283a6c87beac5a43f612786fefd6e43f2c70bf6](5283a6c87b).
ACKs for top commit:
fanquake:
ACK c799a19b4b
Tree-SHA512: 9d116ac1e8c605ee3e8ed7f618586f0de85d8b06bbbb70fe8c298939ce203d2a7e97264a9afac037179993ab54c5f69a65ebb9ab27ca7f45acb963011bd45743
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.
This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
