native_ds_store now takes advantage of Pythons ability to decode binary
plists (since 3.4), so we can drop its biplist dependency.
The call to biplist.Data() in custom_dsstore doesn't seem to do anything,
and from what I can tell can just be removed.
Previously, when running ./configure:
1. With CONFIG_SITE pointed to our depends config.site.in, and
2. PYTHONPATH was not set either in the environment or by the user
The configure would output something like:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages:'
When we really mean:
PYTHONPATH='depends/x86_64-pc-linux-gnu/share/../native/lib/python3/dist-packages'
...without the colon
This change makes sure that:
1. There's no trailing colon, and
2. We use the $PATH_SEPARATOR variable instead of a colon
Files like config.site.in are not referenced by any other script in our
tree, so we need to mark it manually with a "shellcheck shell="
directive and make sure that shellcheck is run on them.
Previously, if ./configure was invoked with:
```
$ env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
```
Where $CONFIG_SITE was a relative path, ./configure would fail with the
following misleading output:
```
checking for boostlib >= 1.58.0 (105800)... yes
checking whether the Boost::System library is available... yes
configure: error: Could not find a version of the Boost::System library!
```
Fully resolving depends_prefix in config.site.in fixes this. To make
sure that there are no other side effects I ran a diff on the
config.status generated by:
1. The scripts prior to this change with CONFIG_SITE set to a full path:
env CONFIG_SITE=$PWD/depends/x86_64-pc-linux-gnu/share/config.site ./configure
2. The scripts after this change with CONFIG_SITE set to a relative path:
env CONFIG_SITE=depends/x86_64-pc-linux-gnu/share/config.site ./configure
And it looks good!
Diff: https://paste.sr.ht/~dongcarl/95b469fbc555c128046e85723d87a9082a754f6b
d0a829e963 build: fix mutex detection when building bdb on macOS (fanquake)
Pull request description:
Starting with the Apple Clang shipped with Xcode 12, [Apple has enabled -Werror=implicit-function-declaration by default](https://developer.apple.com/documentation/xcode-release-notes/xcode-12-release-notes):
> Clang now reports an error when you use a function without an explicit declaration when building C or Objective-C code for macOS (-Werror=implicit-function-declaration flag is on). This additional error detection unifies Clang’s behavior for iOS/tvOS and macOS 64-bit targets for this diagnostic. (49917738)
This causes bdbs mutex detection to fail when building on macOS (not cross-compiling):
```bash
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
```
as previously emitted warnings are being turned into errors. i.e:
```bash
configure:18704: checking for mutexes
configure:18815: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang -mmacosx-version-min=10.12 --sysroot /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX.sdk -o conftest -pipe -O2 -I/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/include -L/Users/michael/github/fanquake-bitcoin/depends/x86_64-apple-darwin19.6.0/lib conftest.c -lpthread >&5
conftest.c:46:1: warning: type specifier missing, defaults to 'int' [-Wimplicit-int]
main() {
^
conftest.c:51:2: error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
exit (
^
conftest.c:51:2: note: include the header <stdlib.h> or explicitly provide a declaration for 'exit'
1 warning and 1 error generated.
```
Append `-Wno-error=implicit-function-declaration` to `cflags` so that `-Wimplicit-function-declaration` [returns to being a warning](https://clang.llvm.org/docs/UsersManual.html#cmdoption-wno-error), and the configure checks succeed.
Fixes#19411.
ACKs for top commit:
laanwj:
Code review ACK d0a829e963
Tree-SHA512: 7813005b1fc0b370f843b6c0672acab32c999416e92c3f02b75d866e9c7aa41fe5822704fc74de6b65f0d7d94f2cdd05cc7c3ee83295ff1ecbc71d8492b9a2bf
Starting with the Clang shipped with Xcode 12, Apple has enabled
-Werror=implicit-function-declaration by default. This causes bdbs mutex
detection to fail when building on macOS (not cross-compiling):
checking for mutexes... UNIX/fcntl
configure: WARNING: NO SHARED LATCH IMPLEMENTATION FOUND FOR THIS PLATFORM.
configure: error: Unable to find a mutex implementation
as previously emitted warnings are being turned into errors. i.e:
error: implicitly declaring library function 'exit' with type 'void (int) __attribute__((noreturn))' [-Werror,-Wimplicit-function-declaration]
Append -Wno-error=implicit-function-declaration to cflags so that
-Wimplicit-function-declaration returns to being a warning, and the
configure checks will succeed.
Fixes#19411.
After #19685 started setting LDFLAGS, the INSTALL_RPATH_USE_LINK_PATH cmake
option used in the libmultiprocess build no longer works, so it is neccessary
to set CMAKE_INSTALL_RPATH as a fallback.
It's unclear currently whether the bad interaction between
INSTALL_RPATH_USE_LINK_PATH and LDFLAGS is a bug, but the issue is reported:
https://github.com/bitcoin/bitcoin/issues/19981#issuecomment-696680877https://discourse.cmake.org/t/install-rpath-use-link-path-not-working-when-cmake-exe-linker-flags-ldflags-is-set/1892
Commands useful for building / testing this change
make -C depends MULTIPROCESS=1 print-libmultiprocess_cmake
make -C depends MULTIPROCESS=1 print-native_libmultiprocess_cmake
make -C depends MULTIPROCESS=1 HOST=x86_64-apple-darwin16 print-libmultiprocess_cmake
rm -rvf depends/x86_64-pc-linux-gnu/native depends/work/staging depends/work/build
make -C depends MULTIPROCESS=1 V=1 native_libmultiprocess_staged
for f in `find -name mpgen`; do echo == $f ==; readelf -d $f | grep -i path; done
make -C depends MULTIPROCESS=1 V=1 native_libmultiprocess_built
find -name CMakeCache.txt
Fixes#19981
7a89f2e6c5 build: Fix target name (Hennadii Stepanov)
Pull request description:
It seems like a typo :)
This PR:
- fixes errors when building a package in depends for `HOST=x86_64-apple-darwin16` (fix#19799)
- is a correct alternative to d25e0e308f from #19764
ACKs for top commit:
icota:
tACK 7a89f2e6c5
dongcarl:
Code Review ACK 7a89f2e6c5
theuni:
ACK 7a89f2e6c5.
Tree-SHA512: a0bcbc6805d3450e201476ef1e22e0eb53903db1586c5515314c19afd337bded887e56de0fbe62feaf359b2de15dbccd49a44f1a8b566b4c64f5ae3d94a2ab6d
This is an alternative to #19751 that fixes the build without requiring
splitting out libpng. This patch can be dropped once we are building qt
5.12.0 or later.
b893688357 depends: Specify LDFLAGS to cmake as well (Carl Dong)
b3f541f618 depends: Prepend CPPFLAGS to C{,XX}FLAGS for CMake (Carl Dong)
8e121e5509 depends: Cleanup CMake invocation (Carl Dong)
8c7cd0c6d9 depends: More robust cmake invocation (Carl Dong)
3ecf0eca63 depends: Use $($(package)_cmake) instead of cmake (Carl Dong)
Pull request description:
- Use `$($(package)_cmake)` instead of invoking `cmake` directly
- Use well-known env vars instead of overriding CMake variables
ACKs for top commit:
ryanofsky:
Code review ACK b893688357. Only changes since last review are new commits adding whitespace, cppflags and ldflags to cmake invocation
Tree-SHA512: cfcd8cc9dcd0b336cf48b82fca9fe4bbc7930ed397cb7a68a07066680eb4c1906a6a9b5bd2589b4b4999e8f16232fa30ee9b376b60f4456d0fff931fbf9cc19a
The usage of pragmas within the macOS SDK requires LLVM Clang 8. This is
the version as our prebuilt Clang, however the minimum is worth noting here
as they may diverge and/or expert users might expect they could use an
earlier version.
If you compile using Clang 7 you'll see output like:
```bash
In file included from kernel/qcore_mac_objc.mm:44:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSText.h:9:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSView.h:19:
In file included from /bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSResponder.h:10:
/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/System/Library/Frameworks/AppKit.framework/Headers/NSEvent.h:19:1: error:
expected 'push' or 'pop' after '#pragma clang attribute'
/bitcoin/depends/SDKs/Xcode-11.3.1-11C505-extracted-SDK-with-libcxx-headers/usr/include/os/availability.h:104:273: note: expanded from macro
'API_UNAVAILABLE_BEGIN'
...__API_UNAVAILABLE_BEGIN5, __API_UNAVAILABLE_BEGIN4, __API_UNAVAILABLE_BEGIN3, __API_UNAVAILABLE_BEGIN2, __API_UNAVAILABLE_BEGIN1, 0)(__VA_A...
^
fatal error: too many errors emitted, stopping now [-ferror-limit=]
20 errors generated.
```
5962522fbc depends: bump native_cctools for fixed lto with external clang (Cory Fields)
00d1ba7aaa depends: enable lto support for Apple's ld64 (Cory Fields)
Pull request description:
This didn't work for a few reasons (various toolchain compatibility issues) the last time I tested it, but after the last round of bumps it works with no apparent issues.
Note that this does not _enable_ LTO by default in any way, only hooks up the machinery for ```-flto``` to work correctly when specified.
Lines were split for an easier rebase after #17919 is merged.
ACKs for top commit:
fanquake:
ACK 5962522fbc. The relevant option upstream is [here](https://github.com/tpoechtrager/cctools-port/blob/master/cctools/m4/llvm.m4#L4).
Tree-SHA512: df2775e74e7bc847e6cef94cb8457d503d6c9e2fdea861e51386fa6ed5a7ba688241db3685561ae1a32f66724c1b3801727252025f00c04b90a3bdc8a4f6f93b
2a701a1c42 build: pass -fcommon when building genisoimage (fanquake)
Pull request description:
Starting with the 10.1 release, GCC [defaults to -fno-common](https://gcc.gnu.org/gcc-10/porting_to.html). This causes
linking issues when building genisoimage:
```bash
[ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
[100%] Linking C executable genisoimage
/usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
```
Rather than patching genisoimage further, just pass -fcommon to preserve
the legacy GCC behaviour.
Noticed [while testing #19530](https://github.com/bitcoin/bitcoin/pull/19530#issuecomment-659802512).
ACKs for top commit:
laanwj:
ACK 2a701a1c42
hebasto:
ACK 2a701a1c42, tested on Fedora 32 (x86_64, GCC 10.1.1):
Tree-SHA512: 0e6dd1b4c7c3f5b6997616da27a7ea54f757c81677766ee6813c776356527787a32b959f834e699fee9798553b13bcb142469f442c5e7a04f810f2c06e3a9505
Starting with the 10.1 release, GCC defaults to -fno-common. This causes
linking issues when building genisoimage:
```bash
[ 98%] Building C object genisoimage/CMakeFiles/genisoimage.dir/checksum.o
[100%] Linking C executable genisoimage
/usr/bin/ld: CMakeFiles/genisoimage.dir/apple.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/boot.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/desktop.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
/usr/bin/ld: CMakeFiles/genisoimage.dir/dvd_file.o:(.bss+0x0): multiple definition of `outfile'; CMakeFiles/genisoimage.dir/genisoimage.o:(.bss+0x0): first defined here
```
Rather than patching genisoimage further, pass -fcommon to preserve the
legacy GCC behaviour.
6457361e90 qt: Fix QFileDialog for static builds (Hennadii Stepanov)
Pull request description:
This change partially reverts 248e22bbc0 (#16386) and makes `QFileDialog`s work again for static builds.
Fixes https://github.com/bitcoin-core/gui/issues/32.
ACKs for top commit:
fanquake:
ACK 6457361e90. Although it would be good to know exactly _why_ this fixes the issue. At this stage I also don't think this should be a blocker for 0.20.1.
theuni:
ACK 6457361e90
Tree-SHA512: 8ad27e0bcae6debd02f73b7c374743e37d4edd806922b103a2fe494cf2d9930fe9ef3107b5a6c61f3c466cf7462de2641171880398954e7f2c4f417f5bb820d7
Note that this does not _enable_ lto by default in any way, only hooks up the
machinery for -flto to work correctly.
enable-lto-support is explicitly used for pinned-clang because we know it
works. It is neither enabled nor disabled in the external clang case so that
it can be auto-detected.
For depends builds this was fixed by fbcfcf69, which deleted the conflicting
headers. When we no longer control the clang installation, we need to ensure
that the SDK's libc++ headers are used rather than the ones shipped with clang.
We can do that by turning off the default include path and hard-coding our own.
This hard-coded path is ok because we control (via SDK packaging) where these
headers end-up.
Side-note: Now that this path is hard-coded in depends, we can potentially
package the SDK differently, as the c++ folder can live wherever is most
convenient for us.
This should be caught by the differing clang --version outputs, but because we
haven't yet extracted our pinned clang, the system one is actually used for the
version check. That's not a problem because bumping our pinned clang will
cause a rebuild of everything anyway.
a8d39b8840 doc: explain why passing -mlinker-version is required (fanquake)
Pull request description:
I have been down a 🐇 hole. Closes#19359.
When Clang is compiled, [a check is run](https://github.com/llvm/llvm-project/blob/release/8.x/clang/CMakeLists.txt#L353) to define `HOST_LINK_VERSION` as the output of `$CMAKE_LINKER -v`. Note the this is the version of the linker being used to compile Clang itself.. and this check is only run when compiling Clang for macOS.
In the Clang driver, if `HOST_LINK_VERSION` has been defined, there is some additional runtime functionality. An `-mlinker-version` argument, with the value of `HOST_LINK_VERSION` [will be added to the linker arguments](89de0d8dfb/clang/lib/Driver/Driver.cpp (L382)), if `-mlinker-version` has not been passed in by the user.
This is a bit weird, as by default, you are setting `-mlinker-version` to the version of the linker that was used to build the Clang binary, not the linker which will be used when compiling. The commit which introduced the functionality, 628fcf4e3b, described it as a "hack", that should be replaced. However, that was 10 years ago, and the behaviour is still here.
In the Darwin driver, [a check is done](89de0d8dfb/clang/lib/Driver/ToolChains/Darwin.cpp (L208)) for the `-mlinker-version` argument. If there is no argument, the version will default to `0`. Given the above, this should never happen when using Clang for macOS. A series of comparisons are then performed, to check whether the linker version is modern enough to enable certain features, like [`-demangle`](89de0d8dfb/clang/lib/Driver/ToolChains/Darwin.cpp (L215)).
### What this means
#### macOS
A Clang compiled for macOS, i.e `clang+llvm-8.0.0-x86_64-apple-darwin`, will have `HOST_LINKER_VERSION` set to the version of the linker used to compile Clang itself.
At runtime, `-mlinker-version=HOST_LINKER_VERSION` will be added to the linker args, if `-mlinker-version` wasn't passed in. In the Darwin driver, additional arguments, like `-demangle`, will be added to the linker arguments, because `HOST_LINKER_VERSION` was likely some very modern version of `lld` or `ld64`.
#### Linux (cross compilation in depends)
A Clang compiled for Linux, i.e `clang+llvm-8.0.0-x86_64-linux-gnu-ubuntu-14.04`, which we now use for macOS builds in depends, will behave differently. As it's built for Linux, `HOST_LINKER_VERSION` was not defined at compile time, and there will be no default behaviour of appending `-mlinker-version=HOST_LINKER_VERSION` to the linker args. Thus, unless you pass in `-mlinker-version` yourself, when the version checks are done in the Darwin driver, no modern linker features will be enabled, as the version will have defaulted to `0`.
Therefore, it's important that we continue to pass `-mlinker-version="our LD64 version"` as part of our compilation flags, if we want to have "modern" linker features enabled for our macOS builds.
#### Summary
[Clang 8](https://releases.llvm.org/download.html#8.0.0). Building a macOS binary. Link line with path arguments trimmed.
| | default behaviour | `-mlinker-version=100` (`-demangle threshold`) | `-mlinker-version=530` |
| - | --------------- | --------------------- | ---------------------- |
| macOS Clang | `-demangle -lto_library ../libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.15.0 -o a.out ../test-b8b9b3.o -lc++ -lSystem ../libclang_rt.osx.a` | `-demangle -dynamic -arch x86_64 -macosx_version_min 10.15.0 -o a.out ../test-a66966.o -lc++ -lSystem ../libclang_rt.osx.a` | same as default |
| Linux Clang | `-dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-bfce57.o -lc++ -lSystem` | `-demangle -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-a846a3.o -lc++ -lSystem` | `-demangle -lto_library ../libLTO.dylib -no_deduplicate -dynamic -arch x86_64 -macosx_version_min 10.12.0 -o a.out ../test-de0280.o -lc++ -lSystem` |
Note: Most links here are pointing to the 8.x branch of LLVM/Clang, as we are using that version in depends.
Note: To add a little more confusion, you wont see `-mlinker-version X` in your compile flags, you'll see [`-target-linker-version X`](431daedee4/clang/lib/Driver/ToolChains/Clang.cpp (L4777)).
ACKs for top commit:
laanwj:
ACK a8d39b8840
Tree-SHA512: 92f93079a5e59a0d561e74336b5cb03e3bf5a34437f5850283b9128c7624494b8285ec16290b1fa8103fe87f8789a53ce44b17902b8c1db5fde24d74b76fb168
