The libs in this dir are the following:
```bash
ls /gnu/store/2vnbkrdin4rrf7ygnr80mlcglin4qqa4-gcc-toolchain-12.3.0-static/lib/lib
libanl.a libc.a libdl.a libm.a
libBrokenLocale.a libcrypt.a libg.a libmcheck.a
libpthread.a librt.a
libresolv.a libutil.a
```
These do not need to be propogated into the Windows build environment.
73d92309d7 guix: use GCC 11 for macOS builds (fanquake)
Pull request description:
Note that this is just the native compiler, which is used to build the toolchain we use to build the actual binaries.
Partially motivated by #29091, where it could now be a bit confusing if we are explicitly using GCC 10 in our release toolchain, when our minimum required is 11 (this can't be bumped to 12 due to build issues with native tools).
At the same time, remove `gcc-toolchain "static"` from the macOS build env.
ACKs for top commit:
hebasto:
ACK 73d92309d7.
Tree-SHA512: 31392290b327cc0e19498cf053b7c9eb19e70295933d650b29b29589356ad455d35b6addcdaae702a9635513c07070fb17d61bcb48445d3cb1a9d4a93aa6ddf3
fd8527a20e guix: remove errant leftover from #29648 (fanquake)
Pull request description:
We no longer build a lib, so a non-existent dir is causing builds to fail.
ACKs for top commit:
josibake:
ACK fd8527a20e
hebasto:
ACK fd8527a20e.
TheCharlatan:
ACK fd8527a20e
Tree-SHA512: 9175a0de3f95f56939b3eaa3e89dca2cfae4996bcd84ef6b8e2872672bef39cb0550c9f4a79475d887eb8fac92c15dfa8c352648ff167d54a0b736978412226c
This includes a commit to fix building LLVM 17 on riscv64, see
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4e26331a5ee87928a16888c36d51e270f0f10f90.
Followup to discussion in
https://github.com/bitcoin/bitcoin/pull/28880#issuecomment-1843313196.
If you don't have riscv64 hardware, this can be tested with the
following:
```bash
guix time-machine --commit=d5ca4d4fd713a9f7e17e074a1e37dda99bbb09fc -- build --target=riscv64-linux-gnu llvm
....
riscv64-linux-gnu-ld: CMakeFiles/dsymutil.dir/dsymutil.cpp.o: undefined reference to symbol '__atomic_fetch_and_1@@LIBATOMIC_1.0'
riscv64-linux-gnu-ld: /gnu/store/i4ga0pnr1b74bir2bjyp8mcrrbsvk7d3-gcc-cross-riscv64-linux-gnu-11.3.0-lib/riscv64-linux-gnu/lib/libatomic.so.1:
error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
guix time-machine --commit=dc4842797bfdc5f9f3f5f725bf189c2b68bd6b5a -- build --target=riscv64-linux-gnu llvm
....
grafting '/gnu/store/7y0j0y8jaz4mjx2nz0y42wdnxxjp6id6-llvm-17.0.6-opt-viewer' -> '/gnu/store/8xvahrrjscbprh6cjj0qp5bm9mm78wwa-llvm-17.0.6-opt-viewer'...
grafting '/gnu/store/bjhw648bz7ijd2p9hgzzdbw1q8hpagk8-llvm-17.0.6' -> '/gnu/store/x50qi8i2ywgpx6azv4k55ms0w5xjxxg5-llvm-17.0.6'...
successfully built /gnu/store/q9xvk8gzzvb4dxfzf6yi5164zd0d1vj2-llvm-17.0.6.drv
```
The zip for codesigned MacOS distribution needs to have all files have
the same timestamp. These files also need to be included in the zip as
zip is not automatically recursive. We use the same pattern for zip as
is done for the other zip files produced by guix.
Without ffile-prefix-map, the debug symbols will contain paths for the
guix store which will include the hashes of each package. However, the
hash for the same package will differ when on different architectures.
In order to be reproducible regardless of the architecture used to build
the package, map all guix store prefixes to something fixed, e.g. /usr.
Now that we use GCC 10 for release builds, we no-longer need to
pass-Wl,-z,noexecstack to get a non-executable stack in RISC-V binaries.
This was originally removed in #21036, but then re-added in #21799, when
we reverted to using GCC 8.
There are two reasons to perform this bump:
* Fixes#25082 by bumping to a commit that includes a fix for time-dependent unit
tests in libgit2 (f5fe0082abe4547f3fb9f29d8351473cfb3a387b).
* Gives us access to clang-toolchain-14 (14.0.3, 998eda3067c7d21e0d9bb3310d2f5a14b8f1c681),
which is useful for the Guix portion of #21778.
Note that with this bump:
Linux kernels headers update from 5.15.28 to 5.15.37.
copy over bitcoin.conf during the build process.
this means `contrib/devtools/gen-bitcoin-conf.sh` will need
to be run and the generated file committed during the release process.
this is the same process used for generating man pages for each release.
97af652788 guix: Drop code for the unsupported `i686-linux-gnu` host (Hennadii Stepanov)
Pull request description:
Now GUIX build for the `i686-linux-gnu` host is broken, and [there are no plans to re-add it](https://github.com/bitcoin/bitcoin/pull/24448).
ACKs for top commit:
fanquake:
ACK 97af652788
Tree-SHA512: 968181aff65e607a7c1a1b06ac7dfd79f6e2ce49b3c4c3828def020e925769fdbab1859d37ea924ded7632405b30539ac3ec81ac714cb9a01a2f7d5c93301dd9
2f356a0ca8 scripted-diff: Drop Darwin version for better maintainability (Hennadii Stepanov)
Pull request description:
After this PR, any macOS tools version bumping in the future will touch fewer files in the repo.
Pointing a Darwin version for the `--host` system does not matter for the following reasons:
- in terms of the resulted binaries, we should only care about the minimum supported macOS version which is a separated parameter in our build system.
- in terms of the build system itself, the usage of the `$(host)` variable is self-consistent enough. Btw `$(host_os)` value already has the version dropped:
```
$ make -C depends --no-print-directory print-host_os HOST=x86_64-apple-darwin19
host_os=darwin
```
ACKs for top commit:
gruve-p:
ACK 2f356a0ca8
promag:
ACK 2f356a0ca8.
fanquake:
ACK 2f356a0ca8
Tree-SHA512: 374896ab0ba02b0d8b4b21431fe963bd213b0d09586e0898c13a4c5fa294c1b693f1b2c92880c245c4157c14217b4825b36522f461930477f4d2a727086ebb2a
a3f61676e8 test: Make more shell scripts verifiable by the `shellcheck` tool (Hennadii Stepanov)
Pull request description:
Some shell scripts from `contrib/guix` and `contrib/shell` are not verifiable by the `shellcheck` tool for the following reasons:
- they have no extension (see 4eccf063b2 from bitcoin/bitcoin#21375)
- they have the `.bash` extension while `.sh` is expected
This PR adds these scripts to the input for the `shellcheck` tool, and it fixes discovered `shellcheck` warnings.
ACKs for top commit:
dongcarl:
Code Review ACK a3f61676e8, this is a good robustness improvement for our shell scripts.
jamesob:
crACK a3f61676e8
Tree-SHA512: 6703f5369d9c04c1a174491f381afa5ec2cc4d37321c1b93615abcdde4dfd3caae82868b699c25b72132d8c8c6f2e9cf24d38eb180ed4d0f0584d8c282e58935
From what I can see the only platform this drops support for is CentOS
7. CentOS 7 reached the end of it's "full update" support at the end of
2020. It does receive maintenance updates until 2024, however I don't
think supporting glibc 2.17 until 2024 is realistic. Note that anyone
wanting to self-compile and target a glibc 2.17 runtime could build with
--disable-threadlocal.
glibc 2.18 was released in August 2013.
https://sourceware.org/legacy-ml/libc-alpha/2013-08/msg00160.html
0f95247246 Integrate univalue into our buildsystem (Cory Fields)
9b49ed656f Squashed 'src/univalue/' changes from 98fadc0909..a44caf65fe (fanquake)
Pull request description:
This PR more tightly integrates building Univalue into our build system. This follows the same approach we use for [LevelDB](https://github.com/bitcoin-core/leveldb/), ([`Makefile.leveldb.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.leveldb.include)), and [CRC32C](https://github.com/bitcoin-core/crc32c) ([`Makefile.crc32c.include`](https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include)), and will be the same approach we use for [minisketch](https://github.com/sipa/minisketch); see #23114.
This approach yields a number of benefits, including:
* Faster configuration due to one less subconfigure being run during `./configure` i.e 22s with this PR vs 26s
* Faster autoconf i.e 13s with this PR vs 17s
* Improved caching
* No more issues with compiler flags i.e https://github.com/bitcoin/bitcoin/pull/12467
* More direct control means we can build exactly the objects we want
There might be one argument against making this change, which is that builders should have the option to use "proper shared/system libraries". However, I think that falls down for a few reasons. The first being that we already don't support building with a number of system libraries (secp256k1, leveldb, crc32c); some for good reason. Univalue is really the odd one out at the moment.
Note that the only fork of Core I'm aware of, that actively patches in support for using system libs, also explicitly marks them as ["DANGEROUS"](a886811721/configure.ac (L1430)) and ["NOT SUPPORTED"](a886811721/configure.ac (L1312)). So it would seem they exist more to satisfy a distro requirement, as opposed to something that anyone should, or would actually use in practice.
PRs like #22412 highlight the "issue" with us operating with our own Univalue fork, where we actively fix bugs, and make improvements, when upstream (https://github.com/jgarzik/univalue) may not be taking those improvements, and by all accounts, is not currently actively maintained. Bitcoin Core should not be hamstrung into not being able to fix bugs in a library, and/or have to litter our source with "workarounds", i.e #22412, for bugs we've already fixed, based on the fact that an upstream project is not actively being maintained. Allowing builders to use system libs is really only exacerbating this problem, with little benefit to our project. Bitcoin Core is not quite like your average piece of distro packaged software.
There is the potential for us to give the same treatment to libsecp256k1, however it seems doing that is currently less straightforward.
ACKs for top commit:
dongcarl:
ACK 0f95247246 less my comment above, always nice to have an include-able `sources.mk` which makes integration easier.
theuni:
ACK 0f95247246. Thanks fanquake for keeping this going.
Tree-SHA512: a7f2e41ee7cba06ae72388638e86b264eca1b9a8b81c15d1d7b45df960c88c3b91578b4ade020f8cc61d75cf8d16914575f9a78fa4cef9c12be63504ed804b99
I used Guix's values for the powerpc64(le) dynamic linkers, and the
/lib-prefix seems to be a Guix-ism rather than standard. The standard
path for the linker-loaders start with /lib64.
I've taken the new loader values from SYSDEP_KNOWN_INTERPRETER_NAMES in
glibc's sysdeps/unix/sysv/linux/powerpc/ldconfig.h file.
For future reference, loader path values can also be found on glibc's
website: https://sourceware.org/glibc/wiki/ABIList?action=recall&rev=16
This addresses issues like the one in #12467, where some of our compiler flags
end up being dropped during the subconfigure of Univalue. Specifically, we're
still using the compiler-default c++ version rather than forcing c++17.
We can drop the need subconfigure completely in favor of a tighter build
integration, where the sources are listed separately from the build recipes,
so that they may be included directly by upstream projects. This is
similar to the way leveldb build integration works in Core.
Core benefits of this approach include:
- Better caching (for ex. ccache and autoconf)
- No need for a slow subconfigure
- Faster autoconf
- No more missing compile flags
- Compile only the objects needed
There are no benefits to Univalue itself that I can think of. These changes
should be a no-op there, and to downstreams as well until they take advantage
of the new sources.mk.
This also removes the option to use an external univalue to avoid similar ABI
issues with mystery binaries.
Co-authored-by: fanquake <fanquake@gmail.com>
Previously, if the builder exported $VERSION in their environment (as
past Gitian-building docs told them to), but their HEAD does not
actually point to v$VERSION, their build outputs will differ from those
of other builders.
This is because the contrib/guix/guix-* scripts only ever act on the
current git worktree, and does not try to check out $VERSION if $VERSION
is set in the environment.
Setting $VERSION only makes the scripts pretend like the current
worktree is $VERSION.
This problem was seen in jonatack's attestation for all.SHA256SUMS,
where only his bitcoin-22.0rc3-osx-signed.dmg differed from everyone
else's.
Here is my deduced sequence of events:
1. Aug 27th: He guix-builds 22.0rc3 and uploads his attestations up to
guix.sigs
2. Aug 30th, sometime after POSIX time 1630310848: he pulls the latest
changes from master in the same worktree where he guix-built 22.0rc3
and ends up at 7be143a960
3. Aug 30th, sometime before POSIX time 1630315907: With his worktree
still on 7be143a960, he guix-codesigns. Normally, this would result
in outputs going in guix-build-7be143a960e2, but he had
VERSION=22.0rc3 in his environment, so the guix-* scripts pretended
like he was building 22.0rc3, and used 22.0rc3's guix-build directory
to locate un-codesigned outputs and dump codesigned ones.
However, our SOURCE_DATE_EPOCH defaults to the POSIX time of HEAD
(7be143a960), which made all timestamps in the resulting codesigned
DMG 1630310848, 7be143a960e2's POSIX timestamp. This differs from the
POSIX timestamp of 22.0rc3, which is 1630348517. Note that the
windows codesigning procedure does not consider SOURCE_DATE_EPOCH.
We resolve this by only allowing VERSION overrides via the FORCE_VERSION
environment variable.
