Using GCC 11 for the macOS build hasn't been required since #21778, and
at this point, given a toolchain is still needed (#30206), it makes more
sense to (re-)use 12, rather than make all builders compile another
GCC toolchain.
If the binaries don't exist, the Guix build has failed for some other
reason.
There's no need to check for unknown architectures, or executable
formats, as the only ones that could be built are those that we've
configured toolchains for in Guix.
We've also been doing this inconsistently across the two scripts.
9010b1343b contrib: c++ify test stubs after switching to c++ compilers (Cory Fields)
261f770333 contrib: rename cc to cxx in binary checking scripts (Cory Fields)
a38c960005 contrib: use c++ rather than c for binary tests (Cory Fields)
Pull request description:
From hebasto's CMake repo. See discussion here: https://github.com/hebasto/bitcoin/pull/252#discussion_r1664657488
Use CXX/CXXFLAGS rather than CC/CFLAGS to test our actual compiler for binary checks rather than the one we only forward to secp256k1.
ACKs for top commit:
hebasto:
ACK 9010b1343b.
fanquake:
ACK 9010b1343b
Tree-SHA512: 7b8788d7d3760103062eff10056c995e1ad14c0c487d9414683ad54d816c255d0ca751f4d0e2d2ad7f9e8a7101d8c7f1e9333fa5b137558ed68fa593c4b4ce6d
34c9cee380 clang-tidy: add check for non-trivial thread_local vars (Cory Fields)
Pull request description:
Forbid thread_local vars with non-trivial destructors.
This is a follow-up from: https://github.com/bitcoin/bitcoin/pull/30095#discussion_r1608423170
ACKs for top commit:
maflcko:
ACK 34c9cee380
TheCharlatan:
Re-ACK 34c9cee380
Tree-SHA512: 3a798607fb189a5bbc714ed6e86dea462fe29d366b790e96d10a7b4ffcf1f194da9b8f4cd0b82154408709b8e3c58d3f613d6311903bd65a76d8b556ab230d21
Introduce diff_addrs subcommand as means for a Bitcoin-centric
comparison of two ASMaps.
In addition to two ASMaps, the subcommand reads addresses from
a (getnodeaddresses-compatible) file, and provides information on
addresses that have mismatching ASN according to the two ASMaps.
b5fc6d46a3 guix: use glibc 2.31 (fanquake)
Pull request description:
Set minimum required glibc to 2.31.
The glibc 2.31 branch is still maintained: https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.
Remove the stack-protector check from test-security-check, as the test
no-longer fails, and given the control we have of the end, the actual
security-check test seems sufficient (this might also be applied to some
of the other checks).
Drops runtime support for Ubuntu Bionic 18.04 and RHEL-8 from the release binaries.
ACKs for top commit:
TheCharlatan:
ACK b5fc6d46a3
Tree-SHA512: ba7e727240fa0ebebfb8b749024c71cbfdec37c33b39627866d78f9318ccdc687fd5103a63ee0e98cf809d9954dde56b1b305691c33d1de275ed0519f716c921
3ab2520190 contrib: Fixup verify-binaries OS platform parsing (Ben Westgate)
Pull request description:
Closes#30145.
This PR solves two major issues in the `parse_version_string` function of verify-binaries:
1. `-aarch64` binaries cannot be specifically downloaded. The -platform string gets interpreted as a release candidate that doesn't exist due to containing sub-string "rc".
2. Specifying a platform with a "-" in the name causes the parser to ignore both "-platform" AND "-rcN" and download the potentially wrong (non-rc) version for every platform. This also prevented specifying just one platform binary the user wished to download.
It also updates the accompanying `test.py` to cover problem two and adds two examples that were formerly broken to `README.md` to show what is now possible. Including the most useful case of downloading only 1 specific platform's binary.
This improves the Bitcoin verify-binaries tools user experience by not:
1. Failing to download for inexplicable reasons,
2. Downloading more files than what the user told it to, or in the worst case
3. Downloading only the wrong files.
* A test was added to cover the command `verify-binaries/verify.py pub 22.0-x86_64-linux-gnu.tar.gz` which checks that _bitcoin-22.0-x86_64-linux-gnu.tar.gz_ downloads successfully AND ONLY _bitcoin-22.0-x86_64-linux-gnu.tar.gz_ downloads.
* The steps to reproduce each bug are in the referenced issue #30145. Explanation of the potential issue as well as reasoning for the way the bug was fixed are in my commit descriptions.
* This delivers the promised feature of "only download the binaries for a certain platform", by allowing strings with '-' to be accepted, allowing for single file downloads for any specific platform which was not always possible before.
* Removes 6 lines of code from the offending `parse_version_string` function, while fixing the bugs/errors, and extending the functionality to be practical for users with slow connections.
* Makes the error message more helpful when no file matches the provided platform string, now prints "Did you mean: `closest-match`" to help correct typos.
Thanks for reading my PR. I look forward to getting this helpful tool in its best shape yet.
Log of this branch passing the new test.py:
```
python3 test.py
✓ 'Nonexistent version should fail' passed
✓ 'Malformed version should fail' passed
✓ '--min-good-sigs 20 should fail' passed
- testing verification (22.0-x86_64-linux-gnu.tar.gz)
✓ '22.0-x86_64-linux-gnu.tar.gz should succeed' passed
- testing verification (22.0)
✓ '22.0 should succeed' passed
```
Log of master failing the new test.py:
```
python3 test.py
✓ 'Nonexistent version should fail' passed
✓ 'Malformed version should fail' passed
✓ '--min-good-sigs 20 should fail' passed
- testing verification (22.0-x86_64-linux-gnu.tar.gz)
✓ '22.0-x86_64-linux-gnu.tar.gz should succeed' passed
Traceback (most recent call last):
File "/home/ben/Documents/GitHub/bitcoin/contrib/verify-binaries/test.py", line 74, in <module>
main()
File "/home/ben/Documents/GitHub/bitcoin/contrib/verify-binaries/test.py", line 27, in main
assert len(v) == 1
^^^^^^^^^^^
AssertionError
```
ACKs for top commit:
stickies-v:
re-ACK 3ab2520190
willcl-ark:
re-ACK 3ab2520190
Tree-SHA512: 6093228bb876cd0ac84d1cd2630074e17a3f09c4b23325b9419d859a5721a802f928844575233b135df52b882287dd18d6fadf4419d88ec0a2cdf82db315329e
Parse platform strings with "-" or '.' correctly such as "linux-gnu" or
"x86_64-linux-gnu.tar.gz" to download the matching files or file. String
partition() is used to tolerate more dashes. Update `VERSION_EXAMPLE`
with a new string parsed correctly now. Fix "-aarch64" interpreted as a
release candidate due to sub-string "rc", causing all downloads to fail.
Now "rc" must immediately follow first "-" to indicate an [-rc] string.
Local variables `version_rc`, `version_os` renamed to `rc`, `platform`.
If "-rcN" is specified, `platform` is reassigned to remove the '-rcN'.
Changes are useful to only download one bitcoin core binary on slow
connections. Making `verify.py pub` more intuitive, robust, and
versatile. Closes#30145
When user types a platform string not found in any filename lets help
and say the platform closest to what they typed in a `f"No files
matched the platform specified. Did you mean: {closest_match}"` log.
Improves UX when unaware how we name our files.
Uses the difflib Python built-in which was already imported elsewhere.
Update test.py to test single file verification
verify-binaries/verify.py can accept an entire filename filter for its
"-platform" parameter now so let us test that it succeeds and downloads
and verifies only one file. `verify.py pub 22.0-x86_64-linux-gnu.tar.gz`
should get and verify only the requested binary. It is placed before the
existing <version> wide verification as it is a faster test and possibly
easier to break.
Update doc with examples now possible after bugfix
Add example to show release candidates now work with "-platform" strings
containing "-" and string provided can be from the middle of filename:
`./contrib/verify-binaries/verify.py --json pub 23.0-rc5-linux-gnu`
Change example 5 to not match example 3.
New examples to show platform can now be provided specifically enough to
download only a single binary down to its file extension:
`./contrib/verify-binaries/verify.py pub 25.2-x86_64-linux`
`./contrib/verify-binaries/verify.py pub 24.1-rc1-darwin`
`./contrib/verify-binaries/verify.py pub 27.0-win64-setup.exe`
This is the most common use if not verifying all files so users see it
as the first example for "only download the binaries for a certain
architecture and/or platform". Downloading one file is intuitively what
most will think this meant and this change delivers on that expectation.
Co-authored-by: stickies-v
7c298fe0df doc: rewrite some of the macdeploy docs (fanquake)
d042230f7a depends: swap mmacosx-version-min for mmacos-version-min (fanquake)
Pull request description:
Whilst `-mmacosx-version-min` and `-mmacos-version-min` remain aliases for each other, the later is preferred,
and I assume the former will be removed at some point in the future; see: https://github.com/llvm/llvm-project/pull/95374.
Somewhat of a followup to #21778. Rewrite some of the mac deploy docs.
ACKs for top commit:
theuni:
utACK 7c298fe0df
TheCharlatan:
ACK 7c298fe0df
hebasto:
ACK 7c298fe0df.
Tree-SHA512: 6493f087fde93e0eec319af0e105d163b3f047d8a03f7d4b0d6cd7c64b58d0a978b7d67c6b8dba5c6ccf8b10e188aab5dc98eec400b0546dc9ee801a689b4332
c7376babd1 doc: Clarify distinction between util and common libraries in libraries.md (Ryan Ofsky)
4f74c59334 util: Move util/string.h functions to util namespace (Ryan Ofsky)
4d05d3f3b4 util: add TransactionError includes and namespace declarations (Ryan Ofsky)
680eafdc74 util: move fees.h and error.h to common/messages.h (Ryan Ofsky)
02e62c6c9a common: Add PSBTError enum (Ryan Ofsky)
0d44c44ae3 util: move error.h TransactionError enum to node/types.h (Ryan Ofsky)
9bcce2608d util: move spanparsing.h to script/parsing.h (Ryan Ofsky)
6dd2ad4792 util: move spanparsing.h Split functions to string.h (Ryan Ofsky)
23cc8ddff4 util: move HexStr and HexDigit from util to crypto (TheCharlatan)
6861f954f8 util: move util/message to common/signmessage (Ryan Ofsky)
cc5f29fbea build: move memory_cleanse from util to crypto (Ryan Ofsky)
5b9309420c build: move chainparamsbase from util to common (Ryan Ofsky)
ffa27af24d test: Add check-deps.sh script to check for unexpected library dependencies (Ryan Ofsky)
Pull request description:
Remove `fees.h`, `errors.h`, and `spanparsing.h` from the util library. Specifically:
- Move `Split` functions from `util/spanparsing.h` to `util/string.h`, using `util` namespace for clarity.
- Move remaining spanparsing functions to `script/parsing.h` since they are used for descriptor and miniscript parsing.
- Combine `util/fees.h` and `util/errors.h` into `common/messages.h` so there is a place for simple functions that generate user messages to live, and these functions are not part of the util library.
Motivation for this change is that the util library is a dependency of the kernel, and we should remove functionality from util that shouldn't be called by kernel code or kernel applications. These changes should also improve code organization and make functions easier to discover. Some of these same moves are (or were) part of #28690, but did not help with code organization, or made it worse, so it is better to move them and clean them up in the same PR so code only has to change one time.
ACKs for top commit:
achow101:
ACK c7376babd1
TheCharlatan:
Re-ACK c7376babd1
hebasto:
re-ACK c7376babd1.
Tree-SHA512: 5bcef16c1255463b1b69270548711e7ff78ca0dd34e300b95e3ca1ce52ceb34f83d9ddb2839e83800ba36b200de30396e504bbb04fa02c6d0c24a16d06ae523d
Set minimum required glibc to 2.31.
The glibc 2.31 branch is still maintained:
https://sourceware.org/git/?p=glibc.git;a=shortlog;h=refs/heads/release/2.31/master.
Remove the stack-protector check from test-security-check, as the test
no-longer fails, and given the control we have of the end, the actual
security-check test seems sufficient (this might also be applied to some
of the other checks).
Drops runtime support for Ubuntu Bionic 18.04 and RHEL-8 from the release binaries.
429ec1aaaa refactor: Rename CTransaction::nVersion to version (Ava Chow)
27e70f1f5b consensus: Store transaction nVersion as uint32_t (Ava Chow)
Pull request description:
Given that the use of a transaction's nVersion is always as an unsigned int, it doesn't make sense to store it as signed and then cast it to unsigned everywhere it is used and displayed.
Since a few alternative implementations have recently been revealed to have made an error with this signedness that would have resulted in consensus failure, I think it makes sense for us to just make this always unsigned to make it clear that the version is treated as unsigned. This would also help us avoid future potential issues with signedness of this value.
I believe that this is safe and does not actually change what transactions would or would not be considered both standard and consensus valid. Within consensus, the only use of the version in consensus is in BIP68 validation which was already casting it to uint32_t. Within policy, although it is used as a signed int for the transaction version number check, I do not think that this change would change standardness. Standard transactions are limited to the range [1, 2]. Negative numbers would have fallen under the < 1 condition, but by making it unsigned, they are still non-standard under the > 2 condition.
Unsigned and signed ints are serialized and unserialized the same way so there is no change in serialization.
ACKs for top commit:
maflcko:
ACK 429ec1aaaa 🐿
glozow:
ACK 429ec1aaaa
shaavan:
ACK 429ec1aaaa💯
Tree-SHA512: 0bcd92a245d7d16c3665d2d4e815a4ef28207ad4a1fb46c6f0203cdafeab1b82c4e95e4bdce7805d80a4f4a46074f6542abad708e970550d38a00d759e3dcef1
In order to ensure that the change of nVersion to a uint32_t in the
previous commit has no effect, rename nVersion to version in this commit
so that reviewers can easily spot if a spot was missed or if there is a
check somewhere whose semantics have changed.
fa52e13ee8 test: Remove struct.pack from almost all places (MarcoFalke)
fa826db477 scripted-diff: test: Use int.to_bytes over struct packing (MarcoFalke)
faf2a975ad test: Use int.to_bytes over struct packing (MarcoFalke)
faf3cd659a test: Normalize struct.pack format (MarcoFalke)
Pull request description:
`struct.pack` has many issues:
* The format string consists of characters that may be confusing and may need to be looked up in the documentation, as opposed to using easy to understand self-documenting code.
This lead to many test bugs, which weren't hit, which is fine, but still confusing. Ref: https://github.com/bitcoin/bitcoin/pull/29400, https://github.com/bitcoin/bitcoin/pull/29399, https://github.com/bitcoin/bitcoin/pull/29363, fa3886b7c6, ...
Fix all issues by using the built-in `int` helpers `to_bytes` via a scripted diff.
Review notes:
* For `struct.pack` and `int.to_bytes` the error behavior is the same, although the error messages are not identical.
* Two `struct.pack` remain. One for float serialization in a C++ code comment, and one for native serialization.
ACKs for top commit:
achow101:
ACK fa52e13ee8
rkrux:
tACK [fa52e13](fa52e13ee8)
theStack:
Code-review ACK fa52e13ee8
Tree-SHA512: ee80d935b68ae43d1654b047e84ceb80abbd20306df35cca848b3f7874634b518560ddcbc7e714e2e7a19241e153dee64556dc4701287ae811e26e4f8c57fe3e
For example:
```bash
ADDITIONAL_GUIX_COMMON_FLAGS set in the ENV
ADDITIONAL_GUIX_ENVIRONMENT_FLAGS="--emulate-fhs" ./contrib/guix/guix-build
<snip>
INFO: Building f75199182133 for platform triple x86_64-linux-gnu:
...using reference timestamp: 1716905119
...running at most 10 jobs
...from worktree directory: '/bitcoin'
...bind-mounted in container to: '/bitcoin'
...in build directory: '/bitcoin/guix-build-f75199182133/distsrc-f75199182133-x86_64-linux-gnu'
...bind-mounted in container to: '/distsrc-base/distsrc-f75199182133-x86_64-linux-gnu'
...outputting in: '/bitcoin/guix-build-f75199182133/output/x86_64-linux-gnu'
...bind-mounted in container to: '/outdir-base/x86_64-linux-gnu'
ADDITIONAL FLAGS (if set)
ADDITIONAL_GUIX_COMMON_FLAGS: --no-substitutes
ADDITIONAL_GUIX_ENVIRONMENT_FLAGS: --emulate-fhs
ADDITIONAL_GUIX_TIMEMACHINE_FLAGS:
```
Adjust the security check for:
ld64.lld: warning: Option `-allow_stack_execute' is not yet implemented.
ld64.lld: error: -fixup_chains is incompatible with -no_pie
and to account for the embedding of LLVMs version number.
b59a027d95 contrib: drop dead get_machine from test sym check (fanquake)
e6aba463ad contrib: use env_flags in get_arch (fanquake)
Pull request description:
This isn't an issue right now (because the get_arch check is simple), but becomes one as soon as we want to use `lld` for linking, and need LDFLAGS (otherwise we call `ld` and fail, see it's usage in #21778). So I've split this out for review. It also makes sense to use the same flags for all compilation in these checks.
Also drops some dead code in test-symbol-check.
ACKs for top commit:
TheCharlatan:
ACK b59a027d95
Tree-SHA512: d8afc4144815369aae63cf6dc6e983af46f208c7043d6ea5c9c811152649c256a8e67eb6864ea9d385d87b6b049fece07710a84b90da325da7fc3f05efcaacd6
7f5ac4520d build: swap otool for (llvm-)objdump (fanquake)
Pull request description:
This tool is used in GUI packaging on macOS, and also somewhat of a blocker for #21778. The main issue is that some distros don't really ship this tool in a standard ways, i.e Ubuntu only ships `llvm-otool` with a version suffix, i.e `llvm-otool-17`, which makes it hard to find and use. Rather than trying to deal with that mess, switch to using the equivalent LLVM tool (objdump), which is a drop-in replacement.
ACKs for top commit:
TheCharlatan:
ACK 7f5ac4520d
theuni:
ACK 7f5ac4520d. Tested `make deploy` on native macOS. Looks good.
hebasto:
ACK 7f5ac4520d.
Tree-SHA512: ac978043f14fb448010542a4a7ce8c6c74b4cbd90f83b4cb4d0bff55974010f10a70b5354f65b239a8bd961d7a3aca22ca165b42954ca87879b9e0524db5f879
6abe772a17 contrib: Add asmap-tool (Fabian Jahr)
Pull request description:
This adds `asmap.py` and `asmap-tool.py` from sipa's `nextgen` branch: https://github.com/sipa/asmap/tree/nextgen
The motivation is that we should maintain the tooling for de- and encoding asmap files within the bitcoin core repository because it is not possible to use an asmap file that is not encoded.
We already had an earlier version of `asmap.py` within the seeds contrib tools. The newer version only had a small amount of changes and is still compatible, so the old version is removed from contrib/seeds and the new version is made available to `makeseeds.py`.
ACKs for top commit:
virtu:
ACK [6abe772](6abe772a17)
0xB10C:
ACK 6abe772a17
achow101:
ACK 6abe772a17
brunoerg:
ACK 6abe772a17
Tree-SHA512: cc2a82ffa4eb46fa0ce4ca769dd82f8d0d2f37fc3652aa748eeb060e1142f9da4035008fe89433e2fd524a4dc153b7b9c085748944b49137b37009b0c0be8afb
Similar to libtool, (llvm-)otool only exists with a version suffix
on some systems (Ubuntu), which makes it annoying to use/find. Avoid
this, by switching to objdump. Which is a drop-in replacement.
This is related to #21778, and the switchover to using vanilla LLVM for
macOS.
b8e084b978 guix: remove no-longer-used bzip2 (fanquake)
bd6e1d6718 depends: switch qrencode to .tar.gz (fanquake)
4a9b71b900 depends: switch libxcb_util_wm to .tar.gz (fanquake)
fad989852d depends: switch libxcb_util_render to .tar.gz (fanquake)
ce28cb31b4 depends: switch libxcb_util_keysyms to .tar.gz (fanquake)
00a6896346 depends: switch libxcb_util_image to .tar.gz (fanquake)
8e9190c6aa depends: switch libxcb_util to .tar.gz (fanquake)
b845029d46 depends: switch xproto to .tar.gz (fanquake)
5996c30384 depends: switch libXau to .tar.gz (fanquake)
e7a8dd5931 depends: switch fontconfig to .tar.gz (fanquake)
58c423def3 depends: switch boost to .tar.gz (fanquake)
Pull request description:
This moves packages in depends that use `.tar.bzip2` to `.tar.gz` (which is what we use for our own release tarballs). Doing so means we can drop `bzip2` from our Guix release env. You can observe that Guix building master without it would currently fail:
```diff
diff --git a/contrib/guix/manifest.scm b/contrib/guix/manifest.scm
index 8f13c642d3..96818c7748 100644
--- a/contrib/guix/manifest.scm
+++ b/contrib/guix/manifest.scm
@@ -499,7 +499,6 @@ inspecting signatures in Mach-O binaries.")
moreutils
;; Compression and archiving
tar
- bzip2
gzip
xz
;; Build tools
```
`FORCE_DIRTY_WORKTREE=1 ./contrib/guix/guix-build`
```bash
Extracting boost...
/sources/boost_1_81_0.tar.bz2: OK
tar (child): lbzip2: Cannot exec: No such file or directory
tar (child): Error is not recoverable: exiting now
tar: Child returned status 2
tar: Error is not recoverable: exiting now
```
Guix Build:
```bash
8f6959d01ae972bae1340dfaf18753607152eca9844e6d8c4fa128314a4ba762 guix-build-b8e084b9781e/output/aarch64-linux-gnu/SHA256SUMS.part
3c9c1cc000e3e6b7c2853c9d530c9afa1c880a43e7ab4c766aaa88283ff0908c guix-build-b8e084b9781e/output/aarch64-linux-gnu/bitcoin-b8e084b9781e-aarch64-linux-gnu-debug.tar.gz
f45fbece697b450538aded11f568e92b2af391e873e113c3038d022eff41688f guix-build-b8e084b9781e/output/aarch64-linux-gnu/bitcoin-b8e084b9781e-aarch64-linux-gnu.tar.gz
08295d770c11b2057206f98aaf4123007c7475bd942840d048f5f9d5efec1ce1 guix-build-b8e084b9781e/output/arm-linux-gnueabihf/SHA256SUMS.part
0a0db6967168019b8b890ec4d31b3a87a88c4956b703938ec4447d514cfc231e guix-build-b8e084b9781e/output/arm-linux-gnueabihf/bitcoin-b8e084b9781e-arm-linux-gnueabihf-debug.tar.gz
3d1538e8bf4edfb66a4875198dfa90b79dcfe44eb9c4e76e47d73a18175c838a guix-build-b8e084b9781e/output/arm-linux-gnueabihf/bitcoin-b8e084b9781e-arm-linux-gnueabihf.tar.gz
87e7805155dbed3bd64763f199ea63843ed8c4eb37873753c7e60b0b42565eaf guix-build-b8e084b9781e/output/arm64-apple-darwin/SHA256SUMS.part
fa33590296aeae2b738b023a4cbf2de4a4e06662a5f7d407c251a8af714bd587 guix-build-b8e084b9781e/output/arm64-apple-darwin/bitcoin-b8e084b9781e-arm64-apple-darwin-unsigned.tar.gz
32b8fbbdf240f9f08e44c7bb0a8ea2e8a40537e59ec2231cf6635edc6592f226 guix-build-b8e084b9781e/output/arm64-apple-darwin/bitcoin-b8e084b9781e-arm64-apple-darwin-unsigned.zip
d176f3b7c8140c8dfde03bd87fd5abd4a89b497ba11fa6849bc92a33cb621a07 guix-build-b8e084b9781e/output/arm64-apple-darwin/bitcoin-b8e084b9781e-arm64-apple-darwin.tar.gz
5273b17087e3565ab042a7989cfba71cf1629331d0267137d7eccabee1a06a13 guix-build-b8e084b9781e/output/dist-archive/bitcoin-b8e084b9781e.tar.gz
b84a9180181994a6a17a1c2a4701f8ba5a82654233d5a8afcf596d28dd8b3924 guix-build-b8e084b9781e/output/powerpc64-linux-gnu/SHA256SUMS.part
fd3396f6b64425a31b5a3565ab4d8a1c1668c291349a0f9e9b8904dad04ee24c guix-build-b8e084b9781e/output/powerpc64-linux-gnu/bitcoin-b8e084b9781e-powerpc64-linux-gnu-debug.tar.gz
73cb4bd2a67934c93ea8e3f3bc04b8917627ec09d75c151bb01977bba97522c8 guix-build-b8e084b9781e/output/powerpc64-linux-gnu/bitcoin-b8e084b9781e-powerpc64-linux-gnu.tar.gz
15938e7f0f71303b96566d60e3b255816e7fd70d628601e592e1d6840eb8d2a1 guix-build-b8e084b9781e/output/riscv64-linux-gnu/SHA256SUMS.part
408b4973865e3a77be833438f71181fd88acd0490127257b3667309e8421030e guix-build-b8e084b9781e/output/riscv64-linux-gnu/bitcoin-b8e084b9781e-riscv64-linux-gnu-debug.tar.gz
a5c02144ffb79cfa0179ff0d7ac0f81192ef1d3b1acfad334adf486e50b776bb guix-build-b8e084b9781e/output/riscv64-linux-gnu/bitcoin-b8e084b9781e-riscv64-linux-gnu.tar.gz
de904843d8bb8601a2d763701ebb929e61b447e01040267a12149a2902489535 guix-build-b8e084b9781e/output/x86_64-apple-darwin/SHA256SUMS.part
414cb3cf3fa10b9a3cda47e98858222f01fdd164371dd54761642e6793099849 guix-build-b8e084b9781e/output/x86_64-apple-darwin/bitcoin-b8e084b9781e-x86_64-apple-darwin-unsigned.tar.gz
6ce43d7f007bf17eca16d3ee48190318e09aacd82c5396f9565e6345ec6bd2fa guix-build-b8e084b9781e/output/x86_64-apple-darwin/bitcoin-b8e084b9781e-x86_64-apple-darwin-unsigned.zip
24eba9c0dd1312a68c2b2a800cc915595e343c0ead982b6cbe025abe7a7bff19 guix-build-b8e084b9781e/output/x86_64-apple-darwin/bitcoin-b8e084b9781e-x86_64-apple-darwin.tar.gz
2869a01ce847298950a91b3b8514bc8fa39cc274a8e9cd4f68f4f038c1bb3040 guix-build-b8e084b9781e/output/x86_64-linux-gnu/SHA256SUMS.part
3f63e1d3b19b640d3994074b344d595bcd6fca420a1a8669b63b4ad22978308b guix-build-b8e084b9781e/output/x86_64-linux-gnu/bitcoin-b8e084b9781e-x86_64-linux-gnu-debug.tar.gz
ccc3eb8eb56c1596981e81c8c95cadee3db799ed69b0cd1fb1e102da10adacfb guix-build-b8e084b9781e/output/x86_64-linux-gnu/bitcoin-b8e084b9781e-x86_64-linux-gnu.tar.gz
1ff6dab6dcde9ddbbe407cca02119c4a5d545034c91389a1c647020902b7b40e guix-build-b8e084b9781e/output/x86_64-w64-mingw32/SHA256SUMS.part
a91c2247fd9d886e3f3ada551c0a4f9f7ffc4874e07ea5ab9de14f2743b9b8c7 guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64-debug.zip
6fbc8d5df571fd535990370009bdfcbb37b9697c33446a08eadb1279ba6e4649 guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64-setup-unsigned.exe
38f7a981fd2999c1e138860e1ddc183dafec090d867e37f5ab5c2d48ad4ef9ee guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64-unsigned.tar.gz
88aca0a40a64a289617aad060a9ccf8c78bc6a201470720d8caf48d793d5207f guix-build-b8e084b9781e/output/x86_64-w64-mingw32/bitcoin-b8e084b9781e-win64.zip
```
ACKs for top commit:
laanwj:
This is fully expected (no weird changes from dropping bzip2 from the build env). ACK b8e084b978
Tree-SHA512: 7da9a75a3ff7fa0c9ff464e3a82f5b1d0cfdd28d5de049c910142179f7e1211c922b705361844c7029ce9baaa8e97e8016b454d2e4eee98e31fae1379674fbe2
1a9aa8d4ee build: better scope usage of -Wl,-headerpad_max_install_names (fanquake)
3bee51427a build: don't use install_name_tool for macOS deploy when cross-compiling (fanquake)
78b6b5c485 build: don't pass strip to macOS deploy if cross-compiling (fanquake)
Pull request description:
Neither of these tools are actually used when we are cross-compiling for macOS. They are used when we have to adjust non-static libs during a deploy after building on a macOS machine. Simplies #29739 (will be rebased on top).
Guix (aarch64):
```bash
8f29bce75d7f574306a0e38d793e0e4e145b547a4b9e5a755a54976121d8ac41 guix-build-5afd3c302051/output/arm64-apple-darwin/SHA256SUMS.part
9ba01fe46be715adcbe80f39dc7dbe449f32ca9d9b660da698f933aef3e6d80b guix-build-5afd3c302051/output/arm64-apple-darwin/bitcoin-5afd3c302051-arm64-apple-darwin-unsigned.tar.gz
37719437e951449341d0e10dcc4afe93e955d59de5312ce6351e1fa01b4927ac guix-build-5afd3c302051/output/arm64-apple-darwin/bitcoin-5afd3c302051-arm64-apple-darwin-unsigned.zip
06a79fc871dcd4290f5f7e7e9de19a5a535203d20279f4555d1c319d07abe2d0 guix-build-5afd3c302051/output/arm64-apple-darwin/bitcoin-5afd3c302051-arm64-apple-darwin.tar.gz
98d2b8b37197dcad36a04eb2f3ff2130b859220a17b83a4186a78dcf0af4eafd guix-build-5afd3c302051/output/dist-archive/bitcoin-5afd3c302051.tar.gz
df63ff44ef41565ff13ce6dde5485173a18d5866ebc316df86f9ebd91fda18f5 guix-build-5afd3c302051/output/x86_64-apple-darwin/SHA256SUMS.part
28362ce9e80d5e78db198efa5f89434fbe76ca91df5fde7455da4d50ceb8523a guix-build-5afd3c302051/output/x86_64-apple-darwin/bitcoin-5afd3c302051-x86_64-apple-darwin-unsigned.tar.gz
534745b679eb9e8e408dd251a6bf0829e62e12f7a41772b8a57a044ded14208c guix-build-5afd3c302051/output/x86_64-apple-darwin/bitcoin-5afd3c302051-x86_64-apple-darwin-unsigned.zip
f53d0c9a1bb83d548c7d274c7d39653a3989fb1b4efec49e73dd1cac7c92074c guix-build-5afd3c302051/output/x86_64-apple-darwin/bitcoin-5afd3c302051-x86_64-apple-darwin.tar.gz
```
ACKs for top commit:
TheCharlatan:
ACK 1a9aa8d4ee
Tree-SHA512: 0aa77ea4d6dc45c226806bb1758b6aa7e8ca17f91045bab4fc6891af7b9de476211cd5692c11cb9d5bcf59744fd86a2534812a77fe304ae10c3518e08fc412be
0244416aac security: restrict abis in bitcoind.service (Charlie)
Pull request description:
[As noted here](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#MemoryDenyWriteExecute=), it's a good idea to pair `MemoryDenyWriteExecute=true` with `SystemCallArchitectures=native` because `MemoryDenyWriteExecute` can be circumvented in some operating systems which support multiple ABIs like x86/x86-64.
This helps restrict the possible application binary interfaces (ABIs) that can be used when running bitcoind through systemd, reducing the attack surface area.
ACKs for top commit:
laanwj:
ACK 0244416aac . This is a sensible security feature.
0xB10C:
ACK 0244416aac
Tree-SHA512: 77a35b0674d8d67d857cd20ae1b8cd011f82d6f5ed21bc106cbe45bfa937e786ddc1bf7261e3bdb8c289df1224e91658760905d2c8f37cc4c6506ef8037ad158
05da2460db guix: remove gcc-toolchain static from Windows build (fanquake)
Pull request description:
The libs in this dir are the following:
```bash
ls /gnu/store/2vnbkrdin4rrf7ygnr80mlcglin4qqa4-gcc-toolchain-12.3.0-static/lib/
libanl.a libc.a libdl.a libm.a
libBrokenLocale.a libcrypt.a libg.a libmcheck.a
libpthread.a librt.a
libresolv.a libutil.a
```
These do not need to be propogated into the Windows build environment.
Guix Build (aarch64):
```bash
450c0c4f45f9cb7ed7fc2ef6e7557b6a23004b82c951399da3b7635e8451a076 guix-build-05da2460db89/output/dist-archive/bitcoin-05da2460db89.tar.gz
5df68ab18636090c387bc90297356d0e148b02931d3a99c0f6d33cd268aa072b guix-build-05da2460db89/output/x86_64-w64-mingw32/SHA256SUMS.part
13e979f60d9296aa11081fbbb360404da9fbb797bb4663ed2d1189d800659b4f guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64-debug.zip
d1cc78437a96f012a59af7c757bef592f163559e523d45014d7804d0be29a8b8 guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64-setup-unsigned.exe
33a9cfd4475677646bb32c9c45c25bd796ca5adb126590bf556d4e6f9592c676 guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64-unsigned.tar.gz
5d2ee251668d3d31bf378826ab06f98542dd20926cdee2df5c3315e11222a519 guix-build-05da2460db89/output/x86_64-w64-mingw32/bitcoin-05da2460db89-win64.zip
```
Somewhat similar to #29673.
ACKs for top commit:
laanwj:
ACK 05da2460db
hebasto:
ACK 05da2460db,
Tree-SHA512: bf514a726a22e2bfae4de645b10d90a66fe090971340c4299aeb9b2ff9cf536ca6cfed274d312ea5d5a172775cbda6db0e609492ec603f5aee55c8de81462cc0
The libs in this dir are the following:
```bash
ls /gnu/store/2vnbkrdin4rrf7ygnr80mlcglin4qqa4-gcc-toolchain-12.3.0-static/lib/lib
libanl.a libc.a libdl.a libm.a
libBrokenLocale.a libcrypt.a libg.a libmcheck.a
libpthread.a librt.a
libresolv.a libutil.a
```
These do not need to be propogated into the Windows build environment.
007ea322a6 depends: switch to building libqrencode with CMake (fanquake)
884330c0a5 guix: make cmake-minimal a global requirement (fanquake)
Pull request description:
Switch to building libqrencode with CMake. Note that upstream (https://github.com/fukuchi/libqrencode) hasn't seen any activity for ~4 years, so the odds of getting anything upstream seems low, but I've made two minor changes to the source here, which I will PR in any case.
From an initial look I couldn't find any significant difference between the Autotools and CMake produced libs. As part of this change we move cmake-minimal in Guix into the global package set.
ACKs for top commit:
TheCharlatan:
ACK 007ea322a6
Tree-SHA512: c784f790ddea958082c8ae96d3744bdf99331a8799765f9d44f00861b8e2cfcab1a88a3d64af5b10e51a8d5938d55eb6a3d271790b565e50492a39d00dc0e30f
73d92309d7 guix: use GCC 11 for macOS builds (fanquake)
Pull request description:
Note that this is just the native compiler, which is used to build the toolchain we use to build the actual binaries.
Partially motivated by #29091, where it could now be a bit confusing if we are explicitly using GCC 10 in our release toolchain, when our minimum required is 11 (this can't be bumped to 12 due to build issues with native tools).
At the same time, remove `gcc-toolchain "static"` from the macOS build env.
ACKs for top commit:
hebasto:
ACK 73d92309d7.
Tree-SHA512: 31392290b327cc0e19498cf053b7c9eb19e70295933d650b29b29589356ad455d35b6addcdaae702a9635513c07070fb17d61bcb48445d3cb1a9d4a93aa6ddf3
fd8527a20e guix: remove errant leftover from #29648 (fanquake)
Pull request description:
We no longer build a lib, so a non-existent dir is causing builds to fail.
ACKs for top commit:
josibake:
ACK fd8527a20e
hebasto:
ACK fd8527a20e.
TheCharlatan:
ACK fd8527a20e
Tree-SHA512: 9175a0de3f95f56939b3eaa3e89dca2cfae4996bcd84ef6b8e2872672bef39cb0550c9f4a79475d887eb8fac92c15dfa8c352648ff167d54a0b736978412226c
eff19fa1c8 build, macos: Drop unused `osx_volname` target (Hennadii Stepanov)
Pull request description:
The `osx_volname` makefile target was introduced in https://github.com/bitcoin/bitcoin/pull/7192 and was used to pass the package name to Gitian scripts as a content of the `osx_volname` file.
With the current Guix scripts, the `osx_volname` file is never read. Therefore, its creation might be omitted.
My Guix builds:
```
x86_64
5e2d254e207d53784621c8df331c9bf4a969da667d185992402f48a5ac49f563 guix-build-eff19fa1c8d7/output/arm64-apple-darwin/SHA256SUMS.part
089dba70685893aca5e7c8ce1d53a07380e87ca50eda8b3a2a75aeaeb1d28e48 guix-build-eff19fa1c8d7/output/arm64-apple-darwin/bitcoin-eff19fa1c8d7-arm64-apple-darwin-unsigned.tar.gz
390c57197c6ab4aefdde1c665d5e4ebdfb4ae5e553f8f93b017f2fad1093d110 guix-build-eff19fa1c8d7/output/arm64-apple-darwin/bitcoin-eff19fa1c8d7-arm64-apple-darwin-unsigned.zip
e1edde7ca28bf26aea8d956b1d3c1725a475f2a9c148f5c36b651db4b814091c guix-build-eff19fa1c8d7/output/arm64-apple-darwin/bitcoin-eff19fa1c8d7-arm64-apple-darwin.tar.gz
d0096ea73a5f75cc4d3cef4ef1761ae3e48c8a63aff918f07371c5c88896e4e6 guix-build-eff19fa1c8d7/output/dist-archive/bitcoin-eff19fa1c8d7.tar.gz
51b4affb9fd6f8aea05b7d25d29f017d0a0a145395f457caa14b9af9646b035b guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/SHA256SUMS.part
b1df081ecf636a92754e673e5388d1d988653d4646f0b0446a4c9f14d865a265 guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/bitcoin-eff19fa1c8d7-x86_64-apple-darwin-unsigned.tar.gz
62e09926029d176da950d3e3db7ff8ae6cbe4c0b2ea17b084fc1d28565f91475 guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/bitcoin-eff19fa1c8d7-x86_64-apple-darwin-unsigned.zip
477dcb2382cbd447bd88a3b644b4bd736f5b67d66d42cb73fe31ffc153d3e181 guix-build-eff19fa1c8d7/output/x86_64-apple-darwin/bitcoin-eff19fa1c8d7-x86_64-apple-darwin.tar.gz
```
ACKs for top commit:
Empact:
ACK eff19fa1c8
Sjors:
tACK eff19fa1c8
TheCharlatan:
ACK eff19fa1c8
Tree-SHA512: 29714be5c58caa07b3eb99846d71bb83366dade769af022059bb3c499878adcd34cdf03b006c5da561291f373ccc59abdb83c925057ec0049465eaa8dd6ef4e1
7850c5fe20 guix: build GCC with --enable-standard-branch-protection (fanquake)
Pull request description:
This is one change extracted from #24123 (which now produces fully BTI & PAC enabled bins), which will mean that everything in depends, for Guix builds, is compiled using `-mbranch-protection=standard`.
Turning this on by default, is similar to what we already do with `--enable-default-ssp`, `--enable-default-pie` etc.
See: https://gcc.gnu.org/install/specific.html#aarch64-x-x
> To enable Branch Target Identification Mechanism and Return Address Signing by default at configure time use the `--enable-standard-branch-protection` option.
> This is equivalent to having `-mbranch-protection=standard` during compilation. This can be explicitly disabled during compilation by passing the `-mbranch-protection=none` option which turns off all types of branch protections.
ACKs for top commit:
TheCharlatan:
ACK 7850c5fe20
Tree-SHA512: 18f898da27021bab502e708ea5fa9b325352f8f6e23d9488a2a0feda87e0af2ac0e4f87b3af9ad6a9a37bbfc99ab0285de4f0bdc174dcd38163d92c122e958e2
To enable Branch Target Identification Mechanism and Return
Address Signing by default at configure time use the
`--enable-standard-branch-protection` option.
This is equivalent to having `-mbranch-protection=standard` during
compilation. This can be explicitly disabled during compilation
by passing the `-mbranch-protection=none` option which turns off
all types of branch protections.
See:
https://gcc.gnu.org/install/specific.html#aarch64-x-x
This includes a commit to fix building LLVM 17 on riscv64, see
https://git.savannah.gnu.org/cgit/guix.git/commit/?id=4e26331a5ee87928a16888c36d51e270f0f10f90.
Followup to discussion in
https://github.com/bitcoin/bitcoin/pull/28880#issuecomment-1843313196.
If you don't have riscv64 hardware, this can be tested with the
following:
```bash
guix time-machine --commit=d5ca4d4fd713a9f7e17e074a1e37dda99bbb09fc -- build --target=riscv64-linux-gnu llvm
....
riscv64-linux-gnu-ld: CMakeFiles/dsymutil.dir/dsymutil.cpp.o: undefined reference to symbol '__atomic_fetch_and_1@@LIBATOMIC_1.0'
riscv64-linux-gnu-ld: /gnu/store/i4ga0pnr1b74bir2bjyp8mcrrbsvk7d3-gcc-cross-riscv64-linux-gnu-11.3.0-lib/riscv64-linux-gnu/lib/libatomic.so.1:
error adding symbols: DSO missing from command line
collect2: error: ld returned 1 exit status
guix time-machine --commit=dc4842797bfdc5f9f3f5f725bf189c2b68bd6b5a -- build --target=riscv64-linux-gnu llvm
....
grafting '/gnu/store/7y0j0y8jaz4mjx2nz0y42wdnxxjp6id6-llvm-17.0.6-opt-viewer' -> '/gnu/store/8xvahrrjscbprh6cjj0qp5bm9mm78wwa-llvm-17.0.6-opt-viewer'...
grafting '/gnu/store/bjhw648bz7ijd2p9hgzzdbw1q8hpagk8-llvm-17.0.6' -> '/gnu/store/x50qi8i2ywgpx6azv4k55ms0w5xjxxg5-llvm-17.0.6'...
successfully built /gnu/store/q9xvk8gzzvb4dxfzf6yi5164zd0d1vj2-llvm-17.0.6.drv
```
Retain native GCC 10 toolchain for macOS, to prevent compile failures in
native tools (this will be removed entirely when we tansition to LLD).
Update the vmov-alignment patch, for changes in GCC 12.
d5228efb53 kernel: Remove dependency on CScheduler (TheCharlatan)
06069b3913 scripted-diff: Rename MainSignals to ValidationSignals (TheCharlatan)
0d6d2b650d scripted-diff: Rename SingleThreadedSchedulerClient to SerialTaskRunner (TheCharlatan)
4abde2c4e3 [refactor] Make MainSignals RAII styled (TheCharlatan)
84f5c135b8 refactor: De-globalize g_signals (TheCharlatan)
473dd4b97a [refactor] Prepare for g_signals de-globalization (TheCharlatan)
3fba3d5dee [refactor] Make signals optional in mempool and chainman (TheCharlatan)
Pull request description:
By defining a virtual interface class for the scheduler client, users of the kernel can now define their own event consuming infrastructure, without having to spawn threads or rely on the scheduler design.
Removing `CScheduler` also allows removing the thread and exception modules from the kernel library.
To make the `CMainSignals` class easier to use from a kernel library perspective, remove its global instantiation and adopt RAII practices.
Renames `CMainSignals` to `ValidationSignals`, which more accurately describes its purpose and scope.
Also make the `ValidationSignals` in the `ChainstateManager` and CTxMemPool` optional. This could be useful in the future for using or testing these classes without having to instantiate any form of signal handling.
---
This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587). It improves the kernel API and removes two modules from the kernel library.
ACKs for top commit:
maflcko:
re-ACK d5228efb53🌄
ryanofsky:
Code review ACK d5228efb53. Just comment change since last review.
vasild:
ACK d5228efb53
furszy:
diff ACK d5228ef
Tree-SHA512: e93a5f10eb6182effb84bb981859a7ce750e466efd8171045d8d9e7fe46e4065631d9f6f533c5967c4d34c9bb7d7a67e9f4593bd4c5b30cd7b3bbad7be7b331b
faa30a4c56 rpc: Do not wait for headers inside loadtxoutset (MarcoFalke)
Pull request description:
While the `loadtxoutset` default 10 minute timeout is convenient when it is sufficient, it may cause hassle where it is not. For example:
* When P2P connections are missing, it seems better to abort early than wait for the timeout.
* When the 10 minute timeout is not sufficient, the RPC will have to be called again, so a check or loop is needed outside the RPC either way. So might as well remove the loop inside the RPC.
ACKs for top commit:
fjahr:
ACK faa30a4c56
theStack:
Code-review ACK faa30a4c56
pablomartin4btc:
tACK faa30a4c56
TheCharlatan:
ACK faa30a4c56
Tree-SHA512: 9167c7d8b2889bb3fd369de4acd2cc4d24a2fe225018d82bd9568ecd737093f6e19be7cc62815b574137b61076a6f773c29bff75398991b5cd702423aab2322b
This is done in preparation for the next two commits, where the
CMainSignals are de-globalized.
This avoids adding new constructor arguments to the ChainstateManager
and CTxMemPool classes over the next two commits.
This could also allow future tests that are only interested in the
internal behaviour of the classes to forgo instantiating the signals.
27f260aa6e net: remove now unused global 'g_initial_block_download_completed' (furszy)
aff7d92b15 test: add coverage for peerman adaptive connections service flags (furszy)
6ed53602ac net: peer manager, dynamically adjust desirable services flag (furszy)
9f36e591c5 net: move state dependent peer services flags (furszy)
f9ac96b8d6 net: decouple state independent service flags from desirable ones (furszy)
97df4e3887 net: store best block tip time inside PeerManager (furszy)
Pull request description:
Derived from #28120 discussion.
By relocating the peer desirable services flags into the peer manager, we
allow the connections acceptance process to handle post-IBD potential
stalling scenarios.
The peer manager will be able to dynamically adjust the services flags
based on the node's proximity to the tip (back and forth). Allowing the node
to recover from the following post-IBD scenario:
Suppose the node has successfully synced the chain, but later experienced
dropped connections and remained inactive for a duration longer than the limited
peers threshold (the timeframe within which limited peers can provide blocks). In
such cases, upon reconnecting to the network, the node might only establish
connections with limited peers, filling up all available outbound slots. Resulting
in an inability to synchronize the chain (because limited peers will not provide
blocks older than the `NODE_NETWORK_LIMITED_MIN_BLOCKS` threshold).
ACKs for top commit:
achow101:
ACK 27f260aa6e
vasild:
ACK 27f260aa6e
naumenkogs:
ACK 27f260aa6e
mzumsande:
Light Code Review ACK 27f260aa6e
andrewtoth:
ACK 27f260aa6e
Tree-SHA512: 07befb9bcd0b60a4e7c45e4429c02e7b6c66244f0910f4b2ad97c9b98258b6f46c914660a717b5ed4ef4814d0dbfae6e18e6559fe9bec7d0fbc2034109200953
52149b7a2c contrib: Fix clang-format-diff.py lint errors (TheCharlatan)
008e81e025 contrib: Latest clang-format-diff.py script (TheCharlatan)
Pull request description:
This was taken from 900bb318b5/clang/tools/clang-format/clang-format-diff.py and is useful for systems where clang tools are shipped with a version suffix.
ACKs for top commit:
maflcko:
lgtm ACK 52149b7a2c🌱
Tree-SHA512: cad720b283650e36c5b3ec597278112899ae6413a42c79b6296c58242000f32ae6ab7ed732a3a0f30f1f2586d7e3c0d1ef856a4821e28272bae6f428ed9497e9
We assume to be using python3, so don't check for it. This removes a
type error on the line `from io import BytesIO as StringIO`.
Specify the encoding as "utf8" when opening a file.
5335e454c0 contrib: add macho branch protection check (fanquake)
Pull request description:
Followup to https://github.com/bitcoin/bitcoin/pull/28459. Add a sanity check that `bti` instructions are present in the arm macho binary, similar to our x86_64 check for control flow.
Could do something similar for aarch64 linux in future, and maybe could use https://github.com/lief-project/LIEF/issues/975.
ACKs for top commit:
TheCharlatan:
ACK 5335e454c0
Tree-SHA512: 6cc8721209fe07fe07f0524ef6f114004e2b98844f73d31ff16547f7055c7cb4a5609480058c45ede21b457b2dea5357f1475eaa5063ea1f9772aa260f49039b
This was take from
900bb318b5/clang/tools/clang-format/clang-format-diff.py
Updating it introduces some new options. For example specifying the
clang-format binary, which is useful for systems where clang tools are
shipped with a version suffix.
1f8450f066 doc: upgrade Bitcoin Core license to 2024 (22388o⚡️)
Pull request description:
See https://github.com/bitcoin/bitcoin/pull/26748.
Cherry-picked these commits from 22388o and then squashed them.
ACKs for top commit:
fanquake:
ACK 1f8450f066
Tree-SHA512: 6d12f24a6c7cd421f7d975d7e445de4583144a4d0902d4e68e7648395074ca804e3ee585b2d3f307d193690e2ed2f9fbd8e7938f8fb8af263888b8701993782a
4fdd836db9 Use hardened runtime on macOS release builds. (Mark Friedenbach)
Pull request description:
The Apple notary service requires submitted app bundles to be configured to use the hardened runtime libraries. This is configured at signing time, and supported by the signapple tool Bitcoin Core uses for reproduceable signed binaries. We simply need to pass "--hardened-runtime" when the signature is created. Once attached to the bundle, the resulting codesigned binary can be successfully submitted to the Apple binary notarization service by any Apple Developer.
This partially resolves#15774. The release maintainer, or any authorized Apple Developer, will need to run `xcrun notarytool` to prevent gatekeeper warnings on macOS. Using `xcrun staple` to generate a binary that doesn't call home on first launch would be bonus, but at least this would massively improve the user experience.
ACKs for top commit:
fanquake:
ACK 4fdd836db9 - we can move ahead with this, and figure out notarisation / stapling as a followup.
Tree-SHA512: 7b8ba50030fb230d44bd63d12ed082537e8eaaa61396114c5df715f8dd6772fd8d84b00dc819f88d9a463996c2170a84981fce1bde7f7999b4bdb914fbcdfdac
The Apple notary service requires submitted app bundles to be configured to use the hardened runtime libraries. This is configured at signing time, and supported by the signapple tool Bitcoin Core uses for reproduceable signed binaries. We simply need to pass "--hardened-runtime" when the signature is created. Once attached to the bundle, the resulting codesigned binary can be successfully submitted to the Apple binary notarization service by any Apple Developer.
ff896d2581 contrib: drop GCC MAX_VERSION to 4.3.0 in symbol-check (fanquake)
Pull request description:
Reflect the actual symbols used, i.e:
```bash
bitcoind: symbol __bswapsi2 from unsupported version GCC_4.3.0(7)
```
ACKs for top commit:
TheCharlatan:
ACK ff896d2581
Tree-SHA512: b38ff8f4dd78d2d1c9063c53544dc4f240c3043f142e1581f7ba42f088a509293f6f17cc402c60ac82bff3b36668866b87e0e9e4d10d929484bb4c7a3e654f25
11b7269d83 script: Enhance validations in utxo_snapshot.sh (pablomartin4btc)
Pull request description:
This PR resolves#27841 and some more:
- Ensure that the snapshot height is higher than the pruned block height when the node is pruned (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/28553#issuecomment-1804941396)).
- Validate the correctness of the file path and check if the file already exists (@hazeycode's [#27845](https://github.com/bitcoin/bitcoin/pull/27845)).
- Make network activity disablement optional for the user (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/16899#discussion_r342735815) and [here](https://github.com/bitcoin/bitcoin/pull/16899#issuecomment-536520911)).
- Ensure the `reconsiderblock` command is triggered on exit (@hazeycode's same PR as above), even in the case of user interruption (Ctrl-C).
In order to perform some testing please follow the instructions in the description of previous @hazeycode's PR #27845.
ACKs for top commit:
Sjors:
tACK 11b7269d83
ryanofsky:
Code review ACK 11b7269d83
Tree-SHA512: 2b699894c6f732ad5104f5a2bcf5dc86ed31edcc9d664690cab55b94a8ab00e2ca5bde901ee1d63acddca7ea80ad1734d8cfe78f9c02f8470f264fe93a2af759
f95af98128 guix: default ssp for Windows GCC (fanquake)
95d55b96c2 guix: remove ssp workaround from Windows GCC (fanquake)
8f43302a0a build: remove explicit libssp linking from Windows build (fanquake)
Pull request description:
I was expecting this to fail to compile somewhere, maybe in the CI, but that doesn't seem to be the case?
Seems workable given the SSP related changes in the newer mingw-w64 headers (which are in Guix):
> Implement some of the stack protector functions/variables so -lssp is now optional when _FORTIFY_SOURCE or -fstack-protector-strong is used.
However I think this would still be broken in some older environments, so we might have to wait for a compiler bump, or similar. The optional -lssp also seems to work when using older headers, which doesn't make sense.
Would fix#28104.
ACKs for top commit:
hebasto:
ACK f95af98128, I've verified binaries from `bitcoin-f95af98128f1-win64.zip` on Windows 11 Pro 23H2.
TheCharlatan:
ACK f95af98128
Tree-SHA512: 71169ec513cfe692dfa7741d2bf37b45da05627c0af1cbd50cf8c3c04cc21c4bf88f3284532bddc1e3e648391ec78dbaca5170987a13c21ac204a7bcaf27f349
fa552e8a4e doc: Simplify guix install doc, after 1.4 release (MarcoFalke)
Pull request description:
Now that 1.4 is out (for a while), remove the recommendation to build a random commit.
ACKs for top commit:
fanquake:
ACK fa552e8a4e
hebasto:
ACK fa552e8a4e.
Tree-SHA512: f5642df201ff0e2af8a7ae9660a66920ddbb5f522b3e921f6f4aa7c411ced23afa91bdfe43b943ac012228eebbaad3396df505d00aa8f721a4358f03fda9d8e3
a478c817b2 test: replace `Callable`/`Iterable` with their `collections.abc` alternative (PEP 585) (stickies-v)
4b9afb18e6 scripted-diff: use PEP 585 built-in collection types for verify-binary script (Sebastian Falbesoner)
d516cf83ed test: use built-in collection types for type hints (Python 3.9 / PEP 585) (Sebastian Falbesoner)
Pull request description:
With Python 3.9 / [PEP 585](https://peps.python.org/pep-0585/), [type hinting has become a little less awkward](https://docs.python.org/3.9/whatsnew/3.9.html#type-hinting-generics-in-standard-collections), as for collection types one doesn't need to import the corresponding capitalized types (`Dict`, `List`, `Set`, `Tuple`, ...) anymore, but can use the built-in types directly (see https://peps.python.org/pep-0585/#implementation for the full list).
This PR applies the replacement for all Python scripts (i.e. in the contrib and test folders) for the basic types, i.e.:
- typing.Dict -> dict
- typing.List -> list
- typing.Set -> set
- typing.Tuple -> tuple
For an additional check, I ran mypy 1.6.1 on both master and the PR branch via
```
$ mypy --ignore-missing-imports --explicit-package-bases $(git ls-files "*.py")
```
and verified that the output is identical -- (from the 22 identified problems, most look like false-positives, it's probably worth it to go deeper here and address them in a follow-up though).
ACKs for top commit:
stickies-v:
ACK a478c817b2
fanquake:
ACK a478c817b2
Tree-SHA512: 6948c905f6abd644d84f09fcb3661d7edb2742e8f2b28560008697d251d77a61a1146ab4b070e65b0d27acede7a5256703da7bf6eb1c7c3a897755478c76c6e8
Running the miner under python >= 3.12 causes a SyntaxWarning. The problem was
already present in previous versions, but it only triggered a
DeprecationWarning, which was not shown by default.
The change is useful for future-proofing the code base, since future python
versions will start to exit with a runtime exception (see the reference given
later).
Command to see the warning at runtime under python3.11 (DeprecationWarning,
needs "-Walways"):
$ python3.11 -Walways ./contrib/signet/miner
<BASE>/contrib/signet/miner:33: DeprecationWarning: invalid escape sequence '\d'
RE_MULTIMINER = re.compile("^(\d+)(-(\d+))?/(\d+)$")
2023-11-15 16:02:49 ERROR Must specify command
Command to see the warning at runtime under python3.12 (SyntaxWarning, no
modifiers needed):
$ python3.12 ./contrib/signet/miner
<BASE>/contrib/signet/miner:33: SyntaxWarning: invalid escape sequence '\d'
RE_MULTIMINER = re.compile("^(\d+)(-(\d+))?/(\d+)$")
2023-11-15 16:03:00 ERROR Must specify command
Reference ( https://docs.python.org/3.8/library/re.html ):
Regular expressions use the backslash character ('\') [...]. This collides
with Python’s usage of the same character for the same purpose in string
literals; [...]
Also, please note that any invalid escape sequences in Python’s usage of the
backslash in string literals now generate a DeprecationWarning and in the
future this will become a SyntaxError.
The solution is to use Python’s raw string notation for regular expression
patterns;
f718a74b12 guix: remove python-macholib (fanquake)
d3cbff16c2 guix: update signapple (fanquake)
Pull request description:
Update to the latest signapple, which includes https://github.com/achow101/signapple/pull/13.
Drop python-macholib and python-altgraph.
ACKs for top commit:
Sjors:
ACK f718a74b12
Tree-SHA512: 199b2108f2f063b6b0fb5354ac79a30b46e848c923ebe7d02f7d7d3f08749817a1f6b4c14d21658fd2f2d68f8be1698e1999edf7e2366b1cae3bf2709a665e30
3c61c60b90 build: Add an old hack to remove bind_at_load from libtool. (Cory Fields)
45257601da build: remove -bind_at_load usage (fanquake)
Pull request description:
This is deprecated on macOS:
```bash
ld: warning: -bind_at_load is deprecated on macOS
```
and likely redundant anyways, given the behaviour of dyld3.
Unfortunately libtool is still injecting a `-bind_at_load`, because it's version check is broken:
```bash
# Don't allow lazy linking, it breaks C++ global constructors
# But is supposedly fixed on 10.4 or later (yay!).
if test CXX = "$tagname"; then
case ${MACOSX_DEPLOYMENT_TARGET-10.0} in
10.[0123])
func_append compile_command " $wl-bind_at_load"
func_append finalize_command " $wl-bind_at_load"
;;
esac
fi
```
so this adds another change to strip them out at the end of configure.
Note that anywhere the ld64 warnings are being emitted, we are already not adding this flag to our hardened ldflags, because of `-Wl,-fatal_warnings`.
ACKs for top commit:
theuni:
utACK 3c61c60b90.
hebasto:
ACK 3c61c60b90, tested on macOS Sonoma 14.1.1 (23B81, Apple M1) and Ubuntu 23.10 (cross-compiling for macOS). Also I've verified the actual diff in the `libtool` script.
Tree-SHA512: 98e6a095dc2d2409f8ec3b9d462e0db3643d7873d7903a12f8acd664829e7e84e797638556fa42ca8ebc1003f13a38fe9bb8a2a50cecfa991155da818574bf08
- Ensure that the snapshot height is higher than the pruned block height when the node is pruned.
- Validate the correctness of the file path and check if the file already exists.
- Make network activity disablement optional for the user.
- Ensure the reconsiderblock command is triggered on exit, even in the case of user interruption (Ctrl-C).
Co-authored-by: Chris Heyes <22148308+hazeycode@users.noreply.github.com>
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
This is deprecated on macOS:
```bash
ld: warning: -bind_at_load is deprecated on macOS
```
and likely redundant anyways, given the behaviour of dyld3.
Unfortunately libtool is still injecting a `-bind_at_load`:
```bash
# Don't allow lazy linking, it breaks C++ global constructors
# But is supposedly fixed on 10.4 or later (yay!).
if test CXX = "$tagname"; then
case ${MACOSX_DEPLOYMENT_TARGET-10.0} in
10.[0123])
func_append compile_command " $wl-bind_at_load"
func_append finalize_command " $wl-bind_at_load"
;;
esac
fi
```
so this doesn't remove all the warnings, but removes us as a potential
source of them.
Note that anywhere the ld64 warnings are being emitted, we are already
not adding this flag to our hardened ldflags, because of `-Wl,-fatal_warnings`.
79539fbfbf guix: update signapple (fanquake)
Pull request description:
Fixes#28449, and removes the need to boostrap Rust, by avoiding the `python-requests` dependency.
Comparing a `--no-substitutes` build of this PR, to master, signapple requires ~1350 _less_ packages to boostrap:
Master derivation - https://gist.github.com/fanquake/dbf69a62c9a78b7ae8c183a160e6d58d
PR derivation - https://gist.github.com/fanquake/0aa2d8eddaba861ba489ed3d936f727d
ACKs for top commit:
achow101:
ACK 79539fbfbf
Tree-SHA512: 341ddcae27e53c31d114465cb5173573dcc9e1c0874ee160715630f686da6f69255f6080ec0181ffeffc26efbdb545599d667784b1cd17dfa7e3da0998ec9bd6
The zip for codesigned MacOS distribution needs to have all files have
the same timestamp. These files also need to be included in the zip as
zip is not automatically recursive. We use the same pattern for zip as
is done for the other zip files produced by guix.
Since Python 3.9, type hinting has become a little less awkward, as for
collection types one doesn't need to import the corresponding
capitalized types (`Dict`, `List`, `Set`, `Tuple`, ...) anymore, but can
use the built-in types directly. [1] [2]
This commit applies the replacement for all Python scripts (i.e. in the
contrib and test folders) for the basic types:
- typing.Dict -> dict
- typing.List -> list
- typing.Set -> set
- typing.Tuple -> tuple
[1] https://docs.python.org/3.9/whatsnew/3.9.html#type-hinting-generics-in-standard-collections
[2] https://peps.python.org/pep-0585/#implementation for a list of type
348e79f7c6 lint: Include test_utxo_snapshots in lint_shell (Fabian Jahr)
Pull request description:
jamesob excluded `test_utxo_snapshots.sh` from the shell linter with this explanation: "Add the script to the shellcheck exception list since the quoted variables rule needs to be violated in order to get bitcoind to pick up on $EARLY_IBD_FLAGS." However, macrofake pointed out that single lines can be excluded from linting.
This fixes one fixable rule violation, excludes the rest of the offending lines from the linter and then removes the exclusion of the `test_utxo_snapshots.sh` file. Also adds documentation.
ACKs for top commit:
Empact:
ACK 348e79f7c6
maflcko:
lgtm ACK 348e79f7c6
pablomartin4btc:
tACK 348e79f7c6
Tree-SHA512: a904cc1cc3c94488dfbd39ea69a3ef17259f991708a797009001669448fef81eed086ecbce1ec433988d88baef293849698e2e0eb86a969b949cc7ef93af7b4b
092daa2f95 contrib: add test for macOS linker version to symbol-check (fanquake)
cefbf0bc20 depends: update LD64_VERSION to 711 (fanquake)
Pull request description:
I forgot to do this in 7d5815293e.
Add a test so it's impossible to forget.
ACKs for top commit:
TheCharlatan:
utACK 092daa2f95
achow101:
ACK 092daa2f95
jarolrod:
ACK 092daa2f95
hebasto:
ACK 092daa2f95.
laanwj:
ACK 092daa2f95
Tree-SHA512: 37f0bdfd6607a7760eabe5efe279532ba0c59c0915161e08d5e3b9a0b7705839d62537d6e17406062f6a0a1db5407575da7cd671e9cb916e422e77f5649c6e2b
36a3004a41 devtools: test_utxo_snapshots.sh sleep cleanup and documentation (Fabian Jahr)
Pull request description:
There were bare sleep statements in the script where it was unclear why they were needed and I think they could also be replaced by existing helpers.
ACKs for top commit:
Sjors:
utACK 36a3004a41
pablomartin4btc:
utACK 36a3004a41
Tree-SHA512: b6e2fc69cd7babcfa7f413f11304f4d648d6d64b3b526862664ccedb0016ad182b9e886aa4e8c33315e4c18824512e11a4fd6365f9c3c95093967d3ef7687e62
