36a3004a41 devtools: test_utxo_snapshots.sh sleep cleanup and documentation (Fabian Jahr)
Pull request description:
There were bare sleep statements in the script where it was unclear why they were needed and I think they could also be replaced by existing helpers.
ACKs for top commit:
Sjors:
utACK 36a3004a41
pablomartin4btc:
utACK 36a3004a41
Tree-SHA512: b6e2fc69cd7babcfa7f413f11304f4d648d6d64b3b526862664ccedb0016ad182b9e886aa4e8c33315e4c18824512e11a4fd6365f9c3c95093967d3ef7687e62
3d420d8f28 Add instructions for headerssync-params.py to release-process.md (Pieter Wuille)
53d7d35b58 Update parameters in headerssync.cpp (Pieter Wuille)
7899402cff Add headerssync-params.py script to the repository (Pieter Wuille)
Pull request description:
Builds upon #25946, as it incorporates changes based on the selected values there.
This adds the headerssync tuning parameters optimization script from https://gist.github.com/sipa/016ae445c132cdf65a2791534dfb7ae1 to the repository, updates the parameters based on its output, and adds release process instructions for doing this update in the future.
A few considerations:
* It would be a bit cleaner to have these parameters be part of `CChainParams`, but due to the nature of the approach, it really only applies to chains with unforgeable proof-of-work, which we really can only reasonably expect from mainnet, so I think it's fine to keep them local to `headerssync.cpp`. Keeping them as compile-time evaluatable constants also has a (likely negligible) performance impact (avoiding runtime modulo operations).
* If we want to make sure the chainparams and headerssync params don't go out of date, it could be possible to run the script in CI, and and possibly even have the parameters be generated automatically at build time. I think that's overkill for how unfrequently these need to change, and running the script has non-trivial cost (~minutes in the normal python interpreter).
* A viable alternative is just leaving this out-of-repo entirely, and just do ad-hoc updating from time to time. Having it in the repo and release notes does make sure it's not forgotten, though adds a cost to contributors/maintainers who follow the process.
ACKs for top commit:
ajtowns:
reACK 3d420d8f28
Tree-SHA512: 03188301c20423c72c1cbd008ccce89b93e2898edcbeecc561b2928a0d64e9a829ab0744dc3b017c23de8b02f3c107ae31e694302d3931f4dc3540e184de1963
360b917674 contrib/bash-completions: use package naming conventions (Erik Arvstedt)
Pull request description:
#### Copy of commit msg
This naming scheme supports auto-detection and on-demand loading of completions.
See
ba109693ee/README.md (faq),
section "Where should I put it to be sure that interactive bash shells will find it and source it", keyword `foo.bash`.
Previously, distro package maintainers had to rename these files manually.
ACKs for top commit:
willcl-ark:
ACK 360b917674
Tree-SHA512: 6dd1f62309e877402fac2f7aec785d3053a1fd3fdeae38abc961c3f61269eb664a2489b6fa8294bd706d8ac387c7af3ac66cd7a852810f69371fa8ae991afacb
Add the script to the shellcheck exception list since the
quoted variables rule needs to be violated in order to get
bitcoind to pick up on $CHAIN_HACK_FLAGS.
This naming scheme supports auto-detection and on-demand loading of completions.
See
ba109693ee/README.md (faq),
section "Where should I put it to be sure that interactive bash shells will find it and source it".
Previously, distro package maintainers had to rename these files manually.
Using the new time-machine results in warnings about consistently using
keyword arguments:
```bash
guix environment: warning: 'cross-kernel-headers' must be used with keyword arguments
guix environment: warning: 'cross-libc' must be used with keyword arguments
```
This is required for bumping the time-machine, for compatibility with
OpenSSL:
oscrypto: openssl backend, 1.2.1, /tmp/guix-build-python-oscrypto-1.2.1.drv-0/source/oscrypto
Traceback (most recent call last):
File "/tmp/guix-build-python-oscrypto-1.2.1.drv-0/source/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
libcrypto.EVP_PKEY_size.argtypes = [
File "/gnu/store/9dkl9fnidcdpw19ncw5pk0p7dljx7ijb-python-3.10.7/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
func = self.__getitem__(name)
File "/gnu/store/9dkl9fnidcdpw19ncw5pk0p7dljx7ijb-python-3.10.7/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /gnu/store/2hr7w64zhr6jjznidyc2xi40d5ynhj9c-openssl-3.0.8/lib/libcrypto.so.3: undefined symbol: EVP_PKEY_size. Did you mean: 'EVP_PKEY_free'?
Refactor our glibc 2.27 to be a single 'package', and avoid the use of
`package-with-extra-configure-variable`. This also lets us drop the
`enable_werror` workaround, and just use --disable-werror directly.
Employ the same workaround as the Guix glibc, to avoid a "permission
denied" failure during build:
```bash
make subdir=sunrpc -C sunrpc ..=../ subdir_install
make[2]: Entering directory '/tmp/guix-build-glibc-cross-x86_64-linux-gnu-2.27.drv-0/source/sunrpc'
.././scripts/mkinstalldirs /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/rpc
mkdir -p -- /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/rpc
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 rpc/netdb.h /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/rpc/netdb.h
.././scripts/mkinstalldirs /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/nfs
mkdir -p -- /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/nfs
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 ../sysdeps/unix/sysv/linux/nfs/nfs.h /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/nfs/nfs.h
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 /tmp/guix-build-glibc-cross-x86_64-linux-gnu-2.27.drv-0/build/gnu/lib-names-64.h /gnu/store/ga8jciqrd5lh52m572x3mk4q1smf5agq-glibc-cross-x86_64-linux-gnu-2.27/include/gnu/lib-names-64.h
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install -c -m 644 etc.rpc /etc/rpc
/gnu/store/kvpvk5wh70wdbjnr83hh85rg22ysxm9h-coreutils-8.32/bin/install: cannot create regular file '/etc/rpc': Permission denied
make[2]: *** [Makefile:197: /etc/rpc] Error 1
```
Split out of #27897. This is some refactoring to the Windows Guix build
that facilitates bumping our Guix time-machine. Namely, avoiding
`package-with-extra-configure-variable`, which is non-functional in the
newer time-machine, see https://issues.guix.gnu.org/64436.
At the same time, consolidate our Windows GCC build into mingw-w64-base-gcc.
Rename `gcc-10-remap-guix-store.patch` to avoid changing it whenever GCC changes.
We move the old `building-on` inside `explicit-cross-configure`, so that
non-windows builds continue to work. Note that `explicit-cross-configure`
will be going away entirely (see #27897).
fa60fa3b0c bitcoin-tidy: Apply bitcoin-unterminated-logprintf to spkm as well (MarcoFalke)
faa11434fe refactor: Enable all clang-tidy plugin bitcoin tests (MarcoFalke)
fa6dc57760 refactor: Enforce C-str fmt strings in WalletLogPrintf() (MarcoFalke)
fa244f3321 doc: Fix bitcoin-unterminated-logprintf tidy comments (MarcoFalke)
Pull request description:
All fmt functions only accept a raw C-string as argument.
There should never be a need to pass a format string that is not a compile-time string literal, so disallow it in `WalletLogPrintf()` to avoid accidentally introducing it.
Apart from consistency, this also fixes the clang-tidy plugin bug https://github.com/bitcoin/bitcoin/pull/26296#discussion_r1286821141.
ACKs for top commit:
theuni:
ACK fa60fa3b0c
Tree-SHA512: fa6f4984c50f9b34e850bdfee7236706af586e512d866cc869cf0cdfaf9aa707029c210ca72d91f85e75fcbd8efe0d77084701de8c3d2004abfd7e46b6fa9072
bb3263d3e3 bitcoin-tidy: fix macOS build (Cory Fields)
Pull request description:
[LLVM uses these options](https://github.com/llvm/llvm-project/blob/main/llvm/cmake/modules/HandleLLVMOptions.cmake#L178) for building as well, so there's precedent.
Also fix the shared library extension which was incorrectly being set to dylib.
Thanks to jonatack for reporting and debugging.
ACKs for top commit:
jonatack:
ACK bb3263d3e3 tested with arm64 macos 13.5, llvm 16.0.6 and cmake 3.27.2
Tree-SHA512: de7bfd497f38f1565a14d217d0b057cbfa788bdda702b5942b7f0b55947ae5e1c05af13e7d6a073ed036bc4db57035868f180034508b6e084ab9b901a5baaf2f
An LLVM installation will have `llvm-config` available to query for
info. Ask it for the `--cmakedir`, and use that in our bitcoin-tidy
example, rather than listing multiple different (potential) paths per
distro/OS etc.
7f96638723 contrib: add macOS fixup_chains check to security-check (fanquake)
3dca683cb7 build: support -no_fixup_chains in ld64 (fanquake)
Pull request description:
Followup to #27676, adding the check for chained fixups.
Somewhat annoyingly, we have to patch support for `-no_fixup_chains` into ld64. As it doesn't seem to have been added [until a later version](59a99ab603/src/ld/Options.cpp (L4172)).
Guix Build:
```bash
0e17d462808f86aa7157e27a957da88fd1adeb491ad6c01138aca93e5ad1d018 guix-build-7f96638723a0/output/arm64-apple-darwin/SHA256SUMS.part
ceb208e6374f5d7367b73128e90ca6eaeea15d50c69e49c8cf75b47212525ad7 guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.dmg
e31663554cfde8a37a9f3438c9c895dde94b90ff87e28f12f78be71ef6421d93 guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin-unsigned.tar.gz
68a7bbc42418641eab391a85725b5c2f3c46d38a7acc07e7a8cef98909be07ec guix-build-7f96638723a0/output/arm64-apple-darwin/bitcoin-7f96638723a0-arm64-apple-darwin.tar.gz
38d966ad93e7384f4f1ce16faded003a675ecce7be1987e6c4eee8e4b82c0432 guix-build-7f96638723a0/output/dist-archive/bitcoin-7f96638723a0.tar.gz
9d314f595d897a715a321a9fba0d552220fbd4bf69aff84eb8c0001cdb48234f guix-build-7f96638723a0/output/x86_64-apple-darwin/SHA256SUMS.part
c218ebfd0e96348c4912e6d522492b621bb043ef45b75105ff1fde979d1004d0 guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.dmg
1c5ff7fa82f5c76d7d8b9582ad5202f4a82a917102ecafdc3c1fb7b783f6bc3e guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin-unsigned.tar.gz
15fb01e5afcc842db6a3e793b42c70c05ce07bec79e0d2d605e241901ff9f639 guix-build-7f96638723a0/output/x86_64-apple-darwin/bitcoin-7f96638723a0-x86_64-apple-darwin.tar.gz
```
ACKs for top commit:
theuni:
utACK 7f96638723.
hebasto:
ACK 7f96638723, I have reviewed the code and the patch, and they look OK.
TheCharlatan:
ACK 7f96638723
Tree-SHA512: 7f94710460f54b2afe3c9f5d57107b71436c59b799b15f78e5e3011c3c4f6b23a3acc1008eccea9c22226a200774c82900bad6c6236ab6c5c48a17dec3f2d5a2
6c97757a48 script: appease spelling linter (Jon Atack)
1316119ce7 script: update ignored-words.txt (Jon Atack)
146c861da2 script: update linter dependencies (Jon Atack)
92408224a4 test: fix PEP484 no implicit optional argument types errors (Jon Atack)
f86a301433 script, test: add missing python type annotations (Jon Atack)
Pull request description:
With these updates, `./test/lint/lint-python.py` and `./test/lint/lint-spelling.py` should be green again for developers using relatively recent Python dependencies, in particular mypy 0.991 (released 11/2022) and later. Please see the commit messages for details.
ACKs for top commit:
fanquake:
ACK 6c97757a48
Tree-SHA512: 8a46a4d36d5978affdcecf4f2ace20ca1b52d483e098304911a2169afe60ccb9b042fa90c04b762d94f3ce53d2cafe6f24476ae839867a770c7f31e7e7242d99
Fix warnings for these files when ./test/lint/lint-python.py is run using
mypy 0.991 (released 11/2022) and later:
"By default the bodies of untyped functions are not checked, consider using
--check-untyped-defs [annotation-unchecked]"
For details, see:
https://mypy-lang.blogspot.com/2022/11/mypy-0990-released.html
This change improves the maintainability of the manifest:
(1) It allows to remove the module when the specified symbols are no
longer used.
(2) It prevents accidental use of other symbols, such as `bash`
instead of `bash-minimal`.
529c92e837 guix: Update `python-lief` package to 0.13.2 (Hennadii Stepanov)
Pull request description:
The Guix's `python-lief` package is going to move to using external deps, rather than the bundled ones (https://lists.gnu.org/archive/html/guix-patches/2023-05/msg01302.html). We want to continue using our own package indefinitely, to keep the build simpler, and allow for easier updating.
Changes in `contrib/devtools/security-check.py` are caused by 6357c6370b.
Also see: https://github.com/bitcoin/bitcoin/pull/27507.
ACKs for top commit:
fanquake:
ACK 529c92e837
Tree-SHA512: ad81111b090a39b380fe25bb27b54a339e78a158f462c7adda25d5ee55f0d654107b1486b29b9687ad0808e27b01e04f53a0e8ffc6600b79103d6bd0dfec64ef
Unfortunately clang 10 does not understand "-mmacosx-version-min=11.0",
as it expects to see only 10.x.
Bump minimally to 11.1 to fix that problem. This will likely be our last
binary toolchain bump, as it will soon be replaced with usage of upstream
vanilla llvm.
015cc5e588 lint: stop ignoring LIEF imports (fanquake)
Pull request description:
Type stubs are now available as of 0.13.0.
See https://github.com/lief-project/LIEF/issues/650.
ACKs for top commit:
TheCharlatan:
ACK 015cc5e588
Tree-SHA512: ebb754f293c2a61a0ef64c3552f7c700ceb3054b50fd3f1573e4a9e87773ddeba47bd9875f6ab055043012dbc20aeb71e4d76cd3da535c76651dfb1fbfc66e89
689a65d878 contrib/init: Better systemd integration (Carl Dong)
Pull request description:
```
1. Make logs available to journalctl (systemd's logging system) by not
specifying -daemonwait, which rightfully has its own set of stdout
and stderr descriptors (a user invoking with -daemonwait on the
command line should not see any logs). It makes more sense not to
daemonize in the systemd context anyway.
2. Make systemd aware of when bitcoind is started and in steady state by
specifying -startupnotify='systemd-notify --ready' and Type=notify.
NotifyAccess=all is necessary so that the spawned thread for
startupnotify is allowed to inform systemd of bitcoind's readiness.
Note that NotifyAccess=exec won't work because it only allows
sd_notify readiness signalling from Exec*= declarations in the
.service file.
Note that we currently don't allow multiple startupnotify commands, but
users can override it in systemd via:
# systemctl edit bitcoind
By specifying something like:
[Service]
ExecStart=/usr/bin/bitcoind -pid=/run/bitcoind/bitcoind.pid \
-conf=/etc/bitcoin/bitcoin.conf \
-datadir=/var/lib/bitcoind \
-startupnotify='systemd-notify --ready; mycommandhere'
```
ACKs for top commit:
real-or-random:
ACK 689a65d878 tested this service file with 25.0
Tree-SHA512: 9a52ad5cf25886c0d8dabc986d8920602a056db25875b5edd910b387043b78bb78c76d6df82e6e322e3be3bfd5c35c80721cbc8308cec946060bd7586820e9c6
1. Make logs available to journalctl (systemd's logging system) by not
specifying -daemonwait, which rightfully has its own set of stdout
and stderr descriptors (a user invoking with -daemonwait on the
command line should not see any logs). It makes more sense not to
daemonize in the systemd context anyway.
2. Make systemd aware of when bitcoind is started and in steady state by
specifying -startupnotify='systemd-notify --ready' and Type=notify.
NotifyAccess=all is necessary so that the spawned thread for
startupnotify is allowed to inform systemd of bitcoind's readiness.
Note that NotifyAccess=exec won't work because it only allows
sd_notify readiness signalling from Exec*= declarations in the
.service file.
3. Also make systemd aware of when bitcoind is stopping by specifying
-shutdownnotify='systemd-notify --stopping'
Note that we currently don't allow multiple *notify commands, but users
can override it in systemd via:
# systemctl edit bitcoind
By specifying something like:
[Service]
ExecStart=/usr/bin/bitcoind -pid=/run/bitcoind/bitcoind.pid \
-conf=/etc/bitcoin/bitcoin.conf \
-datadir=/var/lib/bitcoind \
-startupnotify='systemd-notify --ready; mystartupcommandhere' \
-shutdownnotify='systemd-notify --stopping; myshutdowncommandhere'
These should only be relevant for a glibc that is built as part of a
Guix system, and should not be required for a glibc that is just being
built to compile our binaries against. A x86_64 linux bitcoind produced
with Guix using master vs this change has no difference. i.e:
```diff
@@ -20311,15 +20311,15 @@
This is experimental software.
The source code is available from %s.
Please contribute if you find %s useful. Visit %s for further information about the software.
The %s developers
The Bitcoin Core developers
<https://bitcoincore.org/>
Copyright (C) %i-%i
-v25.99.0-gda0bf1d07639b0490791bbd6aec71bbea8aa2aThe %s developer<https://github.com/bitcoin/bitcDistributed under the MIT software license, see the accompanyingThis is experimeThe source code is available froPlease contribute if you find %s useful. Visit %s for further information about Copyright (C) %ibool BCLog::Logger::StartLogging()
+v25.99.0-gd7700d3a26478d9b1648463c188648c7047b1cThe %s developer<https://github.com/bitcoin/bitcDistributed under the MIT software license, see the accompanyingThis is experimeThe source code is available froPlease contribute if you find %s useful. Visit %s for further information about Copyright (C) %ibool BCLog::Logger::StartLogging()
std::string BCLog::Logger::LogLevelToStr(BCLog::Level) const
std::string LogCategoryToStr(BCLog::LogFlags)
void BCLog::Logger::LogPrintStr(const string&, const string&, const string&, int, BCLog::LogFlags, BCLog::Level)
void BCLog::Logger::ShrinkDebugFile()
Failed to shrink debug log file: fseek(...) failed
logging.cpp
m_buffering
```
```diff
@@ -1505889,15 +1505889,15 @@
call aa3380 <malloc@plt+0xa4edb0>
mov (%rsp),%rdx
movdqa 0x465540(%rip),%xmm0
mov %rax,0x7a0559(%rip)
lea 0x7a0552(%rip),%rsi
lea 0x3957bb(%rip),%rdi
mov %rdx,0x7a0554(%rip)
- mov $0x3038,%edx
+ mov $0x3036,%edx
movups %xmm0,(%rax)
movdqa 0x465524(%rip),%xmm0
mov %dx,0x30(%rax)
mov 0x7a0529(%rip),%rdx
movups %xmm0,0x10(%rax)
movdqa 0x46551d(%rip),%xmm0
movups %xmm0,0x20(%rax)
```
```diff
@@ -37238,17 +37238,17 @@
0x00b73730 65202573 20646576 656c6f70 65727300 e %s developers.
0x00b73740 54686520 42697463 6f696e20 436f7265 The Bitcoin Core
0x00b73750 20646576 656c6f70 65727300 434f5059 developers.COPY
0x00b73760 494e4700 3c687474 70733a2f 2f626974 ING.<https://bit
0x00b73770 636f696e 636f7265 2e6f7267 2f3e0043 coincore.org/>.C
0x00b73780 6f707972 69676874 20284329 2025692d opyright (C) %i-
0x00b73790 25690053 61746f73 68690000 00000000 %i.Satoshi......
- 0x00b737a0 7632352e 39392e30 2d676461 30626631 v25.99.0-gda0bf1
- 0x00b737b0 64303736 33396230 34393037 39316262 d07639b0490791bb
- 0x00b737c0 64366165 63373162 62656138 61613261 d6aec71bbea8aa2a
+ 0x00b737a0 7632352e 39392e30 2d676437 37303064 v25.99.0-gd7700d
+ 0x00b737b0 33613236 34373864 39623136 34383436 3a26478d9b164846
+ 0x00b737c0 33633138 38363438 63373034 37623163 3c188648c7047b1c
0x00b737d0 54686520 25732064 6576656c 6f706572 The %s developer
0x00b737e0 3c687474 70733a2f 2f676974 6875622e <https://github.
0x00b737f0 636f6d2f 62697463 6f696e2f 62697463 com/bitcoin/bitc
0x00b73800 44697374 72696275 74656420 756e6465 Distributed unde
0x00b73810 72207468 65204d49 5420736f 66747761 r the MIT softwa
0x00b73820 7265206c 6963656e 73652c20 73656520 re license, see
0x00b73830 74686520 6163636f 6d70616e 79696e67 the accompanying
```
```diff
@@ -1,5 +1,5 @@
Hex dump of section '.gnu_debuglink':
0x00000000 62697463 6f696e64 2e646267 00000000 bitcoind.dbg....
- 0x00000010 6b6e8eda kn..
+ 0x00000010 345cb865 4\.e
```
a2bef805c1 kernel: update m_assumed_* chain params for 25.x (fanquake)
4128e01dba kernel: update chainTxData for 25.x (fanquake)
00b2b114b4 kernel: update nMinimumChainWork & defaultAssumeValid for 25.x (fanquake)
07fcc0a82c doc: update references to kernel/chainparams.cpp (fanquake)
Pull request description:
Update chainparams pre `25.x` branch off.
Co-Author in the commits as a PR (#27223) had previously been opened too-early to do the same.
Note: Remember that some variance is expected in the `m_assumed_*` sizes.
ACKs for top commit:
achow101:
ACK a2bef805c1
josibake:
ACK a2bef805c1
gruve-p:
ACK a2bef805c1
dergoegge:
ACK a2bef805c1 on the new mainnet params
Tree-SHA512: 0b19c2ef15c6b15863d6a560a1053ee223057c7bfb617ffd3400b1734cee8f75bc6fd7f04d8f8e3f5af6220659a1987951a1b36945d6fe17d06972004fd62610
3cc989da5c Fix checking bad dns seeds without casting (Yusuf Sahin HAMZA)
Pull request description:
- Since seed lines comes with `str` type, comparing `good` column directly with **0** (`int` type) in the if statement was not working at all. This is fixed by casting `int` type to the values in the `good` column of seeds text file.
- Lines that starts with comment in the seeds text file are now ignored.
- If statement for checking bad seeds are moved to the top of the `parseline` function as if a seed is bad; there is no point of going forward from there.
Since this bug-fix eliminates bad seeds over **550k** in the first place, in my case; particular job for parsing all seeds speed is up by **600%** and whole script's speed is up by **%30**.
Note that **stats** in the terminal are not going to include bad seeds after this fix, which would be the same if this bug were never there before.
ACKs for top commit:
achow101:
ACK 3cc989da5c
jonatack:
ACK 3cc989da5c
Tree-SHA512: 13c82681de4d72de07293f0b7f09721ad8514a2ad99b0584d1c94fa5f2818821df2000944f9514d6a222a5dccc82856d16c8c05aa36d905cfa7d4610c629fd38
754fb6bb81 verifybinaries: fix argument type error pointed out by mypy (Cory Fields)
8a65e5145c verifybinaries: catch the correct exception (Cory Fields)
4b23b488d2 verifybinaries: fix OS download filter (Cory Fields)
8cdadd1729 verifybinaries: use recommended keyserver by default (Cory Fields)
4e0396835d verifybinaries: remove unreachable code (Cory Fields)
5668c6473a verifybinaries: Don't delete shasums file (Cory Fields)
46c73b57c6 verifybinaries: README cleanups (Cory Fields)
6d11830265 verifybinaries: remove awkward bitcoin-core prefix handling (Cory Fields)
c44323a717 verifybinaries: move all current examples to the pub subcommand (Cory Fields)
7a6e7ffd06 contrib: Use machine parseable GPG output in verifybinaries (Andrew Chow)
6b2cebfa2f contrib: Add verifybinaries command for specifying files to verify (Andrew Chow)
e4d5778228 contrib: Specify to GPG the SHA256SUMS file that is detached signed (Andrew Chow)
17575c0efa contrib: Refactor verifbinaries to support subcommands (Andrew Chow)
37c9fb7a59 contrib: verifybinaries: allow multisig verification (James O'Beirne)
Pull request description:
Following up on #23020 from jamesob with achow101's additional features on top.
Both mentioned that they will be away for the next few weeks, so this is intended to keep review going.
All credit to the jamesob and achow101. See #23020 for the original description and [here](https://github.com/bitcoin/bitcoin/pull/23020#issuecomment-1480603300) for the added features.
I squashed the last commit from https://github.com/achow101/bitcoin/tree/pr23020-direct-bins-gpg-parse into the first commit here.
Fetching and local verification seem to work as intended for me.
ACKs for top commit:
josibake:
ACK 754fb6bb81
Tree-SHA512: b310c57518daa690a00126308a3e7e94b978ded56d13da15d5189e9e90b71c93888d854f64179150586b0a915db8dadd43c92b716613913c198128db8867257b
In addition to verifying the published releases with the `pub` command,
the verifybinaries script is updated to take a `bin` command where the
user specifies the local files, sums, and sigs to verify.
This commit adds the functionality necessary to transition from
doing binary verification on the basis of a single signature to
requiring a minimum threshold of trusted signatures.
A signature can appear as "good" from GPG output, but it may not come
from an identity the user trusts. We call these "good, untrusted"
signatures.
We report bad signatures but do not necessarily fail in their presence,
since a bad signature might coexist with enough good, trusted signatures
to fulfill our criteria.
If "--import-keys" is enabled, we will prompt the user to
optionally try to retrieve unknown keys. Marking them as trusted locally
is a WIP, but keys which are retrieved successfully and appear on the
builder-keys list will immediately count as being useful towards
fulfilling the threshold.
Logging is improved and an option to output JSON that summarizes the
whole sum signature and binary verification processes has been added.
Co-authored-by: Russ Yanofsky <russ@yanofsky.org>
Co-authored-by: willcl-ark <will8clark@gmail.com>
4becee396f guix: combine and document enable_werror (fanquake)
Pull request description:
Combine into `hardened-glibc`.
Document why we don't use `--disable-werror` directly.
https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html
> By default, the GNU C Library is built with -Werror. If you wish
> to build without this option (for example, if building with a
> newer version of GCC than this version of the GNU C Library was
> tested with, so new warnings cause the build with -Werror to fail),
> you can configure with --disable-werror.
ACKs for top commit:
hebasto:
ACK 4becee396f, the diff is correct.
TheCharlatan:
ACK 4becee396f
Tree-SHA512: 8724415f51b4d72d40c4e797faf52c93a81147fb629332b9388ffd7f113f2b16db3b7496bf3063dd978ac629fd5bde3ec7df4f1ff1ed714cb56f316a9334d119
Combine into hardened-glibc.
Document why we don't use --disable-werror directly.
https://www.gnu.org/software/libc/manual/html_node/Configuring-and-compiling.html
> By default, the GNU C Library is built with -Werror. If you wish
> to build without this option (for example, if building with a
> newer version of GCC than this version of the GNU C Library was
> tested with, so new warnings cause the build with -Werror to fail),
> you can configure with --disable-werror.
This also fixes atleast one --no-substitues build failure I've seen,
where cmake dependencies wouldn't build:
```bash
The following derivations will be built:
/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv
/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv
/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv
building /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv...
/ 'check' phasenote: keeping build directory `/tmp/guix-build-python-sphinx-4.2.0.drv-5'
builder for `/gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv' failed with exit code 1
build of /gnu/store/3wg6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv failed
View build log at '/var/log/guix/drvs/3w/g6ya847id503m5izhzhn1qqs464lfk-python-sphinx-4.2.0.drv.gz'.
cannot build derivation `/gnu/store/f9zwh1ldy63ga0i5w6cbbqlj6sfq226j-cmake-3.21.4.drv': 1 dependencies couldn't be built
cannot build derivation `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv': 1 dependencies couldn't be built
guix environment: error: build of `/gnu/store/7qqvqq2g7l5ylrjv0gn6zha565a12kar-python-lief-0.12.1.drv' failed
```
285edfadca guix: use osslsigncode 2.5 (fanquake)
Pull request description:
Switches to using a newer version of [osslsigncode](https://github.com/mtrojnar/osslsigncode) in our Guix environment.
achow101 can you test this with some sort of WIndows code-signing dry-run (no-rush).
ACKs for top commit:
achow101:
ACK 285edfadca
Tree-SHA512: 2ab8f65e506bd97e74e76f24e791ae20694e567a751cc57d3a27f31f0733e3530d058ef19825a35dc21d1342e3fffc52d8d643258198c669cc68b6db41bda629
4b7aec2951 Add mempool tracepoints (virtu)
Pull request description:
This PR adds multiple mempool tracepoints.
| tracepoint | description |
| ------------- | ------------- |
| `mempool:added` | Is called when a transaction enters the mempool |
| `mempool:removed` | ... when a transaction is removed from the mempool |
| `mempool:replaced` | ... when a transaction is replaced in the mempool |
| `mempool:rejected` | ... when a transaction is rejected from entering the mempool |
The tracepoints are further documented in `docs/tracing.md`. Usage is demonstrated in the example script `contrib/tracing/mempool_monitor.py`. Interface tests are provided in `test/functional/interface_usdt_mempool.py`.
The rationale for passing the removal reason as a string instead of numerically is that the benefits of not having to maintain a redundant enum-string mapping seem to outweigh the small cost of string generation. The reject reason is passed as string as well, although in this instance the string does not have to be generated but is readily available.
ACKs for top commit:
0xB10C:
ACK 4b7aec2951
achow101:
ACK 4b7aec2951
Tree-SHA512: 6deb3ba2d1a061292fb9b0f885f7a5c4d11b109b838102d8a8f4828cd68f5cd03fa3fc64adc6fdf54a08a1eaccce261b0aa90c2b8c33cd5fd3828c8f74978958
Tracepoints for added, removed, replaced, and rejected transactions.
The removal reason is passed as string instead of a numeric value, since
the benefits of not having to maintain a redundant enum-string mapping
seem to outweigh the small cost of string generation. The reject reason
is passed as string as well, although here the string does not have to
be generated but is readily available.
So far, tracepoint PRs typically included two demo scripts: a naive
bpftrace script to show raw tracepoint data and a bcc script for a more
refined view. However, as some of the ongoing changes to bpftrace
introduce a certain degree of unreliability (running some of the
existing bpftrace scripts was not possible with standard kernels and
bpftrace packages on latest stable Ubuntu, Debian, and NixOS), this PR
includes only a single bcc script that fuses the functionality of former
bpftrace and bcc scripts.
