de3ae78eff ci: increase CPU count of sanitizer job to increase memory limit (fanquake)
Pull request description:
According to the [docs](https://cirrus-ci.org/guide/linux/#linux-containers):
> For each CPU you can't get more than 4G of memory.
thus if we want this job to have 24GB of memory, we need to increase the CPU count to 6.
It's currently [failing with](https://github.com/bitcoin/bitcoin/runs/2273962280):
> Requested memory is too high! You can request at most 4G per CPU
Top commit has no ACKs.
Tree-SHA512: 0a4da5649d061425190a373859274c78ca5587cd2d6e27905ec548f124ed114a0133215cb2eff22ffc182f50c3a53df58e7c9832b44db6e37d7ea59ec96a4775
fa5eabe721 refactor: Remove negative lock annotations from globals (MarcoFalke)
Pull request description:
They only make sense for mutexes that are private members. Until cs_main is a private member the negative annotations should be replaced by excluded annotations, which are optional.
ACKs for top commit:
sipa:
utACK fa5eabe721
ajtowns:
ACK fa5eabe721
hebasto:
ACK fa5eabe721
vasild:
ACK fa5eabe721
Tree-SHA512: 06f8a200304f81533010efcc42d9f59b8c4d0ae355920c0a28efb6fa161a3e3e68f2dfffb0c009afd9c2501e6a293c6e5a419a64d718f1f4e79668ab2ab1fcdc
5c09bcadc4 Fix a typo in guix-build output (Pieter Wuille)
Pull request description:
This was overlooked in #21375.
ACKs for top commit:
fanquake:
ACK 5c09bcadc4
Tree-SHA512: 81d8ad4061abb17d5f16ae72ab0c88df76d5a2f100cb9f471ca700d2e87583103036367ea1958a1066f9613f63908d1b2b35734a7eb77ec9850d4c8b079732e1
According to the docs,
https://cirrus-ci.org/guide/linux/#linux-containers, "For each CPU you
can't get more than 4G of memory.", thus if we want this job to have
24GB of memory, we need to increase the CPU count to 6.
The register keyword was deprecated in C++11, and removed in C++17. Now
that we require C++17, we shouldn't have to supress warnings for a
non-existant feature.
Many users have reported problems that stem from having an unclean
working tree. To that end, I've written a guix-clean script which should
help reset the working tree while respecting user-specified precious
directories.
Precious directories, such as:
- SOURCES_PATH
- BASE_CACHE
- SDK_PATH
- OUTDIR
Should be preserved when cleaning the working tree, and are thus
recorded in ./contrib/guix/var/precious_dirs.
The ./contrib/guix/guix-clean script is able to parse that file and make
sure to avoid them when cleaning out the working tree.
7476b46f18 guix: Build dmg as a static binary (Carl Dong)
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb4 guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3 guix: Supply --link-profile (Carl Dong)
d55a1056ee guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b2 guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7b guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5 guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab38 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e9785 guix: Use --cores instead of --max-jobs (Carl Dong)
Pull request description:
This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
- Resolves confusion between `--cores=` and `--max-jobs=`
- `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
- `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
- `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
- Add troubleshooting documentation to `README.md`
- Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
- Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
- `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`
A few robustness changes are also included:
- We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
- We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
- We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.
Finally, we restructure the guix building hierarchy such that it looks something like:
```
guix-build-<short-hash-or-version-tag>
├── distsrc-<short-hash-or-version-tag>-${HOST}
│ ├── contrib
│ ├── depends
│ ├── src
│ └── ...
├── distsrc-<short-hash-or-version-tag>-...
└── output
├── dist-archive
│ └── bitcoin-<short-hash-or-version-tag>.tar.gz
├── *-linux-*
│ ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
│ └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
├── x86_64-apple-darwin18
│ ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
│ ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
│ └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
└── x86_64-w64-mingw32
├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
├── bitcoin-<short-hash-or-version-tag>-win64.zip
└── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
```
Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.
ACKs for top commit:
sipa:
ACK 7476b46f18
laanwj:
ACK 7476b46f18
Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.
This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
fa04eb7d78 test: Document why no symbol names can be used for suppressions (MarcoFalke)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: 02e5d9cb3ca299b5e7f1f0f780af514692147f5f0846e21e5375a99e90d3aed3a3489c674ac0eb90ae34d60f576410ba1d926d5ca70cdce4f9c7a111cd516adf
585854ac66 test: Replace blanket UBSan signed integer overflow suppression for txmempool.cpp with specific suppression (practicalswift)
Pull request description:
Replace file level (`txmempool.cpp`) signed integer overflow suppression with function level suppression (`CTxMemPool::PrioritiseTransaction`). The suppression was added yesterday in #21586.
Rationale: To avoid risk hiding other signed integer overflows in `txmempool.cpp`.
Obviously it would be better if this signed integer overflow fixed instead of suppressed - see details #20626. Any taker? :)
To hit the issue via fuzzing:
```
$ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" FUZZ=validation_load_mempool src/test/fuzz/fuzz
INFO: Seed: 1184244493
INFO: Loaded 1 modules (634418 inline 8-bit counters): 634418 [0x55a09fdfbf98, 0x55a09fe96dca),
INFO: Loaded 1 PC tables (634418 PCs): 634418 [0x55a09fe96dd0,0x55a0a08450f0),
INFO: 1264 files found in mempool/
INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1040698 bytes
INFO: seed corpus: files: 1264 min: 1b max: 1040698b total: 15997133b rss: 197Mb
txmempool.cpp:847:15: runtime error: signed integer overflow: -7211388903327006720 + -7211353718954917888 cannot be represented in type 'long'
#0 0x55a09c3ce2d8 in CTxMemPool::PrioritiseTransaction(uint256 const&, long const&) /home/thomas/bitcoin/src/txmempool.cpp:847:15
```
ACKs for top commit:
JeremyRubin:
utACK 585854a
hebasto:
ACK 585854ac66, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 5a343f028c1e1a1aba3b51a0eced605849184891ffafecb3cd2b424c6cfea01afd7c2672274936b0bac646075ec066408a570bf6b34bc9b87399a53ce20d8a23
