Commit graph

716 commits

Author SHA1 Message Date
Hennadii Stepanov
5deb0b024e
build, test, doc: Temporarily remove Android-related stuff
Previously, our Android builds were geared towards generating APKs,
which relied on Qt. However, after migrating to C++20, compiling for
Android became unfeasible due to Qt 5.15's compatibility limitations
with NDK only up to r25, which includes an outdated embedded libc++.

All removed stuff will be reinstated after migrating the build system to
CMake and upgrading Qt to version 6.x."
2024-05-06 11:29:14 +01:00
fanquake
b088062e68
ci: remove -Wdocumentation from -Werror in multiprocess CI
The issues here are coming from Boost Test code.
2024-05-03 15:31:54 +08:00
fanquake
bec6a88fbc
ci: remove -Warray-bounds from -Werror for win64
These warnings are coming from leveldb, and appear to be fixed starting
with GCC 13.
2024-05-03 15:31:54 +08:00
fanquake
7469ac7032
ci: disable -Werror=maybe-uninitialized for Windows builds
This produces false positives, such as:
```bash
torcontrol.cpp: In static member function ‘static void TorControlConnection::readcb(bufferevent*, void*)’:
torcontrol.cpp:94:28: error: ‘result’ may be used uninitialized [-Werror=maybe-uninitialized]
   94 |         self->message.code = ToIntegral<int>(s.substr(0, 3)).value_or(0);
      |         ~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./netaddress.h:18,
                 from ./torcontrol.h:11,
                 from torcontrol.cpp:6:
./util/strencodings.h:184:7: note: ‘result’ was declared here
  184 |     T result;
      |       ^~~~~~
cc1plus: all warnings being treated as errors
make[2]: *** [Makefile:11088: libbitcoin_node_a-torcontrol.o] Error 1
```
2024-05-03 15:31:54 +08:00
MarcoFalke
fa1964c5b8
build: Bump clang minimum supported version to 15 2024-04-27 14:54:00 +02:00
merge-script
d822d4e849
Merge bitcoin/bitcoin#29837: ci: disable _FORTIFY_SOURCE with MSAN
08ff17d142 ci: disable _FORTIFY_SOURCE with MSAN (fanquake)

Pull request description:

  By undefining `_FORTIFY_SOURCE` we can drop`--disable-hardening`.

ACKs for top commit:
  maflcko:
    lgtm ACK 08ff17d142
  hernanmarino:
    utACK 08ff17d142 . Relevant CI test seems to be working OK.

Tree-SHA512: 948fd075aa648a7e34c37376fb913074ebc07d1c3cb0737d5fcbe7eac0b35c4152139773e4515ccb80f2d11b1ced6c6984da1757c2bcf8dd90e8ff6f664dae8e
2024-04-24 22:46:25 +08:00
Hennadii Stepanov
6f5954acac
ci: Drop no longer needed -I flag in "tidy" task 2024-04-21 14:30:38 +01:00
merge-script
22c86140f8
Merge bitcoin/bitcoin#29848: ci: use Clang 16 for Valgrind
ad21f22948 ci: use clang-16 for Valgrind (fanquake)

Pull request description:

  Switch to Ubuntu Noble.
  Valgrind 3.19 -> 3.22
  Clang 14 -> Clang 16

ACKs for top commit:
  maflcko:
    lgtm ACK ad21f22948

Tree-SHA512: ec79ef9faaec97e34529ae36fff7798f859daca6a1e3563bc50e5d56a56ee4525c736976158a6e950c5b9f810c498d54ab128df984f42441e706033906c2ea3e
2024-04-15 14:26:59 +01:00
fanquake
ad21f22948
ci: use clang-16 for Valgrind
Switch to Ubuntu Noble.
2024-04-14 10:39:14 +01:00
MarcoFalke
fadf7e90dc
ci: Bump s390x to ubuntu:24.04
Re-enable feature_init
2024-04-12 16:30:45 +02:00
fanquake
08ff17d142
ci: disable _FORTIFY_SOURCE with MSAN 2024-04-09 15:58:05 +02:00
fanquake
c15170c27d
Revert "ci: Temporarily disable bpfcc-tools"
This reverts commit fac012c726.
2024-04-09 09:17:34 +02:00
fanquake
61641e2466
ci: remove --with-asm usage (secp256k1) 2024-04-06 09:41:24 +01:00
fanquake
c7efee591a
ci: use LLVM 18.1.3 in MSAN jobs 2024-04-06 09:40:23 +01:00
fanquake
3a8dc562f2
Merge bitcoin/bitcoin#29800: ci: Drop duplicated compiler flags
a3485af67d ci: Drop duplicated compiler flags (Hennadii Stepanov)

Pull request description:

  On the master branch @ 0d509bab45, it is easy to check the _"Options used to compile and link"_ section in the `configure` script output and observe duplicated compiler flags.

  This PR cleans such cases up.

ACKs for top commit:
  maflcko:
    re-ACK a3485af67d
  fanquake:
    ACK a3485af67d - no-longer a change in behaviour.

Tree-SHA512: 7e644fcfad7be48af3b18edd2994c0c78a21ac3f9fff497724be80f74c9e859d156de15ca4024c5c50d1080435576ce63402b48aba5c2fd556e2ed7e318e0e34
2024-04-05 17:09:10 +01:00
Hennadii Stepanov
a3485af67d
ci: Drop duplicated compiler flags 2024-04-05 15:38:16 +01:00
fanquake
5de68e45c2
Merge bitcoin/bitcoin#29788: ci: Temporarily disable bpfcc-tools
fac012c726 ci: Temporarily disable bpfcc-tools (MarcoFalke)

Pull request description:

  This works around package install errors, such as https://github.com/bitcoin/bitcoin/runs/23354020361. Should be possible to reproduce locally via `apt update && apt install bpfcc-tools` on noble:

  ```
   python3-bpfcc : Depends: libbpfcc (>= 0.29.1+ds-1ubuntu4) but it is not going to be installed

ACKs for top commit:
  hebasto:
    ACK fac012c726, I have reviewed the code, it looks OK. And CI is green.
  TheCharlatan:
    ACK fac012c726

Tree-SHA512: 369e89bc0fbf4d75455c7c047ae904849267f82e647eeadd7ee6045393bd9812a5e682a58eb8d2ccd8c89a030e6665f054596fdcd0708f591ed4c693398125d1
2024-04-04 11:14:46 +01:00
MarcoFalke
fac012c726
ci: Temporarily disable bpfcc-tools 2024-04-02 10:57:21 +02:00
fanquake
948ecf181e
Merge bitcoin/bitcoin#29648: Remove libbitcoinconsensus
80f8b92f4f remove libbitcoinconsensus (fanquake)

Pull request description:

  This was deprecated in `v27.0`, for removal in `v28.0`. See discussion in PR #29189.

ACKs for top commit:
  theuni:
    Concept ACK and light review ACK 80f8b92f4f. My only hesitation here is that (afaics?) there's now nothing keeping undesired features like threading or globals from working their way into the interpreter in future commits.
  m3dwards:
    Concept ACK 80f8b92f4f
  TheCharlatan:
    ACK 80f8b92f4f
  hebasto:
    ACK 80f8b92f4f, I have reviewed the code and it looks OK.

Tree-SHA512: 17a62118aeb088f2695c892bb32794dfea3061e3cb7d9e8e9f1c06c3ff6f63a7587fa532e37edbb91fbc5a19b12c9a0f8e05fa9e8864aa07f92665375d847e80
2024-04-01 17:53:31 +02:00
fanquake
c7247bd9e8
Merge bitcoin/bitcoin#29765: ci: Use clang-18
fa75220ac5 ci: Use clang-18 in asan/fuzz/tsan task (MarcoFalke)
fad23a0646 ci: Bump clang+llvm in i686_multiprocess task (MarcoFalke)

Pull request description:

  Use clang-18, which comes with bugfixes and sanitizer upgrades.

ACKs for top commit:
  fanquake:
    ACK fa75220ac5

Tree-SHA512: da251eb55d3a7e537110b03fb0fc10fc6e2f4406d55bdb33bdc91459daabfd3b8d1d20859a66718bb1b09126e11495f6584c52504a85eb09bde9eafafdaf0b0b
2024-04-01 16:27:18 +02:00
MarcoFalke
fa75220ac5
ci: Use clang-18 in asan/fuzz/tsan task 2024-03-29 16:41:18 +01:00
MarcoFalke
fad23a0646
ci: Bump clang+llvm in i686_multiprocess task 2024-03-29 15:19:08 +01:00
laanwj
6c2990416e ci: Pull in qtbase5-dev instead of seperate low-level libraries
Fix CI build for t64 migration.
2024-03-29 11:17:39 +01:00
MarcoFalke
fa22a438fa
ci: Print tsan errors to stderr 2024-03-26 19:19:28 +01:00
MarcoFalke
faecf3a7e6
ci: Bump msan to llvm-18 2024-03-19 17:47:22 +01:00
fanquake
8e95a9cd7a
Merge bitcoin/bitcoin#29094: ci: Better tidy errors
fae70ba00d ci: Better tidy errors (MarcoFalke)

Pull request description:

  Currently tidy errors are not nice, because the user may have to scroll up to see them in a large block of text. See for example (before) https://github.com/bitcoin/bitcoin/runs/19670551485

  Fix that by `tee`ing the output to a file and summarizing the errors in the end again. See for example (after): https://github.com/bitcoin/bitcoin/runs/22647850662

ACKs for top commit:
  hebasto:
    ACK fae70ba00d, logs with errors look cleaner.
  TheCharlatan:
    ACK fae70ba00d

Tree-SHA512: dcaea557fed40089409d16ce2cbaa8a9cfbf047f601d5daadfee0823b0eed7badc12d803addc0b7b6bb3f1eaf5c787fccb2488475d32c4efd80835f386f761dd
2024-03-19 14:50:39 +00:00
fanquake
80f8b92f4f
remove libbitcoinconsensus
This was deprecated in v27.0, for removal in v28.0.
See discussion in PR #29189.
2024-03-18 16:59:39 +00:00
fanquake
9a459e3ab9
Merge bitcoin/bitcoin#29669: ci: Drop --enable-c++20 option
64722e4359 ci: Drop `--enable-c++20` option (Hennadii Stepanov)

Pull request description:

  This option has ceased to exist since https://github.com/bitcoin/bitcoin/pull/28349.

ACKs for top commit:
  maflcko:
    ACK 64722e4359

Tree-SHA512: bd392c331f775605615e1b236682269b83a1e6363a4d3f09c4d8d54495cf3d22973a921ebf6b8a9f813ba6c024d3324761f3291aaf7f473995f5eaa4c195bc43
2024-03-18 16:28:40 +00:00
fanquake
aba9024c0c
Merge bitcoin/bitcoin#29659: ci: Bump TIDY_LLVM_V
636c9862cf ci: Bump `TIDY_LLVM_V` (Hennadii Stepanov)

Pull request description:

  This PR switches to the latest [IWYU 0.22](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.22), which is compatible with Clang 18.

ACKs for top commit:
  fanquake:
    ACK 636c9862cf

Tree-SHA512: 78ce89244c5e487dd1be8b4bd2ca6f06d19b04b78289ebc21985110574053545dcce5eb622edf2bede2cf7bb58360170e976d30a4484a127d34dd17b1c604e9c
2024-03-18 15:10:31 +00:00
Hennadii Stepanov
64722e4359
ci: Drop --enable-c++20 option
This option has ceased to exist since https://github.com/bitcoin/bitcoin/pull/28349.
2024-03-17 16:54:47 +00:00
Hennadii Stepanov
636c9862cf
ci: Bump TIDY_LLVM_V
This change switches to the latest IWYU 0.22, which is compatible with
Clang 18.
2024-03-15 13:34:05 +00:00
MarcoFalke
fa8409e760
build: Bump g++ minimum supported version to 11 2024-03-14 12:15:22 +01:00
MarcoFalke
fae70ba00d
ci: Better tidy errors 2024-03-14 09:14:06 +01:00
fanquake
ce54330cf6
ci: use Debian Bookworm (GCC 12) for ARM ci job
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2024-03-12 16:26:27 +00:00
fanquake
0da6451c58
ci: use Debian Bookworm (GCC 12) for win64 job
Co-authored-by: MarcoFalke <*~=`'#}+{/-|&$^_@721217.xyz>
2024-03-12 16:26:27 +00:00
fanquake
bd55b7a528
Merge bitcoin/bitcoin#29610: ci: Fix "macOS native" job
acc06bc91f ci, macos: Use `--break-system-packages` with Homebrew's python (Hennadii Stepanov)
ae5f72027f ci: Add workaround for Homebrew's python link error (Hennadii Stepanov)

Pull request description:

  Homebrew [promoted](https://github.com/Homebrew/homebrew-core/pull/150390) `python@3.12` to the default `python3`. Now, our "macOS native" CI job is facing the following issues:

  1. Installing `qt@5` [requires](https://github.com/bitcoin/bitcoin/actions/runs/8216848118/job/22471875454#step:4:51) re-installing `python@3.12`:
  ```
  ==> Fetching dependencies for qt@5: readline, python@3.12 and gettext
  ```
  2. Re-installing `python@3.12` [fails](https://github.com/bitcoin/bitcoin/actions/runs/8216848118/job/22471875454#step:4:127) due to symbolic link conflicts on macOS `x86_64`:
  ```
  ==> Pouring python@3.12--3.12.2_1.ventura.bottle.tar.gz
  Error: The `brew link` step did not complete successfully
  ```
  3. Homebrew's `python@3.12` is marked as externally managed (according to PEP 668), necessitating different approaches for installing Python packages.

  This pull request resolves all the issues mentioned above.

ACKs for top commit:
  m3dwards:
    reACK acc06bc91f to get the CI passing again.

Tree-SHA512: 82cf72bff328f1e0725342431ac14ad4bae7a758186d97db6c7a558e4b661dcbf3fabe536978e26e709c5f6f7f5c11aac46642634c6685f1291592d8d825ad87
2024-03-12 11:43:33 +00:00
Hennadii Stepanov
acc06bc91f
ci, macos: Use --break-system-packages with Homebrew's python
Homebrew's python@3.12 is marked as externally managed (PEP 668),
necessitating different approaches for installing Python packages.

For more details, please refer to https://github.com/orgs/Homebrew/discussions/3404.
2024-03-11 19:57:56 +00:00
Martin Zumsande
ecc036c5d6 ci: add --v2transport to an existing CI job 2024-02-29 13:50:30 -05:00
Ryan Ofsky
84388c942c ci: avoid running git diff after patching
Drop `git diff` command so it is easier to run CI locally if git checkout is a
worktree. Currently it fails because the directory is not recognized as a git
repository.

The `git diff` command was added recently in #28359 commit
fa07ac48d8 and can be avoided just by teeing the
patch to stdout
2024-02-21 11:16:56 -05:00
fanquake
bdddf364c9
Merge bitcoin/bitcoin#29441: ci: Avoid CI failures from temp env file reuse
fa91bf2559 ci: Skip git install if it is already installed (MarcoFalke)
c65fde4831 ci: vary /tmp/env (Sjors Provoost)

Pull request description:

  * Currently, running separate CI tasks at the same time may intermittently fail, because they race to read/write `/tmp/env`. Fix this by adding `$CONTAINER_NAME` to the file name.

  * Also, add `$USER`, while touching the line, to allow different users to run the same CI task at the same time.

  * Also, skip the git install if there is no need.

  Ref: https://github.com/bitcoin/bitcoin/pull/29274

ACKs for top commit:
  Sjors:
    ACK fa91bf2559
  BrandonOdiwuor:
    ACK fa91bf2559
  hebasto:
    ACK fa91bf2559.

Tree-SHA512: 9a8479255a2afb6618f9d0796488d9430ba95266b90ce39536a9817c1974ca4049beeaab5355a38b25171f76fc386dbec06b1919aaa079f08a5a0c0a146232c8
2024-02-20 10:21:44 +00:00
fanquake
8531e1e731
ci: Use DEBUG=1 in depends for MSAN jobs
Followup to #27448, which was deffered, as it produces #27448 and
another similar issue in sqlite, see comment here:
https://github.com/bitcoin/bitcoin/pull/27448#issuecomment-1514902450.
2024-01-26 13:39:05 +00:00
fanquake
800ddef6b9
ci: use LLVM 17.0.6 in MSAN jobs 2024-01-26 13:38:52 +00:00
Sjors Provoost
c65fde4831
ci: vary /tmp/env 2024-01-18 18:18:08 +01:00
fanquake
1dc2c9b385
ci: cleanup C*FLAG usage in Valgrind jobs
This was being used to avoid a missing -O2. After the previous commits,
this is no-longer an issue.
2024-01-16 09:51:19 +00:00
fanquake
17e33fb578
Merge bitcoin/bitcoin#29237: depends: Allow PATH with spaces in directory names.
4756114e50 [depends] Allow PATH with spaces in directory names. (Mark Friedenbach)

Pull request description:

  The goal of this PR is to help close https://github.com/bitcoin/bitcoin/pull/28733. I reverted the change on `depends/config.guess` based on the feedback provided in the previous PR. I've also incorporated the test mentioned by maflcko

ACKs for top commit:
  maflcko:
    lgtm ACK 4756114e50
  hebasto:
    ACK 4756114e50, successfully built depends on Ubuntu 22.04.
  TheCharlatan:
    ACK 4756114e50

Tree-SHA512: ee257f6efd235839156bc236384f08d77b91debc3c257168368a71e70742639f28a3289572b8693609c1109062dc9968e461103d1f4f5679906506e94b54e649
2024-01-15 13:14:32 +00:00
fanquake
8c5e4f42d5
Merge bitcoin/bitcoin#29208: build: Bump clang minimum supported version to 14
aaaace2fd1 fuzz: Assume presence of __builtin_*_overflow, without checks (MarcoFalke)
fa223ba5eb Revert "build: Fix undefined reference to __mulodi4" (MarcoFalke)
fa7c751bd9 build: Bump clang minimum supported version to 14 (MarcoFalke)

Pull request description:

  Most supported operating systems ship with clang-14 (or later), so bump the minimum to that and allow new code to drop workarounds for previous clang bugs.

  For reference:
  * https://packages.debian.org/bookworm/clang (`clang-14`)
  * https://packages.ubuntu.com/jammy/clang (`clang-14`)
  * CentOS-like 8/9 Stream: All Clang versions from 15 to 17
  * FreeBSD 12/13: All Clang versions from 15 to 16
  * OpenSuse Tumbleweed ships with https://software.opensuse.org/package/clang (`clang17`); No idea about OpenSuse Leap

  On operating systems where the clang version is not shipped by default, the user would have to use GCC, or install clang in a different way. For example:

  * https://packages.debian.org/bullseye/g++ (g++-10)
  * https://packages.ubuntu.com/focal/g++-10
  * https://apt.llvm.org/, or nix, or guix, or compile clang from source, ...

ACKs for top commit:
  fanquake:
    ACK aaaace2fd1

Tree-SHA512: 81d066b14cc568d27312f1cc814b09540b038a10a0a8e9d71fc9745b024fb6c32a959af673e6819b817ea7cef98da4abfa63dff16cffb7821b40083016b0291f
2024-01-12 10:03:22 +00:00
Mark Friedenbach
4756114e50 [depends] Allow PATH with spaces in directory names.
added test and update code based on feedback
2024-01-11 15:36:18 -03:00
MarcoFalke
fa0c594b33
ci: Rename tasks (previous releases, macOS cross) 2024-01-11 17:32:43 +01:00
fanquake
f3ca6db8d3
ci: move CMake into base packages
This is already used in multiple CIs, and will soon become a requirement
for most CIs, i.e when we migrate depends packages to use CMake, for
example:
https://github.com/bitcoin/bitcoin/pull/21778#issuecomment-1885576324.

Some of the CIs in 21778 are failing because CMake isn't available, so
just break this out and make CMake globally available.
2024-01-11 11:30:04 +00:00
MarcoFalke
fa7c751bd9
build: Bump clang minimum supported version to 14 2024-01-09 14:17:55 +01:00
Hennadii Stepanov
080763a058
build: Drop ALLOW_HOST_PACKAGES support in depends
The `ALLOW_HOST_PACKAGES` variable was introduced in bitcoin#10508 "to
speed up build and avoid timeout".

It is no longer the case for our CI infrastructure, which uses self-
hosted persistent workers and depends caching.

In the current circumstances, it does not seem worth porting this
feature to the upcoming CMake-based build system.
2024-01-08 15:56:25 +00:00
Hennadii Stepanov
5ddd7c65b4
build: Bump native_clang up to 17.0.6 2023-12-21 09:37:32 +00:00
fanquake
f0e829022a
Merge bitcoin/bitcoin#28967: build: disable external-signer for Windows
308aec3e56 build: disable external-signer for Windows (fanquake)
35537318a1 ci: remove --enable-external-signer from win64 job (fanquake)

Pull request description:

  It's come to light that Boost ASIO (a Boost Process sub dep) has in some
  instances, been quietly  initialising our network stack on Windows (see
  PR https://github.com/bitcoin/bitcoin/pull/28486 and discussion in https://github.com/bitcoin/bitcoin/issues/28940).

  This has been shielding a bug in our own code, but the larger issue
  is that Boost Process/ASIO is running code before main, and doing things
  like setting up networking. This undermines our own assumptions about
  how our binary works, happens before we run any sanity checks,
  and before we call our own code to setup networking. Note that ASIO also
  calls WSAStartup with version `2.0`, whereas we call with `2.2`.

  It's also not clear why a feature like external signer would have a
  dependency that would be doing anything network/socket related,
  given it only exists to spawn a local process.

  See also the discussion in https://github.com/bitcoin/bitcoin/issues/24907. Note that the maintaince of Boost Process in general,
  has not really improved. For example, rather than fixing bugs like https://github.com/boostorg/process/issues/111,
  i.e, https://github.com/boostorg/process/pull/317, the maintainer chooses to just wrap exception causing overflows
  in try-catch blocks: 0c42a58eac. These changes get merged in large,
  unreviewed PRs, i.e https://github.com/boostorg/process/pull/319.

  This PR disables external-signer on Windows for now. If, in future, someone
  changes how Boost Process works, or replaces it entirely with some
  properly reviewed and maintained code, we could reenable this feature on
  Windows.

ACKs for top commit:
  hebasto:
    re-ACK 308aec3e56.
  TheCharlatan:
    ACK 308aec3e56

Tree-SHA512: 7405f7fc9833eeaacd6836c4e5b1c1a7845a40c1fdd55c1060152f8d8189e4777464fde650e11eb1539556a75dddf49667105987078b1457493ee772945da66e
2023-12-13 11:55:21 +00:00
fanquake
dce1dfbc47
Merge bitcoin/bitcoin#28622: build: use macOS 14 SDK (Xcode 15.0)
8ea45e626e build: use macOS 14 SDK (Xcode 15.0) (fanquake)
51c97ffb69 build: patch boost process for macOS 14 SDK (fanquake)
423949a13b depends: add -platform_version to macOS build flags (fanquake)

Pull request description:

  This fixes: https://github.com/bitcoin/bitcoin/pull/28349#issuecomment-1748515277 (cross-compiling with C++20 for macOS). See https://developer.apple.com/xcode/cpp/#c++20 for C++20 support in Apples libc++, some features landed with Xcode 14.3, although many more landed with Xcode 15.0.

ACKs for top commit:
  hebasto:
    ACK 8ea45e626e.
  TheCharlatan:
    ACK 8ea45e626e

Tree-SHA512: 274ce2c9b9f8e4d755c07b8d0d4897a7f92708ac64e6afb7a3f75bdb485e863fc7f40badf3a88b129ce36f6cca72f768dc2ed7fba2bdf0bb6da2bf0c8fedee10
2023-12-07 10:00:47 +00:00
fanquake
d854914043
Merge bitcoin/bitcoin#28992: ci: Use Ubuntu 24.04 Noble for asan,tsan,tidy,fuzz
fad2392c58 ci: Use Ubuntu 24.04 Noble for asan (MarcoFalke)
fa83b65ef8 ci: Use Ubuntu 24.04 Noble for tsan,tidy,fuzz (MarcoFalke)

Pull request description:

  23.10 will be EOL mid next year, so a bump is needed before then for the `master` branch (and possibly the `26.x` branch).

  Doing the bump now is fine, because the clang version is pinned to 17 inside the CI tasks. So a default clang version change in the system image should not affect the tasks. Once clang-18 is available and the default in April next year (https://discourse.ubuntu.com/t/noble-numbat-release-schedule/35649#planned-and-potentially-disruptive-archive-wide-activities-2), the pinned version could be bumped (for CI tasks that require a pin, like tidy), or the pin can be removed (for CI tasks that usually do not require a pin, like fuzz or the sanitizers).

ACKs for top commit:
  fanquake:
    ACK fad2392c58

Tree-SHA512: c40aede4e2281a5d539d5f65d2c08a57bf92e4a00b4f45a4260b57b7443a63d1a0603115da4a3bbd100ac5f6ade3f2eda0916e4b565573741162a76294ec0ac5
2023-12-06 14:13:42 +00:00
fanquake
8ea45e626e
build: use macOS 14 SDK (Xcode 15.0) 2023-12-05 09:55:01 +00:00
MarcoFalke
fad2392c58
ci: Use Ubuntu 24.04 Noble for asan 2023-12-04 12:29:02 +01:00
MarcoFalke
fa83b65ef8
ci: Use Ubuntu 24.04 Noble for tsan,tidy,fuzz 2023-12-04 09:38:42 +01:00
fanquake
35537318a1
ci: remove --enable-external-signer from win64 job
This is redundant in any case, because --enable-external-signer is
already in `BITCOIN_CONFIG_ALL`.
2023-12-01 10:45:57 +00:00
fanquake
97d53e90ba
ci: remove libz-dev from macOS build deps
Remove no-longer used libz-dev (from DMG compression).
Followup to #28432.
Missed this in #28932.
2023-11-30 15:17:19 +00:00
MarcoFalke
fad82fea2b
ci: Reduce use of bash -c
It is confusing to treat commands as a single string. This change is
also required to support paths and strings with spaces in them in the
future.

This requires replacing TEST_RUNNER_ENV with a global export, because it
no longer works. See:

```bash
$ export ENV="A=1" && $ENV ls
bash: A=1: command not found...
```

Or in the CI task:

+ DIR_UNIT_TEST_DATA=/ci_container_base/ci/scratch/qa-assets/unit_test_data/
+ LD_LIBRARY_PATH=/ci_container_base/depends/i686-pc-linux-gnu/lib
+ BITCOIND=bitcoin-node make -j10 check VERBOSE=1
/ci_container_base/ci/test/03_test_script.sh: line 166: BITCOIND=bitcoin-node: command not found

https://github.com/bitcoin/bitcoin/pull/28954/checks?check_run_id=19096858944
https://cirrus-ci.com/task/6718317604372480
2023-11-29 15:13:16 +01:00
MarcoFalke
fafcee4874
ci: Rename test script to 03_test_script.sh 2023-11-28 13:25:40 +01:00
fanquake
0ffcc5b680
ci: remove python3-setuptools from mac build deps 2023-11-23 17:52:38 +00:00
Ryan Ofsky
21bfee0720 depends: bump libmultiprocess to fix capnproto deprecation warnings
This incorporates PR https://github.com/chaincodelabs/libmultiprocess/pull/88
"Fix current deprecation warnings as of capnproto-1.0.1" and reverts the
NO_WERROR CI workaround added in https://github.com/bitcoin/bitcoin/pull/28735
2023-11-17 15:27:19 -05:00
fanquake
e862bceb17
Merge bitcoin/bitcoin#27935: fuzz: call lookup functions before calling Ban
fca0a8938e ci: remove "--exclude banman" for fuzzing in mac (brunoerg)
f9b286353f fuzz: call lookup functions before calling `Ban` (brunoerg)

Pull request description:

  Fixes #27924

  To not have any discrepancy, it's required to call lookup functions before calling `Ban`. If we don't do it, the assertion `assert(banmap == banmap_read);` may fail because `BanMapFromJson` will call `LookupSubNet` and cause the discrepancy between the banned and the loaded one. It happens especially in MacOS (#27924).

  Also, calling lookup functions before banning is what RPC `setban` does.

ACKs for top commit:
  maflcko:
    lgtm ACK fca0a8938e
  dergoegge:
    ACK fca0a8938e

Tree-SHA512: a3d635088a556df4507e65542157f10b41d4f87dce42927b58c3b812f262f4544b6b57f3384eef1097ffdd7c32b8dd1556aae201254960cbfbf48d45551200f7
2023-11-13 10:57:01 +00:00
brunoerg
fca0a8938e ci: remove "--exclude banman" for fuzzing in mac 2023-11-09 10:11:59 -03:00
fanquake
b3898e946c
Merge bitcoin/bitcoin#28826: ci: Switch IWYU to clang_17 branch
9f208c0171 ci: Switch IWYU to `clang_17` branch (Hennadii Stepanov)

Pull request description:

  The IWYU version [0.21](https://github.com/include-what-you-use/include-what-you-use/releases/tag/0.21) has been tagged, and the `clang_17` branch is available now.

ACKs for top commit:
  maflcko:
    lgtm ACK 9f208c0171

Tree-SHA512: 8b8f8743d1c2719b6383b5a6a48356ac02a301d1ce9cee77f93cc04c12de22e9ac6b59e23550a589540e68292cfac0d85bacedc9ca26f6b589011d36ee1d38cf
2023-11-09 13:07:05 +00:00
Hennadii Stepanov
9f208c0171
ci: Switch IWYU to clang_17 branch 2023-11-09 12:04:17 +00:00
fanquake
49d953281d
fuzz: explicitly specify llvm-symbolizer path in runner
It's not completely clear to me why this needs to be explicitly
specified in some environments, and not in others, while at the same time
that `llvm-symbolizer` is already in PATH, but this has fixed the 2 issues
outlined in #28147.

Use `LLVM_SYMBOLIZER_PATH` as the env var, as that is somewhat also used
inside LLVM, but not consistently, i.e it's checked for in the asan_symbolize
script, but not in in the ubsan_symbolize script, or from in compiler-rt.
2023-11-07 16:57:23 +00:00
MarcoFalke
3333f14efa
depends: Bump to capnproto-c++-1.0.1 2023-11-03 12:48:42 +01:00
MarcoFalke
faea58eee4
Bump g++ minimum supported version to 10
Also, enable -Werror=maybe-uninitialized in
ci/test/00_setup_env_native_qt5.sh
2023-10-30 15:12:26 +01:00
MarcoFalke
fae379b6b1
build: Bump minimum supported Clang to clang-13 2023-10-24 18:52:00 +02:00
fanquake
43704827b4
Merge bitcoin/bitcoin#28211: Bump python minimum supported version to 3.9
fa25e8b0a1 doc: Recommend lint image build on every call (MarcoFalke)
faf70c1f33 Bump python minimum version to 3.9 (MarcoFalke)
fa8996b930 ci: Bump i686_multiprocess.sh to latest Ubuntu LTS (MarcoFalke)

Pull request description:

  All supported operating systems ship with python 3.9 (or later), so bumping the minimum should not cause any issues. A bump will allow new code to use new python 3.9 features.

  For reference:
  * https://packages.debian.org/bullseye/python3
  * https://packages.ubuntu.com/focal/python3.9
  * FreeBSD 12/13 also ships with 3.9
  * CentOS-like 8/9 also ships with 3.9 (and 3.11)
  * OpenSuse Leap also ships with 3.9 (and 3.11) https://software.opensuse.org/package/python311-base

  This is for Bitcoin Core 27.0 in 2024 (next year), not the soon upcoming 26.0 next month.

ACKs for top commit:
  Sjors:
    ACK fa25e8b0a1
  jamesob:
    ACK fa25e8b0a1 ([`jamesob/ackr/28211.1.MarcoFalke.bump_python_minimum_supp`](https://github.com/jamesob/bitcoin/tree/ackr/28211.1.MarcoFalke.bump_python_minimum_supp))

Tree-SHA512: 86c9f6ac4b5ba94a62ee6a6062dd48a8295d8611a39cdb5829f4f0dbc77aaa1a51edccc7a99275bf699143ad3a6fe826de426d413e5a465e3b0e82b86d10c32e
2023-10-24 17:24:30 +01:00
MarcoFalke
fa65887377
ci: Add missing --external to podman image prune 2023-10-23 14:10:12 +02:00
fanquake
fc1073bb45
Merge bitcoin/bitcoin#28643: ci: Add missing CI_RETRY_EXE before git clone
6889a80766 ci: Add missing CI_RETRY_EXE before git clone (MarcoFalke)
b705bade44 ci: Export `IN_GETOPT_BIN` on macOS (Hennadii Stepanov)
1c2132ddd9 Revert "ci: Upgrading pip version in macos environment" (Hennadii Stepanov)

Pull request description:

  This PR is a resurrection of https://github.com/bitcoin/bitcoin/pull/28623:

  > This should fix [bitcoin/bitcoin/actions/runs/6457002476/job/17527598426#step:7:240](https://github.com/bitcoin/bitcoin/actions/runs/6457002476/job/17527598426#step:7:240):
  >
  > ```
  > + git clone --depth=1 https://github.com/bitcoin-core/qa-assets /Users/runner/work/bitcoin/bitcoin/ci/scratch/qa-assets
  > Cloning into '/Users/runner/work/bitcoin/bitcoin/ci/scratch/qa-assets'...
  > error: RPC failed; curl 18 HTTP/2 stream 5 was reset
  > error: 54975 bytes of body are still expected
  > fetch-pack: unexpected disconnect while reading sideband packet
  > fatal: early EOF
  > fatal: fetch-pack: invalid index-pack output
  > Error: Process completed with exit code 128.
  > ```

ACKs for top commit:
  maflcko:
    lgtm ACK 6889a80766

Tree-SHA512: cd50102061a5c57fcf53ecbf552e7eae09995395e91651d00037e3f101c4906c0a994b2f6b1ccf984f2fc902ca59f0db66206ace56a2b6e2810dfa4167f3e115
2023-10-20 11:23:04 +01:00
stickies-v
e44c574650
ci: always prune all dangling bitcoin-ci-test images
Since all bitcoin-ci-test images are now labeled, we can always
prune all dangling images, regardless of whether we are in
RESTART_CI_DOCKER_BEFORE_RUN.

To be safe, still prune all images if RESTART_CI_DOCKER_BEFORE_RUN
in case the filtering doesn't work, or if images were created on
an earlier version that did not assign labels.
2023-10-13 11:23:30 +01:00
MarcoFalke
6889a80766
ci: Add missing CI_RETRY_EXE before git clone 2023-10-13 10:37:34 +01:00
Hennadii Stepanov
b705bade44
ci: Export IN_GETOPT_BIN on macOS
This variable is required for the `retry` script.
2023-10-13 10:37:34 +01:00
Hennadii Stepanov
1c2132ddd9
Revert "ci: Upgrading pip version in macos environment"
This reverts commit 057750c09d.

It is not needed anymore in the GHA CI.
This change will make the code much simpler in the following commit.
2023-10-13 10:37:28 +01:00
stickies-v
ce1699706e
ci: add label to docker images
This allows us or the user to perform batch operations on all
images produced by the ci, e.g. to prune all dangling images,
without affecting non-ci images.
2023-10-13 10:17:21 +01:00
fanquake
504f1f30ed
Merge bitcoin/bitcoin#28547: ci: Work around podman stop intermittent failure
fa2c894cbb ci: move-only CI_CONTAINER_ID to 02_run_container.sh (MarcoFalke)
fa695b4df0 ci: Work around podman stop bug (MarcoFalke)
fa09a031c1 ci: Add set -ex to 02_run_container.sh (MarcoFalke)
fac9abbf47 ci: Rename 04_install to 02_run_container (MarcoFalke)

Pull request description:

  Sometimes, it seems that `podman stop` does not work. Presumably, it falls back to `podman kill`, which is async.

  Try to work around this intermittent issue by using the `rm --force` over `stop`.

  Example failing log https://cirrus-ci.com/task/4549784611061760?logs=ci#L238:

  ```
  Restart docker before run to stop and clear all containers started with --rm
  ++ podman container stop --all
  e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009
  ++ echo 'Prune all dangling images'
  Prune all dangling images
  ++ docker image prune --force
  Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
  +++ docker run --cap-add LINUX_IMMUTABLE --rm --interactive --detach --tty --mount type=bind,src=/tmp/cirrus-build-1970593815,dst=/tmp/cirrus-build-1970593815,readonly --mount type=volume,src=ci_macos_cross_ccache,dst=/tmp/ccache_dir --mount type=volume,src=ci_macos_cross_depends,dst=/ci_container_base/depends --mount type=volume,src=ci_macos_cross_previous_releases,dst=/ci_container_base/prev_releases --env-file /tmp/env --name ci_macos_cross ci_macos_cross
  Emulate Docker CLI using podman. Create /etc/containers/nodocker to quiet msg.
  time="2023-09-27T20:55:39Z" level=warning msg="The input device is not a TTY. The --tty and --interactive flags might not work properly"
  Error: creating container storage: the container name "ci_macos_cross" is already in use by e4eca0766f87864d89fc230aa884a238c214cfbcd44cf76a4dbdb2a30c982009. You have to remove that container to be able to reuse that name: that name is already in use

ACKs for top commit:
  hebasto:
    ACK fa2c894cbb, I have reviewed the code and tested it locally.

Tree-SHA512: 31fca340c6bedaadf4dd51fa745d9b3969042cebc0c7c904ef18af3f2f986039ec4354ccdff1422fbf77cf223e4423857368dce53cfa67ef15c76b78d007eace
2023-10-13 10:24:07 +02:00
MarcoFalke
fa2843eba4
ci: Bump asan 2023-10-12 13:07:06 +02:00
fanquake
ccf7895e17
Merge bitcoin/bitcoin#28633: ci: Install Qt's default Android API platform
78d3062b68 ci: Install Android API 31 platform as Qt expects (Hennadii Stepanov)

Pull request description:

  When building the `qt` package, it expects that the default (in Qt's view) Android API platform is installed.

  During the recent Qt version [update](https://github.com/bitcoin/bitcoin/pull/28561), it has been changed:
  ```diff
  --- a/mkspecs/features/android/sdk.prf
  +++ b/mkspecs/features/android/sdk.prf
  @@ -1,6 +1,6 @@
   API_VERSION_TO_USE = $$(ANDROID_API_VERSION)
   isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = $$API_VERSION
  -isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-28
  +isEmpty(API_VERSION_TO_USE): API_VERSION_TO_USE = android-31

   ANDROID_JAR_FILE = $$ANDROID_SDK_ROOT/platforms/$$API_VERSION_TO_USE/android.jar
   !exists($$ANDROID_JAR_FILE) {
  ```

  This PR fixes the CI for the Android task and addresses https://github.com/bitcoin/bitcoin/pull/28561#issuecomment-1749180177.

  Qt [docs](https://doc.qt.io/qt-5/android.html) still claim that Android API Level 21 and up are supported, however, I did not test every possible configuration.

  NOTE: https://github.com/bitcoin/bitcoin/pull/28611 is still valid.

ACKs for top commit:
  maflcko:
    lgtm ACK 78d3062b68
  jarolrod:
    tACK 78d3062b68

Tree-SHA512: 781fba6d80aae7e6500854de14af0d30169c258e395b9e482a5430a7b4a2211a6181f8c9ee58543c896b431abf09e3e7c5573b9672ed128658f11f98a2006e7e
2023-10-12 09:11:49 +02:00
Hennadii Stepanov
78d3062b68
ci: Install Android API 31 platform as Qt expects 2023-10-10 19:29:18 +01:00
fanquake
8735e2c136
ci: use LLVM/Clang 17 in tidy job 2023-10-10 11:02:10 +01:00
fanquake
ce46b68941
ci: use LLVM 17.0.2 in MSAN jobs 2023-10-10 10:23:18 +01:00
MarcoFalke
fa2c894cbb
ci: move-only CI_CONTAINER_ID to 02_run_container.sh
This limits the scope of the CI_CONTAINER_ID symbol.

Can be reviewed with --color-moved=dimmed-zebra
2023-10-09 16:17:04 +02:00
MarcoFalke
fa695b4df0
ci: Work around podman stop bug
Force remove any containers, pontentially leaving dangling processes,
which should be fine.
2023-10-09 16:16:55 +02:00
MarcoFalke
fa09a031c1
ci: Add set -ex to 02_run_container.sh
The same is done by the 06 script.
2023-10-09 16:16:41 +02:00
MarcoFalke
fac9abbf47
ci: Rename 04_install to 02_run_container
This reflects what the script does (docker run ...).
2023-10-09 16:16:22 +02:00
MarcoFalke
fac88a874f
ci: Avoid cache depends/build 2023-10-05 13:45:04 +02:00
MarcoFalke
fac054d24c
ci: Print Linux kernel info 2023-10-03 16:37:43 +02:00
fanquake
99ce8366ed
Merge bitcoin/bitcoin#28504: ci: Use nproc over MAKEJOBS in 01_base_install
fa3b5e5e57 ci: Use nproc over MAKEJOBS in 01_base_install (MarcoFalke)

Pull request description:

  Currently `$MAKEJOBS` is the default value in `01_base_install.sh` when building the container image.

  This problem can't be fixed (see below), so just use `nproc` for now.

  Other solutions would be bad:

  * Passing in the `MAKEJOBS` as a dockerfile env would create a new image if the number of tasks are changed, seems verbose and confusing.
  * Leaving `master` as-is would leave CPUs unused if there are more than `4`.

ACKs for top commit:
  fanquake:
    ACK - I think this is fine as-is, it's an improvement over the current state fa3b5e5e57

Tree-SHA512: 15014eb7b548945737b0fed5cd46f0cd4b72b1d54d044f60fce628f914053a2de6518878428a54822d236d08d6f257d8c464d3fb589400f4be56022d2ec8676d
2023-09-20 16:14:30 +00:00
fanquake
e9a4793b82
Merge bitcoin/bitcoin#28432: build: Produce a .zip for macOS distribution
b5790c35f7 build: remove dmg dependencies (fanquake)
33ae0bd1e4 macdeploy: remove DMG generation from deploy script (fanquake)
a128111c29 build: produce a .zip for macOS distribution (Hennadii Stepanov)
c38561d6b1 build: add -zip option to macdeployqtplus (fanquake)

Pull request description:

  It is https://github.com/bitcoin/bitcoin/pull/27099 revived with addressed [comments](https://github.com/bitcoin/bitcoin/pull/27099#issuecomment-1708705686).

  From https://github.com/bitcoin/bitcoin/pull/27099#issue-1584429885:
  > Reviving the discussion around using a `.zip` for the distributed macOS binaries, as opposed to a `.dmg`.
  >
  > Given we only had a single report of the "no finder window" issue (#26176), I wonder if that means macOS users were able to figure it out, they gave up/didn't report, or, we just have very few macOS users.
  >
  > Related to #18128.

  That's how it looks on macOS:

  ![image](https://github.com/bitcoin/bitcoin/assets/32963518/baa637bb-256b-4b24-8645-8c2754c2ae64)

ACKs for top commit:
  Sjors:
    tACK b5790c35f7
  jarolrod:
    ACK b5790c35f7
  TheCharlatan:
    utACK b5790c35f7

Tree-SHA512: 6e9cb3ab0f60f8a92bfec50577e8d096c5b23ec09ebbb334826415609140ddc96d470aea37379495c1c6bb1beec0d306b09460f62e1543bb0f4396c10a1dfbe2
2023-09-20 11:40:47 +00:00
MarcoFalke
fa3b5e5e57
ci: Use nproc over MAKEJOBS in 01_base_install 2023-09-19 16:13:27 +00:00
fanquake
b5790c35f7
build: remove dmg dependencies 2023-09-15 13:47:50 +01:00
Hennadii Stepanov
a128111c29
build: produce a .zip for macOS distribution
Instead of a .dmg.

Co-authored-by: fanquake <fanquake@gmail.com>
2023-09-15 13:47:50 +01:00
fanquake
717a4d8944
Merge bitcoin/bitcoin#28476: ci: LLVM 17 for MSAN jobs
a241d6069c ci: use LLVM 17.0.0 in MSAN jobs (fanquake)

Pull request description:

  See https://libcxx.llvm.org/Hardening.html as well as https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.

ACKs for top commit:
  MarcoFalke:
    review ACK a241d6069c

Tree-SHA512: c374dabf307fe762be0da96f63695a150f6018c1468fe9414fad23f74f5818bbf7a5a699e109084e31467482a900cfebf1d5835821e4da94aa310b2c9570749c
2023-09-15 11:49:29 +01:00
Andrew Chow
541976b42e
Merge bitcoin/bitcoin#27850: test: Add unit & functional test coverage for blockstore
de8f9123af test: cover read-only blockstore (Matthew Zipkin)
5c2185b3b6 ci: enable chattr +i capability inside containers (Matthew Zipkin)
e573f24202 unit test: add coverage for BlockManager (Matthew Zipkin)

Pull request description:

  This PR adds unit and functional tests to cover the behavior described in #2039. In particular, that bitcoind will crash on startup if a reindex is requested but the `blk` files are read-only. Eventually this behavior can be updated with https://github.com/bitcoin/bitcoin/pull/27039. This PR just commits the test coverage from #27039 as suggested in https://github.com/bitcoin/bitcoin/pull/27039#issuecomment-1584915782

ACKs for top commit:
  jonatack:
    ACK de8f9123af modulo suggestions in https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1319010039, tested on macOS, but not on Linux for the Linux-related change in the last push
  achow101:
    ACK de8f9123af
  MarcoFalke:
    lgtm ACK de8f9123af 📶

Tree-SHA512: b9bd684035dcea11c901b649fc39f397a2155a9a8459f3348e67947e387e45312fddeccb52981aef486f8a31deebb5356a7901c1bb94b78f82c24192a369af73
2023-09-14 13:21:14 -04:00
fanquake
a241d6069c
ci: use LLVM 17.0.0 in MSAN jobs
See https://libcxx.llvm.org/Hardening.html as well as
https://discourse.llvm.org/t/rfc-removing-the-legacy-debug-mode-from-libc/71026.
2023-09-14 11:12:24 +01:00