Commit graph

625 commits

Author SHA1 Message Date
MarcoFalke
fad0f21c3c
ci: Use clang in multiprocess task to avoid OOM 2021-05-01 11:17:35 +02:00
fanquake
c29cba44b3
build: Xcode 12.1, macOS SDK 10.15.6 2021-05-01 13:39:45 +08:00
MarcoFalke
faeabef4f3
ci: Enable D_GLIBCXX_DEBUG for multiprocess task 2021-04-30 08:33:07 +02:00
windsok
6f6bb3ebc7 test: fix file permissions on various scripts
Updates permissions on files to comply with the new test added in the following commit
2021-04-23 17:13:28 -07:00
MarcoFalke
00004565cc
ci: Use clang-12 for asan task
Changes from Ubuntu Focal to Hirsute to get clang-12.
Generally, it can be expected that more recent compilers have
better support for sanitizers.
2021-04-20 12:46:03 +02:00
MarcoFalke
fadbd99885
test: Remove spurious double lock tsan suppressions by bumping to clang-12 2021-04-13 19:54:39 +02:00
Hennadii Stepanov
b367745cfe
ci: Make Cirrus CI Windows build with --enable-werror 2021-04-13 10:21:11 +03:00
fanquake
f0b457212f
Merge #21467: Move external signer out of wallet module
88d4d5ff2f rpc: add help for enumeratesigners and walletdisplayaddress (Sjors Provoost)
b0db187e5b ci: use --enable-external-signer instead of --with-boost-process (Sjors Provoost)
b54b2e7b1a Move external signer out of wallet module (Sjors Provoost)

Pull request description:

  In addition, this PR enables external signer testing on CI.

  This PR moves the ExternalSigner class and RPC methods out of the wallet module.

  The `enumeratesigners` RPC can be used without a wallet since #21417. With additional modifications external signers could be used without a wallet in general, e.g. via `signrawtransaction`.

  The `signerdisplayaddress` RPC is ranamed to `walletdisplayaddress` because it requires wallet context. A future `displayaddress` RPC call without wallet context could take a descriptor argument.

  This commit fixes a `rpc_help.py` failure when configured with `--disable-wallet`.

ACKs for top commit:
  ryanofsky:
    Code review ACK 88d4d5ff2f
  fanquake:
    ACK 88d4d5ff2f

Tree-SHA512: 3242a24e22313aed97eee32a520bfcb1c17495ba32a2b8e06a5e151e2611320e2da5ef35b572d84623af0a49a210d2f9377a2531250868d1a0ccf3e144352a97
2021-04-13 14:35:16 +08:00
MarcoFalke
fa278412a0
ci: Restart docker before run
Also, add setting for persistent worker in .cirrus.yml
2021-04-09 15:56:23 +02:00
Sjors Provoost
b0db187e5b
ci: use --enable-external-signer instead of --with-boost-process
An earlier version of #16546 used both --with-boost-process and --enable-external-signer, which was simplified to only use the latter. However I forgot to update CI, so the external signer tests were not run.
2021-04-08 17:56:00 +02:00
MarcoFalke
fa97a17ac3
ci: Bump Android cross-build to Ubuntu Focal
This does not change behavior, but bumping to Focal now means it doesn't
have to be done later when Bionic is no longer used and EOL.
2021-03-28 20:05:46 +02:00
MarcoFalke
fac577d423
ci: Build depends only once for Android build
Depends is currently built twice for the Android build. For example, the
same task building it twice:

* https://cirrus-ci.com/task/6673185279049728?logs=ci#L3418 (aarch64-linux-android)
* https://cirrus-ci.com/task/6673185279049728?logs=ci#L3422 (x86_64-pc-linux-gnu, 4 lines later)
2021-03-28 20:05:31 +02:00
MarcoFalke
fa908a41f3
ci: Set DEPENDS_DIR when setting BASE_ROOT_DIR
The depends dir can not be overwritten by a FILE_ENV file. Also, a FILE_ENV file
might depend on the DEPENDS_DIR value. Thus, set it before reading FILE_ENV.

This commit does not change behavior, but is required for later commits.

Can be reviewed with --color-moved=dimmed-zebra
2021-03-28 20:04:24 +02:00
Igor Cota
7563720e30 CI: add Android APK build script 2021-03-21 22:33:35 +01:00
Hennadii Stepanov
e4c0cada79
ci, gitian: Drop unneeded python3-dev package for macOS builds 2021-03-14 14:55:12 +02:00
Hennadii Stepanov
3272e34f9c
build: Add xkbcommon 0.8.4
Co-authored-by: fanquake <fanquake@gmail.com>
2021-03-10 08:23:47 +08:00
Hennadii Stepanov
f7f3829a68
build, doc: Drop libbz2-dev from macOS cross-compiling dependencies 2021-03-03 22:43:08 +02:00
Hennadii Stepanov
d8239362e2
build, doc: Drop libcap-dev from macOS cross-compiling dependencies 2021-03-03 22:38:45 +02:00
Sjors Provoost
29d6b1da2a test: previous releases: add v0.20.1
Can be reviewed with --ignore-all-space
2021-02-25 08:50:34 +01:00
fanquake
ccb7b6a976
contrib: run test-symbol check for RISCV
Now that we are using Focal for Gitian building (glibc 2.31), we can
user a newer introduced  symbol, and include RISCV in this test.
2021-02-22 08:35:06 +08:00
MarcoFalke
fa330d8fed
ci: Avoid invoking curl on the host 2021-02-20 09:52:55 +01:00
MarcoFalke
6a680a6236
Merge #21226: build: Fix fuzz binary compilation under windows
56ace907b9 Fix fuzz binary compilation under windows (Dan Benjamin)

Pull request description:

  Small change to allow the fuzz binary to compile under windows. Also removed --disable-fuzz-binary from the windows CI test. This fixes #21212.

ACKs for top commit:
  MarcoFalke:
    review ACK 56ace907b9 the best bugfixes are the ones removing code

Tree-SHA512: 6088fd955a5e511b5ca1b3eaa8469a889eb6d994c2827acac7695dac6e4e320a344b45f4015a2f279b16df0d4b23ec4df13304ae6315395ad2fe8c5b526cada4
2021-02-19 11:30:41 +01:00
Dan Benjamin
56ace907b9 Fix fuzz binary compilation under windows 2021-02-18 17:35:24 -05:00
fanquake
cad8b527ea
build: explicitly install libboost-dev package
This package is currently installed as a side-effect of installing our
other libboost-*-dev packages. However as those continue to dissapear,
it makes sense to install boost dev explicitly.
2021-02-17 09:04:20 +08:00
Wladimir J. van der Laan
9996b1806a
Merge #21064: refactor: use std::shared_mutex & remove Boost Thread
060a2a64d4 ci: remove boost thread installation (fanquake)
06e1d7d81d build: don't build or use Boost Thread (fanquake)
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)

Pull request description:

  This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).

  Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.

  Two other points:

  [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
  > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.

  Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
  * The version of Boost.
  * The platform you're building for.
  * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
  * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.

  A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.

  With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.

  Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
  Also related to #21022 - pthread sanity checking.

ACKs for top commit:
  laanwj:
    Code review ACK 060a2a64d4
  vasild:
    ACK 060a2a64d4

Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-12 11:39:36 +01:00
MarcoFalke
faa8afe70b
ci: Re-run wine tests once if they fail 2021-02-10 08:59:35 +01:00
MarcoFalke
fa272dfdff
ci: Properly bump to focal for win cross build 2021-02-09 21:37:14 +01:00
fanquake
ac24af453d
ci: use Ubuntu Focal for macOS cross build 2021-02-09 13:58:59 +08:00
MarcoFalke
ca85449f22
Merge #21036: gitian: Bump descriptors to Focal for 22.0
2ecaf21433 gitian: remove execstack workaround for ricv64 & powerpc64le (fanquake)
5baff2b318 build: use focal in gitian descriptors (fanquake)

Pull request description:

  This PR changes the gitian descriptors to use Ubuntu Focal (20.04), over Bionic (18.04), moving from GCC 7.5 to GCC 8.4 for native Linux builds, mingw-w64 GCC 7.3 to mingw-w64 GCC 9.3 for Windows builds, while continuing to use GCC 8.4 for all cross builds and Clang 8.0.0 for macOS builds.

  It also drops the `-Wl,-z,noexecstack` workaround we've been using for the riscv64 and powerpc64le hosts, as it's no-longer needed. One new package is installed in the osx build, `libtinfo5`, as libtinfo5.so is required by our downloaded Clang 8.

  A bump to Focal will at least be required if we want to update to a newer Qt (5.15, #19716) for 22.0, as we need a newer version of [`g++-mingw-w64`](https://packages.ubuntu.com/focal/g++-mingw-w64-x86-64) and the [`mingw-w64`](https://mingw-w64.org/doku.php) headers. This can still be done while continuing to use GCC 8.4 for Linux builds (see below), however the newer `g++-mingw-w64` will be based off of GCC 9.3.

  **Some considerations**

  GCC 9 is affected by #20005 "memcmp with constants that contain zero bytes are broken in GCC", and the newer `g++-mingw-w64` will be based off of GCC 9.3.

  The `--no-*` variants of the Windows linker flags (i.e `--no-dynamicbase`) we use to [test our `security-check.py` script](16b784d953/contrib/devtools/test-security-check.py (L53)) are not patched into the mingw binutils in Focal (they have been re-added in Groovy (20.10)). This isn't currently an issue, however, we might add a call to `test-security-check` for Guix (#20980), and if we wanted to do the same for gitian, it would not work. Note how it's quite "easy" for us to apply the `--no-*` variant patch to our Guix build; it would be quite a bit harder to do in Gitian.

  Gitian Builds @ 2ecaf21433

  #### Linux
  ```bash
  8882ea78486fbae4fac574b9089eb1107c6372d0dd7dfcda4f0f930576f9d6c1  bitcoin-2ecaf214331b-aarch64-linux-gnu-debug.tar.gz
  50a9e30943b4eee5163edff3331241e745ff32a2c4463c21a6fdc5986e2d0383  bitcoin-2ecaf214331b-aarch64-linux-gnu.tar.gz
  ec4e55a447fddf033fee33cd5f22bfeda3c3612f059194bcf6238859f7989d7a  bitcoin-2ecaf214331b-arm-linux-gnueabihf-debug.tar.gz
  444fe1b3b933c00bcbd4a9d86888cff3b61c1215b1debccd2843e842d1224777  bitcoin-2ecaf214331b-arm-linux-gnueabihf.tar.gz
  88e486ff465980dc1a4aab9687d142ec6f727ed2c52cf539f69db2877dee83b2  bitcoin-2ecaf214331b-powerpc64-linux-gnu-debug.tar.gz
  66144ac264c65cada9d86446e6026c85b04fb88198b8f41b42840f6031db3e6c  bitcoin-2ecaf214331b-powerpc64-linux-gnu.tar.gz
  34bcc13d78d929d575e34e77a6672f23ca7ea23230b28ec2eed563889352ba86  bitcoin-2ecaf214331b-powerpc64le-linux-gnu-debug.tar.gz
  b4c5f959664f3063df4330edfe343c17120eb6b556ee1c15c4aeb2c1c54ffd49  bitcoin-2ecaf214331b-powerpc64le-linux-gnu.tar.gz
  918fa72ab6f6ebce4e9663c93f72fe26651c260477cbb54749f7eb61438b5cc1  bitcoin-2ecaf214331b-riscv64-linux-gnu-debug.tar.gz
  f704f9f8c053ffe37d854e2e81e0f4c0614c435dad7f5d82518c681b73a76ae6  bitcoin-2ecaf214331b-riscv64-linux-gnu.tar.gz
  b59e3a62f1df9d79f30e916b3c9655f654036fe3a420040c53acc8dd9f4162c5  bitcoin-2ecaf214331b-x86_64-linux-gnu-debug.tar.gz
  a4dc9ca877cc97544e65db11be38406d16f15d74fcdcd2318bb92474729bc60d  bitcoin-2ecaf214331b-x86_64-linux-gnu.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  af6ebc91147778e4e6705eade62608dde4d6e60522d79087fa9129bdb7c01199  bitcoin-core-linux-22-res.yml
  ```

  #### Windows
  ```bash
  121a3970a6911cb8c453b2ce37d03f6cbb43333e29db8fa516c68563fb367f43  bitcoin-2ecaf214331b-win-unsigned.tar.gz
  6294e9efebe935092f9ba119dc60ad4094f18b51c4181324e54d3057524d6101  bitcoin-2ecaf214331b-win64-debug.zip
  5b5a236b63e67f5f6c07ad9aa716aa7b72fb63722c96798b332c6d164738f9cf  bitcoin-2ecaf214331b-win64-setup-unsigned.exe
  c1fa5894c5e02a201637567c80b9bde9024f44673dcd06fd4d489c1709179279  bitcoin-2ecaf214331b-win64.zip
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  665fd7eb61aed368150db58a254f15fb5efb51a4efa5abcc52571cb7a1a5de22  bitcoin-core-win-22-res.yml
  ```

  #### macOS
  ```bash
  6a1deae7662aa782baa82a42590f862c6bcdc4f4e38daa9b8c2a9eed1fbb5397  bitcoin-2ecaf214331b-osx-unsigned.dmg
  1ee843266e84928a4323fa255c833528c2617a2c9fd2f98fb26ba19bbfc1227b  bitcoin-2ecaf214331b-osx-unsigned.tar.gz
  097b64dadc167d8e5b733421bf1541a40760ad952990f7cf3f35adc6ae2616d0  bitcoin-2ecaf214331b-osx64.tar.gz
  b40ba2d5da498330ade92a4ccebcceb1452b94c8ffeacb336f87e93b5c88d8af  src/bitcoin-2ecaf214331b.tar.gz
  6e378fb543928e40c7119b96be6ff773d38506a9a888f8b02c7f1b8a0801a80e  bitcoin-core-osx-22-res.yml
  ```

ACKs for top commit:
  laanwj:
    Build script changes review ACK 2ecaf21433

Tree-SHA512: 975d5830b787d2e08988f43cbc6e839294171c1d94c8219636308b05f9b77041421612ae67be24a631674670cfc9c2d96d8177f2b3158a78fc3deea19631febf
2021-02-08 10:44:37 +01:00
Dan Benjamin
32cbb06676 build: build fuzz tests by default.
This fixes issue #19388. The changes are as follows:
  - Add a new flag to configure, --enable-fuzz-binary, which allows building test/fuzz/fuzz regardless of whether we are building to do actual fuzzing
  - Set -DPROVIDE_MAIN_FUNCTION whenever --enable-fuzz is no
  - Add the following libraries to FUZZ_SUITE_LD_COMMON:
    - LIBBITCOIN_WALLET
    - SQLLITE_LIBS
    - BDB_LIBS
    - if necessary, some or all of:
      - NATPMP_LIBS
      - MINIUPNPC_LIBS
      - LIBBITCOIN_ZMQ / ZMQ_LIBS
2021-02-05 19:52:45 -05:00
fanquake
5baff2b318
build: use focal in gitian descriptors
Compilers used change as follows:
Linux native GCC 7.5 -> GCC 8.4
Linux cross GCC 8.4 -> GCC 8.4
Windows mingw-w64 7.3 -> mingw-w64 9.3
macOS Clang 8.0.0 -> Clang 8.0.0

The macOS and Win cross builds in the CI are updated to use Focal, and
per the op, running the security tests is disabled in the Windows
build.
2021-02-03 21:39:54 +08:00
fanquake
060a2a64d4
ci: remove boost thread installation
Adjust fuzzbuzz.yml to only install the Boost components we need.
2021-02-02 12:38:22 +08:00
MarcoFalke
faff3991a9
ci: Fuzz with integer sanitizer 2021-01-26 12:50:43 +01:00
Hennadii Stepanov
e28f9be87a
ci: Add libnatpmp-dev package to some builds 2021-01-07 18:07:10 +02:00
Hennadii Stepanov
a8d9f275d0
net: Add libnatpmp support 2021-01-07 18:07:09 +02:00
MarcoFalke
fa0074e2d8
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-12-31 09:45:41 +01:00
Hennadii Stepanov
09d105ef0f
ci: Drop travis_fold feature as Travis CI is no longer used 2020-12-17 22:02:12 +02:00
fanquake
0df9819126 build: Replace genisoimage with xorriso
xorriso and its mkisofs/genisoimage emulation alter-ego xorrisofs are
more maintained, and has the right toggles for us to achieve output
determinism without using blunt tools like faketime.

In this commit, we use xorrisofs from the build environment rather than
building it ourselves using depends. This is not necessary and can be
changed in the future.

From https://wiki.debian.org/genisoimage?action=recall&rev=11 :

> The classical command line interface for production of ISO 9660
> filesystem images is the option set established by program mkisofs.
> For reasons of licensing and other problems with its author, Debian
> ships a fork of mkisofs, called genisoimage, which was split off in
> 2006 and then developed independently.
>
> Meanwhile, genisoimage gets no new features and not even bug fixes. It
> is first choice only if its options -udf or -hfs are needed.
>
> Replacement in most uses cases, especially for bootable ISO 9660
> filesystems, archiving, and backup, is xorrisofs which starts the -as
> mkisofs emulation mode of program xorriso.
2020-12-16 12:29:14 -05:00
Hennadii Stepanov
2f6fe4e4e9
ci: Build with --enable-werror by default, and document exceptions 2020-12-03 20:51:39 +02:00
Wladimir J. van der Laan
a3186b6da6
Merge #20520: depends: Do not force Precompiled Headers (PCH) for building Qt on Linux
c82d15b6d1 depends: Do not force Precompiled Headers (PCH) for building Qt on Linux (Hennadii Stepanov)

Pull request description:

  On CentOS 8 (Cirrus CI job) the forced `-pch` option breaks Qt build.

  Removing `-pch` option does not affect build time for other systems:

  - master (e2ff5e7b35):
  ```
  $ time make -j 9 -C depends/ qt
  ...
  Caching qt...
  make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'

  real	4m22,359s
  user	18m3,719s
  sys     1m24,769s
  ```

  - this PR:
  ```
  $ time make -j 9 -C depends/ qt
  ...
  Caching qt...
  make: Leaving directory '/home/hebasto/guix/GitHub/bitcoin/depends'

  real	4m14,862s
  user	18m3,355s
  sys 	1m24,506s
  ```

  Qt docs: https://doc.qt.io/qt-5/qmake-precompiledheaders.html

  Fixes #20423

ACKs for top commit:
  MarcoFalke:
    review ACK c82d15b6d1

Tree-SHA512: 0f2a3712e90de881d00f8e56c363edde33dd4f5c117df5744ab4e51d0a8146331de7236bc8329d68ddd91535cd853e68ee80ef4cceb6a909786abfd8881b01e8
2020-12-03 13:23:09 +01:00
Hennadii Stepanov
c82d15b6d1
depends: Do not force Precompiled Headers (PCH) for building Qt on Linux
On CentOS 8 the forced '-pch' option breaks Qt build.
Removing '-pch' option does not affect build time for other Linux
systems.
2020-12-02 13:49:25 +02:00
fanquake
2b356117e9
ci: no-longer exclude feature_block in TSAN job
The TSAN job is now running on Cirrus.
Increase the allocated memory to the maximum allowed.
2020-12-02 16:42:07 +08:00
fanquake
e2c500636c
depends: build zeromq with -std=c++17 2020-11-24 10:18:06 +08:00
fanquake
2374f2fbef
depends: build Boost with -std=c++17 2020-11-24 10:17:35 +08:00
MarcoFalke
fa73674738
ci: Run i686 centos ci config on cirrus 2020-11-23 17:09:12 +01:00
Wladimir J. van der Laan
555b5d1bf9
Merge #20419: build: set minimum supported macOS to 10.14
a52ecc936a build: set minimum supported macOS to 10.14 (fanquake)

Pull request description:

  This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):

  > You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.

  > While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.

  > I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.

  We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).

ACKs for top commit:
  hebasto:
    ACK a52ecc936a, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
2020-11-23 14:24:58 +01:00
MarcoFalke
faaee810e6
build: Require C++17 compiler 2020-11-18 15:15:04 +01:00
fanquake
a52ecc936a
build: set minimum supported macOS to 10.14 2020-11-18 21:46:09 +08:00
MarcoFalke
4b24c3962f
Merge #19504: Bump minimum python version to 3.6
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them (Anthony Towns)
8ae9d314e9 Bump minimum python version to 3.6 (Anthony Towns)

Pull request description:

  Python 3.5 has reached [end-of-life](https://devguide.python.org/#status-of-python-branches) as of September 2020, and 3.6 has some moderately nice [features](https://docs.python.org/3/whatsnew/3.6.html):

  - `f'x = {x}'` as an alternative to `'x = {}'.format(x)` format strings (cf https://github.com/bitcoin/bitcoin/pull/13718#issuecomment-406591027)
  - underscore separators for large numbers, like `1_234_567`
  - improvements to async
  - improvements to typing module

  Note that 3.6 is not available in xenial (16.04), but is available in bionic (18.04), while focal (20.04) has 3.8. CentOS 7 and 8 have 3.6.8, Debian stable has 3.7.3, and [gentoo and arch already had 3.6 and 3.7 in 2018](https://github.com/bitcoin/bitcoin/pull/14954#issuecomment-447118707).

ACKs for top commit:
  MarcoFalke:
    re-ACK 97c738ff1b

Tree-SHA512: ec7fce68845edde4d61a42de12c065fd49e5217311a6fda1323206f091a0afd50f293645dffc27d420127e4e5deb864e953f1b67eff735a0dfbbedd7899a9d60
2020-11-18 10:24:22 +01:00
Wladimir J. van der Laan
4fd37d0a10
Merge #20292: test: Fix intermittent feature_taproot issue
fab900802d ci: Bump timeout factor (MarcoFalke)
50eb0c2512 Small improvements to the Taproot functional tests (Pieter Wuille)
fac865b72d test: Fix intermittent feature_taproot issue (MarcoFalke)
fa1dea19fc test: Fix deser issue in create_block (MarcoFalke)
fa762a3fd4 test: Remove unused unnamed parameter from block.serialize call (MarcoFalke)

Pull request description:

  This fixes three bugs. Also, fix some unrelated code style issues.

  Please refer to the commit messages for more information.

ACKs for top commit:
  laanwj:
    Code review ACK fab900802d

Tree-SHA512: 4e22c240cf345710f3b21fc63243126b90014b3656d0865ff87156e958dd1442e6572c6c0a5701dbbe503eee931a0ceb66eeeb3553137f3d1f5afd27a9f9cada
2020-11-09 15:47:04 +01:00
MarcoFalke
77777c8b5e
ci: Run windows ci config on cirrus 2020-11-09 10:16:45 +01:00
MarcoFalke
3333d6942e
ci: Run macos ci config on cirrus 2020-11-09 10:15:59 +01:00
MarcoFalke
fa0795f54d
ci: Replace TRAVIS_OS_NAME with CI_OS_NAME
Also enable free disk space check unconditionally
2020-11-09 10:15:48 +01:00
MarcoFalke
fafce1a13a
ci: Move documentation to correct config file 2020-11-09 10:15:13 +01:00
Anthony Towns
8ae9d314e9 Bump minimum python version to 3.6 2020-11-09 17:53:47 +10:00
MarcoFalke
fa2c3c0d96
ci: Set LC_ALL=C to allow running the s390x tests in qemu
Without the fix, the tests immediately abort:

terminate called after throwing an instance of 'std::runtime_error'
  what():  locale::facet::_S_create_c_locale name not valid
./qt/test/test_bitcoin-qt: line 2: 116150 Aborted
2020-11-05 12:54:51 +01:00
MarcoFalke
fab900802d
ci: Bump timeout factor
This avoids timeouts when signing a large raw transaction

https://cirrus-ci.com/task/5009228131729408?command=ci#L4981

test_framework.authproxy.JSONRPCException: 'signrawtransactionwithwallet' RPC took longer than 120.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
2020-11-03 19:44:57 +01:00
MarcoFalke
faf58ab139
ci: Add --with-libs=no to one ci config 2020-10-26 14:18:50 +01:00
Wladimir J. van der Laan
3caee16946
Merge #19953: Implement BIP 340-342 validation (Schnorr/taproot/tapscript)
0e2a5e448f tests: dumping and minimizing of script assets data (Pieter Wuille)
4567ba034c tests: add generic qa-asset-based script verification unit test (Pieter Wuille)
f06e6d0345 tests: functional tests for Schnorr/Taproot/Tapscript (Pieter Wuille)
3c226639eb tests: add BIP340 Schnorr signature support to test framework (Pieter Wuille)
206fb180ec --- [TAPROOT] Tests --- (Pieter Wuille)
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342) (Pieter Wuille)
e9a021d7e6 Make Taproot spends standard + policy limits (Pieter Wuille)
865d2c37e2 --- [TAPROOT] Regtest activation and policy --- (Pieter Wuille)
72422ce396 Implement Tapscript script validation rules (BIP 342) (Johnson Lau)
330de894a9 Use ScriptExecutionData to pass through annex hash (Pieter Wuille)
8bbed4b7ac Implement Taproot validation (BIP 341) (Pieter Wuille)
0664f5fe1f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340) (Pieter Wuille)
5de246ca81 Implement Taproot signature hashing (BIP 341) (Johnson Lau)
9eb590894f Add TaggedHash function (BIP 340) (Pieter Wuille)
450d2b2371 --- [TAPROOT] BIP340/341/342 consensus rules --- (Pieter Wuille)
5d62e3a68b refactor: keep spent outputs in PrecomputedTransactionData (Pieter Wuille)
8bd2b4e784 refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script (Pieter Wuille)
107b57df9f scripted-diff: put ECDSA in name of signature functions (Pieter Wuille)
f8c099e220 --- [TAPROOT] Refactors --- (Pieter Wuille)

Pull request description:

  This is an implementation of the Schnorr/taproot consensus rules proposed by BIPs [340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), [341](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), and [342](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki).

  See the list of commits [below](https://github.com/bitcoin/bitcoin/pull/19953#issuecomment-691815830). No signing or wallet support of any kind is included, as testing is done entirely through the Python test framework.

  This is a successor to https://github.com/bitcoin/bitcoin/pull/17977 (see discussion following [this comment](https://github.com/bitcoin/bitcoin/pull/17977#issuecomment-682285983)), and will have further changes squashed/rebased. The history of this PR can be found in #19997.

ACKs for top commit:
  instagibbs:
    reACK 0e2a5e448f
  benthecarman:
    reACK 0e2a5e4
  kallewoof:
    reACK 0e2a5e448f
  jonasnick:
    ACK 0e2a5e448f almost only looked at bip340/libsecp related code
  jonatack:
    ACK 0e2a5e448f modulo the last four commits (tests) that I plan to finish reviewing tomorrow
  fjahr:
    reACK 0e2a5e448f
  achow101:
    ACK 0e2a5e448f

Tree-SHA512: 1b00314450a2938a22bccbb4e177230cf08bd365d72055f9d526891f334b364c997e260c10bc19ca78440b6767712c9feea7faad9a1045dd51a5b96f7ca8146e
2020-10-15 10:22:35 +02:00
Andrew Chow
e87df82580 Add sqlite to travis and depends 2020-10-14 11:18:13 -04:00
Pieter Wuille
4567ba034c tests: add generic qa-asset-based script verification unit test
This adds a unit test that does generic script verification tests,
with positive/negative witnesses/scriptsigs, under various flags.
The test data is large (several MB) so it's stored in the qa-assets
repo.
2020-10-12 17:18:47 -07:00
Fabian Jahr
33df8d46bb
ci: Build Arm64 on Travis without functional tests 2020-10-03 19:20:04 +02:00
Hennadii Stepanov
0374e821bd
util: Hard code previous release tarball checksums 2020-08-29 11:28:53 +03:00
Hennadii Stepanov
bd897ce79f
scripted-diff: Move previous_release.py to test/get_previous_releases.py
-BEGIN VERIFY SCRIPT-
OLD=contrib/devtools/previous_release.py
NEW=test/get_previous_releases.py
sed -i "s|$OLD|$NEW|g" $(git grep -l $OLD)
git mv $OLD $NEW
-END VERIFY SCRIPT-
2020-08-29 11:26:25 +03:00
MarcoFalke
fa6583c30b
ci: Set increased --timeout-factor by default 2020-08-15 09:24:46 +02:00
Sjors Provoost
c17f54ee53
[ci] use boost::process
Explictly opt-out on win64, in case the default changes.
2020-07-31 13:38:10 +02:00
MarcoFalke
24ead1a923
Merge #19519: ci: Increase CCACHE_SIZE in some builds on Travis
fdf697fe75 ci: Increase CCACHE_SIZE in some builds on Travis (Hennadii Stepanov)

Pull request description:

  This PR will decrease build time for the longest Travis builds.
  Up to ~20 min improvement with warmed up cache is expected.

Top commit has no ACKs.

Tree-SHA512: ee3a22162d03537be11e60b53de043247f6a65e35e630e0807b758a846b7e05ef2059d18846644aafa3cd5dc7d3e3f56a8ccb94cc71b95227debc9201d3142a2
2020-07-28 10:25:11 +02:00
MarcoFalke
ea595d39f7
Merge #19205: script: previous_release.sh rewritten in python
9c34aff393 Remove previous_release.sh (Brian Liotti)
e1e5960e10 script: Add previous_release.py (Brian Liotti)

Pull request description:

  Closes #18132

  Added functionality:
  1) checks file hash before untarring when using the binary download option

ACKs for top commit:
  fjahr:
    re-ACK 9c34aff393
  Sjors:
    tACK 9c34aff393

Tree-SHA512: 323f11828736a372a47f048592de8b027ddcd75b38f312dfc73f7b495d1e078bfeb384d9cdf434b3e70f2c6c0ce2da2df48e9a6460ac0e1967c6829a411c52d5
2020-07-21 10:11:39 +02:00
Russell Yanofsky
0cdf2a77dd
ci: add tsan debug symbols option
https://github.com/bitcoin/bitcoin/issues/19049#issuecomment-639255156
2020-07-17 12:14:41 +03:00
Hennadii Stepanov
fdf697fe75
ci: Increase CCACHE_SIZE in some builds on Travis 2020-07-14 20:41:17 +03:00
Hennadii Stepanov
60824b3c3a
ci: Fix configure options for macOS builds 2020-07-14 02:37:15 +03:00
Hennadii Stepanov
557d3f1cc0
ci: Do not activate Travis ccache caching strategy
It is sufficient to specify CCACHE_DIR to cache.
Also this change fixes ccache on native macOS build.
2020-07-14 02:35:02 +03:00
Hennadii Stepanov
2d747428e2
ci: Disable functional tests on forked repos to avoid timeouts for macOS 2020-07-14 02:34:37 +03:00
Brian Liotti
9c34aff393 Remove previous_release.sh 2020-07-05 04:05:02 -04:00
MarcoFalke
7027c67cac
Merge #18288: build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory
870f0cd2a0 build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory (practicalswift)

Pull request description:

  Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory.

  First UBSan, then ASan followed by TSan... and now: yes, the wait is over -- **MSan is finally here!** :)

  Some historical context:
  * 2017: Continuous compilation with Clang Thread Safety analysis enabled (#10866, #10923)
  * 2018: Continuous testing with trapping on signed integer overflows (`-ftrapv`) (#12686)
  * 2018: Continuous testing of use of locale dependent functions (#13041)
  * 2018: Continuous testing of format strings (#13705)
  * 2018: Continuous compilation with MSVC `TreatWarningAsError` (#14151)
  * 2018: Continuous testing under UndefinedBehaviorSanitizer – UBSan (#14252, #14673, #17006)
  * 2018: Continuous testing under AddressSanitizer – ASan (#14794, #17205, #17674)
  * 2018: Continuous testing under ThreadSanitizer – TSan (#14829)
  * 2019: Continuous testing in an unsigned char environment (`-funsigned-char`) (#15134)
  * 2019: Continuous compile-time testing of assumptions we're making (#15391)
  * 2019: Continuous testing of fuzz test cases under Valgrind (#17633, #18159, #18166)
  * 2020: Finally... MemorySanitizer – MSAN! :)

  What is the next step? What tools should we add to CI to keep bugs from entering `master`? :)

ACKs for top commit:
  MarcoFalke:
    ACK 870f0cd2a0

Tree-SHA512: 38327c8b75679d97d469fe42e704cacd1217447a5a603701dd8a58ee50b3be2c10248f8d68a479ed081c0c4b254589d3081c9183f991640b06ef689061f75578
2020-07-02 10:26:50 -04:00
MarcoFalke
fa23fbb42f
ci: Run all tests on native mac again 2020-07-01 17:00:37 -04:00
MarcoFalke
8d3187fd0d
Merge #19412: test: move TEST_RUNNER_EXTRA into native tsan setup
a92e48b02d test: move TEST_RUNNER_EXTRA into native tsan setup (fanquake)

Pull request description:

  `feature_block.py` is being run in the tsan job, i.e [here](https://travis-ci.org/github/bitcoin/bitcoin/jobs/703122309), even though it should be excluded. My hasty assumption is that this will fix it. In any case, all other instances of `TEST_RUNNER_EXTRA` seem to have moved out of `.travis.yml` and into the different CI configurations.

ACKs for top commit:
  MarcoFalke:
    ACK a92e48b02d
  practicalswift:
    ACK a92e48b02d -- patch looks correct
  hebasto:
    ACK a92e48b02d, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 86057bef2cc87c6acdbbf94f8cd7a5147510448c3e67aacde8daf247e3ccf649cfc5afbbd10693e084f426042d98150616c0e49bfa5f32b949dff9cebd2fd95d
2020-06-29 12:47:51 -04:00
Wladimir J. van der Laan
dbadf746e2
Merge #19333: refactor: Fix clang compile failure
fa3b35a189 ci: Add test for clang-3.8 C++11 support (MarcoFalke)
faa7431fee refactor: Fix clang compile failure (MarcoFalke)

Pull request description:

  Fix

  ```
  script/standard.cpp:278:22: error: default initialization of an object of const type 'const (anonymous namespace)::CScriptVisitor' without a user-provided default constructor
  const CScriptVisitor g_script_visitor;
                       ^
                                       {}
  1 error generated.

ACKs for top commit:
  laanwj:
    ACK fa3b35a189

Tree-SHA512: b3251208945b44530224aadbc10fef1260b479c0b43a5e345501fbfd3579a9fe354b946090e023232852bbb99759da4429b58b137b7b286ddac6bd7960851f7f
2020-06-29 16:50:59 +02:00
fanquake
a92e48b02d
test: move TEST_RUNNER_EXTRA into native tsan setup 2020-06-29 22:00:27 +08:00
MarcoFalke
fa74a54fad
ci: Increase test timeout for sanitizer configs 2020-06-24 09:17:37 -04:00
practicalswift
870f0cd2a0 build: Add MemorySanitizer (MSan) in Travis to detect use of uninitialized memory 2020-06-23 09:09:30 +00:00
fanquake
80fd474e40
Merge #19240: build: macOS toolchain simplification and bump
adf543d714 darwin: pass mlinker-version so that clang enables new features (Cory Fields)
2418f739f7 macos: Bump to xcode 11.3.1 and 10.15 SDK (Cory Fields)
5c2c835433 depends: bump MacOS toolchain (Cory Fields)
85b5e42088 contrib: macdeploy: Remove historical extraction notes (Carl Dong)
351beb5c9a contrib: macdeploy: Use apple-sdk-tools instead of xar+pbzx (Carl Dong)
fbcfcf6954 native_cctools: Don't use libc++ from pinned clang (Carl Dong)
3381e4a189 Adapt rest of tooling to new SDK naming scheme (Carl Dong)
b3394ab235 contrib: macdeploy: Correctly generate macOS SDK (Carl Dong)

Pull request description:

  This PR achieves 3 main things:
  1. It simplifies the macOS SDK generation by putting the logic inside a (semi-)portable python3 script `gen-sdk`
  2. It transitions us to using `libc++` headers extracted from the `Xcode.app`, which is more correct as those headers better match the `.tbd` library stubs we use from the `MacOSX.sdk` (located under the same `Xcode.app`). Previously, we used `libc++` headers copied from our downloaded, pinned clang (see `native_cctools.mk`).
  3. It bumps the macOS toolchain in a way that fulfills all of the following constraints:
      1. The new SDK should support compiling with C++17 (our current one doesn't)
      2. The new toolchain should not change our minimum supported macOS version (`-mmacosx-version-min`)
      3. The new toolchain should expect to use a version of `cctools` that is supported by https://github.com/tpoechtrager/cctools-port

  For the constraints in (3), you can reference [this chart](https://en.wikipedia.org/wiki/Xcode#Xcode_7.0_-_11.x_(since_Free_On-Device_Development)) to see that the newest toolchain we can use with our `cctools-port` is `11.3.1`, and the rest of the constraints were tested with local builds.

  #### But [the other Wikipedia chart](https://en.wikipedia.org/wiki/Xcode#Xcode_11.x_(since_SwiftUI_framework)) says that the "min macOS to run" for Xcode 11.3.1 is 10.14.4, doesn't that violate constraint (ii)?

  This confused me at first too, but the "min macOS to run" is for the Xcode.app App itself. The SDK still supports 10.12, as evident in a few plist files and as proven through local builds.

  #### Why bundle all of this together in a single PR?

  We need (1) and (2) together, because if we don't, manually adding the `libc++` headers and writing that out in a `README.md` is going to result in a lot of user error, so it's great to have these together to be more correct and also make it easier on the user at the same time.

  We need (3) together with everything else because bumping (or in the case of (1), renaming) the SDK requires some human coordination and may break some builds. And since it's not that complicated a change, it makes sense to do it together with the rest.

ACKs for top commit:
  theuni:
    ACK adf543d714.
  fanquake:
    ACK adf543d714 - I'll take a look at the linker issue.

Tree-SHA512: 3813b69ebfe9610bee14100f26296fb5438d9bf0dd184ea55e6c38f5ebd94f7c171d98b150fc9e52fde626533f347f7ec51a2b72b79859d946284f578c1084a3
2020-06-23 16:14:49 +08:00
Cory Fields
2418f739f7
macos: Bump to xcode 11.3.1 and 10.15 SDK
This gets us a newer SDK with c++17 support and retains 10.12
back-compat.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-06-22 10:14:33 -04:00
Carl Dong
3381e4a189
Adapt rest of tooling to new SDK naming scheme 2020-06-22 10:13:55 -04:00
MarcoFalke
fa3b35a189
ci: Add test for clang-3.8 C++11 support 2020-06-19 20:33:53 -04:00
MarcoFalke
fa05f44893
ci: Upgrade most ci configs to focal 2020-06-19 10:44:11 -04:00
MarcoFalke
fa880773b4
ci: Have one config run in xenial to test against python3.5
Also, bump the travis env to bionic. This shouldn't matter at all
because all ci configs run inside a docker, but it does seem to fix a
bug. See
https://github.com/bitcoin/bitcoin/pull/19267#issuecomment-643630309
2020-06-19 10:43:21 -04:00
fanquake
968aaae940
tests: run test-security-check.py in CI 2020-06-16 19:52:30 +08:00
MarcoFalke
fa71667597
ci: Move travis workarounds to .travis.yml 2020-06-14 11:33:25 -04:00
MarcoFalke
aa35ea5502
Merge #19173: build: turn on --enable-c++17 by --enable-fuzz
0012471391 build: turn on --enable-c++17 by --enable-fuzz (Vasil Dimov)

Pull request description:

  Fuzzing code uses C++17 specific code (e.g. std::optional), so it is not
  possible to compile with --enable-fuzz and without --enable-c++17.

  Thus, turn on --enable-c++17 whenever --enable-fuzz is used.

ACKs for top commit:
  hebasto:
    ACK 0012471391, tested on Linux Mint 19.3 (x86_64); verified that it fails to compile with `--enable-fuzz` and without `--enable-c++17` on master.

Tree-SHA512: 290531ea8d79de3b9251ea4ad21e793478b18150cc0124eea1e50c3a4ed92bab89c3e70ed0aa526906f8723ea952cdba4268f1560ae4be9bd25b9e4f9b97436c
2020-06-05 07:15:44 -04:00
Vasil Dimov
0012471391
build: turn on --enable-c++17 by --enable-fuzz
Fuzzing code uses C++17 specific code (e.g. std::optional), so it is not
possible to compile with --enable-fuzz and without --enable-c++17.

Thus, turn on --enable-c++17 whenever --enable-fuzz is used.
2020-06-05 11:50:34 +02:00
MarcoFalke
fa7e002d52
ci: tsan with wallet 2020-06-04 18:26:01 -04:00
MarcoFalke
faf62e6ed0
ci: Remove unused workaround 2020-05-31 18:37:56 -04:00
MarcoFalke
fa7c850915
ci: Install llvm to get llvm symbolizer 2020-05-31 18:37:43 -04:00
MarcoFalke
fa0cc02c0a
ci: Mute depends logs completely 2020-05-30 08:33:44 -04:00
MarcoFalke
fa10d85079
ci: Use libc++ instead of libstdc++ for tsan 2020-05-30 08:33:02 -04:00
MarcoFalke
fa0d5ee112
ci: Set halt_on_error=1 for tsan 2020-05-30 08:32:58 -04:00
MarcoFalke
fa2ffe87f7
ci: Deduplicate DOCKER_EXEC 2020-05-30 08:32:33 -04:00
MarcoFalke
faf1c3cc58
test: Replace TEST_PREVIOUS_RELEASES env var with test_framework option 2020-05-21 07:47:46 -04:00
fanquake
97b21b302a
Merge #18677: Multiprocess build support
e2bab2aa16 multiprocess: add multiprocess travis configuration (Russell Yanofsky)
603fd6a2e7 depends: add MULTIPROCESS depends option (Russell Yanofsky)
5d1377b52b build: multiprocess autotools changes (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR consists of build changes only. It adds an `--enable-multiprocess` autoconf option (off by default and marked experimental), that builds new `bitcoin-node` and `bitcoin-gui` binaries. These currently function the same as existing `bitcoind` and `bitcoin-qt` binaries, but are extended in #10102 with IPC features to execute node, wallet, and gui functions in separate processes.

  In addition to adding the `--enable-multiprocess` config flag, it also adds a depends package and autoconf rules to build with the [libmultiprocess](https://github.com/chaincodelabs/libmultiprocess) library, and it adds new travis configuration to exercise the build code and run functional tests with the new binaries.

  The changes in this PR were originally part of #10102 but were moved into #16367 to be able to develop and review the multiprocess build changes independently of the code changes. #16367 was briefly merged and then reverted in #18588. Only change since #16367 has been dropping the `native_boost.mk` depends package which was pointed out to be no longer necessary in https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-596484337 and https://github.com/bitcoin/bitcoin/pull/18588#pullrequestreview-391765649

ACKs for top commit:
  practicalswift:
    ACK e2bab2aa16
  Sjors:
    tACK e2bab2aa16 on macOS 10.15.4
  hebasto:
    ACK e2bab2aa16, tested on Linux Mint 19.3 (x86_64):

Tree-SHA512: b5a76eab5abf63d9d8b6d628cbdff4cc1888eef15cafa0a5d56369e2f9d02595fed623f4b74b2cf2830c42c05a774f0943e700f9c768a82d9d348cad199e135c
2020-05-21 15:34:25 +08:00
MarcoFalke
faf552117e
ci: Set DEBIAN_FRONTEND=noninteractive
Also fix travis environment variables for this build

Previously the resulting env was:

FILE_ENV="./ci/test/00_setup_env_native_tsan.sh" TEST_RUNNER_EXTRA="--exclude feature_block" #= Not= enough= memory= on= travis= machines=
2020-05-18 20:38:56 -04:00
MarcoFalke
fa006caa13
ci: tsan on clang-9 2020-05-18 20:38:54 -04:00
Russell Yanofsky
e2bab2aa16 multiprocess: add multiprocess travis configuration 2020-05-12 09:47:06 -04:00
Russell Yanofsky
cbd661122e Set LD_LIBRARY_PATH consistently in travis tests
Remove inconsistency between functional and unit test environments and make it
possible to substitute bitcoin-qt and bitcoin-node in place of bitcoind in
python tests, or to link bitcoind against shared libraries.
2020-05-10 10:35:17 -04:00
MarcoFalke
fa35c34df7
Remove unused ci configs that have been moved elsewhere
They have been moved to https://github.com/MarcoFalke/btc_nightly
running on Cirrus CI https://cirrus-ci.com/build/6249975761862656
2020-05-10 07:51:31 -04:00
MarcoFalke
3333cb9699
fuzz: Pass down MAKEJOBS to test_runner 2020-05-10 07:49:09 -04:00
Harris
420fa0770f
fuzz: use std::optional for sep_pos variable
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-05-09 11:09:52 +02:00
MarcoFalke
3930014abc
Merge #18864: Add v0.16.3 backwards compatibility test, bump v0.19.0.1 to v0.19.1
d135c29476 [ci] make list of previous releases to download a setting (Sjors Provoost)
9c246b873c [test] backwards compatibility: bump v0.19.0.1 to v0.19.1 (Sjors Provoost)
89a28e02fa [test] add v0.16.3 backwards compatibility test (Sjors Provoost)

Pull request description:

  Thanks to #18774's `adjust_bitcoin_conf_for_pre_17` we can now test backwards compatibility for v0.16.3, both for sync and loading a recent wallet.

  This PR bumps v0.19.0.1 to v0.19.1.

  I also made the version list consistent for the `contrib/devtools/previous_release.sh` instruction, between both tests.

ACKs for top commit:
  MarcoFalke:
    ACK d135c29476

Tree-SHA512: 5ff137a7a934237fa220f1c2807ce9abeeb75929266558bf3e4045bec7dfcd0a8747fa74d700065c568330b18badf58c60c308eb13d1eed444d4bbfe6decc48b
2020-05-08 11:23:40 -04:00
Sjors Provoost
d135c29476
[ci] make list of previous releases to download a setting
Co-Authored-By: MarcoFalke <falke.marco@gmail.com>
2020-05-08 16:13:29 +02:00
MarcoFalke
8c705ff129 travis: Remove s390x 2020-05-07 09:58:57 -04:00
Sjors Provoost
9c246b873c
[test] backwards compatibility: bump v0.19.0.1 to v0.19.1 2020-05-04 13:36:40 +02:00
MarcoFalke
ddc0a600b3
Merge #18617: test: add factor option to adjust test timeouts
2742c34286 test: add factor option to adjust test timeouts (Harris)

Pull request description:

  This PR adds a new option **factor** that can be used to adjust timeouts in various functional tests.
  Several timeouts and functions from `authproxy`, `mininode`, `test_node` and `util` have been adapted to use this option. The factor-option definition is located in `test_framework.py`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18266
  Also Fixes https://github.com/bitcoin/bitcoin/issues/18834

ACKs for top commit:
  MarcoFalke:
    Thanks! ACK 2742c34286

Tree-SHA512: 6d8421933ba2ac1b7db00b70cf2bc242d9842c48121c11aadc30b0985c4a174c86a127d6402d0cd73b993559d60d4f747872d21f9510cf4806e008349780d3ef
2020-05-03 08:58:56 -04:00
Harris
2742c34286
test: add factor option to adjust test timeouts
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2020-05-03 01:42:40 +02:00
MarcoFalke
aaaacff107
ci: Merge C++17 build with one of the existing ones 2020-04-30 12:50:36 -04:00
Wladimir J. van der Laan
35ef3c15ef
Merge #18591: Add C++17 build to Travis
c31cbe7cfe Add C++17 test to Travis (Pieter Wuille)
7829685e27 Add configure option for c++17 (Pieter Wuille)
0fbde488b2 Support conversion between Spans of compatible types (Pieter Wuille)
7cbfebbf3d Update ax_cxx_compile_stdcxx.m4 (Pieter Wuille)

Pull request description:

  This adds a `--enable-c++17` option to the configure script, fixes the only C++17 incompatibility (with a commit taken from #18468), and adds a Travis test for it.

  This is all off by default, and release builds remain C++11.

  It implements the first step of the plan in https://github.com/bitcoin/bitcoin/issues/16684.

ACKs for top commit:
  elichai:
    tACK c31cbe7cfe
  practicalswift:
    Tested ACK c31cbe7cfe
  hebasto:
    ACK c31cbe7cfe, tested on Linux Mint 19.3 both C++11 and C++17 modes. Compiled and passed tests locally.

Tree-SHA512: a4b00776dbceef9c12abbb404c6bcd48f7916ce24c8c7a14116355f64e817578b7fcddbedd5ce435322319d1e4de43429b68553f4d96d970c308fe3e3e59b9d1
2020-04-30 11:16:56 +02:00
MarcoFalke
ab91a0e0fc
Merge #18798: ci: Fix default retry script usage
45615de26c ci: Fix default retry script usage (Hennadii Stepanov)

Pull request description:

  On master (5352d14b37) `CI_RETRY_EXE=${CI_RETRY_EXE:retry}` works as a [Substring Expansion](https://www.gnu.org/software/bash/manual/html_node/Shell-Parameter-Expansion.html), and that is wrong.
  If `CI_RETRY_EXE` variable was unset initially, its new value becomes an empty string, but not "retry" as one could expect. Consequently, the `${CI_RETRY_EXE} ...` command does _not_ use `ci/retry/retry` script.

  This PR makes for `CI_RETRY_EXE` variable a usual parameter expansion, i.e., `${parameter:-word}`.

  Reference: https://github.com/bitcoin/bitcoin/pull/18735#issuecomment-620095489

Top commit has no ACKs.

Tree-SHA512: 108173f6b2677979b9ddf2f9b9df4a6c56f5efa81c36543a1816bb3b984e42984bf3c83fe413ea3a5ca1e2317c4efb02fea7180a6b44863af7cfe6202e9cf94d
2020-04-29 11:14:14 -04:00
MarcoFalke
e302830fae
Merge #18774: test: added test for upgradewallet RPC
66fe7b1a98 test: added test for upgradewallet RPC (Harris)

Pull request description:

  This PR adds tests for the newly merged *upgradewallet* RPC.

  Additionally, it expands `test_framework/util.py` by adding the function `adjust_bitcoin_conf_for_pre_17` to support nodes that don't parse configuration sections.

  This test uses two older node versions, v0.15.2 and v0.16.3, to create older wallet versions to be used by `upgradewallet`.

  Fixes https://github.com/bitcoin/bitcoin/issues/18767

Top commit has no ACKs.

Tree-SHA512: bb72ff1e829e2c3954386cc308842820ef0828a4fbb754202b225a8748f92d4dcc5ec77fb146bfd5484a5c2f29ce95adf9f3fb4483437088ff3ea4a8d2c442c1
2020-04-29 11:09:05 -04:00
Harris
66fe7b1a98
test: added test for upgradewallet RPC 2020-04-29 16:11:49 +02:00
MarcoFalke
a483ffd689
Merge #18735: ci: Add and document BASE_BUILD_DIR
fae49f6e42 ci: Add and document BASE_BUILD_DIR (MarcoFalke)

Pull request description:

  Also fixes #18768

ACKs for top commit:
  hebasto:
    re-ACK fae49f6e42, which is essentially the same as the previously [reviewed changes](https://github.com/bitcoin/bitcoin/pull/18735#pullrequestreview-400581536).

Tree-SHA512: 216565a05ccd513dd9f114b2333d3c283fd71914d32f9b05f145cb7c70633b083ff8ef60798d6f22f4be6a4d652b03806551fd74b5b596c92968501a4d9726d2
2020-04-28 11:50:21 -04:00
MarcoFalke
fa72a75102
ci: Document why tests can not be run on mac 2020-04-28 07:10:19 -04:00
Hennadii Stepanov
45615de26c
ci: Fix default retry script usage 2020-04-28 10:52:31 +03:00
MarcoFalke
fae49f6e42
ci: Add and document BASE_BUILD_DIR
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2020-04-27 12:27:34 -04:00
MarcoFalke
fac24dea00
ci: Run functional tests on mac again 2020-04-26 21:03:16 -04:00
MarcoFalke
faebcd4e8d
ci: Remove xenial tsan workaround 2020-04-19 08:52:49 -04:00
MarcoFalke
d7bdba460c
Merge #18683: ci: Disable valgrind functionl tests on forked repos to avoid timeouts
faceeae49a ci: Disable valgrind functionl tests on forked repos to avoid timeouts (MarcoFalke)

Pull request description:

  Allows people to fork our repo and run the tests again

  Also print more cache stats

ACKs for top commit:
  hebasto:
    ACK faceeae49a, tested on my own repo: https://travis-ci.org/github/hebasto/bitcoin/jobs/676257500

Tree-SHA512: 50e44edf94fcb997438eeaf7308b2b58a0854141ecb1fbb0ba7bf5ed662f19b60899f966f579cca90ad5e789234d0e90122d8c2c854da70339058adc3a475fa6
2020-04-17 14:00:04 -04:00
MarcoFalke
54f812d9d2
Merge #18673: scripted-diff: Sort test includes
fa4632c417 test: Move boost/stdlib includes last (MarcoFalke)
fa488f131f scripted-diff: Bump copyright headers (MarcoFalke)
fac5c37300 scripted-diff: Sort test includes (MarcoFalke)

Pull request description:

  When writing tests, often includes need to be added or removed. Currently the list of includes is not sorted, so developers that write tests and have `clang-format` installed will either have an unrelated change (sorting) included in their commit or they will have to manually undo the sort.

  This pull preempts both issues by just sorting all includes in one commit.

  Please be aware that this is **NOT** a change to policy to enforce clang-format or any other developer guideline or process. Developers are free to use whatever tool they want, see also #18651.

  Edit: Also includes a commit to bump the copyright headers, so that the touched files don't need to be touched again for that.

ACKs for top commit:
  practicalswift:
    ACK fa4632c417
  jonatack:
    ACK fa4632c417, light review and sanity checks with gcc build and clang fuzz build

Tree-SHA512: 130a8d073a379ba556b1e64104d37c46b671425c0aef0ed725fd60156a95e8dc83fb6f0b5330b2f8152cf5daaf3983b4aca5e75812598f2626c39fd12b88b180
2020-04-17 10:12:13 -04:00
MarcoFalke
faceeae49a
ci: Disable valgrind functionl tests on forked repos to avoid timeouts 2020-04-17 09:00:43 -04:00
MarcoFalke
fa488f131f
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-04-16 13:33:09 -04:00
Hennadii Stepanov
0c6318788b
ci: Limit cache size regardless of NO_DEPENDS 2020-04-16 17:56:21 +03:00
Pieter Wuille
c31cbe7cfe Add C++17 test to Travis 2020-04-11 11:30:27 -07:00
MarcoFalke
1b30761360
Merge #16367: Multiprocess build support
b919efadff depends: Use default macos clang compiler (Russell Yanofsky)
d54f64c6c7 Add multiprocess travis configuration (Russell Yanofsky)
787f40668d Set LD_LIBRARY_PATH consistently in travis tests (Russell Yanofsky)
d630646662 libmultiprocess depends build (Russell Yanofsky)
e6e44eedd5 Multiprocess build changes (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  This splits autotools, depends build, and travis changes out of #10102, so code changes and build system changes can be reviewed separately.

ACKs for top commit:
  hebasto:
    re-ACK b919efadff, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/16367#issuecomment-605514556) review.

Tree-SHA512: ebc5e403cc99a0d9629ed7fe1595e01d57e6d1255cbf03968a3196ff6f528f734c78060fdc065724ee1f923bcc5aa2b29470fcb36a7f15957eb57c76d58178a4
2020-04-10 12:55:12 -04:00
Hennadii Stepanov
6d480667b2
ci: Print ccache statistics summary 2020-04-10 19:54:32 +03:00
Hennadii Stepanov
a6eea715ff
ci: Drop unused WINEDEBUG variable 2020-04-10 19:51:21 +03:00
MarcoFalke
1ae366ecb0
Merge #18569: ci: Add big endian native s390x build
6136a96cdf ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST (Hennadii Stepanov)
97ba77aa8e ci: Add native s390x (Hennadii Stepanov)

Pull request description:

  Unlike the Docker wrapped solution (#17591) this PR suggests running on host system directly.

  This approach makes builds quick and stable (see: #18106).

  The excerpt from the Travis log:
  ```
  ...
  Running on host system without docker wrapper
  ...
  Byte Order: Big Endian
  ...
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 6136a96cdf

Tree-SHA512: 1b591de13e38d10a35217e1de11cbd648a359d18d16eed166fac18ea5788b58cc9fc6d407086ed342b99e57e479efd951a0ea693710177e500eb116316b9a788
2020-04-09 22:19:22 +08:00
Hennadii Stepanov
6136a96cdf
ci: Rename RUN_CI_ON_HOST to DANGER_RUN_CI_ON_HOST 2020-04-09 02:59:37 +03:00
Hennadii Stepanov
97ba77aa8e
ci: Add native s390x 2020-04-09 02:52:52 +03:00
MarcoFalke
fa7af33b4c
ci: Run unit tests sequential once 2020-04-08 22:16:55 +08:00
Russell Yanofsky
d54f64c6c7 Add multiprocess travis configuration 2020-04-05 21:48:21 -04:00
Russell Yanofsky
787f40668d Set LD_LIBRARY_PATH consistently in travis tests
Remove inconsistency between functional and unit test environments and make it
possible to substitute bitcoin-qt and bitcoin-node in place of bitcoind in
python tests, or to link bitcoind against shared libraries.
2020-04-05 21:48:21 -04:00
MarcoFalke
faa9491870
ci: Use Focal for fuzzers 2020-03-30 18:57:59 -04:00
Hennadii Stepanov
25c8b73656
ci: Use Homebrew addon on native macOS
Also the macOS image has been updated.
2020-03-26 14:57:14 +02:00
Hennadii Stepanov
596c627a1e
ci: Fix brew in Travis 2020-03-26 14:56:43 +02:00
MarcoFalke
fae1e99289
ci: Only clone bitcoin-core/qa-assets when fuzzing 2020-03-25 11:10:18 -04:00
MarcoFalke
fa92af5af3
ci: Run feature_block and feature_abortnode in valgrind 2020-03-22 10:12:40 -04:00
MarcoFalke
16dfaed1b7
Merge #18304: ci: Enable all functional tests in valgrind
4444edc2e6 ci: Enable all functional tests in valgrind (MarcoFalke)

Pull request description:

  The travis timeout for our repo has been bumped to 2h, so we can run all tests in valgrind now

ACKs for top commit:
  practicalswift:
    ACK 4444edc2e6 -- regarding the three disabled cases (`feature_abortnode`, `feature_block` and `rpc_bind`): not a big deal since MSan will take care of those once #18288 is merged. More is more :)

Tree-SHA512: ea2f798112911b6d1f3d88cfcdf0a7cdb698687248343703d6fe55da144542c961c15d888bffb41672c10aa76765615cb7c7ff93d468bfad3c51f962f24e7abb
2020-03-11 08:39:09 -04:00
MarcoFalke
4444edc2e6
ci: Enable all functional tests in valgrind 2020-03-10 18:38:42 -04:00
fanquake
e21fa542b1
test: remove Boost Chrono installation from CI 2020-03-07 08:34:31 +08:00
Ben Woosley
9b0e16226e
doc: Correct spelling errors in comments
And ci script output.

Identified via test/lint/lint-spelling
2020-03-02 23:07:21 -08:00
practicalswift
12a2f37718 util: Avoid potential uninitialized read in FormatISO8601DateTime(int64_t nTime) by checking gmtime_s/gmtime_r return value 2020-02-19 22:41:06 +00:00
practicalswift
733bbec34f tests: Add --exclude integer,parse_iso8601 (temporarily) to make Travis pass until uninitialized read issue in FormatISO8601DateTime is fixed 2020-02-19 14:11:54 +00:00
practicalswift
a3b539a924 ci: Run fuzz testing test cases under valgrind 2020-02-18 06:56:26 +00:00
Sjors Provoost
c98c26ee99
ci: use --enable-werror on more hosts 2020-02-14 10:40:23 +01:00
Sjors Provoost
c456145b2c
[test] add 0.19 backwards compatibility tests 2020-02-11 21:46:23 +01:00
Sjors Provoost
8b1460dbd1
[tests] check v0.17.1 and v0.18.1 backwards compatibility 2020-02-11 21:46:21 +01:00
fanquake
9e111db088
test: set a name for CI Docker containers 2020-02-10 20:03:52 +08:00
fanquake
7e2104433c
build: use macOS 10.14 SDK
Co-Authored-By: Carl Dong <accounts@carldong.me>
2020-02-03 19:49:46 +08:00
Wladimir J. van der Laan
4a07233076
Merge #17767: ci: Fix qemu issues
fa569e1a9c ci: Set LC_ALL=C when running in qemu-s390x (MarcoFalke)
fa3d77623e ci: Use debian to avoid apt install 404 errors (MarcoFalke)
fabb946090 ci: Install needed gcc and qemu-user iff cross-compiling (MarcoFalke)
faba4672b6 ci: Fix QEMU_USER_CMD parse issues (MarcoFalke)
fa5d709fb2 ci: Move wrap-qemu into separate script (MarcoFalke)

Pull request description:

  Now that the ci system no longer propagates files from the docker container back to the host, the wrap-qemu script is broken.

  To fix it, every statement in the script needs to be executed in the docker (with `DOCKER_EXEC`). Instead of juggling with triple escape sequences like `\\\"`, just move the script to a separate file and call it with `DOCKER_EXEC`.

  Also, fix a bunch of other bugs that prevent running the ci system in qemu

  See the `ci/README.md` on how to test. TLDR: Can be tested with (replace `arm` with `s390x` to run the s390x build):

  ```
  FILE_ENV="./ci/test/00_setup_env_arm.sh" MAKEJOBS="-j9" ./ci/test_run_all.sh
  ```

ACKs for top commit:
  laanwj:
    Code review ACK fa569e1a9c

Tree-SHA512: 84ebc44a4f0261ee6c29605a6896a1833ff6c81d729e6d08dd111941f570ce73221422bd3303e1108a266ec5eab2148bd5ee1cf6bc01477d8cc9a6c5bf2b34c2
2020-01-22 21:09:13 +01:00
MarcoFalke
2aaeca50b2
Merge #17900: ci: Combine 32-bit build with CentOS 7 build
ef63f5fc11 ci: Combine 32-bit build with CentOS 7 build (Sebastian Falbesoner)

Pull request description:

  Combines the CentOS build with the 32-bit (i686) build to avoid Travis bottlenecks, as suggested in #17757 by MarcoFalke. This keeps most of the properties of the 32-bit build (dash as config shell, building QT5 GUI) and just builds it with depends inside the CentOS docker container.

  Making the depends in `05_before_script.sh` with unset config shell (`CONFIG_SHELL=`)

  6196e93001/ci/test/05_before_script.sh (L28)

  caused problems for building the library libevent (resulting in a Makefile with no shell set (`SHELL=`)), that's why I set it explicitely to `/bin/bash` if we have a CentOS Docker container.

  A Travis output of this 32-bit CentOS build can be seen here: https://travis-ci.org/theStack/bitcoin/jobs/634472394 (has been restarted once due to too long build time and appearance of the `CACHE_ERR_MSG`).

  For anyone wanting to verify the outputs, I found these instructions useful to reproduce a Travis build locally: https://github.com/erdc/proteus/wiki/Replicating-the-TravisCI-Environment-on-your-Local-Machine (steps 1-3). In this case it's a bit tricky since you run Docker inside Docker -- within the Travis Docker container, the CentOS Docker container is created. To make this possible, the Docker socket has to be exposed to the Travis container via bind-mounting (`docker run -v /var/run/docker.sock:/var/run/docker.sock ...`), as suggested in https://stackoverflow.com/a/33003273.

Top commit has no ACKs.

Tree-SHA512: af508241cec3a10a66c37673d56691717b78375340e910fcdd3fb3870741eba623a436e1e85b26b54f013375611896f5411c5a7fec2437d367d27172230129fe
2020-01-16 15:45:24 -05:00
MarcoFalke
e09c701e01 scripted-diff: Bump copyright of files changed in 2020
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-01-15 02:18:00 +07:00
Sebastian Falbesoner
ef63f5fc11 ci: Combine 32-bit build with CentOS 7 build 2020-01-08 23:36:46 +01:00
MarcoFalke
fa569e1a9c
ci: Set LC_ALL=C when running in qemu-s390x
This avoids std::runtime_error in the unit tests of the type
"locale::facet::_S_create_c_locale name not valid"
2020-01-03 07:07:11 -05:00
MarcoFalke
fa3d77623e
ci: Use debian to avoid apt install 404 errors
The default ubuntu mirror does not have s390x or armhf packages:

Get:1 http://security.ubuntu.com/ubuntu bionic-security InRelease [88.7 kB]
Get:2 http://archive.ubuntu.com/ubuntu bionic InRelease [242 kB]
Get:3 http://archive.ubuntu.com/ubuntu bionic-updates InRelease [88.7 kB]
Ign:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
Get:5 http://archive.ubuntu.com/ubuntu bionic-backports InRelease [74.6 kB]
Get:6 http://security.ubuntu.com/ubuntu bionic-security/restricted amd64 Packages [19.2 kB]
Get:7 http://security.ubuntu.com/ubuntu bionic-security/multiverse amd64 Packages [6781 B]
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Get:9 http://security.ubuntu.com/ubuntu bionic-security/main amd64 Packages [761 kB]
Ign:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Get:12 http://archive.ubuntu.com/ubuntu bionic/restricted amd64 Packages [13.5 kB]
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Get:14 http://archive.ubuntu.com/ubuntu bionic/main amd64 Packages [1344 kB]
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Get:17 http://security.ubuntu.com/ubuntu bionic-security/universe amd64 Packages [795 kB]
Get:18 http://archive.ubuntu.com/ubuntu bionic/multiverse amd64 Packages [186 kB]
Get:19 http://archive.ubuntu.com/ubuntu bionic/universe amd64 Packages [11.3 MB]
Ign:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Ign:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Err:4 http://security.ubuntu.com/ubuntu bionic-security/multiverse armhf Packages
  404  Not Found [IP: 91.189.88.174 80]
Ign:8 http://security.ubuntu.com/ubuntu bionic-security/universe armhf Packages
Ign:15 http://security.ubuntu.com/ubuntu bionic-security/main armhf Packages
Ign:16 http://security.ubuntu.com/ubuntu bionic-security/restricted armhf Packages
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Ign:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Get:21 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 Packages [1057 kB]
Ign:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
Get:23 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse amd64 Packages [10.5 kB]
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Get:26 http://archive.ubuntu.com/ubuntu bionic-updates/universe amd64 Packages [1322 kB]
Get:27 http://archive.ubuntu.com/ubuntu bionic-updates/restricted amd64 Packages [32.7 kB]
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Ign:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
Get:30 http://archive.ubuntu.com/ubuntu bionic-backports/universe amd64 Packages [4244 B]
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Get:32 http://archive.ubuntu.com/ubuntu bionic-backports/main amd64 Packages [2496 B]
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Ign:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Ign:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Ign:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Err:10 http://archive.ubuntu.com/ubuntu bionic/main armhf Packages
  404  Not Found [IP: 91.189.88.149 80]
Ign:11 http://archive.ubuntu.com/ubuntu bionic/restricted armhf Packages
Ign:13 http://archive.ubuntu.com/ubuntu bionic/multiverse armhf Packages
Ign:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Ign:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Ign:20 http://archive.ubuntu.com/ubuntu bionic/universe armhf Packages
Err:22 http://archive.ubuntu.com/ubuntu bionic-updates/restricted armhf Packages
  404  Not Found [IP: 91.189.88.149 80]
Ign:24 http://archive.ubuntu.com/ubuntu bionic-updates/main armhf Packages
Ign:25 http://archive.ubuntu.com/ubuntu bionic-updates/universe armhf Packages
Ign:28 http://archive.ubuntu.com/ubuntu bionic-updates/multiverse armhf Packages
Err:29 http://archive.ubuntu.com/ubuntu bionic-backports/main armhf Packages
  404  Not Found [IP: 91.189.88.149 80]
Ign:31 http://archive.ubuntu.com/ubuntu bionic-backports/universe armhf Packages
Fetched 17.4 MB in 2s (7076 kB/s)
Reading package lists...
E: Failed to fetch http://security.ubuntu.com/ubuntu/dists/bionic-security/multiverse/binary-armhf/Packages  404  Not Found [IP: 91.189.88.174 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic/main/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-updates/restricted/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
E: Failed to fetch http://archive.ubuntu.com/ubuntu/dists/bionic-backports/main/binary-armhf/Packages  404  Not Found [IP: 91.189.88.149 80]
E: Some index files failed to download. They have been ignored, or old ones used instead.
2020-01-03 07:06:12 -05:00
MarcoFalke
fabb946090
ci: Install needed gcc and qemu-user iff cross-compiling 2020-01-03 07:05:39 -05:00
MarcoFalke
faba4672b6
ci: Fix QEMU_USER_CMD parse issues
* Fix bash syntax by adding missing -

* Disable QEMU_USER_CMD fallback when it is set

  Apparently bash can't distinguish an unset variable from a variable
  that is set to the empty string

* Export the environment variable to the docker env, otherwise it
  couldn't be used there
2020-01-03 07:04:45 -05:00
MarcoFalke
fa5d709fb2
ci: Move wrap-qemu into separate script 2020-01-03 07:03:46 -05:00
Hennadii Stepanov
87744b16b0
ci: Fix brew python link 2020-01-02 18:08:10 +02:00
MarcoFalke
aaaaad6ac9
scripted-diff: Bump copyright of files changed in 2019
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2019-12-30 10:42:20 +13:00
MarcoFalke
890eac8f82
Merge #17725: ci: Add valgrind run
facb416ad5 ci: Add valgrind run (MarcoFalke)

Pull request description:

  Fixes #17460

ACKs for top commit:
  practicalswift:
    ACK facb416ad5

Tree-SHA512: 55396e548a76f976d7b7170b68bc5f93cfd44656162267172f66db7eb549699a2a22d3b1bb0d5f180fe0697931939e652c8cdb86b435e81e7ce572485798009d
2019-12-17 11:46:22 -05:00
Elichai Turkel
e190000869
ci-s390x: Add qemu and depends support in the ci script 2019-12-14 18:27:13 +02:00
MarcoFalke
facb416ad5
ci: Add valgrind run 2019-12-10 19:37:37 -05:00
MarcoFalke
fc0da027e8
Merge #17635: ci: Add CentOS 7 build
711e0449cf ci: Remove trusty build (Hennadii Stepanov)
7f3ae22468 ci: Add CentOS 7 build (Hennadii Stepanov)

Pull request description:

  Arguably, CentOS is the most conservative distro of all the popular ones. Thus, it could be a good way to check the Bitcoin Core compatibility with aged dependencies.

  Currently, CentOS 7 has:
  - Berkeley DB == 4.8.30
  - Boost == 1.53.0
  - GCC == 4.8.5
  - libevent == 2.0.21 < minimum required [2.0.22](https://github.com/bitcoin/bitcoin/blob/master/doc/dependencies.md), but tests passed
  - MiniUPnPc == 2.0
  - Python == 3.6.8
  - qrencode == 3.4.1
  - Qt == 5.9.7
  - ZeroMQ == 4.1.4

  ~Please note that this PR is based on the bugfix #17634.~

  Also trusty build has been removed for the following reasons:
  - https://github.com/bitcoin/bitcoin/issues/17628#issuecomment-559448201:
  > Maybe it'd make sense to replace Ubuntu Trusty with Centos 7 as the "check ancient backward compatibililty" Travis run. It's supported until 2024, apparently.

  - https://github.com/bitcoin/bitcoin/pull/17635#discussion_r354811792:
  > Our travis is currently running at its limit and this doesn't seem like it is adding a lot new coverage compared to the other builds.

  Close #17628

ACKs for top commit:
  MarcoFalke:
    ACK 711e0449cf 🚠

Tree-SHA512: 614ec8394943f482a5867067f7119bffd052924a51e32ffda9a08e10c392c4a955a3539e2f8907cb65bfd9347dadf0ba62f6d1530bbc49927c347360a5a7f73c
2019-12-06 23:32:21 -05:00
Hennadii Stepanov
711e0449cf
ci: Remove trusty build
Co-authored-by: Emil Engler <me@emilengler.com>
2019-12-06 21:54:37 +02:00
Hennadii Stepanov
7f3ae22468
ci: Add CentOS 7 build 2019-12-06 21:44:35 +02:00
practicalswift
1f9d5af4f1 tests: Add initialization order fiasco detection in Travis 2019-12-05 10:54:18 +00:00
MarcoFalke
fa40e48c50
ci: Remove unparseable lines from supp file for old xenial clang tsan 2019-12-04 12:16:52 -05:00
MarcoFalke
fa1bfc476c
ci: ubsan report_error_type=1 and add suppressions 2019-12-04 12:16:38 -05:00
MarcoFalke
faa8023ce9
ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le 2019-12-04 09:32:15 -05:00
MarcoFalke
1705f1949a
Merge #17549: ci: misc cleanups
fad88e6f86 ci: Remove use of cd (MarcoFalke)
fa2941bbf4 ci: Remove unmaintained extended_lint (MarcoFalke)
fa04187528 scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR (MarcoFalke)
fa0656d1eb ci: Add DEPENDS_DIR variable, Add documentation for folders (MarcoFalke)
faeeca87b6 scripted-diff: Move various folders to ci scratch dir (MarcoFalke)

Pull request description:

  Some minor cleanups:

  * Remove unused and unmaintained extended_lint to avoid ci bottlenecks and waste of CPU
  * Move all folders that hold temporary ci files to the ci scratch dir (except for the build dirs)
  * Add some documentation to folders and remove the `BASE_BUILD_DIR` alias for the root directory
  * Fixes #17178 by removing `cd`

Top commit has no ACKs.

Tree-SHA512: f6eb9d47ab98c08af59a79c2a4bf62fc331f0f6f3174909a28f2c2f48b0234423da5aee876225d2e188619b71b008d882a20d29a7aca68248039ea5080be5af5
2019-12-03 11:02:06 -05:00
MarcoFalke
fabd71076c
ci: Print free disk space 2019-11-25 16:23:36 -05:00
MarcoFalke
fa94fc10c8
ci: Run functional tests on s390x 2019-11-25 15:07:28 -05:00
Elichai Turkel
2fa65e0de9
Add ci script to install on s390x 2019-11-25 14:34:47 +02:00
MarcoFalke
fad88e6f86
ci: Remove use of cd 2019-11-22 13:57:55 -05:00
fanquake
7d7bf2ff4a
build: set minimum supported macOS to 10.12 2019-11-21 11:08:47 -05:00
MarcoFalke
fa04187528
scripted-diff: Use ci DEPENDS_DIR, remove BASE_BUILD_DIR
-BEGIN VERIFY SCRIPT-
 # Use ci DEPENDS_DIR
 sed -i -e 's|BASE_BUILD_DIR/depends|DEPENDS_DIR|g' $(git grep -l depends ci)
 sed -i -e 's| depends/| ${DEPENDS_DIR}/|g'         $(git grep -l depends ci)

 # Remove redundant alias
 sed -i -e 's|BASE_BUILD_DIR|BASE_ROOT_DIR|g'       $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-21 11:01:34 -05:00
MarcoFalke
fa0656d1eb
ci: Add DEPENDS_DIR variable, Add documentation for folders 2019-11-21 11:01:17 -05:00
MarcoFalke
faeeca87b6
scripted-diff: Move various folders to ci scratch dir
-BEGIN VERIFY SCRIPT-
 # move ci sanitizer-output
 sed -i -e 's|BASE_BUILD_DIR}/sanitizer-output|BASE_SCRATCH_DIR}/sanitizer-output|g' $(git grep -l BASE_BUILD_DIR ci)
 # move qa-assets
 sed -i -e 's|BASE_BUILD_DIR}/qa-assets|BASE_SCRATCH_DIR}/qa-assets|g'               $(git grep -l BASE_BUILD_DIR ci)
 # move out dir
 sed -i -e 's|BASE_BUILD_DIR/out|BASE_SCRATCH_DIR/out|g'                             $(git grep -l BASE_BUILD_DIR ci)
-END VERIFY SCRIPT-
2019-11-20 19:56:26 -05:00
MarcoFalke
41162c0ba0
Merge #17423: ci: Make ci system read-only on the git work tree
fa7523d3aa ci: Extend docs (MarcoFalke)
fa493ef088 ci: Make ci system read-only on the git work tree (MarcoFalke)
fab1333292 ci: Remove git from required packages on host (MarcoFalke)
fa00393bce ci: Make all filesystem operations inside docker (MarcoFalke)

Pull request description:

  Running the ci completely in a docker, without leaving any traces on the host system is not possible right now because the ccache and depends dir needs to be propagated back and picked up by the host for caching.

  Fixes #17372

ACKs for top commit:
  JeremyRubin:
    tested ACK fa7523d3aa

Tree-SHA512: 4bce1a0f883bcbdb34abf409bdbc80d420c5da2045d2f9c5536ac433f9e5b490f23df084546c8c049f688b487572bbfc4f9c4029e9e672f4d9279739d066ed2e
2019-11-20 16:41:55 -05:00
fanquake
76e777df83
Merge #16161: util: Fix compilation errors in support/lockedpool.cpp
30fb598737 Fix segfault in allocator_tests/arena_tests (Jeffrey Czyz)
15c84f53f4 Define ARENA_DEBUG in Travis test runs (Jeffrey Czyz)
ad71548822 Fix compilation errors in support/lockedpool.cpp (Jeffrey Czyz)

Pull request description:

  Changes in #12048 cause a compilation error in Arena::walk() when
  ARENA_DEBUG is defined. Specifically, Arena's chunks_free map was
  changed to have a different value type.

  Additionally, missing includes cause other compilation errors when
  ARENA_DEBUG is defined.

  Reproduced with:

  make CPPFLAGS=-DARENA_DEBUG

ACKs for top commit:
  laanwj:
    ACK 30fb598737
  fanquake:
    ACK 30fb598737 - thanks for following up jkczyz.

Tree-SHA512: 4eec368a4e9c67e4e2a27bc05608a807c2892d50c60d06ed21490cd274c0369f9671bc05b3006acc2a193316caf4896454c9c299603bfed29bd488f1987ec446
2019-11-20 09:54:31 -05:00
Wladimir J. van der Laan
ea3c7e585c test: Remove libssl-dev packages from CI scripts 2019-11-19 15:04:43 +01:00
fanquake
397dbae070
ci: remove OpenSSL installation 2019-11-18 08:56:48 -05:00
Jeffrey Czyz
15c84f53f4 Define ARENA_DEBUG in Travis test runs
The definition and uses of Arena::walk() are compiled only if
ARENA_DEBUG is defined. Configure Travis to define ARENA_DEBUG so
compilation errors do not go unnoticed.
2019-11-16 08:45:21 -08:00
MarcoFalke
fa6e01b2f3
ci: Use clang-8 for fuzzing to run on aarch64 ci systems 2019-11-13 17:53:04 -05:00
MarcoFalke
fa493ef088
ci: Make ci system read-only on the git work tree 2019-11-09 09:13:41 -05:00
MarcoFalke
fab1333292
ci: Remove git from required packages on host 2019-11-09 09:12:24 -05:00
MarcoFalke
fa00393bce
ci: Make all filesystem operations inside docker 2019-11-09 09:11:32 -05:00
MarcoFalke
faf757a125
ci: Guess the native host when not cross compiling 2019-11-07 13:42:57 -05:00
MarcoFalke
fa8a60bce9
ci: Run non-cross-compile builds natively 2019-11-04 14:10:24 -05:00
MarcoFalke
fa56bcbb01
ci: Run CI_WAIT only on travis 2019-11-04 14:10:07 -05:00
MarcoFalke
def69e5a86
Merge #17233: travis: Run unit and functional tests on native arm
facc0da63a travis: Run unit and functional tests on native arm (MarcoFalke)
fafa064d2a ci: Remove ccache requirement on the host (MarcoFalke)

Pull request description:

  This keeps the cross-compilation to make it easy to run the ci on non-arm hardware. To run this locally in qemu-user as it used to be, just `export QEMU_USER_CMD="qemu-arm -L /usr/arm-linux-gnueabihf/"`.

ACKs for top commit:
  laanwj:
    LGTM ACK facc0da63a
  practicalswift:
    ACK facc0da63a -- diff looks correct and Travis seems happy

Tree-SHA512: 0dc1bc82eb93e2bd8b159e044f20fe3055f8cdfd73aaa238bd2e178397582144dfc0c6a87bd8270115dafea1a623e642bde5d5f30254f94140f1a2cdb12fc2da
2019-11-04 08:23:21 -05:00
MarcoFalke
facc0da63a
travis: Run unit and functional tests on native arm 2019-10-30 14:45:32 -04:00
MarcoFalke
fafa064d2a
ci: Remove ccache requirement on the host
ccache is only needed to create the cache dir on the host, if it didn't
already exist. The same can be achieved with mkdir, so just use that
instead.
2019-10-24 18:37:38 -04:00
fanquake
3548e4aac7
Remove BIP70 Support 2019-10-24 16:01:43 -04:00
fanquake
67328bb7ca
build: remove protobuf from depends 2019-10-24 16:01:42 -04:00
MarcoFalke
fa710066b9
ci: Disable functional tests on mac host 2019-10-24 12:43:55 -04:00
MarcoFalke
4aaeb04497
Merge #17205: ci: Enable address sanitizer (ASan) stack-use-after-return checking
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking (practicalswift)

Pull request description:

  Enable address sanitizer (ASan) stack-use-after-return checking (`detect_stack_use_after_return=1`).

  Example:

  ```
  #include <iostream>
  #include <string>

  const std::string& get_string(int i) {
      return std::to_string(i);
  }

  int main() {
      std::cout << get_string(41) << "\n";
  }
  ```

  Without address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ./stack-use-after-return

  $
  ```

  With address sanitizer (ASan) stack-use-after-return checking:

  ```
  $ ASAN_OPTIONS="detect_stack_use_after_return=1" ./stack-use-after-return
  =================================================================
  ==10400==ERROR: AddressSanitizer: stack-use-after-return on address 0x7f7fa0400030 at pc 0x00000049d2cc bp 0x7ffcbd617070 sp 0x7ffcbd616820
  READ of size 2 at 0x7f7abbecd030 thread T0
      #0 0x439781 in fwrite
      #1 0x7f7ac0504cb3 in std::basic_ostream<char, std::char_traits<char> >& std::__ostream_insert<char, std::char_traits<char> >(std::basic_ostream<char, std::char_traits<char> >&, char const*, long) (/usr/lib/x86_64-linux-gnu/libstdc++.so.6+0x113cb3)
      #2 0x4f9b5f in main stack-use-after-return.cpp:9:15
      #3 0x7f7abf440b96 in __libc_start_main
      #4 0x41bbc9 in _start
  …
  $
  ```

Top commit has no ACKs.

Tree-SHA512: 6557a9ff184023380fd9aa433cdf413e01a928ea99dbc59ec138e5d69cb9e13592e8bb5951612f231ff17a37a895bec5c0940c8db5f328a5c840a5771bdeeba5
2019-10-22 13:14:16 -04:00
practicalswift
8d22ab0e50 ci: Enable address sanitizer (ASan) stack-use-after-return checking 2019-10-21 10:04:28 +00:00
MarcoFalke
fa677d1801
ci: Remove redundant check for TRAVIS_OS_NAME
Can be reviewed with
git diff --ignore-all-space --function-context
2019-10-17 11:30:36 -04:00
Sjors Provoost
1f6c650c99
travis: run tests on macOS native
Review hint:
git show -w

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
Co-authored-by: keneanung <keneanung@googlemail.com>
Co-authored-by: Vadim Peretokin <vperetokin@gmail.com>
2019-10-14 23:00:49 +02:00
MarcoFalke
fa79dff624
ci: Run tests on arm 2019-10-09 09:33:07 -04:00
Wladimir J. van der Laan
34919e0033
Merge #17011: ci: Use busybox utils for one build
ddddd8961b ci: Use busybox utils for one build (MarcoFalke)

Pull request description:

  To make sure Bitcoin Core can be built with BusyBox, see https://github.com/bitcoin/bitcoin/pull/16927#issuecomment-536483706

ACKs for top commit:
  laanwj:
    ACK ddddd8961b

Tree-SHA512: da3a4654ee7975206d04643675d309b4973a510ca344acaec97fb1ed19c43cf13489bdf236c92c4a90499ec5b3c18c3338fff096110b26abee5ffe955089f267
2019-10-08 11:31:48 +02:00
MarcoFalke
fa95503d64
ci: Make apt-get more verbose, to debug travis timeouts 2019-10-03 10:50:33 -04:00
MarcoFalke
ddddd8961b
ci: Use busybox utils for one build 2019-10-02 10:01:04 -04:00
practicalswift
aa81e2cc0e Enable UBSan for Travis fuzzer job 2019-10-01 08:23:34 +00:00
MarcoFalke
fadd76acc2
ci: Remove TRAVIS env vars 2019-09-24 16:43:44 -04:00
MarcoFalke
fa449b89b5
ci: Set $HOST before setting fallback values 2019-09-24 16:21:49 -04:00
fanquake
107e030723
build: make protobuf optional in depends
Those that want to build it can now pass PROTOBUF=1.
2019-09-14 12:58:59 +08:00
fanquake
376f4929f8
build: disable BIP70 support by default 2019-09-12 20:28:17 +10:00
MarcoFalke
fa27372e65
ci: Move CCACHE_DIR and test_runner tmp dir into ./ci/scratch/ 2019-08-27 15:58:50 -04:00
MarcoFalke
fa60583d23
ci: Pass down $MAKEJOBS to test_runner.py 2019-08-27 15:58:49 -04:00
MarcoFalke
8b42db130f
Merge #16623: ci: Add environment files for all settings
fa21737ba7 ci: Add environment files for all settings (MarcoFalke)

Pull request description:

  This moves all environment settings from travis to files in the ci folder. Now, it is possible to easily run each travis configuration with a single command.

Top commit has no ACKs.

Tree-SHA512: 989c6b62eb3839eb1fa5461e986496e9660167e2438a789c7588a6fee4f9c37b332782c010fe5c7de8f606bcf98dffb2481d2777cbce88f87cc9f0c42fb2d7fc
2019-08-19 09:39:53 -04:00
MarcoFalke
faba46da07
ci: Set --ansi in test_runner 2019-08-16 08:50:01 -04:00
MarcoFalke
fa21737ba7
ci: Add environment files for all settings 2019-08-16 08:46:37 -04:00
MarcoFalke
fa6cbdc3c9
ci: Use ./ci/ on non-travis host 2019-08-15 11:13:09 -04:00
MarcoFalke
fa31bc35eb
ci: Remove dependence on travis, use it as fallback env 2019-08-15 11:12:34 -04:00
MarcoFalke
fafe78f6ae
ci: Rename .travis/ to ./ci/ 2019-08-15 11:11:45 -04:00