Commit graph

28442 commits

Author SHA1 Message Date
Carl Dong
d522d8006b guix: Attest to inputs in inputs.SHA256SUMS
At build/codesigning-time, hash build inputs and output the digest to
${OUTDIR}/inputs.SHA256SUMS, which gets included in the final SHA256SUMS
constructed by guix-attest.

Example final SHA256SUMS:
ee832d2a35b7701bff581dea05a536118b118e3ad0a587a2855b6ee8cd6fba20  inputs/bitcoin-78199266af7b.tar.gz
ca765e70a0c12866dd63c0be228b675278a26329e5f8f5b5c52fd09200fedf21  bitcoin-78199266af7b-powerpc64le-linux-gnu-debug.tar.gz
dae95327d7f2c324e2728c4b73627be6cb2c0d2f2e5bea940d1d5e6463939327  bitcoin-78199266af7b-powerpc64le-linux-gnu.tar.gz
2021-05-03 13:18:19 -04:00
Carl Dong
f9e2960c01 guix: Construct $OUTDIR in ${DISTSRC}/output
While files are being output to $OUTDIR, it will be under
${DISTSRC}/output, and only when everything is done, will
${DISTSRC}/output be moved to the actual $OUTDIR.

This makes it so that a Ctrl-C in the middle of a build is less likely
to result in a partially-constructed $OUTDIR. In fact, if I understand
correctly, if $OUTDIR and $DISTSRC reside on the same filesystem, the
move (rename) is likely atomic.

Also, since the "working $OUTDIR" is under ${DISTSRC}/output, it will be
cleaned properly by the guix-clean script.
2021-05-03 13:18:19 -04:00
Carl Dong
022abc85fc guix: Minor quoting fix in libexec/build.sh 2021-05-03 13:18:19 -04:00
Carl Dong
c83c4fa5b7 guix-attest: Allow skipping GPG signing with NO_SIGN 2021-05-03 13:18:19 -04:00
Carl Dong
0e1c2e448c guix-attest: Use ascii-armor signatures 2021-05-03 13:18:19 -04:00
Carl Dong
b5fd89c4c8 guix-attest: Only use cross-platform flags for find+xargs 2021-05-03 13:18:19 -04:00
Carl Dong
5926432ba6 guix: Add guix-verify script 2021-05-03 13:18:19 -04:00
Carl Dong
30daf76a97 guix: Add guix-attest script 2021-05-03 13:18:19 -04:00
W. J. van der Laan
0c9597ce7d
Merge #21304: guix: Add guix-clean script + establish gc-root for container profiles
867a5e172a guix: Register garbage collector root for containers (Carl Dong)
8f8b96fb54 guix: Update hint messages to mention guix-clean (Carl Dong)
44f6d4f56b guix: Record precious directories and add guix-clean (Carl Dong)
84912d4b24 build: Remove spaces from variable-printing rules (Carl Dong)

Pull request description:

  ```
  guix: Record precious directories and add guix-clean

  Many users have reported problems that stem from having an unclean
  working tree. To that end, I've written a guix-clean script which should
  help reset the working tree while respecting user-specified precious
  directories.

  Precious directories, such as:

  - SOURCES_PATH
  - BASE_CACHE
  - SDK_PATH
  - OUTDIR

  Should be preserved when cleaning the working tree, and are thus
  recorded in ./contrib/guix/var/precious_dirs.

  The ./contrib/guix/guix-clean script is able to parse that file and make
  sure to avoid them when cleaning out the working tree.
  ```

ACKs for top commit:
  laanwj:
    ACK 867a5e172a

Tree-SHA512: c498fad781ff5e6406639df2b91b687fc528273fdf266bcdba8f6eec3b3b37ecce544b6da0252f0b9c6717f9d88e844e4c7b72d1877bdbabfc6871ddd0172af5
2021-04-08 23:19:54 +02:00
MarcoFalke
6664211be2
Merge #21574: Drop JSONRPCRequest constructors after #21366
9044522ef7 Drop JSONRPCRequest constructors after #21366 (Russell Yanofsky)

Pull request description:

  This just makes an additional simplification after #21366 replaced
  util::Ref with std::any. It was originally suggested
  https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351 but
  delayed for a followup. It would have prevented usage bug
  https://github.com/bitcoin/bitcoin/pull/21572.

ACKs for top commit:
  promag:
    ACK 9044522ef7, fixed conflict in src/wallet/interfaces.cpp.

Tree-SHA512: e909411b8f75013620b94e1a609296befb832fdcb574cd2e6689bfe3c636b03cd4ac1ccb2b32b532daf0f2131bb043464024966310fffc7e3cad77713d4bd0ef
2021-04-08 09:08:03 +02:00
fanquake
2e9031f95d
Merge #21626: doc: Fix typos from codespell
94c7dd9ac8 doc: Fix typos from codespell lint (Yerzhan Mazhkenov)

Pull request description:

  Typos from codespell linter: https://cirrus-ci.com/task/6677401661865984?logs=lint#L856
  - txrequest.cpp: `annoucements` ==> `announcements`
  - contrib/guix/README.md:298: `stil` ==> `still`
  - contrib/guix/guix-build:18: `invokable` ==> `invocable`
  - contrib/guix/libexec/prelude.bash:12: `invokable` ==> `invocable`
  - src/test/fuzz/tx_pool.cpp:37: `acess` ==> `access`
  - src/txorphanage.h:29: `orginating` ==> `originating`

ACKs for top commit:
  practicalswift:
    cr ACK 94c7dd9ac8: thnaks fro fiixng tpyos!
  jarolrod:
    ACK 94c7dd9ac8

Tree-SHA512: e0fac462a2f9e68b6a161c9f5d95b4d0648ce5c618fd7cd243d57db8f0256138b8823b166ea406b21e95586eae43047df1ef0df04616858082a39c1d1eb13a86
2021-04-08 08:16:04 +08:00
Carl Dong
867a5e172a guix: Register garbage collector root for containers
By registering the container profiles as garbage collector roots, it
will prevent `guix gc` from garbage collecting derivations which our
container needs and inconvieniencing the user with a rebuild.
2021-04-07 15:21:00 -04:00
Carl Dong
8f8b96fb54 guix: Update hint messages to mention guix-clean 2021-04-07 15:20:25 -04:00
Yerzhan Mazhkenov
94c7dd9ac8 doc: Fix typos from codespell lint 2021-04-07 19:26:25 +01:00
W. J. van der Laan
cb79cabdd9
Merge #21594: rpc: add network field to getnodeaddresses
5c446784b1 rpc: improve getnodeaddresses help (Jon Atack)
1b9189866a rpc: simplify/constify getnodeaddresses code (Jon Atack)
3bb6e7b655 rpc: add network field to rpc getnodeaddresses (Jon Atack)

Pull request description:

  This patch adds a network field to RPC `getnodeaddresses`, which is useful on its own, particularly with the addition of new networks like I2P and others in the future, and which I also found helpful for adding a new CLI command as a follow-up to this pull that calls `getnodeaddresses` and needs to know the network of each address.

  While here, also improve the `getnodeaddresses` code and help.

  ```
  $ bitcoin-cli -signet getnodeaddresses 3
  [
    {
      "time": 1611564659,
      "services": 1033,
      "address": "2600:1702:3c30:734f:8f2e:744b:2a51:dfa5",
      "port": 38333,
      "network": "ipv6"
    },
    {
      "time": 1617531931,
      "services": 1033,
      "address": "153.126.143.201",
      "port": 38333,
      "network": "ipv4"
    },
    {
      "time": 1617473058,
      "services": 1033,
      "address": "nsgyo7begau4yecc46ljfecaykyzszcseapxmtu6adrfagfrrzrlngyd.onion",
      "port": 38333,
      "network": "onion"
    }
  ]

  $ bitcoin-cli help getnodeaddresses
  getnodeaddresses ( count )

  Return known addresses, which can potentially be used to find new nodes in the network.

  Arguments:
  1. count    (numeric, optional, default=1) The maximum number of addresses to return. Specify 0 to return all known addresses.

  Result:
  [                         (json array)
    {                       (json object)
      "time" : xxx,         (numeric) The UNIX epoch time when the node was last seen
      "services" : n,       (numeric) The services offered by the node
      "address" : "str",    (string) The address of the node
      "port" : n,           (numeric) The port number of the node
      "network" : "str"     (string) The network (ipv4, ipv6, onion, i2p) the node connected through
    },
    ...
  ]
  ```
  Future idea: allow passing `getnodeaddresses` a network (or networks) as an argument to return only addresses in that network.

ACKs for top commit:
  laanwj:
    Tested ACK 5c446784b1
  jarolrod:
    re-ACK 5c446784b1
  promag:
    Code review ACK 5c446784b1.

Tree-SHA512: ab0101f50c76d98c3204133b9f2ab6b7b17193ada31455ef706ad11afbf48f472fa3deb33e96028682369b35710ccd07d81863d2fd55c1485f32432f2b75efa8
2021-04-07 18:56:01 +02:00
Jon Atack
5c446784b1
rpc: improve getnodeaddresses help 2021-04-07 12:57:11 +02:00
Jon Atack
1b9189866a
rpc: simplify/constify getnodeaddresses code 2021-04-07 12:57:09 +02:00
Jon Atack
3bb6e7b655
rpc: add network field to rpc getnodeaddresses 2021-04-07 12:57:07 +02:00
Russell Yanofsky
9044522ef7 Drop JSONRPCRequest constructors after #21366
This just makes an additional simplification after #21366 replaced
util::Ref with std::any. It was originally suggested
https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351 but
delayed for a followup. It would have prevented usage bug
https://github.com/bitcoin/bitcoin/pull/21572.
2021-04-07 04:53:26 -04:00
MarcoFalke
aa69471ecd
Merge #21572: Fix wrong wallet RPC context set after #21366
937fd4a66f Fix wrong wallet RPC context set after #21366 (Russell Yanofsky)

Pull request description:

  This bug doesn't have any effects currently because it only affects
  external signer RPCs which aren't currently using the wallet context,
  but it does cause an appveyor failure in a upcoming PR:

  https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/38512882

  This bug is subtle and could have been avoided if JSONRPCRequest didn't
  have constructors that were so loose with type checking.  Suggested
  change
  https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351
  eliminates these and would be a good followup for a future PR.

  This PR just implements the simplest possible fix.

ACKs for top commit:
  theStack:
    Code-review ACK 937fd4a66f
  meshcollider:
    Code review ACK 937fd4a66f

Tree-SHA512: 53e6265ed6c7abb47d2b3e77d1604edfeb993c3a2440f0c19679cfeb23516965e6707ff486196a0acfbeff21c79a9a08b5cd33bae9a232d33d0134bca1bd0ff3
2021-04-07 10:53:26 +02:00
fanquake
2b3e5bf4c0
Merge #21613: build: enable -Wdocumentation
a4e970adb6 build: enable -Wdocumentation if suppressing external warnings (fanquake)
3b0078f958 doc: fixup -Wdocumentation issues (fanquake)
c6edcf1c71 build: suppress libevent warnings if supressing external warnings (fanquake)

Pull request description:

  Enable `-Wdocumentation` by taking advantage of our `--enable-suppress-external-warnings` flag. Most of the CIs are using this flag now, so any regressions should be caught.

  This also required modifying libevents flags when suppressing warnings, as depending on the version being built against, that could generate a large number of warnings. i.e:
  ```bash
  In file included from httpserver.cpp:34:
  In file included from ./support/events.h:12:
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:464:11: warning: parameter 'req' not found in the function declaration [-Wdocumentation]
     @param req a request object
            ^~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:465:11: warning: parameter 'databuf' not found in the function declaration [-Wdocumentation]
     @param databuf the data chunk to send as part of the reply.
            ^~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:467:11: warning: parameter 'call' not found in the function declaration [-Wdocumentation]
     @param call back's argument.
            ^~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:939:4: warning: declaration is marked with '@deprecated' command but does not have a deprecation attribute [-Wdocumentation-deprecated-sync]
    @deprecated  This function is deprecated; you probably want to use
    ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:946:1: note: add a deprecation attribute to the declaration to silence this warning
  char *evhttp_decode_uri(const char *uri);
  ^
  __AVAILABILITY_INTERNAL_DEPRECATED
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:979:5: warning: declaration is marked with '@deprecated' command but does not have a deprecation attribute [-Wdocumentation-deprecated-sync]
     @deprecated This function is deprecated as of Libevent 2.0.9.  Use
     ~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:987:1: note: add a deprecation attribute to the declaration to silence this warning
  int evhttp_parse_query(const char *uri, struct evkeyvalq *headers);
  ^
  __AVAILABILITY_INTERNAL_DEPRECATED
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:1002:11: warning: parameter 'query_parse' not found in the function declaration [-Wdocumentation]
     @param query_parse the query portion of the URI
            ^~~~~~~~~~~
  /usr/local/Cellar/libevent/2.1.12/include/event2/http.h:1002:11: note: did you mean 'uri'?
     @param query_parse the query portion of the URI
            ^~~~~~~~~~~
            uri
  69 warnings generated.
  ```

  Note that a lot of these have already been fixed upstream.

ACKs for top commit:
  laanwj:
    Concept and code review ACK a4e970adb6
  practicalswift:
    cr ACK a4e970adb6: automatic compiler feedback comes sooner and is more reliable than manual reviewer feedback
  jonatack:
    Light ACK a4e970adb6 skimmed the changes, clang 11 build is clean with the change, verified -Wdocumentation build warnings with this change when a doc fix was reverted

Tree-SHA512: 57a1e30cffcc8bcceee72d85f58ebe29eae525861c70acb237541bd480c51ede89875c033042c0af376fdbb49fb7f588ef9282a47c6e78f9d4501c41f1b21eb6
2021-04-07 16:49:57 +08:00
MarcoFalke
6154291cf9
Merge #21617: fuzz: Fix uninitialized read in i2p test
33333755f2 fuzz: Fix uninitialized read in test (MarcoFalke)

Pull request description:

  Can be tested with:

  ```
  ./test/fuzz/test_runner.py -l DEBUG --valgrind ../btc_qa_assets/fuzz_seed_corpus/ i2p
  ```

  ```
  ==22582== Conditional jump or move depends on uninitialised value(s)
  ==22582==    at 0x6BB2D8: __sanitizer_cov_trace_const_cmp1 (in /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz)
  ==22582==    by 0xB305DB: ConnectSocketDirectly(CService const&, Sock const&, int, bool) (netbase.cpp:570)
  ==22582==    by 0x8AAA5D: i2p::sam::Session::Hello() const (i2p.cpp:284)
  ==22582==    by 0x8A6FA0: i2p::sam::Session::CreateIfNotCreatedAlready() (i2p.cpp:352)
  ==22582==    by 0x8A6742: i2p::sam::Session::Listen(i2p::Connection&) (i2p.cpp:134)
  ==22582==    by 0x7A6C42: i2p_fuzz_target(Span<unsigned char const>) (i2p.cpp:37)

ACKs for top commit:
  sipa:
    utACK 33333755f2
  vasild:
    ACK 33333755f2

Tree-SHA512: 36073582b26b541324b3e55f3fd4a44abf89cb3081f36d361525daf8c27602fbc25f736510ec30df7cb4ca0c4e395e8d8a60f531bf6af358b5a3e65dbabf72c0
2021-04-07 10:39:27 +02:00
fanquake
c0160ea52e
Merge #21540: wallet: refactor: dedup sqlite statement preparations/deletions
ea19cc844e wallet: refactor: dedup sqlite statement deletions (Sebastian Falbesoner)
9a3670930e wallet: refactor: dedup sqlite statement preparations (Sebastian Falbesoner)

Pull request description:

  This refactoring PR deduplicates repeated SQLite statement preparation calls (`sqlite3_prepare_v2(...)`) / deletions (`sqlite3_finalize(...)`) and its surrounding logic by putting each prepared statement and its corresponding text representation into a ~std::map~ ~`std::array`~ `std::vector`. This should be more readable and less error-prone, e.g. in case an additional statement needs to be added in the  future or the error handling has to be adapted.

ACKs for top commit:
  achow101:
    ACK ea19cc844e
  meshcollider:
    utACK ea19cc844e

Tree-SHA512: ced89869b2147e088e7a4cda2acbbdd4a806f66dbc2d6999953d0d702c0655aa53c0eb699cc7e5e3732f2d24206d577a9d9e1b5de7f439100dead2696ade1092
2021-04-07 14:17:25 +08:00
MarcoFalke
41a8d2b96f
Merge #21582: Fix assumeutxo crash due to missing base_blockhash
fa9b74f5ea Fix assumeutxo crash due to missing base_blockhash (MarcoFalke)
fa8fffebe8 refactor: Prefer clean assert over UB in coinstats (MarcoFalke)

Pull request description:

  This fixes an UB (which results in a crash with sanitizers enabled). Can be reproduced by cherry-picking the test without the other code changes. The fix:

  * Adds an `Assert` to transform the UB into a clean crash, even when sanitizers are disabled
  * Adds an early-fail condition to avoid the crash

ACKs for top commit:
  jamesob:
    ACK fa9b74f5ea ([`jamesob/ackr/21582.1.MarcoFalke.fix_assumeutxo_crash_due`](https://github.com/jamesob/bitcoin/tree/ackr/21582.1.MarcoFalke.fix_assumeutxo_crash_due))
  ryanofsky:
    Code review ACK fa9b74f5ea with no code changes since last review, just splitting up combocommit a little.

Tree-SHA512: dd36808a09f49c647543a9eaa6fdb785b3f1109af48ba4cc983153b22a144da9ca61af22034dcfaa0e192a65b1ee7de744f187555079aff55bec0efa0ce87cd4
2021-04-07 07:33:27 +02:00
fanquake
245a5cd560
Merge #21166: Introduce DeferredSignatureChecker and have SignatureExtractorClass subclass it
a97a9298ce Test that signrawtx works when a signed CSV and CLTV inputs are present (Andrew Chow)
6965456c10 Introduce DeferringSignatureChecker and inherit with SignatureExtractor (Andrew Chow)

Pull request description:

  Previously SignatureExtractorChecker took a MutableTransactionSignatureChecker and passed through function calls to that. However not all functions were implemented so not everything passed through as it should have. To solve this, SignatureExctractorChecker now implements all of those functions via a new class - DeferredSignatureChecker. DeferredSignatureChecker is introduced to allow for future signature checkers which use another SignatureChecker but need to be able to do somethings outside of just the signature checking.

  Fixes #21151

ACKs for top commit:
  sipa:
    utACK a97a9298ce
  meshcollider:
    Code review ACK a97a9298ce
  instagibbs:
    utACK a97a9298ce

Tree-SHA512: bca784c75c2fc3fcb74e81f4e3ff516699e8debaa2db81e12843abdfe9cf265dac11db8619751cb9b3e9bbe779805d029fabe5f3cbca5e86bfd72de3664b0b94
2021-04-07 12:47:41 +08:00
MarcoFalke
33333755f2
fuzz: Fix uninitialized read in test 2021-04-06 12:43:33 +02:00
W. J. van der Laan
9be7fe4849
Merge #21560: net: Add Tor v3 hardcoded seeds
b2ee8b207d net: Deserialize hardcoded seeds from BIP155 blob (W. J. van der Laan)
9b29d5df7f contrib: Add explicit port numbers for testnet seeds (W. J. van der Laan)
2a257de113 contrib: Add a few TorV3 seed nodes (W. J. van der Laan)
06030f7a42 contrib: generate-seeds.py generates output in BIP155 format (W. J. van der Laan)

Pull request description:

  Closes #20239 and mitigates my node's problem in #21351.

  - Add a few hardcoded seeds for TorV3
    - As the [bitcoin-seeder](https://github.com/sipa/bitcoin-seeder) doesn't collect TorV3 addresses yet, I have extracted these from my own node using [a script](https://gist.github.com/laanwj/b3d7b01ef61ce07c2eff0a72a6b90183) and added them manually. This is intended to be a temporary stop gap until 22.0's seeds update.

  - Change hardcoded seeds to variable length BIP155 binary format.
    - It is stored as a single serialized blob in a byte array, instead of pseudo-IPv6 address slots. This is more flexible and, assuming most of the list is IPv4, more compact.
    - Only the (networkID, addr, port) subset (CService). Services and time are construed on the fly as before.

  - Change input format for `nodes_*.txt`.
    - Drop legacy `0xAABBCCDD` format for IPv4. It is never generated by `makeseeds.py`.
    - Stop interpreting lack of port as default port, interpret it as 'no port', to accomodate I2P and other port-less protocols (not handled in this PR). An explicit port is always generated by `makeseeds.py` so in practice this makes no difference right now.

  A follow-up to this PR could do the same for I2P.

ACKs for top commit:
  jonatack:
    ACK b2ee8b207d

Tree-SHA512: 11a6b54f9fb0192560f2bd7b218f798f86c1abe01d1bf37f734cb88b91848124beb2de801ca4e6f856e9946aea5dc3ee16b0dbb9863799e42eec1b239d40d59d
2021-04-06 10:47:51 +02:00
MarcoFalke
1a7dec77f6
Merge #21571: test: make sure non-IP peers get discouraged and disconnected (vasild)
81747b2171 test: make sure non-IP peers get discouraged and disconnected (Vasil Dimov)
637bb6da36 test: also check disconnect in denialofservice_tests/peer_discouragement (Vasil Dimov)
4d6e246fa4 test: use pointers in denialofservice_tests/peer_discouragement (Vasil Dimov)

Pull request description:

  Split up from #20966, so that it can be backported easier. Merging this ahead of #20966 will also reduce the number of conflicts for that pull.

ACKs for top commit:
  jonatack:
    ACK 81747b2171

Tree-SHA512: 8f0e30b95baba7f056920d7fc3b37bd49ee13e69392fe80e2d333c6bb09fd25f4603249301b8795cca26a2f2d15b9f8904798a55cd9c04fd28afb316e95c551c
2021-04-06 10:26:33 +02:00
MarcoFalke
02fb5bdc21
Merge #21610: build: remove -Wdeprecated-register from NOWARN flags
1a011b3a82 build: remove -Wdeprecated-register from NOWARN flags (fanquake)

Pull request description:

  The `register` keyword was deprecated in C++11, and [removed in C++17](https://en.cppreference.com/w/cpp/keyword/register). Now that we require C++17, we shouldn't have to suppress warnings for a non-existent feature.

ACKs for top commit:
  sipa:
    utACK 1a011b3a82
  hebasto:
    ACK 1a011b3a82

Tree-SHA512: 7546b3870fe819507deea57d5c3179bc5debd4513df41b3f74d191995116b1507ff2a77cf2081ac32871194d4e5c2a2913c668c56244dff39853a5da18194f9f
2021-04-06 08:52:54 +02:00
fanquake
a4e970adb6
build: enable -Wdocumentation if suppressing external warnings
Co-authored-by: Ben Woosley <ben.woosley@gmail.com>
2021-04-06 14:50:26 +08:00
fanquake
3b0078f958
doc: fixup -Wdocumentation issues 2021-04-06 14:50:17 +08:00
MarcoFalke
7b4934e550
Merge #21557: test: small cleanup in RPCNestedTests tests
6526a1644c test: small cleanup in RPCNestedTests tests (fanquake)

Pull request description:

  Remove QtDir & QtGlobal (dea086f498)
  Add missing includes.
  Remove obsolete comment about Qt 5.3 (fd46c4c001)

Top commit has no ACKs.

Tree-SHA512: 097e603fc31a19be1817459ad4c5a9692708f8a39a0ae87e4a60eabc22bf4f6141b577ba68746044fd594f92e36848b7cd56d60dccd262f83f8ec7310ab7d1bc
2021-04-06 08:45:08 +02:00
fanquake
c6edcf1c71
build: suppress libevent warnings if supressing external warnings 2021-04-06 14:44:50 +08:00
MarcoFalke
b2ec7d398f
Merge #21609: ci: increase CPU count of sanitizer job to increase memory limit
de3ae78eff ci: increase CPU count of sanitizer job to increase memory limit (fanquake)

Pull request description:

  According to the [docs](https://cirrus-ci.org/guide/linux/#linux-containers):
  > For each CPU you can't get more than 4G of memory.

  thus if we want this job to have 24GB of memory, we need to increase the CPU count to 6.

  It's currently [failing with](https://github.com/bitcoin/bitcoin/runs/2273962280):
  >  Requested memory is too high! You can request at most 4G per CPU

Top commit has no ACKs.

Tree-SHA512: 0a4da5649d061425190a373859274c78ca5587cd2d6e27905ec548f124ed114a0133215cb2eff22ffc182f50c3a53df58e7c9832b44db6e37d7ea59ec96a4775
2021-04-06 07:57:30 +02:00
MarcoFalke
9ac8f6d7dd
Merge #21598: refactor: Remove negative lock annotations from globals
fa5eabe721 refactor: Remove negative lock annotations from globals (MarcoFalke)

Pull request description:

  They only make sense for mutexes that are private members. Until cs_main is a private member the negative annotations should be replaced by excluded annotations, which are optional.

ACKs for top commit:
  sipa:
    utACK fa5eabe721
  ajtowns:
    ACK fa5eabe721
  hebasto:
    ACK fa5eabe721
  vasild:
    ACK fa5eabe721

Tree-SHA512: 06f8a200304f81533010efcc42d9f59b8c4d0ae355920c0a28efb6fa161a3e3e68f2dfffb0c009afd9c2501e6a293c6e5a419a64d718f1f4e79668ab2ab1fcdc
2021-04-06 07:54:12 +02:00
fanquake
328aaac80d
Merge #21597: test: Document race:validation_chainstatemanager_tests suppression
fab19871ba test: Document race:validation_chainstatemanager_tests suppression (MarcoFalke)

Pull request description:

ACKs for top commit:
  jamesob:
    ACK fab19871ba
  practicalswift:
    ACK fab19871ba

Tree-SHA512: 3f1838b4cf11eba768ce06826cd4b57c9065669b61a5530af44216fc96535ebf37124b47a8de8f72aedf32345157a72d2208cd63214481a9cb56c063f05db5dd
2021-04-06 10:33:30 +08:00
fanquake
e2777b5328
Merge #21611: Fix a typo in guix-build output
5c09bcadc4 Fix a typo in guix-build output (Pieter Wuille)

Pull request description:

  This was overlooked in #21375.

ACKs for top commit:
  fanquake:
    ACK 5c09bcadc4

Tree-SHA512: 81d8ad4061abb17d5f16ae72ab0c88df76d5a2f100cb9f471ca700d2e87583103036367ea1958a1066f9613f63908d1b2b35734a7eb77ec9850d4c8b079732e1
2021-04-06 09:12:36 +08:00
Pieter Wuille
5c09bcadc4 Fix a typo in guix-build output 2021-04-05 17:58:49 -07:00
fanquake
de3ae78eff
ci: increase CPU count of sanitizer job to increase memory limit
According to the docs,
https://cirrus-ci.org/guide/linux/#linux-containers, "For each CPU you
can't get more than 4G of memory.", thus if we want this job to have
24GB of memory, we need to increase the CPU count to 6.
2021-04-06 08:40:24 +08:00
fanquake
1a011b3a82
build: remove -Wdeprecated-register from NOWARN flags
The register keyword was deprecated in C++11, and removed in C++17. Now
that we require C++17, we shouldn't have to supress warnings for a
non-existant feature.
2021-04-06 08:12:22 +08:00
fanquake
511e58223b
Merge #21565: build: make bitcoin_qt.m4 more generic
fecb3723b6 build: additional PKG_CHECK_MODULES calls in bitcoin_qt.m4 (fanquake)
a53eff3ce5 build: misc doc changes in bitcoin_qt.m4 (fanquake)
340fa6c0ff build: set QT_*_CFLAGS & QT_*_LIBS in PKG_CHECK_MODULES() calls (fanquake)
4b8ad97c5b build: use QT_*_LIBS rather than passing lib names (fanquake)
13313b2904 scripted-diff: replace Qt5 with ${qt_lib_prefix} in _BITCOIN_QT_CHECK_STATIC_LIBS (fanquake)
477df3623c build: reorder libs in _BITCOIN_QT_CHECK_STATIC_LIBS (move-only) (fanquake)

Pull request description:

  These are some small changes to simplify this code before we may end up supporting Qt6. Replaces usages of `Qt5` with `qt_lib_prefix`, minor docs cleanups and takes care of [a suggestion](https://github.com/bitcoin/bitcoin/pull/21376#discussion_r590693514) left over from #21376.

  Guix builds:
  ```bash
  8749dca4a86c39607ff86b0107b76daea60d415a15b5022824f4a4469c6edc37  output/bitcoin-fecb3723b63d-aarch64-linux-gnu-debug.tar.gz
  7b162ad6d953ceed9402f5fef03a8199a76c6dd300a71b03c1814c33ce0126f6  output/bitcoin-fecb3723b63d-aarch64-linux-gnu.tar.gz
  869c42616320fc5bb25e942d7e20c0a8a4b8510939f8f8f6021a784e4afda83e  output/bitcoin-fecb3723b63d-arm-linux-gnueabihf-debug.tar.gz
  278b1481045e93b3ac7c31ae14ea3b6a3030391c2abc90e04ab80072f4c33dd3  output/bitcoin-fecb3723b63d-arm-linux-gnueabihf.tar.gz
  e5501939838f930bf5d5a6da4f6d3665c008d736aa916ef0e7e0e06db91e1a36  output/bitcoin-fecb3723b63d-osx-unsigned.dmg
  834a94ea3fbc5e090618842b339b301bebd622d1f392918f0f500ec6637733ac  output/bitcoin-fecb3723b63d-osx-unsigned.tar.gz
  6cf70af01a48bea9270e6883ed15ce458c46c6840f826414df7a244fffa3338d  output/bitcoin-fecb3723b63d-osx64.tar.gz
  d80e9e2789fdee931f80c049c70912e9ff146d5fe2ae7bcd26244d7fe5dd4910  output/bitcoin-fecb3723b63d-powerpc64-linux-gnu-debug.tar.gz
  d924e161f1c32b3980c72b4949c99269e3d5fad8656749edae9fbbaafb1b7ff9  output/bitcoin-fecb3723b63d-powerpc64-linux-gnu.tar.gz
  d2f339fc78608ac6cdc505b21abc5f397e52283914113f45dc0adad0c023df54  output/bitcoin-fecb3723b63d-powerpc64le-linux-gnu-debug.tar.gz
  d537c014adbdd712b7b20f3171bc4342622ba56f4fa655bba85afbf35aee2840  output/bitcoin-fecb3723b63d-powerpc64le-linux-gnu.tar.gz
  0f42646fa5be154e8717df0e39ba843a61928225b00e0d48b1b2179966fea315  output/bitcoin-fecb3723b63d-riscv64-linux-gnu-debug.tar.gz
  f7b9cc85f74c006ac484e20915c84a12069447102d685124c5246b6eff8545c8  output/bitcoin-fecb3723b63d-riscv64-linux-gnu.tar.gz
  b60b51544a26dfa2e6a05a6daa99c63aeb62d7a1893191824867e1da9bbb73c4  output/bitcoin-fecb3723b63d-win-unsigned.tar.gz
  abc4cdb44146edf16fc47aa11d72648fa53c00d8f70937cab74cdaaaa9947c58  output/bitcoin-fecb3723b63d-win64-debug.zip
  3c6d437b09acc479060d922c64c1f6688df6385a6a3950d31c7a6e0977d757eb  output/bitcoin-fecb3723b63d-win64-setup-unsigned.exe
  d4f015fcfca15f23a797786fe67ca12ac73d3f594dd9e1b885e33d765f21beff  output/bitcoin-fecb3723b63d-win64.zip
  5fcb478027cb790ab7a3480fab91f764b10da4fd25d82b08f5ecfe42b4b7d316  output/bitcoin-fecb3723b63d-x86_64-linux-gnu-debug.tar.gz
  eef74c6b301bd799c6522b60c26d700a496ecf2bd357f09c83cbd88132086783  output/bitcoin-fecb3723b63d-x86_64-linux-gnu.tar.gz
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  output/src/bitcoin-fecb3723b63d.tar.gz
  ```

  Gitian builds:
  ```bash
  # macOS:
  b3926b7956b31b87ec96885369401141cc5a3f37e91a0ad9a03c6f7c1bd2bee7  bitcoin-fecb3723b63d-osx-unsigned.dmg
  61792260b3afd0b921d84ddb0bb3639c2b4e063c7b7fb715d515b3601c7ad38b  bitcoin-fecb3723b63d-osx-unsigned.tar.gz
  953081b66a1feababe041cae0b31664856260f7364f87c5c50f7f7c1fd6b710a  bitcoin-fecb3723b63d-osx64.tar.gz
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  src/bitcoin-fecb3723b63d.tar.gz
  aecaa381205e9ed6585cdffa22c7b46d3ece989c1774855d4a3b4cae5d75567a  bitcoin-core-osx-22-res.yml

  # Windows:
  21c23951c96bcbf79fa75a6b915c40c24445964cced6242f4e9181464f66ba40  bitcoin-fecb3723b63d-win-unsigned.tar.gz
  327cc290f5f709892e54cdc7bbf1745892eed497cd636c8f80d2310419e585c2  bitcoin-fecb3723b63d-win64-debug.zip
  e86a36c26897be064339cda866cb9563d08c867eceae59a677329390d729cbfe  bitcoin-fecb3723b63d-win64-setup-unsigned.exe
  844754a178b3b7ba62fc39bcc3f5e76b3903c9cd86adc26ec224ecf84d697239  bitcoin-fecb3723b63d-win64.zip
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  src/bitcoin-fecb3723b63d.tar.gz

  # Linux:
  b6224d86755e6c609fd1cfff79f6514bdd0cf1af8689eca692352c515e6cb7dd  bitcoin-fecb3723b63d-aarch64-linux-gnu-debug.tar.gz
  371a2533f3510c7a7e7bd2a9b504accffdcdd65bcc9d6c393570e0378f470f93  bitcoin-fecb3723b63d-aarch64-linux-gnu.tar.gz
  6c171cf1168f7bc6a1f78d5662c6cd1de7dd6157559fe4f7f95d34df7e398354  bitcoin-fecb3723b63d-arm-linux-gnueabihf-debug.tar.gz
  ac455dcb33c92191eb0124c78fa356cea78c88e3976bfe8b4b9f9052458b7b24  bitcoin-fecb3723b63d-arm-linux-gnueabihf.tar.gz
  166ec0874d09b1e3d1b682ed120873b2715dee5ed13dd721443ee29784074107  bitcoin-fecb3723b63d-powerpc64-linux-gnu-debug.tar.gz
  2b32d1a761625c28ed59fdbe8463dcb89b4f583a3119c57fd6815c9882241c78  bitcoin-fecb3723b63d-powerpc64-linux-gnu.tar.gz
  b9b37f0097f39fb16c2612166ffe21a09b436feea42394935831e2cb102c532a  bitcoin-fecb3723b63d-powerpc64le-linux-gnu-debug.tar.gz
  e98fb672a5205a8d18f6ca4a39dec0c26353031c455c4e1b3917b917ce687d8d  bitcoin-fecb3723b63d-powerpc64le-linux-gnu.tar.gz
  f92b07bd155fc469a18aa34571534f2c0bf12706f381ca973bb7a51692e25754  bitcoin-fecb3723b63d-riscv64-linux-gnu-debug.tar.gz
  94321b7db0446457c372bea24d71d84d0f0181f7607bf9f96a8b773e69ee1370  bitcoin-fecb3723b63d-riscv64-linux-gnu.tar.gz
  350bc9be945e3410bf73d9631fe4bb750f9a2ee3d538d2ce3efd42b15eba926a  bitcoin-fecb3723b63d-x86_64-linux-gnu-debug.tar.gz
  74aa3745942a9d76d3db23fa0251a27bbe1ea95683a901df37333caf9c6d13ab  bitcoin-fecb3723b63d-x86_64-linux-gnu.tar.gz
  af897573bd4f1e9acfef43af19a79e0e1c6a662cb3dc0bc8f5f6d2656c441953  src/bitcoin-fecb3723b63d.tar.gz
  959c25666565646279659fccb93278875bb926b67faf685185f461bf55b4058e  bitcoin-core-linux-22-res.yml
  ```

ACKs for top commit:
  hebasto:
    ACK fecb3723b6

Tree-SHA512: 00edf084852bfab404fb32d0a726088a50283a58a240d4e03cbd746cbffbe085d309434b2c22a4ed18c28be4df6958eb51d92d4ccd7548dd0ee49f99d458ff03
2021-04-06 08:05:41 +08:00
Carl Dong
44f6d4f56b guix: Record precious directories and add guix-clean
Many users have reported problems that stem from having an unclean
working tree. To that end, I've written a guix-clean script which should
help reset the working tree while respecting user-specified precious
directories.

Precious directories, such as:

- SOURCES_PATH
- BASE_CACHE
- SDK_PATH
- OUTDIR

Should be preserved when cleaning the working tree, and are thus
recorded in ./contrib/guix/var/precious_dirs.

The ./contrib/guix/guix-clean script is able to parse that file and make
sure to avoid them when cleaning out the working tree.
2021-04-05 19:13:54 -04:00
Carl Dong
84912d4b24 build: Remove spaces from variable-printing rules
This simplifies parsing when using these rules from scripts.
2021-04-05 19:13:54 -04:00
W. J. van der Laan
0102f80b51
Merge #21375: guix: Misc feedback-based fixes + hier restructuring
7476b46f18 guix: Build dmg as a static binary (Carl Dong)
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb4 guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3 guix: Supply --link-profile (Carl Dong)
d55a1056ee guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b2 guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7b guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5 guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab38 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e9785 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f18
  laanwj:
    ACK 7476b46f18

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2021-04-06 01:00:27 +02:00
Carl Dong
7476b46f18 guix: Build dmg as a static binary
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.

This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
2021-04-05 11:00:39 -04:00
Carl Dong
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching
CMake's RPATH patching apparently causes non-reproducibility in the
executables which are produced, manifesting in a difference in padding
in the .dynstr section (we found this while investigating
non-reproducibility in the "dmg" tool). This RPATH patching can be
safely skipped for executables which don't depend on internal shared
libraries.

Documentation sources:
1. https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling
2. https://reproducible-builds.org/docs/deterministic-build-systems/#cmake-notes

Prior debugging art:
1. https://stackoverflow.com/questions/63438206/cmake-g-reproducible-build-issue-with-changing-build-path
2. https://github.com/NXPmicro/mfgtools/pull/229/files
2021-04-05 11:00:39 -04:00
Carl Dong
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball 2021-04-05 11:00:39 -04:00
Carl Dong
ca85679eb4 guix: Use clang-toolchain instead of clang 2021-04-05 11:00:39 -04:00
Carl Dong
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users 2021-04-05 11:00:39 -04:00
Carl Dong
1742f8e12d guix: Add early health check for guix-daemon 2021-04-05 11:00:39 -04:00