Commit graph

29255 commits

Author SHA1 Message Date
W. J. van der Laan
0102f80b51
Merge #21375: guix: Misc feedback-based fixes + hier restructuring
7476b46f18 guix: Build dmg as a static binary (Carl Dong)
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching (Carl Dong)
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball (Carl Dong)
ca85679eb4 guix: Use clang-toolchain instead of clang (Carl Dong)
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users (Carl Dong)
1742f8e12d guix: Add early health check for guix-daemon (Carl Dong)
c1ae726a13 guix: More thoroughly control native toolchain (Carl Dong)
39741128d3 guix: Supply --link-profile (Carl Dong)
d55a1056ee guix: Add troubleshooting documentation entries (Carl Dong)
7f401c953f guix: Adapt guix-build to prelude, restructure hier (Carl Dong)
4eccf063b2 guix: Remove guix-build.sh filename extension (Carl Dong)
7753357a7b guix: Add source-able bash prelude and utils (Carl Dong)
e5b49a01f5 guix: Create windeploy inside distsrc-* (Carl Dong)
3e9982ab38 contrib: Silence git-describe when looking for tag (Carl Dong)
d5a71e9785 guix: Use --cores instead of --max-jobs (Carl Dong)

Pull request description:

  This PR addresses a few hiccups encountered by the brave souls who've been experimenting with the Guix scripts:
  - Resolves confusion between `--cores=` and `--max-jobs=`
    - `guix`'s `--cores=` actually corresponds to make's `--jobs=`, so let's just control `--cores=` with our overridable env var
  - `git-describe` will scream `fatal: no tag exactly matches '<hash>'` when looking for a tag, but we don't care, so silence that
  - `windeploy/unsigned` should be inside `distsrc-*` and created idempotently (sorry I know this one annoyed people)
  - Add troubleshooting documentation to `README.md`
  - Add early health check for `guix-daemon` in case user forgot to start a `guix-daemon`
  - Depending on configuration, a `--fallback` flag may be needed to tell Guix to not fail if substitutes fail but fallback to building locally
  - `codesign_allocate` and `pagestuff` are now unnecessary for codesigning as we're now using `signapple`

  A few robustness changes are also included:
  - We supply the `--link-profile` flag, as some Guix packages may expect the profile to be available under `$HOME/.guix-profile`
  - We now clear and manually set all toolchain-related env vars (e.g. `C*_INCLUDE_PATH`) ourselves, after patching a Qt::moc bug
  - We use the native `clang-toolchain` package for darwin builds instead of `clang`, lining up with all our other toolchain packages.

  Finally, we restructure the guix building hierarchy such that it looks something like:
  ```
  guix-build-<short-hash-or-version-tag>
  ├── distsrc-<short-hash-or-version-tag>-${HOST}
  │   ├── contrib
  │   ├── depends
  │   ├── src
  │   └── ...
  ├── distsrc-<short-hash-or-version-tag>-...
  └── output
      ├── dist-archive
      │   └── bitcoin-<short-hash-or-version-tag>.tar.gz
      ├── *-linux-*
      │   ├── bitcoin-<short-hash-or-version-tag>-*-linux-*-debug.tar.gz
      │   └── bitcoin-<short-hash-or-version-tag>-*-linux-*.tar.gz
      ├── x86_64-apple-darwin18
      │   ├── bitcoin-<short-hash-or-version-tag>-osx64.tar.gz
      │   ├── bitcoin-<short-hash-or-version-tag>-osx-unsigned.dmg
      │   └── bitcoin-<short-hash-or-version-tag>-osx-unsigned.tar.gz
      └── x86_64-w64-mingw32
          ├── bitcoin-<short-hash-or-version-tag>-win64-debug.zip
          ├── bitcoin-<short-hash-or-version-tag>-win64-setup-unsigned.exe
          ├── bitcoin-<short-hash-or-version-tag>-win64.zip
          └── bitcoin-<short-hash-or-version-tag>-win-unsigned.tar.gz
  ```
  Separating guix builds by their version identifier (basically namespacing them) allows us to change the layout in the future without worry about potential naming conflicts.

ACKs for top commit:
  sipa:
    ACK 7476b46f18
  laanwj:
    ACK 7476b46f18

Tree-SHA512: 0e899aa941aafdf552b2a7e8a08131ee9283180bbef7334439e2461a02aa7235ab7b9ca9c149b80fc5d0a9f4bbd35bc80fcee26197c0836ba8eaf2d86ffa0386
2021-04-06 01:00:27 +02:00
MarcoFalke
fa121b628d
blockstorage: [refactor] Use chainman reference where possible
Also, add missing { } for style.

Can be reviewed with `--word-diff-regex=.`
2021-04-05 20:26:32 +02:00
MarcoFalke
fa0c7d9ad2
move-only: Move *Disk functions to blockstorage
Can be reviewed with the git options
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-05 20:26:14 +02:00
Carl Dong
d7824acdb9 rest: Use existing NodeContext 2021-04-05 11:14:33 -04:00
Carl Dong
3f08934799 rest: Pass in NodeContext to rest_block 2021-04-05 11:14:33 -04:00
Carl Dong
7be0671b95 rpc/rawtx: Use existing NodeContext
Also pass in appropriate object to:
- TxToJSON
2021-04-05 11:14:33 -04:00
Carl Dong
60dc05afc6 rpc/mining: Use existing NodeContext
Also pass in appropriate object to:
- GetNetworkHashPS
- [gG]enerateBlock{,s}

Also:
- Misc style/constness changes
2021-04-05 11:14:28 -04:00
Carl Dong
d485e815e2 rpc/blockchain: Use existing NodeContext
Also pass in appropriate object to:
- BIP9SoftForkDescPushBack
- BuriedForkDescPushBack
2021-04-05 11:13:54 -04:00
Carl Dong
d0abf0bf42 rpc/*,rest: Add review-only assertion to EnsureChainman 2021-04-05 11:13:54 -04:00
Carl Dong
cced0f46c9 miner: Pass in previous CBlockIndex to RegenerateCommitments 2021-04-05 11:13:51 -04:00
Carl Dong
7476b46f18 guix: Build dmg as a static binary
This relatively easy change eliminates all runtime dependencies (except
for the kernel) for dmg, which is the only native build tool that gets
put in our output tarballs.

This allows much more flexibility when constructing the codesigning
environment, and is much more robust.
2021-04-05 11:00:39 -04:00
Carl Dong
06d6cf6784 depends: libdmg-hfsplus: Skip CMake RPATH patching
CMake's RPATH patching apparently causes non-reproducibility in the
executables which are produced, manifesting in a difference in padding
in the .dynstr section (we found this while investigating
non-reproducibility in the "dmg" tool). This RPATH patching can be
safely skipped for executables which don't depend on internal shared
libraries.

Documentation sources:
1. https://gitlab.kitware.com/cmake/community/-/wikis/doc/cmake/RPATH-handling
2. https://reproducible-builds.org/docs/deterministic-build-systems/#cmake-notes

Prior debugging art:
1. https://stackoverflow.com/questions/63438206/cmake-g-reproducible-build-issue-with-changing-build-path
2. https://github.com/NXPmicro/mfgtools/pull/229/files
2021-04-05 11:00:39 -04:00
Carl Dong
65176ab573 guix: Remove codesign_allocate+pagestuff from unsigned tarball 2021-04-05 11:00:39 -04:00
Carl Dong
ca85679eb4 guix: Use clang-toolchain instead of clang 2021-04-05 11:00:39 -04:00
Carl Dong
1aec0eda8f guix: Fallback to local build for substitute-enabled Guix users 2021-04-05 11:00:39 -04:00
Carl Dong
1742f8e12d guix: Add early health check for guix-daemon 2021-04-05 11:00:39 -04:00
Carl Dong
c1ae726a13 guix: More thoroughly control native toolchain 2021-04-05 11:00:39 -04:00
Carl Dong
39741128d3 guix: Supply --link-profile 2021-04-05 11:00:39 -04:00
Carl Dong
d55a1056ee guix: Add troubleshooting documentation entries 2021-04-05 11:00:39 -04:00
Carl Dong
7f401c953f guix: Adapt guix-build to prelude, restructure hier 2021-04-05 11:00:39 -04:00
Carl Dong
4eccf063b2 guix: Remove guix-build.sh filename extension 2021-04-05 11:00:39 -04:00
Carl Dong
7753357a7b guix: Add source-able bash prelude and utils 2021-04-05 11:00:21 -04:00
MarcoFalke
1ea5c7ec78
Merge #21604: test: Document why no symbol names can be used for suppressions
fa04eb7d78 test: Document why no symbol names can be used for suppressions (MarcoFalke)

Pull request description:

Top commit has no ACKs.

Tree-SHA512: 02e5d9cb3ca299b5e7f1f0f780af514692147f5f0846e21e5375a99e90d3aed3a3489c674ac0eb90ae34d60f576410ba1d926d5ca70cdce4f9c7a111cd516adf
2021-04-05 16:25:58 +02:00
MarcoFalke
fa04eb7d78
test: Document why no symbol names can be used for suppressions 2021-04-05 16:23:13 +02:00
Hennadii Stepanov
b8e5d0d3fe
qt: Handle exceptions in SendCoinsDialog::sendButtonClicked slot
Also, uic automatic connection replaced with an explicit one.
2021-04-05 16:47:31 +03:00
Hennadii Stepanov
1ac2bc7ac0
qt: Handle exceptions in TransactionView::bumpFee slot
Also the parameter list of the TransactionView::bumpFee slot is made
compatible with one of the QAction::triggered signal.
2021-04-05 16:47:31 +03:00
Hennadii Stepanov
bc00e13bc8
qt: Handle exceptions in WalletModel::pollBalanceChanged slot
Actually, the private QTimer::timeout signal has one
QTimer::QPrivateSignal parameter.
2021-04-05 16:47:08 +03:00
W. J. van der Laan
5c9b06db81
Merge #21302: wallet: createwallet examples for descriptor wallets
5039e0e55a test: HelpExampleCliNamed and HelpExampleRpcNamed (Ivan Metlushko)
591735ef0b rpc: Add HelpExampleCliNamed and use it for `createwallet` doc (Wladimir J. van der Laan)
5d5a90e819 rpc: Add HelpExampleRpcNamed (Ivan Metlushko)

Pull request description:

  Rationale: make descriptor wallets more visible and just a bit easier to setup

  `bitcoin-cli help createwallet`

  **Before**:
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

  **After**
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  > bitcoin-cli createwallet "descriptors" false false "" true true true
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["descriptors", false, false, "", true, true, true]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 5039e0e55a

Tree-SHA512: d37210e6ce639addee881377092d8f6fb2a537a60a259c561899e24cf68a0254d7ff45a213573c938f626677e46770cd21113aae5974f26c66b9a2e137699c14
2021-04-05 15:31:41 +02:00
W. J. van der Laan
b2ee8b207d net: Deserialize hardcoded seeds from BIP155 blob
Switch from IPv6 slot-based format to more compact and flexible BIP155
format.
2021-04-05 14:00:48 +02:00
W. J. van der Laan
9b29d5df7f contrib: Add explicit port numbers for testnet seeds
This is necessary now due to parsing change.
2021-04-05 14:00:48 +02:00
W. J. van der Laan
2a257de113 contrib: Add a few TorV3 seed nodes 2021-04-05 14:00:48 +02:00
W. J. van der Laan
06030f7a42 contrib: generate-seeds.py generates output in BIP155 format 2021-04-05 13:59:40 +02:00
MarcoFalke
fab19871ba
test: Document race:validation_chainstatemanager_tests suppression 2021-04-05 13:31:04 +02:00
MarcoFalke
fa5eabe721
refactor: Remove negative lock annotations from globals 2021-04-05 08:42:15 +02:00
MarcoFalke
824eea5643
Merge #21599: test: Replace file level integer overflow suppression with function level suppression
585854ac66 test: Replace blanket UBSan signed integer overflow suppression for txmempool.cpp with specific suppression (practicalswift)

Pull request description:

  Replace file level (`txmempool.cpp`) signed integer overflow suppression with function level suppression (`CTxMemPool::PrioritiseTransaction`). The suppression was added yesterday in #21586.

  Rationale: To avoid risk hiding other signed integer overflows in `txmempool.cpp`.

  Obviously it would be better if this signed integer overflow fixed instead of suppressed - see details #20626. Any taker? :)

  To hit the issue via fuzzing:

  ```
  $ UBSAN_OPTIONS="print_stacktrace=1:halt_on_error=1:report_error_type=1" FUZZ=validation_load_mempool src/test/fuzz/fuzz
  INFO: Seed: 1184244493
  INFO: Loaded 1 modules   (634418 inline 8-bit counters): 634418 [0x55a09fdfbf98, 0x55a09fe96dca),
  INFO: Loaded 1 PC tables (634418 PCs): 634418 [0x55a09fe96dd0,0x55a0a08450f0),
  INFO:     1264 files found in mempool/
  INFO: -max_len is not provided; libFuzzer will not generate inputs larger than 1040698 bytes
  INFO: seed corpus: files: 1264 min: 1b max: 1040698b total: 15997133b rss: 197Mb
  txmempool.cpp:847:15: runtime error: signed integer overflow: -7211388903327006720 + -7211353718954917888 cannot be represented in type 'long'
      #0 0x55a09c3ce2d8 in CTxMemPool::PrioritiseTransaction(uint256 const&, long const&) /home/thomas/bitcoin/src/txmempool.cpp:847:15
  ```

ACKs for top commit:
  JeremyRubin:
    utACK 585854a
  hebasto:
    ACK 585854ac66, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 5a343f028c1e1a1aba3b51a0eced605849184891ffafecb3cd2b424c6cfea01afd7c2672274936b0bac646075ec066408a570bf6b34bc9b87399a53ce20d8a23
2021-04-05 07:28:45 +02:00
practicalswift
585854ac66 test: Replace blanket UBSan signed integer overflow suppression for txmempool.cpp with specific suppression 2021-04-04 21:10:15 +00:00
MarcoFalke
fa91b2b2b3
move-only: Move AbortNode to shutdown
Can be reviewed with the git option
--color-moved=dimmed-zebra
2021-04-04 18:08:36 +02:00
MarcoFalke
fa413f07a1
move-only: Move ThreadImport to blockstorage
Can be reviewed with the git options
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-04 18:07:24 +02:00
Hennadii Stepanov
b95f7f8ac0
build, qt, refactor: Drop sed commands for win32-g++/qmake.conf 2021-04-04 14:40:06 +03:00
MarcoFalke
590e49ccf2
Merge #21586: test: Add missing suppression for signed-integer-overflow:txmempool.cpp
fad8a97b21 test: Add missing suppression for signed-integer-overflow:txmempool.cpp (MarcoFalke)

Pull request description:

  Otherwise the fuzzer will crash:

  ```
  txmempool.cpp:847:15: runtime error: signed integer overflow: 8138645194045128704 + 4611686018427387904 cannot be represented in type 'long'
      #0 0x558ff1838d4c in CTxMemPool::PrioritiseTransaction(uint256 const&, long const&) /root/fuzz_dir/scratch/fuzz_gen/code/src/txmempool.cpp:847:15
      #1 0x558ff196e723 in LoadMempool(CTxMemPool&, CChainState&, std::function<_IO_FILE* (boost::filesystem::path const&, char const*)>) /root/fuzz_dir/scratch/fuzz_gen/code/src/validation.cpp:5053:22
      #2 0x558ff13f37ab in validation_load_mempool_fuzz_target(Span<unsigned char const>) /root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/validation_load_mempool.cpp:32:11
      #3 0x558ff1083378 in std::_Function_handler<void (Span<unsigned char const>), void (*)(Span<unsigned char const>)>::_M_invoke(std::_Any_data const&, Span<unsigned char const>&&) /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:300:2
      #4 0x558ff22a749d in std::function<void (Span<unsigned char const>)>::operator()(Span<unsigned char const>) const /usr/bin/../lib/gcc/x86_64-linux-gnu/9/../../../../include/c++/9/bits/std_function.h:688:14
      #5 0x558ff22a70e8 in LLVMFuzzerTestOneInput /root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz.cpp:63:5
      #6 0x558ff0f83543 in fuzzer::Fuzzer::ExecuteCallback(unsigned char const*, unsigned long) fuzzer.o
      #7 0x558ff0f6d442 in fuzzer::RunOneTest(fuzzer::Fuzzer*, char const*, unsigned long) fuzzer.o
      #8 0x558ff0f7323a in fuzzer::FuzzerDriver(int*, char***, int (*)(unsigned char const*, unsigned long)) fuzzer.o
      #9 0x558ff0f9ef82 in main (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x196df82)
      #10 0x7f1237f310b2 in __libc_start_main /build/glibc-eX1tMB/glibc-2.31/csu/../csu/libc-start.c:308:16
      #11 0x558ff0f4816d in _start (/root/fuzz_dir/scratch/fuzz_gen/code/src/test/fuzz/fuzz+0x191716d)

Top commit has no ACKs.

Tree-SHA512: 94c13771054b4acfb83e3dcfa09beb3f9d0ca0e025d3993cdf2e46df6456f227565b31fd4377b8dd86c567aeee800f293ac57a470c6f5f81e9177d460e7bd705
2021-04-04 07:44:00 +02:00
MarcoFalke
fa9b74f5ea
Fix assumeutxo crash due to missing base_blockhash 2021-04-04 07:38:02 +02:00
MarcoFalke
fa8fffebe8
refactor: Prefer clean assert over UB in coinstats 2021-04-04 07:37:43 +02:00
MarcoFalke
fa73ce6e65
Fix assumeutxo crash due to truncated file 2021-04-03 17:52:58 +02:00
MarcoFalke
fad8a97b21
test: Add missing suppression for signed-integer-overflow:txmempool.cpp 2021-04-03 17:39:41 +02:00
fanquake
fecb3723b6
build: additional PKG_CHECK_MODULES calls in bitcoin_qt.m4
Add checks for the edid, input and service support modules.
2021-04-03 15:27:04 +08:00
MarcoFalke
ad4bf8a945
Merge #20459: rpc: Fail to return undocumented return values
fa8192f42e rpc: Fail to return undocumented return values (MarcoFalke)

Pull request description:

  Currently a few return values are undocumented. This is causing confusion at the least. See for example #18476

  Fix this by treating it as an internal bug to return undocumented return values.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa8192f42e. Only changes: rebase, no const_cast suggestion, and tostring cleanups needed after suggestion

Tree-SHA512: c006905639bafe3045de152b00c34d9864731becb3c4f468bdd61a392f10d7e7cd89a54862c8daa8c11ac4eea0eb5f13b0f647d21e21a0a797b54191cff7238c
2021-04-03 09:26:31 +02:00
fanquake
a53eff3ce5
build: misc doc changes in bitcoin_qt.m4 2021-04-03 14:07:22 +08:00
fanquake
340fa6c0ff
build: set QT_*_CFLAGS & QT_*_LIBS in PKG_CHECK_MODULES() calls 2021-04-03 14:04:58 +08:00
fanquake
4b8ad97c5b
build: use QT_*_LIBS rather than passing lib names 2021-04-03 14:04:58 +08:00
fanquake
13313b2904
scripted-diff: replace Qt5 with ${qt_lib_prefix} in _BITCOIN_QT_CHECK_STATIC_LIBS
-BEGIN VERIFY SCRIPT-
sed -i -e "s/\[Qt5/\[\$\{qt_lib_prefix\}/g" build-aux/m4/bitcoin_qt.m4
sed -i -e "s/Qt5Core/\$\{qt_lib_prefix\}Core/g" build-aux/m4/bitcoin_qt.m4
-END VERIFY SCRIPT-
2021-04-03 14:04:58 +08:00