autotools and automake changes to support multiprocess execution.
This adds a new --enable-multiprocess flag, and build configuration code to
detect libraries needed for multiprocess support. The --enable-multiprocess
flag builds new bitcoin-node and bitcoin-gui executables, which are updated in
https://github.com/bitcoin/bitcoin/pull/10102 to communicate across processes.
But for now they are functionally equivalent to existing bitcoind and
bitcoin-qt executables.
9e071b0089 test: remove rapidcheck integration and tests (fanquake)
Pull request description:
Whilst the property tests are interesting, ultimately [rapidcheck](https://github.com/emil-e/rapidcheck) integration in this repository has not gained much traction. We have a limited number of tests, and they are rarely (if ever) run. Have discussed this with Chris Stewart.
ACKs for top commit:
practicalswift:
ACK 9e071b0089
Tree-SHA512: d0c12af3163382eee8413da420c63e39265a7b700709a05d518445832d45e049aed9508e32524db5228fe3ac114609a00b7bb890be047c07032e44a5ef4611e9
0ed2d8e07d test: add BIP37 remote crash bug [CVE-2013-5700] test to p2p_filter.py (Sebastian Falbesoner)
Pull request description:
Integrates the missing message type `filteradd` to the test framework and checks that the BIP37 implementation is not vulnerable to the "remote crash bug" [CVE-2013-5700](https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2013-5700) anymore. Prior to v.0.8.4, it was possible to trigger a division-by-zero error on the following line in the function `CBloomFilter::Hash()`:
f0d6487e29/src/bloom.cpp (L45)
By setting a zero-length filter via `filterload`, `vData.size()` is 0, so the modulo operation above, called on any .insert() or .contains() operation then crashed the node. The test uses the approach of just sending an arbitrary `filteradd` message after, which calls `CBloomFilter::insert()` (and in turn `CBloomFilter::Hash()`) on the node. The vulnerability was fixed by commit 37c6389c5a (an intentional covert fix, [according to gmaxwell](https://github.com/bitcoin/bitcoin/issues/18483#issuecomment-608224095)), which introduced flags `isEmpty`/`isFull` that wouldn't call the `Hash()` member function if `isFull` is true (set to true by default constructor).
To validate that the test fails if the implementation is vulnerable, one can simply set the flags to false in the member function `UpdateEmptyFull()` (that is called after a filter received via `filterload` is constructed), which activates the vulnerable code path calling `Hash` in any case on adding or testing for data in the filter:
```diff
diff --git a/src/bloom.cpp b/src/bloom.cpp
index bd6069b..ef294a3 100644
--- a/src/bloom.cpp
+++ b/src/bloom.cpp
@@ -199,8 +199,8 @@ void CBloomFilter::UpdateEmptyFull()
full &= vData[i] == 0xff;
empty &= vData[i] == 0;
}
- isFull = full;
- isEmpty = empty;
+ isFull = false;
+ isEmpty = false;
}
```
Resulting in:
```
$ ./p2p_filter.py
[...]
2020-04-03T14:38:59.593000Z TestFramework (INFO): Check that division-by-zero remote crash bug [CVE-2013-5700] is fixed
2020-04-03T14:38:59.695000Z TestFramework (ERROR): Assertion failed
[...]
[... some exceptions following ...]
```
ACKs for top commit:
naumenkogs:
utACK 0ed2d8e07d
Tree-SHA512: 02d0253d13eab70c4bd007b0750c56a5a92d05d419d53033523eeb3ed80318bc95196ab90f7745ea3ac9ebae7caee3adbf2a055a40a4124e0915226e49018fe8
691e2a7af7 build: create test_fuzz library from src/test/fuzz/fuzz.cpp (Harris)
Pull request description:
This PR creates a static library **libtest_fuzz.a** to speed up the compilation of fuzz tests. It is functionally similar to https://github.com/bitcoin/bitcoin/pull/17542
Fixes https://github.com/bitcoin/bitcoin/issues/18527
ACKs for top commit:
MarcoFalke:
ACK 691e2a7af7🦁
Tree-SHA512: 39d7d2731ca4370db518dbb969eb17ddbf9c030c3fe0dec0d04ff6578f24a128563fe5aced78300c92ce296623a7079fea5aea70619819a20c56fb34191f00ef
4670006762 test: remove redundant sync_with_ping after add_p2p_connection (Jon Atack)
Pull request description:
Now that #18247 is merged, these calls are redundant.
ACKs for top commit:
vasild:
utACK 4670006
Tree-SHA512: bdbfe8bcf9dbdde0a8115e3a62bfe359910798d7a3010d920ffca07049cb5f97bf8fb9b6f70079b0607105192b61a6d665774e59a2b678597b47ad6237595ad5
3dc8c012f0 test: remaining replacements of (send_message+sync_with_ping) with send_and_ping (Sebastian Falbesoner)
Pull request description:
This is a tiny follow-up PR to #18494, substituting the remaining occurences of `send_message(...)`/`sync_with_ping(...)` pairs with `send_and_ping(...)`, as suggested in the comment https://github.com/bitcoin/bitcoin/pull/18494#pullrequestreview-386418913. Thanks to jonatack and [MarcoFalke](https://github.com/bitcoin/bitcoin/pull/18494#issuecomment-608496342) for giving me the hint to do this follow-up.
ACKs for top commit:
practicalswift:
ACK 3dc8c012f0
Tree-SHA512: 44d64332933c23a7f59c0415e008ce1b2b2e07177f81cb9473b7c71558188f1c698e8973de5cc940280e4697f9553af852d9a42841304f82469673d1c8162852
facc332dc5 fuzz: Avoid running over all inputs after merging them (MarcoFalke)
Pull request description:
This cuts the time it takes to merge inputs by half
ACKs for top commit:
practicalswift:
ACK facc332dc5
Tree-SHA512: bb22992c463dd985d3b1e9b8908c591d0c8e620c38eba0a932d880f87133bfe4ca2036b166c4f79b92ddf7940f56c044e9cb8cc50309c74204df122b369c167d
fab32557f2 rpc: Make rpc documentation not depend on rpc args (MarcoFalke)
Pull request description:
This is required to host the documentation on a static resource (like a website or pdf)
ACKs for top commit:
emilengler:
utACK fab32557f2
promag:
ACK fab32557f2.
Tree-SHA512: 3ca2691c7fbd5f17c75df2887753da152f66521dcb7dee4c29af6339fdea011cecdd51f825b96bde9c6aaf82f4d915cbd5aacb52e4eae3898d9dbc216f627171
f32ab443a9 Bugfix: RPC: JSON null is not "None" (Luke Dashjr)
26dcf39581 Bugfix: RPC: Don't use a continuation elipsis after an elision elipsis (Luke Dashjr)
eca65caadc Bugfix: RPC: Add missing commas and correct indentation of explicit ELISION (Luke Dashjr)
Pull request description:
1. listsinceblock had a double ellipsis (elision + continuation); this looks ugly, just one is needed.
2. Elision ellipsis wasn't getting a comma, so was truncated to `".."` by comma-removal code.
3. Elision ellipsis was indented incorrectly (as if it was a subitem).
4. Similarly, type "none" would get truncated to `"Non"`, when it should really be `"null"` anyway.
ACKs for top commit:
MarcoFalke:
ACK f32ab443a9🐰
Tree-SHA512: 34e1c72673790ed11cdee838d64ea5e0ac498de19258df99d54b5322e003060123c65ad27ac2fd4729a1dfe52066a0629602a132b1ef85d4154affd99a065a3f
fad2f68353 init: Replace URL_WEBSITE with PACKAGE_URL (MarcoFalke)
Pull request description:
This is needed for rebranding efforts such as #18489
ACKs for top commit:
hebasto:
ACK fad2f68353, tested on Linux Mint 19.3:
fanquake:
ACK fad2f68353 - clicked a link.
Tree-SHA512: c26e18cd328d3dd3fd7e25413e1bab780026687a148f126b8673e5f6cc13249f6c16689e45eba9da1545915c6001f96cd33f4e656c08cda3eae1c3fd88da23ea
error: unknown switch `a'
usage: git add [<options>] [--] <pathspec>...
-n, --dry-run dry run
-v, --verbose be verbose
-i, --interactive interactive picking
-p, --patch select hunks interactively
-e, --edit edit current diff and apply
-f, --force allow adding otherwise ignored files
-u, --update update tracked files
--renormalize renormalize EOL of tracked files (implies -u)
-N, --intent-to-add record only the fact that the path will be added later
-A, --all add changes from all tracked and untracked files
--ignore-removal ignore paths removed in the working tree (same as --no-all)
--refresh don't add, only refresh the index
--ignore-errors just skip files which cannot be added because of errors
--ignore-missing check if - even missing - files are ignored in dry run
--chmod (+|-)x override the executable bit of the listed files
7b2975ae8f doc: Update webchat URLs in README.md (Suriyaa Sundararuban)
Pull request description:
#### What happend?
Web links in `doc/README.md` redirected from `http://webchat.freenode.net?channels=bitcoin` to `https://webchat.freenode.net/#bitcoin`.
#### What did I changed?
* Remove URL redirection. (Update all webchat links.)
* Use HTTPS protocol instead of HTTP.
ACKs for top commit:
fanquake:
ACK 7b2975ae8f
Tree-SHA512: e55970ad368ce6dbb79295d12629d0f16318e7c43d8d194876e16e81c6e325bb136c9e8b361d61c5a04e2f3624350f81c99131b1e85d9ee5410ac22f524a4ef2
d695eb4c21 doc: note the costs of fetching all pull requests (Vasil Dimov)
Pull request description:
Also mention that it is possible to fetch just one pull request.
ACKs for top commit:
MarcoFalke:
ACK d695eb4c21
fanquake:
ACK d695eb4c21
Tree-SHA512: afe080fd018b2e773fb974956937e819085831bf0c1c5623f7f12c728639906b80666b785234058ee39fd98115a53a2fad431c54ee0840667e60bb317e4a828d
332f373a9d [scripts] previous_release: improve failed download error message (Sebastian Falbesoner)
Pull request description:
Currently, if the earlier release build/fetch script `previous_release.sh` is invoked with the option `-b` (intending to fetch a binary package from `https://bitcoin.org`) and the download fails, the user sees the following confusing output:
```
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
gzip: stdin: not in gzip format
tar: Child returned status 1
tar: Error is not recoverable: exiting now
```
This implies that the download worked, but the archive is corrupted, when in reality the HTML document containing the delivery fail reason (most likely 404 Not Found) is saved and tried to get unpacked. In contrast to wget, curl is a bit stubborn and needs explicit instructions to react to server errors via the flag `-f` (outputs error message and returns error code, ideal for scripts): https://curl.haxx.se/docs/manpage.html#-f
On the PR branch, the output on failed download looks now the following:
```
$ contrib/devtools/previous_release.sh -r -b v0.9.5
[...]
curl: (22) The requested URL returned error: 404 Not Found
Download failed.
```
ACKs for top commit:
fanquake:
ACK 332f373a9d
Tree-SHA512: 046c931ad9e78aeb2d13faa4866d46122ed325aa142483547c2b04032d03223ed2411783b00106fcab0cd91b2f78691531ac526ed7bb3ed7547b6e2adbfb2e93
f65c9ad40f Check for overflow when calculating sum of outputs (Elichai Turkel)
Pull request description:
This was reported by practicalswift here #18046
The exact order of the if, is important, we first do `!MoneyRange(tx_out.nValue)` to make sure the amount is non-negative. and then `std::numeric_limits<CAmount>::max() - tx_out.nValue < nValueOut` checks that the addition cannot overflow (if we won't check that the amount is positive this check can also overflow! (by doing something like `max - -max`))
and only then we make sure that the some is also valid `!MoneyRange(nValueOut + tx_out.nValue)`
if any of these conditions fail we throw.
the overflowing logic:
```
a + b > max // we want to fail if a+b is more than the maximum -> will overflow
b > max - a
max - a < b
```
Closes: #18046
ACKs for top commit:
MarcoFalke:
ACK f65c9ad40f, checked that clang with O2 produces identical binaries 💕
practicalswift:
ACK f65c9ad40f
instagibbs:
utACK f65c9ad40f
vasild:
ACK f65c9ad40f modulo `s/assert.h/cassert/`
Tree-SHA512: 512d6cf4762f24c41cf9a38da486b17b19c634fa3f4efbdebfe6608779e96fc3014d5d2d29adb8001e113152c0217bbd5b3900ac4edc7b8abe77f82f36209e33
6112a20982 test: replace (send_message + sync_with_ping) with send_and_ping (Jon Atack)
Pull request description:
This is a follow-up to faf1d04731 yesterday.
ACKs for top commit:
vasild:
utACK 6112a20
MarcoFalke:
ACK 6112a20982 🎞
Tree-SHA512: 749644ac9a1ef0e1aa6c3ac5e899eb3fa7fb9c0909352f922a80412df2bc0e539692a7757af550eff4d4914cbe57b0c75ce3948f569acc7a52852e91a55ad457
9eefc6e92f gui: Delete progress dialog instead of hidding it (João Barbosa)
ee9e88ba27 wallet: Handle duplicate fileid exception (João Barbosa)
Pull request description:
Handle the duplicate fileid exception thrown at `CheckUniqueFileid` in tow cases:
- when duplicate wallets are set on the command line - catch in `LoadWallets`;
- when a duplicate wallet is loaded dynamically - catch in `LoadWallet`.
Fixes#16776.
ACKs for top commit:
jonatack:
Re-ACK 9eefc6e92f no change since last review 68e0ff0e1f530c942721aab49cf67ffc07104628
hebasto:
re-ACK 9eefc6e92f
Tree-SHA512: 46e3c1cd6708b54e2d1c4973a74c8d5428822e04cecbc147cf200eb034efa385e867bd749c7c639020e83c9813fae8fed64a851bdd99abf60c33b07e0363f5d5
a46484c8b3 build: Detect gmtime_* definitions via configure (Ben Woosley)
Pull request description:
Something has changed in the mingw-w64 headers such that we
no-longer compile when using 7.0.0.
```bash
util/time.cpp: In function 'std::__cxx11::string FormatISO8601DateTime(int64_t)':
util/time.cpp:84:9: error: 'gmtime_r' was not declared in this scope
if (gmtime_r(&time_val, &ts) == nullptr) {
^~~~~~~~
util/time.cpp: In function 'std::__cxx11::string FormatISO8601Date(int64_t)':
util/time.cpp:97:9: error: 'gmtime_r' was not declared in this scope
if (gmtime_r(&time_val, &ts) == nullptr) {
```
Looking at time.h, it seems that `gmtime_r()` is only available when
`_POSIX_C_SOURCE` is defined. This must have been the case for 6.0.0
(which we compile fine using), but no-longer seems to be for 7.0.0?
I've checked that adding `-D_POSIX_C_SOURCE=200112L` to our compile
flags does fix the issue above.
However, an alternative solution seems to be to just use `gmtime_s()`
instead, when compiling with `mingw-w64`, as `gmtime_r()` [just wraps
`gmtime_s()` anyways](7c03b11bf1/mingw-w64-headers/crt/time.h (L284)).
I've tested this change crosss-compiling on Debian Bullseye ([mingw-w64 7.0.0](https://packages.debian.org/source/bullseye/mingw-w64))
and Buster ([mingw-w64 6.0.0](https://packages.debian.org/source/buster/mingw-w64)).
ACKs for top commit:
laanwj:
ACK a46484c8b3
Tree-SHA512: 7cf1a81060b9625d64de40b77341d74704cc8ae1358d25d7e2909685dc83a7a9762260d72e47806e9f0a5cbabf88d0239ec9e0fd0ebd3731b1d206b075f43a63
740614e3c3 qt: Translations update pre-branch (Wladimir J. van der Laan)
Pull request description:
Update translations from transifex before the 0.20 branch-off. As we don't have translations for the master branch (only for release versions), this is the last opportunity (for this release) to get up to date translations on master.
ACKs for top commit:
elichai:
utACK 740614e3c3
fanquake:
ACK 740614e3c3
Tree-SHA512: 108b82bac967009cab2257d77f0fbc38d18497ffdfe34b06aba35a1894b9b7ebff8ba4671219f4e20ebc42f984ed9ef65c5d41d21ac08dadfef798474f0b2b3b
faa9491870 ci: Use Focal for fuzzers (MarcoFalke)
Pull request description:
This gives us access to clang-10, as well as a newer version of valgrind
ACKs for top commit:
fanquake:
ACK faa9491870 - [Clang 10](https://packages.ubuntu.com/focal/clang) and [valgrind 3.15](https://packages.ubuntu.com/focal/valgrind).
practicalswift:
ACK faa9491870 -- diff looks correct & contemporary clang is better than vintage clang
Tree-SHA512: 0e67232673434c0309db79c1054e3e981115083585945967e346f4d58792635832100f89911428aab928155e44e5f401207a023681ae008fdb5280cf02c4d427
7777703958 doc: Explain new test logging (MarcoFalke)
Pull request description:
Explain logging added in #18472 and #16975
ACKs for top commit:
jonatack:
ACK 7777703
Tree-SHA512: 3a0aa7bab32a6753d8894d29cf82604b044b23e512102dd275b717eefda3c2212dbf43ea7e9155267350dd9f3bc5badba2eb660152db3efeab30a04f52126c95
