Commit graph

24140 commits

Author SHA1 Message Date
MarcoFalke
aa8d76806c
Merge #17946: Fix GBT: Restore "!segwit" and "csv" to "rules" key
412d5fe879 QA: feature_segwit: Check that template "rules" includes "!segwit" as appropriate (Luke Dashjr)
2abe8cc3b7 Bugfix: Include "csv","!segwit" in "rules" (Luke Dashjr)

Pull request description:

  #16060 removed CSV & segwit from versionbits, breaking the "rules" key returned by GBT.

  Without this, miners don't know they're mining segwit blocks, and should fall back to pre-segwit block creation.

ACKs for top commit:
  sipa:
    ACK 412d5fe879
  jnewbery:
    Tested ACK 412d5fe879.

Tree-SHA512: 825d72e257dc0dd4941f2fe498d8d4f4f2a21b9505cd21a8f9eb7fb5d6d7dd9219347928cf90bb57a777920ce24295859763e64fa8a22ebb58fc2380f80f5615
2020-05-19 08:54:23 -04:00
Jonas Schnelli
d44dd51322
Merge #18152: qt: Use SynchronizationState enum for signals to GUI
a0d0f1c6c3 refactor: Remove Node:: queries from GUI (Hennadii Stepanov)
06d519f0b4 qt: Add SynchronizationState enum to signal parameter (Hennadii Stepanov)
3c709aa69d refactor: Remove Node::getReindex() call from GUI (Hennadii Stepanov)
1dab574edf refactor: Pass SynchronizationState enum to GUI (Hennadii Stepanov)
2bec309ad6 refactor: Remove unused bool parameter in RPCNotifyBlockChange() (Hennadii Stepanov)
1df77014d8 refactor: Remove unused bool parameter in BlockNotifyGenesisWait() (Hennadii Stepanov)

Pull request description:

  This PR is a followup of #18121 and:
  - addresses confusion about GUI notification throttling conditions (**luke-jr**'s [comment](https://github.com/bitcoin/bitcoin/pull/18121#discussion_r378552386), **ryanofsky**'s [comment](https://github.com/bitcoin/bitcoin/pull/18121#discussion_r378975960))
  - removes `isInitialBlockDownload()` call from the GUI back to the node (on macOS). See:  **ryanofsky**'s [comment](https://github.com/bitcoin/bitcoin/pull/18121#pullrequestreview-357730284)

ACKs for top commit:
  jonasschnelli:
    Core Review ACK a0d0f1c6c3 (modulo [question](https://github.com/bitcoin/bitcoin/pull/18152#pullrequestreview-414140601)).
  ryanofsky:
    Code review ACK a0d0f1c6c3. Only changes since last review were rebase and tweaking SynchronizationState enum declaration as suggested (thanks!)

Tree-SHA512: b6a712a710666e763aeee0d5440de1391a4c6c8f7fa661888773e1ba59e9e0f83654ee384d4edc704031be7eb25616e5eca2a6e26058d3efb7f64c47f9ed7316
2020-05-19 14:35:02 +02:00
fanquake
e7736854ef
Merge #19008: ci: tsan on clang-9
faf552117e ci: Set DEBIAN_FRONTEND=noninteractive (MarcoFalke)
fa006caa13 ci: tsan on clang-9 (MarcoFalke)

Pull request description:

  Bump the compiler runtime library that includes the sanitizers from clang-8 to clang-9 to get a more recent version. Also, bump the system packages from xenial to bionic to test packages closer to what is commonly used in production.

  The second commit is needed to install the `tzdata` package, which is missing on some operating systems. See https://travis-ci.org/github/MarcoFalke/bitcoin-core/jobs/688455828#L1727

ACKs for top commit:
  hebasto:
    ACK faf552117e
  practicalswift:
    ACK faf552117e -- patch looks correct and Travis is happy

Tree-SHA512: aa38fdae5f716966a83a21d5f7c121675cf7d663148ab3baa065142c8b3850bcd4bf88526d7da0fa51f5e08f2c317b537f950fcc9eb1e69fdacb0eac8863e1c6
2020-05-19 16:10:23 +08:00
fanquake
042ff52142
Merge #18999: log: Remove "No rpcpassword set" from logs
fa243be1dc log: Remove "No rpcpassword set" from logs (MarcoFalke)

Pull request description:

  rpcpassword is deprecated and not recommended anymore. So remove it from the logs, which indicate that an rpcpassword should be set and cause confusion. See #18998.

ACKs for top commit:
  ryanofsky:
    Code review ACK fa243be1dc. New log message makes more sense
  elichai:
    Re Code Review ACK (Checked the diff) fa243be1dc

Tree-SHA512: de3e0800a204b15a59a59a7e6f345013ee9d38e8c5d0c9a94d6142780faa9cce672ed358c7571f53c1eb843bf5afb0b7bcbfd289d3b9e2e0bf8ff2fd361e98a9
2020-05-19 15:41:21 +08:00
fanquake
c73bd004ae
Merge #18861: Do not answer GETDATA for to-be-announced tx
2896c412fa Do not answer GETDATA for to-be-announced tx (Pieter Wuille)
f2f32a3dee Push down use of cs_main into FindTxForGetData (Pieter Wuille)
c6131bf407 Abstract logic to determine whether to answer tx GETDATA (Pieter Wuille)

Pull request description:

  This PR intends to improve transaction-origin privacy.

  In general, we should try to not leak information about what transactions we have (recently) learned about before deciding to announce them to our peers. There is a controlled transaction dissemination process that reveals our transactions to peers that has various safeguards for privacy (it's rate-limited, delayed & batched, deterministically sorted, ...), and ideally there is no way to test which transactions we have before that controlled process reveals them. The handling of the `mempool` BIP35 message has protections in this regard as well, as it would be an obvious way to bypass these protections (handled asynchronously after a delay, also deterministically sorted).

  However, currently, if we receive a GETDATA for a transaction that we have not yet announced to the requester, we will still respond to it if it was announced to *some* other peer already (because it needs to be in `mapRelay`, which only happens on the first announcement). This is a slight privacy leak.

  Thankfully, this seems easy to solve: `setInventontoryTxToSend` keeps track of the txids we have yet to announce to a peer - which almost(*) exactly corresponds to the transactions we know of that we haven't revealed to that peer. By checking whether a txid is in that set before responding to a GETDATA, we can filter these out.

  (*) Locally resubmitted or rebroadcasted transactions may end up in setInventoryTxToSend while the peer already knows we have them, which could result in us incorrectly claiming we don't have such transactions if coincidentally requested right after we schedule reannouncing them, but before they're actually INVed. This is made even harder by the fact that filterInventoryKnown will generally keep known reannouncements out of setInventoryTxToSend unless it overflows (which needs 50000 INVs in either direction before it happens).

  The condition for responding now becomes:

  ```
    (not in setInventoryTxToSend) AND
    (
      (in relay map) OR
      (
        (in mempool) AND
        (old enough that it could have expired from relay map) AND
        (older than our last getmempool response)
      )
    )
  ```

ACKs for top commit:
  naumenkogs:
    utACK 2896c41
  ajtowns:
    ACK 2896c412fa
  amitiuttarwar:
    code review ACK 2896c412fa
  jonatack:
    ACK 2896c412fa per `git diff 2b3f101 2896c41` only change since previous review is moving the recency check up to be verified first in `FindTxForGetData`, as it was originally in 353a391 (good catch), before looking up the transaction in the relay pool.
  jnewbery:
    code review ACK 2896c412fa

Tree-SHA512: e7d5bc006e626f60a2c108a9334f3bbb67205ace04a7450a1e4d4db1d85922a7589e0524500b7b4953762cf70554c4a08eec62c7b38b486cbca3d86321600868
2020-05-19 15:18:06 +08:00
MarcoFalke
faf552117e
ci: Set DEBIAN_FRONTEND=noninteractive
Also fix travis environment variables for this build

Previously the resulting env was:

FILE_ENV="./ci/test/00_setup_env_native_tsan.sh" TEST_RUNNER_EXTRA="--exclude feature_block" #= Not= enough= memory= on= travis= machines=
2020-05-18 20:38:56 -04:00
MarcoFalke
fa006caa13
ci: tsan on clang-9 2020-05-18 20:38:54 -04:00
Hennadii Stepanov
a0d0f1c6c3
refactor: Remove Node:: queries from GUI 2020-05-19 03:01:53 +03:00
Hennadii Stepanov
06d519f0b4
qt: Add SynchronizationState enum to signal parameter 2020-05-19 03:01:42 +03:00
MarcoFalke
362f9c60a5
Merge #18986: tests: Add capability to disable RPC timeout in functional tests
38c3dd9c70 docs: Add notes on how to diasble rpc timeout in functional tests while attatching gdb. (codeShark149)
784ae09625 test: Add capability to disable RPC timeout in functional tests. (codeShark149)

Pull request description:

  Many times, especially while debugging RPC callbacks to core using gdb, the test timeout kicks in before the response can get back. This can be annoying and requires restarting the functional test as well as gdb attachment.

  This PR adds a `--notimeout` flag into `test_framework` and sets the `rpc_timeout` accordingly if the flag is set.

  The same effect can be achieved with newly added `--factor` flag but keeping a separate flag that explicitly disables the timeout can be easier for new testers to find it out and separates its purpose from the `--factor` flag.

  Requesting review ryanofsky jnewbery as per the IRC discussion.

  Update: After initial round of review, the approach is modified to accommodate the functionality in already existing `--factor` flag. `--factor` is changed to `--timeout-factor` to express its intent better.

ACKs for top commit:
  MarcoFalke:
    ACK 38c3dd9c70 and thanks for fixing up all my typos 😅
  jnewbery:
    ACK 38c3dd9c70.

Tree-SHA512: 9458dd1010288c62f8bb83f7a4893284fbbf938882dd65fc9e08810a910db07ef676e3100266028e5d4c8ce407b2267b3860595015da070c84a9d4a9816797db
2020-05-18 20:01:13 -04:00
Hennadii Stepanov
3c709aa69d
refactor: Remove Node::getReindex() call from GUI 2020-05-19 02:49:48 +03:00
Hennadii Stepanov
1dab574edf
refactor: Pass SynchronizationState enum to GUI
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-05-19 02:49:32 +03:00
Hennadii Stepanov
2bec309ad6
refactor: Remove unused bool parameter in RPCNotifyBlockChange() 2020-05-19 02:39:45 +03:00
Hennadii Stepanov
1df77014d8
refactor: Remove unused bool parameter in BlockNotifyGenesisWait() 2020-05-19 02:39:33 +03:00
codeShark149
38c3dd9c70 docs: Add notes on how to diasble rpc timeout in functional tests while attatching gdb. 2020-05-18 21:18:40 +05:30
codeShark149
784ae09625 test: Add capability to disable RPC timeout in functional tests.
Modifies the existing --factor flag to --timeout-factor to better express intent.
Adds rules to disable timeout if --timeout-factor is set to 0.
Modfies --timeout-factor help doc to inform users about this feature.
2020-05-18 21:18:04 +05:30
MarcoFalke
fa243be1dc
log: Remove "No rpcpassword set" from logs 2020-05-17 12:39:26 -04:00
MarcoFalke
dc5333d31f
Merge #18938: tests: Fill fuzzing coverage gaps for functions in consensus/validation.h, primitives/block.h and util/translation.h
cd34038cbd Switch from Optional<T> to std::optional<T> (C++17). Run clang-format. (practicalswift)
fb559c1170 tests: Fill fuzzing coverage gaps for functions in util/translation.h (practicalswift)
b74f3d6c45 tests: Fill fuzzing coverage gaps for functions in consensus/validation.h (practicalswift)
c0bbf8193d tests: Fill fuzzing coverage gaps for functions in primitives/block.h (practicalswift)

Pull request description:

  * Fill fuzzing coverage gaps for functions in `consensus/validation.h`
  * Fill fuzzing coverage gaps for functions in `primitives/block.h`
  * Fill fuzzing coverage gaps for functions in `util/translation.h`
  * Switch from `Optional<T>` to `std::optional<T>` (C++17). Run `clang-format`.

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

Top commit has no ACKs.

Tree-SHA512: d6aa4634c3953ade173589a8239bd230eb317ef897835a8557acb73df01b25e5e17bf46f837838e59ec04c1f3d3b7d1309ba68c8a264d17b938215512c9e6085
2020-05-17 08:19:39 -04:00
MarcoFalke
f8123d483c
Merge #18952: test: avoid os-dependant path
8a22fd0114 avoided os-dependant path (Ferdinando M. Ametrano)

Pull request description:

  The current code fails on windows because of the forward slashes; using os.path.join solves the problem and it is in general more robust

ACKs for top commit:
  MarcoFalke:
    ACK 8a22fd0114

Tree-SHA512: 813f27aea33f97c8afac52e716a55fc5d7fb69621023aba99d40df7e1d145e0ec8d1eee49ddd403b219bf0e0e168e0e987b05c78eaef611f744d99bf2fc8bc91
2020-05-16 06:16:47 -04:00
MarcoFalke
5a454d7882
Merge #18634: ci: Add fuzzbuzz integration configuration file
8d306862ef ci: Add fuzzbuzz integration (practicalswift)

Pull request description:

  Add fuzzbuzz integration.

  Just like #15338 enabled optional FreeBSD building via Cirrus CI (`.cirrus.yml`) this PR adds optional fuzzing via fuzzbuzz (`.fuzzbuzz.yml`).

  Having this merged makes is easier for people to fuzz Bitcoin Core (via their forked repos) using their fuzzbuzz account and then hopefully submit coverage increasing corpus additions upstreams to to https://github.com/bitcoin-core/qa-assets.

  Historically it has been mostly been me and MarcoFalke who submit test cases to `qa-assets`, but with this change hopefully more people will join the hunt for coverage increasing fuzzing inputs :)

Top commit has no ACKs.

Tree-SHA512: c7d8e354996c673da36cc9add260383c82a5325bfaa7ce6141ad6cd6b7d6adf3a6c900ea2db17fb70147b3625fa7f6a1ff8ba813aeaa299f316d8f6cabb3a65c
2020-05-16 06:14:45 -04:00
MarcoFalke
951870807e
Merge #18781: Add templated GetRandDuration<>
0000ea3265 test: Add test for GetRandMillis and GetRandMicros (MarcoFalke)
fa0e5b89cf Add templated GetRandomDuration<> (MarcoFalke)

Pull request description:

  A naive implementation of this template is dangerous, because the call site might accidentally omit the template parameter:

  ```cpp
  template <typename D>
  D GetRandDur(const D& duration_max)
  {
      return D{GetRand(duration_max.count())};
  }

  BOOST_AUTO_TEST_CASE(util_time_GetRandTime)
  {
      std::chrono::seconds rand_hour = GetRandDur(std::chrono::hours{1});
      // Want seconds to be in range [0..1hour), but always get zero :((((
      BOOST_CHECK_EQUAL(rand_hour.count(), 0);
  }
  ```

  Luckily `std::common_type` is already specialised in the standard lib for `std::chrono::duration` (https://en.cppreference.com/w/cpp/chrono/duration/common_type). And its effect seem to be that the call site must always specify the template argument explicitly.

  So instead of implementing the function for each duration type by hand, replace it with a templated version that is safe to use.

ACKs for top commit:
  laanwj:
    Code review ACK 0000ea3265
  promag:
    Code review ACK 0000ea3265.
  jonatack:
    ACK 0000ea3 thanks for the improved documentation. Code review, built, ran `src/test/test_bitcoin -t random_tests -l test_suite` for the new unit tests, `git diff fa05a4c 0000ea3` since previous review:
  hebasto:
    ACK 0000ea3265 with non-blocking [nit](https://github.com/bitcoin/bitcoin/pull/18781#discussion_r424924671).

Tree-SHA512: e89d46e31452be6ea14269ecbbb2cdd9ae83b4412cd14dff7d1084283092722a2f847cb501e8054394e4a3eff852f9c87f6d694fd008b3f7e8458cb5a3068af7
2020-05-15 08:58:49 -04:00
fanquake
e2f6866cca
Merge #18975: test: Remove const to work around compiler error on xenial
050e2ee6f2 test: Remove const to work around compiler error on xenial (Wladimir J. van der Laan)

Pull request description:

  Fix the following error in travis:

      test/validationinterface_tests.cpp:26:36: error: default initialization of an object of const type 'const BlockValidationState' without a user-provided default constructor

      const BlockValidationState state_dummy;

ACKs for top commit:
  MarcoFalke:
    Tested ACK 050e2ee6f2 on xenial with clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
  fanquake:
    ACK 050e2ee6f2 - I see why we didn't hit this on master. We are installing the `clang-8` packages for the tsan job. However on the 0.20 branch we are still just installing `clang`, which is 3.8.

Tree-SHA512: 8a1d57289dbe9895ab79f81ca87b4fd723426b8d72f3a34bec9553226fba69f6dc19551c1f1d52db6c4b2652164a02ddc60f3187c3e2ad7bcacb0aaca7fa690a
2020-05-15 08:05:45 +08:00
practicalswift
cd34038cbd Switch from Optional<T> to std::optional<T> (C++17). Run clang-format. 2020-05-14 18:52:57 +00:00
practicalswift
fb559c1170 tests: Fill fuzzing coverage gaps for functions in util/translation.h 2020-05-14 18:52:57 +00:00
practicalswift
b74f3d6c45 tests: Fill fuzzing coverage gaps for functions in consensus/validation.h 2020-05-14 18:45:42 +00:00
practicalswift
c0bbf8193d tests: Fill fuzzing coverage gaps for functions in primitives/block.h 2020-05-14 18:45:42 +00:00
Wladimir J. van der Laan
553bb3fc3d
Merge #18962: net processing: Only send a getheaders for one block in an INV
746736639e [net processing] Only send a getheaders for one block in an INV (John Newbery)

Pull request description:

  Headers-first is the primary method of announcement on the network. If a node fell back sending blocks by inv, it's probably for a re-org. The final block hash provided should be the highest, so send a getheaders and then fetch the blocks we need to catch up.

  Sending many GETHEADERS messages to the peer would cause them to send a large number of potentially large HEADERS messages with redundant data, which is a waste of bandwidth.

ACKs for top commit:
  sipa:
    utACK 746736639e
  mzumsande:
    utACK 746736639e as per ajtowns' reasoning.
  naumenkogs:
    utACK 7467366
  ajtowns:
    ACK 746736639e
  jonatack:
    ACK 746736639e

Tree-SHA512: 59e243b80d3f0873709dfacb2e4ffba34689aad7de31ec7f69a64e0e3a0756235a0150e4082ff5de823949ba4411ee1aed2344b4749b62e0eb1ea906e41f5ea9
2020-05-14 20:43:45 +02:00
Wladimir J. van der Laan
2d7489be8f
Merge #18796: scripts: security-check.py refactors
eacedfb023 scripts: add additional type annotations to security-check.py (fanquake)
83d063e954 scripts: add run_command to security-check.py (fanquake)
13f606b4f9 scripts: remove NONFATAL from security-check.py (fanquake)
061acf62a1 scripts: no-longer check for 32 bit windows in security-check.py (fanquake)

Pull request description:

  * Remove 32-bit Windows checks.
  * Remove NONFATAL checking. Added in #8249, however unused since #13764.
  * Add `run_command` to de-duplicate all of the subprocess calls. Mentioned in #18713.
  * Add additional type annotations.
  * Print stderr when there is an issue running a command.

ACKs for top commit:
  laanwj:
    ACK eacedfb023

Tree-SHA512: 69a7ccfdf346ee202b3e8f940634c5daed1d2b5a5d15ac9800252866ba3284ec66e391a66a0b341f5a4e5e8482fe1b614d4671e8e766112ff059405081184a85
2020-05-14 20:00:06 +02:00
Wladimir J. van der Laan
4dd2e5255a
Merge #18946: rpcwallet: Replace boost::optional<T>::emplace with simple assignment of T{}
fa1f840596 rpcwallet: Replace pwallet-> with wallet. (MarcoFalke)
fa182a8794 rpcwallet: Replace boost::optional<T>::emplace with simple assignment of T{} (MarcoFalke)

Pull request description:

  Closes #18943

ACKs for top commit:
  laanwj:
    ACK fa1f840596
  ryanofsky:
    Code review ACK fa1f840596 and thanks for using a standalone commit for the fix
  promag:
    Code review ACK fa1f840596.
  hebasto:
    ACK fa1f840596, tested on Linux Mint 19.3.

Tree-SHA512: 0838485d1f93f737ce5bf12740669dcafeebb78dbc3fa15dbcc511edce64bf024f60f0497a04149a1e799d893d57b0c9ffe442020c1b9cfc3c69db731f50e712
2020-05-14 19:26:17 +02:00
Wladimir J. van der Laan
050e2ee6f2 test: Remove const to work around compiler error on xenial
Fix the following error in travis:

    test/validationinterface_tests.cpp:26:36: error: default initialization of an object of const type 'const BlockValidationState' without a user-provided default constructor

    const BlockValidationState state_dummy;
2020-05-14 18:40:57 +02:00
fanquake
b9c504cbc4
Merge #18742: miner: Avoid stack-use-after-return in validationinterface
7777f2a4bb miner: Avoid stack-use-after-return in validationinterface (MarcoFalke)
fa5ceb25fc test: Remove UninterruptibleSleep from test and replace it by SyncWithValidationInterfaceQueue (MarcoFalke)
fa770ce7fe validationinterface: Rework documentation, Rename pwalletIn to callbacks (MarcoFalke)
fab6d060ce test: Add unregister_validation_interface_race test (MarcoFalke)

Pull request description:

  When a validationinterface has itself unregistered in one thread, but is about to get executed in another thread [1], there is a race:

  * The validationinterface destructing itself
  * The validationinterface getting dereferenced for execution

  [1] 64139803f1/src/validationinterface.cpp (L82-L83)

  This happens in the miner. More generally it happens everywhere where at least one thread is generating notifications and another one is unregistering a validationinterface.

  This issue has been fixed in commit ab31b9d6fe, but the fix has not been applied to the miner.

  Example where this happened in practice: https://travis-ci.org/github/bitcoin/bitcoin/jobs/675322230#L4414

ACKs for top commit:
  promag:
    Code review ACK 7777f2a4bb.
  laanwj:
    Code review ACK 7777f2a4bb

Tree-SHA512: 8087119243c71ba18a823a63515f3730d127162625d8729024278b447af29e2ff206f4840ee3d90bf84f93a2c5ab73b76c7e7044c83aa93b5b51047a166ec3d3
2020-05-14 20:40:55 +08:00
fanquake
eacedfb023
scripts: add additional type annotations to security-check.py 2020-05-14 15:30:52 +08:00
fanquake
83d063e954
scripts: add run_command to security-check.py
Deduplicate all the subprocess code as mentioned in 18713.
2020-05-14 15:29:58 +08:00
fanquake
13f606b4f9
scripts: remove NONFATAL from security-check.py 2020-05-14 14:36:27 +08:00
fanquake
061acf62a1
scripts: no-longer check for 32 bit windows in security-check.py 2020-05-14 14:36:27 +08:00
MarcoFalke
7777f2a4bb
miner: Avoid stack-use-after-return in validationinterface
This is achieved by switching to a shared_ptr.

Also, switch the validationinterfaces in the tests to use shared_ptrs
for the same reason.
2020-05-13 19:58:20 -04:00
MarcoFalke
fa5ceb25fc
test: Remove UninterruptibleSleep from test and replace it by SyncWithValidationInterfaceQueue
For the purpose of this test the two have the same outcome, but this one
is shorter and avoids a sleep for 0.1 seconds.
2020-05-13 19:58:11 -04:00
MarcoFalke
fa770ce7fe
validationinterface: Rework documentation, Rename pwalletIn to callbacks 2020-05-13 19:57:55 -04:00
MarcoFalke
fab6d060ce
test: Add unregister_validation_interface_race test
This commit is (intentionally) adding a broken test. The test is broken
because it registering a subscriber object that can go out of scope
while events are still being sent.

To run the broken test and reproduce the bug:
  - Remove comment /** and */
  - ./configure --with-sanitizers=address
  - export ASAN_OPTIONS=detect_leaks=0
  - make
  - while ./src/test/test_bitcoin -t validationinterface_tests/unregister_validation_interface_race --catch_system_errors=no  ; do true; done
2020-05-13 19:57:50 -04:00
Wladimir J. van der Laan
04c09553d8
Merge #18887: build: enable -Werror=gnu
a30b0a24e9 build: enable -Werror=gnu (Vasil Dimov)

Pull request description:

  Stop the build if a warning is emitted due to `-Wgnu` and
  `--enable-werror` has been used. As usual - this would help notice such
  a warning that is about to be introduced in new code.

  This is a followup to
  https://github.com/bitcoin/bitcoin/pull/18088 build: ensure we aren't using GNU extensions

ACKs for top commit:
  practicalswift:
    ACK a30b0a24e9
  Empact:
    ACK a30b0a24e9

Tree-SHA512: f81b71cf3ee4db88b6f664c571075e0d30800a604f067f44273f256695a1dea533779db2ac859dd0a4cd8b66289c3e45f4aff1cfadfa160a1c354237167b05e2
2020-05-13 22:20:13 +02:00
MarcoFalke
a9a6d946e4
Merge #18888: test: Remove RPCOverloadWrapper boilerplate
faa26d3744 test: Remove RPCOverloadWrapper boilerplate (MarcoFalke)

Pull request description:

  There are too many wrappers in test_node already, so at least the code that implements the wrappers should be as minimal as possible.

ACKs for top commit:
  laanwj:
    code review ACK faa26d3744

Tree-SHA512: 94e593907de22187524e2445afb3101e40b3b599d4b4015aa8c6ca902d7586ff9daf520828759029d199a3af79e61b96b490a822a5a193ac7bf946beacb11a24
2020-05-13 15:36:10 -04:00
Jonas Schnelli
51825aea7f
Merge #18922: gui: Do not translate InitWarning messages in debug.log
78be8d97d3 util: Drop OpOriginal() and OpTranslated() (Hennadii Stepanov)
da16f95c3f gui: Do not translate InitWarning messages in debug.log (Hennadii Stepanov)
4c9b9a4882 util: Enhance Join() (Hennadii Stepanov)
fe05dd0611 util: Enhance bilingual_str (Hennadii Stepanov)

Pull request description:

  This PR forces the `bitcoin-qt` to write `InitWarning()` messages to the `debug.log` file in untranslated form, i.e., in English.

  On master (376294cde6):
  ```
  $ ./src/qt/bitcoin-qt -lang=nl -debug=vladidation -printtoconsole | grep 'vladi'
  Warning: Niet-ondersteunde logcategorie -debug=vladidation.
  2020-05-09T12:39:59Z Warning: Niet-ondersteunde logcategorie -debug=vladidation.
  2020-05-09T12:40:02Z Command-line arg: debug="vladidation"
  ```

  With this PR:
  ```
  $ ./src/qt/bitcoin-qt -lang=nl -debug=vladidation -printtoconsole | grep 'vladi'
  Warning: Unsupported logging category -debug=vladidation.
  2020-05-09T12:42:04Z Warning: Unsupported logging category -debug=vladidation.
  2020-05-09T12:42:35Z Command-line arg: debug="vladidation"
  ```

  ![Screenshot from 2020-05-09 15-42-31](https://user-images.githubusercontent.com/32963518/81474073-c7a50e00-920b-11ea-8775-c41122dacafe.png)

  Related to #16218.

ACKs for top commit:
  laanwj:
    ACK 78be8d97d3
  jonasschnelli:
    utACK 78be8d97d3
  MarcoFalke:
    ACK 78be8d97d3 📢

Tree-SHA512: 48e9ecd23c4dd8ec262e3eb94f8e30944bcc9c6c163245fb837b2e0c484d4d0b4f47f7abc638c14edc27d635d340ba3ee4ba4506b062399e9cf59a1564c98755
2020-05-13 20:30:39 +02:00
Wladimir J. van der Laan
fc895d7700
Merge #18616: refactor: Cleanup clientversion.cpp
c269e618cf Drop unused GIT_COMMIT_DATE macro (Hennadii Stepanov)
8f9f4ba5e2 refactor: Remove duplicated code (Hennadii Stepanov)
35f1189ea7 build: Rename BUILD_* macros and the code self-descriptive (Hennadii Stepanov)
dc1fba9389 scripted-diff: Rename share/genbuild.sh macros to more meaningful ones (Hennadii Stepanov)
1e06bb68be Drop unused CLIENT_VERSION_SUFFIX macro (Hennadii Stepanov)

Pull request description:

  This PR:
  - removes unused macros and duplicated code
  - renames macros in a way, that makes the code self-descriptive.

ACKs for top commit:
  dongcarl:
    Yup! ACK c269e618cf

Tree-SHA512: c469f6269b578ccfae33d960e317eca8efaf27d49638f4c3830948c11b12ef728494d7e18c31e4a410945b7d83af5b246c7b83661b4eca17cf41ee4c4583649b
2020-05-13 20:14:51 +02:00
Wladimir J. van der Laan
5d18c0ae18
Merge #18862: Remove fdelt_chk back-compat code and sanity check
df6bde031b test: remove glibc fdelt sanity check (fanquake)
8bf1540cc2 build: remove fdelt_chk backwards compatibility code (fanquake)

Pull request description:

  ae30d40e50
  The return type of [`fdelt_chk`](https://sourceware.org/git/?p=glibc.git;a=blob;f=debug/fdelt_chk.c;h=f62ce7349707cb68f55831c1c591fd7387a90258;hb=HEAD) changed from `unsigned  long int` to `long int` in glibc 2.16. See [this commit](https://sourceware.org/git/?p=glibc.git;a=commit;h=ceb9e56b3d1f8c1922e0526c2e841373843460e2). Now that we require [glibc >=2.17](https://github.com/bitcoin/bitcoin/pull/17538) we can remove our back-compat code.

  ab7bce584a
  While looking at the above changes, I noticed that our glibc fdelt sanity check doesn't seem to be checking anything. `fdelt_warn()` also isn't something we'd want to actually "trigger" at runtime, as doing so would cause `bitcoind` to abort.

  The comments:
  > // trigger: Call FD_SET to trigger __fdelt_chk. FORTIFY_SOURCE must be defined
  > //   as >0 and optimizations must be set to at least -O2.

  suggest calling FD_SET to check the invocation of `fdelt_chk` (this is [aliased with fdelt_warn in glibc](https://sourceware.org/git/?p=glibc.git;a=blob;f=debug/fdelt_chk.c;h=f62ce7349707cb68f55831c1c591fd7387a90258;hb=HEAD)). However just calling `FD_SET()` will not necessarily cause the compiler to insert a call to `fd_warn()`.

  Whether or not GCC (recent Clang should work, but may use different heuristics) inserts a call to `fdelt_warn()` depends on if the compiler can determine if the value passed in is a compile time constant (using [`__builtin_constant_p`](https://gcc.gnu.org/onlinedocs/gcc/Other-Builtins.html)) and whether the value is < 0 or >= `FD_SETSIZE`. The glibc implementation is [here](https://sourceware.org/git/?p=glibc.git;a=blob;f=misc/bits/select2.h;h=7e17430ed94dd1679af10afa3d74795f9c97c0e8;hb=HEAD). This means our check should never cause a call to be inserted.

  Compiling master without `--glibc-back-compat` (if you do pass `--glibc-back-compat` the outcome is still the same; however the abort will only happen with >=`FD_SETSIZE` as that is what our [fdelt_warn()](https://github.com/bitcoin/bitcoin/blob/master/src/compat/glibc_compat.cpp#L24) checks for), there are no calls to `fdelt_warn()` inserted by the compiler:
  ```bash
  objdump -dC bitcoind | grep sanity_fdelt
  ...
  0000000000399d20 <sanity_test_fdelt()>:
    399d20:       48 81 ec 98 00 00 00    sub    $0x98,%rsp
    399d27:       b9 10 00 00 00          mov    $0x10,%ecx
    399d2c:       64 48 8b 04 25 28 00    mov    %fs:0x28,%rax
    399d33:       00 00
    399d35:       48 89 84 24 88 00 00    mov    %rax,0x88(%rsp)
    399d3c:       00
    399d3d:       31 c0                   xor    %eax,%eax
    399d3f:       48 89 e7                mov    %rsp,%rdi
    399d42:       fc                      cld
    399d43:       f3 48 ab                rep stos %rax,%es:(%rdi)
    399d46:       48 8b 84 24 88 00 00    mov    0x88(%rsp),%rax
    399d4d:       00
    399d4e:       64 48 33 04 25 28 00    xor    %fs:0x28,%rax
    399d55:       00 00
    399d57:       75 0d                   jne    399d66 <sanity_test_fdelt()+0x46>
    399d59:       b8 01 00 00 00          mov    $0x1,%eax
    399d5e:       48 81 c4 98 00 00 00    add    $0x98,%rsp
    399d65:       c3                      retq
    399d66:       e8 85 df c8 ff          callq  27cf0 <__stack_chk_fail@plt>
    399d6b:       0f 1f 44 00 00          nopl   0x0(%rax,%rax,1)

  ```

  If you modify the sanity test to pass `-1` or `FD_SETSIZE` to `FD_SET`, you'll see calls to `fdelt_warn` inserted, and the runtime behaviour is an abort as expected.

  ```diff
  diff --git a/src/compat/glibc_sanity_fdelt.cpp b/src/compat/glibc_sanity_fdelt.cpp
  index 87140d0c7..16974bfa0 100644
  --- a/src/compat/glibc_sanity_fdelt.cpp
  +++ b/src/compat/glibc_sanity_fdelt.cpp
  @@ -20,7 +20,7 @@ bool sanity_test_fdelt()
   {
       fd_set fds;
       FD_ZERO(&fds);
  -    FD_SET(0, &fds);
  +    FD_SET(FD_SETSIZE, &fds);
       return FD_ISSET(0, &fds);
   }
   #endif
  ```

  ```bash
  0000000000399d20 <sanity_test_fdelt()>:
    399d20:	48 81 ec 98 00 00 00 	sub    $0x98,%rsp
    399d27:	b9 10 00 00 00       	mov    $0x10,%ecx
    399d2c:	64 48 8b 04 25 28 00 	mov    %fs:0x28,%rax
    399d33:	00 00
    399d35:	48 89 84 24 88 00 00 	mov    %rax,0x88(%rsp)
    399d3c:	00
    399d3d:	31 c0                	xor    %eax,%eax
    399d3f:	48 89 e7             	mov    %rsp,%rdi
    399d42:	fc                   	cld
    399d43:	f3 48 ab             	rep stos %rax,%es:(%rdi)
    399d46:	48 c7 c7 ff ff ff ff 	mov    $0xffffffffffffffff,%rdi
    399d4d:	e8 3e ff ff ff       	callq  399c90 <__fdelt_warn>
    399d52:	0f b6 04 24          	movzbl (%rsp),%eax
    399d56:	83 e0 01             	and    $0x1,%eax
    399d59:	48 8b 94 24 88 00 00 	mov    0x88(%rsp),%rdx
    399d60:	00
    399d61:	64 48 33 14 25 28 00 	xor    %fs:0x28,%rdx
    399d68:	00 00
    399d6a:	75 08                	jne    399d74 <sanity_test_fdelt()+0x54>
    399d6c:	48 81 c4 98 00 00 00 	add    $0x98,%rsp
    399d73:	c3                   	retq
    399d74:	e8 77 df c8 ff       	callq  27cf0 <__stack_chk_fail@plt>
    399d79:	0f 1f 80 00 00 00 00 	nopl   0x0(%rax)
   ```

   ```bash
   src/bitcoind
  *** buffer overflow detected ***: src/bitcoind terminated
  Aborted
   ```

  I think the test should should be removed and replaced (if possible) with additional checks in security-check.py. I was thinking about adding a version of [this script](https://github.com/fanquake/core-review/blob/master/fortify.py) as part of the output, but that needs more thought. I'll address this in a follow up.

ACKs for top commit:
  laanwj:
    ACK  df6bde031b

Tree-SHA512: d8b3af4f4eb2d6c767ca6e72ece51d0ab9042e1bbdfcbbdb7ad713414df21489ba3217662b531b8bfdac0265d2ce5431abfae6e861b6187d182ff26c6e59b32d
2020-05-13 19:35:25 +02:00
Wladimir J. van der Laan
99c03728c0
Merge #18878: test: Add test for conflicted wallet tx notifications
f963a68051 test: Add test for conflicted wallet tx notifications (Russell Yanofsky)

Pull request description:

  Add test coverage for conflicted wallet transaction notifications so we can improve current behavior and avoid future regressions

  https://github.com/bitcoin/bitcoin/pull/9240 - accidental break
  https://github.com/bitcoin/bitcoin/issues/9479 - bug report
  https://github.com/bitcoin/bitcoin/pull/9371 - fix
  https://github.com/bitcoin/bitcoin/pull/16624 - accidental break
  https://github.com/bitcoin/bitcoin/issues/18325 - bug report
  https://github.com/bitcoin/bitcoin/pull/18600 - potential fix

ACKs for top commit:
  laanwj:
    ACK f963a68051
  jonatack:
    re-ACK f963a68051
  MarcoFalke:
    ACK f963a68051

Tree-SHA512: d3a7952a2d3dc2ff0800ef857575ea4ef9759c0917d58a7fc91e2db0ca3cc3baf0dd0cf9af61683f691e5fefb11afe8120bb5810c7037ed9ecedee385dd4aa07
2020-05-13 15:52:58 +02:00
fanquake
6c647c89db
Merge #18738: build: Suppress -Wdeprecated-copy warnings
0c63f80854 build: Suppress -Wdeprecated-copy warnings (Hennadii Stepanov)

Pull request description:

  Tomorrow, on Apr 23 the Ubuntu 20.04 release is expected. It packaged with Qt 5.12 LTS that has a nasty peculiarity to cause modern compilers, including Clang 10.0 and GCC 9.3, to emit spammy `-Wdeprecated-copy` warnings (#15822, #18419).

  This PR suppress such warnings _temporarily_, until the [upstream is fixed](https://codereview.qt-project.org/c/qt/qtbase/+/272258).

  Here are some affected systems (with system packages):
  - Ubuntu 20.04 LTS + Qt 5.12.8 LTS + { Clang 10.0 | GCC 9.3 }
  - Fedora 32 + Qt 5.13.2 + Clang 10.0

  Reference: [QTBUG-75210](https://bugreports.qt.io/browse/QTBUG-75210)

  Also see **fanquake**'s [comment](https://github.com/bitcoin/bitcoin/pull/18738#issuecomment-622956100).

ACKs for top commit:
  MarcoFalke:
    ACK 0c63f80854 seems fine to disable this warning for the 0.21.0 release temporarily and then enable it for 0.22.0, when boost is removed.
  fanquake:
    ACK 0c63f80854 - I think it's ok to suppress these for now, given that `-Wdeprecated-copy` is enabled (via `-Wextra`) in GCC 9 and Clang 10. The Qt output is pretty noisy, and there's a few warnings from Boost as well.

Tree-SHA512: 7064a3272bc9eae00b73a16c421ac58be148f374cbef87320e8f092f52761f6e98166eff60346b70867f8a69a9698a79455dc16b42d92f8fbe7c56519571ac08
2020-05-13 21:17:07 +08:00
fanquake
a33901cb6d
Merge #18814: rpc: Relock wallet only if most recent callback
9f59dde974 rpc: Relock wallet only if most recent callback (João Barbosa)
a2e6db5c4f rpc: Add mutex to guard deadlineTimers (João Barbosa)

Pull request description:

  This PR fixes an early relocking race condition from #18811 where old relock callback runs after new wallet unlock code and nRelockTime update but before rpcRunLater call, causing early relock and incorrect nRelockTime time

  Issue introduced in #18487.
  Fixes #18811.

ACKs for top commit:
  MarcoFalke:
    ACK 9f59dde974
  ryanofsky:
    Code review ACK 9f59dde974. No changes since last review except squashing commits.
  jonatack:
    ACK 9f59dde974

Tree-SHA512: 2f7fc03e5ab6037337f2d82dfad432495cc337c77d07c968ee2355105db6292f24543c03456f5402e0e759577a4327758f9372f7ea29de6d56dc3695fda9b379
2020-05-13 17:36:06 +08:00
Jonas Schnelli
246e878e78
Merge #18894: gui: Fix manual coin control with multiple wallets loaded
a8b5f1b133 gui: Fix manual coin control with multiple wallets loaded (João Barbosa)

Pull request description:

  This PR ensures each loaded wallet has a dedicated coin control in the send view which is manipulated by the coin control dialog.

  This is an alternative to #17457. Two main differences are:
   - scope reduced - no unnecessary changes unrelated to the fix;
   - approach taken - coin control instance now belongs to the send view.

  All problems raised in #17457 reviews no longer apply due to the approach taken - https://github.com/bitcoin/bitcoin/pull/17457#pullrequestreview-319297589 and https://github.com/bitcoin/bitcoin/pull/17457#issuecomment-555920829)

  No change in behavior if only one wallet is loaded.

  Closes #15725.

ACKs for top commit:
  jonasschnelli:
    utACK a8b5f1b133
  ryanofsky:
    Code review ACK a8b5f1b133. Code changes are very straightforward, just replacing global CCoinControl object with SendCoinsDialog member. Not sure if this means coin control settings are reset between payments. It would be good to note in the PR description or release notes if single wallet behavior is affected
  hebasto:
    ACK a8b5f1b133
  Sjors:
    tACK a8b5f1b133

Tree-SHA512: 3ad9c51bab6f28ec0e90efbd6f43fa510c81dafb2eff0b8c3724efcee3e030054a10be013e27cefe35763374c5f6d7af8c02658736964f733d7e38b646b5df65
2020-05-13 10:15:32 +02:00
Jonas Schnelli
8d17f8dc17
Merge #18578: gui: Fix leak in CoinControlDialog::updateView
e8123eae40 gui: Fix itemWalletAddress leak when not tree mode (João Barbosa)

Pull request description:

  Taken from #17457, the first commit is a similar to 88a94f7bb8 but for test binary, and the second commit fixes a leak where `CCoinControlWidgetItem` are unnecessarily created and leaked.

ACKs for top commit:
  jonasschnelli:
    utACK e8123eae40
  hebasto:
    ACK e8123eae40, tested on Linux Mint 19.3.

Tree-SHA512: 8b43cb29de103842ce5f048de51222919540d3212d2873c16731145e856178644041924ad0e9a58c2ff08f209a9b4ac26dc9965289eb719da233c0984f93631e
2020-05-13 10:13:06 +02:00
fanquake
219c55da75
Merge #16710: build: Enable -Wsuggest-override if available
839add193b build: Enable -Wsuggest-override (Hennadii Stepanov)
de5e91c303 refactor: Add BerkeleyDatabaseVersion() function (Hennadii Stepanov)

Pull request description:

  From GCC [docs](https://gcc.gnu.org/onlinedocs/gcc-9.2.0/gcc/Warning-Options.html):
  > `-Wsuggest-override`
  > Warn about overriding virtual functions that are not marked with the override keyword.

  ~This PR is based on #16722 (the first commit).~ See: https://github.com/bitcoin/bitcoin/pull/16722#issuecomment-584111086

ACKs for top commit:
  fanquake:
    ACK 839add193b
  vasild:
    ACK 839add193
  practicalswift:
    ACK 839add193b assuming Travis is happy: patch looks correct

Tree-SHA512: 1e8cc085da30d41536deff9b181962c1882314ab252c2ad958294087ae1e5a0dfa4886bdbe36f21cf6ae71df776a8420f349f007d4b5b49fd79ba98ce308965a
2020-05-13 15:19:05 +08:00