Commit graph

2868 commits

Author SHA1 Message Date
gzhao408
158a0b268c Apply maximal validation flags to tx_valid tests
- Apply all validation flags by default
- Invert the meaning of verifyFlags as flags being excluded

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
0a76a39b63 [test] fix CSV test missing OP_ADD
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
19db590d04 [test] remove unnecessary OP_1s from CSV and CLTV tests
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
fanquake
8e55981ef8
refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests
Co-authored-by: MarcoFalke falke.marco@gmail.com
Co-authored-by: sinetek pitwuu@gmail.com
2021-02-02 12:35:40 +08:00
Wladimir J. van der Laan
2c0fc856a6
Merge #20464: refactor: Treat CDataStream bytes as uint8_t
fa29272459 Remove redundant MakeUCharSpan wrappers (MarcoFalke)
faf4aa2f47 Remove CDataStream::Init in favor of C++11 member initialization (MarcoFalke)
fada14b948 Treat CDataStream bytes as uint8_t (MarcoFalke)
fa8bdb048e refactor: Drop CDataStream constructors in favor of one taking a Span of bytes (MarcoFalke)
faa96f841f Remove unused CDataStream methods (MarcoFalke)

Pull request description:

  Using `uint8_t` for raw bytes has a style benefit:
  * The signedness is clear from reading the code, as it does not depend on the architecture

  Other clean-ups in this pull include:
  * Remove unused methods
  * Constructor is simplified with `Span`
  * Remove `Init()` member in favor of C++11 member initialization

ACKs for top commit:
  laanwj:
    code review ACK fa29272459
  theStack:
    ACK fa29272459 🍾

Tree-SHA512: 931ee28bd99843d7e894b48e90e1187ffb0278677c267044b3c0c255069d9bbd9298ab2e539b1002a30b543d240450eaec718ef4ee95a7fd4be0a295e926343f
2021-02-01 15:17:28 +01:00
Wladimir J. van der Laan
d0d256536c
Merge #21016: refactor: remove boost::thread_group usage
dc8be12510 refactor: remove boost::thread_group usage (fanquake)

Pull request description:

  Post #18710, there isn't much left using `boost::thread_group`, so should just be able to replace it with the standard library. This also removes the last use of `boost::thread_interrupted`.

  After this change, last piece of Boost Thread we'd be using is `boost::shared_mutex`. See the commentary [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) as to why it may be non-trivial to swap that for `std::shared_mutex` in the near future.

  Closes #17307

ACKs for top commit:
  laanwj:
    Code review re-ACK dc8be12510
  MarcoFalke:
    review ACK dc8be12510 🔁
  jonatack:
    Non-expert code review ACK dc8be12510, also checked range-diff since last review and that local debug build is clean with gcc 10.2.1-6 on Debian

Tree-SHA512: 5510e2d760cce824234207dc86b1551ca8f21cbf3a2ce753c0254a0d03ffd83c94e449aec202fb7bd76e6fc64df783a6b70a736b0add9ece3734bb9c8ce8fc2f
2021-02-01 13:27:28 +01:00
Wladimir J. van der Laan
44f4bcd302
Merge #20749: [Bundle 1/n] Prune g_chainman usage related to ::LookupBlockIndex
67c9a83df1 style-only: Remove redundant sentence in ActivateBestChain comment (Carl Dong)
b8e95658d5 style-only: Make TestBlockValidity signature readable (Carl Dong)
0cdad75390 validation: Use accessible chainstate in ChainstateManager::ProcessNewBlock (Carl Dong)
ea4fed9021 validation: Use existing chainstate in ChainstateManager::ProcessNewBlockHeaders (Carl Dong)
e0dc305727 validation: Move LoadExternalBlockFile to CChainState (Carl Dong)
5f8cd7b3a5 validation: Remove global ::ActivateBestChain (Carl Dong)
2a696472a1 validation: Pass in chainstate to ::NotifyHeaderTip (Carl Dong)
9c300cc8b3 validation: Pass in chainstate to TestBlockValidity (Carl Dong)
0e17c833cd validation: Make CChainState.m_blockman public (Carl Dong)
d363d06bf7 validation: Pass in blockman to ContextualCheckBlockHeader (Carl Dong)
f11d11600d validation: Move GetLastCheckpoint to BlockManager (Carl Dong)
e4b95eefbc validation: Move GetSpendHeight to BlockManager (Carl Dong)
b026e318c3 validation: Move FindForkInGlobalIndex to BlockManager (Carl Dong)
3664a150ac validation: Remove global LookupBlockIndex (Carl Dong)
eae54e6e60 scripted-diff: Use BlockManager::LookupBlockIndex (Carl Dong)
15d20f40e1 validation: Move LookupBlockIndex to BlockManager (Carl Dong)
f92dc6557a validation: Guard the active_chainstate with cs_main (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

ACKs for top commit:
  jnewbery:
    utACK 67c9a83df1
  laanwj:
    re-ACK 67c9a83df1
  ryanofsky:
    Code review ACK 67c9a83df1. Changes since last review:

Tree-SHA512: 8744aba2dd57a40cd2fedca809b0fe24d771bc60da1bffde89601999384aa0df428057a86644a3f72fbeedbc8b04db6c4fd264ea0db2e73c279e5acc6d056cbf
2021-02-01 13:09:46 +01:00
Sebastian Falbesoner
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17)
Removes the macro ARRAYLEN and also substitutes all other uses of the same
"sizeof(a)/sizeof(a[0])" pattern by std::size, available since C++17.
2021-01-31 17:35:16 +01:00
Sebastian Falbesoner
63d4ee1968 refactor: iterate arrays via C++11 range-based for loops if idx is not needed 2021-01-31 17:07:46 +01:00
practicalswift
dee2d6fbf9 fuzz: Avoid designated initialization (C++20) in fuzz tests 2021-01-30 08:22:32 +00:00
fanquake
dc8be12510
refactor: remove boost::thread_group usage 2021-01-29 15:39:44 +08:00
Carl Dong
e0dc305727 validation: Move LoadExternalBlockFile to CChainState
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

LoadExternalBlockFile mainly acts on CChainState.
2021-01-28 14:15:26 -05:00
Carl Dong
5f8cd7b3a5 validation: Remove global ::ActivateBestChain
Instead use CChainState::ActivateBestChain, which is what the global one
calls anyway.
2021-01-28 14:15:26 -05:00
Carl Dong
eae54e6e60 scripted-diff: Use BlockManager::LookupBlockIndex
[META] In a previous commit, we moved ::LookupBlockIndex to become a
       member function of BlockManager. This commit is split out from
       that one since it can be expressed nicely as a scripted-diff.

-BEGIN VERIFY SCRIPT-
find_regex='LookupBlockIndex' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E "s@${find_regex}@g_chainman.m_blockman.LookupBlockIndex@g"
-END VERIFY SCRIPT-
2021-01-28 14:15:26 -05:00
MarcoFalke
4d5eaf7a90
Merge #20995: fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION
fad3d7625a fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION (MarcoFalke)
fa99e33aeb fuzz: move-only FillNode implementation to cpp file (MarcoFalke)

Pull request description:

  This fixes a fuzz bug introduced in #20881. Previously the nodes in the fuzz tests had their version initialized to a constant (`PROTOCOL_VERSION`). After #20881, the nodes have their version initialized to an arbitrary signed integer. This is problematic for several reasons:

  * Both `nVersion` and `m_greatest_common_version` may be initialized to `0`. If a `version` message is processed, this leads to a crash, because `m_greatest_common_version` must be `INIT_PROTO_VERSION` while the `version` message is processed. See #20138
  * The "valid" range for `nVersion` is `[MIN_PEER_PROTO_VERSION, std::numeric_limits<int32_t>::max()]` (see check in net_processing)
  * The "valid" range for `m_greatest_common_version` is `std::min(nVersion, PROTOCOL_VERSION)` (see net_processing)

  Fix all issues by initializing `nVersion` and `m_greatest_common_version` to their valid ranges.

  -----

  The crashers, if someone wants to try this at home:

  ```
  ( echo 'dmVyc2lvbgAWFhYWFhYWFhYWFhYWFhYWFhYWFhZp/29uAPX//xYWFhYWFhYWFhYWFhYWFhYWFhYW
  FhYWFhYWaW9uAOr1//8WFhYWFha0ZXJzaW9uAPX//wAAAAAAABAAAAAAAAAAAAC0ZXJzaW9uAPX/
  /wBPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08AAAAAABAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAB2ZXJzaW9uAACDJIO9vXYKAAAAAAAAAAAAAAAAAAAAAAB2ZfS1qmu1qhUVFWs=' | base64 --decode > /tmp/a ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/a
  ```
  ```
  ( echo 'dmVyc2lvbgD//wAhTmiqN///NDcAAACENDL/iv//8DYAAHL///////79/RtcAJqamhqa/QEAAAD/
  ///+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZyq8SaGVhZGVycwAAAAD/NDcAAACENDL/iv//
  8DYAAHL///////79/RtcAJqamhqa/QEAAAD////+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZ
  yq8SaGVhZGVycwAAAADPAQAAACAGIm5GERoLWS1wb3J061u/KMNPOkwFXqZ///b5IgIAAD+5ubkb
  XD5hZGRyAJqamhqasP0BAAAAAAAAAP0BAAAAIf39/R0dHQAAAAAAMgAA///7//+gXqZ///b5IgIA
  AD+5ubm5ubm5AAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAFgAAAAAAAAAAAAlBmv39/f1/f39B
  f39hZGRyAG5vAACaLgAdGzY2zwEAAAAgBiJuRhEaC1ktcG9ydOtbvyjDTzpMBV6mf//2+SICAAA/
  ubm5G1w+YWRkcgCampoamrD9AQAAAAAAAAD9AQAAACH9/f0dHR0AAAAAADIAAP//+///oF6mf//2
  +SICAAA/ubm5ubm5uQAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAABYAAAAAAAAAAAAJQZr9/f39
  f39/QX9/YWRkcgBubwAAmi4AHRs2NjY2NjY2NjYCAgI2NgIA/f39/f39Nv39/TUmABxc' | base64 --decode > /tmp/b ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/b
  ```

ACKs for top commit:
  practicalswift:
    cr ACK fad3d7625a

Tree-SHA512: ea64ee99b94d8e619e3949d2d21252c1236412c0e40f44f2b73595ca70cd2da0bdab005fb1a54f65fb291e7b07fdd33577ce4a3a078ca933246b511ebcb0e52a
2021-01-28 15:13:25 +01:00
MarcoFalke
e130ff38c9
Merge bitcoin-core/gui#183: Add include for std::bind.
2a39ccf133 Add include for std::bind. (sinetek)

Pull request description:

  Hi, this patch adds in <functional> because the GUI code makes use of std::bind.
  That's all.

ACKs for top commit:
  jonasschnelli:
    utACK 2a39ccf133

Tree-SHA512: fb5ac07d9cd5d006182b52857b289a9926362a2f1bfa4f7f1c78a088670e2ccf39ca28214781df82e8de3909fa3e69685fe1124a7e3ead758575839f5f2277a9
2021-01-26 11:14:27 +01:00
MarcoFalke
32d44d2b1c
Merge #21000: fuzz: Add UBSan suppressions needed for fuzz tests to not warn under -fsanitize=integer
f0f8b1a076 fuzz: Add UBSan suppressions needed for fuzz tests to not warn under -fsanitize=integer (practicalswift)
58232e3ffb fuzz: Avoid -fsanitize=integer warnings in fuzzing harnesses (practicalswift)

Pull request description:

  Add UBSan suppressions needed for fuzz tests to not warn under `-fsanitize=integer`.

  Avoid `-fsanitize=integer` warnings in fuzzing harnesses.

  Suppressed warnings (excluding warnings from `src/crypto/` and `src/test/`):

  ```
  addrman.cpp:306:24: runtime error: implicit conversion from type 'long' of value 5190149478 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 895182182 (32-bit, unsigned)
  addrman.h:446:43: runtime error: implicit conversion from type 'int' of value -22 (32-bit, signed) to type 'const uint8_t' (aka 'const unsigned char') changed the value to 234 (8-bit, unsigned)
  arith_uint256.cpp:32:35: runtime error: left shift of 1712128 by 24 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
  arith_uint256.cpp:47:39: runtime error: left shift of 4294966784 by 31 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
  chain.cpp:151:12: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'unsigned long' changed the value to 18446744073709551615 (64-bit, unsigned)
  coins.cpp:114:22: runtime error: unsigned integer overflow: 0 - 96 cannot be represented in type 'unsigned long'
  compressor.cpp:162:33: runtime error: unsigned integer overflow: 15617702637291228364 * 10 cannot be represented in type 'unsigned long'
  compressor.cpp:188:11: runtime error: unsigned integer overflow: 2265760372865400000 * 10 cannot be represented in type 'unsigned long'
  hash.cpp:13:15: runtime error: left shift of 1692305888 by 15 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
  pubkey.h:152:23: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'
  streams.h:570:31: runtime error: left shift of 350879 by 52 places cannot be represented in type 'uint64_t' (aka 'unsigned long')
  util/bip32.cpp:57:36: runtime error: left shift of 3241096244 by 1 places cannot be represented in type 'unsigned int'
  util/strencodings.cpp:562:38: runtime error: implicit conversion from type 'unsigned char' of value 255 (8-bit, unsigned) to type 'char' changed the value to -1 (8-bit, signed)
  util/strencodings.h:164:24: runtime error: implicit conversion from type 'int' of value -74 (32-bit, signed) to type 'unsigned long' changed the value to 18446744073709551542 (64-bit, unsigned)
  ```

  The warnings above happen here:

  32b191fb66/src/addrman.cpp (L306)
  32b191fb66/src/addrman.h (L446)
  32b191fb66/src/arith_uint256.cpp (L32)
  32b191fb66/src/arith_uint256.cpp (L47)
  32b191fb66/src/chain.cpp (L151)
  32b191fb66/src/coins.cpp (L114)
  32b191fb66/src/compressor.cpp (L162)
  32b191fb66/src/compressor.cpp (L188)
  32b191fb66/src/hash.cpp (L13)
  32b191fb66/src/pubkey.h (L152)
  32b191fb66/src/streams.h (L570)
  32b191fb66/src/util/bip32.cpp (L57)
  32b191fb66/src/util/strencodings.cpp (L562)
  32b191fb66/src/util/strencodings.h (L164)

ACKs for top commit:
  MarcoFalke:
    review ACK f0f8b1a076 🤚

Tree-SHA512: a8f04f7cc055d03653161de1d9d14d106a6280cea1e86a1243abcd57cf8e61dcf5f731d0ab0da5b390790e816022ff7a70759a641463bc7e3303076b8667009f
2021-01-26 10:00:43 +01:00
fanquake
f827e151a2
refactor: remove straggling boost::mutex usage
After the merge of #18710, the linter is warning:
```bash
A new Boost dependency in the form of "boost/thread/mutex.hpp" appears to have been introduced:
src/sync.cpp:#include <boost/thread/mutex.hpp>
src/test/sync_tests.cpp:#include <boost/thread/mutex.hpp>

^---- failure generated from test/lint/lint-includes.sh
```

the interim #19337 was merged, which introduced more `boost::mutex` usage.

Given we no longer use `boost::mutex`, just remove the double lock test
and remaining includes.
2021-01-26 15:57:28 +08:00
practicalswift
58232e3ffb fuzz: Avoid -fsanitize=integer warnings in fuzzing harnesses 2021-01-25 20:55:36 +00:00
Wladimir J. van der Laan
b386d37360
Merge #18710: Add local thread pool to CCheckQueue
bb6fcc75d1 refactor: Drop boost::thread stuff in CCheckQueue (Hennadii Stepanov)
6784ac471b bench: Use CCheckQueue local thread pool (Hennadii Stepanov)
dba30695fc test: Use CCheckQueue local thread pool (Hennadii Stepanov)
01511776ac Add local thread pool to CCheckQueue (Hennadii Stepanov)
0ef938685b refactor: Use member initializers in CCheckQueue (Hennadii Stepanov)

Pull request description:

  This PR:
  - gets rid of `boost::thread_group` in the `CCheckQueue` class
  - allows thread safety annotation usage in the `CCheckQueue` class
  - is alternative to #14464 (https://github.com/bitcoin/bitcoin/pull/18710#issuecomment-616618525, https://github.com/bitcoin/bitcoin/pull/18710#issuecomment-617291612)

  Also, with this PR (I hope) it could be easier to resurrect a bunch of brilliant ideas from #9938.

  Related: #17307

ACKs for top commit:
  laanwj:
    Code review ACK bb6fcc75d1
  LarryRuane:
    ACK bb6fcc75d1
  jonatack:
    Code review ACK bb6fcc75d1 and verified rebase to master builds cleanly with unit/functional tests green

Tree-SHA512: fddeb720d5a391b48bb4c6fa58ed34ccc3f57862fdb8e641745c021841c8340e35c5126338271446cbd98f40bd5484f27926aa6c3e76fa478ba1efafe72e73c1
2021-01-25 20:21:19 +01:00
MarcoFalke
fad3d7625a
fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION 2021-01-23 20:04:37 +01:00
MarcoFalke
fa99e33aeb
fuzz: move-only FillNode implementation to cpp file
This allows to modify the implementation without having to recompile all
fuzz targets.

Can be reviewed with --color-moved=dimmed-zebra
2021-01-23 20:01:12 +01:00
MarcoFalke
fa61b9d1a6
util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet
Co-Authored-by: Anthony Towns <aj@erisian.com.au>
2021-01-21 19:31:28 +01:00
MarcoFalke
45952dab9d
Merge #20932: refactor: Replace fs::absolute calls with AbsPathJoin calls
da9caa1ced Replace fs::absolute calls with AbsPathJoin calls (Kiminuo)
66576c4fd5 test: Clear forced -walletdir setting after wallet init_tests (Kiminuo)

Pull request description:

  This adds better test coverage and will make it easier in #20744 to remove our dependency on the two-argument boost::filesystem::absolute() function which does not have a direct equivalent in C++17.

  This PR doesn't change behavior aside from adding an assert and fixing a test bug.

ACKs for top commit:
  jonatack:
    Code review ACK da9caa1ced only doxygen improvements since my last review per `git diff d867d7a da9caa1`
  MarcoFalke:
    review ACK da9caa1ced 📯
  ryanofsky:
    Code review ACK da9caa1ced. Just comment and test tweaks since previous review.

Tree-SHA512: c940ee60f3ba374d4927cf34cf12d27c4c735c94af591fbc0ca408c641b30f8f8fbcfe521d66bfbddf9877a1fc8cd99bd8a47ebcd2fa59789de6bd87a7b9cf4d
2021-01-21 18:48:03 +01:00
MarcoFalke
85fee49c39
Merge #20946: fuzz: Consolidate fuzzing TestingSetup initialization
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate (Carl Dong)
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization (Carl Dong)

Pull request description:

  ```
  Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:

  1. Calling InitializeFuzzingContext, which implicitly constructs a static
     const BasicTestingSetup
  2. Directly constructing a static const BasicTestingSetup in the initialize_*
     function
  3. Directly constructing a static TestingSetup and reproducing the
     initialization arguments (I'm assuming because
     InitializeFuzzingContext only initializes a BasicTestingSetup)

  The new, relatively-simple MakeFuzzingContext function allows us to
  consolidate these methods of initialization by being flexible enough to
  be used in all situations. It:

  1. Is templated so that we can choose to initialize any of
     the *TestingSetup classes
  2. Has sane defaults which are often used in fuzzers but are also
     easily overridable
  3. Returns a unique_ptr, explicitly transferring ownership to the caller
     to deal with according to its situation
  ```

  ~~Question for fuzzing people: was it intentional that `src/test/fuzz/net.cpp` would directly instantiate the `BasicTestingSetup` and thus omit the `"-nodebuglogfile"` flag?~~ [Answered](https://github.com/bitcoin/bitcoin/pull/20946#issuecomment-761537108)

ACKs for top commit:
  MarcoFalke:
    ACK abb6fa7285

Tree-SHA512: 96a5ca6f4cd5ea0e9483b60165b31ae3e9003918c700a7f6ade48010f419f2a6312e10b816b3187f1d263798827571866e4c4ac0bbfb2e0c79dfad254cda68e7
2021-01-21 16:04:31 +01:00
Carl Dong
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate
A full TestingSetup is required for both coins_view and
load_external_block_file as they interact with the active chainstate.
2021-01-21 09:29:42 -05:00
Carl Dong
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization
Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:

1. Calling InitializeFuzzingContext, which implicitly constructs a static
   const BasicTestingSetup
2. Directly constructing a static const BasicTestingSetup in the initialize_*
   function
3. Directly constructing a static TestingSetup and reproducing the
   initialization arguments (I'm assuming because
   InitializeFuzzingContext only initializes a BasicTestingSetup)

The new, relatively-simple MakeFuzzingContext function allows us to
consolidate these methods of initialization by being flexible enough to
be used in all situations. It:

1. Is templated so that we can choose to initialize any of
   the *TestingSetup classes
2. Has sane defaults which are often used in fuzzers but are also
   easily overridable
3. Returns a unique_ptr, explicitly transferring ownership to the caller
   to deal with according to its situation
2021-01-21 09:29:42 -05:00
Kiminuo
da9caa1ced Replace fs::absolute calls with AbsPathJoin calls
This adds better test coverage and will make it easier in #20744 to remove our dependency on the two-argument boost::filesystem::absolute() function which does not have a direct equivalent in C++17.
2021-01-15 22:48:15 +01:00
MarcoFalke
0a1cf6c347
Merge #20908: fuzz: Use mocktime in process_message* fuzz targets
fa0a864b38 fuzz: Use mocktime in process_message* fuzz targets (MarcoFalke)

Pull request description:

  Use mocktime to allow time to advance deterministically during execution of a fuzz input. This also allows to drop the call to `JumpOutOfIbd`.

ACKs for top commit:
  practicalswift:
    cr ACK fa0a864b38

Tree-SHA512: e92fc70ec6bd49760173cb202549f364304e22b3f7127b9a4da8447cf9341008e477ad42c2599c2fde167bbcbc0e2d139709b4ef6371788bc2c1c3b7f589e11d
2021-01-15 19:56:18 +01:00
practicalswift
b4511e2e2e log: Prefix log messages with function name if -logsourcelocations is set 2021-01-15 09:57:32 +00:00
MarcoFalke
29d2aeb4a2
Merge #20828: fuzz: Introduce CallOneOf helper to replace switch-case
fa75d40ef8 fuzz: Introduce CallOneOf helper to replace switch-case (MarcoFalke)

Pull request description:

  The current `switch (fuzzed_data_provider.ConsumeIntegralInRange<int>(0, nn)) { case 0: ... case 1: ... case nn: ...` has several problems:

  * It makes it hard to review newly added targets, because it requires manual counting of cases
  * It makes it hard to update a target, because updating all case labels is trivial, but tedious to review and causes merge conflicts
  * ~~Updating the target raises the question whether the case labels should be preserved to not invalidate the existing fuzz inputs format. Fuzz input format might already change implicitly on every commit, so this isn't something worthwhile to pursue.~~ Edit: This pull doesn't fix this problem.

  Fix all issues by adding a new `CallOneOf` helper

ACKs for top commit:
  ajtowns:
    ACK fa75d40ef8 - code review only
  jnewbery:
    utACK fa75d40ef8

Tree-SHA512: 2daa602b240b86c8e85a024e008f03a57ba60349377eed771f4d21a97a9dba9b66e93fff16ff1992018d4330be7a1a276944c3dfdf698748ce135626c380e563
2021-01-14 11:07:22 +01:00
Wladimir J. van der Laan
60427ee35f
Merge #20811: refactor: move net_processing implementation details out of header
c97f70c861 net_processing: move Peer definition to .cpp (Anthony Towns)
e0f2e6d2df net_processing: move PeerManagerImpl into cpp file (Anthony Towns)
a568b82feb net_processing: split PeerManager into interface and implementation classes (Anthony Towns)
0df3d3fd6b net_processing: make more of PeerManager private (Anthony Towns)
0d246a59b6 net, net_processing: move NetEventsInterface method docs to net.h (Anthony Towns)

Pull request description:

  Moves the implementation details of `PeerManager` and all of `struct Peer` into net_processing.cpp.

ACKs for top commit:
  jnewbery:
    ACK c97f70c861
  Sjors:
    ACK c97f70c861
  laanwj:
    Code review ACK c97f70c861
  vasild:
    ACK c97f70c861

Tree-SHA512: 2e081e491d981c61bd78436a6a6c2eb41d3c2d86a1a8ef9d4d6f801b82cb64a8bf707a96db3429418d1704cffb60a657d1037a0336cbc8173fb79aef9eb72001
2021-01-13 09:48:06 +01:00
MarcoFalke
fa0a864b38
fuzz: Use mocktime in process_message* fuzz targets 2021-01-13 07:48:41 +01:00
sinetek
2a39ccf133 Add include for std::bind. 2021-01-13 02:05:00 +01:00
fanquake
7838db141b
Merge #20495: sync: Use decltype(auto) return type for WITH_LOCK
3eb94ec81b sync: Use decltype(auto) return type for WITH_LOCK (Carl Dong)

Pull request description:

  > Now that we're using C++17, we can use the decltype(auto) return type
  > for functions and lambda expressions.
  >
  > As demonstrated in this commit, this can simplify cases where previously
  > the compiler failed to deduce the correct return type.
  >
  > Just for reference, for the "assign to ref" cases fixed here, there are
  > 3 possible solutions:
  >
  > - Return a pointer and immediately deref as used before this commit
  > - Make sure the function/lambda returns declspec(auto) as used after
  >   this commit
  > - Class& i = WITH_LOCK(..., return std::ref(...));
  >
  > -----
  >
  > References:
  > 1. https://en.cppreference.com/w/cpp/language/function#Return_type_deduction
  > 2. https://en.cppreference.com/w/cpp/language/template_argument_deduction#Other_contexts
  > 3. https://en.cppreference.com/w/cpp/language/auto
  > 4. https://en.cppreference.com/w/cpp/language/decltype
  >
  > Explanations:
  > 1. https://stackoverflow.com/a/21369192
  > 2. https://stackoverflow.com/a/21369170

  Thanks to sipa and ryanofsky for helping me understand this

ACKs for top commit:
  jnewbery:
    utACK 3eb94ec81b
  hebasto:
    ACK 3eb94ec81b, I have reviewed the code and it looks OK, I agree it can be merged. I have verified possible warnings:
  ryanofsky:
    Code review ACK 3eb94ec81b

Tree-SHA512: 5f55c7722aeca8ea70e5c1a8db93e93ba0e356e8967e7f607ada38003df4b153d73c29bd2cea8d7ec1344720d37d857ea7dbfd2a88da1d92e0e9cbb9abd287df
2021-01-12 15:56:19 +08:00
fanquake
6d81d7aa87
Merge #20787: Use C++17 std::array deduction for OUTPUT_TYPES, ALL_FEE_ESTIMATE_HORIZONS
aaaa987840 refactor: Use C++17 std::array deduction for ALL_FEE_ESTIMATE_HORIZONS (MarcoFalke)
fa39cdd072 refactor: Use C++17 std::array deduction for OUTPUT_TYPES (MarcoFalke)

Pull request description:

  With the new C++17 array deduction rules, an array encompassing all values in an enum can be specified in the same header file that specifies the enum. This is useful to avoid having to repeatedly enumerate all enum values in the code. E.g. the RPC code, but also the fuzz code.

ACKs for top commit:
  theStack:
    cr ACK aaaa987840 ⚙️
  fanquake:
    ACK aaaa987840

Tree-SHA512: b71bd98f3ca07ddfec385735538ce89a4952e418b52dc990fb160187ccef1fc7ebc139d42988b6f7b48df24823af61f803b83d47fb7a3b82475f0c0b109bffb7
2021-01-11 21:46:09 +08:00
Wladimir J. van der Laan
675af2a515
Merge #20852: net: allow CSubNet of non-IP networks
39b43298d9 test: add test for banning of non-IP addresses (Vasil Dimov)
94d335da7f net: allow CSubNet of non-IP networks (Vasil Dimov)

Pull request description:

  Allow creation of valid `CSubNet` objects of non-IP networks and only
  match the single address they were created from (like /32 for IPv4 or
  /128 for IPv6).

  This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
  and in `BanMan` which assume that creating a subnet from any address
  using the `CSubNet(CNetAddr)` constructor would later match that address
  only. Before this change a non-IP subnet would be invalid and would not
  match any address.

ACKs for top commit:
  jonatack:
    Code review re-ACK 39b43298d9 per `git diff 5e95ce6 39b4329`; only change since last review is improvements to the functional test; verified the test fails on master @ 616eace0 where expected (`assert(self.is_banned(node, tor_addr))` fails and unban unfails)
  laanwj:
    code review ACK 39b43298d9

Tree-SHA512: 3239b26d0f2fa2d1388b4fdbc1d05ce4ac1980be699c6ec46049409baefcb2006b1e72b889871e2210e897f6725c48e873f68457eea7e6e4958ab4f959d20297
2021-01-11 11:27:47 +01:00
MarcoFalke
fa75d40ef8
fuzz: Introduce CallOneOf helper to replace switch-case
Can be reviewed with --ignore-all-space
2021-01-11 10:37:16 +01:00
fanquake
bd6af53e1f
Merge #20480: Replace boost::variant with std::variant
faa8f68943 Replace boost::variant with std::variant (MarcoFalke)

Pull request description:

  Now that we can use std::variant from the vanilla standard library, drop the third-party boost variant dependency

ACKs for top commit:
  fjahr:
    Code review ACK faa8f68943
  fanquake:
    ACK faa8f68943

Tree-SHA512: 6e3aecd33b00c2e31a763f999247944d5b2ce5e3018f1965c516c1000cd08ff6703a8d50fb0be64883153da2925ae72986b8a6b96586db74057bd05d6f4986e6
2021-01-11 12:05:46 +08:00
Vasil Dimov
94d335da7f
net: allow CSubNet of non-IP networks
Allow creation of valid `CSubNet` objects of non-IP networks and only
match the single address they were created from (like /32 for IPv4 or
/128 for IPv6).

This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
and in `BanMan` which assume that creating a subnet from any address
using the `CSubNet(CNetAddr)` constructor would later match that address
only. Before this change a non-IP subnet would be invalid and would not
match any address.
2021-01-10 15:51:15 +01:00
MarcoFalke
555fc0789d
Merge #20881: fuzz: net permission flags in net processing
fad327ca65 fuzz: net permission flags in net processing (MarcoFalke)

Pull request description:

  to increase coverage

ACKs for top commit:
  Crypt-iQ:
    cr ACK fad327c
  practicalswift:
    ACK fad327ca65

Tree-SHA512: f8643d1774ff13524ab97ab228ad070489e080435e5742af26e6e325fd002e4c1fd78b9887e11622e79d6fe0c4daaddce5e033e6cd4b32e50fd68b434aab7333
2021-01-10 10:33:57 +01:00
Anthony Towns
a568b82feb net_processing: split PeerManager into interface and implementation classes 2021-01-09 23:27:45 +10:00
MarcoFalke
fa44417fcb
fuzz: Add missing muhash registration 2021-01-08 09:40:50 +01:00
MarcoFalke
fad327ca65
fuzz: net permission flags in net processing 2021-01-07 19:07:02 +01:00
Wladimir J. van der Laan
b6a71b80d2
Merge #19055: Add MuHash3072 implementation
9815332d51 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr)
01297fb3ca fuzz: Add MuHash consistency fuzz test (Fabian Jahr)
b111410914 test: Add MuHash3072 fuzz test (Fabian Jahr)
c122527385 bench: Add Muhash benchmarks (Fabian Jahr)
7b1242229d test: Add MuHash3072 unit tests (Fabian Jahr)
adc708c98d crypto: Add MuHash3072 implementation (Fabian Jahr)
0b4d290bf5 crypto: Add Num3072 implementation (Fabian Jahr)
589f958662 build: Check for 128 bit integer support (Fabian Jahr)

Pull request description:

  This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`.

ACKs for top commit:
  laanwj:
    Code review ACK 9815332d51

Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
2021-01-07 17:57:17 +01:00
MarcoFalke
3a6acd1772
Merge #20789: fuzz: Rework strong and weak net enum fuzzing
eeee43bc48 fuzz: Use ConsumeWeakEnum for ServiceFlags (MarcoFalke)
fa9949b914 fuzz: Add ConsumeWeakEnum helper, Extract ALL_NET_PERMISSION_FLAGS (MarcoFalke)
faaef9434c fuzz: [refactor] Extract ALL_CONNECTION_TYPES constant (MarcoFalke)
fa42da2d54 fuzz: Use ConsumeNode in process_message target (MarcoFalke)
fa121f058f fuzz: Use ConsumeNode in process_messages target (MarcoFalke)

Pull request description:

  The fuzz tests have several problems:
  * The array passed to the fuzz engine to pick `net_permission_flags` is outdated
  * The process_message* targets has the service flags as well as connection type hardcoded, limiting potential coverage
  * The service flags deserialization from the fuzz engine doesn't allow for easy "exact matches". The fuzz engine has to explore a 64-bit space to hit an "exact match" (only one bit set)

  Fix all issues in the commits in this pull

ACKs for top commit:
  mzumsande:
    ACK eeee43bc48 after rebase.

Tree-SHA512: 1ad9520c7e708b7f4994ae8f77886ffca33d7c542756e2a3e07dbbbe59e360f9fcaccf2e2fb57d9bc731d4aeb4938fb1c5c546e9d2744b007af5626f5cb377fe
2021-01-07 17:04:56 +01:00
MarcoFalke
f13e03cda2
Merge #20584: Declare de facto const reference variables/member functions as const
31b136e580 Don't declare de facto const reference variables as non-const (practicalswift)
1c65c075ee Don't declare de facto const member functions as non-const (practicalswift)

Pull request description:

  _Meta: This is the second and final part of the `const` refactoring series (part one: #20581). **I promise: no more refactoring PRs from me in a while! :)** I'll now go back to focusing on fuzzing/hardening!_

  Changes in this PR:
  * Don't declare de facto const member functions as non-const
  * Don't declare de facto const reference variables as non-const

  Awards for finding candidates for the above changes go to:
  * `clang-tidy`'s [`readability-make-member-function-const`](https://clang.llvm.org/extra/clang-tidy/checks/readability-make-member-function-const.html)  check ([list of `clang-tidy` checks](https://clang.llvm.org/extra/clang-tidy/checks/list.html))
  * `cppcheck`'s `constVariable` check ([list of `cppcheck` checks](https://sourceforge.net/p/cppcheck/wiki/ListOfChecks/))

  See #18920 for instructions on how to analyse Bitcoin Core using Clang Static Analysis, `clang-tidy` and `cppcheck`.

ACKs for top commit:
  ajtowns:
    ACK 31b136e580
  jonatack:
    ACK 31b136e580
  theStack:
    ACK 31b136e580 ❄️

Tree-SHA512: f58f8f00744219426874379e9f3e9331132b9b48e954d24f3a85cbb858fdcc98009ed42ef7e7b4619ae8af9fc240a6d8bfc1c438db2e97b0ecd722a80dcfeffe
2021-01-07 09:05:09 +01:00
MarcoFalke
fafd725a7c
Merge #19846: build: enable unused member function diagnostic
819d03b932 refactor: took out unused member functions (Zero)
ed69213c2b build: enable unused member function diagnostic (Zero)

Pull request description:

  This PR enables the `-Wunused-member-function` compiler diagnostic, as discussed in #19702.

  > **Notice**: The `unused-member-function` diagnostic is only available on clang. Therefore, clang should be used to test this PR.

  - [x] Include the `-Wunused-member-function`diagnostic in `./configure.ac`. (ed69213c2b)
  - [x] Resolve the reported warnings. (819d03b932)

  Currently, enabling this flag no longer reports the following warnings:

  > **Note**: output from `make 2>&1 | grep "warning: unused member function" | sort | uniq -c`

  ```
  1 index/blockfilterindex.cpp:54:5: warning: unused member function 'DBHeightKey' [-Wunused-member-function]
  2 script/bitcoinconsensus.cpp:50:9: warning: unused member function 'GetType' [-Wunused-member-function]
  1 test/util_tests.cpp:1975:14: warning: unused member function 'operator=' [-Wunused-member-function]
  ```

  All tests have passed locally (from `make check` & `src/test/test_bitcoin`).

  This PR closes #19702.

ACKs for top commit:
  practicalswift:
    ACK 819d03b932 - patch still looks correct :)
  MarcoFalke:
    ACK 819d03b932
  pox:
    Tested ACK 819d03b932 with clang after `make clean`. No unused member function warnings.
  theStack:
    tested ACK 819d03b932

Tree-SHA512: 5fdfbbb02b3dc618a90a874a5caa5e01e596fc1d14a209e75a6981f01b253f9bca0cfac8fdd758dd7151986609fb76571c3745124a29cfd4f8cbb8d82a07272e
2021-01-05 12:06:40 +01:00
MarcoFalke
faa8f68943
Replace boost::variant with std::variant 2021-01-05 10:10:50 +01:00
Ikko Ashimine
1112035d32
doc: fix various typos
Co-authored-by: Peter Yordanov <ppyordanov@yahoo.com>
2021-01-04 12:31:31 +08:00