Commit graph

3172 commits

Author SHA1 Message Date
MarcoFalke
99993f0664
fuzz: Avoid excessively large min fee rate in tx_pool 2021-05-09 10:53:24 +02:00
MarcoFalke
facfc0f65d
fuzz: Remove strprintf test cases that are known to fail 2021-05-09 10:25:21 +02:00
MarcoFalke
fa1aa6c571
fuzz: Limit ParseISO8601DateTime fuzzing to 32-bit 2021-05-09 10:04:01 +02:00
MarcoFalke
fa27d6d3ac
fuzz: Remove unused --enable-danger-fuzz-link-all option 2021-05-08 09:32:45 +02:00
MarcoFalke
fa5cb6b268
fuzz: Add WRITE_ALL_FUZZ_TARGETS_AND_ABORT 2021-05-07 11:01:05 +02:00
MarcoFalke
1fcf66af68
Merge bitcoin/bitcoin#21798: fuzz: Create a block template in tx_pool targets
fa03d0acd6 fuzz: Create a block template in tx_pool targets (MarcoFalke)
fa61ce5cf5 fuzz: Limit mocktime to MTP in tx_pool targets (MarcoFalke)
fab646b8ea fuzz: Use correct variant of ConsumeRandomLengthString instead of hardcoding a maximum size (MarcoFalke)
fae2c8bc54 fuzz: Allow to pass min/max to ConsumeTime (MarcoFalke)

Pull request description:

  Relatively simple check to ensure a block can always be created from the mempool

ACKs for top commit:
  practicalswift:
    Tested ACK fa03d0acd6

Tree-SHA512: e613376ccc88591cbe594db14ea21ebc9b2b191f6325b3aa4ee0cd379695352ad3b480e286134ef6ee30f043d486cf9792a1bc7e44445c41045ac8c3b931c7ff
2021-05-06 16:06:20 +02:00
MarcoFalke
fa2204f6ad
streams: Accept URef obj for VectorReader unserialize 2021-05-05 20:19:03 +02:00
MarcoFalke
fa03d0acd6
fuzz: Create a block template in tx_pool targets 2021-05-05 20:12:00 +02:00
MarcoFalke
128b98fce3
Merge bitcoin/bitcoin#21681: validation: fix ActivateSnapshot to use hardcoded nChainTx
91d93aac4e validation: remove nchaintx from assumeutxo metadata (James O'Beirne)
931684b24a validation: fix ActivateSnapshot to use hardcoded nChainTx (James O'Beirne)

Pull request description:

  This fixes an oversight from the move of nChainTx from the user-supplied
  snapshot metadata into the hardcoded assumeutxo chainparams.

  Since the nChainTx is now unused in the metadata, it should be removed
  in a future commit.

  See: https://github.com/bitcoin/bitcoin/pull/19806#discussion_r612165410

ACKs for top commit:
  Sjors:
    utACK 91d93aac4e
  ryanofsky:
    Code review ACK 91d93aac4e. No change to previous commit, just new commit removing now unused utxo snapshot field and updating tests.

Tree-SHA512: 445bdd738faf007451f40bbcf360dd1fb4675e17a4c96546e6818c12e33dd336dadd95cf8d4b5f8df1d6ccfbc4bf5496864bb5528e416cea894857b6b732140c
2021-05-05 18:34:16 +02:00
MarcoFalke
32f1f021bf
Merge bitcoin/bitcoin#21817: refactor: Replace &foo[0] with foo.data()
fac30eec42 refactor: Replace &foo[0] with foo.data() (MarcoFalke)
faece47c47 refactor: Avoid &foo[0] on C-Style arrays (MarcoFalke)
face961109 refactor: Use only one temporary buffer in CreateObfuscateKey (MarcoFalke)
fa05dddc42 refactor: Use CPubKey vector constructor where possible (MarcoFalke)
fabb6dfe6e script: Replace address-of idiom with vector data() method (Guido Vranken)

Pull request description:

  The main theme of this refactor is to replace `&foo[0]` with `foo.data()`.

  The first commit is taken from #21781 with the rationale:

  * In CSignatureCache::ComputeEntryECDSA, change the way a vector pointer is resolved to prevent invoking undefined behavior if the vector is empty.

  The other commits aim to remove all `&foo[0]`, where `foo` is any kind of byte representation. The rationale:

  * Sometimes alternative code without any raw data pointers is easier to read (refer to the respective commit message for details)
  * If the raw data pointer is needed, `foo.data()` should be preferred, as pointed out in the developer notes. This addresses the instances that have been missed in commit 592404f03f, and https://github.com/bitcoin/bitcoin/pull/9804

ACKs for top commit:
  laanwj:
    Code review ACK fac30eec42
  practicalswift:
    cr ACK fac30eec42: patch looks correct
  promag:
    Code review ACK fac30eec42.

Tree-SHA512: e7e73146edbc78911a8e8c728b0a1c6b0ed9a88a008e650aa5dbffe72425bd42c76df70199a9cf7e02637448d7593e0eac52fd0f91f59240283e1390ee21bfa5
2021-05-05 18:24:09 +02:00
W. J. van der Laan
75b3a32f03
Merge bitcoin/bitcoin#21821: test: Add missing test for empty P2WSH redeem
fa80a11c3b test: Add missing test for empty P2WSH redeem (MarcoFalke)

Pull request description:

ACKs for top commit:
  0xB10C:
    ACK fa80a11c3b

Tree-SHA512: 2d28fe2e80c0770d82e4164b235345a3a828e6b71445a82aa144b3efb3298e494a8bc456a329f175a3cb542fa1b669d869dc1fb1b711611feeb76ab312a661d5
2021-05-05 15:47:19 +02:00
MarcoFalke
cf83b82cf0
fuzz: Limit toxic test globals to their respective scope 2021-05-04 09:24:17 +02:00
MarcoFalke
fac30eec42
refactor: Replace &foo[0] with foo.data() 2021-05-04 06:55:31 +02:00
MarcoFalke
e2d4e67a8f
Merge bitcoin/bitcoin#21840: test: Misc refactor to get rid of &foo[0] raw pointers
fa8a88849c bench: Remove duplicate constants (MarcoFalke)
000098f964 test: Use throwing variant accessor (MarcoFalke)
fa2197c8b3 test: Use loop to register RPCs (MarcoFalke)

Pull request description:

  Simplify test code

ACKs for top commit:
  Empact:
    Code Review ACK fa8a88849c
  practicalswift:
    cr ACK fa8a88849c
  promag:
    Code review ACK fa8a88849c.

Tree-SHA512: 6a5bebaa9a3f43e9c332f4fbff606e9ece6dc8b95a769980082cc022f8e9bde6083c1e4a0145dcbf3741f514d6e97b4198f201a1bf1370ebf43bd3a5c0f85981
2021-05-04 06:49:22 +02:00
MarcoFalke
ea71726a54
Merge bitcoin/bitcoin#21810: fuzz: Various RPC fuzzer follow-ups
5252f86eb6 fuzz: Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of #ifdef forests (practicalswift)
54549dda31 fuzz: RPC fuzzer post-merge follow-ups. Remove unused includes. Update list of fuzzed RPC commands. (practicalswift)

Pull request description:

  Various RPC fuzzer follow-ups:
  * Remove unused includes.
  * Update list of fuzzed RPC commands.
  * Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of `#ifdef` forests.

  Context: https://github.com/bitcoin/bitcoin/pull/21169#pullrequestreview-646723483

ACKs for top commit:
  MarcoFalke:
    Concept ACK 5252f86eb6

Tree-SHA512: 286d70798131706ffb157758e1c73f7f00ed96ce120c7d9dc849e672b283f1362df47b206cfec9da44d5debb5869225e721761dcd5c38a7d5d1019dc6c912ab2
2021-05-03 19:47:18 +02:00
MarcoFalke
000098f964
test: Use throwing variant accessor
It does not matter if the tests fail due to a BOOST_CHECK failure or
due to a thrown exception. Prefer the exception because it is less
code.

Example fail with the throwing accessor:

unknown location(0): fatal error: in "script_standard_tests/script_standard_ExtractDestinations": std::bad_variant_access: std::get: wrong index for variant
test/script_standard_tests.cpp(314): last checkpoint

*** 1 failure is detected in the test module "Bitcoin Core Test Suite"
2021-05-03 11:50:29 +02:00
fanquake
60132382a7
Merge bitcoin/bitcoin#20867: Support up to 20 keys for multisig under Segwit context
ebd4be43cc doc: add release notes for 20867 (Antoine Poinsot)
5aa50ab9cc rpc/util: multisig: only check redeemScript size is <= 520 for P2SH (Antoine Poinsot)
063df9e897 test/functional: standardness sanity checks for P2(W)SH multisig (Antoine Poinsot)
ae0429d3af script: allow up to 20 keys in wsh() descriptors (Antoine Poinsot)
9fc68faf35 script: match multisigs with up to MAX_PUBKEYS_PER_MULTISIG keys (Antoine Poinsot)

Pull request description:

  As described in https://github.com/bitcoin/bitcoin/issues/20620 multisigs are currently limited to 16 keys in descriptors and RPC helpers, even for P2WSH and P2SH-P2WSH.

  This adds support for multisig with up to 20 keys (which are already standard) for Segwit v0 context for descriptors (`wsh()`, `sh(wsh())`) and RPC helpers.

  Fixes https://github.com/bitcoin/bitcoin/issues/20620

ACKs for top commit:
  meshcollider:
    re-utACK ebd4be43cc
  instagibbs:
    re-ACK ebd4be43cc

Tree-SHA512: 36141f10a8288010d17d5c4fe8d24878bcd4533b88a8aba3a44fa8f74ceb3182d70fee01427e0ab7f53ce7fab46c88c1cd3ac3b18ab8a10bd4a6b8b74ed79e46
2021-05-03 12:44:23 +08:00
MarcoFalke
fa80a11c3b
test: Add missing test for empty P2WSH redeem 2021-05-01 10:12:11 +02:00
W. J. van der Laan
2b45cf0bcd
Merge bitcoin/bitcoin#19521: Coinstats Index
5f96d7d22d rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height (Fabian Jahr)
23fe50436b test: Add test for coinstatsindex behavior in reorgs (Fabian Jahr)
90c966b0f3 rpc: Allow gettxoutsetinfo and getblockstats for stale blocks (Fabian Jahr)
b9362392ae index, rpc: Add use_index option for gettxoutsetinfo (Fabian Jahr)
bb7788b121 test: Test coinstatsindex robustness across restarts (Fabian Jahr)
e0938c2909 test: Add tests for block_info in gettxoutsetinfo (Fabian Jahr)
2501576ecc rpc, index: Add verbose amounts tracking to Coinstats index (Fabian Jahr)
655d929836 test: add coinstatsindex getindexinfo coverage, improve current tests (Jon Atack)
ca01bb8d68 rpc: Add Coinstats index to getindexinfo (Fabian Jahr)
57a026c30f test: Add unit test for Coinstats index (Fabian Jahr)
6a4c0c09ab test: Add functional test for Coinstats index (Fabian Jahr)
3f166ecc12 rpc: gettxoutsetinfo can be requested for specific blockheights (Fabian Jahr)
3c914d58ff index: Coinstats index can be activated with command line flag (Fabian Jahr)
dd58a4de21 index: Add Coinstats index (Fabian Jahr)
a8a46c4b3c refactor: Simplify ApplyStats and ApplyHash (Fabian Jahr)
9c8a265fd2 refactor: Pass hash_type to CoinsStats in stats object (Fabian Jahr)
2e2648a902 crypto: Make MuHash Remove method efficient (Fabian Jahr)

Pull request description:

  This is part of the coinstats index project tracked in #18000

  While the review of the new UTXO set hash algorithm (MuHash) takes longer recently #19328 was merged which added the possibility to run `gettxoutsetinfo` with a specific hash type. As the first type it added `hash_type=none` which skips the hashing of the UTXO set altogether. This alone did not make `gettxoutsetinfo` much faster but it allows the use of an index for the remaining coin statistics even before a new hashing algorithm has been added. Credit to Sjors for the idea to take this intermediate step.

  Features summary:
  - Users can start their node with the option `-coinstatsindex` which syncs the index in the background
  - After the index is synced the user can  use `gettxoutsetinfo` with `hash_type=none` or `hash_type=muhash` and will get the response instantly out of the index
  - The user can specify a height or block hash when calling `gettxoutsetinfo` to see coin statistics at a specific block height

ACKs for top commit:
  Sjors:
    re-tACK 5f96d7d22d
  jonatack:
    Code review re-ACK 5f96d7d22d per `git range-diff 13d27b4 07201d3 5f96d7d`
  promag:
    Tested ACK 5f96d7d22d. Light code review ACK 5f96d7d22d.

Tree-SHA512: cbca78bee8e9605c19da4fbcd184625fb280200718396c694a56c7daab6f44ad23ca9fb5456d09f245d8b8d9659fdc2b3f3ce5e953c1c6cf4003dbc74c0463c2
2021-04-30 17:27:19 +02:00
John Newbery
0829516d1f [refactor] Remove unused ForEachNodeThen() template 2021-04-30 11:29:17 +01:00
John Newbery
76568a3351 [net processing] Move addr relay data and logic into net processing 2021-04-30 11:29:14 +01:00
practicalswift
5252f86eb6 fuzz: Reduce maintenance requirements by allowing RPC annotations also for conditionally available RPC commands (such as wallet commands) without the fragility of #ifdef forests 2021-04-29 18:40:25 +00:00
Hennadii Stepanov
a508f718f3
refactor: Use appropriate thread constructor 2021-04-29 18:39:01 +03:00
MarcoFalke
4cfe6c37d9
Merge bitcoin/bitcoin#18847: compressor: use a prevector in CompressScript serialization [ZAP1]
83a425d25a compressor: use a prevector in compressed script serialization (William Casarin)

Pull request description:

  This function was doing millions of unnecessary heap allocations during IBD.

  I'm start to catalog unnecessary heap allocations as a pet project of mine: as-zero-as-possible-alloc IBD. This is one small step.

  before:
  ![May01-174536](https://user-images.githubusercontent.com/45598/80850964-9a38de80-8bd3-11ea-8eec-08cd38ee1fa1.png)

  after:
  ![May01-174610](https://user-images.githubusercontent.com/45598/80850974-a91f9100-8bd3-11ea-94a1-e2077391f6f4.png)

  ~should I type alias this?~ *I type aliased it*

  This is a part of the Zero Allocations Project #18849 (ZAP1). This code came up as a place where many allocations occur.

ACKs for top commit:
  Empact:
    ACK 83a425d25a
  elichai:
    tACK 83a425d25a
  sipa:
    utACK 83a425d25a

Tree-SHA512: f0ffa6ab0ea1632715b0b76362753f9f6935f05cdcc80d85566774401155a3c57ad45a687942a1806d3503858f0bb698da9243746c8e2edb8fdf13611235b0e0
2021-04-28 21:13:44 +02:00
MarcoFalke
fa61ce5cf5
fuzz: Limit mocktime to MTP in tx_pool targets
This is needed for the next commit to generate blocks.

Also, apply the same mocking strategies to both targets.
2021-04-28 20:54:07 +02:00
MarcoFalke
fab646b8ea
fuzz: Use correct variant of ConsumeRandomLengthString instead of hardcoding a maximum size
This is technically a breaking change.

This allows the fuzz engine to pick the right size,
also larger sizes, if needed.
2021-04-28 20:53:50 +02:00
MarcoFalke
fae2c8bc54
fuzz: Allow to pass min/max to ConsumeTime 2021-04-28 20:52:29 +02:00
practicalswift
54549dda31 fuzz: RPC fuzzer post-merge follow-ups. Remove unused includes. Update list of fuzzed RPC commands. 2021-04-28 09:27:50 +00:00
MarcoFalke
549d20a31b
Merge bitcoin/bitcoin#20772: fuzz: bolster ExtractDestination(s) checks
a29f522ba4 fuzz: bolster ExtractDestination(s) checks (Michael Dietz)

Pull request description:

ACKs for top commit:
  practicalswift:
    Tested ACK a29f522ba4

Tree-SHA512: 0fc194edb7b0fce77c7bb725fe65dec7976598edcd53882b5a0eb7cd83281a3ddcd2b3de00282468be659a7e5bc9991eb482816418f55b30e657cdc5a3bd7438
2021-04-28 11:02:25 +02:00
Antoine Poinsot
ae0429d3af
script: allow up to 20 keys in wsh() descriptors
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:29 +02:00
Antoine Poinsot
9fc68faf35
script: match multisigs with up to MAX_PUBKEYS_PER_MULTISIG keys
We were previously ruling out 17-20 pubkeys multisig, while they are
only invalid under P2SH context.
This makes multisigs with up to 20 keys be detected as valid by the
solver. This is however *not* a policy change as it would only apply
to bare multisigs, which are already limited to 3 pubkeys.

Note that this does not change the sigOpCount calculation (as it would
break consensus). Therefore 1-16 keys multisigs are counted as 1-16 sigops
and 17-20 keys multisigs are counted as 20 sigops.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:28 +02:00
MarcoFalke
e45863166f
Merge bitcoin/bitcoin#21169: fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70%.
545404e7e1 fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70%. (practicalswift)

Pull request description:

  Add RPC interface fuzzing.

  This PR increases overall fuzzing line coverage from [~65%](https://marcofalke.github.io/btc_cov/fuzz.coverage/) to ~70% 🎉

  To test this PR:

  ```
  $ make distclean
  $ ./autogen.sh
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make -C src/ test/fuzz/fuzz
  $ FUZZ=rpc src/test/fuzz/fuzz
  ```

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for more information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  MarcoFalke:
    Concept ACK 545404e7e1

Tree-SHA512: 35fc1b508af42bf480ee3762326b09ff2eecdb7960a1917ad16345fadd5c0c21d666dafe736176e5a848ff6492483c782e4ea914cd9000faf50190df051950fd
2021-04-28 09:45:30 +02:00
practicalswift
545404e7e1 fuzz: Add RPC interface fuzzing. Increase fuzzing coverage from 65% to 70%. 2021-04-28 06:34:20 +00:00
James O'Beirne
91d93aac4e
validation: remove nchaintx from assumeutxo metadata
This value is no longer used and is instead specified statically
in chainparams. This change means that previously generated
snapshots will no longer be usable.
2021-04-26 13:22:37 -04:00
Hennadii Stepanov
30e4448215
refactor: Make TraceThread a non-template free function
Also it is moved into its own module.
2021-04-25 12:28:44 +03:00
MarcoFalke
fa1fdeb230
fuzz: Ensure prevout is consensus-valid 2021-04-25 10:36:00 +02:00
MarcoFalke
8f80092d78
Merge bitcoin/bitcoin#21563: net: Restrict period when cs_vNodes mutex is locked
8c8237a4a1 net, refactor: Fix style in CConnman::StopNodes (Hennadii Stepanov)
229ac1892d net: Combine two loops into one, and update comments (Hennadii Stepanov)
a3d090d110 net: Restrict period when cs_vNodes mutex is locked (Hennadii Stepanov)

Pull request description:

  This PR restricts the period when the `cs_vNodes` mutex is locked, prevents the only case when `cs_vNodes` could be locked before the `::cs_main`.

  This change makes the explicit locking of recursive mutexes in the explicit order redundant.

ACKs for top commit:
  jnewbery:
    utACK 8c8237a4a1
  vasild:
    ACK 8c8237a4a1
  ajtowns:
    utACK 8c8237a4a1 - logic seems sound
  MarcoFalke:
    review ACK 8c8237a4a1 👢

Tree-SHA512: a8277924339622b188b12d260a100adf5d82781634cf974320cf6007341f946a7ff40351137c2f5369aed0d318f38aac2d32965c9b619432440d722a4e78bb73
2021-04-25 10:08:57 +02:00
Hennadii Stepanov
a3d090d110
net: Restrict period when cs_vNodes mutex is locked 2021-04-22 17:28:39 +03:00
W. J. van der Laan
906ecb87c8
Merge #21238: A few descriptor improvements to prepare for Taproot support
0b188b751f Clean up context dependent checks in descriptor parsing (Pieter Wuille)
33275a9649 refactor: move uncompressed-permitted logic into ParsePubkey* (Pieter Wuille)
17e006ff8d refactor: split off subscript logic from ToStringHelper (Pieter Wuille)
6ba5dda0c9 Account for key cache indices in subexpressions (Pieter Wuille)
4441c6f3c0 Make DescriptorImpl support multiple subscripts (Pieter Wuille)
a917478db0 refactor: move population of out.scripts from ExpandHelper to MakeScripts (Pieter Wuille)
84f3939ece Remove support for subdescriptors expanding to multiple scripts (Pieter Wuille)

Pull request description:

  These are a few refactors and non-invasive improvements to the descriptors code to prepare for adding Taproot descriptors.

  None of the commits change behavior in any way, except the last one which improves error reporting a bit.

ACKs for top commit:
  S3RK:
    reACK 0b188b7
  Sjors:
    re-ACK 0b188b7
  achow101:
    re-ACK 0b188b751f

Tree-SHA512: cb4e999134aa2bace0e13d4883454c65bcf1369e1c8585d93cc6444ddc245f3def5a628d58af7dab577e9d5a4a75d3bb46f766421fcc8cc5c85c01a11f148b3f
2021-04-20 08:59:09 +02:00
MarcoFalke
de77cbc9d8
Merge bitcoin/bitcoin#21691: test: Check that no versionbits are re-used
fa8eaee6a8 test: Run versionbits_sanity for all chains (MarcoFalke)
faec1e9ee1 test: Address outstanding versionbits_test feedback (MarcoFalke)
fad4167871 test: Check that no versionbits are re-used (MarcoFalke)

Pull request description:

ACKs for top commit:
  jnewbery:
    Code review ACK fa8eaee6a8
  ajtowns:
    ACK fa8eaee6a8 code review only

Tree-SHA512: e99ffcca8970921fd07fa9e04cf1ea2515a317409865d34ddfd70be0f0b0616b29d1fad58262d96a3c3418c0cf7018a6a955802a178b8f78f6ecfaa30a37d91c
2021-04-20 07:15:12 +02:00
fanquake
a839303edc
Merge #21244: Move GetDataDir to ArgsManager
bb8d1c6e02 Change ClearDataDirPathCache() to ArgsManager.ClearPathCache(). (Kiminuo)
b4190eff72 Change GetBlocksDir() to ArgsManager.GetBlocksDirPath(). (Kiminuo)
83292e2a70 scripted-diff: Modify unit tests to use the ArgsManager in the BasicTestingSetup class instead of implicitly relying on gArgs. (Kiminuo)
55c68e6f01 scripted-diff: Replace m_args with m_local_args in getarg_tests.cpp (Kiminuo)
511ce3a26b BasicTestingSetup: Add ArgsManager. (Kiminuo)
1cb52ba065 Modify "util_datadir" unit test to not use gArgs. (Kiminuo)
1add318704 Move GetDataDir(fNetSpecific) implementation to ArgsManager. (Kiminuo)
70cdf679f8 Move StripRedundantLastElementsOfPath before ArgsManager class. (Kiminuo)

Pull request description:

  This PR attempts to contribute to "Remove gArgs" (#21005).

  Main changes:

  * `GetDataDir()` function is moved to `ArgsManager.GetDataDirPath()`.
  * `GetBlocksDir()` function is moved to `ArgsManager.GetBlocksDirPath()`.

ACKs for top commit:
  ryanofsky:
    Code review ACK bb8d1c6e02. Just minor const/naming changes and splitting/scripting commits since last review
  MarcoFalke:
    review ACK bb8d1c6e02 📓
  hebasto:
    re-ACK bb8d1c6e02, addressed comments, and two commits made scripted-diffs since my [previous](https://github.com/bitcoin/bitcoin/pull/21244#pullrequestreview-638270583) review.

Tree-SHA512: ba9408c22129d6572beaa103dca0324131766f06d562bb7d6b9e214a0a4d40b0216ce861384562bde24b744003b3fbe6fac239061c8fd798abd3981ebc1b9019
2021-04-20 08:42:56 +08:00
Fabian Jahr
57a026c30f
test: Add unit test for Coinstats index 2021-04-19 20:31:02 +02:00
Fabian Jahr
9c8a265fd2
refactor: Pass hash_type to CoinsStats in stats object 2021-04-19 20:28:48 +02:00
MarcoFalke
cfec4a1dad
Merge bitcoin/bitcoin#21280: test: bug fix in transaction_tests
b109bde46a [test] check that mapFlagNames is up to date (glozow)
5d3ced72f9 [test] remove unnecessary OP_1s from invalid tests (glozow)
5aee73d175 [test] minor improvements / followups (glozow)
8a365df558 [test] fix bug in ExcludeIndividualFlags (glozow)
8cac2923f5 [test] remove invalid test from tx_valid.json (glozow)

Pull request description:

  This is a followup to #19698.

  - There was a bug in the `ExcludeIndividualFlags` function which is fixed here.
  - Fixing this bug also showed that there is a test that's supposed to fail (already existing in tx_invalid.json) in tx_valid.json, so I removed it. Other than that, the tests should all pass.
  - Also implements a few suggestions I received offline: removing the `OP_1`s from the invalid tests (similar to 19db590d04), comments, and style.
  - A few other small fixes, like adding asserts, putting all the flags in `mapFlagNames`, better error messages

ACKs for top commit:
  jnewbery:
    utACK b109bde46a

Tree-SHA512: 7233a8c0f1ae1172fac8000ea6e05384ecf79074c39948d118464868505c7f02f17e96503c81bd05c07adb2087648a5d93d9899e16fdefa6b7efcb51319444a9
2021-04-19 11:26:42 +02:00
Jon Atack
36fb036d25
p2p: allow NetPermissions::ClearFlag() only with PF_ISIMPLICIT
NetPermissions::ClearFlag() is currently only called in the codebase with
an `f` value of NetPermissionFlags::PF_ISIMPLICIT.

If that should change in the future, ClearFlag() should not be called
with `f` being a subflag of a multiflag, e.g. NetPermissionFlags::PF_RELAY
or NetPermissionFlags::PF_DOWNLOAD, as that would leave `flags` in an
invalid state corresponding to none of the existing NetPermissionFlags.

Therefore, allow only calling ClearFlag with the implicit flag for now.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-04-18 16:32:28 +02:00
Jon Atack
4e0d5788ba
test: add net permissions noban/download unit test coverage
to clarify/test the relationship and NetPermissions operations
involving the NetPermissionFlags PF_NOBAN and PF_DOWNLOAD.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-04-18 16:32:16 +02:00
Kiminuo
bb8d1c6e02 Change ClearDataDirPathCache() to ArgsManager.ClearPathCache(). 2021-04-18 12:07:00 +02:00
Kiminuo
83292e2a70 scripted-diff: Modify unit tests to use the ArgsManager in the BasicTestingSetup class instead of implicitly relying on gArgs.
-BEGIN VERIFY SCRIPT-
git ls-files src/test/dbwrapper_tests.cpp src/test/denialofservice_tests.cpp src/test/flatfile_tests.cpp src/test/fs_tests.cpp src/test/settings_tests.cpp src/test/util_tests.cpp | xargs sed -i 's/GetDataDir()/m_args.GetDataDirPath()/g';
-END VERIFY SCRIPT-
2021-04-18 11:59:28 +02:00
Kiminuo
55c68e6f01 scripted-diff: Replace m_args with m_local_args in getarg_tests.cpp
-BEGIN VERIFY SCRIPT-
git ls-files src/test/getarg_tests.cpp | xargs sed -i "s/m_args/m_local_args/g";
-END VERIFY SCRIPT-
2021-04-18 11:59:28 +02:00
Kiminuo
511ce3a26b BasicTestingSetup: Add ArgsManager. 2021-04-18 11:59:28 +02:00
Kiminuo
1cb52ba065 Modify "util_datadir" unit test to not use gArgs. 2021-04-18 11:59:25 +02:00
Kiminuo
1add318704 Move GetDataDir(fNetSpecific) implementation to ArgsManager. 2021-04-17 20:09:01 +02:00
MarcoFalke
0dd7b23489
Merge #21391: [Bundle 5/n] Prune g_chainman usage in RPC modules
586190f0b4 rpc/rest: Take and reuse local Chain/ChainState obj (Carl Dong)
bc3bd36902 rpc: style: Improve BuriedForkDescPushBack signature (Carl Dong)
f99913969f rpc: Remove unnecessary casting of block height (Carl Dong)
6a3d192020 rpc: Tidy up local references (see commit message) (Carl Dong)
038854f31e rest/rpc: Remove now-unused old Ensure functions (Carl Dong)
6fb65b49f4 scripted-diff: rest/rpc: Use renamed EnsureAny*() (Carl Dong)
1570c7ee98 rpc: Add renamed EnsureAny*() functions (Carl Dong)
306b1cd3ee rpc: Add alt Ensure* functions acepting NodeContext (Carl Dong)
d7824acdb9 rest: Use existing NodeContext (Carl Dong)
3f08934799 rest: Pass in NodeContext to rest_block (Carl Dong)
7be0671b95 rpc/rawtx: Use existing NodeContext (Carl Dong)
60dc05afc6 rpc/mining: Use existing NodeContext (Carl Dong)
d485e815e2 rpc/blockchain: Use existing NodeContext (Carl Dong)
d0abf0bf42 rpc/*,rest: Add review-only assertion to EnsureChainman (Carl Dong)
cced0f46c9 miner: Pass in previous CBlockIndex to RegenerateCommitments (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Based on:
  - [x] #21270 | [Bundle 4/n] Prune g_chainman usage in validation-adjacent modules
  - [x] #21525 | [Bundle 4.5/n] Followup fixups to bundle 4

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  	1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  	2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  	3. Remove `old_function`

ACKs for top commit:
  ryanofsky:
    Code review ACK 586190f0b4. Since last review, no changes to existing commits, just some simple new commits added: three new commits renaming std::any Ensure functions (scripted diff commit and manual pre/post commits), and one new commit factoring out a repeated `ActiveChain()` call made in a loop. Thanks for the updates!
  jnewbery:
    utACK 586190f0b4
  MarcoFalke:
    review ACK 586190f0b4 🍯

Tree-SHA512: 64b677fb50141805b55c3f1afe68fcd298f9a071a359bdcd63256d52e334f83e462f31fb3ebee9b630da8f1d912a03a128cfc38179e7aaec29a055744a98478c
2021-04-17 17:37:37 +02:00
MarcoFalke
fa8eaee6a8
test: Run versionbits_sanity for all chains 2021-04-17 11:29:50 +02:00
MarcoFalke
f5e8bcf985
Merge #21689: test: Remove intermittently failing and not very meaningful BOOST_CHECK in cnetaddr_basic
63631beef6 test: Remove intermittently failing and not very meaningful `BOOST_CHECK` in `cnetaddr_basic` (practicalswift)

Pull request description:

  Remove intermittently failing and not very meaningful `BOOST_CHECK` in `cnetaddr_basic`.

  Fixes #21682.

  Rationale from https://github.com/bitcoin/bitcoin/issues/21682#issuecomment-819897122:

  > I've looked at that test before and I don't think that specific `BOOST_CHECK` makes much sense TBH :)
  >
  > 1.) I don't understand why we test if `ToString()` output includes `%zone_index`: it clearly doesn't on some platforms, so we cannot rely on it anyways. Then why test it?
  >
  > 2.) And perhaps more fundamentally: why would we even _want_ to have `%zone_index` in our textual `ToString()` output? I think the expectation is to get say `fe80::1ff:fe23:4567:890a` (without zone index) and not say `fe80::1ff:fe23:4567:890a%eth2 ` or `fe80::1ff:fe23:4567:890a%3 `when doing `ipv6_addr.ToString()` :)

ACKs for top commit:
  MarcoFalke:
    review ACK 63631beef6

Tree-SHA512: 06863d1edfb9ad1ca9bcae09cf3f0f47b58bb29d222b70799c3dc059b96452889026e4b99b132782846d9896e3e798d17c7f9406e0e6a0bec1bffc6edb54e9df
2021-04-17 11:18:51 +02:00
MarcoFalke
faec1e9ee1
test: Address outstanding versionbits_test feedback
* https://github.com/bitcoin/bitcoin/pull/21377#discussion_r609585080
* https://github.com/bitcoin/bitcoin/pull/21377#discussion_r613702341
2021-04-17 10:40:47 +02:00
MarcoFalke
fad4167871
test: Check that no versionbits are re-used
This allows to remove check that windows for the same bit are disjoint

This addresses https://github.com/bitcoin/bitcoin/pull/21377#discussion_r611492633
2021-04-17 10:40:43 +02:00
MarcoFalke
c6b30ccb2e
Merge #21630: fuzz: split FuzzedSock interface and implementation
549c82ad3a fuzz: use ConsumeBool() instead of !ConsumeBool() (Vasil Dimov)
29ae1c13a5 fuzz: split FuzzedSock interface and implementation (Vasil Dimov)
9668e43d8e fuzz: make FuzzedSock::Wait() sometimes simulate an occurred event (Vasil Dimov)
0c90ff1429 fuzz: set errno from FuzzedSock::Wait() if it simulates a failure (Vasil Dimov)
5198a02de4 style: remove extra white space (Vasil Dimov)

Pull request description:

  * split FuzzedSock interface and implementation
  * make FuzzedSock::Wait() sometimes simulate an occurred event
  * set errno from FuzzedSock::Wait() if it simulates a failure

  (this is a followup from https://github.com/bitcoin/bitcoin/pull/21617)

ACKs for top commit:
  practicalswift:
    cr ACK 549c82ad3a: patch looks correct and touches only `src/test/fuzz/`
  MarcoFalke:
    re-ACK 549c82ad3a only change is rebase 🎬

Tree-SHA512: 8ba965a8319074ad2ef840219c35c77e37cc79f00fb3926f20ccbf5f58e9616f5a3ac96434ad33996b47d292fa760d5d00a529001ac0d1d254262e5df93f616f
2021-04-15 10:48:39 +02:00
MarcoFalke
a5e756b74e
Merge #21676: test: Use mocktime to avoid intermittent failure in rpc_tests
fa40d6a1c4 test: Reset mocktime in the common setup (MarcoFalke)
fa78590a8f test: Use mocktime to avoid intermittent failure (MarcoFalke)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/21602#discussion_r611176103

ACKs for top commit:
  jonatack:
    Code review ACK fa40d6a1c4
  jarolrod:
    ACK fa40d6a1c4

Tree-SHA512: 4967e006f3d2c4eb92f03c9086a6abe3190ad54755d251c30d20422c574bb1a154c06f3d5bcb0d4deaa3c4abfd3864d743b71d84897edd358e829bb42233ad12
2021-04-15 10:05:21 +02:00
Vasil Dimov
549c82ad3a
fuzz: use ConsumeBool() instead of !ConsumeBool()
The former is shorter and ends up with a "random" bool anyway.
2021-04-15 08:51:39 +02:00
Vasil Dimov
29ae1c13a5
fuzz: split FuzzedSock interface and implementation
Move the `FuzzedSock`'s implementation from `src/test/fuzz/util.h` to
`src/test/fuzz/util.cpp`.

A separate interface and implementation make the code more readable for
consumers who don't need to (better not) know the implementation
details.
2021-04-15 08:51:36 +02:00
Vasil Dimov
9668e43d8e
fuzz: make FuzzedSock::Wait() sometimes simulate an occurred event 2021-04-15 08:19:49 +02:00
Vasil Dimov
0c90ff1429
fuzz: set errno from FuzzedSock::Wait() if it simulates a failure 2021-04-15 08:19:48 +02:00
Vasil Dimov
5198a02de4
style: remove extra white space 2021-04-15 08:19:44 +02:00
practicalswift
63631beef6 test: Remove intermittently failing and not very meaningful BOOST_CHECK in cnetaddr_basic 2021-04-15 06:19:30 +00:00
MarcoFalke
9712f75746
Merge #21677: fuzz: Avoid use of low file descriptor ids (which may be in use) in FuzzedSock
6262182b3f Avoid use of low file descriptor ids (which may be in use) in FuzzedSock and StaticContentsSock (practicalswift)

Pull request description:

  Avoid use of low file descriptor ids (which may be in use) in `FuzzedSock`.

  Context: https://github.com/bitcoin/bitcoin/pull/21630/files#r610694541

ACKs for top commit:
  vasild:
    ACK 6262182b3f

Tree-SHA512: e622acb4d01446c3db01adbbbb779038be7247e13f3f4e72c614bc2880c3efd710fd3b189f87abb00f236fa5ddf91f4c215f420ca4eb08a97aaba31593254c3d
2021-04-15 08:02:22 +02:00
fanquake
2cd834e6c0
Merge #21377: Speedy trial support for versionbits
ffe33dfbd4 chainparams: drop versionbits threshold to 90% for mainnnet and signet (Anthony Towns)
f054f6bcd2 versionbits: simplify state transitions (Anthony Towns)
55ac5f568a versionbits: Add explicit NEVER_ACTIVE deployments (Anthony Towns)
dd07e6da48 fuzz: test versionbits delayed activation (Anthony Towns)
dd85d5411c tests: test versionbits delayed activation (Anthony Towns)
73d4a70639 versionbits: Add support for delayed activation (Anthony Towns)
9e6b65f6fa tests: clean up versionbits test (Anthony Towns)
5932744450 tests: test ComputeBlockVersion for all deployments (Anthony Towns)
63879f0a47 tests: pull ComputeBlockVersion test into its own function (Anthony Towns)

Pull request description:

  BIP9-based implementation of "speedy trial" activation specification, see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-March/018583.html

  Edge cases are tested by fuzzing added in #21380.

ACKs for top commit:
  instagibbs:
    tACK ffe33dfbd4
  jnewbery:
    utACK ffe33dfbd4
  MarcoFalke:
    review ACK ffe33dfbd4 💈
  achow101:
    re-ACK ffe33dfbd4
  gmaxwell:
    ACK ffe33dfbd4
  benthecarman:
    ACK ffe33dfbd4
  Sjors:
    ACK ffe33dfbd4
  jonatack:
    Initial approach ACK ffe33dfbd4 after a first pass of review, building and testing each commit, mostly looking at the changes and diffs. Will do a more high-level review iteration. A few minor comments follow to pick/choose/ignore.
  ariard:
    Code Review ACK ffe33df

Tree-SHA512: f79a7146b2450057ee92155cbbbcec12cd64334236d9239c6bd7d31b32eec145a9781c320f178da7b44ababdb8808b84d9d22a40e0851e229ba6d224e3be747c
2021-04-15 10:04:14 +08:00
practicalswift
6262182b3f Avoid use of low file descriptor ids (which may be in use) in FuzzedSock and StaticContentsSock 2021-04-14 22:21:17 +00:00
James O'Beirne
931684b24a
validation: fix ActivateSnapshot to use hardcoded nChainTx
This fixes an oversight from the move of nChainTx from the user-supplied
snapshot metadata into the hardcoded assumeutxo chainparams.

Since the nChainTx is now unused in the metadata, it should be removed
in a future commit.
2021-04-14 13:29:27 -04:00
MarcoFalke
fa40d6a1c4
test: Reset mocktime in the common setup
Doing it there will reduce code bloat and also ensure no test can "forget" to reset it
2021-04-14 17:38:07 +02:00
MarcoFalke
fa78590a8f
test: Use mocktime to avoid intermittent failure 2021-04-14 10:05:09 +02:00
Vasil Dimov
e286cd0d7b
net: flag relevant Sock methods with [[nodiscard]] 2021-04-13 17:26:20 +02:00
fanquake
1f14130cb0
Merge #21575: refactor: Create blockstorage module
fadcd3f78e doc: Remove irrelevant link to GitHub (MarcoFalke)
fa121b628d blockstorage: [refactor] Use chainman reference where possible (MarcoFalke)
fa0c7d9ad2 move-only: Move *Disk functions to blockstorage (MarcoFalke)
fa91b2b2b3 move-only: Move AbortNode to shutdown (MarcoFalke)
fa413f07a1 move-only: Move ThreadImport to blockstorage (MarcoFalke)
faf843c07f refactor: Move load block thread into ChainstateManager (MarcoFalke)

Pull request description:

  This picks up the closed pull request #21030 and is the first step toward fixing #21220.

  The basic idea is to move all disk access into a separate module with benefits:
  * Breaking down the massive files init.cpp and validation.cpp into logical units
  * Creating a standalone-module to reduce the mental complexity
  * Pave the way to fix validation related circular dependencies
  * Pave the way to mock disk access for testing, especially where it is performance critical (like fuzzing)

ACKs for top commit:
  promag:
    Code review ACK fadcd3f78e, checked (almost) moved only changes. This is a nice tidy up change and doesn't change behavior. Easily reviewed commit by commit.
  jamesob:
    ACK fadcd3f78e ([`jamesob/ackr/21575.1.MarcoFalke.refactor_create_blocksto`](https://github.com/jamesob/bitcoin/tree/ackr/21575.1.MarcoFalke.refactor_create_blocksto))
  ryanofsky:
    Code review ACK fadcd3f78e. New organization makes sense, moves extraneous things outside of validation.cpp. PR is also easy to review with helpfully split up moveonly commits.

Tree-SHA512: 917996592b6d8f9998289d8cb2b1b78b23d1fdb3b07216c9caec1380df33baa09dc2c1e706da669d440b497e79c9c62a01ca20dc202df5ad974a75f3ef7a143b
2021-04-13 22:00:28 +08:00
fanquake
88331aa8a7
Merge #21633: refactor: add [[noreturn]] attribute where applicable
003929c0d5 refactor: add [[noreturn]] attribute where applicable (fanquake)

Pull request description:

  Similar to #10843. We could build with `-Wmissing-noreturn`, however that would also mean modifying something like `--suppress-external-warnings` to suppress warnings for leveldb, which I don't think we want to do. In any case, the functions where this is applicable are only added/removed very rarely.

ACKs for top commit:
  vasild:
    ACK 003929c0d5

Tree-SHA512: 33dfa6547d6b84f38a941f24d4c2effe8fde7b93dbc0b27a9309716420e4a879fdbe689d789fa5439d65f5f78292f89fd9dc1b61c97acf69316dfed954086705
2021-04-13 21:17:20 +08:00
fanquake
bd65a76b9d
Merge #21330: Deal with missing data in signature hashes more consistently
725d7ae049 Use PrecomputedTransactionData in signet check (Pieter Wuille)
497718b467 Treat amount<0 also as missing data for P2WPKH/P2WSH (Pieter Wuille)
3820090bd6 Make all SignatureChecker explicit about missing data (Pieter Wuille)
b77b0cc507 Add MissingDataBehavior and make TransactionSignatureChecker handle it (Pieter Wuille)

Pull request description:

  Currently we have 2 levels of potentially-missing data in the transaction signature hashes:
  * P2WPKH/P2WSH hashes need the spent amount
  * P2TR hashes need all spent outputs (amount + scriptPubKey)

  Missing amounts are treated as -1 (thus leading to unexpected signature failures), while missing outputs in P2TR validation cause assertion failure. This is hard to extend for signing support, and also quite ugly in general.

  In this PR, an explicit configuration option to {Mutable,}TransactionSignatureChecker is added (MissingDataBehavior enum class) to either select ASSERT_FAIL or FAIL. Validation code passes ASSERT_FAIL (as at validation time all data should always be passed, and anything else is a serious bug in the code), while signing code uses FAIL.

  The existence of the ASSERT_FAIL option is really just an abundance of caution. Always using FAIL should be just fine, but if there were for some reason a code path in consensus code was introduced that misses certain data, I think we prefer as assertion failure over silently introducing a consensus change.

  Potentially useful follow-ups (not for this PR, in my preference):
  * Having an explicit script validation error code for missing data.
  * Having a MissingDataBehavior::SUCCEED option as well, for use in script/sign.cpp DataFromTransaction (if a signature is present in a witness, and we don't have enough data to fully validate it, we should probably treat it as valid and not touch it).

ACKs for top commit:
  sanket1729:
    reACK 725d7ae049
  Sjors:
    ACK 725d7ae049
  achow101:
    re-ACK 725d7ae049
  benthecarman:
    ACK 725d7ae049
  fjahr:
    Code review ACK 725d7ae049

Tree-SHA512: d67dc51bae9ca7ef6eb9acccefd682529f397830f77d74cd305500a081ef55aede0e9fa380648c3a8dd4857aa7eeb1ab54fe808979d79db0784ac94ceb31b657
2021-04-13 10:24:31 +08:00
fanquake
003929c0d5
refactor: add [[noreturn]] attribute where applicable 2021-04-13 08:59:21 +08:00
Anthony Towns
f054f6bcd2 versionbits: simplify state transitions
This removes the DEFINED->FAILED transition and changes the
STARTED->FAILED transition to only occur if signalling didn't pass the
threshold. This ensures that it is always possible for activation to
occur, no matter what settings are chosen, or the speed at which blocks
are found.
2021-04-12 11:14:49 +10:00
Anthony Towns
55ac5f568a versionbits: Add explicit NEVER_ACTIVE deployments
Previously we used deployments that would timeout prior to Bitcoin's
invention, which allowed the deployment to still be activated in unit
tests. This switches those deployments to be truly never active.
2021-04-12 11:14:49 +10:00
Anthony Towns
dd07e6da48 fuzz: test versionbits delayed activation 2021-04-12 11:14:49 +10:00
Anthony Towns
dd85d5411c tests: test versionbits delayed activation 2021-04-12 11:14:49 +10:00
Anthony Towns
73d4a70639 versionbits: Add support for delayed activation 2021-04-12 11:14:49 +10:00
Anthony Towns
9e6b65f6fa tests: clean up versionbits test
Simplify the versionbits unit test slightly to make the next set of
changes a little easier to follow.
2021-04-12 10:47:42 +10:00
Anthony Towns
5932744450 tests: test ComputeBlockVersion for all deployments
This generalises the ComputeBlockVersion test so that it can apply to
any activation parameters we might set, and checks all the parameters
set for each deployment on each chain, to simultaneously ensure that the
deployments we have configured work sensibly, and that the test code
does not suffer bitrot in the event that all interesting deployments
are buried.
2021-04-12 10:47:42 +10:00
Anthony Towns
63879f0a47 tests: pull ComputeBlockVersion test into its own function
The intent here is to allow checking ComputeBlockVersion behaviour with
each deployment, rather than only testdummy on mainnet. This commit does
the trivial refactoring component of that change.
2021-04-12 10:44:04 +10:00
MarcoFalke
f6c44e999b
Merge #21602: rpc: add additional ban time fields to listbanned
d3b0b08b0f doc: release notes for new listbanned fields (Jarol Rodriguez)
60290d3f5e test: increase listbanned unit test coverage (Jon Atack)
3e978d1a5d rpc: add time_remaining field to listbanned (Jarol Rodriguez)
5456b34531 rpc: add ban_duration field to listbanned (Jarol Rodriguez)
c95c61657a doc: improve listbanned help (Jarol Rodriguez)
dd3c8eaa33 rpc: swap position of banned_until and ban_created fields (Jarol Rodriguez)

Pull request description:

  This PR adds a `ban_duration` and `time_remaining` field to the `listbanned` RPC command. Thanks to jonatack, this PR also expands the `listbanned` test coverage to include these new fields

  It's useful to keep track of `ban_duration` as this is another data point on which to sort banned peers. I found this helpful in adding additional context columns to the GUI `bantablemodel` as part of a follow-up PR. As [suggested](https://github.com/bitcoin/bitcoin/pull/21602#issuecomment-813486134) by jonatack, `time_remaining` is another useful user-centric data point.

  Since a ban always expires after its created, the `ban_created` field is now placed before the `banned_until` field. This new ordering is more logical.

  This PR also improves the `help listbanned` output by providing additional context to the descriptions of the `address`, `ban_created`, and `banned_until` fields.

  **Master: listbanned**
  ```
  [
    {
      "address": "1.2.3.4/32",
      "banned_until": 1617691101,
      "ban_created": 1617604701
    },
    {
      "address": "135.181.41.129/32",
      "banned_until": 1649140716,
      "ban_created": 1617604716
    }
  ]
  ```

  **PR: listbanned**
  ```
  [
    {
      "address": "1.2.3.4/32",
      "ban_created": 1617775773,
      "banned_until": 1617862173,
      "ban_duration": 86400,
      "time_remaining": 86392
    },
    {
      "address": "3.114.211.172/32",
      "ban_created": 1617753165,
      "banned_until": 1618357965,
      "ban_duration": 604800,
      "time_remaining": 582184
    }
  ]
  ```

ACKs for top commit:
  jonatack:
    re-ACK d3b0b08b0f
  hebasto:
    ACK d3b0b08b0f, tested on Linux Mint 20.1 (x86_64).
  MarcoFalke:
    review ACK d3b0b08b0f 🕙

Tree-SHA512: 5b83ed2483344e546d57e43adc8a1ed7a1fff292124b14c86ca3a1aa2aec8b0f7198212fabff2c5145e7f726ca04ae567fe667b141254c7519df290cf63774e5
2021-04-11 13:36:29 +02:00
MarcoFalke
f0fa32450e
Merge #21606: fuzz: Extend psbt fuzz target a bit
faaf3954e2 fuzz: Extend psbt fuzz target a bit (MarcoFalke)

Pull request description:

  Previously it only merged the psbt with itself, now it tries to merge another.

ACKs for top commit:
  practicalswift:
    Tested ACK faaf3954e2

Tree-SHA512: e1b1d31a47d35e1767285bc2fda176c79cb0550d6d383fe467104272e61e1c83f6cbc0c7d6bbc0c3027729eec13ae1f289f8950117ee91e0fb3703e66d5e6918
2021-04-09 18:54:17 +02:00
MarcoFalke
faaf3954e2
fuzz: Extend psbt fuzz target a bit 2021-04-09 13:17:37 +02:00
MarcoFalke
4ad83a9597
Merge #21592: test: Remove option to make TestChain100Setup non-deterministic
fa6183d776 test: Remove option to make TestChain100Setup non-deterministic (MarcoFalke)
fa732bccb3 test: Use compressed keys in TestChain100Setup (MarcoFalke)

Pull request description:

  Seems odd to have an option for non-deterministic tests
  when the goal should be for all tests to be deterministic.

ACKs for top commit:
  jamesob:
    ACK fa6183d776
  practicalswift:
    cr ACK fa6183d776: patch looks deterministic!

Tree-SHA512: 6897a9f36e0dfb7d63b25dd6984414b3ee8a62458ad232cb21ed5077184fdb0bc626996e4ac84ef0bdd452b9f17c54aac75a71575b8e723b84cac07c9f9d5611
2021-04-09 07:43:10 +02:00
MarcoFalke
6664211be2
Merge #21574: Drop JSONRPCRequest constructors after #21366
9044522ef7 Drop JSONRPCRequest constructors after #21366 (Russell Yanofsky)

Pull request description:

  This just makes an additional simplification after #21366 replaced
  util::Ref with std::any. It was originally suggested
  https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351 but
  delayed for a followup. It would have prevented usage bug
  https://github.com/bitcoin/bitcoin/pull/21572.

ACKs for top commit:
  promag:
    ACK 9044522ef7, fixed conflict in src/wallet/interfaces.cpp.

Tree-SHA512: e909411b8f75013620b94e1a609296befb832fdcb574cd2e6689bfe3c636b03cd4ac1ccb2b32b532daf0f2131bb043464024966310fffc7e3cad77713d4bd0ef
2021-04-08 09:08:03 +02:00
MarcoFalke
fa6183d776
test: Remove option to make TestChain100Setup non-deterministic
Seems odd to have an option for non-deterministic tests
when the goal should be for all tests to be deterministic.

Can be reviewed with `--ignore-all-space`.
2021-04-08 08:59:00 +02:00
MarcoFalke
fa732bccb3
test: Use compressed keys in TestChain100Setup
coinbaseKey.MakeNewKey(true); creates a compressed key and there is no reason
for the deterministic setup to use uncompressed ones.
2021-04-08 08:58:44 +02:00
glozow
b109bde46a [test] check that mapFlagNames is up to date
There is no way to iterate through all script verification flags, and
it's not guaranteed that every power of 2 is used. Just make sure that
all flags in STANDARD_SCRIPT_VERIFY_FLAGS are present in mapFlagNames;
this covers all consensus and policy flags. If mapFlagNames has more
flags than STANDARD_SCRIPT_VERIFY_FLAGS, that's okay. Nonexistent flags
will be caught by the compiler.
2021-04-07 19:00:23 -07:00
glozow
5d3ced72f9 [test] remove unnecessary OP_1s from invalid tests
Similar to 19db590d04, which removed these
for the valid tests. Not removing ones that cause a false/empty stack
error because these tests should fail due to being invalid with CSV/CLTV
2021-04-07 19:00:17 -07:00
glozow
5aee73d175 [test] minor improvements / followups
Add missing script verify flags to mapFlagNames.
iterate through mapFlagNames values instead of bits.

BOOST_CHECK_MESSAGE better reports which test failed exactly, whereas
BOOST_ERROR was just incrementing the error counter.
2021-04-07 19:00:17 -07:00
glozow
8a365df558 [test] fix bug in ExcludeIndividualFlags
PR #19168 introduced this function but it always returns an empty vector.
2021-04-07 19:00:17 -07:00
fanquake
2e9031f95d
Merge #21626: doc: Fix typos from codespell
94c7dd9ac8 doc: Fix typos from codespell lint (Yerzhan Mazhkenov)

Pull request description:

  Typos from codespell linter: https://cirrus-ci.com/task/6677401661865984?logs=lint#L856
  - txrequest.cpp: `annoucements` ==> `announcements`
  - contrib/guix/README.md:298: `stil` ==> `still`
  - contrib/guix/guix-build:18: `invokable` ==> `invocable`
  - contrib/guix/libexec/prelude.bash:12: `invokable` ==> `invocable`
  - src/test/fuzz/tx_pool.cpp:37: `acess` ==> `access`
  - src/txorphanage.h:29: `orginating` ==> `originating`

ACKs for top commit:
  practicalswift:
    cr ACK 94c7dd9ac8: thnaks fro fiixng tpyos!
  jarolrod:
    ACK 94c7dd9ac8

Tree-SHA512: e0fac462a2f9e68b6a161c9f5d95b4d0648ce5c618fd7cd243d57db8f0256138b8823b166ea406b21e95586eae43047df1ef0df04616858082a39c1d1eb13a86
2021-04-08 08:16:04 +08:00
Yerzhan Mazhkenov
94c7dd9ac8 doc: Fix typos from codespell lint 2021-04-07 19:26:25 +01:00
Russell Yanofsky
9044522ef7 Drop JSONRPCRequest constructors after #21366
This just makes an additional simplification after #21366 replaced
util::Ref with std::any. It was originally suggested
https://github.com/bitcoin/bitcoin/pull/21366#issuecomment-792044351 but
delayed for a followup. It would have prevented usage bug
https://github.com/bitcoin/bitcoin/pull/21572.
2021-04-07 04:53:26 -04:00
MarcoFalke
6154291cf9
Merge #21617: fuzz: Fix uninitialized read in i2p test
33333755f2 fuzz: Fix uninitialized read in test (MarcoFalke)

Pull request description:

  Can be tested with:

  ```
  ./test/fuzz/test_runner.py -l DEBUG --valgrind ../btc_qa_assets/fuzz_seed_corpus/ i2p
  ```

  ```
  ==22582== Conditional jump or move depends on uninitialised value(s)
  ==22582==    at 0x6BB2D8: __sanitizer_cov_trace_const_cmp1 (in /tmp/bitcoin-core/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/test/fuzz/fuzz)
  ==22582==    by 0xB305DB: ConnectSocketDirectly(CService const&, Sock const&, int, bool) (netbase.cpp:570)
  ==22582==    by 0x8AAA5D: i2p::sam::Session::Hello() const (i2p.cpp:284)
  ==22582==    by 0x8A6FA0: i2p::sam::Session::CreateIfNotCreatedAlready() (i2p.cpp:352)
  ==22582==    by 0x8A6742: i2p::sam::Session::Listen(i2p::Connection&) (i2p.cpp:134)
  ==22582==    by 0x7A6C42: i2p_fuzz_target(Span<unsigned char const>) (i2p.cpp:37)

ACKs for top commit:
  sipa:
    utACK 33333755f2
  vasild:
    ACK 33333755f2

Tree-SHA512: 36073582b26b541324b3e55f3fd4a44abf89cb3081f36d361525daf8c27602fbc25f736510ec30df7cb4ca0c4e395e8d8a60f531bf6af358b5a3e65dbabf72c0
2021-04-07 10:39:27 +02:00
Jon Atack
60290d3f5e test: increase listbanned unit test coverage
Add test coverage for the new ban_duration and time_remaining fields.
While here, some code improvements.
2021-04-07 01:57:26 -04:00
MarcoFalke
41a8d2b96f
Merge #21582: Fix assumeutxo crash due to missing base_blockhash
fa9b74f5ea Fix assumeutxo crash due to missing base_blockhash (MarcoFalke)
fa8fffebe8 refactor: Prefer clean assert over UB in coinstats (MarcoFalke)

Pull request description:

  This fixes an UB (which results in a crash with sanitizers enabled). Can be reproduced by cherry-picking the test without the other code changes. The fix:

  * Adds an `Assert` to transform the UB into a clean crash, even when sanitizers are disabled
  * Adds an early-fail condition to avoid the crash

ACKs for top commit:
  jamesob:
    ACK fa9b74f5ea ([`jamesob/ackr/21582.1.MarcoFalke.fix_assumeutxo_crash_due`](https://github.com/jamesob/bitcoin/tree/ackr/21582.1.MarcoFalke.fix_assumeutxo_crash_due))
  ryanofsky:
    Code review ACK fa9b74f5ea with no code changes since last review, just splitting up combocommit a little.

Tree-SHA512: dd36808a09f49c647543a9eaa6fdb785b3f1109af48ba4cc983153b22a144da9ca61af22034dcfaa0e192a65b1ee7de744f187555079aff55bec0efa0ce87cd4
2021-04-07 07:33:27 +02:00
MarcoFalke
33333755f2
fuzz: Fix uninitialized read in test 2021-04-06 12:43:33 +02:00
MarcoFalke
1a7dec77f6
Merge #21571: test: make sure non-IP peers get discouraged and disconnected (vasild)
81747b2171 test: make sure non-IP peers get discouraged and disconnected (Vasil Dimov)
637bb6da36 test: also check disconnect in denialofservice_tests/peer_discouragement (Vasil Dimov)
4d6e246fa4 test: use pointers in denialofservice_tests/peer_discouragement (Vasil Dimov)

Pull request description:

  Split up from #20966, so that it can be backported easier. Merging this ahead of #20966 will also reduce the number of conflicts for that pull.

ACKs for top commit:
  jonatack:
    ACK 81747b2171

Tree-SHA512: 8f0e30b95baba7f056920d7fc3b37bd49ee13e69392fe80e2d333c6bb09fd25f4603249301b8795cca26a2f2d15b9f8904798a55cd9c04fd28afb316e95c551c
2021-04-06 10:26:33 +02:00
MarcoFalke
fa0c7d9ad2
move-only: Move *Disk functions to blockstorage
Can be reviewed with the git options
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-04-05 20:26:14 +02:00
Carl Dong
cced0f46c9 miner: Pass in previous CBlockIndex to RegenerateCommitments 2021-04-05 11:13:51 -04:00
W. J. van der Laan
5c9b06db81
Merge #21302: wallet: createwallet examples for descriptor wallets
5039e0e55a test: HelpExampleCliNamed and HelpExampleRpcNamed (Ivan Metlushko)
591735ef0b rpc: Add HelpExampleCliNamed and use it for `createwallet` doc (Wladimir J. van der Laan)
5d5a90e819 rpc: Add HelpExampleRpcNamed (Ivan Metlushko)

Pull request description:

  Rationale: make descriptor wallets more visible and just a bit easier to setup

  `bitcoin-cli help createwallet`

  **Before**:
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

  **After**
  ```
  Examples:
  > bitcoin-cli createwallet "testwallet"
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["testwallet"]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  > bitcoin-cli createwallet "descriptors" false false "" true true true
  > curl --user myusername --data-binary '{"jsonrpc": "1.0", "id": "curltest", "method": "createwallet", "params": ["descriptors", false, false, "", true, true, true]}' -H 'content-type: text/plain;' http://127.0.0.1:8332/
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 5039e0e55a

Tree-SHA512: d37210e6ce639addee881377092d8f6fb2a537a60a259c561899e24cf68a0254d7ff45a213573c938f626677e46770cd21113aae5974f26c66b9a2e137699c14
2021-04-05 15:31:41 +02:00
MarcoFalke
fa9b74f5ea
Fix assumeutxo crash due to missing base_blockhash 2021-04-04 07:38:02 +02:00
Vasil Dimov
81747b2171 test: make sure non-IP peers get discouraged and disconnected 2021-04-02 18:33:45 +02:00
Vasil Dimov
637bb6da36 test: also check disconnect in denialofservice_tests/peer_discouragement
Use `CConnmanTest` instead of `CConnman` and add the nodes to it
so that their `fDisconnect` flag is set during disconnection.
2021-04-02 18:32:51 +02:00
Vasil Dimov
4d6e246fa4 test: use pointers in denialofservice_tests/peer_discouragement
This is a non-functional change that replaces the `CNode` on-stack
variables with `CNode` pointers.

The reason for this is that it would allow us to add those `CNode`s
to `CConnman::vNodes[]` which in turn would allow us to check that they
are disconnected properly - a `CNode` object must be in
`CConnman::vNodes[]` in order for its `fDisconnect` flag to be set.

If we store pointers to the on-stack variables in `CConnman` then it
would crash at the end, trying to `delete` them.
2021-04-02 18:32:46 +02:00
MarcoFalke
80a699fda9
Merge #21525: [Bundle 4.5/n] Followup fixups to bundle 4
693414d271 node/ifaces: ChainImpl: Use an accessor for ChainMan (Carl Dong)
98c4e252f0 node/ifaces: NodeImpl: Use an accessor for ChainMan (Carl Dong)
7e8b5ee814 validation: Make BlockManager::LookupBlockIndex const (Carl Dong)
88aead263c node: Avoid potential UB by asserting assumptions (Carl Dong)
1dd8ed7a84 net_processing: Move comments to declarations (Carl Dong)
07156eb387 node/coinstats: Replace #include with fwd-declaration (Carl Dong)
7b8e976cd5 miner: Add chainstate member to BlockAssembler (Carl Dong)
e62067e7bc Revert "miner: Pass in chainstate to BlockAssembler::CreateNewBlock" (Carl Dong)
eede0647b0 Revert "scripted-diff: Invoke CreateNewBlock with chainstate" (Carl Dong)
0c1b2bc549 Revert "miner: Remove old CreateNewBlock w/o chainstate param" (Carl Dong)

Pull request description:

  Chronological history of this changeset:
  1. Bundle 4 (#21270) got merged
  2. Posthumous reviews were posted
  3. These changes were prepended in bundle 5
  4. More reviews were added in bundle 5
  5. Someone suggested that we split the prepended changes up to another PR
  6. This is that PR

  In the future, I will just do posthumous review changes in another PR instead. I apologize for the confusion.

  Addresses posthumous reviews on bundle 4:
    - From jnewbery:
      - https://github.com/bitcoin/bitcoin/pull/21270#issuecomment-796738048
        - I didn't fix this one, but I added a `TODO` comment so that we don't lost track of it
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592291225
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592296942
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592299738
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r592301704
    - From MarcoFalke:
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593096212
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593097032
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593097867
      - https://github.com/bitcoin/bitcoin/pull/21270#discussion_r593100570

  Addresses reviews on bundle 5:
  - Checking chainman existence before locking cs_main
    - MarcoFalke
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r596601776
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r596601876
  - Appropriate locking, usage of chainman, and control flow in `src/node/interfaces.cpp`
    - MarcoFalke
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r596601383
    - jnewbery
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597029360
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597029921
    - ryanofsky
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597163828
  - Style/comment formatting changes
    - jnewbery
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597026552
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597027186
  - Making LookupBlockIndex const
    - jnewbery
      - https://github.com/bitcoin/bitcoin/pull/21391#discussion_r597035062

ACKs for top commit:
  MarcoFalke:
    review ACK 693414d271 🛐
  ryanofsky:
    Code review ACK 693414d271. I reviewed this previously as part of #21391. I am a fan of the increasingly complicated bundle numbering, and kind of hope there in the next round there is some way we can get bundles 5.333333 and 5.666667!
  jamesob:
    ACK 693414d271 ([`jamesob/ackr/21525.1.dongcarl.bundle_4_5_n_followup_f`](https://github.com/jamesob/bitcoin/tree/ackr/21525.1.dongcarl.bundle_4_5_n_followup_f))

Tree-SHA512: 9bdc199f70400d01764e1bd03c25bdb6cff26dcef60e4ca3b649baf8d017a2dfc1f058099067962b4b6ccd32d078002b1389d733039f4c337558cb70324c0ee3
2021-04-01 10:58:53 +02:00
W. J. van der Laan
602b038d43
Merge #21366: refactor: replace util::Ref with std::any (C++17)
916ab0195d remove unused class util::Ref and its unit test (Sebastian Falbesoner)
8dbb87a393 refactor: replace util::Ref by std::any (C++17) (Sebastian Falbesoner)
95cccf8a4b util: introduce helper AnyPtr to access std::any instances (Sebastian Falbesoner)

Pull request description:

  As described in `util/ref.h`: "_This implements a small subset of the functionality in C++17's std::any class, and **can be dropped when the project updates to C++17**_". For accessing the contained object of a `std::any` instance, a helper template function `AnyPtr` is introduced (thanks to ryanofsky).

ACKs for top commit:
  hebasto:
    re-ACK 916ab0195d, with command
  ryanofsky:
    Code review ACK 916ab0195d. Changes since last review: rebase and replacing types with `auto`. I might have used `const auto*` and `auto*` instead of plain `auto` because I think the qualifiers are useful, but this is all good.

Tree-SHA512: fe2c3e4f5726f8ad40c61128339bb24ad11d2c261f71f7b934b1efe3e3279df14046452b0d9b566917ef61d5c7e0fd96ccbf35ff810357e305710f5002c27d47
2021-03-31 20:17:39 +02:00
glozow
8cac2923f5 [test] remove invalid test from tx_valid.json
This exact test is also already in tx_invalid.json#L29
It will also test with no-P2SH flags, so duplicating with
different flags is not necessary.
2021-03-31 06:14:26 -07:00
MarcoFalke
267b60f800
Merge #21553: fuzz: Misc refactor
fa4926cca6 fuzz: [refactor] Use IsValidFlagCombination in signature_checker fuzz target (MarcoFalke)
eeee8f5be1 fuzz: Removed unused try-catch in coins_view (MarcoFalke)
fa98f3f66e fuzz: [refactor] Use ConsumeScript in signature_checker fuzz target (MarcoFalke)

Pull request description:

  Some small refactors to remove unused and redundant fuzz code

ACKs for top commit:
  practicalswift:
    cr re-ACK fa4926cca6

Tree-SHA512: eb07a2140caad7b31495b76385fc7634cf5b6daa4947f430ebb127eb1375583dc11e541a0a42d0e5d93d430480b8a815b93974450fd5ed897528a2d47c752f86
2021-03-30 20:07:53 +02:00
Wladimir J. van der Laan
f9e86d8966
Merge #21387: p2p: Refactor sock to add I2P fuzz and unit tests
40316a37cb test: add I2P test for a runaway SAM proxy (Vasil Dimov)
2d8ac77970 fuzz: add tests for the I2P Session public interface (Vasil Dimov)
9947e44de0 i2p: use pointers to Sock to accommodate mocking (Vasil Dimov)
82d360b5a8 net: change ConnectSocketDirectly() to take a Sock argument (Vasil Dimov)
b5861100f8 net: add connect() and getsockopt() wrappers to Sock (Vasil Dimov)
5a887d49b2 fuzz: avoid FuzzedSock::Recv() repeated errors with EAGAIN (Vasil Dimov)
3088f83d01 fuzz: extend FuzzedSock::Recv() to support MSG_PEEK (Vasil Dimov)
9b05c49ade fuzz: implement unimplemented FuzzedSock methods (Vasil Dimov)

Pull request description:

  Change the networking code and the I2P code to be fully mockable and use `FuzzedSocket` to fuzz the I2P methods `Listen()`, `Accept()` and `Connect()`.

  Add a mocked `Sock` implementation that returns a predefined data on reads and use it for a regression unit test for the bug fixed in https://github.com/bitcoin/bitcoin/pull/21407.

ACKs for top commit:
  practicalswift:
    Tested ACK 40316a37cb
  MarcoFalke:
    Concept ACK 40316a37cb
  jonatack:
    re-ACK 40316a37cb reviewed `git range-diff 01bb3afb 23c861d 40316a3` and the new unit test commit, debug built, ran unit tests, ran bitcoind with an I2P service and network operation with seven I2P peers (2 in, 5 out) is looking nominal
  laanwj:
    Code review ACK 40316a37cb

Tree-SHA512: 7fc4f129849e16e0c7e16662d9f4d35dfcc369bb31450ee369a2b97bdca95285533bee7787983e881e5a3d248f912afb42b4a2299d5860ace7129b0b19623cc8
2021-03-30 17:41:13 +02:00
Wladimir J. van der Laan
dede9eb924
Merge #20197: p2p: protect onions in AttemptToEvictConnection(), add eviction protection test coverage
0cca08a8ee Add unit test coverage for our onion peer eviction protection (Jon Atack)
caa21f586f Protect onion+localhost peers in ProtectEvictionCandidatesByRatio() (Jon Atack)
8f1a53eb02 Use EraseLastKElements() throughout SelectNodeToEvict() (Jon Atack)
8b1e156143 Add m_inbound_onion to AttemptToEvictConnection() (Jon Atack)
72e30e8e03 Add unit tests for ProtectEvictionCandidatesByRatio() (Jon Atack)
ca63b53ecd Use std::unordered_set instead of std::vector in IsEvicted() (Jon Atack)
41f84d5ecc Move peer eviction tests to a separate test file (Jon Atack)
f126cbd6de Extract ProtectEvictionCandidatesByRatio from SelectNodeToEvict (Jon Atack)

Pull request description:

  Now that #19991 and #20210 have been merged, we can determine inbound onion peers using `CNode::m_inbound_onion` and add it to the localhost peers protection in `AttemptToEvictConnection`, which was added in #19670 to address issue #19500.

  Update 28 February 2021: I've updated this to follow gmaxwell's suggestion in https://github.com/bitcoin/bitcoin/pull/20197#issuecomment-713865992.

  This branch now protects up to 1/4 onion peers (connected via our tor control service), if any, sorted by longest uptime. If any (or all) onion slots remain after that operation, they are then allocated to protect localhost peers, or a minimum of 2 localhost peers in the case that no onion slots remain and 2 or more onion peers were protected, sorted as before by longest uptime.

  This patch also adds test coverage for the longest uptime, localhost, and onion peer eviction protection logic to build on the welcome initial unit testing of #20477.

  Suggest reviewing the commits that move code with `colorMoved = dimmed-zebra` and `colorMovedWs = allow-indentation-change`.

  Closes #11537.

ACKs for top commit:
  laanwj:
    Code review ACK 0cca08a8ee
  vasild:
    ACK 0cca08a8ee

Tree-SHA512: 2f5a63f942acaae7882920fc61f0185dcd51da85e5b736df9d1fc72343726dd17da740e02f30fa5dc5eb3b2d8345707aed96031bec143d48a2497a610aa19abd
2021-03-30 16:20:47 +02:00
MarcoFalke
1999baac30
Merge #20228: addrman: Make addrman a top-level component
3fc06d3d7b [net] remove fUpdateConnectionTime from FinalizeNode (John Newbery)
7c4cc67c0c [net] remove CConnman::AddNewAddresses (John Newbery)
bcd7f30b79 [net] remove CConnman::MarkAddressGood (John Newbery)
8073673dbc [net] remove CConnman::SetServices (John Newbery)
392a95d393 [net_processing] Keep addrman reference in PeerManager (John Newbery)
1c25adf6d2 [net] Construct addrman outside connman (John Newbery)

Pull request description:

  Addrman is currently a member variable of connman. Make it a top-level component with lifetime owned by node.context, and add a reference to addrman in peerman. This allows us to eliminate some functions in connman that are simply forwarding requests to addrman, and simplifies the connman-peerman interface.

  By constructing the addrman in init, we can also add parameters to the ctor, which allows us to test it better. See #20233, where we enable consistency checking for addrman in our functional tests.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3fc06d3d7b only change is squash 🏀
  vasild:
    ACK 3fc06d3d7b

Tree-SHA512: 17662c65cbedcd9bd1c194914bc4bb4216f4e3581a06222de78f026d6796f1da6fe3e0bf28c2d26a102a12ad4fbf13f815944a297f000e3acf46faea42855e07
2021-03-30 12:28:15 +02:00
MarcoFalke
fa4926cca6
fuzz: [refactor] Use IsValidFlagCombination in signature_checker fuzz target
Can be reviewed with --color-moved=dimmed-zebra
2021-03-30 10:42:45 +02:00
MarcoFalke
eeee8f5be1
fuzz: Removed unused try-catch in coins_view 2021-03-30 10:16:22 +02:00
MarcoFalke
fa98f3f66e
fuzz: [refactor] Use ConsumeScript in signature_checker fuzz target 2021-03-30 09:30:43 +02:00
Pieter Wuille
0b188b751f Clean up context dependent checks in descriptor parsing
This changes all context dependent checks in the parser to be
disjunctions of equality checks, rather than also including inequalities.
This makes sure that adding a new context enum in the future won't change
semantics for existing checks.

The error messages are also made a bit more consistent.
2021-03-29 17:44:13 -07:00
Sebastian Falbesoner
916ab0195d remove unused class util::Ref and its unit test 2021-03-29 23:29:47 +02:00
Sebastian Falbesoner
8dbb87a393 refactor: replace util::Ref by std::any (C++17) 2021-03-29 23:29:42 +02:00
MarcoFalke
1c7be9ab90
Merge #20286: rpc: deprecate addresses and reqSigs from rpc outputs
90ae3d8ca6 doc: Add release notes for -deprecatedrpc=addresses and bitcoin-tx (Michael Dietz)
085b3a7299 rpc: deprecate `addresses` and `reqSigs` from rpc outputs (Michael Dietz)

Pull request description:

  Considering the limited applicability of `reqSigs` and the confusing output of `1` in all cases except bare multisig, the `addresses` and `reqSigs` outputs are removed for all rpc commands.

  1) add a new sane "address" field (for outputs that have an identifiable address, which doesn't include bare multisig)
  2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact (with all weird/wrong behavior they have now)
  3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely always.

  Note: Some light refactoring done to allow us to very easily delete a few chunks of code (marked with TODOs) when we remove this deprecated behavior.

  Using `IsDeprecatedRPCEnabled` in core_write.cpp caused some circular dependencies involving core_io

  Circular dependencies were caused by rpc/util unnecessarily importing node/coinstats and node/transaction. Really what rpc/util needs are some fundamental type/helper-function definitions. So this was cleaned up to make more sense.

  This fixes #20102.

ACKs for top commit:
  MarcoFalke:
    re-ACK 90ae3d8ca6 📢

Tree-SHA512: 8ffb617053b5f4a8b055da17c06711fd19632e0037d71c4c8135e50c8cd7a19163989484e4e0f17a6cc48bd597f04ecbfd609aef54b7d1d1e76a784214fcf72a
2021-03-29 15:14:31 +02:00
MarcoFalke
cf11f9c22f
Merge #21531: test: remove qt byteswap compattests
9ac86bcc0d test: remove qt byteswap compattests (fanquake)

Pull request description:

  These were added as part of #9366 when with fixing issues with Protobuf.

  Now that we no-longer use Protobuf, there's no reason to maintain a duplicate set of byteswap tests in the qt tests. Our other set of byteswap tests are here: https://github.com/bitcoin/bitcoin/blob/master/src/test/bswap_tests.cpp.

ACKs for top commit:
  laanwj:
    Code review ACK 9ac86bcc0d

Tree-SHA512: 72ba131a5f8fbd9fdbbc4e1f95baa794496c960b12e0271700c632c6511b7e1b331e8db07a201838b4d56b2aeeb43d4de4e10265ea07ab14241307fa14d3342e
2021-03-29 11:52:58 +02:00
MarcoFalke
97e6a7f98c
Merge #21477: test: Add test for CNetAddr::ToString IPv6 address formatting (RFC 5952)
732c7bddeb tests: Add test for CNetAddr::ToString IPv6 address formatting (RFC 5952) (practicalswift)

Pull request description:

  Test that `CNetAddr::ToString` formats IPv6 addresses with zero compression and canonicalisation as described in [RFC 5952 ("A Recommendation for IPv6 Address Text Representation")](https://tools.ietf.org/html/rfc5952).

  Solving #21466 will hopefully be trivial with the ability to check zero compression correctness against these tests.

ACKs for top commit:
  vasild:
    ACK 732c7bddeb

Tree-SHA512: 31a1378aa435ba4171490a2e15d7280a175292270eb001b47d367e010c6ac9b83420b82bbeab22211f8f500c69e21878047c87adf216263b3420b6bb2a5d2bfb
2021-03-29 09:27:05 +02:00
fanquake
9ac86bcc0d
test: remove qt byteswap compattests
These were added as part of #9366 to fix issues with Protobuf.

Now that we no-longer use Protobuf, there's no reason to maintain a
duplicate set of byteswap tests for qt.
2021-03-29 11:12:26 +08:00
MarcoFalke
9217f9fe73
Merge #21522: fuzz: [refactor] Use PickValue where possible
fa818ca202 fuzz: [refactor] Use PickValue where possible (MarcoFalke)

Pull request description:

  `PickValue` is a bit less typing, so I think it should be used where possible

ACKs for top commit:
  practicalswift:
    cr ACK fa818ca202: patch looks correct and `PickValue` is better :)

Tree-SHA512: 49ed030694e3b7676654f1615f033287d26e2f0bc29647e1db56e0d84e14d29080f3e1898f5df8d644d834b8ded3ce713d2425ea86a37c9279d01f86ad03c202
2021-03-25 08:02:11 +01:00
Carl Dong
7b8e976cd5 miner: Add chainstate member to BlockAssembler 2021-03-24 15:40:56 -04:00
Carl Dong
eede0647b0 Revert "scripted-diff: Invoke CreateNewBlock with chainstate"
This reverts commit 46b7f29340.
2021-03-24 15:40:56 -04:00
fanquake
f95071a3f5
Merge #21489: fuzz: cleanups for versionbits fuzzer
aa7f418fe3 fuzz: cleanups for versionbits fuzzer (Anthony Towns)

Pull request description:

  Followups for #21380, shouldn't change coverage. Marking as draft to avoid introducing conflicts for the speedy trial PRs.

ACKs for top commit:
  MarcoFalke:
    cr ACK aa7f418fe3
  practicalswift:
    Tested ACK aa7f418fe3

Tree-SHA512: 6792364e3bb036cc903b4a5f5805d00afceeae475ce84660da962d28335bd98e59d5f45e68718657d3aa526123e351edadda39e99e49f1c6cfab629e98df35ed
2021-03-24 14:15:35 +08:00
MarcoFalke
fa818ca202
fuzz: [refactor] Use PickValue where possible 2021-03-24 06:57:55 +01:00
Michael Dietz
085b3a7299
rpc: deprecate addresses and reqSigs from rpc outputs
1) add a new sane "address" field (for outputs that have an
   identifiable address, which doesn't include bare multisig)
2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact
   (with all weird/wrong behavior they have now)
3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely,
   always.
2021-03-23 10:51:43 -04:00
MarcoFalke
fac921f23f
fuzz: Fix tx_pool target to properly fuzz immature outpoints 2021-03-23 10:58:36 +01:00
MarcoFalke
fa2b95f861
fuzz: Style fixups 2021-03-23 10:58:32 +01:00
MarcoFalke
fd2b22bf24
Merge #21142: fuzz: Add tx_pool fuzz target
faa9ef49d1 fuzz: Add tx_pool fuzz targets (MarcoFalke)

Pull request description:

ACKs for top commit:
  AnthonyRonning:
    reACK faa9ef49d1
  practicalswift:
    Tested ACK faa9ef49d1
  glozow:
    code review ACK faa9ef49d1, a bunch of comments but non blocking

Tree-SHA512: 8d404398faa46d8e7bf93060a2fe9afd5c0c2bd6e549ff6588d2f3dd1b912dff6c5416d5477c18edecc2e85b00db4fdf4790c3e6597a5149b0d40c9d5014d82f
2021-03-23 09:59:40 +01:00
MarcoFalke
d400e672a0
Merge #21487: fuzz: Use ConsumeWeakEnum in addrman for service flags
55554463c1 fuzz: Use ConsumeWeakEnum in addrman for service flags (MarcoFalke)

Pull request description:

  This has minimally better performance. Reported by me in https://github.com/bitcoin/bitcoin/pull/20228#discussion_r598081787

ACKs for top commit:
  practicalswift:
    Tested ACK 55554463c1
  vasild:
    ACK 55554463c1

Tree-SHA512: 4e5f51fe4f2bd7b2f37d0690e41203341ba45c0c9bc9247449cd26cfb5f77dc2ec61df3e4963276f68694e4b3ca3d0a76367a51c4d775501edeb3224d305a261
2021-03-23 09:43:15 +01:00
John Newbery
3fc06d3d7b [net] remove fUpdateConnectionTime from FinalizeNode
PeerManager can just call directly into CAddrMan::Connected() now.
2021-03-22 10:25:39 +00:00
MarcoFalke
1e4a3c057a
Merge #21317: util: Make Assume() usable as unary expression
fa4cebadcf util: Make Assume() usable as unary expression (MarcoFalke)

Pull request description:

  Assume shouldn't behave different at the call site depending on build flags. Currently compilation fails if it is used as expression. Fix that by using the lambda approach from `Assert()` without the `assert()`.

ACKs for top commit:
  jnewbery:
    ACK fa4cebadcf
  practicalswift:
    cr ACK fa4cebadcf: patch looks correct and commit hash starts with `fa`

Tree-SHA512: 9ec9ac8d410cdaf5e4e28df571a89e3d23d38e05a7027bb726cae3da6e9314734277e5a218e9e090cc17e10db763da71052c229ad642077ca5824ee42022f3ed
2021-03-22 08:35:21 +01:00
Jon Atack
7e3444805e
test: remove duplicate assertions in util_tests
Co-authored-by: Kiminuo <kiminuo@protonmail.com>
2021-03-21 11:14:35 +01:00
MarcoFalke
d2a78ee928
Merge #21488: test: add ParseUInt16() unit test and fuzz coverage
3d086f42ab test: add ParseUInt16() test coverage (Jon Atack)

Pull request description:

  `ParseUInt16()` was just added in #21328.

ACKs for top commit:
  practicalswift:
    cr ACK 3d086f42ab: patch looks correct & more coverage is better than less coverage

Tree-SHA512: bf7f96deb7c1531419565907f0ea8a8e32b368d4b823a3e80928b2c118edbf643ea06e357b4b5504a89f855caeed289daa9f823c740231ed6ad1b8ed00285ce8
2021-03-21 08:12:56 +01:00
MarcoFalke
9dbec05600
Merge #21349: build: Fix fuzz-cuckoocache cross-compiling with DEBUG=1
52a43b0c7d build: Fix fuzz-cuckoocache cross-compiling for Windows with DEBUG=1 (Hennadii Stepanov)

Pull request description:

  Fix #21348.

ACKs for top commit:
  practicalswift:
    Tested ACK 52a43b0c7d

Tree-SHA512: 6592f829edfb740a1e9d0691acf04b2372e91b0a53ca395b08350cb0b80031d3b55fa7331bdaddf857d450eb30b605af9fe8fe02559cda19374a48f9634fae70
2021-03-21 08:05:00 +01:00
Anthony Towns
aa7f418fe3 fuzz: cleanups for versionbits fuzzer 2021-03-21 11:21:41 +10:00
MarcoFalke
55554463c1
fuzz: Use ConsumeWeakEnum in addrman for service flags 2021-03-20 12:03:12 +01:00
John Newbery
7c4cc67c0c [net] remove CConnman::AddNewAddresses
It just forwards calls to CAddrMan::Add.
2021-03-20 10:24:40 +00:00
John Newbery
bcd7f30b79 [net] remove CConnman::MarkAddressGood
It just forwards calls to CAddrMan::Good.
2021-03-20 10:24:40 +00:00
John Newbery
8073673dbc [net] remove CConnman::SetServices
It just forwards calls to CAddrMan::SetServices.
2021-03-20 10:24:40 +00:00
John Newbery
392a95d393 [net_processing] Keep addrman reference in PeerManager 2021-03-20 10:24:40 +00:00
John Newbery
1c25adf6d2 [net] Construct addrman outside connman
node.context owns the CAddrMan. CConnman holds a reference to
the CAddrMan.
2021-03-20 10:24:36 +00:00
Jon Atack
3d086f42ab
test: add ParseUInt16() test coverage 2021-03-19 23:50:36 +01:00