Commit graph

361 commits

Author SHA1 Message Date
W. J. van der Laan
020c2b7609
Merge bitcoin/bitcoin#22923: test: Switch multiprocess to clang i686 build
fa309ee61c bench: Fix 32-bit compilation failure in addrman bench (MarcoFalke)
fae0295a79 ci: Switch multiprocess to i686 build (MarcoFalke)

Pull request description:

  Building for i686 with clang helps to catch bugs early for:
  * The OSS-Fuzz i686 clang libFuzzer build
  * The arm 32-bit native clang build

  Fixes  #22889

ACKs for top commit:
  hebasto:
    ACK fa309ee61c

Tree-SHA512: 581820d319aae2fcd4dd44979ee3d4164a575f0438476890aa2a7447f1392a5da26766cd6ab954530499b54f66eec2417bdeefdd7efb19bc27dd679cd2b9d0ce
2021-09-09 15:52:26 +02:00
MarcoFalke
fa309ee61c
bench: Fix 32-bit compilation failure in addrman bench 2021-09-09 08:30:53 +02:00
MarcoFalke
fa57fa1a2e
Enable clang-tidy bugprone-argument-comment and fix violations 2021-09-07 09:11:10 +02:00
MarcoFalke
a8fdfea77b
Merge bitcoin/bitcoin#22791: init: Fix asmap/addrman initialization order bug
724c497562 [fuzz] Add ConsumeAsmap() function (John Newbery)
5840476714 [addrman] Make m_asmap private (John Newbery)
f9002cb5db [net] Rename the copyStats arg from m_asmap to asmap (John Newbery)
f572f2b204 [addrman] Set m_asmap in CAddrMan initializer list (John Newbery)
593247872d [net] Remove CConnMan::SetAsmap() (John Newbery)
50fd77045e [init] Read/decode asmap before constructing addrman (John Newbery)

Pull request description:

  Commit 181a1207 introduced an initialization order bug: CAddrMan's m_asmap must be set before deserializing peers.dat.

  The first commit restores the correct initialization order. The remaining commits make `CAddrMan::m_asmap` usage safer:

  - don't reach into `CAddrMan`'s internal data from `CConnMan`
  - set `m_asmap` in the initializer list and make it const
  - make `m_asmap` private, and access it (as a reference to const) from a getter.

  This ensures that peers.dat deserialization must happen after setting m_asmap, since m_asmap is set during CAddrMan construction.

ACKs for top commit:
  mzumsande:
    Tested ACK 724c497562
  amitiuttarwar:
    code review but utACK 724c497562
  naumenkogs:
    utACK 724c497562
  vasild:
    ACK 724c497562
  MarcoFalke:
    review ACK 724c497562 👫

Tree-SHA512: 684a4cf9e3d4496c9997fb2bc4ec874809987055c157ec3fad1d2143b8223df52b5a0af787d028930b27388c8efeba0aeb2446cb35c337a5552ae76112ade726
2021-09-06 12:41:36 +02:00
Russell Yanofsky
b11a195ef4 refactor: Detach wallet transaction methods (followup for move-only)
Followup to commit "MOVEONLY: CWallet transaction code out of
wallet.cpp/.h" that detaches and renames some CWalletTx methods, making
into them into standalone functions or CWallet methods instead.

There are no changes in behavior and no code changes that aren't purely
mechanical. It just gives spend and receive functions more consistent
names and removes the circular dependencies added by the earlier
MOVEONLY commit.

There are also no comment or documentation changes. Removed comments
from transaction.h are just migrated to spend.h, receive.h, and
wallet.h.
2021-09-01 02:22:58 -05:00
John Newbery
f572f2b204 [addrman] Set m_asmap in CAddrMan initializer list
This allows us to make it const.
2021-08-27 10:55:41 +01:00
John Newbery
406be5ff96 [addrman] Remove all public uses of CAddrMan.Clear() from the tests
Just use unique_ptr<CAddrMan>s and reset the pointer if a frest addrman is required.
Also make CAddrMan::Clear() private to ensure that no call sites are missed.
2021-08-19 11:26:53 +01:00
John Newbery
a4d78546b0 [addrman] Make addrman consistency checks a runtime option
Currently addrman consistency checks are a compile time option, and are not
enabled in our CI. It's unlikely anyone is running these consistency checks.

Make them a runtime option instead, where users can enable addrman
consistency checks every n operations (similar to mempool tests). Update
the addrman unit tests to do internal consistency checks every 100
operations (checking on every operations causes the test runtime to
increase by several seconds).

Also assert on a failed addrman consistency check to terminate program
execution.
2021-08-12 10:41:11 +01:00
John Newbery
fa9710f62c [addrman] Add deterministic argument to CAddrMan ctor
Removes the need for tests to update nKey and insecure_rand after constructing
a CAddrMan.
2021-08-05 17:10:30 +01:00
MarcoFalke
9faa4b68db
Merge bitcoin/bitcoin#22232: refactor: Pass interpreter flags as uint32_t instead of signed int
fa621ededd refactor: Pass script verify flags as uint32_t (MarcoFalke)

Pull request description:

  The flags are cast to unsigned in the interpreter anyway, so avoid the confusion (and fuzz crashes) by just passing them as unsigned from the beginning.

  Also, the flags are often inverted bit-wise with the `~` operator, which also works on signed integers, but might cause confusion as the sign bit is flipped.

  Fixes #22233

ACKs for top commit:
  theStack:
    Concept and code review ACK fa621ededd
  kristapsk:
    ACK fa621ededd
  jonatack:
    ACK fa621ededd

Tree-SHA512: ea0720f32f823fa7f075309978672aa39773c6019d12b6c1c9d611fc1983a76115b7fe2a28d50814673bb6415c311ccc05b99d6e871575fb6900faf75ed17769
2021-07-20 15:36:23 +02:00
Jon Atack
e49d50cf40
bench: fix 32-bit narrowing warning in bench/peer_eviction.cpp 2021-07-15 23:05:10 +02:00
Jon Atack
5adb064574
bench: add peer eviction protection benchmarks
Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2021-07-08 12:28:23 +02:00
Jon Atack
84e2d5b781
bench: bench_bitcoin.cpp help fixups
- remove unneeded strprintf
- consistent punctuation (no EOL periods)
- sort helps by order they are printed (alphabetical order)
2021-06-24 11:13:12 +02:00
Jon Atack
10f4ce2078
bench: bench.h fixes and improvements 2021-06-24 11:13:10 +02:00
MarcoFalke
fa621ededd
refactor: Pass script verify flags as uint32_t
They are cast to unsigned anyway when calling VerifyScript,
bitcoinconsensus_verify_script*, or CountWitnessSigOps.
2021-06-14 08:02:45 +02:00
Carl Dong
6c3b5dc0c1 scripted-diff: tree-wide: Remove all review-only assertions
-BEGIN VERIFY SCRIPT-
find_regex='((assert|CHECK_NONFATAL)\(std::addressof|TODO: REVIEW-ONLY)' \
    && git grep -l -E "$find_regex" -- . \
        | xargs sed -i -E "/${find_regex}/d"
-END VERIFY SCRIPT-
2021-06-10 15:05:24 -04:00
Samuel Dobson
58f8b156ed
Merge bitcoin/bitcoin#22008: wallet: Cleanup and refactor CreateTransactionInternal
96c2c9520e scripted-diff: Rename SelectCoinsMinConf to AttemptSelection (Andrew Chow)
b583f73354 Move vin filling to before final fee setting (Andrew Chow)
d39cac0547 Set m_subtract_fee_outputs during recipients vector loop (Andrew Chow)
364e0698a5 Move variable initializations to where they are used (Andrew Chow)
32ab430651 Move recipients vector checks to beginning of CreateTransaction (Andrew Chow)
cd1d6d3324 Rename nSubtractFeeFromAmount in CreateTransaction (Andrew Chow)
dac21c793f Rename nValue and nValueToSelect (Andrew Chow)
d2aee3bbc7 Remove extraneous scope in CreateTransactionInternal (Andrew Chow)
b2995963b5 Move cs_wallet lock in CreateTransactionInternal to top of function (Andrew Chow)

Pull request description:

  #17331 did some refactors and cleanup of `CreateTransactionInternal` to make it easier to understand, however it is still a bit convoluted even though it doesn't have to be. This PR does additional cleanup and refactoring to `CreateTransactionInternal` so that it is easier to understand. Some unnecessary code was removed, some variables moved around to where they matter, and several indents removed.

ACKs for top commit:
  glozow:
    reACK 96c2c95
  ryanofsky:
    Code review ACK 96c2c9520e also acked previously (was reverted).
  meshcollider:
    re-utACK 96c2c9520e

Tree-SHA512: 3dba67ed436968a07bfd82d435d566ad74e116c6e50ac9baed7144a46ad5c0f630b1ba59d91e8e8972ac2af559d7c0576f0560f09684d2ab20fad6689902866f
2021-06-09 22:37:34 +12:00
MarcoFalke
2fccd9cf30
Merge bitcoin/bitcoin#22082: test: update nanobench from release 4.0.0 to 4.3.4
44d05d0a69 test: remove sanitizer suppression for nanobench (Martin Ankerl)
e3c866e3ca test: update nanobench from release 4.0.0 to 4.3.4 (Martin Ankerl)

Pull request description:

  This updates the third-party library nanobench with the latest release. It contains mostly minor bugfixes, a new pyperf output format, ability to suppress warnings with environment variable `NANOBENCH_SUPPRESS_WARNINGS`. Full changelog:

  v4.0.2
  * Changed `doNotOptimizeAway` to what google benchmark is doing. The old code did not work on some machines.
  * fix: display correct "total" value
  * minor Documentation updates

  v4.1.0
  * Updated link to new pyperf home
  * Adds ability to configure console output time unit
  *  Add support for environment variable `NANOBENCH_SUPPRESS_WARNINGS`
  * Nanobench is now usable with CMake's FetchContent (see documentation: https://nanobench.ankerl.com/tutorial.html#cmake-integration)

  v4.2.0
  * Ability to store and later compare results added, through `pyperf`.
  * See https://nanobench.ankerl.com/tutorial.html#pyperf-python-pyperf-module-output
  * Added lots of build targets to travis, similar to bitcoin's build.
  * Some minor API & documentation improvements

  v4.3.0
  * `ankerl::nanobench::Rng` can now return the state with `std::vector<uint64_t> Rng::state()`, and this can also be used to initialize the Rng.

  v4.3.1
  * Minor cmake improvements when integrationg as a third-party library: add alias `nanobench::nanobench`, default to C++17

  v4.3.2
  * Fixed a MSVC 2015 build problem
  * updates license to 2021.
  * build should now work with very old linux headers
  * Also disable UBSAN (bitcoin needed to add a suppression)

  v4.3.3
  * Do not use locale-dependent `std::to_string`

  v4.3.4
  * Add missing sanitizer suppression to `rotl`

ACKs for top commit:
  MarcoFalke:
    review ACK 44d05d0a69

Tree-SHA512: 3291c85057720cfc84a44bfaa305a7d0df4dc35779169d20de73d32e40d4cdbf3f005bf343f79710eca517441de2459e8118c195c5f5136f99d1f50ebd5dfd08
2021-06-02 09:20:16 +02:00
MarcoFalke
f63fc53c2a
Merge bitcoin/bitcoin#21767: [Bundle 6/n] Prune g_chainman usage in auxiliary modules
7a799c9c2b index: refactor-only: Reuse CChain ref (Carl Dong)
db33cde80f index: Add chainstate member to BaseIndex (Carl Dong)
f4a47a1feb bench: Use existing chainman in AssembleBlock (Carl Dong)
91226eb917 bench: Use existing NodeContext in DuplicateInputs (Carl Dong)
e6b4aa6eb5 miner: Pass in chainman to RegenerateCommitments (Carl Dong)
9ecade1425 rest: Add GetChainman function and use it (Carl Dong)
fc1c282845 rpc/blockchain: Use existing blockman in gettxoutsetinfo (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  The first 2 commits are fixups addressing review for the last bundle: #21391

  NEW note:
  1. I have opened #21766 which keeps track of potential improvements where the flaws already existed before the de-globalization work, please post on that issue about these improvements, thanks!

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

ACKs for top commit:
  jarolrod:
    ACK  7a799c9
  ariard:
    Code Review ACK 7a799c9
  fjahr:
    re-ACK 7a799c9c2b
  MarcoFalke:
    review ACK 7a799c9c2b 🌠
  ryanofsky:
    Code review ACK 7a799c9c2b. Basically no change since last review except fixed rebase conflicts and a new comment about REST Ensure()
  jamesob:
    conditional ACK 7a799c9c2b ([`jamesob/ackr/21767.1.dongcarl.bundle_6_n_prune_g_chai`](https://github.com/jamesob/bitcoin/tree/ackr/21767.1.dongcarl.bundle_6_n_prune_g_chai))

Tree-SHA512: 531c00ddcb318817457db2812d9a9d930bc664e58e6f7f1c746350732b031dd624270bfa6b9f49d8056aeb6321d973f0e38e4ff914acd6768edd8602c017d10e
2021-06-01 13:34:18 +02:00
Martin Ankerl
e3c866e3ca test: update nanobench from release 4.0.0 to 4.3.4
This updates the third-party library nanobench with the latest release. It contains mostly minor bugfixes, a new pyperf output format, ability to suppress warnings with environment variable `NANOBENCH_SUPPRESS_WARNINGS`. Full changelog:

v4.0.2
* Changed `doNotOptimizeAway` to what google benchmark is doing. The old code did not work on some machines.
* fix: display correct "total" value
* minor Documentation updates

v4.1.0
* Updated link to new pyperf home
* Adds ability to configure console output time unit
*  Add support for environment variable `NANOBENCH_SUPPRESS_WARNINGS`
* Nanobench is now usable with CMake's FetchContent (see documentation: https://nanobench.ankerl.com/tutorial.html#cmake-integration)

v4.2.0
* Ability to store and later compare results added, through `pyperf`.
* See https://nanobench.ankerl.com/tutorial.html#pyperf-python-pyperf-module-output
* Added lots of build targets to travis, similar to bitcoin's build.
* Some minor API & documentation improvements

v4.3.0
* `ankerl::nanobench::Rng` can now return the state with `std::vector<uint64_t> Rng::state()`, and this can also be used to initialize the Rng.

v4.3.1
* Minor cmake improvements when integrationg as a third-party library: add alias `nanobench::nanobench`, default to C++17

v4.3.2
* Fixed a MSVC 2015 build problem
* updates license to 2021.
* build should now work with very old linux headers
* Also disable UBSAN (bitcoin needed to add a suppression)

v4.3.3
* Do not use locale-dependent `std::to_string`

v4.3.4
* Add missing sanitizer suppression to `rotl`
2021-06-01 12:00:00 +02:00
Andrew Chow
96c2c9520e scripted-diff: Rename SelectCoinsMinConf to AttemptSelection
SelectCoinsMinConf is a bit of a misnomer now since it really just does
all of the coin selection given some parameters. So rename this to
something less annoying to say and makes a bit more sense.

-BEGIN VERIFY SCRIPT-
sed -i 's/SelectCoinsMinConf/AttemptSelection/g' $(git grep -l SelectCoinsMinConf ./src)
-END VERIFY SCRIPT-
2021-05-30 14:10:10 -04:00
Carl Dong
f4a47a1feb bench: Use existing chainman in AssembleBlock 2021-05-27 13:50:11 -04:00
Carl Dong
91226eb917 bench: Use existing NodeContext in DuplicateInputs 2021-05-27 13:50:11 -04:00
Samuel Dobson
6b254814c0
Merge bitcoin/bitcoin#17331: Use effective values throughout coin selection
51a3ac242c Have OutputGroup determine the value to use (Andrew Chow)
6d6d278475 Change SelectCoins_test to actually test SelectCoins (Andrew Chow)
9d3bd74ab4 Remove CreateTransaction while loop and some related variables (Andrew Chow)
6f0d5189af Remove use_bnb and bnb_used (Andrew Chow)
de26eb0e1f Do both BnB and Knapsack coin selection in SelectCoinsMinConf (Andrew Chow)
01dc8ebda5 Have KnapsackSolver actually use effective values (Andrew Chow)
bf26e018de Roll static tx fees into nValueToSelect instead of having it be separate (Andrew Chow)
cc3f14b27c Move output reductions for fee to after coin selection (Andrew Chow)
d97d25d950 Make cost_of_change part of CoinSelectionParams (Andrew Chow)
af5867c896 Move some calculations to common code in SelectCoinsMinConf (Andrew Chow)
1bf4a62cb6 scripted-diff: rename some variables (Andrew Chow)

Pull request description:

  Changes `KnapsackSolver` to use effective values instead of just the nominal txout value. Since fees are taken into account during the selection itself, we finally get rid of the `CreateTransaction` loop as well as a few other things that only were only necessary because of that loop.

  This should not change coin selection behavior at all (except maybe remove weird edge cases that were caused by the loop). In order to keep behavior the same, `KnapsackSolver` will select outputs with a negative effective value (as it did before).

ACKs for top commit:
  ryanofsky:
    Code review ACK 51a3ac242c. Looks good to go!
  instagibbs:
    review ACK 51a3ac242c
  meshcollider:
    re-light-utACK 51a3ac242c

Tree-SHA512: 372c27e00edcd5dbf85177421ba88f20bfdaf1791b6e3dc022c44876ecc379403e2375ed69e71c512c49e6af87641001ff385c4b25ab93684b3a08a53bf3824e
2021-05-26 01:35:43 +12:00
W. J. van der Laan
37e9f07996
Merge bitcoin/bitcoin#21843: p2p, rpc: enable GetAddr, GetAddresses, and getnodeaddresses by network
ce6bca88e8 doc: release note for getnodeaddresses by network (Jon Atack)
3f89c0e990 test: improve getnodeaddresses coverage, test by network (Jon Atack)
6c98c09991 rpc: enable filtering getnodeaddresses by network (Jon Atack)
80ba294854 p2p: allow CConnman::GetAddresses() by network, add doxygen (Jon Atack)
a49f3ddbba p2p: allow CAddrMan::GetAddr() by network, add doxygen (Jon Atack)
c38981e748 p2p: pull time call out of loop in CAddrMan::GetAddr_() (João Barbosa)
d35ddca91e p2p: enable CAddrMan::GetAddr_() by network, add doxygen (Jon Atack)

Pull request description:

  This patch allows passing a network argument to CAddrMan::GetAddr(), CConnman::GetAddresses(), and rpc getnodeaddresses to return only addresses of that network.

  It also contains a performance optimisation by promag.

ACKs for top commit:
  laanwj:
    Code review and lightly tested ACK ce6bca88e8
  vasild:
    ACK ce6bca88e8

Tree-SHA512: 40e700d97091248429c73cbc0639a1f03ab7288e636a7b9026ad253e9708253c6b2ec98e7d9fb2d56136c0f762313dd648915ac98d723ee330d713813a43f99d
2021-05-20 20:53:05 +02:00
Andrew Chow
6f0d5189af Remove use_bnb and bnb_used
These booleans are no longer needed
2021-05-19 14:37:17 -04:00
Andrew Chow
bf26e018de Roll static tx fees into nValueToSelect instead of having it be separate
The fees for transaction overhead and recipient outputs are now included
in nTargetValue instead of being a separate parameter. For the coin
selection algorithms, it doesn't matter that these are separate as in
either case, the algorithm needs to select enough to cover these fees.

Note that setting nValueToSelect is changed as it now includes
not_input_fees. Without the change to how nValueToSelect is increased
for KnapsackSolver, this would result in overpaying fees. The change to
increase by the difference between nFeeRet and not_input_fees allows
this to have the same behavior as previously.

Additionally, because we assume that KnapsackSolver will always find a
solution that requires change (we assume that BnB always finds a
non-change solution), we also include the fee for the change output in
KnapsackSolver's target. As part of this, we also use the changeless
nFeeRet when iterating for KnapsackSolver. This is because we include
the change fee when doing KnapsackSolver, so nFeeRet on further
iterations won't include the change fee.
2021-05-19 13:33:31 -04:00
Jon Atack
a49f3ddbba
p2p: allow CAddrMan::GetAddr() by network, add doxygen 2021-05-19 13:04:11 +02:00
Ivan Metlushko
e2a47ce085 refactor: move first run detection to client code 2021-05-19 08:50:16 +02:00
MarcoFalke
fa8a88849c
bench: Remove duplicate constants 2021-05-03 11:50:30 +02:00
fanquake
bd65a76b9d
Merge #21330: Deal with missing data in signature hashes more consistently
725d7ae049 Use PrecomputedTransactionData in signet check (Pieter Wuille)
497718b467 Treat amount<0 also as missing data for P2WPKH/P2WSH (Pieter Wuille)
3820090bd6 Make all SignatureChecker explicit about missing data (Pieter Wuille)
b77b0cc507 Add MissingDataBehavior and make TransactionSignatureChecker handle it (Pieter Wuille)

Pull request description:

  Currently we have 2 levels of potentially-missing data in the transaction signature hashes:
  * P2WPKH/P2WSH hashes need the spent amount
  * P2TR hashes need all spent outputs (amount + scriptPubKey)

  Missing amounts are treated as -1 (thus leading to unexpected signature failures), while missing outputs in P2TR validation cause assertion failure. This is hard to extend for signing support, and also quite ugly in general.

  In this PR, an explicit configuration option to {Mutable,}TransactionSignatureChecker is added (MissingDataBehavior enum class) to either select ASSERT_FAIL or FAIL. Validation code passes ASSERT_FAIL (as at validation time all data should always be passed, and anything else is a serious bug in the code), while signing code uses FAIL.

  The existence of the ASSERT_FAIL option is really just an abundance of caution. Always using FAIL should be just fine, but if there were for some reason a code path in consensus code was introduced that misses certain data, I think we prefer as assertion failure over silently introducing a consensus change.

  Potentially useful follow-ups (not for this PR, in my preference):
  * Having an explicit script validation error code for missing data.
  * Having a MissingDataBehavior::SUCCEED option as well, for use in script/sign.cpp DataFromTransaction (if a signature is present in a witness, and we don't have enough data to fully validate it, we should probably treat it as valid and not touch it).

ACKs for top commit:
  sanket1729:
    reACK 725d7ae049
  Sjors:
    ACK 725d7ae049
  achow101:
    re-ACK 725d7ae049
  benthecarman:
    ACK 725d7ae049
  fjahr:
    Code review ACK 725d7ae049

Tree-SHA512: d67dc51bae9ca7ef6eb9acccefd682529f397830f77d74cd305500a081ef55aede0e9fa380648c3a8dd4857aa7eeb1ab54fe808979d79db0784ac94ceb31b657
2021-04-13 10:24:31 +08:00
Pieter Wuille
3820090bd6 Make all SignatureChecker explicit about missing data
Remove the implicit MissingDataBehavior::ASSERT_FAIL in the
*TransationSignatureChecker constructors, and instead specify
it explicit in all call sites:
* Test code uses ASSERT_FAIL
* Validation uses ASSERT_FAIL (through CachingTransactionSignatureChecker)
  (including signet)
* libconsensus uses FAIL, matching the existing behavior of the
  non-amount API (and the extended required data for taproot validation
  is not available yet)
* Signing code uses FAIL
2021-03-15 17:29:39 -07:00
Wladimir J. van der Laan
8ec881d3b6
Merge #20861: BIP 350: Implement Bech32m and use it for v1+ segwit addresses
03346022d6 naming nits (Fabian Jahr)
2e7c80fb5b Add signet support to gen_key_io_test_vectors.py (Pieter Wuille)
fe5e495c31 Use Bech32m encoding for v1+ segwit addresses (Pieter Wuille)
25b1c6e13d Add Bech32m test vectors (Pieter Wuille)
da2bb6976d Implement Bech32m encoding/decoding (Pieter Wuille)

Pull request description:

  This implements [BIP 350](https://github.com/bitcoin/bips/blob/master/bip-0350.mediawiki):
  * For segwit v1+ addresses, a new checksum algorithm called Bech32m is used.
  * Segwit v0 address keep using Bech32 as specified in [BIP 173](https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki).

ACKs for top commit:
  Sjors:
    utACK 0334602
  jnewbery:
    utACK 03346022d6
  achow101:
    ACK 0334602
  fjahr:
    re-ACK 0334602
  benthecarman:
    ACK 03346022d6

Tree-SHA512: 4424cfd44869d813d6152fb3ed867b204036736bc2344a039b93700b6f36a43e9110478f138eb81c97c77ab27ecb776dada5ba632cb5a3a9d244924d2540a557
2021-03-18 20:37:21 +01:00
Wladimir J. van der Laan
a9d1b40d53
Merge #21415: refactor: remove Optional & nullopt
ebc4ab721b refactor: post Optional<> removal cleanups (fanquake)
57e980d13c scripted-diff: remove Optional & nullopt (fanquake)

Pull request description:

  Same rationale & motivation as #21404, which turned out to be quite low in the number of potential conflicts. Lets see what the bot has to say here.

ACKs for top commit:
  practicalswift:
    cr ACK ebc4ab721b: patch looks correct
  jnewbery:
    utACK ebc4ab721b
  laanwj:
    Code review ACK ebc4ab721b

Tree-SHA512: 550fbeef09b9d35ddefaa805d1755c18c8fd499c4b0f77ebfece8c20296a7abd1cf6c699e2261f92fe3552deeb7555ec2a2287ffe3ab9e98bb9f8612a4d43be3
2021-03-17 12:17:33 +01:00
fanquake
ebc4ab721b
refactor: post Optional<> removal cleanups 2021-03-17 14:56:20 +08:00
Samuel Dobson
d25e28c20b
Merge #21083: wallet: Avoid requesting fee rates multiple times during coin selection
f9cd2bfbcc Rename CoinSelectionParams::effective_fee to m_effective_feerate (Andrew Chow)
bdd0c2934b wallet: Move discard feerate fetching to CreateTransaction (Andrew Chow)
448d04b931 wallet: Move long term feerate setting to CreateTransaction (Andrew Chow)
e2f429e6bb wallet: Replace nFeeRateNeeded with effective_fee (Andrew Chow)
1a6a0b0dfb wallet: Use existing feerate instead of getting a new one (Andrew Chow)

Pull request description:

  During coin selection, there are various places where we need to have a feerate. We need the feerate for the transaction itself, the discard fee rate, and long term feerate. Fetching these each time we need them can lead to a race condition where two feerates that should be the same are actually different. One particular instance where this can happen is during the loop in `CreateTransactionInternal`. After inputs are chosen, the expected transaction fee is calculated using a newly fetched feerate. If `pick_new_inputs == false`, the loop will go again with the assumption that the fee for the transaction remains the same. However because the feerate is fetched again, it is possible that it actually isn't and this causes coin selection to fail.

  Instead of fetching the feerate each time it is needed, we fetch them all at once at the top of `CreateTransactionInternal`, store them in `CoinSelectionParams`, and use them where needed.

  While some of these fee rates probably don't need this caching, I've done it for consistency and the guarantee that they remain the same.

  Fixes #19229

ACKs for top commit:
  glozow:
    reACK f9cd2bfbcc
  fjahr:
    Code review re-ACK f9cd2bfbcc
  Xekyo:
    tACK f9cd2bfbcc
  meshcollider:
    Code review + test run ACK f9cd2bfbcc

Tree-SHA512: be83ff64ba473c3cdd3469c812e214659b6e2a9584c22ed2b1595618fce0d4b35d0901e61068cd1069fc1a8fb911db01dd7312d05c3b8cbafbe2504ab7a3e863
2021-03-17 13:14:48 +13:00
Andrew Chow
f9cd2bfbcc Rename CoinSelectionParams::effective_fee to m_effective_feerate
It's a feerate, not a fee. Also follow the style guide for member names.
2021-03-16 17:16:57 -04:00
Andrew Chow
bdd0c2934b wallet: Move discard feerate fetching to CreateTransaction
Instead of fetching the discard feerate for each SelectCoinsMinConf
iteration, fetch and cache it once during CreateTransaction so that it
is shared for each SelectCoinsMinConf through
coin_selection_params.m_discard_feerate.

Does not change behavior.
2021-03-16 16:33:27 -04:00
Andrew Chow
448d04b931 wallet: Move long term feerate setting to CreateTransaction
Instead of setting the long term feerate for each SelectCoinsMinConf
iteration, set it once during CreateTransaction and let it be shared
with each SelectCoinsMinConf through
coin_selection_params.m_long_term_feerate.

Does not change behavior.
2021-03-16 16:32:38 -04:00
Pieter Wuille
da2bb6976d Implement Bech32m encoding/decoding 2021-03-15 17:26:35 -07:00
fanquake
57e980d13c
scripted-diff: remove Optional & nullopt
-BEGIN VERIFY SCRIPT-
git rm src/optional.h

sed -i -e 's/Optional</std::optional</g' $(git grep -l 'Optional<' src)

sed -i -e 's/{nullopt}/{std::nullopt}/g' $(git grep -l 'nullopt' src)
sed -i -e 's/ nullopt;/ std::nullopt;/g' $(git grep -l 'nullopt' src)
sed -i -e 's/ nullopt)/ std::nullopt)/g' $(git grep -l 'nullopt' src)
sed -i -e 's/(nullopt)/(std::nullopt)/g' $(git grep -l 'nullopt' src)
sed -i -e 's/ nullopt,/ std::nullopt,/g' $(git grep -l 'nullopt' src)
sed -i -e 's/? nullopt :/? std::nullopt :/g' $(git grep -l 'nullopt' src)
sed -i -e 's/: nullopt}/: std::nullopt}/g' $(git grep -l 'nullopt' src)

sed -i -e '/optional.h \\/d' src/Makefile.am

sed -i -e '/#include <optional.h>/d' src/test/fuzz/autofile.cpp src/test/fuzz/buffered_file.cpp src/test/fuzz/node_eviction.cpp

sed -i -e 's/#include <optional.h>/#include <optional>/g' $(git grep -l '#include <optional.h>' src)
-END VERIFY SCRIPT-
2021-03-15 10:41:30 +08:00
fanquake
3ba2840e7e
scripted-diff: remove MakeUnique<T>()
-BEGIN VERIFY SCRIPT-
git rm src/util/memory.h
sed -i -e 's/MakeUnique/std::make_unique/g' $(git grep -l MakeUnique src)
sed -i -e '/#include <util\/memory.h>/d' $(git grep -l '#include <util/memory.h>' src)
sed -i -e '/util\/memory.h \\/d' src/Makefile.am
-END VERIFY SCRIPT-
2021-03-11 13:45:14 +08:00
MarcoFalke
fa576b4532
Move MakeNoLogFileContext to common libtest_util, and use it in bench
Can be reviewed with --color-moved=dimmed-zebra
2021-03-03 09:17:37 +01:00
Wladimir J. van der Laan
05e821ee19
Merge #21170: bench: Add benchmark to write JSON into a string
e3e0a2432c Add benchmark to write JSON into a string (Martin Ankerl)

Pull request description:

  The benchmark `BlockToJsonVerbose` only tests generating (and destroying)
  the JSON data structure, but serializing into a string is also a
  performance critical aspect of the RPC calls.

  Extracts test setup into a `struct TestBlockAndIndex`, and uses it in
  both `BlockToJsonVerbose` and `BlockToJsonVerboseWrite`.

  Also, use `ankerl::nanobench::doNotOptimizeAway` to make sure the compiler
  can't optimize the result of the calls away.

  Here are benchmark results on my Intel i7-8700:

  |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |         bra/op |   miss% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
  |       71,807,017.00 |               13.93 |    0.4% |  555,782,961.00 |  220,788,645.00 |  2.517 | 102,279,341.00 |    0.4% |      0.80 | `BlockToJsonVerbose`
  |       27,916,835.00 |               35.82 |    0.1% |  235,084,034.00 |   89,033,525.00 |  2.640 |  42,911,139.00 |    0.3% |      0.32 | `BlockToJsonVerboseWrite`

ACKs for top commit:
  laanwj:
    Code review ACK e3e0a2432c

Tree-SHA512: bc4d6d1588d47d4bd7af8e7908e44b8561bc391a2d73eccd7c0aa37fc40e8a9ce1fa1f3c29b416eef24a73c6bce3036839c0bbfe1b8dbd6d1bba3718b7ca5383
2021-03-01 19:12:09 +01:00
Carl Dong
3704433c4f scripted-diff: Invoke ::AcceptToMemoryPool with chainstate
-BEGIN VERIFY SCRIPT-
find_regex='\bAcceptToMemoryPool\(' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E 's@'"$find_regex"'@\0::ChainstateActive(), @g'
-END VERIFY SCRIPT-
2021-02-18 14:49:06 -05:00
MarcoFalke
cd66d8b1d8
Merge #20429: refactor: replace (sizeof(a)/sizeof(a[0])) with C++17 std::size
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17) (Sebastian Falbesoner)
365539c846 refactor: init vectors via std::{begin,end} to avoid pointer arithmetic (Sebastian Falbesoner)
63d4ee1968 refactor: iterate arrays via C++11 range-based for loops if idx is not needed (Sebastian Falbesoner)

Pull request description:

  This refactoring PR picks up the idea of #19626 and replaces all occurences of `sizeof(x)/sizeof(x[0])` (or `sizeof(x)/sizeof(*x)`, respectively) with the now-available C++17 [`std::size`](https://en.cppreference.com/w/cpp/iterator/size)  (as [suggested by sipa](https://github.com/bitcoin/bitcoin/pull/19626#issuecomment-666487228)), making the macro `ARRAYLEN` obsolete.

  As preparation for this, two other changes are done to eliminate `sizeof(x)/sizeof(x[0])` usage:
  * all places where arrays are iterated via an index are changed to use C++11 range-based for loops If the index' only purpose is to access the array element (as [suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/19626#discussion_r463404541)).
  * `std::vector` initializations are done via `std::begin` and `std::end` rather than using pointer arithmetic to calculate the end (also [suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/20429#discussion_r567418821)).

ACKs for top commit:
  practicalswift:
    cr ACK e829c9afbf: patch looks correct
  fanquake:
    ACK e829c9afbf
  MarcoFalke:
    review ACK e829c9afbf 🌩

Tree-SHA512: b01d32c04b9e04d562b7717cae00a651ec9a718645047a90761be6959e0cc2adbd67494e058fe894641076711bb09c3b47a047d0275c736f0b2218e1ce0d193d
2021-02-18 07:53:37 +01:00
Martin Ankerl
e3e0a2432c Add benchmark to write JSON into a string
The benchmark BlockToJsonVerbose only tests generating (and destroying)
the JSON data structure, but serializing into a string is also a
performance critical aspect of the RPC calls.

Also, use ankerl::nanobench::doNotOptimizeAway to make sure the compiler
can't optimize the result of the calls away.
2021-02-13 13:14:16 +01:00
MarcoFalke
8e1913ae02
Merge #21062: refactor: return MempoolAcceptResult from ATMP
53e716ea11 [refactor] improve style for touched code (gzhao408)
174cb5330a [refactor] const ATMPArgs and non-const Workspace (gzhao408)
f82baf0762 [refactor] return MempoolAcceptResult (gzhao408)
9db10a5506 [refactor] clean up logic in testmempoolaccept (gzhao408)

Pull request description:

  This is the first 4 commits of #20833, and does refactoring only. It should be relatively simple to review, and offers a few nice things:
  - It makes accessing values that don't make sense (e.g. fee) when the tx is invalid an error.
  - Returning `MempoolAcceptResult` from ATMP makes the interface cleaner. The caller can get a const instead of passing in a mutable "out" param.
  - We don't have to be iterating through a bunch of lists for package validation, we can just return a `std::vector<MempoolAcceptResult>`.
  - We don't have to refactor all ATMP call sites again if/when we want to return more stuff from it.

ACKs for top commit:
  MarcoFalke:
    ACK 53e716ea11 💿
  jnewbery:
    Code review ACK 53e716ea11
  ariard:
    Code Review ACK 53e716e, I did tweak a bit the touched paths to see if we had good test coverage. Didn't find holes.

Tree-SHA512: fa6ec324a08ad9e6e55948615cda324cba176255708bf0a0a0f37cedb7a75311aa334ac6f223be7d8df3c7379502b1081102b9589f9a9afa1713ad3d9ab3c24f
2021-02-11 14:45:41 +01:00
gzhao408
f82baf0762 [refactor] return MempoolAcceptResult
This creates a cleaner interface with ATMP, allows us to make results const,
and makes accessing values that don't make sense (e.g. fee when tx is
invalid) an error.
2021-02-09 07:01:52 -08:00
Wladimir J. van der Laan
1e69800d5e
Merge #21059: Drop boost/preprocessor dependencies
e99db77a6e Drop boost/preprocessor dependencies (Hennadii Stepanov)
12f5028d49 refactor: Move STRINGIZE macro to macros.h (Hennadii Stepanov)

Pull request description:

  Use own macros instead of boost's ones.

ACKs for top commit:
  laanwj:
    Code review ACK e99db77a6e
  practicalswift:
    cr ACK e99db77a6e

Tree-SHA512: 7ec15c2780a661e293c990f64c41b5b451d894cc191aa7872fbcaf96da91915a351209b1f1003ab12a7a16cb464e50ac58a028db02beeedfa5f6931752c2d9e2
2021-02-02 12:44:27 +01:00