Commit graph

20644 commits

Author SHA1 Message Date
Andrew Chow
d3dbb16168 Separate individual HD Keypath serialization into separate functions 2021-12-10 08:29:47 -05:00
Andrew Chow
a69332fd89 Store version bytes and be able to serialize them in CExtPubKey
CExtPubKey does not store the version bytes for the extended public key.
We store these so that a CExtPubKey can be serialized and deserialized with
the same version bytes.
2021-12-10 08:29:47 -05:00
Andrew Chow
5fdaf6a2ad moveonly: Move (Un)Serialize(To/From)Vector, (De)SerializeHDKeypaths to psbt module
SerializeToVector, UnserializeFromVector, DeserializeHDKeypaths, and SerializeHDKeypaths
were in sign.h where PSBT was originally implemented. Since all of the PSBT serialization
has moved to its own file, these functions should follow.
2021-12-10 08:29:47 -05:00
Andrew Chow
a4cf810174 Output proprietary type info in decodepsbt 2021-12-10 08:29:47 -05:00
Andrew Chow
aebe758e54 Implement PSBT proprietary type 2021-12-10 08:29:46 -05:00
Andrew Chow
10ba0b593d Output psbt version in decodepsbt 2021-12-10 08:29:26 -05:00
Andrew Chow
df84fa99c5 Add GetVersion helper to PSBT 2021-12-10 08:29:26 -05:00
Andrew Chow
c3eb416b88 Implement PSBT versions 2021-12-10 08:29:26 -05:00
Andrew Chow
3235847473 Types are compact size uints 2021-12-10 08:29:24 -05:00
MarcoFalke
fa72dd314f
fuzz: Move ISO8601 to one place 2021-12-10 13:58:55 +01:00
MarcoFalke
011d6e429b
Merge bitcoin/bitcoin#22514: psbt: Actually use SIGHASH_DEFAULT for PSBT signing
c0405ee27f rpc: Document that DEFAULT is for Taproot, ALL for everything else (Andrew Chow)
d3992669df psbt: Actually use SIGHASH_DEFAULT (Andrew Chow)
eb9a1a2c59 psbt: Make sighash_type std::optional<int> (Andrew Chow)

Pull request description:

  Make the behavior align with the help text by actually using SIGHASH_DEFAULT as the default sighash for signing PSBTs.

ACKs for top commit:
  Sjors:
    re-utACK c0405ee27f

Tree-SHA512: 5199fb41de416b2f10ac451f824e7c94b428ba11fdb9e50f0027c692e959ce5813a340c34a4e52d7aa128e12008303d80939a693eff36a869720e45442119828
2021-12-10 10:17:36 +01:00
MarcoFalke
9f7661c0c4
Merge bitcoin/bitcoin#19499: p2p: Make timeout mockable and type safe, speed up test
fadc0c80ae p2p: Make timeout mockable and type safe, speed up test (MarcoFalke)
fa6d5a238d scripted-diff: Rename m_last_send and m_last_recv (MarcoFalke)

Pull request description:

  Use type-safe time for better code readability/maintainability and mockable time for better testability. This speeds up the p2p_timeout test.

  This is also a bugfix for intermittent test issues like: https://cirrus-ci.com/task/4769904156999680?command=ci#L2836

  Fixes #20654

ACKs for top commit:
  laanwj:
    Code review ACK fadc0c80ae
  naumenkogs:
    ACK fadc0c80ae

Tree-SHA512: 28c6544c97f188c8a0fbc80411c74ab74ffd055885322c325aa3d1c404b29c3fd70a737e86083eecae58ef394db1cb56bc122d06cff63742aa89a8e868730c64
2021-12-10 10:02:12 +01:00
fanquake
09ad512369
Merge bitcoin/bitcoin#23628: Check descriptors returned by external signers
5493e92501 Check descriptors returned by external signers (sstone)

Pull request description:

  Check that descriptors returned by external signers have been parsed properly when creating a new wallet.
  See https://github.com/bitcoin/bitcoin/issues/23627 for context.

  The problem is that parsing an invalid descriptor will return `null` which is not checked for in `CWallet::SetupDescriptorScriptPubKeyMans()`.

  I'm not completely sure what the best fix is since there several strategies for dealing with errors in the current codebase but the proposed fix is very simple and consistent with other validation checks in `CWallet::SetupDescriptorScriptPubKeyMans()`.

ACKs for top commit:
  jamesob:
    Code review ACK 5493e92501
  achow101:
    ACK 5493e92501

Tree-SHA512: 63259f4aa519405a86c554b6813efdb741314bdaa18bf005b70ea8bb92a27abc6e2b65f7c584641dc257fc78a6499f42b51b5310c243e611c4663430dccf3d04
2021-12-10 09:17:35 +08:00
W. J. van der Laan
c840ab0231
Merge bitcoin/bitcoin#22019: wallet: Introduce SelectionResult for encapsulating a coin selection solution
05300c1439 Use SelectionResult in SelectCoins (Andrew Chow)
9d9b101d20 Use SelectionResult in AttemptSelection (Andrew Chow)
bb50850a44 Use SelectionResult for waste calculation (Andrew Chow)
e8f7ae5eb3 Make an OutputGroup for preset inputs (Andrew Chow)
51a9c00b4d Return SelectionResult from SelectCoinsSRD (Andrew Chow)
0ef6184575 Return SelectionResult from KnapsackSolver (Andrew Chow)
60d2ca72e3 Return SelectionResult from SelectCoinsBnB (Andrew Chow)
a339add471 Make member variables of SelectionResult private (Andrew Chow)
cbf0b9f4ff scripted-diff: Use SelectionResult in coin selector tests (Andrew Chow)
9d1d86da04 Introduce SelectionResult struct (Andrew Chow)
94d851d28c Fix bnb_search_test to use set equivalence for (Andrew Chow)

Pull request description:

  Instead of returning a set of selected coins and their total value as separate items, encapsulate both of these, and other variables, into a new `SelectionResult` struct. This allows us to have all of the things relevant to a coin selection solution be in a single object. `SelectionResult` enables us to implement the waste calculation in a cleaner way.

  All of the coin selection functions (`SelectCoinsBnB`, `KnapsackSolver`, `AttemptSelection`, and `SelectCoins`) are changed to use a `SelectionResult` as the output parameter.

  Based on #22009

ACKs for top commit:
  laanwj:
    Code review ACK 05300c1439

Tree-SHA512: e4dbb4d78a6cda9c237d230b19e7265591efac5a101a64e6970f0654e2c4f93d13bb5d07b98e8c7b8d37321753dbfc94c28c3a7810cb1c59b5bc29b08a8493ef
2021-12-09 17:21:46 +01:00
MarcoFalke
7ce8d74156
Merge bitcoin/bitcoin#23346: util, refactor: Improve headers for bitcoin-wallet tool
3431839c33 util, refactor: Improve headers for bitcoin-wallet tool (Hennadii Stepanov)

Pull request description:

  This PR:
  - removes unneeded `#include <wallet/wallet.h>` from `<wallet/wallettool.h>`
  - introduces class forward declaration in `<wallet/wallettool.h>`
  - added `#include <config/bitcoin-config.h>` to `wallet/wallettool.cpp` where the `USE_BDB` macro is used

Top commit has no ACKs.

Tree-SHA512: a0de560d821f8b570ae806a1165b9b382c9e0b339687d932052fa4c38ab2ba493e7e050f19adc02ad7db40c42cf88ac1d37209f9071494a0ab268ed33ff22b9f
2021-12-09 13:44:24 +01:00
sstone
5493e92501 Check descriptors returned by external signers
Check that descriptors returned by external signers have been parsed properly when creating a new wallet.
2021-12-09 11:17:04 +01:00
fanquake
7908772244
Merge bitcoin/bitcoin#23703: scripted-diff: Use named args in RPC docs
fa9aaf8694 scripted-diff: Use named args in RPC docs (MarcoFalke)

Pull request description:

  Incorrect named args are source of bugs, like #22979.

  To allow them being checked by `clang-tidy`, use a format it can understand.

ACKs for top commit:
  fanquake:
    ACK fa9aaf8694 - checked `clang-tidy` and it's fine here, (but throwing errors in other files. i.e `wallet/test/wallet_tests.cpp`).

Tree-SHA512: e09dae8ee999a5c4819e6f848c12139593ca0e915e645c8fabeb97c379188fb9104d286c02c71f590abc64cdec125f78026735f83e016111976baa49d588a9bc
2021-12-09 17:14:26 +08:00
MarcoFalke
faa0833c43
doc: Normalize RPC description whitespace 2021-12-08 19:43:24 +01:00
Andrew Chow
c0405ee27f rpc: Document that DEFAULT is for Taproot, ALL for everything else 2021-12-08 09:43:30 -05:00
Andrew Chow
d3992669df psbt: Actually use SIGHASH_DEFAULT
Make the behavior align with the help text by actually using
SIGHASH_DEFAULT as the default sighash for signing PSBTs.
2021-12-08 09:43:30 -05:00
Andrew Chow
eb9a1a2c59 psbt: Make sighash_type std::optional<int>
It is better to ues an optional to determine whether the sighash type
is set rather than using 0 as a magic number.
2021-12-08 09:43:26 -05:00
MarcoFalke
fa77f95c2f
fuzz: Fix RPC internal bug detection 2021-12-08 14:20:16 +01:00
MarcoFalke
577bd51a4b
Merge bitcoin/bitcoin#23702: doc: Add missing optional to getblockfrompeer
aaaa34e34d doc: Add missing optional to getblockfrompeer (MarcoFalke)

Pull request description:

  Can be reviewed with `--word-diff-regex=. --ignore-all-space`

ACKs for top commit:
  Sjors:
    utACK aaaa34e34d

Tree-SHA512: 7f46c82a46b8cc19f7eb549b9aa13be8cd6849a8ef8a2ddda6d1eee6978d099fccadd29a2bf817f44d601b905f5d5f6b5d8f4f54be5ee8b914b520359c058e68
2021-12-08 13:22:31 +01:00
MarcoFalke
fa9aaf8694
scripted-diff: Use named args in RPC docs
-BEGIN VERIFY SCRIPT-
 sed -i -e 's|, /\* optional \*/ true,|, /*optional=*/true,|g' $( git grep -l ', /\* optional \*/ true,' )
-END VERIFY SCRIPT-
2021-12-08 11:54:12 +01:00
MarcoFalke
aaaa34e34d
doc: Add missing optional to getblockfrompeer 2021-12-08 11:39:31 +01:00
Samuel Dobson
b692e61d61
Merge bitcoin/bitcoin#23254: doc: Fix typo and grammar
ffd11ea876 Fix typo and grammar (Heebs)

Pull request description:

  Fix typo and grammar in the coin selection algorithm's description.

ACKs for top commit:
  meshcollider:
    ACK ffd11ea876

Tree-SHA512: bba07c2efd5140fb3e021618739d70aaa761bbc274afb8158809492b0606773c217e42e58e58b18a2454b9c45ebc883ebece17cdc467ac60e3d3140d7a979db7
2021-12-08 22:43:56 +13:00
MarcoFalke
f6013265b7
Merge bitcoin/bitcoin#20295: rpc: getblockfrompeer
dce8c4c381 rpc: getblockfrompeer (Sjors Provoost)
b884ababc2 rpc: move Ensure* helpers to server_util.h (Sjors Provoost)

Pull request description:

  This adds an RPC method to fetch a block directly from a peer. This can used to fetch stale blocks with lower proof of work that are normally ignored by the node (`headers-only` in `getchaintips`).

  Usage:
  ```
  bitcoin-cli getblockfrompeer HASH peer_n
  ```

  Closes #20155

  Limitations:
  * you have to specify which peer to fetch the block from
  * the node must already have the header

ACKs for top commit:
  jnewbery:
    ACK dce8c4c381
  fjahr:
     re-ACK dce8c4c381

Tree-SHA512: 843ba2b7a308f640770d624d0aa3265fdc5c6ea48e8db32269b96a082b7420f7953d1d8d1ef2e6529392c7172dded9d15639fbc9c24e7bfa5cfb79e13a5498c8
2021-12-08 10:39:37 +01:00
MarcoFalke
84d921e79c
Merge bitcoin/bitcoin#23465: Remove CTxMemPool params from ATMP
f1f10c0514 Remove CTxMemPool params from ATMP (lsilva01)

Pull request description:

  Remove `CTxMemPool` parameter from `AcceptToMemoryPool` function, as suggested in https://github.com/bitcoin/bitcoin/pull/23437#issuecomment-962536149 .

  This requires that `CChainState` has access to `MockedTxPool` in  `tx_pool.cpp` as mentioned https://github.com/bitcoin/bitcoin/pull/23173#discussion_r731895386. So the `MockedTxPool` is attributed to `CChainState::m_mempool` before calling `AcceptToMemoryPool`.

  Requires #23437.

ACKs for top commit:
  jnewbery:
    utACK f1f10c0514
  MarcoFalke:
    review ACK f1f10c0514 🔙

Tree-SHA512: 2a4885f4645014fc1fa98bb1090f13721c1a0796bc0021b9cb43bc8cc13920b6eaf057d1f5ed796e0a110e7813e41fe0196334ce7c80d1231fc057a9a3bdf349
2021-12-08 10:00:55 +01:00
Samuel Dobson
e46fc935aa Add warnings field to addmultisigaddress to warn about uncompressed keys 2021-12-08 17:14:40 +13:00
Samuel Dobson
d1a9742623 Add warnings field to createmultisig to warn about uncompressed keys 2021-12-08 17:11:46 +13:00
Samuel Dobson
b36e738285 MOVEONLY: Move abortrescan from backup.cpp to transactions.cpp 2021-12-08 11:54:08 +13:00
Samuel Dobson
d794d0da8f Remove unused imports from rpc/wallet and reorder RPCs 2021-12-08 11:45:21 +13:00
Samuel Dobson
e116b9747d MOVEONLY: Move rpcwallet to rpc/wallet 2021-12-08 11:45:21 +13:00
Samuel Dobson
8e30875fde MOVEONLY: Move spending RPCs to spend.cpp 2021-12-08 11:45:21 +13:00
Samuel Dobson
9ce521a61b MOVEONLY: Move balance and utxo RPCs to coins.cpp 2021-12-08 11:45:19 +13:00
Samuel Dobson
7b45f5c059 MOVEONLY: Move address related functions from rpcwallet to addresses.cpp 2021-12-08 11:42:57 +13:00
Samuel Dobson
f7646b407f MOVEONLY: Move transaction related wallet RPCs to transactions.cpp 2021-12-08 11:40:59 +13:00
lsilva01
f1f10c0514 Remove CTxMemPool params from ATMP
Co-authored-by: John Newbery <1063656+jnewbery@users.noreply.github.com>
Co-authored-by: Jon Atack <jon@atack.com>
2021-12-07 18:56:29 -03:00
MarcoFalke
63c63b5533
Merge bitcoin/bitcoin#14707: [RPC] Include coinbase transactions in receivedby RPCs
1dcba996d3 Coinbase receivedby rpcs release notes (Andrew Toth)
b5696750a9 Test including coinbase transactions in receivedby wallet rpcs (Andrew Toth)
bce20c34d6 Include coinbase transactions in receivedby wallet rpcs (Andrew Toth)

Pull request description:

  The current `*receivedby*` RPCs filter out coinbase transactions. This doesn't seem correct since an output to your address in a coinbase transaction *is* receiving those coins.

  This PR corrects this behaviour. Also, a new option `include_immature_coinbase` is added (default=`false`) that includes immature coinbase transactions when set to true.

  However, since this is potentially a breaking change this PR introduces a hidden configuration option `-deprecatedrpc=exclude_coinbase`. This can be set to revert to previous behaviour. If no reports of broken workflow are received, then this option can be removed in a future release.

  Fixes https://github.com/bitcoin/bitcoin/issues/14654.

ACKs for top commit:
  jnewbery:
    reACK 1dcba996d3

Tree-SHA512: bfc43b81279fea5b6770a4620b196f6bc7c818d221b228623e9f535ec75a2406bc440e3df911608a3680f11ab64c5a4103917162114f5ff7c4ca8ab07bb9d3df
2021-12-07 20:52:13 +01:00
Carl Dong
7f15eff2dd style-only: Remove redundant scope in *Chainstate
I strongly recommend reviewing with the following git-diff flags:
  --ignore-space-change
2021-12-07 14:48:49 -05:00
Carl Dong
89bec827fd Collapse the 2 cs_main locks in LoadChainstate 2021-12-07 14:48:49 -05:00
Carl Dong
3b1584b794 Remove all #include // for * comments 2021-12-07 14:48:49 -05:00
Carl Dong
9a5a5a3d08 test/setup: Use LoadChainstate
This commit coalesces the chainstate loading sequence between our unit
test and non-unit test init codepaths.
2021-12-07 14:48:49 -05:00
Carl Dong
c541da0d62 node/chainstate: Add options for in-memory DBs
[META] In a future commit, these options will be used in TestingSetup to
       ensure that the DBs are in-memory.
2021-12-07 14:48:49 -05:00
Carl Dong
ceb9790341 node/caches: Remove intermediate variables 2021-12-07 14:48:49 -05:00
Carl Dong
ac4bf138b8 node/caches: Extract cache calculation logic
I strongly recommend reviewing with the following git-diff flags:
  --color-moved=dimmed_zebra --color-moved-ws=allow-indentation-change

[META] In a future commit, this function will be re-used in TestingSetup
       so that the behaviour matches across test and non-test init
       codepaths.
2021-12-07 14:48:49 -05:00
Carl Dong
15f2e33bb3 validation: VerifyDB only needs Consensus::Params
Previously we were passing in CChainParams, when VerifyDB only needed
the Consensus::Params subset.
2021-12-07 14:48:49 -05:00
Carl Dong
4da9c076d1 node/chainstate: Decouple from ShutdownRequested
...instead allow optionally passing in a std::function<bool()>
2021-12-07 14:48:49 -05:00
Carl Dong
05441c2dc5 node/chainstate: Decouple from GetTime
...instead pass in a std::function<int64_t()>

Note that the static_cast is needed (apparently) for the compiler to
know which overloaded GetTime to choose.
2021-12-07 14:48:49 -05:00
Carl Dong
2414ebc18b init: Delay RPC block notif until warmup finished
See added code comment for more details.
2021-12-07 14:48:06 -05:00
MarcoFalke
eaf1c56502
Merge bitcoin/bitcoin#23692: mining, refactor: add m_mempool.cs thread safety lock assertions
275e9390e1 mining, refactor: add m_mempool.cs thread safety lock assertions (Jon Atack)

Pull request description:

  in src/node/miner to

  - BlockAssembler::addPackageTxs()
  - BlockAssembler::SkipMapTxEntry()
  - BlockAssembler::UpdatePackagesForAdded()

  These functions have thread safety lock annotations in their declarations but are missing the corresponding run-time lock assertions in their definitions.

  Per doc/developer-notes.md: "Combine annotations in function declarations with run-time asserts in function definitions."

ACKs for top commit:
  shaavan:
    ACK 275e9390e1. Thanks for catching and fixing this!

Tree-SHA512: 1c6f1ad1bbd94ff391fc8ce1e3b95d88bd3db5db804a1a5ef4636e54b29f5801f79aa9ed753d34c9a79a58cf01c7ed890e7681ff1c7b0f16335dc062bbac31cc
2021-12-07 18:48:33 +01:00
MarcoFalke
fa1571b156
doc: Add missing optional to MempoolEntryDescription 2021-12-07 15:48:04 +01:00
MarcoFalke
4fd0ce75c5
Merge bitcoin/bitcoin#22689: rpc: deprecate top-level fee fields in getmempool RPCs
2f9515f37a rpc: move fees object to match help (josibake)
07ade7db8f doc: add release note for fee field deprecation (josibake)
2ee406ce3e test: add functional test for deprecatedrpc=fees (josibake)
35d928c632 rpc: deprecate fee fields from mempool entries (josibake)

Pull request description:

  per #22682 , top level fee fields for mempool entries have been deprecated since 0.17 but are still returned. this PR properly deprecates them so that they are no longer returned unless `-deprecatedrpc=fees` is passed.

  the first commit takes care of deprecation and also updates `test/functional/mempool_packages.py` to only use the `fees` object. the second commit adds a new functional test for `-deprecatedrpc=fees`

  closes #22682

  ## questions for the reviewer

  * `-deprecatedrpc=fees` made the most sense to me, but happy to change if there is a name that makes more sense
  * #22682 seems to indicate that after some period of time, the fields will be removed all together. if we have a rough idea of when this will be, i can add a `TODO: fully remove in vXX` comment to `entryToJSON`

  ## testing
  to get started on testing, compile, run the tests, and start your node with the deprecated rpcs flag:

  ```bash
  ./src/bitcoind -daemon -deprecatedrpc=fees
  ```
  you should see entries with the deprecated fields like so:
  ```json
  {
    "<txid>": {
      "fees": {
        "base": 0.00000671,
        "modified": 0.00000671,
        "ancestor": 0.00000671,
        "descendant": 0.00000671
      },
      "fee": 0.00000671,
      "modifiedfee": 0.00000671,
      "descendantfees": 671,
      "ancestorfees": 671,
      "vsize": 144,
      "weight": 573,
     ...
    },
  ```
  you can also check `getmempoolentry` using any of the txid's from the output above.

  next start the node without the deprecated flag, repeat the commands from above and verify that the deprecated fields are no longer present at the top level, but present in the "fees" object

ACKs for top commit:
  jnewbery:
    reACK 2f9515f37a
  glozow:
    utACK 2f9515f37a

Tree-SHA512: b175f4d39d26d96dc5bae26717d3ccfa5842d98ab402065880bfdcf4921b14ca692a8919fe4e9969acbb5c4d6e6d07dd6462a7e0a0a7342556279b381e1a004e
2021-12-07 15:26:06 +01:00
Jon Atack
275e9390e1 mining, refactor: add m_mempool.cs thread safety lock assertions
in src/node/miner to:

- BlockAssembler::addPackageTxs()
- BlockAssembler::SkipMapTxEntry()
- BlockAssembler::UpdatePackagesForAdded()

These functions have thread safety lock annotations in
their declarations but are missing the corresponding
run-time lock assertions in their definitions.

Per doc/developer-notes.md: "Combine annotations in function
declarations with run-time asserts in function definitions."
2021-12-07 15:01:43 +01:00
MarcoFalke
95fe477fd1
Merge bitcoin/bitcoin#23693: Revert "Fixes Bug in Transaction generation in ComplexMempool benchmark"
faa185bb3a Revert "Fixes Bug in Transaction generation in ComplexMempool benchmark" (MarcoFalke)

Pull request description:

  Developers are reporting crashes (potentially OOM) on IRC, but I can't reproduce. Still, revert this for now, since one developer reported the bare metal this was running on crashed.

Top commit has no ACKs.

Tree-SHA512: 080db4fcfc682b68f4cc40dfabd9d3e0e3f6e6297ce4b782d5de2c83bc18f85f60efb1cda64c51e23c4fd2a05222a904e7a11853d9f9c052dcd26a53aa00b235
2021-12-07 14:52:18 +01:00
MarcoFalke
faa185bb3a
Revert "Fixes Bug in Transaction generation in ComplexMempool benchmark"
This reverts commit 29e983386b.
2021-12-07 14:29:18 +01:00
W. J. van der Laan
6ac8c4f700
Merge bitcoin/bitcoin#23634: rpc: add missing scantxoutset examples
1ed5681407 rpc: add missing scantxoutset examples (Sebastian Falbesoner)

Pull request description:

  The scantxoutset RPC and its help text was at last improved in #16285, but it's still missing examples (see https://github.com/bitcoin/bitcoin/pull/16285#issuecomment-529313781).

  ~Note that the example descriptor used doesn't follow the developer guideline of using invalid bech32 addresses, as the RPC is not wallet-related and it's use-case is merely to look up state information (i.e. there is no danger of sending funds to a wrong address).~ For the sake of simplicity, the raw descriptor for an early coinbase payout address (block 9) is taken, i.e. it yields results even at an early stage of IBD. Happy to change that though if there are other suggestions.

ACKs for top commit:
  shaavan:
    reACK 1ed5681407

Tree-SHA512: 057ad9ac0d019035bee2332440128de0ef08580bbeae80182ff74771beead3555c4bf7008071a97bbb6a8d85fb85d0f0754fb7941db2c5b755eae1ac9aa65318
2021-12-07 12:38:23 +01:00
MarcoFalke
abc26fa378
Merge bitcoin/bitcoin#22856: test: Fix bug in transaction generation in ComplexMempool benchmark
29e983386b Fixes Bug in Transaction generation in ComplexMempool benchmark (Shorya)

Pull request description:

  This fixes issues with `ComplexMempool` benchmark introduced in [#17292](https://github.com/bitcoin/bitcoin/pull/17292) , this stress test benchmarks performance of ancestor and descendant tracking of mempool graph algorithms on a complex Mempool.

  This Benchmark first creates 100 base transactions and stores them in `available_coins` vector. `available_coins` is used for selecting ancestor transactions while creating 800 new transactions. For this a random transaction is picked from `available_coins` and some of its outputs are mapped to the inputs of the new transaction being created.

  Now in case we exhaust all the outputs of an entry in `available_coins` then we need to remove it from `available_coins` before the next iteration of choosing a potential ancestor , it is now implemented with this patch.

   As the index of the entry is randomly chosen from `available_coins` , In order to remove it from the vector , if index of the selected entry is not at the end of `available_coins` vector , it is swapped with the entry at the back of the vector , then the entry at the end of `available_coins` is popped out.

  Earlier the code responsible for constructing outputs of the newly created transaction was inside the loop used for assigning ancestors to the transaction , which does some unnecessary work as it creates outputs of the transaction again and again , now it is moved out of the loop so outputs of the transaction are created just once before adding it to the final list of the transactions created. This one is a minor change to save some computation.

   These changes have changed the `ComplexMempool` benchmark results on `bitcoin:master` as follows :

  **Before**

  >
  |               ns/op |                op/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |      232,881,625.00 |                4.29 |    0.7% |      2.55 | `ComplexMemPool`

  **After**

  >
  |               ns/op |                op/s |    err% |     total | benchmark
  |--------------------:|--------------------:|--------:|----------:|:----------
  |      497,275,135.00 |                2.01 |    0.5% |      5.49 | `ComplexMemPool`

Top commit has no ACKs.

Tree-SHA512: d6946d7e65c55f54c84cc49d7abee52e59ffc8b7668b3c80b4ce15a57690ab00a600c6241cc71a2a075def9c30792a311256fed325ef162f37aeacd2cce93624
2021-12-07 10:46:11 +01:00
fanquake
e457513eb1
Merge bitcoin/bitcoin#23631: p2p: Don't use timestamps from inbound peers for Adjusted Time
0c85dc30e6 p2p: Don't use timestamps from inbound peers (Martin Zumsande)

Pull request description:

  `GetAdjustedTime()` (used e.g. in validation and addrman) returns a time with an offset that is influenced by timestamps that our peers have sent us in their version message.

  Currently, timestamps from all peers are used for this.
  However, I think that it would make sense to ignore the timedata samples from inbound peers, making it much harder for others to influence the Adjusted Time in a targeted way.
  With the extra feeler connections (every 2 minutes on average) and extra block-relay-only connections (every 5 minutes on average) there are also now plenty of opportunities to gather a meaningful number of timedata samples from outbound peers.

  There are some measures in place to prevent abuse: the `-maxtimeadjustment` parameter with a default of 70 minutes, warnings in cases of large deviations, only using the first 200 samples ([explanation](383d350bd5/src/timedata.cpp (L57-L72))), but I think that only using samples from outbound connections in the first place would be an additional safety measure that would make sense.

  See also issue #4521 for further context and links: There have been several discussions in the past about replacing or abolishing the existing timedata system.

ACKs for top commit:
  jnewbery:
    Concept and code review ACK 0c85dc30e6
  naumenkogs:
    ACK 0c85dc30e6
  vasild:
    ACK 0c85dc30e6

Tree-SHA512: 2d6375305bcae034d68b58b7a07777b40ac430dfed554c88e681a048c527536691e1b7d08c0ef995247d356f8e81aa0a4b983bf2674faf6a416264e5f1af0a96
2021-12-07 17:36:53 +08:00
MarcoFalke
084c81c8b6
Merge bitcoin/bitcoin#23547: Bugfix: RPC/mining: Fail properly in estimatesmartfee if smart fee data is unavailable
cd8d156354 Bugfix: RPC/mining: Fail properly in estimatesmartfee if smart fee data is unavailable (Luke Dashjr)

Pull request description:

  Fixes a regression introduced by #22722

  (Not entirely sure on the solution)

ACKs for top commit:
  prayank23:
    crACK cd8d156354
  darosior:
    utACK cd8d156354
  kristapsk:
    utACK cd8d156354

Tree-SHA512: eb4aa3cc345c69c44ffd5733b51b90eefe1d7854b7a2855e8cbb98268db24d43b7d0ae9fbb0eccf9b6dc01da644d19433cc77fec52ff67bf890be1fc53a67fc4
2021-12-07 10:16:19 +01:00
MarcoFalke
89ea2b3809
Merge bitcoin/bitcoin#20583: rpc: Add missing BlockUntilSyncedToCurrentChain to wallet RPCs
fa5362a9a0 rpc: Add missing BlockUntilSyncedToCurrentChain to wallet RPCs (MarcoFalke)

Pull request description:

  Wallet RPCs that allow a rescan based on block-timestamp or block-height
  need to sync with the active chain first, because the user might assume
  the wallet is up-to-date with the latest block they got reported via a
  blockchain RPC.

ACKs for top commit:
  meshcollider:
    utACK fa5362a9a0

Tree-SHA512: d4831f1f08f854f9a49fc969de86c438f856e41c2163c801a6ff36dc2f6299cb342b44663279c524a8b7ca9a50895db1243cd7d49bed79277ada857213f20a26
2021-12-07 09:24:34 +01:00
MarcoFalke
b7e63306e8
Merge bitcoin/bitcoin#23687: Remove unused (and broken) functionality in SpanReader
31ba1af74a Remove unused (and broken) functionality in SpanReader (Pieter Wuille)

Pull request description:

  This removes the ability to set an offset in the `SpanReader::SpanReader` constructor, as the current code is broken since #23653. All call sites use `pos=0`, so it is actually unused. If future call sites need it, `SpanReader{a, b, c, d}` is equivalent to `SpanReader{a, b, c.subspan(d)}`.

  It also removes the ability to deserialize from `SpanReader` directly from the constructor. This too is unused, and can be more idiomatically simulated using `(SpanReader{a, b, c} >> x >> y >> z)` instead of `SpanReader{a, b, c, x, y, z}`.

  This was pointed out by achow101 in https://github.com/bitcoin/bitcoin/pull/23653#discussion_r763370432.

ACKs for top commit:
  jb55:
    crACK 31ba1af74a
  achow101:
    ACK 31ba1af74a

Tree-SHA512: 700ebcd74147628488c39168dbf3a00f8ed41709a26711695f4bf036250a9b115574923bbf96040ec7b7fee4132d6dbbcb5c6e5a2977c4beb521dc1500e6ed53
2021-12-07 09:19:18 +01:00
MarcoFalke
9a53ba4618
Merge bitcoin/bitcoin#23676: rpc: correct getnewaddress/getrawchangeaddress address_type helptext
5767208504 correct rpc address_type helptext (brianddk)

Pull request description:

  RPC calls `getnewaddress`/`getrawchangeaddress` support the address_type of `bech32m` but it is omitted in the `RPCHelpMan` help text.

  The `createmultisig` and `addmultisigaddress` help text was not updated since `bech32m` is not yet supported in these.

ACKs for top commit:
  shaavan:
    ACK 5767208504

Tree-SHA512: 3c0cfb96019ca6d316c4a2fe27786d1b621c49b31b3aa61068bad737a5a0ceed89babad704b9923f9aedcabfa670d752916803bdf22236403061ddf9295a2637
2021-12-07 09:07:14 +01:00
MarcoFalke
42b25025fa
Merge bitcoin/bitcoin#23644: wallet: Replace confusing getAdjustedTime() with GetTime()
fa37e798b2 wallet: Replace confusing getAdjustedTime() with GetTime() (MarcoFalke)

Pull request description:

  Setting `nTimeReceived` to the adjusted time has several issues:

  * `m_best_block_time` is set to the "unadjusted" time, thus a comparison of the two times is like comparing apples to oranges. In the worst case this opens up an attack vector where remote peers can force a premature re-broadcast of wallet txs.
  * The RPC documentation for `"timereceived"` doesn't mention that the network adjusted time is used, possibly confusing users when the time reported by RPC is off by a few seconds compared to their local timestamp.

  Fix all issues by replacing the call with `GetTime()`. Also a style fix: Use non-narrowing integer conversion in the RPC method.

ACKs for top commit:
  theStack:
    Code-review ACK fa37e798b2
  shaavan:
    crACK fa37e798b2

Tree-SHA512: 8d020ba400521246b7aed4b6c41319fc70552e8c69e929a5994500375466a9edac02a0ae64b803dbc6695df22276489561a23bd6e030c44c97d288f7b9b2b3fa
2021-12-07 09:02:06 +01:00
Carl Dong
8d466a8504 Move -checkblocks LogPrintf to AppInitMain 2021-12-06 16:41:58 -05:00
Carl Dong
aad8d59789 node/chainstate: Reduce coupling of LogPrintf
...by moving the try/catch out of LoadChainstate

I strongly recommend reviewing with the following git-diff flags:
  --color-moved=dimmed_zebra --color-moved-ws=allow-indentation-change
2021-12-06 16:41:58 -05:00
Carl Dong
b345979a2b node/chainstate: Decouple from concept of uiInterface
...instead allow the caller to optionally pass in callbacks which are
triggered for certain events.

Behaviour change: The string "Verifying blocks..." was previously
printed for each chainstate in chainman which did not have an
effectively empty coinsview, now it will be printed once unconditionally
before we call VerifyLoadedChain.
2021-12-06 16:41:33 -05:00
Pieter Wuille
31ba1af74a Remove unused (and broken) functionality in SpanReader
This removes the ability to set an offset in the SpanReader constructor,
as the current code is broken. All call sites use pos=0, so it is actually
unused. If future call sites need it, SpanReader{a, b, c, d} is equivalent
to SpanReader{a, b, c.subspan(d)}.

It also removes the ability to deserialize from SpanReader directly from
the constructor. This too is unused, and can be more idiomatically
simulated using (SpanReader{a, b, c} >> x >> y >> z) instead of
SpanReader{a, b, c, x, y, z}.
2021-12-06 16:18:14 -05:00
Carl Dong
ca7c0b934d Split off VerifyLoadedChainstate 2021-12-06 15:58:10 -05:00
Carl Dong
adf4912d77 node/chainstate: Remove do/while loop
I strongly recommend reviewing with the following git-diff flags:
  --ignore-space-change
2021-12-06 15:57:46 -05:00
Carl Dong
975235ca0a Move init logistics message for BAD_GENESIS_BLOCK to init.cpp 2021-12-06 15:56:55 -05:00
Carl Dong
8715658983 Move mempool nullptr Assert out of LoadChainstate 2021-12-06 15:56:55 -05:00
Carl Dong
9162a4f93e node/chainstate: Decouple from concept of NodeContext
...instead pass in only the necessary information

Also allow mempool to be a nullptr
2021-12-06 15:56:55 -05:00
Carl Dong
c7a5c46e6f node/chainstate: Decouple from ArgsManager
...instead pass in only the necessary information
2021-12-06 15:56:55 -05:00
Carl Dong
ae9121f958 node/chainstate: Decouple from stringy errors
This allows us to separate the initialization code from translations and
error reporting.

This change changes the caller semantics of LoadChainstate quite
drastically.

To see that this change doesn't change behaviour, observe that:

1. Prior to this change, LoadChainstate returned false only in the "bad
   genesis block" failure case (by returning InitError()), indicating
   that the caller should immediately bail. After this change, the
   corresponding ERROR_BAD_GENESIS_BLOCK handler in src/init.cpp
   maintains behavioue by also bailing immediately.

2. The failed_* temporary booleans were only used to break out of the
   outer do/while(false) loop. They can therefore be safely removed.
2021-12-06 15:56:50 -05:00
Carl Dong
cbac28b72f node/chainstate: Decouple from GetTimeMillis
...instead just move it out
2021-12-06 15:55:49 -05:00
Carl Dong
cb64af9635 node: Extract chainstate loading sequence
I strongly recommend reviewing with the following git-diff flags:
  --color-moved=dimmed_zebra --color-moved-ws=allow-indentation-change

[META] This commit is intended to be as close to a move-only commit as
       possible, and lingering ugliness will be resolved in subsequent
       commits.

A few variables that are passed in by value instead of by reference
deserve explanation:

- fReset and fReindexChainstate are both local variables in AppInitMain
  and are not modified in the sequence

- fPruneMode, despite being a global, is only modified in
  AppInitParameterInteraction, long before LoadChainstate is called

----

[META] This semantic will change in a future commit named
       "node/chainstate: Decouple from stringy errors"
2021-12-06 15:55:16 -05:00
W. J. van der Laan
786ffb3ae4
Merge bitcoin/bitcoin#17160: refactor: net: subnet lookup: use single-result LookupHost()
a989f98d24 refactor: net: subnet lookup: use single-result LookupHost() (Sebastian Falbesoner)

Pull request description:

  plus describe single IP subnet case for more clarity

ACKs for top commit:
  jonatack:
    utACK a989f98d24 the patch rebases cleanly to master, the debug build is green, and it is essentially the same patch as c8991f0251dd2a modulo local variable naming, braced initialization, and a comment
  vasild:
    ACK a989f98d24

Tree-SHA512: 082d3481b1fa5e5f3267b7c4a812954b67b36d1f94c5296fe20110699f053e5042dfa13f728ae20249e9b8d71e930c3b119410125d0faeccdfbdc259223ee3a6
2021-12-06 19:59:35 +01:00
W. J. van der Laan
695ba2fe54
Merge bitcoin/bitcoin#23486: rpc: Only allow specific types to be P2(W)SH wrapped in decodescript
99993425af rpc: Only allow specific types to be P2(W)SH wrapped in decodescript (MarcoFalke)

Pull request description:

  It seems confusing to return a P2SH wrapping address that is eventually either policy- or consensus-unspendable.

ACKs for top commit:
  laanwj:
    Code review re-ACK 99993425af

Tree-SHA512: 3cd530442acee7c295d244995f0f17b2cae7212f1e0970bb5807621f8ff8e4308a3236b385d77087cd493d32ee524813d8edd15e91d937ef9a800094b7bc4946
2021-12-06 19:26:04 +01:00
Andrew Toth
bce20c34d6
Include coinbase transactions in receivedby wallet rpcs 2021-12-06 12:30:58 -05:00
MarcoFalke
dca9ab48b8
Merge bitcoin/bitcoin#23661: cover DisconnectBlock with lock annotation
7da4a8ffb3 cover DisconnectBlock with lock annotation (James O'Beirne)

Pull request description:

  While reviewing #23630, I noticed that `DisconnectBlock` is uncovered by lock annotations. CoinsTip() access requires cs_main and therefore so should this function.

ACKs for top commit:
  jonatack:
    ACK 7da4a8ffb3

Tree-SHA512: 3e2b0247c138b31deeadcd48eb3f7bc8d32c0b6bb6d6e94ccf8ea0cbbc50b1b35d83f662eee432f2bd2d87a3fe9c94604da806ec711df93298bfb0ab34a5a05b
2021-12-06 13:38:54 +01:00
MarcoFalke
8b2c0df83e
Merge bitcoin/bitcoin#23614: test: add unit test for block-relay-only eviction
4740fe8212 test: Add test for block relay only eviction (Martin Zumsande)

Pull request description:

  Adds a unit test for block-relay-only eviction logic added in #19858, which was not covered by any tests before. The added test is very similar to the existing `stale_tip_peer_management` unit test, which tests the analogous logic for regular outbound peers.

ACKs for top commit:
  glozow:
    reACK 4740fe8212
  rajarshimaitra:
    tACK 4740fe8212
  shaavan:
    ACK 4740fe8212. Great work @ mzumsande!
  LarryRuane:
    ACK 4740fe8212

Tree-SHA512: 5985afd7d8f7ae311903dbbf6b7d526e16309c83c88ae6dd6551960c0b186156310a6be0cf6b684f82ac1378d0fc5aa3717f0139e078471013fceb6aebe81bf6
2021-12-06 13:23:55 +01:00
W. J. van der Laan
22feb7fee9
Merge bitcoin/bitcoin#23577: Follow-ups to Bech32 error detection
a4fe70171b Make Bech32 LocateErrors return error list rather than using out-arg (Samuel Dobson)
2fa4fd1961 Use std::iota instead of manually pushing range (Samuel Dobson)
405c96fc9f Use bounds-checked array lookups in Bech32 error detection code (Samuel Dobson)
28d9c2857f Simplify encoding of e in GF(1024) tables to (1,0) (Samuel Dobson)
14358a029d Replace GF1024 tables and syndrome constants with compile-time generated constexprs. (Samuel Dobson)
63f7b69779 Update release note for bech32 error detection (Samuel Dobson)
c8b9a224e7 Report encoding type in bech32 error message (Samuel Dobson)
92f0cafdca Improve Bech32 boost tests (Samuel Dobson)
bb4d3e9b97 Address review comments for Bech32 error validation (Samuel Dobson)

Pull request description:

  A number of follow-ups and improvements to the bech32 error location code, introduced in #16807.

  Notably, this removes the hardcoded GF1024 tables in favour of constexpr table generation.

ACKs for top commit:
  laanwj:
    Re-ACK a4fe70171b

Tree-SHA512: 6312373c20ebd6636f5797304876fa0d70fa777de2f6c507245f51a652b3d1224ebc55b236c9e11e6956c1e88e65faadab51d53587078efccb451455aa2e2276
2021-12-06 12:18:17 +01:00
MarcoFalke
fadc0c80ae
p2p: Make timeout mockable and type safe, speed up test 2021-12-06 10:47:52 +01:00
MarcoFalke
99993425af
rpc: Only allow specific types to be P2(W)SH wrapped in decodescript 2021-12-06 10:33:12 +01:00
Samuel Dobson
a4fe70171b Make Bech32 LocateErrors return error list rather than using out-arg 2021-12-06 14:17:41 +13:00
Andrew Chow
05300c1439 Use SelectionResult in SelectCoins
Replace setCoinsRet and nValueRet with SelectionResult
2021-12-05 13:44:10 -05:00
Andrew Chow
9d9b101d20 Use SelectionResult in AttemptSelection
Replace setCoinsRet and nValueRet with a SelectionResult in
AttemptSelection
2021-12-05 13:44:09 -05:00
Andrew Chow
bb50850a44 Use SelectionResult for waste calculation 2021-12-05 13:43:28 -05:00
Andrew Chow
e8f7ae5eb3 Make an OutputGroup for preset inputs
In SelectCoins, for our preset inputs, we combine all of the preset
inputs into a single OutputGroup. This allows us to combine the preset
inputs with additional selection algo results.
2021-12-05 13:40:48 -05:00
Andrew Chow
51a9c00b4d Return SelectionResult from SelectCoinsSRD
Changes SelectCoinsSRD to return a SelectionResult.
2021-12-05 13:40:48 -05:00
Andrew Chow
0ef6184575 Return SelectionResult from KnapsackSolver
Returns a std::optional<SelectionResult> from KnapsackSolver instead of
using out parameters for the inputs set and selected value.
2021-12-05 13:40:42 -05:00
Andrew Chow
60d2ca72e3 Return SelectionResult from SelectCoinsBnB
Removes coins_out and value_ret has SelectCoinsBnB return a
std::optional<SelectionResult>
2021-12-05 13:39:51 -05:00
Andrew Chow
a339add471 Make member variables of SelectionResult private 2021-12-05 13:39:51 -05:00
Andrew Chow
cbf0b9f4ff scripted-diff: Use SelectionResult in coin selector tests
Replace the CoinSet actual_selection with a SelectionResult
expected_result. We don't use the SelectionResult functions yet, but
will soon.

-BEGIN VERIFY SCRIPT-
sed -i 's/CoinSet actual_selection/SelectionResult expected_result(CAmount(0))/' src/wallet/test/coinselector_tests.cpp
sed -i 's/actual_selection/expected_result.m_selected_inputs/' src/wallet/test/coinselector_tests.cpp
sed -i 's/expected_result.m_selected_inputs.clear/expected_result.Clear/' src/wallet/test/coinselector_tests.cpp
-END VERIFY SCRIPT-
2021-12-05 13:39:51 -05:00
Andrew Chow
9d1d86da04 Introduce SelectionResult struct
Introduces a SelectionResult struct which contains the set of selected
inputs and the total transaction fee for the transaction. This will be
used by the various SelectCoins* functions. Additionally helpers are
provided to compute the total input value and result comparisons.
2021-12-05 13:39:41 -05:00
brianddk
5767208504
correct rpc address_type helptext
added address_type of `bech32m` to rpc calls `getnewaddress`/`getrawchangeaddress`
2021-12-04 18:47:15 -06:00
Hennadii Stepanov
ea989deecc
Merge bitcoin-core/gui#493: qt: Handle Android back key in the Node window
a56a104938 qt: Handle Android back key in the Node window (Hennadii Stepanov)
f045f98717 qt, android: Add GUIUtil::IsEscapeOrBack helper (Hennadii Stepanov)

Pull request description:

  On master (4633199cc8) there are no means to return from the Node window to the main one on Android.

  This PR assigns this functionality to the Android back key:

  ![Screenshot_1638395318](https://user-images.githubusercontent.com/32963518/144320316-af5599ac-0379-40e6-9887-7f5ee30b97ae.png)

ACKs for top commit:
  icota:
    utACK a56a104938

Tree-SHA512: 379c1ad8c6bffa037e861b88c66eb33872d7f7d54aa2f76289a51c55d79a37a0c16262b20f22d00fda11522c7df1f3561c1ceae34cd7a85da94aee4c6cdcfaaf
2021-12-05 02:45:13 +02:00
James O'Beirne
7da4a8ffb3
cover DisconnectBlock with lock annotation
CoinsTip() access requires cs_main and therefore so should this function.
2021-12-03 10:34:07 -05:00
MarcoFalke
57982f419e
Merge bitcoin/bitcoin#23654: fuzz: Rework rpc fuzz target
fa52a86fd3 fuzz: Rework rpc fuzz target (MarcoFalke)

Pull request description:

  Changes (reason):

  * Return `void` in `CallRPC` (the result is unused anyway)
  * Reduce the `catch`-scope of `std::runtime_error` to `RPCConvertValues` (Code clarity and easier bug-finding)
  * Crash when an internal bug is detected (bugs are bad)

ACKs for top commit:
  shaavan:
    Code Review ACK fa52a86fd3

Tree-SHA512: 576411a0e50bca9be3e6ffaf745001b1808fd37029251f8ec2c279e0671efe91d43dd81fd4ca26871c28b119e593ee2a0043d4b75f44da578f17541ee3afd696
2021-12-03 14:02:51 +01:00