Commit graph

4607 commits

Author SHA1 Message Date
Andreas Kouloumos
f2f6068b69 test: MiniWallet: add send_self_transfer_chain to create chain of txns
With this new method, a chain of transactions can be created. This
method is introduced to further simplify the mempool_package_limits.py
tests.
2022-08-01 19:11:36 +03:00
Andreas Kouloumos
1d6b438ef0 test: use MiniWallet to simplify mempool_package_limits.py tests
Moved `bulk_transaction` into MiniWallet class as `_bulk_tx` private
helper method to be used when the newly added `target_weight` option is
passed to `create_self_transfer*`
2022-08-01 19:11:35 +03:00
brunoerg
155344960b test: negative/unknown rpcserialversion should throw an init error 2022-08-01 10:55:05 -03:00
MacroFake
2bca32b7c3
Merge bitcoin/bitcoin#24799: Add test case mimicking issue 24765
395767e9f1 Add test case mimicking issue 24765 (Pieter Wuille)

Pull request description:

  This adds a functional test for the concern brought up in #24765. It turned out to be a non-issue, but since I wrote it anyway, it can't hurt to add it.

Top commit has no ACKs.

Tree-SHA512: fc8d57129d8c68f6d9a41b94b5ff676c87b31f53bc958195d4fe312530ec3e038ebd0bc5e8b9d56be77b7b63fd94574685901901404a4ab8726a5e09d89e86c8
2022-08-01 11:58:57 +02:00
MacroFake
eeb5a94e27
Merge bitcoin/bitcoin#25528: ci: run USDT interface tests in the CI
cc7335edc8 ci: run USDT interface test in a VM (0xb10c)
dba6f82342 test: adopt USDT utxocache interface tests (0xb10c)
220a5a2841 test: hook into PID in tracing tests (0xb10c)

Pull request description:

  Changes a CI task that runs test the previously not run `test/functional/interface_usdt_*.py` functional tests (added in https://github.com/bitcoin/bitcoin/pull/24358).

  This task is run as CirussCI `compute_engine_instance` VM as hooking into the tracepoints is not possible in CirrusCI docker containers (https://github.com/bitcoin/bitcoin/issues/23296#issuecomment-1024920845). We use an unoffical PPA and untrusted  `bpfcc-tools` package in the CI as the Ubuntu jammy and Debian bullseye packages are outdated. We hope use an official package when new Ubuntu/Debian releases are available for the use with Google Compute Engine.

  We make sure to hook into `bitcoind` binaries in USDT interface tests via their PID, instead of their path. This makes sure multiple functional tests running in parallel don't interfere with each other.

  The utxocache USDT interface tests is adopted to a change of the functional test framework that wasn't detected as the tests weren't run in the CI. As the tracepoints expose internals, it can happen that we need to adopt the interface test when internals change. This is a bit awkward, and if it happens to frequently, we should consider generalizing the tests a bit more. For now it's fine, I think.

  See the individual commit messages for more details on the changes.

  Fixes https://github.com/bitcoin/bitcoin/issues/24782
  Fixes https://github.com/bitcoin/bitcoin/issues/23296

  I'd like to hear from reviewers:
  - Are we OK with using the [`hadret/bpfcc`](https://launchpad.net/~hadret/+archive/ubuntu/bpfcc) PPA for now? There is a clear plan when to drop it and as is currently, it could only impact the newly added VM task.
  - ~~Adding a new task increases CI runtime and costs. Should an existing `container` CI task be ported to a VM and reused instead?~~ Yes, see https://github.com/bitcoin/bitcoin/pull/25528#issuecomment-1179509525

ACKs for top commit:
  MarcoFalke:
    cr ACK cc7335edc8

Tree-SHA512: b7fddccc0a77d82371229d048abe0bf2c4ccaa45906497ef3040cf99e7f05561890aef4c253c40e4afc96bb838c9787fae81c8454c6fd9db583276e005a4ccb3
2022-08-01 11:27:29 +02:00
MacroFake
c5ba1d92b6
Merge bitcoin/bitcoin#25610: wallet, rpc: Opt in to RBF by default
ab3c06db1a doc: Release notes for default RBF (Andrew Chow)
61d9149e78 rpc: Default rbf enabled (Andrew Chow)
e3c33637ba wallet: Enable -walletrbf by default (Andrew Chow)

Pull request description:

  The GUI currently opts in to RBF by default, but RPCs do not, and `-walletrbf` is default disabled. This PR makes the default in those two places to also opt in.

  The last time this was proposed (#9527), the primary objections were the novelty at the time, the inability to bump transactions, and the gui not having the option to disable rbf. In the 5 years since, RBF usage has steadily grown, with ~27% of txs opting in. The GUI has the option to enable/disable RBF, and is also defaulted to having it enabled. And we have the ability to bump RBF'd transactions in both the RPC and the GUI. So I think it makes sense to finally change the default to always opt in to RBF.

ACKs for top commit:
  darosior:
    reACK ab3c06db1a
  aureleoules:
    ACK ab3c06db1a.
  glozow:
    utACK ab3c06db1a

Tree-SHA512: 81b012c5033e270f86a87a6a196ccc549eb54b158eebf88e917cc6621d40d7bdcd1566b602688907dd5d364b95a557b29f97dce869cea512e339588262c027b6
2022-08-01 10:53:11 +02:00
Andrew Chow
1abbae65eb
Merge bitcoin/bitcoin#24584: wallet: avoid mixing different OutputTypes during coin selection
71d1d13627 test: add unit test for AvailableCoins (josibake)
da03cb41a4 test: functional test for new coin selection logic (josibake)
438e04845b wallet: run coin selection by `OutputType` (josibake)
77b0707206 refactor: use CoinsResult struct in SelectCoins (josibake)
2e67291ca3 refactor: store by OutputType in CoinsResult (josibake)

Pull request description:

  # Concept

  Following https://github.com/bitcoin/bitcoin/pull/23789, Bitcoin Core wallet will now generate a change address that matches the payment address type. This improves privacy by not revealing which of the outputs is the change at the time of the transaction in scenarios where the input address types differ from the payment address type. However, information about the change can be leaked in a later transaction. This proposal attempts to address that concern.

  ## Leaking information in a later transaction

  Consider the following scenario:

  ![mix input types(1)](https://user-images.githubusercontent.com/7444140/158597086-788339b0-c698-4b60-bd45-9ede4cd3a483.png)

  1. Alice has a wallet with bech32 type UTXOs and pays Bob, who gives her a P2SH address
  2. Alice's wallet generates a P2SH change output, preserving her privacy in `txid: a`
  3. Alice then pays Carol, who gives her a bech32 address
  4. Alice's wallet combines the P2SH UTXO with a bech32 UTXO and `txid: b` has two bech32 outputs

  From a chain analysis perspective, it is reasonable to infer that the P2SH input in `txid: b` was the change from `txid: a`. To avoid leaking information in this scenario, Alice's wallet should avoid picking the P2SH output and instead fund the transaction with only bech32 Outputs. If the payment to Carol can be funded with just the P2SH output, it should be preferred over the bech32 outputs as this will convert the P2SH UTXO to bech32 UTXOs via the payment and change outputs of the new transaction.

  **TLDR;** Avoid mixing output types, spend non-default `OutputTypes` when it is economical to do so.

  # Approach

  `AvailableCoins` now populates a struct, which makes it easier to access coins by `OutputType`. Coin selection tries to find a funding solution by each output type and chooses the most economical by waste metric. If a solution can't be found without mixing, coin selection runs over the entire wallet, allowing mixing, which is the same as the current behavior.

  I've also added a functional test (`test/functional/wallet_avoid_mixing_output_types.py`) and unit test (`src/wallet/test/availablecoins_tests.cpp`.

ACKs for top commit:
  achow101:
    re-ACK 71d1d13627
  aureleoules:
    ACK 71d1d13627.
  Xekyo:
    reACK 71d1d13627 via `git range-diff master 6530d19 71d1d13`
  LarryRuane:
    ACK 71d1d13627

Tree-SHA512: 2e0716efdae5adf5479446fabc731ae81d595131d3b8bade98b64ba323d0e0c6d964a67f8c14c89c428998bda47993fa924f3cfca1529e2bd49eaa4e31b7e426
2022-07-28 18:16:51 -04:00
Andrew Chow
317ef0368b
Merge bitcoin/bitcoin#25670: test: check that combining PSBTs with different txs fails
4e616d20c9 test: check that combining PSBTs with different txs fails (Sebastian Falbesoner)
2a428c7989 test: support passing PSBTMaps directly to PSBT ctor (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `combinepsbt` RPC, in the case of combining two PSBTs with different transactions:
  b8067cd435/src/psbt.cpp (L24-L27)
  The calling function `CombinePSBTs` checks for the false return value and then returns the transaction error string `PSBT_MISMATCH`:
  b8067cd435/src/psbt.cpp (L433-L435)
  b8067cd435/src/util/error.cpp (L30-L31)

ACKs for top commit:
  instagibbs:
    reACK 4e616d20c9
  achow101:
    ACK 4e616d20c9

Tree-SHA512: 45b2b224b13b44ad69ae62e4bc20f74cab32770cf8127b026ec47a7520f7253148fdbf1fad612afece59e45a6738bef9a351ae87ea98dc83d095cc78f6db0318
2022-07-28 17:34:28 -04:00
Aurèle Oulès
7ab43eb811
test: remove unused if statements 2022-07-25 09:59:05 +02:00
Sebastian Falbesoner
4e616d20c9 test: check that combining PSBTs with different txs fails 2022-07-23 09:08:54 +02:00
Sebastian Falbesoner
2a428c7989 test: support passing PSBTMaps directly to PSBT ctor
This will allow to create simple PSBTs as short one-liners, without the
need to have three individual assignments (globals, inputs, outputs).
2022-07-23 08:48:08 +02:00
Andrew Chow
d67f89bd95
Merge bitcoin/bitcoin#25625: test: add test for decoding PSBT with per-input preimage types
71a751f6c3 test: add test for decoding PSBT with per-input preimage types (Sebastian Falbesoner)
faf43378e2 refactor: move helper `random_bytes` to util library (Sebastian Falbesoner)
fdc1ca3896 test: add constants for PSBT key types (BIP 174) (Sebastian Falbesoner)
1b035c03f9 refactor: move PSBT(Map) helpers from signet miner to test framework (Sebastian Falbesoner)
7c0dfec2dd refactor: move `from_binary` helper from signet miner to test framework (Sebastian Falbesoner)
597a4b35f6 scripted-diff: rename `FromBinary` helper to `from_binary` (signet miner) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `decodepsbt` RPC in the case that a PSBT with on of the per-input preimage types (`PSBT_IN_RIPEMD160`, `PSBT_IN_SHA256`, `PSBT_IN_HASH160`, `PSBT_IN_HASH256`; see [BIP 174](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki#Specification)) is passed. As preparation, the first four commits move the already existing helpers for (de)serialization of PSBTs and PSBTMaps from the signet miner to the test framework (in a new module `psbt.py`), which should be quite useful for further tests to easily create PSBTs.

ACKs for top commit:
  achow101:
    ACK 71a751f6c3

Tree-SHA512: 04f2671612d94029da2ac8dc15ff93c4c8fcb73fe0b8cf5970509208564df1f5e32319b53ae998dd6e544d37637a9b75609f27a3685da51f603f6ed0555635fb
2022-07-20 16:46:39 -04:00
fanquake
895937edb2
Merge bitcoin/bitcoin#25285: Add AutoFile without ser-type and ser-version and use it where possible
facc2fa7b8 Use AutoFile where possible (MacroFake)
6666803c89 streams: Add AutoFile without ser-type and ser-version (MacroFake)

Pull request description:

  This was done in the context of https://github.com/bitcoin/bitcoin/pull/25284 , but I think it also makes sense standalone.

  The basic idea is that serialization type should not be initialized when it is not needed. Same for the serialization version.

  So do this here for `AutoFile`. `CAutoFile` remains in places where it is not yet possible.

ACKs for top commit:
  laanwj:
    Code review ACK facc2fa7b8
  fanquake:
    ACK facc2fa7b8

Tree-SHA512: d82d024d55af57565ac53d9d1517afafc12b46964effba0332de62a6c77869356fa77f89e6d4834438fff44c45b64fccdf5a1358bfea03e28dfe55013b3c099d
2022-07-20 09:32:11 +01:00
Pieter Wuille
544b4332f0 Add wallet tests for spending rawtr() 2022-07-19 18:17:20 -04:00
Pieter Wuille
8d9670ccb7 Add rawtr() descriptor for P2TR with unknown tweak 2022-07-19 17:36:08 -04:00
josibake
da03cb41a4
test: functional test for new coin selection logic
Create a wallet with mixed OutputTypes and send a volley of payments,
ensuring that there are no mixed OutputTypes in the txs. Finally,
verify that OutputTypes are mixed only when necessary.
2022-07-19 18:42:21 +02:00
Sebastian Falbesoner
71a751f6c3 test: add test for decoding PSBT with per-input preimage types 2022-07-19 17:44:50 +02:00
Sebastian Falbesoner
faf43378e2 refactor: move helper random_bytes to util library
Can be easily reviewed with `--color-moved=dimmed-zebra`.
2022-07-19 17:42:35 +02:00
Sebastian Falbesoner
fdc1ca3896 test: add constants for PSBT key types (BIP 174)
Also take use of the constants in the signet miner to get rid of
magic numbers and increase readability and maintainability.
2022-07-19 15:40:51 +02:00
Sebastian Falbesoner
1b035c03f9 refactor: move PSBT(Map) helpers from signet miner to test framework
Can be easily reviewed with `--color-moved=dimmed-zebra`.
2022-07-19 15:40:51 +02:00
Sebastian Falbesoner
7c0dfec2dd refactor: move from_binary helper from signet miner to test framework
Can be easily reviewed with `--color-moved=dimmed-zebra`.
2022-07-19 15:40:51 +02:00
Antoine Poinsot
55f98d087e
rpc: output parent wallet descriptors for coins in listunspent 2022-07-19 12:46:15 +02:00
Antoine Poinsot
b724476158
rpc: output wallet descriptors for received entries in listsinceblock
The descriptor wallets allow an application to track coins of multiple
descriptors in a single wallet. However, such an application would not
previously be able to (easily) tell what received coin "belongs" to what
descriptor.

This commit tackles this issues by adding a "wallet_desc" entry to the
entries for received coins in 'listsinceblock'.
2022-07-19 12:46:01 +02:00
fanquake
47dad42833
Merge bitcoin/bitcoin#25629: univalue: Return more detailed type check error messages
fae5ce8795 univalue: Return more detailed type check error messages (MacroFake)
fafab147e7 move-only: Move UniValue::getInt definition to keep class with definitions only (MacroFake)

Pull request description:

  Print the current type and the expected type

ACKs for top commit:
  aureleoules:
    ACK fae5ce8795.

Tree-SHA512: 4ae720a012ff8245baf5cd7f844f93b946c58feebe62de6dfd84ebc5c8afb988295a94de7c01aef98aaf4c6228f7184ed622f37079c738924617e0f336ac5b6e
2022-07-19 11:24:53 +01:00
MacroFake
1b285b7807
Merge bitcoin/bitcoin#25590: wallet: Precompute Txdata after setting PSBT inputs' UTXOs
d2ed97656b wallet: Precompute Txdata after setting PSBT inputs' UTXOs (Andrew Chow)

Pull request description:

  If we are given a PSBT that is missing one or more input UTXOs, our
  PrecomputedTransactionData will be incorrect and missing information
  that it should otherwise have, and therefore we may not produce a
  signature when we should. To avoid this problem, we can do the
  precomputation after we have set the UTXOs the wallet is able to set for
  the PSBT.

  Also adds a test for this behavior.

ACKs for top commit:
  instagibbs:
    reACK d2ed97656b
  Sjors:
    ACK d2ed97656b
  aureleoules:
    ACK d2ed97656b.

Tree-SHA512: 71beb6c7946096e82cfca83f36277302aa9e69d27b4f6d73d7d8f2f9f0ea1c0d653e846fa6aebee5e4763f56f950b4481240e953f6a2412caa84908d519171e1
2022-07-19 10:58:25 +02:00
Andrew Chow
4aaa3b5200
Merge bitcoin/bitcoin#25351: rpc, wallet: Scan mempool after import* - Second attempt
1be7964189 test, wallet: Add mempool rescan test for import RPCs (Fabian Jahr)
833ce76df7 rpc, wallet: Document mempool rescan after importdescriptor, importwallet (Fabian Jahr)
0e396d1ba7 rpc, wallet: Document mempool scan after importmulti (Fabian Jahr)
e6d3ef8586 rpc, wallet: Document mempool scan after importpubkey (Fabian Jahr)
6d3db52e66 rpc, wallet: Document and test mempool scan after importprivkey (João Barbosa)
3abdbbb90a rpc, wallet: Document and test mempool scan after importaddress (João Barbosa)
236239bd40 wallet: Rescan mempool for transactions as well (Fabian Jahr)

Pull request description:

  This PR picks up the work from #18964 and closes #18954.

  It should incorporate all the unaddressed feedback from the PR:
  - Mempool rescan now expanded to all relevant import* RPCs
  - Added documentation in the help of each RPC
  - More tests

ACKs for top commit:
  Sjors:
    re-utACK 1be7964189 (only a test change)
  achow101:
    ACK 1be7964189
  w0xlt:
    reACK 1be7964189

Tree-SHA512: b62fed5f97c6c242b2af417b41c9696a1f18878483d9e1c9429791f9c05257f57a00540a9a84df23c49faf6a61c3109c22972de81540083f38b506217804fcc5
2022-07-18 14:26:21 -04:00
glozow
821f5c824f
Merge bitcoin/bitcoin#25487: [kernel 3b/n] Decouple {Dump,Load}Mempool from ArgsManager
cb3e9a1e3f Move {Load,Dump}Mempool to kernel namespace (Carl Dong)
aa30676541 Move DEFAULT_PERSIST_MEMPOOL out of libbitcoinkernel (Carl Dong)
06b88ffb8a LoadMempool: Pass in load_path, stop using gArgs (Carl Dong)
b857ac60d9 test/fuzz: Invoke LoadMempool via CChainState (Carl Dong)
b3267258b0 Move FopenFn to fsbridge namespace (Carl Dong)
ae1e8e3756 mempool: Use NodeClock+friends for LoadMempool (Carl Dong)
f9e8e5719f mempool: Improve comments for [GS]etLoadTried (Carl Dong)
813962da0b scripted-diff: Rename m_is_loaded -> m_load_tried (Carl Dong)
413f4bb52b DumpMempool: Pass in dump_path, stop using gArgs (Carl Dong)
bd4407817e DumpMempool: Use std::chrono instead of weird int64_t arthmetics (Carl Dong)
c84390b741 test/mempool_persist: Test manual savemempool when -persistmempool=0 (Carl Dong)

Pull request description:

  This is part of the `libbitcoinkernel` project: #24303, https://github.com/bitcoin/bitcoin/projects/18

  -----

  This PR moves `{Dump,Load}Mempool` into its own `kernel/mempool_persist` module and introduces `ArgsManager` `node::` helpers in `node/mempool_persist_args`to remove the scattered calls to `GetBoolArg("-persistmempool", DEFAULT_PERSIST_MEMPOOL)`.

  More context can be gleaned from the commit messages.

  -----

  One thing I was reflecting on as I wrote this was that in the long run, I think we should probably invert the validation <-> mempool relationship. Instead of mempool not depending on validation, it might make more sense to have validation not depend on mempool. Not super urgent since `libbitcoinkernel` will include both validation and mempool, but perhaps something for the future.

ACKs for top commit:
  glozow:
    re ACK cb3e9a1e3f via `git range-diff 7ae032e...cb3e9a1`
  MarcoFalke:
    ACK cb3e9a1e3f 🔒
  ryanofsky:
    Code review ACK cb3e9a1e3f

Tree-SHA512: 979d7237c3abb5a1dd9b5ad3dbf3b954f906a6d8320ed7b923557f41a4472deccae3e8a6bca0018c8e7a3c4a93afecc502acd1e26756f2054f157f1c0edd939d
2022-07-18 16:09:27 +01:00
MacroFake
fae5ce8795
univalue: Return more detailed type check error messages 2022-07-18 11:31:36 +02:00
Andrew Chow
61d9149e78 rpc: Default rbf enabled 2022-07-15 11:46:34 -04:00
Andrew Chow
85b601e043
Merge bitcoin/bitcoin#24148: Miniscript support in Output Descriptors
ffc79b8e49 qa: functional test Miniscript watchonly support (Antoine Poinsot)
bfb036756a Miniscript support in output descriptors (Antoine Poinsot)
4a082887be qa: better error reporting on descriptor parsing error (Antoine Poinsot)
d25d58bf5f miniscript: add a helper to find the first insane sub with no child (Antoine Poinsot)
c38c7c5817 miniscript: don't check for top level validity at parsing time (Antoine Poinsot)

Pull request description:

  This adds Miniscript support for Output Descriptors without any signing logic (yet). See the OP of #24147 for a description of Miniscript and a rationale of having it in Bitcoin Core.
  On its own, this PR adds "watchonly" support for Miniscript descriptors in the descriptor wallet. A follow-up adds signing support.

  A minified corpus of Miniscript Descriptors for the `descriptor_parse` fuzz target is available at https://github.com/bitcoin-core/qa-assets/pull/92.
  The Miniscript descriptors used in the unit tests here and in #24149 were cross-tested against the Rust implementation at https://github.com/rust-bitcoin/rust-miniscript.

  This PR contains code and insights from Pieter Wuille.

ACKs for top commit:
  Sjors:
    re-utACK ffc79b8e49
  achow101:
    ACK ffc79b8e49
  w0xlt:
    reACK ffc79b8e49

Tree-SHA512: 02d919d38bb626d3c557eca3680ce71117739fa161b7a92cfdb6c9c432ed88870b1ed127ba24248574c40c7428217d7e9bdd986fd8cd7c51fae8c776e1271fb9
2022-07-14 14:54:19 -04:00
Antoine Poinsot
ffc79b8e49
qa: functional test Miniscript watchonly support 2022-07-14 12:11:44 +02:00
MacroFake
8efa73e7ce
Merge bitcoin/bitcoin#25557: p2p: Eliminate atomic for m_last_getheaders_timestamp
613e221149 test: remove unnecessary parens (Suhas Daftuar)
e939cf2b76 Remove atomic for m_last_getheaders_timestamp (Suhas Daftuar)

Pull request description:

  Eliminate the unnecessary atomic guarding `m_last_getheaders_timestamp`, which is only accessed in a single thread (thanks to MarcoFalke for pointing this out).

  Also address a nit that came up in #25454.

ACKs for top commit:
  MarcoFalke:
    review ACK 613e221149
  vasild:
    ACK 613e221149

Tree-SHA512: 6d6c473735b450b8ad43aae5cf16ed419154d72f4a05c0a6ce6f26caecab9db2361041398b70bf9395611c107d50897f501fa5fdbabb2891144bbc2b479dfdad
2022-07-14 09:55:44 +02:00
Andrew Chow
e3c33637ba wallet: Enable -walletrbf by default 2022-07-13 16:20:35 -04:00
Carl Dong
c84390b741 test/mempool_persist: Test manual savemempool when -persistmempool=0 2022-07-12 22:37:17 -04:00
Suhas Daftuar
613e221149 test: remove unnecessary parens 2022-07-12 13:38:14 -04:00
MacroFake
faace13b71
test: Remove duplicate MAX_BIP125_RBF_SEQUENCE constant 2022-07-12 18:51:18 +02:00
MacroFake
fa0404dbb7
scripted-diff: [test] Rename BIP125_SEQUENCE_NUMBER to MAX_BIP125_RBF_SEQUENCE
-BEGIN VERIFY SCRIPT-
 sed -i 's:BIP125_SEQUENCE_NUMBER:MAX_BIP125_RBF_SEQUENCE:g' $(git grep -l BIP125_SEQUENCE_NUMBER ./test)
-END VERIFY SCRIPT-
2022-07-12 18:49:08 +02:00
MacroFake
46fcb52cb1
Merge bitcoin/bitcoin#24944: rpc: add getblockfrompeer RPCTypeCheck and invalid input test coverage
2ef5294a5b rpc: add RPCTypeCheck for getblockfrompeer inputs (Jon Atack)
734b9669ff test: add getblockfrompeer coverage of invalid inputs (Jon Atack)

Pull request description:

  The new getblockfrompeer RPC lacks test coverage for invalid arguments, and its error messages are not harmonized with the existing RPCs.

  Fix all issues.

ACKs for top commit:
  brunoerg:
    ACK 2ef5294a5b

Tree-SHA512: 454782cf6a44fd0e05483bb152153667ef5c8021358385ddcf89724fbbbd35e187362bdff757e00c99319527bc4c0b20c7187f67241d4585d767a29787142f25
2022-07-12 17:28:26 +02:00
MacroFake
01ae8d9cd2
Merge bitcoin/bitcoin#25592: test persistence of non-mempool tx prioritisation
a9790ba95f [test] persist prioritisation of transactions not in mempool (glozow)

Pull request description:

  We persist tx prioritisation/fee deltas in mempool.dat (see `DumpMempool`). It seems we have test coverage for persistence of modified fees of mempool entries (see `vinfo` loop), but not for the prioritisation of transactions not in mempool (see `mapDeltas`).

  Relevant: https://github.com/bitcoin/bitcoin/pull/25487#discussion_r917490221

ACKs for top commit:
  darosior:
    utACK a9790ba95f
  w0xlt:
    ACK a9790ba95f

Tree-SHA512: 3f2769a917041f12414584f69b2239eef57586f9975869e826f356633fcaf893fde15500619b302e7663de14f3661c6cba22c7524cab5286e715e2c105726521
2022-07-12 17:08:36 +02:00
glozow
39d111aee7
Merge bitcoin/bitcoin#25575: Address comments remaining from #25353
1056bbdfcd Address comments remaining from #25353 (Antoine Riard)

Pull request description:

  This PR should address the remaining comments from #25353.

ACKs for top commit:
  MarcoFalke:
    cr ACK 1056bbdfcd
  glozow:
    ACK 1056bbdfcd
  w0xlt:
    cr ACK 1056bbdfcd

Tree-SHA512: 194524193b1f087742c04d3cbe221e2ccf62e1f9303dc6668d62b73bd2dc0c039b7d68b33658dbee7809bd14bb8a5479f8e7928180b18c3180fdfbe3876c3ca1
2022-07-12 15:58:39 +01:00
glozow
a9790ba95f
[test] persist prioritisation of transactions not in mempool 2022-07-12 10:37:13 +01:00
Antoine Riard
1056bbdfcd Address comments remaining from #25353 2022-07-11 18:48:26 -04:00
Andrew Chow
d2ed97656b wallet: Precompute Txdata after setting PSBT inputs' UTXOs
If we are given a PSBT that is missing one or more input UTXOs, our
PrecomputedTransactionData will be incorrect and missing information
that it should otherwise have, and therefore we may not produce a
signature when we should. To avoid this problem, we can do the
precomputation after we have set the UTXOs the wallet is able to set for
the PSBT.

Also adds a test for this behavior.
2022-07-11 18:08:32 -04:00
furszy
76a84c0a6c
test: speedup wallet_coinbase_category.py
No need to create a chain for it (nor use the cache).
2022-07-11 15:13:32 -03:00
MacroFake
0817cc379f
Merge bitcoin/bitcoin#25512: test: remove wallet dependency and refactor rpc_signrawtransaction.py
0ee43d13e9 test: refactor rpc_signrawtransaction.py (Ayush Sharma)

Pull request description:

  `rpc_signrawtransaction.py` currently tests the `signrawtransactionwithkey` and `signrawtransactionwithwallet` RPCs.

  This PR splits `rpc_signrawtransaction.py` into

  1. `rpc_signrawtransactionwithkey.py`: the tests for `signrawtransactionwithkey` are moved here and this test can now be run with the wallet disabled.
  2.  `wallet_signrawtransactionwithwallet.py`: wallet only tests for `signrawtransactionwithwallet.py`

ACKs for top commit:
  aureleoules:
    tACK 0ee43d13e9.

Tree-SHA512: c7bd2ea746345c978eae231a781fc52953b9d277eb9f8abb9c3270fe1d9f678f23f3784377d7303a2aa23d7ad90097245e517d386b27b4e0016585dfddcb9d49
2022-07-11 15:33:18 +02:00
Sebastian Falbesoner
6cbe65c5d7 test: refactor: pass absolute fee in create_lots_of_big_transactions helper 2022-07-10 13:09:51 +02:00
Andrew Chow
b9f9ed4640
Merge bitcoin/bitcoin#25337: refactor: encapsulate wallet's address book access
d69045e291 test: add coverage for 'listreceivedbyaddress' no change addrs return (furszy)
324f00a642 refactor: 'ListReceived' use optional for filtered address (furszy)
b459fc122f refactor: RPC 'ListReceived', encapsulate m_address_book access (furszy)
fa9f2ab8fd refactor: RPC 'listlabels', encapsulate 'CWallet::ListAddrBookLabels' functionality (furszy)
83e42c4b94 refactor: use 'ForEachAddrBookEntry' in RPC 'getaddressesbylabel' (furszy)
2b48642499 refactor: use ForEachAddrBookEntry in interfaces::getAddresses (furszy)
032842ae41 wallet: implement ForEachAddrBookEntry method (furszy)
09649bc95d refactor: implement general 'ListAddrBookAddresses' for addressbook destinations lookup (furszy)
192eb1e61c refactor: getAddress don't access m_address_book, use FindAddressEntry function (furszy)

Pull request description:

  ### Context

  The wallet's `m_address_book` field is being accessed directly from several places across the sources.

  ### Problem

  Code structure wise, we shouldn't be accessing it directly. It could end up being modified by mistake (from a place that has nothing to do with the wallet like an RPC command or the GUI) and cause a bigger issue: like an address book entry 'purpose' string change, which if done badly (from 'send' to 'receive'), could end up in a user sharing a "receive" address that he/she doesn't own.

  ### Solution

  Encapsulate `m_address_book` access inside the wallet.

  -------------------------------------------------------

  Extra Note:

  This is the initial step towards decoupling the address book functionality from the wallet's sources. In other words, the creation of the `AddressBookManager` (which will be coming in a follow-up PR).

ACKs for top commit:
  achow101:
    ACK d69045e291
  theStack:
    ACK d69045e291 
  w0xlt:
    ACK d69045e291

Tree-SHA512: dba17acd86f171b4e9af0223bbbcad380048570f6a2f6a92732a51f01abe8806debaf65c9e9e5569fa76a541903cbb50adcb5f56ef77858151c698ae6b218e2a
2022-07-08 10:16:08 -04:00
MacroFake
a7f3479ba3
Merge bitcoin/bitcoin#25353: Add a -mempoolfullrbf node setting
4c9666bd73 Mention `mempoolfullrbf` in policy/mempool-replacements.md (Antoine Riard)
aae66ab43d Update getmempoolinfo RPC with `mempoolfullrbf` (Antoine Riard)
3e27e31727 Introduce `mempoolfullrbf` node setting. (Antoine Riard)

Pull request description:

  This is ready for review.

  Recent discussions among LN devs have brought back on the surface concerns about the security of multi-party funded transactions against pinnings attacks and other mempool-based nuisances. The lack of full-rbf transaction-relay topology connected to miners open the way to cheap and naive DoS against multi-party funded transactions (e.g coinjoins, dual-funded channels, on-chain DLCs, ...) without solutions introducing an overhead cost or centralization vectors afaik . For more details, see [0].

  This PR implements a simple `fullrbf` setting, where the node always allows transaction replacement, ignoring BIP125 opt-in flag. The default value of the setting stays **false**, therefore opt-in replacement is still the default Bitcoin Core replacement policy. Contrary to a previous proposal of mine and listening to feedbacks collected since then [1], I think this new setting simply offers more flexibility in a node transaction-relay policy suiting one's application requirements, without arguing a change of the default behavior.

  I [posted](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2022-June/020557.html) on the ML to invite operators with a bitcoin application sensitive to full-rbf (e.g dual-funded LN channels service providers) or mempool researchers to join a bootstrapped full-rbf activated peers network for experimentation and learning. If people have strong opinions against the existence of such full-rbf transaction-relay network, I'm proposing to express them on the future thread.

  [0] https://lists.linuxfoundation.org/pipermail/lightning-dev/2021-May/003033.html
  [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-June/019074.html

  Follow-up suggestions :
  - soft-enable opt-in RBF in the wallet : https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1154918789
  - p2p discovery and additional outbound connection to full-rbf peers : https://github.com/bitcoin/bitcoin/pull/25353#issuecomment-1156044401
  - match the code between RPC, wallet and mempool about disregard of inherited signaling : #22698

ACKs for top commit:
  instagibbs:
    reACK 4c9666bd73
  glozow:
    ACK 4c9666bd73, a few nits which are non-blocking.
  w0xlt:
    ACK 4c9666bd73

Tree-SHA512: 9e288bf22e06a9808804e58178444ef1830c3fdd42fd8a7cd7ffb101f8f586e08b000679be407d63ca76a56f7216227b368ff630c81f3fac3243db1a1202ab1c
2022-07-08 11:06:24 +02:00
MacroFake
8ef096d4f8
Merge bitcoin/bitcoin#25522: test: Remove -acceptnonstdtxn=1 from feature_rbf.py
fad690ba0a test: Remove -acceptnonstdtxn=1 from feature_rbf.py (MacroFake)
fa5059b7df test: Make the scriptPubKey of MiniWallet created txs mutable (MacroFake)
fa29245827 test: Allow setting sequence per input in MiniWallet create_self_transfer_multi (MacroFake)
fac3800d2c test: Allow amount_per_output in MiniWallet create_self_transfer_multi (MacroFake)
2222842ae7 test: Allow absolute fee in MiniWallet create_self_transfer (MacroFake)

Pull request description:

  On the main network, nonstandard transactions are hardly relayed, so it seems odd that one of our policy test requires a policy setting opposite of the norm.

  Surely it is also important to test that nonstandard transactions can be replaced. However, rbf code should not care about the standardness at all. Moreover, I think testing nonstandardness rbf is of lower priority than testing the stuff that actually happens in production.

ACKs for top commit:
  theStack:
    re-ACK fad690ba0a
  jamesob:
    ACK fad690ba0a ([`jamesob/ackr/25522.1.MarcoFalke.test_remove_acceptnonstd`](https://github.com/jamesob/bitcoin/tree/ackr/25522.1.MarcoFalke.test_remove_acceptnonstd))

Tree-SHA512: e0a0c808bccdddf738fb6a84e5e5672d7c341bffd941c4f0c232112bfc68265fa81a2e42ddcab107d586358ffdb3dccc46bb5533d46999fd9ab024169dac0f78
2022-07-07 16:12:12 +02:00
MacroFake
67c6b61f96
Merge bitcoin/bitcoin#25525: test: remove wallet dependency from mempool_updatefromblock.py
eac1099e00 test: remove wallet dependency  from mempool_updatefromblock.py (Ayush Sharma)

Pull request description:

  This PR enables one of the non-wallet functional tests(`mempool_updatefromblock.py`) to be run with the wallet disabled.

ACKs for top commit:
  aureleoules:
    tACK eac1099e00.

Tree-SHA512: 9734815f2d2e65e8813bd776cf1d847a55ba4181e218c5e7b066ec69a556261069214f675681d672f5d7b0ba8e06342c4a456619dcc005cbf5618a0527303b7f
2022-07-07 13:55:43 +02:00
Antoine Riard
3e27e31727 Introduce mempoolfullrbf node setting.
This new node policy setting enables to accept replaced-by-fee
transaction without inspection of the replaceability signaling
as described in BIP125 "explicit signaling".

If turns on, the node mempool accepts transaction replacement
as described in `policy/mempool-replacements.md`.

The default setting value is `false`, implying opt-in RBF
is enforced.
2022-07-06 20:57:29 -04:00
MacroFake
fad690ba0a
test: Remove -acceptnonstdtxn=1 from feature_rbf.py 2022-07-06 10:21:44 +02:00
MacroFake
195e07feaf
Merge bitcoin/bitcoin#19393: test: Add more tests for orphan tx handling
c0a5fceee9 test: Add test for erase orphan tx conflicted by block (Hennadii Stepanov)
fa45bb2119 test: Add test for erase orphan tx included by block (Hennadii Stepanov)
5c049780c8 test: Add test for erase orphan tx from peer (Hennadii Stepanov)

Pull request description:

  This PR adds test coverage for the following cases:
  - erase orphan transactions when a peer is disconnected
  - erase an orphan transaction when it is included in a new tip block
  - erase an orphan transaction when it is conflicted with other transactions included in a new tip block

  Found useful while working on #19374.

ACKs for top commit:
  aureleoules:
    tACK c0a5fceee9 (`make check` and `test/functional/test_runner.py`).
  kouloumos:
    ACK c0a5fceee9 with a nit per https://github.com/bitcoin/bitcoin/pull/19393#discussion_r899156623.
  pg156:
    Reviewed to c0a5fceee9. Concept ACK. Agree due to the lack of RPC calls to inspect orphan pool, using `assert_debug_log` to match strings in log is a reasonable way to test.

Tree-SHA512: 98f8deeee2d1c588c7e28a82e513d4a18655084198369db33fe2710458251eeaffed030626940072d7576f57fcbf7d856d761990129e2ca9e372d2ccbd86d07d
2022-07-05 18:55:56 +02:00
fanquake
87d012324a
Merge bitcoin/bitcoin#25454: p2p: Avoid multiple getheaders messages in flight to the same peer
99f4785cad Replace GetTime() with NodeClock in MaybeSendGetHeaders() (Suhas Daftuar)
abf5d16c24 Don't send getheaders message when another request is outstanding (Suhas Daftuar)
ffe87db247 Cleanup received_new_header calculation to use WITH_LOCK (Suhas Daftuar)
6d95cd3e74 Move peer state updates from headers message into separate function (Suhas Daftuar)
2b341db731 Move headers direct fetch to end of ProcessHeadersMessage (Suhas Daftuar)
29c4518522 Move headers-direct-fetch logic into own function (Suhas Daftuar)
bf8ea6df75 Move additional headers fetching to own function (Suhas Daftuar)
9492e93bf9 Add helper function for checking header continuity (Suhas Daftuar)
7f2450871b Move handling of unconnecting headers into own function (Suhas Daftuar)

Pull request description:

  Change `getheaders` messages so that we wait up to 2 minutes for a response to a prior `getheaders` message before issuing a new one.

  Also change the handling of the `getheaders` message sent in response to a block INV, so that we no longer use the hashstop variable (including the hash stop will just mean that if our peer's headers chain is longer, then we won't learn
  it, so there's no benefit to using hashstop).

  Also, now respond to a `getheaders` during IBD with an empty headers message (rather than nothing) -- this better conforms to the intent of the new logic that it's better to not ignore a peer's `getheaders` message, even if you have nothing to give. This also avoids a lot of functional tests breaking.

  This PR also reworks the headers processing logic to make it more readable.

ACKs for top commit:
  ajtowns:
    ACK 99f4785cad ; code review, check over new logic of when to send getheaders messages
  dergoegge:
    Code review ACK  99f4785cad
  mzumsande:
    Code Review ACK 99f4785cad
  sipa:
    utACK 99f4785cad
  w0xlt:
    tACK 99f4785cad Good improvement in the code.

Tree-SHA512: b8a63f6f71ac83e292edc0200def7835ad8b06b2955dd34e3ea6fac85980fa6962efd31d689ef5ea121ff5477ec14aafa4bbe2d0db134c05f4a31a57a8ced365
2022-07-04 21:28:21 +01:00
Fabian Jahr
1be7964189
test, wallet: Add mempool rescan test for import RPCs 2022-07-03 21:06:49 +02:00
João Barbosa
6d3db52e66
rpc, wallet: Document and test mempool scan after importprivkey
co-authored-by: Fabian Jahr <fjahr@protonmail.com>
2022-07-03 21:06:49 +02:00
João Barbosa
3abdbbb90a
rpc, wallet: Document and test mempool scan after importaddress
co-authored-by: Fabian Jahr <fjahr@protonmail.com>
2022-07-03 21:06:49 +02:00
Fabian Jahr
236239bd40
wallet: Rescan mempool for transactions as well 2022-07-03 21:06:47 +02:00
Sebastian Falbesoner
1770be72d5 test: pass dustrelayfee=0 option for tests using dust (instead of acceptnonstdtxn=1)
By specifying the `dustrelayfee=0` option instead of the more
generic `acceptnonstdtxn=1`, we can be more specific about what
part of the transaction is non-standard and can be sure that all
other aspects follow the standard policy.
2022-07-03 16:30:11 +02:00
0xb10c
dba6f82342
test: adopt USDT utxocache interface tests
The USDT interface exposes process internals via the tracepoints. This
means, the USDT interface tests somewhat awardly depend on these
internals. If internals change, the tests have to adopt to that change.
Previously, the USDT interface tests weren't run in the CI so changes
could break the USDT interface tests without being noticed (e.g.
https://github.com/bitcoin/bitcoin/pull/25486).

In fa13375aa3 a 'self.rescan_utxos()' call
was added in the 'generate()' function of the test framework.
'rescan_utxos()' causes the UTXO cache to be flushed. In the USDT
interface tests for the 'utxocache:flush' trancepoint, 'generate()' is
used. As the utxo cache is now flushed more often, the number of flushes
the tests expectes need to be adopted. Also, the utxo cache has now a
different size when being flushed.

The utxocache tracepoint is tested by shutting the node down and
pruning blocks, to test the 'for_prune' argument.

Changes:
- A list 'expected_flushes' is now used which contains 'mode',
  'for_prune', and 'size' for each expected flush.
- When a flush happens, the expected-flush is removed from the list.
  This list is checked to be empty (unchanged).
- Previously, shutting down caused these two flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=104, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  now it causes these flushes:
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=2, memory=*, for_prune=False)
    UTXOCacheFlush(duration=*, mode=ALWAYS, size=0, memory=*, for_prune=False)
  The 104 UTXOs flushed previously were mainly coinbase UTXOs generated
  in previous tests and the test setup. These are now already flushed.
- In the 'for_prune' test we previously hooked into the tracepoint
  before mining blocks. This changed to only get notified about the
  tracepoint being triggered for the prune. Here, the utxo cache is
  empty already as it has just been flushed in 'generate()'.
  old:
    UTXOCacheFlush(duration=*, mode=NONE, size=350, memory=*, for_prune=True)
  new:
    UTXOCacheFlush(duration=*, mode=NONE, size=0, memory=*, for_prune=True)
2022-07-02 14:37:32 +02:00
0xb10c
220a5a2841
test: hook into PID in tracing tests
This makes sure to NOT hook into other bitcoind binaries run in
paralell in the test framework. We only want to trace the intended
binary.

In interface_usdt_utxocache.py:
While testing the utxocache flush with pruning, bitcoind is
restarted and we need to hook into the new PID again.
2022-07-02 14:37:29 +02:00
Ayush Sharma
eac1099e00 test: remove wallet dependency from mempool_updatefromblock.py
This functional  test can now be run with the wallet disabled.
2022-07-01 19:27:34 +05:30
MacroFake
fa5059b7df
test: Make the scriptPubKey of MiniWallet created txs mutable
This makes individual bytes of the scriptPubKey mutable, previously it
could only be re-assigned as a whole.
2022-07-01 12:29:23 +02:00
MacroFake
fa29245827
test: Allow setting sequence per input in MiniWallet create_self_transfer_multi
Previously it was only possible to set the same sequence in all inputs
2022-07-01 12:29:22 +02:00
MacroFake
fac3800d2c
test: Allow amount_per_output in MiniWallet create_self_transfer_multi 2022-07-01 12:29:14 +02:00
MacroFake
2222842ae7
test: Allow absolute fee in MiniWallet create_self_transfer 2022-07-01 12:29:00 +02:00
MacroFake
5d68d6840d
Merge bitcoin/bitcoin#25364: test: remove wallet dependency from feature_nulldummy.py
50ba6697f3 remove unused functions (Ayush Sharma)
eec23dad1e test: remove wallet dependency from feature_nulldummy.py (Ayush Sharma)

Pull request description:

  This PR enables one of the non-wallet functional tests (`feature_nulldummy.py`) to be run even with the Bitcoin Core wallet disabled.

  Commit 1: removes wallet dependency and `test_runner.py` is edited to make sure the test only runs once.
  Commit 2: the functions `create_transaction()` and `create_raw_transaction()` in `blocktools.py` are no longer needed and hence removed.

ACKs for top commit:
  kouloumos:
    re-ACK 50ba6697f3, all comments have been addressed.

Tree-SHA512: 3bc3d2766e53dba3d56a03f2c476442608ac693f51d84f4632a22a2cf169bc02c10bf92b676f7d57acb4f0ad86f307d37ab63f936b44b3585ee3c9d08cd0335f
2022-06-30 17:39:45 +02:00
fanquake
6adae27f8c
Merge bitcoin/bitcoin#24836: add RPC (-regtest only) for testing package policy
e866f0d066 [functional test] submitrawpackage RPC (glozow)
fa076515b0 [rpc] add new submitpackage RPC (glozow)

Pull request description:

  It would be nice for LN/wallet/app devs to test out package policy, package RBF, etc., but the only interface to do so right now is through unit tests. This PR adds a `-regtest` only RPC interface so people can test by submitting raw transaction data. It is regtest-only, as it would be unsafe/confusing to create an actual mainnet interface while package relay doesn't exist.

  Note that the functional tests are there to ensure the RPC interface is working properly; they aren't for testing policy itself. See src/test/txpackage_tests.cpp.

ACKs for top commit:
  t-bast:
    Tested ACK against eclair e866f0d066
  ariard:
    Code Review ACK e866f0d0
  instagibbs:
    code review ACK e866f0d066

Tree-SHA512: 824a26b10d2240e0fd85e5dd25bf499ee3dd9ba8ef4f522533998fcf767ddded9f001f7a005fe3ab07ec95e696448484e26599803e6034ed2733125c8c376c84
2022-06-30 15:43:50 +01:00
MacroFake
1ee597817f
Merge bitcoin/bitcoin#25503: test: pass datacarriersize option for tests using large outputs (instead of acceptnonstdtxn)
475aae846e test: pass `datacarriersize` option for tests using large outputs (instead of `acceptnonstdtxn`) (Sebastian Falbesoner)
b1ba3ed155 test: let `gen_return_txouts` create a single large OP_RETURN output (Sebastian Falbesoner)
f319287d81 test: assert serialized txouts size of `gen_return_txouts` helper (Sebastian Falbesoner)

Pull request description:

  By specifying the `datacarriersize` option instead of the more generic `acceptnonstdtxn` for functional tests, we can be more specific about what part of the transaction is non-standard and can be sure that all other aspects follow the standard policy. Transactions with more than one OP_RETURN output are [never considered standard](749b80b29e/src/policy/policy.cpp (L149-L153)), i.e. we have to change the `gen_return_txouts` helper to create only a single output in order to get rid of the `acceptnonstdxtn` option. Note that on master there is currently no test using the `datacarriersize` parameter, so this PR indirectly also increases the test coverage.

  The change affects the tests `mempool_limit.py`, `mining_prioritisetransaction.py` (call `gen_return_txouts` directly) and `feature_maxuploadtarget.py` (calls `gen_return_txouts` indirectly via the `mine_large_block(...)` helper).

Top commit has no ACKs.

Tree-SHA512: c17f032e00d28f5e5880a4d378773fbc8b1995ea9c377f237598d412628fe117f497a44ebdfa8af8cd8a3b1e3127e0cf7692efbf5c833c713764a71a85301f23
2022-06-30 16:37:49 +02:00
Ayush Sharma
0ee43d13e9 test: refactor rpc_signrawtransaction.py
rpc_signrawtransaction.py is split into rpc_signrawtransactionwithkey.py and wallet_signrawtransactionwithwallet.py.
rpc_signrawtransactionwithkey.py can be run with the wallet disabled.
2022-06-30 19:12:01 +05:30
brunoerg
d22bd543cc test: passing a non-positive integer value to -peertimeout should throw an error 2022-06-30 10:18:36 -03:00
Sebastian Falbesoner
475aae846e test: pass datacarriersize option for tests using large outputs (instead of acceptnonstdtxn)
By specifying the `datacarriersize` option instead of the more
generic `acceptnonstdtxn`, we can be more specific about what
part of the transaction is non-standard and can be sure that all
other aspects follow the standard policy.
2022-06-29 18:05:59 +02:00
Sebastian Falbesoner
b1ba3ed155 test: let gen_return_txouts create a single large OP_RETURN output
Transactions with more than one datacarrier (OP_RETURN) output
are never considered standard, i.e. this change is necessary in
order to to get rid of the `acceptnonstdtxn` option for some
tests.
2022-06-29 17:42:51 +02:00
Sebastian Falbesoner
f319287d81 test: assert serialized txouts size of gen_return_txouts helper
This assures that changing the internals of the helper function
still leads to the expected outcome sizewise (preparation for the
next commit).
2022-06-29 17:28:33 +02:00
MacroFake
6666803c89
streams: Add AutoFile without ser-type and ser-version
The moved parts can be reviewed with "--color-moved=dimmed-zebra".
The one-char changes can be reviewed with "--word-diff-regex=.".
2022-06-29 10:31:53 +02:00
MacroFake
e4e201dfd9
Merge bitcoin/bitcoin#25290: [kernel 3a/n] Decouple CTxMemPool from ArgsManager
d1684beabe fees: Pass in a filepath instead of referencing gArgs (Carl Dong)
9a3d825c30 init: Remove redundant -*mempool*, -limit* queries (Carl Dong)
6c5c60c412 mempool: Use m_limit for UpdateTransactionsFromBlock (Carl Dong)
9e93b10301 node/ifaces: Use existing MemPoolLimits (Carl Dong)
38af2bcf35 mempoolaccept: Use limits from mempool in constructor (Carl Dong)
9333427014 mempool: Introduce (still-unused) MemPoolLimits (Carl Dong)
716bb5fbd3 scripted-diff: Rename anc/desc size limit vars to indicate SI unit (Carl Dong)
1ecc77321d scripted-diff: Rename DEFAULT_MEMPOOL_EXPIRY to indicate time unit (Carl Dong)
aa9141cd81 mempool: Pass in -mempoolexpiry instead of referencing gArgs (Carl Dong)
51c7a41a5e init: Only determine maxmempool once (Carl Dong)
386c9472c8 mempool: Make GetMinFee() with custom size protected (Carl Dong)
82f00de7a6 mempool: Pass in -maxmempool instead of referencing gArgs (Carl Dong)
f1941e8bfd pool: Add and use MemPoolOptions, ApplyArgsManOptions (Carl Dong)
0199bd35bb fuzz/rbf: Add missing TestingSetup (Carl Dong)
ccbaf546a6 scripted-diff: Rename DEFAULT_MAX_MEMPOOL_SIZE to indicate SI unit (Carl Dong)
fc02f77ca6 ArgsMan: Add Get*Arg functions returning optional (Carl Dong)

Pull request description:

  This is part of the `libbitcoinkernel` project: #24303, https://github.com/bitcoin/bitcoin/projects/18

  -----

  As mentioned in the Stage 1 Step 2 description of [the `libbitcoinkernel` project](https://github.com/bitcoin/bitcoin/issues/24303), `ArgsManager` will not be part of `libbitcoinkernel`. Therefore, it is important that we remove any dependence on `ArgsManager` by code that will be part of `libbitcoinkernel`. This is the first in a series of PRs aiming to achieve this.

  This PR removes `CTxMemPool+MempoolAccept`'s dependency on `ArgsManager` by introducing a `CTxMemPool::Options` struct, which is used to specify `CTxMemPool`'s various options at construction time.

  These options are:
  - `-maxmempool` -> `CTxMemPool::Options::max_size`
  - `-mempoolexpiry` -> `CTxMemPool::Options::expiry`
  - `-limitancestorcount` -> `CTxMemPool::Options::limits::ancestor_count`
  - `-limitancestorsize` -> `CTxMemPool::Options::limits::ancestor_size`
  - `-limitdescendantcount` -> `CTxMemPool::Options::limits::descendant_count`
  - `-limitdescendantsize` -> `CTxMemPool::Options::limits::descendant_size`

  More context can be gleaned from the commit messages. The important commits are:

  - 56eb479ded8bfb2ef635bb6f3b484f9d5952c70d "pool: Add and use MemPoolOptions, ApplyArgsManOptions"
  - a1e08b70f3068f4e8def1c630d8f50cd54da7832 "mempool: Pass in -maxmempool instead of referencing gArgs"
  - 6f4bf3ede5812b374828f08fc728ceded2f10024 "mempool: Pass in -mempoolexpiry instead of referencing gArgs"
  - 5958a7fe4806599fc620ee8c1a881ca10fa2dd16 "mempool: Introduce (still-unused) MemPoolLimits"

  Reviewers: Help needed in the following commits (see commit messages):
  - a1e08b70f3068f4e8def1c630d8f50cd54da7832 "mempool: Pass in -maxmempool instead of referencing gArgs"
  - 0695081a797e9a5d7787b78b0f8289dafcc6bff7 "node/ifaces: Use existing MemPoolLimits"

  Note to Reviewers: There are perhaps an infinite number of ways to architect `CTxMemPool::Options`, the current one tries to keep it simple, usable, and flexible. I hope we don't spend too much time arguing over the design here since that's not the point. In the case that you're 100% certain that a different design is strictly better than this one in every regard, please show us a fully-implemented branch.

  -----

  TODO:
  - [x] Use the more ergonomic `CTxMemPool::Options` where appropriate
  - [x] Doxygen comments for `ApplyArgsManOptions`, `MemPoolOptions`

  -----

  Questions for Reviewers:
  1. Should we use `std::chrono::seconds` for `CTxMemPool::Options::expiry` and `CTxMemPool::m_expiry` instead of an `int64_t`? Something else? (`std::chrono::hours`?)
  2. Should I merge `CTxMemPool::Limits` inside `CTxMemPool::Options`?

ACKs for top commit:
  MarcoFalke:
    ACK d1684beabe 🍜
  ryanofsky:
    Code review ACK d1684beabe. Just minor cleanups since last review, mostly switching to brace initialization

Tree-SHA512: 2c138e52d69f61c263f1c3648f01c801338a8f576762c815f478ef5148b8b2f51e91ded5c1be915e678c0b14f6cfba894b82afec58d999d39a7bb7c914736e0b
2022-06-29 09:13:31 +02:00
Suhas Daftuar
abf5d16c24 Don't send getheaders message when another request is outstanding
Change getheaders messages so that we wait up to 2 minutes for a response to a
prior getheaders message before issuing a new one.

Also change the handling of the getheaders message sent in response to a block
INV, so that we no longer use the hashstop variable (including the hash stop
will just mean that if our peer's headers chain is longer, then we won't learn
it, so there's no benefit to using hashstop).

Also, now respond to a getheaders during IBD with an empty headers message
(rather than nothing) -- this better conforms to the intent of the new logic
that it's better to not ignore a peer's getheaders message, even if you have
nothing to give. This also avoids a lot of functional tests breaking.

p2p_segwit.py is modified to use this same strategy, as the test logic (of
expecting a getheaders after a block inv) would otherwise be broken.
2022-06-28 15:53:25 -04:00
Carl Dong
1ecc77321d scripted-diff: Rename DEFAULT_MEMPOOL_EXPIRY to indicate time unit
Better to be explicit when it comes to time to avoid unintentional bugs.

-BEGIN VERIFY SCRIPT-
find_regex="DEFAULT_MEMPOOL_EXPIRY" \
    && git grep -l -E "$find_regex" \
        | xargs sed -i -E "s@$find_regex@\0_HOURS@g"
-END VERIFY SCRIPT-
2022-06-28 15:42:40 -04:00
Sjors Provoost
796b020c37
wallet: add taproot support to external signer 2022-06-28 17:15:25 +02:00
laanwj
5bf65ec66e
Merge bitcoin/bitcoin#22558: psbt: Taproot fields for PSBT
b80de4c505 test: Test signing psbts without explicitly having scripts (Andrew Chow)
a73b56888a wallet: also search taproot pubkeys in FillPSBT (Andrew Chow)
6cff82722f sign: Use sigdata taproot spenddata when signing (Andrew Chow)
5f12fe3f36 psbt: Implement merge for Taproot fields (Andrew Chow)
1ece9a3715 psbt, test: Check for taproot fields in taproot psbt test (Andrew Chow)
496a1bbe5e taproot: Use pre-existing signatures if available (Andrew Chow)
0ad21e7c55 tests: Test taproot fields for PSBT (Andrew Chow)
103c6fd279 psbt: Remove non_witness_utxo for segwit v1+ (Andrew Chow)
7dccdd3157 Implement decodepsbt for Taproot fields (Andrew Chow)
ac7747585f Fill PSBT Taproot output data to/from SignatureData (Andrew Chow)
25b6ae46e7 Assert that TaprootBuilder is Finalized during GetSpendData (Andrew Chow)
3ae5b6af21 Store TaprootBuilder in SigningProviders instead of TaprootSpendData (Andrew Chow)
4d1223e512 Fetch key origins for Taproot keys (Andrew Chow)
52e3f2f88e Fill PSBT Taproot input data to/from SignatureData (Andrew Chow)
05e2cc9a30 Implement de/ser of PSBT's Taproot fields (Andrew Chow)
d557eff2ad Add serialization methods to XOnlyPubKey (Andrew Chow)
d43923c381 Add TaprootBuilder::GetTreeTuples (Andrew Chow)
ce911204e4 Move individual KeyOriginInfo de/ser to separate function (Andrew Chow)

Pull request description:

  Implements the Taproot fields for PSBT described in [BIP 371](https://github.com/bitcoin/bips/blob/master/bip-0371.mediawiki).

ACKs for top commit:
  laanwj:
    Code review ACK b80de4c505

Tree-SHA512: 50b79bb44f353c9ec2ef4c98aac08a81eba560987e5264a5684caa370e9c4e7a8255c06747fc47749511be45b32d01492e015f92b82be8d22bc8bf192073bd26
2022-06-28 16:44:03 +02:00
Andrew Chow
b80de4c505 test: Test signing psbts without explicitly having scripts 2022-06-27 16:48:04 -04:00
Andrew Chow
1ece9a3715 psbt, test: Check for taproot fields in taproot psbt test 2022-06-27 16:47:48 -04:00
Andrew Chow
496a1bbe5e taproot: Use pre-existing signatures if available
Actually use pre-existing signatures in CreateTaprootScriptSig if a
signature is found for the given key and leaf hash.
2022-06-27 16:47:48 -04:00
Andrew Chow
0ad21e7c55 tests: Test taproot fields for PSBT 2022-06-27 16:47:48 -04:00
Sebastian Falbesoner
f665c6ecda test: fix failing test interface_usdt_utxocache.py
The `from_node` argument doesn't exist anymore for
`MiniWallet.create_self_transfer` since PR #25435 (commit
fa8421bc5b).
2022-06-27 19:40:28 +02:00
Jon Atack
2ef5294a5b rpc: add RPCTypeCheck for getblockfrompeer inputs 2022-06-27 13:03:24 +02:00
MacroFake
fa83c0c44f
test: Remove unused call to generate in rpc_mempool_info
There are already enough blocks
2022-06-27 11:09:01 +02:00
MacroFake
fa13375aa3
test: Sync MiniWallet utxo state after each generate call 2022-06-27 11:08:50 +02:00
MacroFake
dddd7c4d39
test: Drop spent utxos in MiniWallet scan_tx 2022-06-27 11:08:29 +02:00
MacroFake
fa04ff61b6
test: Return new_utxos from create_self_transfer_multi in MiniWallet 2022-06-27 11:07:34 +02:00
MacroFake
fa34e44e98
test: Return new_utxo from create_self_transfer in MiniWallet 2022-06-27 11:07:29 +02:00
MacroFake
f52d074363
Merge bitcoin/bitcoin#25439: rpc: Return incrementalrelayfee in getmempoolinfo
fafee78188 rpc: Return incrementalrelayfee in getmempoolinfo (MacroFake)

Pull request description:

  Seems odd to return other policy info, but not the incremental relay fee

ACKs for top commit:
  1440000bytes:
    ACK fafee78188
  w0xlt:
    Code Review ACK fafee78188
  jarolrod:
    tACK fafee78188

Tree-SHA512: faad0af6c039b8257acbeac913bc5dcdb2ea2db304c95e52601536c8de60eb1186e9fbb4a64a68adf476605f18022aeda16a5644a0d7912592b0977e4c029638
2022-06-27 08:19:14 +02:00
MacroFake
c1acd34984
Merge bitcoin/bitcoin#25476: test: Remove unnecessary mining from importdescriptors test
e3d8d72703 test: Remove unnecessary block mining from importdescriptors test (Fabian Jahr)

Pull request description:

  This removes generation of 6 blocks and replaces is with a `sync_all` in the `importdescriptors` test.

  The generated blocks themself don't seem to serve any purpose in the test. Instead they could make the test flaky (although I did not find open issues pointing to this happening in practice in the CI). Right before the blocks being generated a transaction is created (L454) and later in the test this tx is assumed to be still in the mempool. If the nodes were to sync their mempools before the blocks are generated, the test fails. It currently only seems to work because one node sends the tx while the other generates the blocks and the mempools are not synced fast enough.

  The `sync_all` is still needed to let nodes catch up at that point. Otherwise races happen further below which the generate call seems to have prevented so far.

ACKs for top commit:
  laanwj:
    Code review ACK e3d8d72703

Tree-SHA512: 14f3dc2938d779d1ad43e09a7e046523fc3c92f41df012833f279a2e88e74c2fcab301fe4f3fcc038bd8460ea1360725a8d1eb5b59acd1039495bacb484fd790
2022-06-27 08:16:28 +02:00
Fabian Jahr
e3d8d72703
test: Remove unnecessary block mining from importdescriptors test 2022-06-26 16:34:11 +02:00
Ayush Sharma
50ba6697f3 remove unused functions
the functions `create_transaction()` and `create_raw_transaction()` were no longer used hence removed.
2022-06-24 18:04:48 +05:30
Ayush Sharma
eec23dad1e test: remove wallet dependency from feature_nulldummy.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-06-24 17:56:53 +05:30
glozow
e866f0d066 [functional test] submitrawpackage RPC 2022-06-23 14:39:47 +01:00
MacroFake
01e9e2d1ca
Merge bitcoin/bitcoin#25451: test: -whitebind and -bind with -listen=0 should throw an error
ceec6808d3 test: `-whitebind` and `-bind`  with `-listen=0` should throw an error (brunoerg)

Pull request description:

  This PR adds test coverage for the following init error:
  b9122e95f0/src/init.cpp (L872-L875)

ACKs for top commit:
  laanwj:
    Code review ACK ceec6808d3

Tree-SHA512: 03068abe7199b1235f029871ab87a3dd4943738c592ad62d82cdcd3e0201e627624960bd3ea1fc6fc1e7da4b8e215ba3393d1cb8130e1108049f764e51dc75c0
2022-06-23 12:08:19 +02:00
brunoerg
ceec6808d3 test: -whitebind and -bind with -listen=0 should throw an error 2022-06-22 15:22:25 -03:00
furszy
d69045e291
test: add coverage for 'listreceivedbyaddress' no change addrs return 2022-06-22 12:51:30 -03:00
MacroFake
faee330c7b
test: Fail if connect_nodes fails
Also replace the use of wait_until_helper, which is not allowed to be
called directly. Otherwise, --timeout-factor will not be honoured.
2022-06-22 09:15:33 +02:00
MacroFake
fafee78188
rpc: Return incrementalrelayfee in getmempoolinfo 2022-06-21 18:03:29 +02:00
MacroFake
fa8421bc5b
test: Remove from_node from create_self_transfer* MiniWallet helpers
The from_node argument is no longer used as of commit
a55606c3bd
2022-06-21 12:02:01 +02:00
Sebastian Falbesoner
be8d0dba15 test: refactor: save MiniWallet mode explicitly
Rather than abusing the member variables self._priv_key and
self._address to determine the MiniWallet mode, save it explicitly
instead in the constructor to increase the readability and
maintainability of the code.
2022-06-21 10:54:42 +02:00
laanwj
f8586b25f6
Merge bitcoin/bitcoin#25289: test: implement 'bech32m' mode for getnewdestination() helper
dcf36fe8e3 test: implement 'bech32m' mode for `getnewdestination()` helper (Sebastian Falbesoner)
1999dcfa40 test: add helpers for creating P2TR scripts/addresses from output key (Sebastian Falbesoner)

Pull request description:

  This PR adds the missing 'bech32m' mode for the `getnewdestination()` helper and sets it as default, i.e. the function returns a tuple (output x-only-pubkey, scriptPubKey, taproot address) now if not specified otherwise. In a preparation commit, the helpers `output_key_to_p2tr{_script}` are introduced. Note that in contrast to all other common script output types, there are usually _two_ keys involved in creating a taproot output (internal key and output key), hence the prefix `output_` is used to clarify that the  output key is expected and the helpers don't do any key tweaking.

  Thanks to michaelfolkson (for pointing out this TODO that I forgot about) and sipa (for patiently explaining basic things about BIP341).

ACKs for top commit:
  michaelfolkson:
    ACK dcf36fe8e3
  w0xlt:
    reACK dcf36fe8e3

Tree-SHA512: 5bb8d5fd96c63092ede10c3f022ffb2e13c14e333c4aa73348d95deb70cbf0a74745218dc4a7c419eb846793dd69e8217a7b4332a13ae2b2758e100b51fb1a9f
2022-06-17 22:51:42 +02:00
Andrew Chow
b0c8306349
Merge bitcoin/bitcoin#24649: wallet: do not count wallet utxos as external
7832e9438f test: fundrawtransaction preset input weight calculation (S3RK)
c3981e379f wallet: do not count wallet utxos as external (S3RK)

Pull request description:

  Correctly differentiating between external vs non-external utxos in coin control produces more accurate weight and fee estimations.

  Weight for external utxos is estimated based on the maximum signature size, while for the wallet utxos we expect minimal signature due to signature grinding.

ACKs for top commit:
  achow101:
    re-ACK 7832e9438f
  Xekyo:
    re-ACK 7832e9438f
  furszy:
    ACK 7832e943

Tree-SHA512: bb5635b0bd85fa9a76922a53ad3fa062286424c06a695a0e87407c665713e80a33555b644fbb13bcc1ab503dcd7f53aacbdc368d69ac0ecff8005603623ac94f
2022-06-16 14:11:19 -04:00
Andrew Chow
51eebe082d
Merge bitcoin/bitcoin#25368: doc: Update importaddress mention incompatibility with descriptor wallet
e3609cdc01 doc: Update importaddress mention incompatibility with descriptor wallet (BrokenProgrammer)

Pull request description:

  This is related to #25363 and offers a small update to the error messages from `EnsureLegacyScriptPubKeyMan` and `EnsureConstLegacyScriptPubKeyMan` to mention that they only are compatible with legacy wallets.

  The RPC documentation for `importaddress` is also updated to mention this as well as guide the user to the alternative `importdescriptors` for cases when using descriptor wallets.

  I'm thinking that we can introduce a "porting guide" document mentioned in #25363 in a separate PR since I would have to make myself more familiar with the subject before being able to tackle that.

ACKs for top commit:
  laanwj:
    Code review ACK e3609cdc01
  achow101:
    ACK e3609cdc01

Tree-SHA512: c7a924a7283fe59dc4e04c8c8fa034c15601f0b25eff09d975e98e2e8db5268ff470336b2d978d6916af9f782f9257b840d64bd15485b1742b4a8b8bfd0bb50f
2022-06-15 13:40:32 -04:00
MacroFake
6acba84603
Merge bitcoin/bitcoin#25358: test: passing a value below 5 MB to -maxmempool should throw an error
216c9b00ec test: passing a value below 5 MB to -maxmempool should throw an error (brunoerg)

Pull request description:

  This PR adds test coverage for the following init error:
  5174a139c9/src/init.cpp (L931-L935)

  By default, the minimum value is 5 MB. See:
  https://github.com/bitcoin/bitcoin/blob/master/doc/reduce-memory.md#memory-pool

ACKs for top commit:
  laanwj:
    Code review ACK 216c9b00ec
  furszy:
    Code review ACK 216c9b00

Tree-SHA512: 0c8fdcefb85e3dabb986a6294ad18503168a04246926614cbfa2d09d9e997312c937b01994f2999b1dc583e2eac5cdb8058bd58577baeb3eb23fdc690400cab9
2022-06-15 19:24:14 +02:00
MacroFake
4c0d1fec16
Merge bitcoin/bitcoin#25374: test: remove unused create_confirmed_utxos helper
42b2fdfd5f test: remove unused `create_confirmed_utxos` helper (Sebastian Falbesoner)

Pull request description:

  After more and more non-wallet tests have been converted to use MiniWallet (#25087, #24839, #24749 etc.), the `create_confirmed_utxos` helper is now not used anymore and can be removed. An alternative would be to create a MiniWallet version of `create_confirmed_utxos`, but it seems that it's not worth it, considering that would be only two lines (calling MiniWallet's `send_self_transfer_multi` with a subsequent `generate` call), see comment https://github.com/bitcoin/bitcoin/pull/24839#discussion_r896472729.

ACKs for top commit:
  MarcoFalke:
    cr ACK 42b2fdfd5f

Tree-SHA512: 274418156265a6071940f53cbcd77f6779af5e951cfa1e5efbf07a5c61487b521ee19f36b4105e5c0a808139d121e5e262e77525ea3d1486a0421f01abcf58fd
2022-06-15 08:38:19 +02:00
MacroFake
a57492f65d
Merge bitcoin/bitcoin#25370: test: check for getblocktxn request with out-of-bounds tx index
5a8c321444 test: check for `getblocktxn` request with out-of-bounds tx index (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `getblocktxn` message handler, in the case that any of the contained indices is out-of-bounds:
  a05876619a/src/net_processing.cpp (L2180-L2183)

ACKs for top commit:
  dunxen:
    ACK 5a8c321

Tree-SHA512: 2743c2c6d8aed57b22f825aefd60ba3e670321b60625a42ea7248e7b0fc41c73e9a5945153567c02824ba3b5f0fce7f4125bffc974973fc608b6ffbe49e14b65
2022-06-15 08:19:45 +02:00
Sebastian Falbesoner
42b2fdfd5f test: remove unused create_confirmed_utxos helper
Confirmed UTXOs in functional tests can simply be created by using
MiniWallet's `send_self_transfer_multi` method with a subsequent
`generate` call to mine a block.
2022-06-15 00:58:02 +02:00
BrokenProgrammer
e3609cdc01 doc: Update importaddress mention incompatibility with descriptor wallet 2022-06-14 20:54:45 +02:00
Sebastian Falbesoner
5a8c321444 test: check for getblocktxn request with out-of-bounds tx index 2022-06-14 18:11:22 +02:00
Sebastian Falbesoner
dcf36fe8e3 test: implement 'bech32m' mode for getnewdestination() helper 2022-06-14 13:37:18 +02:00
Sebastian Falbesoner
1999dcfa40 test: add helpers for creating P2TR scripts/addresses from output key 2022-06-14 13:32:57 +02:00
MacroFake
fafddafc2c
refactor: Introduce PeerManagerImpl::RejectIncomingTxs
Currently there are some confusions in net_processing:

* There is confusion between `-blocksonly mode` and `block-relay-only`,
  so adjust all comments to use the same nomenclature.
* Whether to disconnect peers for providing invs/txs is implemented
  differently. For example, it seems a bit confusing to disconnect
  `block-relay-only` peers with `relay` permission when they send a tx
  message, but not when they send an inv message. Also, keeping track of
  their inv announcements seems both wasteful and confusing, as it does
  nothing. This isn't possible in practice, as outbound connections do
  not have permissions assigned, but sees fragile to rely on. Especially
  in light of proposed changes to make that possible:
  https://github.com/bitcoin/bitcoin/pull/17167
2022-06-14 08:39:55 +02:00
MacroFake
fa779de665
test: Remove MiniWallet mempool_valid option 2022-06-13 18:09:16 +02:00
MacroFake
506d9b25a3
Merge bitcoin/bitcoin#24839: test: use MiniWallet for mining_prioritisetransaction.py
b167e536d0 test: refactor: use `create_lots_of_big_transactions` to dedup where possible (Sebastian Falbesoner)
8973eeb412 test: use MiniWallet for mining_prioritisetransaction.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (mining_prioritisetransaction.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in #20078. Note that the adapted helper function `create_lots_of_big_transactions` is currently only used in this test, i.e. there was no need to change any others.

ACKs for top commit:
  ayush933:
    tACK b167e53
  danielabrozzoni:
    tACK b167e536d0
  kouloumos:
    ACK b167e536d0
  furszy:
    ACK b167e536

Tree-SHA512: ccae20d7d414a720efdeea9c2ae399aa53a3a0e7db72bff8d0cb75d90621a7ae7c019ba68d24f9d06f7b111f87ff33bb9d8e5aa08b763e606cf10268780e205c
2022-06-13 17:59:01 +02:00
brunoerg
216c9b00ec test: passing a value below 5 MB to -maxmempool should throw an error 2022-06-13 09:58:22 -03:00
MacroFake
fa7a711a30
test: Fix out-of-range port collisions 2022-06-10 15:56:07 +02:00
amadeuszpawlik
292b1a3e9c GetExternalSigner(): fail if multiple signers are found
If there are multiple external signers, `GetExternalSigner()` will
just pick the first one in the list. If the user has two or more
hardware wallets connected at the same time, he might not notice this.

This PR adds a check and fails with suitable message.
2022-06-09 20:34:46 +02:00
dergoegge
ea54ba2f42 [test] Fix port collisions caused by p2p_getaddr_caching.py 2022-06-08 21:00:45 +02:00
dergoegge
f9682e75ac [test_framework] Set PortSeed.n directly after initialising params
This allows us to use `p2p_port()` with `set_test_params()`.
2022-06-08 21:00:18 +02:00
MacroFake
455780b1ae
Merge bitcoin/bitcoin#25294: test: Fix wait_for_debug_log UnicodeDecodeError
fa74b63c01 test: Fix wait_for_debug_log UnicodeDecodeError (MacroFake)

Pull request description:

  Fix the intermittent `UnicodeDecodeError` when the debug log is truncated on an (multi-byte) unicode character by treating everything as bytes.

  Also, remove the `ignore_case` option and the`re.search+re.escape` wrap. All of this is unused and doesn't exist on raw byte strings.

  Fixes https://github.com/bitcoin/bitcoin/issues/24575

ACKs for top commit:
  jonatack:
    ACK fa74b63c01
  brunoerg:
    ACK fa74b63c01

Tree-SHA512: c67c9355073e784fa8d9d48b8e79ff0c98f5ae9cd4d704ad12a76d2604733946054bc74b8ab346aa2184db23d740b85c8c13eb892d76cba92e42ebfd73f2f1bf
2022-06-08 17:53:06 +02:00
fanquake
b9416c3847
Merge bitcoin/bitcoin#25096: [net] Minor improvements to addr caching
292828cd77 [test] Test addr cache for multiple onion binds (dergoegge)
3382905bef [net] Seed addr cache randomizer with port from binding address (dergoegge)
f10e80b6e4 [net] Use ConnectedThroughNetwork() instead of GetNetwork() to seed addr cache randomizer (dergoegge)

Pull request description:

  The addr cache id randomizer is currently supposed to be seeded with the network of the inbound connection and the local socket (only the address is used not the port):  a8098f2cef/src/net.cpp (L2800-L2804)

  For inbound onion connections `CNode::addr.GetNetwork()` returns `NET_UNROUTABLE` and `CNode::addrBind` is set to `127.0.0.1:<onion bind port>`. This results in the same addr cache for all inbound connections on 127.0.0.1 binds.

  To avoid the same addr cache across all onion and other 127.0.0.1 binds, we should seed the addr cache randomizer with the correct network for inbound onion connections (using `CNode::ConnectedThroughNetwork()`) as well as the port of `CNode::addrBind`.

ACKs for top commit:
  sipa:
    utACK 292828cd77
  mzumsande:
    Code Review ACK 292828cd77
  naumenkogs:
    utACK 292828cd77

Tree-SHA512: d0be13bab6bc121c2926d4b168687f6c2ed4ce0c9dd19be71eb4886adeba8afc3daacdc4e232a0ba3b03a89d69b618abc5595b69abd1ad0c476d825bc6ea1f9f
2022-06-08 11:21:38 +01:00
MacroFake
fa74b63c01
test: Fix wait_for_debug_log UnicodeDecodeError 2022-06-07 20:54:37 +02:00
laanwj
e282764e04
Merge bitcoin/bitcoin#25228: test: add BIP-125 rule 5 testcase with default mempool
687addaf13 test: add BIP-125 rule 5 testcase with default mempool (James O'Beirne)
6120e8e287 test: allow passing sequence through create_self_transfer_multi (James O'Beirne)

Pull request description:

  Currently, we only test rule 5 of BIP-125 (replacement transactions cannot evict more than 100 transactions) by changing default mempool parameters to allow for more descendants. The current test works on a single transaction graph that has over 100 descendants.

  This patch adds a test to exercise rule 5 using the default mempool parameters. The case is a little more sophisticated: instead of working on a single transaction graph, it uses a replacement transaction to "unite" several UTXOs which join independent transaction graphs. The total number of transactions in these graphs sum to more than the max allowable replacement.

  I think the difference in transaction topology makes this a worthwhile testcase to have, setting aside the fact that this testcase works without having to use atypical mempool params.

  See also: [relevant discussion from IRC](https://www.erisian.com.au/bitcoin-core-dev/log-2022-05-27.html#l-126)

ACKs for top commit:
  laanwj:
    Code review ACK 687addaf13
  LarryRuane:
    ACK 687addaf13

Tree-SHA512: e589aeaf9d6f137d546b7809f8795d6f6043d87b15e97c2efe85b42ce8b49d977ee7d79440c542ca4b0b5ca2de527488029841a1ffc0d96c5771897df4b3f324
2022-06-07 20:49:33 +02:00
MacroFake
581e2bdbac
Merge bitcoin/bitcoin#24629: Bugfix: RPC/blockchain: pruneblockchain: Return the height of the actual last pruned block
e593ae07c4 Bugfix: RPC/blockchain: pruneblockchain: Return the height of the actual last pruned block (Luke Dashjr)

Pull request description:

  From 0.14 (2017 Mar) until before 0.19 (2019 Nov), the height of the last block pruned was returned, subject to a bug if there were blocks left unpruned due to sharing files with later blocks.

  In #15991, this was "fixed" to the current implementation, introducing a new bug: now, it returns the first *unpruned* block.

  Since the user provides the parameter as a block to include in pruning, it makes more sense to fix the behaviour to match the documentation.

  ~~(Additionally, the description of "pruneheight" in getblockchaininfo is fixed to be technically correct)~~

ACKs for top commit:
  fjahr:
    utACK e593ae07c4
  ryanofsky:
    Code review ACK e593ae07c4. Just rebased since last review. Maybe some of the original reviewers of #15991 will want to take a look at this to correct the mistake that was introduced there!

Tree-SHA512: c2d511df80682d57260aae8af1665f9d7eaed16448f185f4c9f23c78fa9b8289a02053da7a0b83643fef57610d601ea63b59ff39661a51f4827f1eb27cc30594
2022-06-07 08:04:51 +02:00
laanwj
06ea2783a2
Merge bitcoin/bitcoin#25220: rpc: fix incorrect warning for address type p2sh-segwit in createmultisig
3a9b9bb38e test: ensure createmultisig and addmultisigaddress are not returning any warning for expected cases (brunoerg)
eaf6f630c0 rpc: fix inappropriate warning for address type p2sh-segwit in createmultisig and addmultisigaddress (brunoerg)

Pull request description:

  Fixes #25127

  If there are any uncompressed keys when calling `AddAndGetMultisigDestination`, it will just default to a legacy address regardless of the chosen `address_type`. So, #23113 added a warnings field which will warn the user why their address format is different.

  However, when creating a multisig (p2sh-segwit), it is returning an inappropriate warning, because when getting the output type from destination (`OutputTypeFromDestination`), it returns `ScriptHash` for both legacy and `P2SH_SEGWIT`. So, since `P2SH_SEGWIT` is different from `ScriptHash`, it returns the warning:
  192d639a6b/src/rpc/output_script.cpp (L166-L169)

  So, to avoid this mistake I changed `OutputTypeFromDestination` to `descriptor->GetOutputType()` to get the appropriate output type.

ACKs for top commit:
  jonatack:
    ACK 3a9b9bb38e
  laanwj:
    Code review ACK 3a9b9bb38e

Tree-SHA512: 49f717479c2b8906277e7591ddd4747f7961c2d5c77494b5124045de9036a4277d46b9ad99279d51f0c4484284c445f1e1d3c55c49bbf0716741bad426a89369
2022-06-06 17:13:22 +02:00
brunoerg
3a9b9bb38e test: ensure createmultisig and addmultisigaddress are not returning any warning for expected cases 2022-06-06 09:46:42 -03:00
Fabian Jahr
5826bf546e
test: Add test for getblockfrompeer on syncing pruned nodes 2022-06-06 01:36:16 +02:00
Luke Dashjr
e593ae07c4 Bugfix: RPC/blockchain: pruneblockchain: Return the height of the actual last pruned block
From 0.14 (2017 Mar) until before 0.19 (2019 Nov), the height of the last
block pruned was returned, subject to a bug if there were blocks left unpruned
due to sharing files with later blocks.

In #15991, this was "fixed" to the current implementation, introducing a new
bug: now, it returns the first *unpruned* block.

Since the user provides the parameter as a block to include in pruning, it
makes more sense to fix the behaviour to match the documentation.
2022-06-03 07:20:07 +00:00
laanwj
00ce8543f1
Merge bitcoin/bitcoin#24171: p2p: Sync chain more readily from inbound peers during IBD
48262a00f5 Add functional test for block sync from inbound peers (Suhas Daftuar)
0569b5c4bb Sync chain more readily from inbound peers during IBD (Suhas Daftuar)

Pull request description:

  When in IBD, if the honest chain is only known by inbound peers, then we must
  eventually sync from them in order to learn it. This change allows us to
  perform initial headers sync and fetch blocks from inbound peers, if we have no
  blocks in flight.

  The restriction on having no blocks in flight means that we will naturally
  throttle our block downloads to any such inbound peers that we may be
  downloading from, until we leave IBD. This is a tradeoff between preferring
  outbound peers for most of our block download, versus making sure we always
  eventually will get blocks we need that are only known by inbound peers even
  during IBD, as otherwise we may be stuck in IBD indefinitely (which could have
  cascading failure on the network, if a large fraction of the network managed to
  get stuck in IBD).

  Note that the test in the second commit fails on master, without the first commit.

ACKs for top commit:
  ajtowns:
    ACK 48262a00f5
  sipa:
    ACK 48262a00f5

Tree-SHA512: ffad3a05fa9a32a92226843c9128f52c275e8d51930fde7368badc340227f2ed680561c4c9f2937b4e3bd722474464849ec9b624f912f5e380ce98d71b55764d
2022-06-02 22:35:05 +02:00
dergoegge
292828cd77 [test] Test addr cache for multiple onion binds 2022-06-02 19:14:17 +02:00
James O'Beirne
687addaf13 test: add BIP-125 rule 5 testcase with default mempool
This testcase exercises rule 5 of BIP-125 (no more than 100 evictions
due to replacement) without having to test under non-default mempool
parametmers.
2022-06-02 10:19:24 -04:00
Sebastian Falbesoner
1bace0cfee test: check replaceable mismatch error in createrawtransaction RPC 2022-06-02 12:59:13 +02:00
MacroFake
fafaad98f7
test: Set maxfeerate=0 in MiniWallet sendrawtransaction() 2022-06-01 17:07:05 +02:00
MacroFake
9cc010f5a9
Merge bitcoin/bitcoin#25087: test: use MiniWallet for feature_dbcrash.py
1da5e45725 test: use MiniWallet for feature_dbcrash.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (feature_dbcrash.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in https://github.com/bitcoin/bitcoin/issues/20078.

ACKs for top commit:
  laanwj:
    Code review ACK 1da5e45725
  brunoerg:
    crACK 1da5e45725

Tree-SHA512: 75ee9a32fd1451254004797d695d18032bd0fcb66ebd88cf737e147e43812525f6e884ec05fcc4f76f566dc71174c8ed7347bcdce16567db6511746ae64cead0
2022-06-01 16:43:49 +02:00
Sebastian Falbesoner
7d0f67a0d5 test: check pre-segwit peer error in getblockfrompeer RPC 2022-05-31 23:04:13 +02:00
brunoerg
ebfc308ea4 test: add coverage for non-hex value to -minimumchainwork 2022-05-31 08:23:38 -03:00
MacroFake
5f65afff9c
Merge bitcoin/bitcoin#24178: p2p: Respond to getheaders if we have sufficient chainwork
a35f963edf Add test for getheaders behavior (Suhas Daftuar)
ef6dbe6863 Respond to getheaders if we have sufficient chainwork (Suhas Daftuar)

Pull request description:

  Previously, we would check to see if we were in IBD and ignore getheaders requests accordingly. However, the IBD criteria -- an optimization mostly targeted at behavior when we have peers serving us many blocks we need to download -- is difficult to reason about in edge-case scenarios, such as if the network were to go a long time without any blocks found and nodes are getting restarted during that time.

  To make things simpler to reason about, just use `nMinimumChainWork` as our anti-DoS threshold for responding to a getheaders request; as long as our chain has that much work, it should be fine to respond to a peer asking for our headers (and this should allow such a peer to request blocks from us if needed).

ACKs for top commit:
  klementtan:
    crACK a35f963edf
  naumenkogs:
    ACK a35f963edf
  MarcoFalke:
    review ACK a35f963edf 🗯

Tree-SHA512: 131e3872e7fe80382ea9c1ec202d6c2dc59c006355c69000aa3f4ce6bccd02a6c689c8cb8f3542b5d9bc48bfa61edcbd1a78535c0b79018971d02bed2655d284
2022-05-31 12:05:46 +02:00
MacroFake
269fa667f2
Merge bitcoin/bitcoin#25044: test: Use MiniWallet in rpc_rawtransaction.py
e8959000b6 test: Use MiniWallet in rpc_rawtransaction.py (Daniela Brozzoni)
e93046c10b MOVEONLY: Move signrawtransactionwithwallet test (Daniela Brozzoni)

Pull request description:

  This PR allows `rpc_rawtransaction.py` to be run even without the Core wallet by using the MiniWallet instead, as proposed in https://github.com/bitcoin/bitcoin/issues/20078.
  This test was previously run twice, once with `--legacy-wallet` and once with
  `--descriptors`. Since this would have meant running the same test twice
  if the wallet wasn't compiled, now we run it just once with the legacy
  wallet.

ACKs for top commit:
  jonatack:
    ACK e8959000b6

Tree-SHA512: d1580570a54dad8e30a5df1ab7d03ecb3f824efe6843323e1f3aef63592045d823c7d54fc86321dc7c1d414854a253431a01a7baa9f30426ea9a09ef11ae3a04
2022-05-30 16:57:47 +02:00
Daniela Brozzoni
e8959000b6
test: Use MiniWallet in rpc_rawtransaction.py
This test was previously run twice, once with `--legacy-wallet` and once with
`--descriptors`.
Now we run it only with `--legacy-wallet`, as all the tests has been
ported to the MiniWallet but `raw_multisig_transaction_legacy_tests`,
which can be run only with the legacy wallet.
We also decrease the number of nodes used from 4 to 3, making the test
run slightly faster.
2022-05-30 16:25:18 +02:00
Daniela Brozzoni
e93046c10b
MOVEONLY: Move signrawtransactionwithwallet test
Put signrawtransactionwithwallet_tests in rpc_signrawtransaction.py,
as the test is mainly testing the signrawtransaction RPC.
Review with `git show --color-moved=dimmed-zebra`
2022-05-30 16:25:17 +02:00
James O'Beirne
6120e8e287 test: allow passing sequence through create_self_transfer_multi
And some little type annotation additions.
2022-05-27 13:40:06 -04:00
Sebastian Falbesoner
387ae8bc09 rpc: remove deprecated fee fields from mempool entries 2022-05-27 17:29:04 +02:00
MacroFake
3ba6dd6f4b
Merge bitcoin/bitcoin#24408: rpc: add rpc to get mempool txs spending specific prevouts
4185570340 Add RPC to get mempool txs spending outputs (t-bast)

Pull request description:

  We add an RPC to fetch mempool transactions spending any of the given outpoints.

  Without this RPC, application developers need to first call `getrawmempool` which returns a long list of `txid`, then fetch each of these transactions individually (`getrawtransaction`) to check whether they spend the given outpoints, which wastes a lot of bandwidth (in the worst case we need to transfer the whole mempool).

  For example in lightning, when we discover that one of our channel funding transactions has been spent, we need to find the spending transaction to claim our outputs from it. We are currently forced to fetch the whole mempool to do the analysis ourselves, which is quite costly.

  I believe that this RPC is also generally useful when doing some introspection on your mempool after one of your transactions failed to broadcast, for example when you implement RBF at the application level. Fetching and analyzing the conflicting transaction gives you more information to successfully replace it.

ACKs for top commit:
  darosior:
    re-utACK 4185570340
  vincenzopalazzo:
    re-ACK 4185570340
  danielabrozzoni:
    re-tACK 4185570340
  w0xlt:
    reACK 4185570340

Tree-SHA512: 206687efb720308b7e0b6cf16dd0a994006c0b5a290c8eb386917a80130973a6356d0d5cae1c63a01bb29e066dd721594969db106cba7249214fcac90d2c3dbc
2022-05-27 15:16:00 +02:00
brunoerg
295ff61934 test: add coverage for unknown -blockfilterindex 2022-05-23 18:06:13 -03:00
Andrew Chow
5ebff43025
Merge bitcoin/bitcoin#25122: rpc: getreceivedbylabel, return early if no addresses were found in the address book
baa3ddc49c doc: add release notes about `getreceivedbylabel` returning an error if the label is not in the address book. (furszy)
8897a21658 rpc: getreceivedbylabel, don't loop over the entire wallet txs map if no destinations were found for the input label. (furszy)

Pull request description:

  Built on top of #23662, coming from comment https://github.com/bitcoin/bitcoin/pull/23662#pullrequestreview-971407999.

  If `wallet.GetLabelAddresses()` returns an empty vector (the wallet does not have stored destinations with that label in the addressbook) or if none of the returned destinations are from the wallet, we can return the function right away.
  Otherwise, we are walking through all the wallet txs + outputs for no reason (`output_scripts` is empty).

ACKs for top commit:
  achow101:
    ACK baa3ddc49c
  theStack:
    re-ACK baa3ddc49c
  w0xlt:
    ACK baa3ddc49c

Tree-SHA512: 00e10365b179bf008da2f3ef8fbb3ee04a330426374020e3f2d0151b16991baba4ef2b944e4659452f3e4d6cb20f128d0918ddf0453933a25a4d9fd8414a1911
2022-05-23 12:15:14 -04:00
furszy
8897a21658
rpc: getreceivedbylabel, don't loop over the entire wallet txs map if no destinations were found for the input label.
If wallet.GetLabelAddresses() returns an empty vector (the wallet does not have addresses with that label in the addressbook) or if none of the returned destinations are from the wallet, we can return the function right away.
2022-05-20 16:32:09 -03:00
MacroFake
a7e3afb221
Merge bitcoin/bitcoin#25171: rpc: wallet: remove -deprecatedrpc=exclude_coinbase logic
a4703ce9d7 doc: add release notes about removal of the `deprecatedrpc=exclude_coinbase` (Sebastian Falbesoner)
ef0aa74836 rpc: wallet: remove `-deprecatedrpc=exclude_coinbase` logic (Sebastian Falbesoner)

Pull request description:

  Including coinbase transactions in `receivedby` RPCs and adding the `-deprecatedrpc=exclude_coinbase` was done in PR #14707 (released in v23.0). For the next release v24.0, this configuration option can be removed.

ACKs for top commit:
  fanquake:
    ACK a4703ce9d7

Tree-SHA512: 97cd4e78501e64f678c78d2ebb5be5376688c023e34fced71dd24e432d27aa31a74b5483545f49ba0bdf48656d8b8b7bee74e3db26cf6daf112613f1caa4dfa4
2022-05-20 08:48:09 +01:00
MacroFake
4a8709821e
Merge bitcoin/bitcoin#24830: init: Allow -proxy="" setting values
1d4122dfef init: Allow -proxy="" setting values (Ryan Ofsky)

Pull request description:

  This drops the `No proxy server specified. Use -proxy=<ip> or -proxy=<ip:port>` error when a empty `-proxy=` command line argument, `bitcoin.conf` value, or `settings.json` value is specified, and just makes bitcoin connect and listen normally in these cases.

  The error was originally added in https://github.com/bitcoin/bitcoin/pull/20003 to prevent a bare `-proxy` command line argument with no assignment from clearing proxy settings. But it was implemented in an overbroad way breaking empty `-proxy=` assignments as well.

  The motivation for this change is to prevent a GUI bug that happens with https://github.com/bitcoin/bitcoin/pull/15936, reported in https://github.com/bitcoin/bitcoin/pull/15936#pullrequestreview-937685759 by vasild, that happens after a proxy setting is enabled and disabled in the GUI. But this change also makes sense on its own to remove a potentially confusing error message.

ACKs for top commit:
  hebasto:
    re-ACK 1d4122dfef, only rebased since my recent [review](https://github.com/bitcoin/bitcoin/pull/24830#pullrequestreview-941255672).

Tree-SHA512: 753adfce199ed078a6cd9e0ea78e76c0b14070f8fcfe2a4632cd0c6dfe6b4e135ddffbe11a97e5e30520ea9e5bda00bad1493cbaef74cf425aa8613249167f53
2022-05-20 08:28:08 +01:00
MacroFake
d433f59f1e
Merge bitcoin/bitcoin#25173: test: add coverage for unknown network in -onlynet
055d94d1ab test: add coverage for unknown network in -onlynet (brunoerg)

Pull request description:

  This PR adds test coverage for the following init error by passing an unknown network in -onlynet
  0de36941ec/src/init.cpp (L1311)

ACKs for top commit:
  MarcoFalke:
    rACK 055d94d1ab

Tree-SHA512: 01bbb297afff371f6345889fa04117ff195b68f0bbf934878ba446049791fdbd7d2ce119ee4f9b3616cc0a81330d7055507dc81151acf68532c077f3575258e9
2022-05-20 08:18:02 +01:00
brunoerg
055d94d1ab test: add coverage for unknown network in -onlynet 2022-05-19 18:39:23 -03:00
Sebastian Falbesoner
1da5e45725 test: use MiniWallet for feature_dbcrash.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-05-19 17:53:30 +02:00
Sebastian Falbesoner
ef0aa74836 rpc: wallet: remove -deprecatedrpc=exclude_coinbase logic 2022-05-19 16:10:59 +02:00
fanquake
fdb82a30be
Merge bitcoin/bitcoin#25147: Net processing: follow ups to #20799 (removing support for v1 compact blocks)
bf6526f4a0 [test] Remove segwit argument from build_block_on_tip() (John Newbery)
c65bf50b44 Remove fUseWTXID parameter from CBlockHeaderAndShortTxIDs constructor (John Newbery)

Pull request description:

  This implements two of the suggestions from code reviews of PR 20799:

  - Remove fUseWTXID parameter from CBlockHeaderAndShortTxIDs constructor
  - Remove segwit argument from build_block_on_tip()

ACKs for top commit:
  dergoegge:
    Code review ACK bf6526f4a0
  naumenkogs:
    ACK bf6526f4a0

Tree-SHA512: d553791d1364b9e655183755e829b195c9b47f59c62371dbae49d9c0f8d84fec58cf18f4dde89591672ef5658e18c9cf0206c2efd70606980f87e506bc3bd4e5
2022-05-19 09:37:32 +01:00
MacroFake
bb83aba6c9
Merge bitcoin/bitcoin#25161: rpc: Put undocumented JSON failure mode behind a runtime flag
b953ea6cc6 rpc: Put undocumented JSON failure mode behind a runtime flag (Suhail Saqan)

Pull request description:

  Fixes #24695 (Put undocumented JSON failure mode behind a runtime flag)

ACKs for top commit:
  luke-jr:
    utACK b953ea6cc6
  vincenzopalazzo:
    ACK b953ea6cc6

Tree-SHA512: 2005ee1b1f3b637918390b2ecd4166f2fd8c86e3c59fba3da8a0cbd5b1dffd03190c92f6dca3c489ecce4276eaf3108b2edcf9cd6224b713adb52f5bb848163b
2022-05-19 06:44:55 +02:00
Suhail Saqan
b953ea6cc6 rpc: Put undocumented JSON failure mode behind a runtime flag
rpc: Put undocumented JSON failure mode behind a runtime flag
2022-05-18 10:50:59 -07:00
MacroFake
e016c00e98
Merge bitcoin/bitcoin#25126: test: add BIP157 message parsing support (via MESSAGEMAP)
5dc6d92077 test: make BIP157 messages default-constructible (MESSAGEMAP compatibility) (Sebastian Falbesoner)
71e4cfefe7 test: p2p: add missing BIP157 message types to MESSAGEMAP (Sebastian Falbesoner)

Pull request description:

  The script [message-capture-parser.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/message-capture/message-capture-parser.py) currently doesn't support parsing the BIP157 messages `getcfilters`, `getcfheaders` and `getcfcheckpt`, e.g.
  ```
  $ ./contrib/message-capture/message-capture-parser.py msgs_recv.dat
  ...
      WARNING - Unrecognized message type b'getcfcheckpt' in /home/thestack/bitcoin/msgs_recv.dat
  ...
  ```

  This PR fixes this by adding the missing message type mappings to the [`MESSAGEMAP`](225e5b57b2/test/functional/test_framework/p2p.py (L95-L127)) in the test framework and add default-constructors for the corresponding `msg_`... classes.

  Without the second commit, the following error message would occur:
  ```
    File "/home/thestack/bitcoin/./contrib/message-capture/message-capture-parser.py", line 141, in process_file
      msg = MESSAGEMAP[msgtype]()
  TypeError: __init__() missing 2 required positional arguments: 'filter_type' and 'stop_hash'
  ```

ACKs for top commit:
  dunxen:
    tACK [5dc6d92](5dc6d92077)

Tree-SHA512: d656c4d38a856373f01d7c293ae7d2b27378a9fc248048ebf2a64725ef8b498b3ddf4f420704abdb20d0c68ca548f1777602c5e73b66821a20c97ae618f1d63f
2022-05-18 19:08:48 +02:00
MacroFake
139f789d7a
Merge bitcoin/bitcoin#25124: test: Fix intermittent race in p2p_unrequested_blocks.py
faac67cab0 test: Fix intermittent race in p2p_unrequested_blocks.py (MacroFake)

Pull request description:

  Disconnect may also result in an `OSError`, not only an `AssertionError`. Instead of maintaining a dead code path and enumerating disconnect reasons, just assume disconnection happens every time.

ACKs for top commit:
  jamesob:
    Code review ACK faac67cab0

Tree-SHA512: d2cec003168e421a5faed275cb2e1ef9fc63f9e8514f41d21da17e8964c79e5b453ccd72cd7ec62805f45293cf877be5bc8124ae98a515c0aa42d6e053409653
2022-05-18 15:39:20 +02:00
John Newbery
bf6526f4a0 [test] Remove segwit argument from build_block_on_tip()
The only place that segwit=True is for a block that contains only the
coinbase transaction. Since the witness commitment is optional if none
of the transactions have a witness, we can leave it out. This doesn't
change the test coverage, which is testing p2p compact block logic.

Suggested in https://github.com/bitcoin/bitcoin/pull/20799#discussion_r867782119
2022-05-18 13:47:54 +01:00
S3RK
7832e9438f test: fundrawtransaction preset input weight calculation 2022-05-18 08:25:08 +02:00
ishaanam
c6122f560b test: use sendall in wallet_taproot.py tests
Fixes #25129 (subtractfeefromamount=true fails with insufficient
funds)
2022-05-17 13:40:15 -04:00
fanquake
d5d40d59f8
Merge bitcoin/bitcoin#23679: Sanitize port in addpeeraddress()
ada8358ef5 Sanitize port in `addpeeraddress()` (amadeuszpawlik)

Pull request description:

  In connection to #22087, it has been [pointed out](https://github.com/bitcoin/bitcoin/pull/22087#pullrequestreview-674786285) that `addpeeraddress` needs to get its port-value sanitized.

ACKs for top commit:
  fanquake:
    ACK ada8358ef5

Tree-SHA512: 48771cd4f6940aa7840fa23488565c09dea86bd5ec5a5a1fc0374afb4857aebcd2a1f51e2d4cb7348460e0ad9793dc5d2962df457084ed2b8d8142cae650003f
2022-05-17 16:39:10 +01:00
Suhas Daftuar
48262a00f5 Add functional test for block sync from inbound peers 2022-05-17 09:36:49 -04:00
John Newbery
42882fc8fc [net processing] Only accept sendcmpct with version=2
Subsequent commits will remove support for other versions of compact blocks.

Add a test that a received `sendcmpct` message with version = 1 is
ignored.
2022-05-15 15:37:56 -04:00
John Newbery
16730b64bb [net processing] Only advertise support for version 2 compact blocks
Subsequent commits will remove support.
2022-05-15 15:37:56 -04:00
John Newbery
cba909eaf9 [net] Stop testing version 1 compact blocks.
Support for version 1 is removed in the following commits.
2022-05-15 15:37:56 -04:00
MacroFake
b74a6dde8c
Merge bitcoin/bitcoin#25123: test: Fix race condition in index prune test
4faa550072 test: Fix race condition in index pruning test (Fabian Jahr)

Pull request description:

  Fixes #25031

  The `feature_index_prune.py` test seems to be racy because connections are reestablished after restarts and the blocks are synced via the `sync_blocks` function. The `sync_blocks` function has a sanity check at the beginning to check that all nodes in the set have at least one established connection and that is not always the case.

  As a solution nodes are not connected via the `-connect` parameter on start but instead via the `connect_nodes` helper.

Top commit has no ACKs.

Tree-SHA512: f88377715f455f1620725fe8ebd6b486fa0209660b193bf68d1ce1452e2086ac5d169d8ca4c2b61443566232e96fb9c6386ee482bc546cce38078d72e7c3c29f
2022-05-15 09:19:43 +02:00
Fabian Jahr
4faa550072
test: Fix race condition in index pruning test
Nodes are restarted and reconnected as part of the test. Afterwards
`sync_blocks` is called immediately on the nodes. `sync_blocks`
first checks that all the included nodes have at least one
connection. Since adding a connection is usually happening in a
thread, sometimes nodes could run into this check before the
connection was fully established so that it would fail the entire
test.

This fix uses the `connect_nodes` helper to make the connection the
nodes. `connect_nodes` has a wait for the connection built into it.
2022-05-14 17:33:41 +02:00
amadeuszpawlik
ada8358ef5 Sanitize port in addpeeraddress()
- Ensures port sanitization in `addpeeraddress()`
- Adds test to check for invalid port values
2022-05-14 10:22:16 +02:00
Sebastian Falbesoner
5dc6d92077 test: make BIP157 messages default-constructible (MESSAGEMAP compatibility)
In order to deserialize received or read messages via lookup in
MESSAGEMAP (e.g.: `t = MESSAGEMAP[msgtype]()`), the messages must have a
default constructor, i.e. there needs to be the possibility to
initialize them with zero arguments.
2022-05-13 13:53:25 +02:00
Sebastian Falbesoner
71e4cfefe7 test: p2p: add missing BIP157 message types to MESSAGEMAP 2022-05-13 13:37:46 +02:00
MacroFake
faac67cab0
test: Fix intermittent race in p2p_unrequested_blocks.py 2022-05-13 09:15:12 +02:00
MacroFake
1d5325a8f9
Merge bitcoin/bitcoin#25117: test: Check msg type in msg capture is followed by zeros
faa5a7a573 test: Check msg type in msg capture is followed by zeros (MacroFake)

Pull request description:

  Checking that they are not printable is an odd (and wrong) way to check that all chars are zero.

ACKs for top commit:
  theStack:
    Code-review ACK faa5a7a573

Tree-SHA512: 63e001bd25298dcf47606f8ab11ddfb704ca963304149b0f6e188eb7dcf45c41f92d39f26bda32bceb03384720c9bdddb2673dba513cd9242dc9663d498b3f29
2022-05-13 07:49:22 +02:00
brunoerg
1df42bc262 test: compare /mempool/info response with getmempoolinfo RPC 2022-05-12 17:49:50 -03:00
MacroFake
faa5a7a573
test: Check msg type in msg capture is followed by zeros 2022-05-12 17:07:35 +02:00
Sebastian Falbesoner
9feb887082 rpc: check fopen return code in dumptxoutset
This change improves the usability of the `dumptxoutset` RPC in two ways,
in the case that an invalid path is passed:
  1. return from the RPC immediately, rather then when the file is first
     tried to be written (which is _after_ calculating the UTXO set hash)
  2. return a proper return code and error message instead of the cryptic
     "CAutoFile::operator<<: file handle is nullptr: unspecified
      iostream_category error" (-1)
2022-05-11 16:03:40 +02:00
laanwj
ed4eeafbb6
Merge bitcoin/bitcoin#24793: test: Change color of skipped functional tests
3258bad996 changes color of skipped functional tests (Jacob P. Fickes)

Pull request description:

  changes the color of skipped functional tests (currently grey and can be hard to read/invisible on dark backgrounds) to yellow.

  resolves #24791

ACKs for top commit:
  theStack:
    Tested ACK 3258bad996
  jarolrod:
    Tested ACK 3258bad996

Tree-SHA512: 3fe5ae0d3b4902b2b6bda6e89ab780feb8bf4b7cb1ce7e8467057b94a1e0a26ddeaf3cac0bc19b06ef10d8bccaac9c495029d42740fbedab8fb0d5fdd7d02eaf
2022-05-10 13:12:38 +02:00
MacroFake
77a9997d97
Merge bitcoin/bitcoin#25063: test: previous releases: add v23.0
dba1231672 test: previous releases: add v23.0 (Sjors Provoost)

Pull request description:

  Follows the same pattern as d8b705f1ca (v22.0) and 8a57a06a50 (v0.21.0).

  Starting from v23.0 there is a separate macOS release for x86_64 and aarch64.

ACKs for top commit:
  prusnak:
    Approach ACK dba1231672

Tree-SHA512: 249aeddd5e80e163578581e5c8e9b6579f3694abc3d1fb68dddb7b42d75021ad85266688ec4a365a6631d82a65a19873aff7ba61c0ea59d21f8adbe4b772dc16
2022-05-06 11:38:03 +02:00
Sjors Provoost
dba1231672
test: previous releases: add v23.0
Starting from v23.0 there is a separate macOS release for x86_64 and aarch64.

Co-authored-by: Pavol Rusnak <pavol@rusnak.io>
2022-05-06 10:00:47 +02:00
t-bast
4185570340
Add RPC to get mempool txs spending outputs
We add an RPC to fetch the mempool transactions spending given outpoints.
Without this RPC, application developers would need to first call
`getrawmempool` which returns a long list of `txid`, then fetch each of
these txs individually to check whether they spend the given outpoint(s).

This RPC can later be enriched to also find confirmed transactions instead
of being restricted to mempool transactions.
2022-05-05 14:56:48 +02:00
laanwj
0047d9b89b
Merge bitcoin/bitcoin#24993: test, contrib, refactor: use with when opening a file
027aab663a test, contrib, refactor: use `with` when opening a file (brunoerg)

Pull request description:

  When manipulating a file in Python without using `with()`, you have to close the file manually, so this PR does it in `get_block_hashes` (`contrib/linearize/linearize-data.py`).

  Edit: this PR does it for all occurances that previously weren't using `with`.

ACKs for top commit:
  laanwj:
    Code review ACK 027aab663a

Tree-SHA512: 879400968e0013e8678ec16f1fe5d0963a73c1e0d442ca34802d885214f0783d2e9a9b500fc6be7c3b93560a367b6a3d685eee24d2f9ce53fddf064ea6feecf8
2022-05-04 19:52:16 +02:00
MacroFake
9b42d62f42
Merge bitcoin/bitcoin#25045: test: add coverage for invalid requests for blockfilterheaders (REST)
d1bfe5ebdb test: add coverage for invalid requests for `blockfilterheaders` (brunoerg)

Pull request description:

  This PR adds test coverage for invalid requests (`Invalid hash` and `Unknown filtertype`) for `/blockfilterheaders` in REST functional test.

ACKs for top commit:
  jonatack:
    ACK d1bfe5ebdb
  vincenzopalazzo:
    ACK d1bfe5ebdb

Tree-SHA512: 9ab7efe7131296577c60642f95921799cf1dbae9c2aaea6752d2ac9f35a1bcc72b9d742a146c314f82fe1848190a80c88836ab78fc28773ed12e97fa327828e7
2022-05-04 09:57:40 +02:00
brunoerg
d1bfe5ebdb test: add coverage for invalid requests for blockfilterheaders 2022-05-03 15:04:54 -03:00
MacroFake
d24318a40c
Merge bitcoin/bitcoin#24941: test: MiniWallet: support skipping mempool checks (feature_fee_estimation.py performance fix)
a498acce45 test: MiniWallet: skip mempool check if `mempool_valid=False` (Sebastian Falbesoner)
01552e8f67 test: MiniWallet: always rehash after signing (P2PK mode) (Sebastian Falbesoner)

Pull request description:

  MiniWallet's core method for creating txs (`create_self_transfer`) right now always executes the `testmempoolaccept` RPC to check for mempool validity or invalidity. In some test cases where we use MiniWallet to create a huge number of transactions this can lead to performance issues, in particular feature_fee_estimation.py where the execution time after MiniWallet usage (PR #24817) doubled, see https://github.com/bitcoin/bitcoin/issues/24828#issuecomment-1100058100, https://github.com/bitcoin/bitcoin/issues/24828#issuecomment-1100301980. This PR mitigates this by skipping the mempool check if the parameter `mempool_valid` is set to `False`.

  As a preparatory commit, the test feature_csv_activation.py has to be adapted w.r.t. to rehashing of transactions, as we now hash all transactions immediately in `create_self_transfer` in order to get the txid (before we relied on the result of `testmempoolaccept`).

  On my machine, this decreases the execution time quite noticably:

  master branch:
  ```
  $ time ./test/functional/feature_fee_estimation.py
  real    3m20.771s
  user    2m52.360s
  sys     0m39.340s
  ```

  PR branch:
  ```
  $ time ./test/functional/feature_fee_estimation.py
  real    2m1.386s
  user    1m42.510s
  sys     0m22.980s
  ```

  Partly fixes #24828 (hopefully).

ACKs for top commit:
  danielabrozzoni:
    tACK a498acce45

Tree-SHA512: f20c358ba42b2ded86175f46ff3ff9eaefb84175cbd1c2624f44904c8d8888e67ce64d6dcbb26aabbf07906e6f5bdea40353eba9ae668618cadcfc517ef7201b
2022-05-03 09:59:52 +02:00
MacroFake
fa12706fc6
Reject invalid rpcauth formats 2022-04-30 12:53:35 +02:00
Martin Zumsande
a3cd7dbfd8 test: stop node before calling assert_start_raises_init_error
...in feature_coinstatsindex and feature_pruning.
Also add an assert to assert_start_raises_init_error that the node is
not already running.
2022-04-29 22:50:26 +02:00
Andrew Chow
606ce05ec2
Merge bitcoin/bitcoin#18554: wallet: ensure wallet files are not reused across chains
5f213213cb tests: add tests for cross-chain wallet use prevention (Seibart Nedor)
968765973b wallet: ensure wallet files are not reused across chains (Seibart Nedor)

Pull request description:

  This implements a proposal in #12805 and is a rebase of #14533.

  This seems to be a working approach, but I'm not sure why the `p2p_segwit.py` functional test needed a change, so I'll look into it more.

ACKs for top commit:
  achow101:
    ACK 5f213213cb
  dongcarl:
    Code Review ACK 5f213213cb
  [deleted]:
    tACK 5f213213cb

Tree-SHA512: 2c934300f113e772fc31c16ef5588526300bbc36e4dcef7d77bd0760c5c8f0ec77f766b1bed5503eb0157fa26dc900ed54d2ad1b41863c1f736ce5c1f3b67bec
2022-04-28 15:59:47 -04:00
laanwj
47b8256da8
Merge bitcoin/bitcoin#24937: test: Remove previous release check in feature_taproot.py
fafd67479a test: Remove previous release check (MarcoFalke)

Pull request description:

  Now that the commit (7c08d81e11) which changes taproot to be enforced for all blocks is sufficiently buried by other commits, and thus less likely to be reverted,  it seems a good time to remove no longer needed test code.

  The `feature_taproot` functional test is cleaned up to no longer run against a previous release. Since previous releases are static and impossible to change, it is sufficient to run the test once against the release. Now that this is done, the check can be removed without decreasing test coverage.

ACKs for top commit:
  laanwj:
    Concept and code review ACK fafd67479a
  vincenzopalazzo:
    ACK fafd67479a

Tree-SHA512: fcb1a93f3bf9deb5f5c7327a7cd23be10ba09c9f4cbfa73ee2764a93c6ce7d6fa98ca32f2cf4023c20ab624aee601beec949fd02a57a3a658fdbd4be1a9ff338
2022-04-28 19:25:27 +02:00
MacroFake
4381681e55
Merge bitcoin/bitcoin#25011: tests: Do not always create a descriptor wallet in wallet_createwallet
786b3a7c44 tests: Do not always create a descriptor wallet in wallet_createwallet (Andrew Chow)

Pull request description:

  The createwallet test for some invalid parameters incorrectly always creates a descriptor wallet. This is unnecessary and also breaks the test when bdb is not compiled in.

  Fixes #25007

ACKs for top commit:
  jacobpfickes:
    ACK 786b3a7c44

Tree-SHA512: 97b0953a08adf83d5ea84cac2651253d790b43d606a2f746dd45d3ccd1fb576bab63e3835e3de592715ef8a5cb133e6f19a3ab810fedf4684072143c3cb578d4
2022-04-28 07:41:22 +02:00
brunoerg
027aab663a test, contrib, refactor: use with when opening a file 2022-04-27 20:04:33 -03:00
Andrew Chow
786b3a7c44 tests: Do not always create a descriptor wallet in wallet_createwallet
The createwallet teswt for some invalid parameters incorrectly always
creates a descriptor wallet. This is unnecessary and also breaks the
test when bdb is not compiled in.
2022-04-27 14:50:04 -04:00
MacroFake
f0a834e2f1
Merge bitcoin/bitcoin#18642: Use std::chrono for the time to rotate destination of addr messages + tests
2ff8f4dd81 Add tests for addr destination rotation (Gleb Naumenko)
77ccb7fce1 Use std::chrono for salting when randomizing ADDR destination (Gleb Naumenko)

Pull request description:

  We currently assign a destination peer for relaying particular addresses of nodes every 24 hours, and then rotate. This is done for rate-limiting (ultimately for privacy leak reduction I think?).

  Before this change, 24 hours was defined as uint. I replaced it with std::chrono, which is mockable and type-safe.

  Also added couple tests for this behavior.

ACKs for top commit:
  jonatack:
    ACK 2ff8f4dd81

Tree-SHA512: 16f703ef3ffee13ce3afa82ca7b4baa27308af18cd2eececdce5565badfb68656a2ad9c4594b73772e4bfa99b3fb15f8e4089c1cb4be98c0bae6730a9d2f8a25
2022-04-27 18:59:46 +02:00
MacroFake
f58c1f1a44
Merge bitcoin/bitcoin#24739: test: Fix intermittent test failure in wallet_listreceivedby.py
fa1f6df21e test: Fix intermittent test failure in wallet_listreceivedby.py (MarcoFalke)

Pull request description:

  * Remove not needed "Generate block to get out of IBD"
  * Sync blocks where possible to avoid incoming blocks on the p2p `msghand` thread while blocks are mined in the RPC thread. See https://github.com/bitcoin/bitcoin/issues/24730 for discussion.

Top commit has no ACKs.

Tree-SHA512: eca0242e7793886535555fec62f7acd4c0955bf26fab78725b4fe53f84f0b118cb12c9ee35627503fc68b83c3a228842e861fab89aab1226e08e18596357aaae
2022-04-27 08:44:21 +02:00
fanquake
34ae04d775
Merge bitcoin/bitcoin#21726: Improve Indices on pruned nodes via prune blockers
71c3f0356c move-only: Rename index + pruning functional test (Fabian Jahr)
de08932efa test: Update test for indices on pruned nodes (Fabian Jahr)
825d19839b Index: Allow coinstatsindex with pruning enabled (Fabian Jahr)
f08c9fb0c6 Index: Use prune locks for blockfilterindex (Fabian Jahr)
2561823531 blockstorage: Add prune locks to BlockManager (Fabian Jahr)
231fc7b035 refactor: Introduce GetFirstStoredBlock helper function (Fabian Jahr)

Pull request description:

  # Motivation
  The main motivation of this change and only behavior change noticeable by user is to allow running `coinstatsindex` on pruned nodes as has been requested [here for example](https://twitter.com/benthecarman/status/1388170854140452870?s=20).

  # Background
  `coinstatsindex` on pruned nodes can be enabled in a much simpler than it is done here but it comes with downside. The ability to run `blockfilterindex`on pruned nodes was added in #15946 but it also added the `blockfilterindex` as a dependency to `validation` and it introduced two new circular dependencies. Enabling `coinstatsindex` on pruned nodes in a similar way would add it as a dependency as well and introduce another circular dependency.

  Instead, this PR introduces a `m_prune_blockers` map to `BlockManager` as a flexible approach to block pruning. Entities like `blockfilterindex`, for example, can add a key and a height to block pruning over that height. These entities need to update that value to allow more pruning when they are ready.

  # Alternative approach
  Upon completing the first draft of this PR I found #19463 as an alternative that follows the same but follows a very different approach. I am listing the main differences here as I see them:
  - Usage of globals
  - Blocks pruning with a start and a stop height
  - Can persist blockers across restarts
  - Blockers can be set/unset via RPCs

  Personally, I don't think any of these are necessary to be added here but if the general approach or specific features are more appealing to reviewers I am happy to change to a solution based on that PR or port over specific parts of it here.

ACKs for top commit:
  mzumsande:
    Code review ACK 71c3f0356c
  ryanofsky:
    Code review ACK 71c3f0356c. Changes since last review: just tweaking comments and asserts, and rebasing
  w0xlt:
    tACK 71c3f0356c on signet.

Tree-SHA512: de7efda08b44aa31013fbebc47a02cd2de32db170b570f9643e1f013fee0e8e7ca3068952d1acc6e5e74a70910735c5f263437981ad73df841ad945b52d36b71
2022-04-26 19:42:45 +01:00
fanquake
260ede1d99
Merge bitcoin/bitcoin#24644: wallet: add tracepoints and algorithm information to coin selection
ab5af9ca72 test: Add test for coinselection tracepoints (Andrew Chow)
ca02b68e8a doc: document coin selection tracepoints (Andrew Chow)
8e3f39e4fa wallet: Add some tracepoints for coin selection (Andrew Chow)
15b58383d0 wallet: compute waste for SelectionResults of preset inputs (Andrew Chow)
912f1ed181 wallet: track which coin selection algorithm produced a SelectionResult (Andrew Chow)

Pull request description:

  Tracepoints can be useful for coin selection as they would allow us to observe what is being selected, selection parameters, and calculation results. So this PR adds 4 new tracepoints:

  1. After `SelectCoins` returns in order to observe the `SelectionResult`
  2. After the first `CreateTransactionInternal` to observe the created transaction
  3. Prior to the second `CreateTransactionInternal` to notify that the optimistic avoid partial spends selection is occurring
  4. After the second `CreateTransactionInternal` to observe the created transaction and inform which solution is being used.

  This PR also adds an algorithm enum to `SelectionResult` so that the first tracepoint will be able to report which algorithm was used to produce that result.

  The primary use case for these tracepoints is in running coin selection simulations. The script I use to run these simulations use these tracepoints in order to gather data on the algorithm used and the calculated waste.

ACKs for top commit:
  jb55:
    crACK ab5af9ca72
  josibake:
    crACK ab5af9ca72
  0xB10C:
    ACK ab5af9ca72. Code reviewed, ran the `interface_usdt_coinselection.py` test, and tested with the above bpftrace script (updated `%d` -> `%ld` where necessary, ty achow101).

Tree-SHA512: a4bf7a910cdf464622f2f3b5d44c15b891f24852df6e7f8c5b177fe3d8aaa4a1164593a24c3960eb22b16544fa7140e5c745345367b9e291b78395084c0ac8ff
2022-04-26 19:16:27 +01:00
Ryan Ofsky
1d4122dfef init: Allow -proxy="" setting values
This drops the `No proxy server specified. Use -proxy=<ip> or -proxy=<ip:port>`
error when a empty `-proxy=` command line argument, `bitcoin.conf` value, or
`settings.json` value is specified, and just makes bitcoin connect and listen
normally in these cases.

The error was originally added in https://github.com/bitcoin/bitcoin/pull/20003
to prevent a bare `-proxy` command line argument with no assignment from
clearing proxy settings. But it was implemented in an overbroad way breaking
empty `-proxy=` assignments as well.

The motivation for this change is to prevent a GUI bug that happens with
https://github.com/bitcoin/bitcoin/pull/15936, reported in
https://github.com/bitcoin/bitcoin/pull/15936#pullrequestreview-937685759 by
vasild, that happens after a proxy setting is enabled and disabled in the GUI.
But this change also makes sense on its own to remove a potentially confusing
error message.
2022-04-26 10:09:39 -04:00
fanquake
269dcad16e
Merge bitcoin/bitcoin#24789: init, index: disallow indexes when running reindex-chainstate
dac44fc06f init: disallow reindex-chainstate with optional indexes (Martin Zumsande)
62e14285f9 doc: Add note that -reindex will rebuild optional indexes (Martin Zumsande)

Pull request description:

  When started together with `-reindex-chainstate`, currently coinstatsindex gets corrupted and the blockfilterindex flatfiles duplicated. See the OP of #24630 for more a more detailed explanation on why this happens.

  This is an alternative to #24630 which does not wipe and rebuild the indexes but returns an `InitError` when they are activated, thus requiring the user to deactivate them temporarily until the `-reindex-chainstate` run is finished.

  This also disallows `-reindex-chainstate` in combination with `-txindex`, which is not leading to corruption, but currently still rebuilds the index unnecessarily and unexpectedly.

  As a long-term goal, it would be desirable to have the indexes tolerate `reindex-chainstate` by ignoring their `BlockConnected` notifications (there is discussion in #24630 about this) or possibly move `reindex-chainstate` option  into a `bitcoin-chainstate` executable, which could also solve the problem. But these would be larger projects - until then, it might be better to disallow the interaction than having corrupted indexes.

  The first commit adjusts the `-reindex` doc to mention that this option does rebuild all active indexes.

ACKs for top commit:
  ryanofsky:
    Code review ACK dac44fc06f. Just fixed IsArgSet call and edited error messages since last review

Tree-SHA512: c1abf7d350648ae227c3fd6c95d9a54c3bac9de70915275dea1c87cca6d9a76a056c0e306d95ef8cfe4df1f8525b418e0e7a4f52ded3be464041c0dc297f8930
2022-04-26 12:11:39 +01:00
Fabian Jahr
71c3f0356c
move-only: Rename index + pruning functional test 2022-04-25 23:22:00 +02:00
Fabian Jahr
de08932efa
test: Update test for indices on pruned nodes 2022-04-25 23:22:00 +02:00
Fabian Jahr
825d19839b
Index: Allow coinstatsindex with pruning enabled 2022-04-25 23:22:00 +02:00
Sjors Provoost
7e02a33297
rpc: bumpfee signer support 2022-04-25 18:13:23 +02:00
fanquake
aa54132bac
Merge bitcoin/bitcoin#24454: tests: Fix calculation of external input weights
9f5ab670e7 tests: Use descriptor that requires both legacy and segwit (Andrew Chow)
8a04a386f7 tests: Calculate input weight more accurately (Andrew Chow)

Pull request description:

  The external input tests with specifying input weight would sometimes result in a test failure because it would add 2 to the calculated byte size in order to account for some of the variation in signature and script sizes. However 1 in 128 signatures are actually 1 byte smaller than we expect, so the difference between the actual signature size and our calculated size becomes 3 bytes which is outside of the tolerance of `assert_fee_amount` and would thus cause the test failure.

  To resolve this, the 2 byte buffer is reduced to 1 byte, so in the above scenario, the difference is 2 bytes which is within the tolerance of `assert_fee_amount`. Additionally, instead of putting a fixed size that we assume is the correct size for the length of the compact size length prefix of data, we actually get the length of the compact size uint.

  Lastly, the size calculation for a scriptWitness was simply incorrect and used fields that did not exist. This is fixed, and the test slightly modified so that it also produces a scriptWitness.

  Fixes #24151

ACKs for top commit:
  jonatack:
    re-ACK 9f5ab670e7
  glozow:
    code review ACK 9f5ab670e7

Tree-SHA512: b7c7ffe8fb0c07bc9e72fbff1f9ef57ee01a57c56bf54b8873345c8b9572c3ce9402b24dc211910b478114a9e6420faef5a4bf8866f38c299971354e54ec4745
2022-04-25 09:54:40 +01:00
Martin Zumsande
dac44fc06f init: disallow reindex-chainstate with optional indexes
It currently leads to corruption (coinstatsindex) or
data duplication (blockfilterindex), so disable it.
2022-04-24 22:28:25 +02:00
MarcoFalke
b1c5991eeb
Merge bitcoin/bitcoin#24812: util/check: Add CHECK_NONFATAL identity function and NONFATAL_UNREACHABLE macro
ee02c8bd9a util/check: Add CHECK_NONFATAL identity function, NONFATAL_UNREACHABLE AND UNREACHABLE macros (Aurèle Oulès)

Pull request description:

  This PR replaces the macro `CHECK_NONFATAL` with an identity function.
  I simplified the usage of `CHECK_NONFATAL` where applicable in `src/rpc`.
  This function is useful in sanity checks for RPC and command-line interfaces.

  Context: https://github.com/bitcoin/bitcoin/pull/24804#discussion_r846182474.

  Also adds `UNREACHABLE_NONFATAL` macro.

ACKs for top commit:
  jonatack:
    ACK ee02c8bd9a
  MarcoFalke:
    ACK ee02c8bd9a 🍨

Tree-SHA512: 3cba09223cd7b22e62fe5d0b46c4a024c1d9957d4268ba6d3fb07fcc0a5854fc0886bb3266184e6a7df5df91373b3e84edd6adf6999c4e934aeef8c043b01aa2
2022-04-24 12:00:05 +02:00
Sebastian Falbesoner
a498acce45 test: MiniWallet: skip mempool check if mempool_valid=False
MiniWallet's core method for creating txs (`create_self_transfer`)
right now always executes the `testmempoolaccept` RPC to check for
mempool validity or invalidity. In some test cases where we use
MiniWallet to create a huge number of transactions this can lead
to performance issues (e.g. feature_fee_estimation.py where the
execution time after MiniWallet usage almost doubled). Providing
the possibility to skip the mempool checks is a mitigation for
this.

master branch:
$ time ./test/functional/feature_fee_estimation.py
real    3m20.771s
user    2m52.360s
sys     0m39.340s

PR branch:
$ time ./test/functional/feature_fee_estimation.py
real    2m1.386s
user    1m42.510s
sys     0m22.980s
2022-04-22 15:07:10 +02:00
Sebastian Falbesoner
01552e8f67 test: MiniWallet: always rehash after signing (P2PK mode)
Also explicitly rehash in the cases where we modify a tx after signing
in feature_csv_activation.py. Parts of this test relied on the fact that
rehashing of transactions is done in the course of calculating a block's
merkle root (`calc_merkle_root`), which only works if no hash was
calculated before due to a caching mechanism.

In the following commit the txid in MiniWallet is calculated via
`rehash()`, i.e. this doesn't work anymore and we always have to
explicitely have the right hash before we calculate the merkle root.
2022-04-22 15:06:44 +02:00
Jon Atack
734b9669ff test: add getblockfrompeer coverage of invalid inputs 2022-04-22 11:27:15 +02:00
Andrew Chow
9f5ab670e7 tests: Use descriptor that requires both legacy and segwit 2022-04-21 21:00:36 -04:00
MarcoFalke
7a4ac713aa
Merge bitcoin/bitcoin#24936: test: compare /mempool/contents response with getrawmempool RPC
bef61496ab test: compare `/mempool/contents` response with `getrawmempool` RPC (brunoerg)
5bc5cbaf31 doc: add reference to `getrawmempool` RPC in `/mempool/contents` REST doc (brunoerg)

Pull request description:

  This PR is similar to #24797, it compares `/mempool/contents` REST response with `getrawmempool` RPC (verbose=True) since they use the same `MempoolToJSON` function.

  Also, adds a reference to `getrawmempool` RPC help to get details about the fields from `/mempool/contents`.

ACKs for top commit:
  0xB10C:
    ACK bef6149

Tree-SHA512: b7e9e9c765ee837986ba167b9234a9b95c9ef0a9ebcc2a03d50f6be6d3aba1480bd77c78111d95df1e4023cde6dfc64bf1e7908d9e5b6f96ca46b76611a4a9b4
2022-04-21 19:01:00 +02:00
Andrew Chow
ab5af9ca72 test: Add test for coinselection tracepoints 2022-04-21 11:17:00 -04:00
MarcoFalke
fafd67479a
test: Remove previous release check 2022-04-21 14:58:55 +02:00
MarcoFalke
346e780442
Merge bitcoin/bitcoin#24918: test: Remove unused taproot node from wallet_taproot.py
fa2153b05b test: Remove unused taproot node from wallet_taproot.py (MarcoFalke)

Pull request description:

  Now that the wallet considers taproot always active after commit 064c729a96, there is no need to test for it.

ACKs for top commit:
  dunxen:
    Code review ACK fa2153b
  brunoerg:
    crACK fa2153b05b

Tree-SHA512: 24e4a66e43d1391acb63fd0c0c52677b0eef7f618b87a5b1a75224a9be58c9c3f8bba2de3b7510f25a686865b027f7f535e653d40d519d0e00ace38f0c7aba0c
2022-04-21 14:45:22 +02:00
brunoerg
bef61496ab test: compare /mempool/contents response with getrawmempool RPC 2022-04-21 08:31:01 -03:00
MarcoFalke
8d3743a365
Merge bitcoin/bitcoin#24896: test: use MiniWallet for p2p_segwit.py
917a89a814 test: use MiniWallet for p2p_segwit.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (p2p_segwit.py) to be run even with the Bitcoin Code wallet by using the MiniWallet instead, as proposed in https://github.com/bitcoin/bitcoin/issues/20078.

  This change only affects the subtest `test_superfluous_witness`. Note that instead of creating a raw transaction first and then signing it, we go the other direction here: MiniWallet creates a transaction spending a segwit v1 output (i.e. including a witness), then we turn it into a raw transaction by dropping the witness. Therefore, the debug log asserts are swapped.

Top commit has no ACKs.

Tree-SHA512: 163a93a527f60100487f0aff49a9d7baf392ceb4417c54521157b2678685f5728dd751a9747c6cf51666aae78252dd3bc44130e659f7a1262ec1c86e30225622
2022-04-19 14:17:31 +02:00
Hennadii Stepanov
e245c5ccd5
doc: Fix a link to test/lint/lint-python.py 2022-04-19 12:19:32 +02:00
MarcoFalke
fa2153b05b
test: Remove unused taproot node from wallet_taproot.py 2022-04-19 11:57:54 +02:00
Andrew Chow
2095f19db9
Merge bitcoin/bitcoin#24859: wallet: Change wallet validation order
6f29409ad1 test: Add a test that creates a wallet with invalid parameters (w0xlt)
0359d9b6a3 Change wallet validation order (w0xlt)

Pull request description:

  In the current code, the database is created before the last validation, which checks that passphrase is set and private keys are disabled.

  Therefore, if this validation fails, it will result in an empty database and the user will not be able to recreate a wallet with the same name and with the correct parameters.

  Behavior on the master branch:
  ```
  $ ./src/bitcoin-cli -regtest -named createwallet wallet_name="invalid_wallet_01" disable_private_keys=true passphrase="passphrase"
  error code: -4
  error message:
  Passphrase provided but private keys are disabled. A passphrase is only used to encrypt private keys, so cannot be used for wallets with private keys disabled.

  $ ./src/bitcoin-cli -regtest -named createwallet wallet_name="invalid_wallet_01"
  error code: -4
  error message:
  Wallet file verification failed. Failed to create database path '/home/w/.bitcoin/regtest/wallets/invalid_wallet'. Database already exists.
  ```

  Behavior on the PR branch:
  ```
  $ ./src/bitcoin-cli -regtest -named createwallet wallet_name="invalid_wallet_02" disable_private_keys=true passphrase="passphrase"
  error code: -4
  error message:
  Passphrase provided but private keys are disabled. A passphrase is only used to encrypt private keys, so cannot be used for wallets with private keys disabled.

  $ ./src/bitcoin-cli -regtest -named createwallet wallet_name="invalid_wallet_02"
  {
    "name": "invalid_wallet_01",
    "warning": ""
  }
  ```

ACKs for top commit:
  achow101:
    ACK 6f29409ad1

Tree-SHA512: d192955fc2285bf27ae5dd4c1b7cfd3d85441a7f3554b189b974aefb319c6b997543991dbb0ca2c8cb980f7058913a77cf0164c02e9b51ceb9c2cb601317c428
2022-04-18 11:29:29 -04:00
Sebastian Falbesoner
917a89a814 test: use MiniWallet for p2p_segwit.py
This change only affects the subtest `test_superfluous_witness`.

Note that instead of creating a raw transaction first and then
signing it, we go the other direction here: MiniWallet creates a
transaction spending a segwit v1 output (i.e. including a witness),
then we turn it into a raw transaction by dropping the witness.
Therefore, the debug log asserts are swapped.
2022-04-17 18:39:41 +02:00
Sebastian Falbesoner
b167e536d0 test: refactor: use create_lots_of_big_transactions to dedup where possible 2022-04-16 21:37:52 +02:00
Sebastian Falbesoner
8973eeb412 test: use MiniWallet for mining_prioritisetransaction.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-04-16 21:15:29 +02:00
Aurèle Oulès
ee02c8bd9a util/check: Add CHECK_NONFATAL identity function, NONFATAL_UNREACHABLE AND UNREACHABLE macros 2022-04-16 15:07:41 +02:00
fanquake
d1b3dfb275
Merge bitcoin/bitcoin#24855: rpc: Fix setwalletflag disabling of flags
88376c623c test: Test for disabling wallet flags (Andrew Chow)
17ab31aa46 rpc, wallet: setwalletflags warnings are optional (Andrew Chow)

Pull request description:

  Trying to disable a wallet flag with `setwalletflag` results in `Internal bug detected: 'std::any_of(m_results.m_results.begin(), m_results.m_results.end(), [ret](const RPCResult& res) { return res.MatchesType(ret); })'`. This occurs because the `warnings` field was not marked as optional. This PR makes `warnings` optional to avoid this error.

  Also added a test case because apparently we didn't already have one.

ACKs for top commit:
  w0xlt:
    ACK 88376c6

Tree-SHA512: 4f5d3bebf0d022a5ad0f75d70c6562a43c7da6e39e9c3118733327d015c435e2c8d5004fdb039d42407dde5b21231a0f8827623d718abf611a1f06c15af5c806
2022-04-16 10:45:15 +01:00
w0xlt
6f29409ad1 test: Add a test that creates a wallet with invalid parameters
Invalid parameters must not prevent a new wallet with the same name
from being created with the correct parameters
2022-04-16 04:46:22 -03:00
Andrew Chow
88376c623c test: Test for disabling wallet flags 2022-04-14 14:42:34 -04:00
Sebastian Falbesoner
038d2a607f test: add test for signet miner script 2022-04-14 00:28:37 +02:00
Sebastian Falbesoner
449b96ed97 test: add is_bitcoin_util_compiled helper 2022-04-11 21:54:49 +02:00
Sebastian Falbesoner
dde33eca63 test: determine path to bitcoin-util in test framework
The path is stored in `self.options.bitcoinutil`, points to
`src/bitcoin-util` by default and can be overrided with the
`BITCOINUTIL` environment variable.
2022-04-11 21:54:24 +02:00
MarcoFalke
cd110cdd0e
Merge bitcoin/bitcoin#24817: test: use MiniWallet for feature_fee_estimation.py
494455f8a5 test: use MiniWallet for feature_fee_estimation.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (feature_fee_estimation.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in https://github.com/bitcoin/bitcoin/issues/20078. It takes use of the recently introduced methods `{create,send}_self_transfer_multi` (#24637) which allows to specify multiple UTXOs to be spent rather than only one. Very likely the test can still be simplified (e.g. coin selection in `small_txpuzzle_randfee`), but this is a first step.

ACKs for top commit:
  ayush933:
    tACK 494455f8 . The test runs successfully with the wallet disabled.
  vincenzopalazzo:
    tACK 494455f8a5

Tree-SHA512: 89789fc34a4374c79c4b90acd926ac69153aad655dab50450ed796f03c770bd675ad872e906f516f90e8d4cb40b83b55f3c78a94b13bfb8fe8f5e27624937748
2022-04-11 11:18:03 +02:00
MarcoFalke
4a58b6acd2
Merge bitcoin/bitcoin#24797: test: compare /chaininfo response with getblockchaininfo RPC
0f7dc893ea test: compare `/chaininfo` response with `getblockchaininfo` RPC (brunoerg)

Pull request description:

  The `/chaininfo` REST endpoint gets its infos from `getblockchaininfo` RPC, so this PR adds an `assert_equal` (in `interface_rest`) to ensure both responses are the same. Obs: other endpoints do the same for their respective RPC.

ACKs for top commit:
  0xB10C:
    Concept and Code Review ACK 0f7dc893ea. Belts-and-spenders.

Tree-SHA512: 51cbcf988090272e406a47dc869710740b74e2222af29c05ddcbf53bd49765cdc59efb525e970867f091b3d2efec4fb13371a342d9e484e51144b760265bc5b8
2022-04-11 11:04:21 +02:00
Sebastian Falbesoner
494455f8a5 test: use MiniWallet for feature_fee_estimation.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-04-10 02:11:38 +02:00
fanquake
c1059c9fef
Merge bitcoin/bitcoin#24770: Put lock logging behind DEBUG_LOCKCONTENTION preprocessor directive
4394733331 Add DEBUG_LOCKCONTENTION documentation to the developer notes (Jon Atack)
39a34b6877 Put lock logging behind DEBUG_LOCKCONTENTION preprocessor directive (Jon Atack)

Pull request description:

  This is a more minimal, no-frills version of #24734 for backport. The other fixes and improvements in that pull can be done after.

  *Copy of the PR 24734 description:*

  PRs #22736, #22904 and #23223 changed lock contention logging from a `DEBUG_LOCKCONTENTION` compile-time preprocessor directive to a runtime `lock` log category and improved the logging output. This changed the locking from using `lock()` to `try_lock()`:

  - `void Mutex::UniqueLock::lock()` acquires the mutex and blocks until it gains access to it

  - `bool Mutex::UniqueLock::try_lock()` doesn't block but instead immediately returns whether it acquired the mutex; it may be used by `lock()` internally as part of the deadlock-avoidance algorithm

  In theory the cost of `try_lock` might be essentially the [same](https://www.erisian.com.au/bitcoin-core-dev/log-2022-03-31.html#l-697) relative to `lock`. The test-and-set logic of these calls is purported to be ~ constant time, optimised and light/quick if used carefully (i.e. no mutex convoying), compared to system calls, memory/cache coherency and fences, wait queues, and (particularly) lock contentions. See the discussion around https://github.com/bitcoin/bitcoin/pull/22736#issuecomment-902851054 and after with respect to performance/cost aspects.  However, there are reasonable concerns (see [here](https://github.com/bitcoin/bitcoin/pull/22736#discussion_r691277896) and [here](https://www.erisian.com.au/bitcoin-core-dev/log-2022-03-31.html#l-620)) that `Base::try_lock()` may be potentially [costly](https://www.erisian.com.au/bitcoin-core-dev/log-2022-03-31.html#l-700) or [risky](https://github.com/bitcoin/bitcoin/pull/22904#issuecomment-930484001) compared to `Base::lock()` in this very frequently called code.

  One alternative to keep the run-time lock logging would be to gate the `try_lock` call behind the logging conditional, for example as proposed in ccd73de1dd and ACKed [here](https://github.com/bitcoin/bitcoin/pull/22736#issuecomment-901980815). However, this would add the [cost](https://github.com/bitcoin/bitcoin/pull/22736#issuecomment-910102353) of `if (LogAcceptCategory(BCLog::LOCK))` to the hotspot, instead of replacing `lock` with `try_lock`, for the most frequent happy path (non-contention).

  It turns out we can keep the advantages of the runtime lock contention logging (the ability to turn it on/off at runtime) while out of prudence putting the `try_lock()` call and `lock` logging category behind a  `DEBUG_LOCKCONTENTION` compile-time preprocessor directive, and also still retain the lock logging enhancements of the mentioned PRs, as suggested in https://github.com/bitcoin/bitcoin/pull/24734#issuecomment-1085785480 by W. J. van der Laan, in https://github.com/bitcoin/bitcoin/pull/22736#discussion_r691280693, and in the linked IRC discussion.

  Proposed here and for backport to v23.

ACKs for top commit:
  laanwj:
    Code review ACK 4394733331

Tree-SHA512: 89b1271cae1dca0eb251914b1a60fc5b68320aab4a3939c57eec3a33a3c8f01688f05d95dfc31f91d71a6ed80cfe2d67b77ff14742611cc206175e47b2e5d3b1
2022-04-08 13:30:24 +01:00
Jacob P. Fickes
3258bad996 changes color of skipped functional tests
Changes the color of skipped functional tests to the default text color of the terminal. This will make skipped tests easy to read on the majority of background colors rather than the original grey color (hard to read on dark backgrounds) and the proposed yellow change (hard to read on white backgrounds)
2022-04-07 12:17:01 -04:00
Pieter Wuille
395767e9f1 Add test case mimicking issue 24765 2022-04-07 11:40:58 -04:00
brunoerg
0f7dc893ea test: compare /chaininfo response with getblockchaininfo RPC 2022-04-06 17:59:37 -03:00
fanquake
10f629e644
Merge bitcoin/bitcoin#24576: contrib: testgen: remove redundant base58 implementation
65c49ac750 test: throw `ValueError` for invalid base58 checksum (Sebastian Falbesoner)
219d2c7ee1 contrib: testgen: use base58 methods from test framework (Sebastian Falbesoner)
605fecfb66 scripted-diff: rename `chars` to `b58chars` in test_framework.address (Sebastian Falbesoner)
11c63e374d contrib: testgen: import OP_* constants from test framework (Sebastian Falbesoner)
7d755bb31c contrib: testgen: avoid need for manually setting PYTHONPATH (Sebastian Falbesoner)

Pull request description:

  This PR removes the redundant base58 implementation [contrib/testgen/base58.py](https://github.com/bitcoin/bitcoin/blob/master/contrib/testgen/base58.py) for the test generation script `gen_key_io_test_vectors.py` and uses the one from the test framework instead. Additionally, three other cleanups/improvements are done:
  - import script operator constants `OP_*` from test framework instead of manually defining them
  - add Python path to test framework directly in the script (via `sys.path.append(...)`) instead of needing the caller to specify `PYTHONPATH=...` on the command line (the same approach is done for the signet miner and the message capture scripts)
  - rename `chars` to `b58chars` in the test_framework.address module (is more explicit and makes the diff for the base58 replacement smaller)

ACKs for top commit:
  laanwj:
    Code review ACK 65c49ac750

Tree-SHA512: 92e1534cc320cd56262bf455de7231c6ec821bfcd0ed58aa5718271ecec1a89df7951bf31527a2306db6398e7f2664d2ff8508200c28163c0b164d3f5aaf8b0e
2022-04-06 14:03:00 +01:00
laanwj
6c9460edae
Merge bitcoin/bitcoin#24358: test: USDT tracepoint interface tests
76c60d7b31 test: validation:block_connected tracepoint test (0xb10c)
260e28ece8 test: utxocache:* tracepoint tests (0xb10c)
34b27bac68 test: net:in/out_message tracepoint tests (0xb10c)
c934087b62 test: checks for tracepoint tests (0xb10c)

Pull request description:

  This adds functional tests for the USDT tracepoints added in https://github.com/bitcoin/bitcoin/pull/22006 and https://github.com/bitcoin/bitcoin/pull/22902. This partially fixes #23296. The tests **are probably skipped** on most systems as these tests require:
  - a Linux system with a kernel that supports BPF (and available kernel headers)
  - that Bitcoin Core is compiled with tracepoints for USDT support (default when compiled with depends)
  - [bcc](https://github.com/iovisor/bcc) installed
  - the tests are run with a privileged user that is able to e.g. do BPF syscalls and load BPF maps

  The tests are not yet run in our CI as the CirrusCI containers lack the required permissions (see https://github.com/bitcoin/bitcoin/issues/23296#issuecomment-1024920845). Running the tests in a VM in the CI could work, but I haven't experimented with this yet. The priority was to get the actual tests done first to ensure the tracepoints work as intended for the v23.0 release. Running the tracepoint tests in the CI is planned as the next step to finish #23296.

  The tests can, however, be run against e.g. release candidates by hand. Additionally, they provide a starting point for tests for future tracepoints. PRs adding new tracepoint should include tests. This makes reviewing these PRs easier.

  The tests require privileges to execute BPF sycalls (`CAP_SYS_ADMIN` before Linux kernel 5.8 and `CAP_BPF` and `CAP_PERFMON` on 5.8+) and permissions to `/sys/kernel/debug/tracing/`. It's currently recommended to run the tests in a virtual machine (or on a VPS) where it's sensible to use the `root` user to gain these privileges. Never run python scripts you haven't carefully reviewed with `root` permissions! It's unclear if a non-root user can even gain the required privileges. This needs more experimenting.

  The goal here is to test the tracepoint interface to make sure the [documented interface](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#tracepoint-documentation) does not break by accident. The tracepoints expose implementation details. This means we also need to rely on implementation details of Bitcoin Core in these functional tests to trigger the tracepoints. An example is the test of the `utxocache:flush` tracepoint: On Bitcoin Core shutdown, the UTXO cache is flushed twice. The corresponding tracepoint test expects two flushes, too - if not, the test fails. Changing implementation details could cause these tests to fail and the tracepoint API to break. However, we purposefully treat the tracepoints only as [**semi-stable**](https://github.com/bitcoin/bitcoin/blob/master/doc/tracing.md#semi-stable-api). The tracepoints should not block refactors or changes to other internals.

ACKs for top commit:
  jb55:
    tACK 76c60d7b31
  laanwj:
    Tested ACK 76c60d7b31

Tree-SHA512: 9a63d945c68102e59d751bd8d2805ddd7b37185408fa831d28a9cb6641b701961389b55f216c475df7d4771154e735625067ee957fc74f454ad7a7921255364c
2022-04-06 13:07:26 +02:00
MarcoFalke
79bf1a0fa2
Merge bitcoin/bitcoin#24732: Remove buggy and confusing IncrementExtraNonce
cccc4e879a Remove nHeightEnd and nHeight in generateBlocks helper (MarcoFalke)
fa38b1c8bd Remove buggy and confusing IncrementExtraNonce (MarcoFalke)

Pull request description:

  IncrementExtraNonce has many issues:

  * It is test-only code, but part of bitcoind
  * It is using the block height of the tip, as opposed to the block's previous block as reference for the new height. See https://github.com/bitcoin/bitcoin/issues/24730#issuecomment-1085586193
  * It has no use case in regtest testing. With a low difficulty the extra nonce won't be incremented. With a high difficulty the test-only functions are clumsy to handle anyway. For example, the generate* RPCs will return an empty array once they reached `maxtries`, as opposed to an error. Also the calls can't be aborted early unless the node shuts down completely. So I think it is fine to just remove the extra nonce functionality and leave it to the outside to implement, if needed. For example, a wrapper script can call the `generate*` RPCs once every second, to use the timestamp as extra nonce.

ACKs for top commit:
  ajtowns:
    ACK cccc4e879a

Tree-SHA512: d8a3989ad280ebd4b1b574159b3a396b8a42134347e6be3c88445162d86624d221c416456f45ae75aea62ed8c8a1a9bb3a2532924abca2ef7a879cb8e6b15654
2022-04-06 11:12:10 +02:00
MarcoFalke
27cfaeed1e
Merge bitcoin/bitcoin#24098: rest: Use query parameters to control resource loading
54b39cfb34 Add release notes (stickies-v)
f959fc0397 Update /<count>/ endpoints to use a '?count=' query parameter instead (stickies-v)
a09497614e Add GetQueryParameter helper function (stickies-v)
fff771ee86 Handle query string when parsing data format (stickies-v)
c1aad1b3b9 scripted-diff: rename RetFormat to RESTResponseFormat (stickies-v)
9f1c54787c Refactoring: move declarations to rest.h (stickies-v)

Pull request description:

  In RESTful APIs, [typically](https://rapidapi.com/blog/api-glossary/parameters/query/) path parameters  (e.g. `/some/unique/resource/`) are used to represent resources, and query parameters (e.g. `?sort=asc`) are used to control how these resources are being loaded through e.g. sorting, pagination, filtering, ...

  As first [discussed in #17631](https://github.com/bitcoin/bitcoin/pull/17631#discussion_r733031180), the [current REST api](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md) contains two endpoints `/headers/` and `/blockfilterheaders/` that rather unexpectedly use path parameters to control how many (filter) headers are returned in the response. While this is no critical issue, it is unintuitive and we are still early enough to easily phase this behaviour out and ensure new endpoints (if any) do not have to stick to non-standard behaviour just for internal consistency.

  In this PR, a new `HTTPRequest::GetQueryParameter` method is introduced to easily parse query parameters, as well as two new `/headers/` and `/blockfilterheaders/` endpoints that use a count query parameter are introduced. The old path parameter-based endpoints are kept without too much overhead, but the documentation now points to the new query parameter-based endpoints as the default interface to encourage standardness.

  ## Behaviour change
  ### New endpoints and default values
  `/headers/` and `/blockfilterheaders/` now have 2 new endpoints that contain query parameters (`?count=<count>`) instead of path parameters (`/<count>/`), as described in REST-interface.md. Since query parameters can easily have default values, I have set this at 5 for both endpoints.

  **headers**
  `GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  **blockfilterheaders**
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  ### Some previously invalid API calls are now valid
  API calls that contained query strings in the URI could not be parsed prior to this PR. This PR changes behaviour in that previously invalid calls (e.g. `GET /rest/headers/5/somehash.json?someunusedparam=foo`) would now become valid, as the query parameters are properly parsed, and discarded if unused.
  For example, prior to this PR, adding an irrelevant `someparam` parameter would be illegal:
  ```
  GET /rest/headers/5/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ->
  Invalid hash: 0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ```
  **This behaviour change affects all rest endpoints, not just the 2 new ones introduced here.**

  *(Note: I'd be open to implementing additional logic to refuse requests containing unrecognized query parameters to minimize behaviour change, but for the endpoints that we currently have I don't really see the point for that added complexity. E.g. I don't see any scenarios where misspelling a parameter could lead to harmful outcomes)*

  ## Using the REST API

  To run the API HTTP server, start a bitcoind instance with the `-rest` flag enabled. To use the
  `blockfilterheaders` endpoint, you'll also need to set `-blockfilterindex=1`:
  ```
  ./bitcoind -signet -rest -blockfilterindex=1
  ```

  As soon as bitcoind is fully up and running, you should be able to query the API, for example by
  using curl on the command line: ```curl "127.0.0.1:38332/rest/chaininfo.json"```.
  To more easily parse the JSON output, you can also use tools like 'jq' or `json_pp`, e.g.:
  ```
  curl -s "localhost:38332/rest/blockfilterheaders/basic/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?count=2" | json_pp .
  ```

  ## To do
  - [x] update `doc/release-notes`

  ## Feedback
  This is my first PR (hooray!). Please don't hold back on any feedback/comments/nits/... you may have, big or small, whether they are code, process, language, ... related. I welcome private messages too if there's anything you don't want to clutter the PR with. I'm here to learn and am grateful for everyone's input.

ACKs for top commit:
  stickies-v:
    I've had to push a tiny doc update to `REST-interface.md` (`git range-diff 219d728 9aac438 54b39cf`) since this was not merged for v23, but since there are no significant changes beyond theStack and jnewbery's ACKs I think this PR is now ready to be considered for merging? @MarcoFalke
  jnewbery:
    ACK 54b39cfb34
  theStack:
    re-ACK 54b39cfb34

Tree-SHA512: 3b393ffde34f25605ca12c0b1300799a19684b816a1d03aed38b0f5439df47bfe6a589ffbcd7b83fd2def6c9d00a1bae5e45b1d18df4ae998c617c709990f83f
2022-04-06 09:25:56 +02:00
Sebastian Falbesoner
65c49ac750 test: throw ValueError for invalid base58 checksum 2022-04-05 20:07:33 +02:00
Sebastian Falbesoner
605fecfb66 scripted-diff: rename chars to b58chars in test_framework.address
-BEGIN VERIFY SCRIPT-
sed -i 's/chars/b58chars/g' ./test/functional/test_framework/address.py
-END VERIFY SCRIPT-
2022-04-05 19:51:40 +02:00
stickies-v
f959fc0397
Update /<count>/ endpoints to use a '?count=' query parameter instead
In most RESTful APIs, path parameters are used to represent resources, and
query parameters are used to control how these resources are being filtered/sorted/...

The old /<count>/ functionality is kept alive to maintain backwards compatibility,
but new paths with query parameters are introduced and documented as the default
interface so future API methods don't break consistency by using query parameters.
2022-04-05 13:19:37 -04:00
laanwj
f421de5be6
Merge bitcoin/bitcoin#24236: Remove utxo db upgrade code
fa9112aac0 Remove utxo db upgrade code (MarcoFalke)

Pull request description:

  It is not possible to upgrade Bitcoin Core pre-segwit (pre-0.13.1) to a recent version without a full IBD from scratch after  commit 19a56d1519 (released in version 22.0).

  Any Bitcoin Core version with the new database format after commit 1088b02f0c (released in version 0.15), can upgrade to any version that is supported as of today.

  This leaves the versions 0.13.1-0.14.x. Even though those versions are unsupported, some users with an existing datadir may want to upgrade to a recent version. However, it seems reasonable to simply ask them to `-reindex` to run a full IBD from scratch. This allows us to remove the utxo db upgrade code.

ACKs for top commit:
  Sjors:
    re-ACK fa9112aac0
  laanwj:
    Code review ACK fa9112aac0

Tree-SHA512: 4243bb35df9ac4892f9fad30fe486d338745952bcff4160bcb0937c772d57b13b800647da14695e21e3655e85ee0d95fa3dc7789ee309d59ad84f422297fecb8
2022-04-05 15:38:14 +02:00
Jon Atack
39a34b6877
Put lock logging behind DEBUG_LOCKCONTENTION preprocessor directive 2022-04-05 12:49:48 +02:00
Ayush Sharma
d2ba43fec8 test: use MiniWallet for mempool_unbroadcast.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-04-04 19:06:29 +05:30
MarcoFalke
fa1f6df21e
test: Fix intermittent test failure in wallet_listreceivedby.py 2022-04-01 15:19:17 +02:00
MarcoFalke
fa38b1c8bd
Remove buggy and confusing IncrementExtraNonce 2022-04-01 11:00:42 +02:00
MarcoFalke
c8ac7e6a65
Merge bitcoin/bitcoin#24698: test: -peerblockfilters without -blockfilterindex raises an error
d6bc2322ed test: -peerblockfilters without -blockfilterindex raises an error (brunoerg)

Pull request description:

  This PR adds test coverage for the following init error:
  2a3e8fb359/src/init.cpp (L850)

  Setting -peerblockfilters without -blockfilterindex should raise an error when initializing.

ACKs for top commit:
  ccdle12:
    Tested ACK d6bc2322ed

Tree-SHA512: e740c2ccde6bb1bb8381bb676a6d01bd5746cf9ce0c8dadd62067a6b9b380027bfe8b8cdeae9846a0ab18385f3dc5dff607fe5274cb55107d47470db00015fb2
2022-03-31 08:37:30 +02:00
MarcoFalke
f4e5d704f2
Merge bitcoin/bitcoin#24118: Add 'sendall' RPC née sweep
bb84b7145b add tests for no recipient and using send_max while inputs are specified (ishaanam)
49090ec402 Add sendall RPC née sweep (Murch)
902793c777 Extract FinishTransaction from send() (Murch)
6d2208a3f6 Extract interpretation of fee estimation arguments (Murch)
a31d75e5fb Elaborate error messages for outdated options (Murch)
35ed094e4b Extract prevention of outdated option names (Murch)

Pull request description:

  Add sendall RPC née sweep

  _Motivation_
  Currently, the wallet uses a fSubtractFeeAmount (SFFO) flag on the
  recipients objects for all forms of sending calls. According to the
  commit discussion, this flag was chiefly introduced to permit sweeping
  without manually calculating the fees of transactions. However, the flag
  leads to unintuitive behavior and makes it more complicated to test
  many wallet RPCs exhaustively. We proposed to introduce a dedicated
  `sendall` RPC with the intention to cover this functionality.

  Since the proposal, it was discovered in further discussion that our
  proposed `sendall` rpc and SFFO have subtly different scopes of
  operation.
  • sendall:
    Use _given UTXOs_ to pay a destination the remainder after fees.
  • SFFO:
    Use a _given budget_ to pay an address the remainder after fees.

  While `sendall` will simplify cases of spending a given set of
  UTXOs such as paying the value from one or more specific UTXOs, emptying
  a wallet, or burning dust, we realized that there are some cases in
  which SFFO is used to pay other parties from a limited budget,
  which can often lead to the creation of change outputs. This cannot be
  easily replicated using `sendall` as it would require manual
  computation of the appropriate change amount.

  As such, sendall cannot replace all uses of SFFO, but it still has a
  different use case and will aid in simplifying some wallet calls and
  numerous wallet tests.

  _Sendall call details_
  The proposed sendall call builds a transaction from a specific
  subset of the wallet's UTXO pool (by default all of them) and assigns
  the funds to one or more receivers. Receivers can either be specified
  with a given amount or receive an equal share of the remaining
  unassigned funds. At least one recipient must be provided without
  assigned amount to collect the remainder. The `sendall` call will
  never create change. The call has a `send_max` option that changes the
  default behavior of spending all UTXOs ("no UTXO left behind"), to
  maximizing the output amount of the transaction by skipping uneconomic
  UTXOs. The `send_max` option is incompatible with providing a specific
  set of inputs.

  ---
  Edit: Replaced OP with latest commit message to reflect my updated motivation of the proposal.

ACKs for top commit:
  achow101:
    re-ACK bb84b7145b

Tree-SHA512: 20aaf75d268cb4b144f5d6437d33ec7b5f989256b3daeeb768ae1e7f39dc6b962af8223c5cb42ecc72dc38cecd921c53c077bc0ec300b994e902412213dd2cc3
2022-03-30 15:02:49 +02:00
ishaanam
bb84b7145b
add tests for no recipient and using send_max while inputs are specified 2022-03-29 16:37:49 -04:00
Murch
49090ec402
Add sendall RPC née sweep
_Motivation_
Currently, the wallet uses a fSubtractFeeAmount (SFFO) flag on the
recipients objects for all forms of sending calls. According to the
commit discussion, this flag was chiefly introduced to permit sweeping
without manually calculating the fees of transactions. However, the flag
leads to unintuitive behavior and makes it more complicated to test
many wallet RPCs exhaustively. We proposed to introduce a dedicated
`sendall` RPC with the intention to cover this functionality.

Since the proposal, it was discovered in further discussion that our
proposed `sendall` rpc and SFFO have subtly different scopes of
operation.
• sendall:
  Use _specific UTXOs_ to pay a destination the remainder after fees.
• SFFO:
  Use a _specific budget_ to pay an address the remainder after fees.

While `sendall` will simplify cases of spending from specific UTXOs,
emptying a wallet, or burning dust, we realized that there are some
cases in which SFFO is used to pay other parties from a limited budget,
which can often lead to the creation of change outputs. This cannot be
easily replicated using `sendall` as it would require manual computation
of the appropriate change amount.

As such, sendall cannot replace all uses of SFFO, but it still has a
different use case and will aid in simplifying some wallet calls and
numerous wallet tests.

_Sendall call details_
The proposed sendall call builds a transaction from a specific subset of
the wallet's UTXO pool (by default all of them) and assigns the funds to
one or more receivers. Receivers can either be specified with a specific
amount or receive an equal share of the remaining unassigned funds. At
least one recipient must be provided without assigned amount to collect
the remainder. The `sendall` call will never create change. The call has
a `send_max` option that changes the default behavior of spending all
UTXOs ("no UTXO left behind"), to maximizing the output amount of the
transaction by skipping uneconomic UTXOs. The `send_max` option is
incompatible with providing a specific set of inputs.
2022-03-29 16:37:47 -04:00
brunoerg
d6bc2322ed test: -peerblockfilters without -blockfilterindex raises an error 2022-03-28 15:28:37 -03:00
MarcoFalke
3297f5c11c
Merge bitcoin/bitcoin#24623: test: Add diamond-shape prioritisetransaction test
fa0758e145 test: Add diamond-shape prioritisetransaction test (MarcoFalke)
fa450c18db test: Rework create_self_transfer_multi (MarcoFalke)

Pull request description:

  Looks like there is no test for diamonds, only for chains (in `mempool_packages.py`)

ACKs for top commit:
  jamesob:
    ACK fa0758e145

Tree-SHA512: d261184a81df77d24fc256f58ad5ed4a13b7cd4e33f74c8b79495c761ff417817602d8e5d4f63f4bb1000ac63f89bbfa54d8d8994a7b2bb2e8a484c467330984
2022-03-28 09:09:34 +02:00
MarcoFalke
161dd7e864
Merge bitcoin/bitcoin#24687: test: Check an invalid -i2psam will raise an init error
45e67b2695 test: invalid -i2psam will raise an init error (brunoerg)

Pull request description:

  This PR adds test coverage (at `feature_proxy.py`) for the following init error:

  2f0f056e08/src/init.cpp (L1791)

  It starts the node with an invalid -i2psam (`-i2psam=invalidhere`) and test if it raises an error when initializing.

ACKs for top commit:
  dunxen:
    Code review ACK 45e67b2

Tree-SHA512: b24e3f6e7a9316b9ebc0b6c8bcf1315faff60a9e258d7bb3dbeb9f6695a728bb3083aea2f81114072fe13822bfca34d4a0f44f229825f7c97a81619d810010c0
2022-03-28 09:07:22 +02:00
MarcoFalke
88709e9dc5
Merge bitcoin/bitcoin#24258: test: check localaddresses in getnetworkinfo for nodes with proxy
89bb25d22a test: check localaddresses in getnetworkinfo for nodes with proxy (brunoerg)

Pull request description:

  This PR adds test coverage for the field `localaddresses` for `getnetworkinfo`. In this case, it verifies if this field is empty for all nodes since they are using proxy.

  Reference:
  515200298b/src/init.cpp (L449)

ACKs for top commit:
  jonatack:
    ACK 89bb25d22a

Tree-SHA512: 3c765c7060b6972c1ae5a1104734cd7669b650b5f6aa4f623f4299567732260da5083fef306a7c1e71c931f5d1396f24abad251d95c3d82b1f3ee0efee7fcd1f
2022-03-28 09:00:11 +02:00
brunoerg
45e67b2695 test: invalid -i2psam will raise an init error 2022-03-26 21:50:34 -03:00
fanquake
6d5771ba07
Merge bitcoin/bitcoin#24494: wallet: generate random change target for each tx for better privacy
9053f64fcb [doc] release notes for random change target (glozow)
46f2fed6c5 [wallet] remove MIN_CHANGE (glozow)
a44236addd [wallet] randomly generate change targets (glozow)
1e52e6bd0a refactor coin selection for parameterizable change target (glozow)

Pull request description:

  Closes #24458 - the wallet always chooses 1 million sats as its change target, making it easier to fingerprint transactions created by the Core wallet. Instead of using a fixed value, choose one randomly each time (within a range). Using 50ksat (around $20) as the lower bound and `min(1 million sat, 2 * average payment value)` as the upper bound.
  RFC: If the payment is <25ksat, this doesn't work, so we're using the range (payment amount, 50ksat) instead.

ACKs for top commit:
  achow101:
    ACK 9053f64fcb
  Xekyo:
    reACK 9053f64fcb

Tree-SHA512: 45ce5d064697065549473347648e29935733f3deffc71a6ab995449431f60302d1f9911a0994dfdb960b48c48b5d8859f168b396ff2a62db67d535a7db041d35
2022-03-25 21:03:32 +00:00
MarcoFalke
f66c827c2d
Merge bitcoin/bitcoin#24502: wallet: don't create long chains by default
da2bc865d6 [wallet] don't create long chains by default (glozow)

Pull request description:

  Default mempool policy doesn't let you have chains longer than 25 transactions. This is locally configurable of course, but it's not really safe to assume that a chain longer than 25 transactions will propagate. Thus, the wallet should probably avoid creating such transactions by default; set `DEFAULT_WALLET_REJECT_LONG_CHAINS` to true.

  Closes #9752
  Closes #10004

ACKs for top commit:
  MarcoFalke:
    re-ACK da2bc865d6 only change is fixing typos in tests 🎏

Tree-SHA512: 65d8e4ec437fe928adf554aa7e819a52e0599b403d5310895f4e371e99bbc838219b3097c4d2f775bc870ac617ef6b4227b94291f2b376f824f14e8f2b152f31
2022-03-25 17:16:13 +01:00
glozow
da2bc865d6 [wallet] don't create long chains by default 2022-03-25 16:02:37 +00:00
MarcoFalke
9745e18051
Merge bitcoin/bitcoin#24670: test: move-only: Move all generate* tests to a single file
0000ff0d6b test: move-only: Move all generate* tests to a single file (MarcoFalke)

Pull request description:

  Seems a bit overkill to spread tests for the `generate*` methods over several files. Combining them into a single file has also a nice side-effect of requiring less node (re)starts, which are expensive in valgrind.

ACKs for top commit:
  glozow:
    utACK 0000ff0d6b

Tree-SHA512: 8269eb05649a871011bbfbd1838d0f7d1dac4a35b3b198fc43fe85131fda8a53803b75da78cbf422eabf086006dee4421e622fbe706f6781a3848b989024001b
2022-03-25 16:53:00 +01:00
fanquake
9344697e57
Merge bitcoin/bitcoin#21160: net/net processing: Move tx inventory into net_processing
1066d10f71 scripted-diff: rename TxRelay members (John Newbery)
575bbd0dea [net processing] Move tx relay data to Peer (John Newbery)
785f55f7ee [net processing] Move m_wtxid_relay to Peer (John Newbery)
36346703f8 [net] Add CNode.m_relays_txs and CNode.m_bloom_filter_loaded (John Newbery)

Pull request description:

  This continues the work of moving application layer data into net_processing, by moving all tx data into the new Peer object added in #19607.

  For motivation, see #19398.

ACKs for top commit:
  dergoegge:
    ACK 1066d10f71 - This is a good layer separation improvement with no behavior changes.
  glozow:
    utACK 1066d10f71

Tree-SHA512: 0c9d6b8a0a05e2d816b6d6588b7df133842ec960ae67667813422aa7bd8eb5308599c714f3822a98ddbdf364ffab9050b055079277ba4aff24092557ff99ebcc
2022-03-25 15:16:00 +00:00
laanwj
7c08d81e11
Merge bitcoin/bitcoin#23536: Enforce Taproot script flags whenever WITNESS is set
cccc1e70b8 Enforce Taproot script flags whenever WITNESS is set (MarcoFalke)
fa42299411 Remove nullptr check in GetBlockScriptFlags (MarcoFalke)
faadc606c7 refactor: Pass const reference instead of pointer to GetBlockScriptFlags (MarcoFalke)

Pull request description:

  Now that Taproot is active, it makes sense to enforce its rules on all blocks, even historic ones, regardless of the deployment status.

  ### Benefits:

  (With "script flags" I mean "taproot script verification flags".)

  * Script flags are known ahead for all blocks (even blocks not yet created) and do not change. This may benefit static analysis, code review, and development of new script features that build on Taproot.
  * Any future bugs introduced in the deployment code won't have any effect on the script flags, as they are independent of deployment.
  * Enforcing the taproot rules regardless of the deployment status makes testing easier because invalid blocks after activation are also invalid before activation. So there is no need to differentiate the two cases.
  * It gives belt-and-suspenders protection against a practically expensive and theoretically impossible IBD reorg attack where the node is eclipsed. While `nMinimumChainWork` already protects against this, the cost for a few months worth of POW might be lowered until a major version release of Bitcoin Core reaches EOL. The needed work for the attack is the difference between `nMinimumChainWork` and the work at block 709632.

  For reference, previously the same was done for P2SH and WITNESS in commit 0a8b7b4b33.

  ### Implementation:

  I found one block which fails verification with the flags applied, so I added a `TaprootException`, similar to the `BIP16Exception`.

  For reference, the debug log:

  ```
  ERROR: ConnectBlock(): CheckInputScripts on b10c007c60e14f9d087e0291d4d0c7869697c6681d979c6639dbd960792b4d41 failed with non-mandatory-script-verify-flag (Witness program was passed an empty witness)
  BlockChecked: block hash=0000000000000000000f14c35b2d841e986ab5441de8c585d5ffe55ea1e395ad state=non-mandatory-script-verify-flag (Witness program was passed an empty witness)
  InvalidChainFound: invalid block=0000000000000000000f14c35b2d841e986ab5441de8c585d5ffe55ea1e395ad  height=692261  log2_work=92.988459  date=2021-07-23T08:24:20Z
  InvalidChainFound:  current best=0000000000000000000067b17a4c0ffd77c29941b15ad356ca8f980af137a25d  height=692260  log2_work=92.988450  date=2021-07-23T07:47:31Z
  ERROR: ConnectTip: ConnectBlock 0000000000000000000f14c35b2d841e986ab5441de8c585d5ffe55ea1e395ad failed, non-mandatory-script-verify-flag (Witness program was passed an empty witness)
  ```

  Hint for testing, make sure to set `-noassumevalid`.

  ### Considerations

  Obviously this change can lead to consensus splits on the network in light of massive reorgs. Currently the last block before Taproot activation, that is the last block without the Taproot script flags set, is only buried by a few days of POW. However, when and if this patch is included in the next major release, it will be buried by a few months of POW. BIP90 considerations apply when looking at reorgs this large.

ACKs for top commit:
  Sjors:
    tACK cccc1e70b8
  achow101:
    ACK cccc1e70b8
  laanwj:
    Code review ACK cccc1e70b8
  ajtowns:
    ACK cccc1e70b8 ; code review; wrote a "getblockscriptflags" rpc to quickly check that blocks just had bit 17 (taproot) added; review of earlier revisions had established non-exception blocks do validate with taproot rules enabled.
  jamesob:
    ACK cccc1e70b8 ([`jamesob/ackr/23536.1.MarcoFalke.enforce_taproot_script_f`](https://github.com/jamesob/bitcoin/tree/ackr/23536.1.MarcoFalke.enforce_taproot_script_f))

Tree-SHA512: 00044de68939caef6420ffd588c1291c041a8b397c80a3df1e3e3487fbeae1821d23975c51c95e44e774558db76f943b00b4e27cbd0213f64a9253116dc6edde
2022-03-25 14:11:18 +01:00
glozow
a44236addd [wallet] randomly generate change targets
If the wallet always chooses 1 million sats as its change target, it is
easier to fingerprint transactions created by the Core wallet.
2022-03-25 11:56:46 +00:00
MarcoFalke
0000ff0d6b
test: move-only: Move all generate* tests to a single file
Can be reviewed with
--color-moved=dimmed-zebra
2022-03-25 11:55:49 +01:00
MarcoFalke
f0c9ba2b48
Merge bitcoin/bitcoin#24205: init, test: improve network reachability test coverage and safety
58a14795b8 test: passing -onlynet=onion with -onion=0/-noonion raises expected init error (Jon Atack)
7000f66d36 test: passing -onlynet=onion without -proxy/-onion raises expected init error (Jon Atack)
8332e6e4cf test: passing invalid -onion raises expected init error (Jon Atack)
d5edb08708 test: passing invalid -proxy raises expected init error (Jon Atack)
bd57dcbaf2 test: hoist proxy out of 2 network loops in feature_proxy.py (Jon Atack)
afdf2de282 test: add CJDNS to LimitedAndReachable_Network unit tests (Jon Atack)
2b7a8180a9 net, init: assert each network reachability is true by default (Jon Atack)

Pull request description:

  Adds missing network reachability test coverage and an assertion during init, noticed while reviewing #22834:

  - assert during init that each network reachability is  true by default
  - add CJDNS to the `LimitedAndReachable_Network` unit tests
  - hoist proxy out of two network loops in feature_proxy.py
  - test that passing invalid `-proxy` raises expected init error
  - test that passing invalid `-onion` raises expected init error
  - test that passing `-onlynet=onion` without `-proxy` and `-onion` raises expected init error
  - test that passing `-onlynet=onion` with `-onion=0` and with `-noonion` raises expected init error

ACKs for top commit:
  vasild:
    ACK 58a14795b8
  brunoerg:
    ACK 58a14795b8
  dongcarl:
    Code Review ACK 58a14795b8

Tree-SHA512: bdee6dd0c12bb63591ce7c9321fe77b509ab1265123054e774adc38a187746dddafe1627cbe89e990bcc78b45e194bfef8dc782710d5b217e2e2106ab0158827
2022-03-24 21:17:46 +01:00
MarcoFalke
a697a3fc91
Merge bitcoin/bitcoin#24653: test: use MiniWallet in test/functional/interface_zmq
bc90b8d869 [move only] remove `is_wallet_compiled` checks (josibake)
0bfbf7fb24 test: use MiniWallet in `interfaces_zmq` (josibake)

Pull request description:

  While working on #24584 , `interface_zmq` started failing due to coin selection not running deterministically. The test doesn't actually need the wallet, so this PR migrates it to use MiniWallet

  _Note for reviewers:_ the second commit moves large chunks of code out of an if block, so it may be helpful to review with something that ignores whitespace, e.g `git diff -w master`

ACKs for top commit:
  vincenzopalazzo:
    ACK bc90b8d869

Tree-SHA512: c618e23d00635d72dafdef28e68cbc88b9cc2030d4898fc5b7eac926fd621684c1958c075ed167192716b18308da5a0c1f1393396e31b99d0d3bde78b78fefc5
2022-03-24 19:57:48 +01:00
Andrew Chow
8a04a386f7 tests: Calculate input weight more accurately
The external input test with specifying input weight would make a
pessimistic estimate of the input weight. However this would result in a
test failure as it is sometimes too pessimistic when an ECDSA signature
ends up being smaller than usual. To correct this, we can calculate the
input weight more accurately.
2022-03-24 11:49:56 -04:00
MarcoFalke
fa0758e145
test: Add diamond-shape prioritisetransaction test 2022-03-24 14:33:41 +01:00
MarcoFalke
fa450c18db
test: Rework create_self_transfer_multi
* Add fallback for utxos_to_spend if none are provided
* Refactor a for-loop
2022-03-24 14:33:28 +01:00
MarcoFalke
4a0ab355b3
Merge bitcoin/bitcoin#24626: init: disallow reindex-chainstate when pruning
b2813980b8 init: disallow reindex-chainstate when pruning (Martin Zumsande)

Pull request description:

  The combination of `-reindex-chainstate` and `-prune` currently makes the node stuck in an endless loop:

  - `LoadChainstate()` will wipe the existing chainstate (so we have no genesis block anymore). It won't clean up unusable block files by calling `CleanupBlockRevFiles()` as for full `-reindex`.
  - `ThreadImport()` has [logic](91d12344b1/src/node/blockstorage.cpp (L855)) of reloading Genesis after reindexing. This is what makes full `-reindex` work with `-prune` but it's not executed for `-reindex-chainstate`.
  - Since we still don't have a genesis block, init will wait for it forever in an endless loop ([code](91d12344b1/src/init.cpp (L1630-L1640))).

  Fix this by disallowing `-reindex-chainstate` together with `-prune`. This is discouraged in the help for `-reindex-chainstate` anyway ("When in pruning mode or if blocks on disk might be corrupted, use full -reindex instead.") but wasn't enforced.

  Fixes #24242

ACKs for top commit:
  MarcoFalke:
    cr ACK b2813980b8

Tree-SHA512: 7220842daaf9a4f972d82b13b81fdeac2833bf5e665c5b0f8eaf6a4bcd0725c8e97d19ec956ca4b730065a983475bb3a2732713d338f4caf8666ccbf63d4d988
2022-03-24 14:27:13 +01:00
Martin Zumsande
b2813980b8 init: disallow reindex-chainstate when pruning
This fixes a bug where the node would be stuck in an
endless loop when combining these parameters.
2022-03-24 13:03:40 +01:00
fanquake
26d98d51f2
Merge bitcoin/bitcoin#24574: test: Actually print TSan tracebacks
fa76d8d4d7 test: Actually print TSan tracebacks (MarcoFalke)

Pull request description:

  Commit 5e5138a721 made the TSan logs to be printed before returning an error from the ci script.

  However, it seems that on Cirrus CI, the `--failfast` option will kill not only all python process and bitcoind child process, but also the parent CI bash script, rendering the `trap` inefficient. I believe this bug was introduced in commit 451b96f7d2.

ACKs for top commit:
  fanquake:
    utACK fa76d8d4d7

Tree-SHA512: 686f889d38a343882cb62ad6e0c2080196330e7cc7086891a7ff66d9443b455c82ba8d7e4a5cc42daa0513b0ad2743055bfe90e2f6ac88a910ee3b663fabddcd
2022-03-24 11:56:14 +00:00
MarcoFalke
864fb89b2f
Merge bitcoin/bitcoin#24637: test: use MiniWallet for mempool_package_onemore.py
2b6dd4e75b test: use MiniWallet for mempool_package_onemore.py (Sebastian Falbesoner)
eb3c5c4ef2 test: MiniWallet: add helper methods `{send,create}_self_transfer_multi` (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (mempool_package_onemore.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in https://github.com/bitcoin/bitcoin/issues/20078. For this purpose helper methods `MiniWallet.{create,send}_self_transfer_multi` are introduced which serve as a replacement for `chain_transaction`. With this, it should be also quite straight-forward to change the larger related test `mempool_packages.py` to use MiniWallet.

ACKs for top commit:
  MarcoFalke:
    ACK 2b6dd4e75b 💾

Tree-SHA512: 0c97fa0519ca5eaa6df8953a04678aa8a6a66905a82db6ff40042a675d0c0682aee829a48db84e4e7983d8f766875021f0d39d65e12889342610b8861bc29cd5
2022-03-24 12:37:11 +01:00
fanquake
8234cdaf62
Merge bitcoin/bitcoin#24587: test: use MiniWallet for rpc_createmultisig.py
2726b60a3a test: use MiniWallet for rpc_createmultisig.py (Ayush Sharma)

Pull request description:

  This PR enables one of the non-wallet functional tests (rpc_createmultisig.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078 .

ACKs for top commit:
  danielabrozzoni:
    re-ACK 2726b60a3a

Tree-SHA512: fb0ef22d3f1c161ca5963cb19ce76533ac3941f15102fc0aa2286ef3bec48f219e5934d504b41976f9f295fb6ca582b737e0fea896df4eb964cdaba1b2c91650
2022-03-24 11:11:56 +00:00
josibake
bc90b8d869
[move only] remove is_wallet_compiled checks 2022-03-24 11:00:22 +01:00
josibake
0bfbf7fb24
test: use MiniWallet in interfaces_zmq
make interfaces_zmg run deterministically.
this test is for the zmg notifications,
so it doesn't need the wallet compiled to run
2022-03-24 10:57:38 +01:00
fanquake
ce05f838f1
Merge bitcoin/bitcoin#24635: test: Run non-wallet tests only once
fa7a576391 test: Run non-wallet tests only once (MarcoFalke)

Pull request description:

  I don't see why non-wallet tests should run for two wallet configs, even though they never use a wallet.

ACKs for top commit:
  achow101:
    ACK fa7a576391

Tree-SHA512: 2a135acf3c3c83a2704ae11f40c72882b23a676828647be1a066653c4d00e4523704f377eb8745c6386829601cc5d643abdce376831c1db91a07e999e1d5e01f
2022-03-23 20:02:23 +00:00
brunoerg
89bb25d22a test: check localaddresses in getnetworkinfo for nodes with proxy 2022-03-22 16:07:01 -03:00
Sebastian Falbesoner
2b6dd4e75b test: use MiniWallet for mempool_package_onemore.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-03-22 18:44:27 +01:00
Sebastian Falbesoner
eb3c5c4ef2 test: MiniWallet: add helper methods {send,create}_self_transfer_multi 2022-03-22 18:43:51 +01:00
Ayush Sharma
2726b60a3a test: use MiniWallet for rpc_createmultisig.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-03-22 14:17:51 +05:30
MarcoFalke
b8f498f80d
Merge bitcoin/bitcoin#24535: test: Fix generate calls and comments in feature_segwit
fa8593f898 test: Fix generate calls and comments in feature_segwit (MarcoFalke)

Pull request description:

  There are currently a few incorrect comments: Block `432` is mined "twice" (The second one is actually 433).

  There isn't any need to mine this many blocks anyway, so remove a few calls.

ACKs for top commit:
  theStack:
    Tested ACK fa8593f898

Tree-SHA512: b034077b85e6c978a80aa4de493797b4ae451d686cfb3e4fe40f37a38f41f7cb886f8e00a1c245a284be3502164b17414097fcb0bef66d155a1c1db5cfbe9e8f
2022-03-22 09:10:55 +01:00
MarcoFalke
fa7a576391
test: Run non-wallet tests only once 2022-03-22 08:11:46 +01:00
MarcoFalke
e3206c9445
Merge bitcoin/bitcoin#24605: test: Use MiniWallet in feature_coinstatsindex
fa48ea3067 Use MiniWallet in feature_coinstatsindex (MarcoFalke)
fab61437f6 test: Refactor MiniWallet get_utxo helper (MarcoFalke)

Pull request description:

  Allows the test to be run even without a wallet compiled

ACKs for top commit:
  josibake:
    ACK fa48ea3067
  ayush933:
    tACK  fa48ea3 . The test runs successfully with the wallet disabled.
  willcl-ark:
    tACK fa48ea3067 both with and without wallet compiled in.

Tree-SHA512: e04e04ea0f236c062d6be68909ece2770130ce1d5343823893073d95aebc6eedb1ad1dc5bc41e5b0cb0bf2cd9018bb1d668f0e7f5f1101ed4e0b007ed6b00f69
2022-03-22 07:56:29 +01:00
MarcoFalke
fa8593f898
test: Fix generate calls and comments in feature_segwit 2022-03-18 16:23:43 +01:00
John Newbery
1066d10f71 scripted-diff: rename TxRelay members
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren cs_filter             m_bloom_filter_mutex
ren fRelayTxes            m_relay_txs
ren pfilter               m_bloom_filter
ren cs_tx_inventory       m_tx_inventory_mutex
ren filterInventoryKnown  m_tx_inventory_known_filter
ren setInventoryTxToSend  m_tx_inventory_to_send
ren fSendMempool          m_send_mempool
ren nNextInvSend          m_next_inv_send_time
ren minFeeFilter          m_fee_filter_received
ren lastSentFeeFilter     m_fee_filter_sent
-END VERIFY SCRIPT-
2022-03-18 11:35:58 +00:00
MarcoFalke
fa48ea3067
Use MiniWallet in feature_coinstatsindex 2022-03-17 20:49:10 +01:00
MarcoFalke
fab61437f6
test: Refactor MiniWallet get_utxo helper 2022-03-17 20:48:43 +01:00
DrahtBot
d6b24e14d2 test: Fix intermittent failure in feature_segwit 2022-03-16 19:56:31 +01:00
MarcoFalke
fa76d8d4d7
test: Actually print TSan tracebacks 2022-03-15 19:16:35 +01:00
MarcoFalke
7b83c7d609
Merge bitcoin/bitcoin#24510: test: check for importprunedfunds RPC errors
7573789925 test: check for importprunedfunds RPC errors (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the following errors of the `importprunedfunds` RPC:
  7003b6ab24/src/wallet/rpc/backup.cpp (L320-L322)
  7003b6ab24/src/wallet/rpc/backup.cpp (L332-L334)
  7003b6ab24/src/wallet/rpc/backup.cpp (L338-L340)
  7003b6ab24/src/wallet/rpc/backup.cpp (L343-L345)

ACKs for top commit:
  MarcoFalke:
    review ACK 7573789925

Tree-SHA512: b054520d102e5940bdeed2456ca644e91afb187d169b751b1262ce34480e4e9fbe1616ab184a78777c184350dced23508c3d367ed5825cab78bb5ad687fd7dac
2022-03-15 16:04:17 +01:00
MarcoFalke
e0881aa5f0
Merge bitcoin/bitcoin#24505: wallet: Add a deprecation warning for newly created legacy wallets
61152183ab wallet: Add a deprecation warning for newly created legacy wallets (Andrew Chow)

Pull request description:

  As we slowly deprecate legacy wallets, we need to warn users that are making new legacy wallets that their wallet type is going to be unsupported in the future.

ACKs for top commit:
  jonatack:
    ACK 61152183ab
  S3RK:
    reACK 61152183ab
  theStack:
    ACK 61152183ab

Tree-SHA512: e89bfb8168869542498958f0c9a2ab302dfd43287f8a49e7d9e09f60438a567bb8b7219a4e569797ee819b30b624f532fcc0b70c6aa0edcb392a301b8ce8b541
2022-03-14 08:37:46 +01:00
Gleb Naumenko
2ff8f4dd81
Add tests for addr destination rotation
Check that within 24h addr of a given node is forwarded
to the same peer(s), and then the destination is
rotated every 24h.

Co-authored-by: Jon Atack <jon@atack.com>
2022-03-13 16:54:43 +01:00
MarcoFalke
e04720ec33
Merge bitcoin/bitcoin#24528: rpc: rename getdeploymentinfo status-next to status_next
5d7c69b887 rpc: rename getdeploymentinfo status-next to status_next (Jon Atack)

Pull request description:

  Rename the `status-next` field to `status_next` in getdeploymentinfo before the RPC is released in v23.

  Before
  ```
  Result:
  {                                       (json object)
    "hash" : "str",                       (string) requested block hash (or tip)
    "height" : n,                         (numeric) requested block height (or tip)
    "deployments" : {                     (json object)
      "xxxx" : {                          (json object) name of the deployment
        "type" : "str",                   (string) one of "buried", "bip9"
        "height" : n,                     (numeric, optional) height of the first block which the rules are or will be enforced (only for "buried" type, or "bip9" type with "active" status)
        "active" : true|false,            (boolean) true if the rules are enforced for the mempool and the next block
        "bip9" : {                        (json object, optional) status of bip9 softforks (only for "bip9" type)
          "bit" : n,                      (numeric, optional) the bit (0-28) in the block version field used to signal this softfork (only for "started" and "locked_in" status)
          "start_time" : xxx,             (numeric) the minimum median time past of a block at which the bit gains its meaning
          "timeout" : xxx,                (numeric) the median time past of a block at which the deployment is considered failed if not yet locked in
          "min_activation_height" : n,    (numeric) minimum height of blocks for which the rules may be enforced
          "status" : "str",               (string) status of deployment at specified block (one of "defined", "started", "locked_in", "active", "failed")
          "since" : n,                    (numeric) height of the first block to which the status applies
          "status-next" : "str",          (string) status of deployment at the next block
          "statistics" : {                (json object, optional) numeric statistics about signalling for a softfork (only for "started" and "locked_in" status)
            "period" : n,                 (numeric) the length in blocks of the signalling period
            "threshold" : n,              (numeric, optional) the number of blocks with the version bit set required to activate the feature (only for "started" status)
            "elapsed" : n,                (numeric) the number of blocks elapsed since the beginning of the current period
            "count" : n,                  (numeric) the number of blocks with the version bit set in the current period
            "possible" : true|false       (boolean, optional) returns false if there are not enough blocks left in this period to pass activation threshold (only for "started" status)
          },
          "signalling" : "str"            (string) indicates blocks that signalled with a # and blocks that did not with a -
        }
      }
    }
  }
  ```
  After
  ```
  Result:
  {                                       (json object)
    "hash" : "str",                       (string) requested block hash (or tip)
    "height" : n,                         (numeric) requested block height (or tip)
    "deployments" : {                     (json object)
      "xxxx" : {                          (json object) name of the deployment
        "type" : "str",                   (string) one of "buried", "bip9"
        "height" : n,                     (numeric, optional) height of the first block which the rules are or will be enforced (only for "buried" type, or "bip9" type with "active" status)
        "active" : true|false,            (boolean) true if the rules are enforced for the mempool and the next block
        "bip9" : {                        (json object, optional) status of bip9 softforks (only for "bip9" type)
          "bit" : n,                      (numeric, optional) the bit (0-28) in the block version field used to signal this softfork (only for "started" and "locked_in" status)
          "start_time" : xxx,             (numeric) the minimum median time past of a block at which the bit gains its meaning
          "timeout" : xxx,                (numeric) the median time past of a block at which the deployment is considered failed if not yet locked in
          "min_activation_height" : n,    (numeric) minimum height of blocks for which the rules may be enforced
          "status" : "str",               (string) status of deployment at specified block (one of "defined", "started", "locked_in", "active", "failed")
          "since" : n,                    (numeric) height of the first block to which the status applies
          "status_next" : "str",          (string) status of deployment at the next block
          "statistics" : {                (json object, optional) numeric statistics about signalling for a softfork (only for "started" and "locked_in" status)
            "period" : n,                 (numeric) the length in blocks of the signalling period
            "threshold" : n,              (numeric, optional) the number of blocks with the version bit set required to activate the feature (only for "started" status)
            "elapsed" : n,                (numeric) the number of blocks elapsed since the beginning of the current period
            "count" : n,                  (numeric) the number of blocks with the version bit set in the current period
            "possible" : true|false       (boolean, optional) returns false if there are not enough blocks left in this period to pass activation threshold (only for "started" status)
          },
          "signalling" : "str"            (string) indicates blocks that signalled with a # and blocks that did not with a -
        }
      }
    }
  }
  ```

Top commit has no ACKs.

Tree-SHA512: 4facfd7af3cfb7b6f5495758c4387602802f5e39d9270b162d17350a7f954eab0b74d895f17f0d8dfbc7814d36db7cff56d08c42728432885ea6f4e37aea4aa8
2022-03-13 10:23:20 +01:00
Sebastian Falbesoner
aab552fa30 test: use MiniWallet for feature_maxuploadtarget.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-03-11 16:14:48 +01:00
fanquake
23e8c702bc
Merge bitcoin/bitcoin#24421: miner: always assume we can build witness blocks
40e871d9b4 [miner] always assume we can create witness blocks (glozow)

Pull request description:

  Given the low possibility of a reorg reverting the segwit soft fork, there is no longer a need to check whether segwit is active to see if it's okay to add to the block template (see also #23512, #21009, etc). `TestBlockValidity()` is also run on the block template at the end of `CreateNewBlock()`, so any invalid block would be caught there.

ACKs for top commit:
  gruve-p:
    ACK 40e871d9b4
  jnewbery:
    utACK 40e871d9b4, although I disagree about changing the test for segwit transaction in mempool before activagtion, instead of just removing it: https://github.com/bitcoin/bitcoin/pull/24421#discussion_r822933721.
  achow101:
    ACK 40e871d9b4
  theStack:
    Code-review ACK 40e871d9b4

Tree-SHA512: bf4860bf2bed8339622d05228d11d60286edb0c32a9a3c434b8d154913c07ea56e50649f4af7009c2a1c6a58a81d2299ab43b41a6f16dee7d08cc89cc1603019
2022-03-11 15:00:38 +00:00
Jon Atack
5d7c69b887
rpc: rename getdeploymentinfo status-next to status_next 2022-03-11 10:21:48 +01:00
Andrew Chow
61152183ab wallet: Add a deprecation warning for newly created legacy wallets 2022-03-10 07:32:02 -05:00
MarcoFalke
fa9112aac0
Remove utxo db upgrade code 2022-03-10 13:05:29 +01:00
Sebastian Falbesoner
7573789925 test: check for importprunedfunds RPC errors 2022-03-09 16:16:07 +01:00
MarcoFalke
aa83bbb1fe
Merge bitcoin/bitcoin#24490: tests: Ensure sorted/multi_a descriptors always generate different addrs
db27ac9354 tests: Ensure sorted/multi_a descriptors always generate different addrs (Andrew Chow)

Pull request description:

  Sometimes the multi_a and sortedmulti_a descriptors will produce some of the same addresses in the tests. This causes the wallets to start generating addresses at a different index as they detect that one of the addresses is used. This subsequently causes a test failure.

  To avoid this problem, use descriptors that will produce unique addresses by putting one of the multi_a in a different branch.

ACKs for top commit:
  ajtowns:
    ACK db27ac9354
  theStack:
    Tested ACK db27ac9354

Tree-SHA512: 0f57822bf4c7c79da304f092d7d43d6118e78a087cbeb0766fbbf634dc27911ae723d5d41350884d3b63a24d3b3817944f7e5fa534afb849161dd008a1e4a62f
2022-03-09 11:33:03 +01:00
Andrew Chow
9d22dbe2e1
Merge bitcoin/bitcoin#24198: wallet, rpc: add wtxid in WalletTxToJSON
7abd8b21ba doc: include wtxid in TransactionDescriptionString (brunoerg)
2d596bce6f doc: add wtxid info in release-notes (brunoerg)
a5b66738f1 test: add wtxid in expected_fields for wallet_basic (brunoerg)
e8c659a297 wallet: add wtxid in WalletTxToJSON (brunoerg)
7482b6f895 wallet: add GetWitnessHash() (brunoerg)

Pull request description:

  This PR add `wtxid` in `WalletTxToJSON` which allows to return this field in `listsinceblock`, `listtransactions` and `gettransaction` (RPCs).

ACKs for top commit:
  achow101:
    re-ACK 7abd8b21ba
  w0xlt:
    crACK 7abd8b2
  luke-jr:
    re-utACK 7abd8b21ba

Tree-SHA512: f86f2dbb5e38e7b19932006121802f47b759d31bdbffe3263d1db464f6a3a30fddd68416f886a44f6d3a9fd570f7bd4f8d999737ad95c189e7ae5e8ec1ffbdaa
2022-03-08 14:32:10 -05:00
MarcoFalke
b07fdd7f9e
Merge bitcoin/bitcoin#24312: addrman: Log too low compat value
fa097d074b addrman: Log too low compat value (MarcoFalke)

Pull request description:

  Before this patch, when writing a negative `lowest_compatible` value, it would be read as a positive value. For example `-32` will be read as `224`. There is generally nothing wrong with that. Though, similarly there shouldn't be anything wrong with refusing to read a negative value. I find the code after this patch more logical than before. Also, this allows dropping a file-wide sanitizer suppression.

  In practice none of this should ever happen. Bitcoin Core would never write a negative `lowest_compatible` in normal operation, unless the file storage is later corrupted by external influence.

ACKs for top commit:
  mzumsande:
    re-ACK fa097d074b

Tree-SHA512: 9aae7b8fe666f52f667f149667025e0160cef1a793cc4d392e36608f65c2bee8096da429235118f40a3368f327aabe30f3732ae78c5874648ea6f423f2687b65
2022-03-08 16:48:22 +01:00
Sebastian Falbesoner
31846b006d test: refactor: use random.sample for choosing random keys in wallet_taproot.py 2022-03-07 18:40:17 +01:00
Andrew Chow
db27ac9354 tests: Ensure sorted/multi_a descriptors always generate different addrs
Sometimes the multi_a and sortedmulti_a descriptors will produce some of
the same addresses in the tests. This causes the wallets to start
generating addresses at a different index as they detect that one of
the addresses is used. This subsequently causes a test failure.

To avoid this problem, use descriptors that will produce unique
addresses by putting one of the multi_a in a different branch.
2022-03-07 06:06:16 -05:00
Andrew Chow
bada9636d7
Merge bitcoin/bitcoin#24043: Add (sorted)multi_a descriptor for k-of-n multisig inside tr
4828d53ecc Add (sorted)multi_a descriptors to doc/descriptors.md (Pieter Wuille)
b5f33ac1f8 Simplify wallet_taproot.py functional test (Pieter Wuille)
eb0667ea96 Add tests for (sorted)multi_a derivation/signing (Pieter Wuille)
c17c6aa08d Add signing support for (sorted)multi_a scripts (Pieter Wuille)
3eed6fca57 Add multi_a descriptor inference (Pieter Wuille)
79728c4a3d Add (sorted)multi_a descriptor and script derivation (Pieter Wuille)
25e95f9ff8 Merge/generalize IsValidMultisigKeyCount/GetMultisigKeyCount (Pieter Wuille)

Pull request description:

  This adds a new `multi_a(k,key_1,key_2,...,key_n)` (and corresponding `sortedmulti_a`) descriptor for k-of-n policies inside `tr()`. Semantically it is very similar to the existing `multi()` descriptor, but with the following changes:
  * The corresponding script is `<key1> OP_CHECKSIG <key2> OP_CHECKSIGADD <key3> OP_CHECKSIGADD ... <key_n> OP_CHECKSIGADD <k> OP_NUMEQUAL`, rather than the traditional `OP_CHECKMULTISIG`-based script, making it usable inside the `tr()` descriptor.
  * The keys can optionally be specified in x-only notation.
  * Both the number of keys and the threshold can be as high as 999; this is the limit due to the consensus stacksize=1000 limit

  I expect that this functionality will later be replaced with a miniscript-based implementation, but I don't think it's necessary to wait for that.

  Limitations:
  * The wallet code will for not estimate witness size incorrectly for script path spends, which may result in a (dramatic) fee underpayment with large multi_a scripts.
  * The multi_a script construction is (slightly) suboptimal for n-of-n (where a `<key1> OP_CHECKSIGVERIFY ... <key_n-1> OP_CHECKSIGVERIFY <key_n> OP_CHECKSIG` would be better). Such a construction is not included here.

ACKs for top commit:
  achow101:
    ACK 4828d53ecc
  gruve-p:
    ACK 4828d53ecc
  sanket1729:
    code review ACK 4828d53ecc
  darosior:
    Code review ACK 4828d53ecc

Tree-SHA512: 5dcd434b79585f0ff830f7d501d27df5e346f5749f47a3109ec309ebf2cbbad0e1da541eec654026d911ab67fd7cf7793fab0f765628d68d81b96ef2a4d234ce
2022-03-04 07:28:23 -05:00
Jon Atack
58a14795b8
test: passing -onlynet=onion with -onion=0/-noonion raises expected init error 2022-03-03 15:31:44 +01:00
Jon Atack
7000f66d36
test: passing -onlynet=onion without -proxy/-onion raises expected init error 2022-03-03 15:31:37 +01:00
Jon Atack
8332e6e4cf
test: passing invalid -onion raises expected init error 2022-03-03 15:31:30 +01:00
Jon Atack
d5edb08708
test: passing invalid -proxy raises expected init error 2022-03-03 15:31:15 +01:00
Vasil Dimov
7d64ea4a01
net: only assume all local addresses if listening on any
If `-bind=` is provided then we would bind only to a particular address
and should not add all the other addresses of the machine to the list of
local addresses.

Fixes https://github.com/bitcoin/bitcoin/issues/20184 (case 4.)
2022-03-02 15:42:40 +01:00
Vasil Dimov
0cfc0cd322
net: fix GetListenPort() to derive the proper port
`GetListenPort()` uses a simple logic: "if `-port=P` is given, then we
must be listening on `P`, otherwise we must be listening on `8333`".
This is however not true if `-bind=` has been provided with `:port` part
or if `-whitebind=` has been provided. Thus, extend `GetListenPort()` to
return the port from `-bind=` or `-whitebind=`, if any.

Fixes https://github.com/bitcoin/bitcoin/issues/20184 (cases 1. 2. 3. 5.)
2022-03-02 15:42:37 +01:00
Vasil Dimov
3cb9d9c861
net: make CaptureMessage() mockable
Rename `CaptureMessage()` to `CaptureMessageToFile()` and introduce a
`std::function` variable called `CaptureMessage` whose value can be
changed by unit tests, should they need to inspect message contents.
2022-03-02 15:40:36 +01:00
laanwj
267917f563
Merge bitcoin/bitcoin#23304: wallet: Derive inactive HD chains in additional places
c4d76c6faa tests: Tests for inactive HD chains (Andrew Chow)
8077862c5e wallet: Refactor TopUp to be able to top up inactive chains too (Andrew Chow)
70134eb34f wallet: Properly set hd chain counters when loading (Andrew Chow)
961b9e4e40 wallet: Parse hdKeypath if key_origin is not available (Andrew Chow)
0652ee73ec Add size check on meta.key_origin.path (Rob Fielding)

Pull request description:

  Currently inactive HD chains are only derived from at the time a key in that chain is found to have been used. However, at that time, the wallet may not be able to derive keys (e.g. it is locked). Currently we would just move on and not derive any new keys, however this could result in missing funds.

  This PR resolves this problem by adding memory only variables to `CHDChain` which track the highest known index. `TopUp` is modified to always try to top up the inactive HD chains, and this process will use the new variables to determine how much to top up. In this way, after an encrypted wallet is unlocked, the inactive HD chains will be topped up and hopefully funds will not be missed.

  Note that because these variables are not persisted to disk (because `CHDChain`s for inactive HD chains are not written to disk), if an encrypted wallet is not unlocked in the same session as a key from an inactive chain is found to be used, then it will not be topped up later unless more keys are found.

  Additionally, wallets which do not have upgraded key metadata will not derive any keys from inactive HD chains. This is resolved by using the derivation path string in `CKeyMetadata.hdKeypath` to determine what indexes to derive.

ACKs for top commit:
  laanwj:
    Code review ACK c4d76c6faa

Tree-SHA512: b2b572ad7f1b1b2847edece09f7583543d63997e18ae32764e5a27ad608dd64b9bdb2d84ea27137894e986a8e82f047a3dba9c8015b74f5f179961911f0c4095
2022-03-02 09:35:07 +01:00
Jon Atack
bd57dcbaf2
test: hoist proxy out of 2 network loops in feature_proxy.py 2022-03-01 21:04:58 +01:00
laanwj
848b11615b
Merge bitcoin/bitcoin#22834: net: respect -onlynet= when making outbound connections
0eea83a85e scripted-diff: rename `proxyType` to `Proxy` (Vasil Dimov)
e53a8505db net: respect -onlynet= when making outbound connections (Vasil Dimov)

Pull request description:

  Do not make outbound connections to hosts which belong to a network
  which is restricted by `-onlynet`.

  This applies to hosts that are automatically chosen to connect to and to
  anchors.

  This does not apply to hosts given to `-connect`, `-addnode`,
  `addnode` RPC, dns seeds, `-seednode`.

  Fixes https://github.com/bitcoin/bitcoin/issues/13378
  Fixes https://github.com/bitcoin/bitcoin/issues/22647
  Supersedes https://github.com/bitcoin/bitcoin/pull/22651

ACKs for top commit:
  naumenkogs:
    utACK 0eea83a85e
  prayank23:
    reACK 0eea83a85e
  jonatack:
    ACK 0eea83a85e code review, rebased to master, debug built, and did some manual testing with various config options on signet

Tree-SHA512: 37d68b449dd6d2715843fc84d85f48fa2508be40ea105a7f4a28443b318d0b6bd39e3b2ca2a6186f2913836adf08d91038a8b142928e1282130f39ac81aa741b
2022-03-01 18:32:01 +01:00
laanwj
159f89c118
Merge bitcoin/bitcoin#24365: wallet: Don't generate keys for wallets with private keys disabled during upgradewallet
c7376cc8d7 tests: Test upgrading wallet with privkeys disabled (Andrew Chow)
3d985d4f43 wallet: Don't generate keys when privkeys disabled when upgrading (Andrew Chow)

Pull request description:

  When we're upgrading a wallet, we shouldn't be trying to generate new keys for wallets where private keys are disabled.

  Fixes #23610

ACKs for top commit:
  laanwj:
    Code review ACK c7376cc8d7
  benthecarman:
    tACK c7376cc8d7 this fixed the issue for me

Tree-SHA512: fa07cf37df9196ff98671bb1ce5c9aa0bab46495066b4dab796d7e8e5d5c7adb414ff56adae4fd3e15658a610995bd19a9e1edb00c46144b0df635c5b343f3a6
2022-02-28 13:15:11 +01:00
MarcoFalke
c7da61dcc3
Merge bitcoin/bitcoin#24403: Avoid implicit-integer-sign-change in VerifyLoadedChainstate
fa7991601c Fixup style of VerifyDB (MarcoFalke)
fa462ea787 Avoid implicit-integer-sign-change in VerifyLoadedChainstate (MarcoFalke)

Pull request description:

  This happens when checking all blocks (`-1`).

  To test:

  ```
  ./configure CC=clang CXX=clang++ --with-sanitizers=undefined,integer
  make
  UBSAN_OPTIONS="suppressions=$(pwd)/test/sanitizer_suppressions/ubsan:print_stacktrace=1:halt_on_error=1:report_error_type=1" ./test/functional/rpc_blockchain.py

ACKs for top commit:
  theStack:
    Code-review ACK fa7991601c
  brunoerg:
    crACK fa7991601c

Tree-SHA512: bcbe6becf2fbedd21bbde83a544122e79465937346802039532143b2e4165784905a8852c0ccb088b964874df5e5550931fdde3629cbcee3ae237f2f63c43a8e
2022-02-28 12:33:32 +01:00
MarcoFalke
fa097d074b
addrman: Log too low compat value
Also remove uint8_t{} casts from values that are already of the same
type.
2022-02-25 14:16:32 +01:00
MarcoFalke
b00b60ed4f
Merge bitcoin/bitcoin#24201: p2p: Avoid InitError when downgrading peers.dat
d41ed32153 p2p: Avoid InitError when downgrading peers.dat (junderw)

Pull request description:

  fixes #24188 (also see https://github.com/bitcoin/bitcoin/pull/22762#issuecomment-951063826)
  When downgrading, a peers.dat with a future version that has a minimum
  required version larger than the downgraded Bitcoin Core version would cause an InitError.

  This commit changes this behavior to overwrite the existing peers.dat with
  a new empty one.

ACKs for top commit:
  prayank23:
    reACK d41ed32153
  kallewoof:
    reACK d41ed32153

Tree-SHA512: c8e625fe36ce0b1aab6c8ef7241c8954038bb856f2de27bdc4814dc9a60e51be28815c7d77d0f96eace49687a0cea02deb713978bbd3a5add742f50a675f2a40
2022-02-25 08:45:11 +01:00
junderw
d41ed32153
p2p: Avoid InitError when downgrading peers.dat
fixes #24188
When downgrading, a peers.dat with a future version that has a minimum
required version larger than the downgraded version would cause an InitError.

This commit changes this behavior to overwrite the existing peers.dat with
a new empty one, while creating a backup in peers.dat.bak.
2022-02-25 09:53:10 +09:00
MarcoFalke
f1ce67f09f
Merge bitcoin/bitcoin#19013: test: add v0.20.1, v0.21.0 and v22.0 to backwards compatibility test
24cec4b5c0 test: Fix intermittent test failure in feature_backwards_compatibility (MarcoFalke)
d8b705f1ca test: previous releases: add v22.0 (Sjors Provoost)
40849eebd9 test: bump sandbox argument minimum version (Sjors Provoost)
8a57a06a50 test: previous releases: add v0.21.0 (Sjors Provoost)
8cba75f5fd test: v0.20.1 backwards compatibility (Sjors Provoost)
0e4b695b6a test: backwards compatibility: misc fixes (Sjors Provoost)
76557cbe4c test: Remove i686 from test/get_previous_releases.py (MarcoFalke)

Pull request description:

  This also simplifies the tests a bit.

ACKs for top commit:
  ryanofsky:
    Code review ACK 24cec4b5c0. Only change since last review is rebasing and adding comment and whitelist args.

Tree-SHA512: 85a603ddd70fd8f0180d00fb84eb2ad2f92d6199b7d3f7c1abd660bfba53f869faf40f1a4183a8ce15dbd496ee3132d879c1258651c9d443ece69e5fe328bd26
2022-02-24 17:42:28 +01:00
laanwj
358fe779cb
Merge bitcoin/bitcoin#24381: test: Run symlink regression tests on Windows
fad7ddf9e3 test: Run symlink regression tests on Windows (MarcoFalke)

Pull request description:

  Seems odd to add tests, but not run them on the platform that needs them most.

ACKs for top commit:
  laanwj:
    Code review ACK fad7ddf9e3
  ryanofsky:
    Code review ACK fad7ddf9e3, just removing new test. Would be nice if the test could be added later, of course.

Tree-SHA512: 64b235967a38c2eb90657e8d7a0447bcc8ce81d1b75a275b6c48bd42efd9ea7e7939257e484f297ee84598def3738eaeb289561aeba1dd6a99b258d389995139
2022-02-23 15:48:55 +01:00
glozow
40e871d9b4 [miner] always assume we can create witness blocks
Given the low possibility of a reorg reverting the segwit soft fork,
there is no need to check whether segwit is active here. Also,
TestBlockValidity is run on the block template after it has been
created.
2022-02-23 10:55:05 +00:00
Andrew Chow
c4d76c6faa tests: Tests for inactive HD chains
test cases are added for inactive HD chains: a basic case, a case
where the wallet is encrypted, and a case for the 21605 segfault.
2022-02-22 14:41:52 -05:00
Andrew Chow
c7376cc8d7 tests: Test upgrading wallet with privkeys disabled 2022-02-22 13:52:40 -05:00
Suhas Daftuar
a35f963edf Add test for getheaders behavior
Expect responses to a getheaders iff the node has a chain with more than nMinimumChainWork
2022-02-22 11:34:05 -05:00
laanwj
8add59d77d
Merge bitcoin/bitcoin#24367: User-facing content and codebase doc fixups from transifex translator feedback
48742693ac Replace "can not" with "cannot" in docs, user messages, and tests (Jon Atack)
e670edd434 User-facing content fixups from transifex translator feedback (Jon Atack)

Pull request description:

  Closes #24366.

ACKs for top commit:
  laanwj:
    Code review re-ACK 48742693ac
  hebasto:
    re-ACK 48742693ac, only suggested change since my previous [review](https://github.com/bitcoin/bitcoin/pull/24367#pullrequestreview-885938219).

Tree-SHA512: 4dcdcb417251a413e65fab6070515e13a1267c8e0dbcf521386b842511391f24c84a0c2168fe13458c977682034466509bf2a3453719d4d94d3c568fd9f4adb4
2022-02-22 13:08:08 +01:00
Jon Atack
48742693ac
Replace "can not" with "cannot" in docs, user messages, and tests 2022-02-21 19:07:29 +01:00
MarcoFalke
fa30e62cc6
doc: Rework generate* doc
Can be reviewed with --word-diff-regex=. --ignore-all-space
2022-02-21 11:02:20 +01:00
MarcoFalke
fa462ea787
Avoid implicit-integer-sign-change in VerifyLoadedChainstate 2022-02-21 10:29:37 +01:00
0xb10c
76c60d7b31
test: validation:block_connected tracepoint test
This adds a test for the validation:block_connected tracepoint.
2022-02-20 14:59:15 +01:00
0xb10c
260e28ece8
test: utxocache:* tracepoint tests
This adds tests for the
- utxocache:flush
- utxocache:uncache
- utxocache:add
- utxocache:spent
tracepoint interfaces.
2022-02-20 14:59:13 +01:00
0xb10c
34b27bac68
test: net:in/out_message tracepoint tests
This adds tests for the net:inbound_message and net:outbound_message
tracepoint interface.
2022-02-20 14:59:12 +01:00
0xb10c
c934087b62
test: checks for tracepoint tests
For testing the USDT tracepoint API in the functional tests we
require:
 - that we are on a Linux system*
 - that Bitcoin Core is compiled with tracepoints
 - that bcc and the the Python bcc module [0] is installed
 - that we run the tests with the required permissions**
otherwise we skip the tests.

*:  We currently only support tracepoints on Linux. Tracepoints are
    not compiled on other platforms.
**: Currently, we check for root permissions via getuid == 0. It's
    unclear if it's even possible to run the tests a non-root user
    with e.g. CAP_BPF, CAP_PERFMON, and access to /sys/kernel/debug/
    tracing/. Anyone running these tests as root should carefully
    review them first and then run them in a disposable VM.

[0]: https://github.com/iovisor/bcc/blob/master/INSTALL.md
2022-02-20 14:59:08 +01:00
MarcoFalke
fad7ddf9e3
test: Run symlink regression tests on Windows 2022-02-18 15:27:08 +01:00
laanwj
b223c3c21e test: Add functional test for symlinked blocks directory 2022-02-17 12:33:30 +01:00
Seibart Nedor
5f213213cb tests: add tests for cross-chain wallet use prevention 2022-02-16 15:02:26 +02:00
MarcoFalke
1e8aa02ec5
Merge bitcoin/bitcoin#24117: index: make indices robust against init aborts
bfcd60f5d5 test: activate all index types in feature_init.py (Martin Zumsande)
0243907fae index: Don't commit without valid m_best_block_index (Martin Zumsande)

Pull request description:

  When an index thread receives an interrupt during init before it got to index anything (so `m_best_block_index == nullptr` still), it will still try to commit previous "work" before stopping the thread. That means that `BaseIndex::CommitInternal()` calls `GetLocator(nullptr)`, which returns an locator to the tip ([code](06b6369766/src/chain.cpp (L31-L32))), and saves it to the index DB.
  On the next startup, this locator will be read and it will be assumed that we have successfully synced the index to the tip, when in reality we have indexed nothing.
  In the case of coinstatsindex, this would lead to a shutdown of bitcoind without any indication what went wrong. For the other indexes, there would be no immediate shutdown, but the index would be corrupt.

  This PR fixes this by not committing when `m_best_block_index==nullptr`, and it also adds an error log message to the silent coinstatsindex shutdown path.

  This is another small bug found by `feature_init.py` - the second commit enables blockfilterindex and coinstatsindex for this test, enabling coinstatsindex without the first commit would have led to frequent failures.

ACKs for top commit:
  fjahr:
    reACK bfcd60f5d5
  shaavan:
    reACK bfcd60f5d5

Tree-SHA512: 8e2bac0fc40cde209518a9e59b597ae0a5a875a2a90898673987c91733718d40e528dada942bf552b58bc021bf46e59da2d0cc5a61045f48f9bae2b1baf6033b
2022-02-15 19:57:25 +01:00
MarcoFalke
7164e00e1b
Merge bitcoin/bitcoin#24324: test: refactor: remove unneeded bytes<->hex conversions in byte_to_base58
f11dad22a5 test: refactor: remove unneeded bytes<->hex conversions in `byte_to_base58` (Sebastian Falbesoner)

Pull request description:

  It seems like the only reason for using hex strings in this method was to have a convenient way to convert to an integer from the input data interpreted as big-endian. In Python3 we have `int.from_bytes(..., 'big')` for that purpose, hence there is no need for that anymore and we can simply operate on bytes only.

ACKs for top commit:
  laanwj:
    Code review ACK f11dad22a5

Tree-SHA512: 9b1563010066ca74d85139c3b9259e9a5bb49e1f141c30b6506a0445afddb2bde7fd421fdd917dc516956e66f93610e2c21d720817640daee8f57f803be76ee4
2022-02-15 09:31:58 +01:00
brunoerg
460fa8e0d9 test: remove import socket in test_ipv6_local 2022-02-14 19:27:33 -03:00
Sebastian Falbesoner
f11dad22a5 test: refactor: remove unneeded bytes<->hex conversions in byte_to_base58
It seems like the only reason for using hex strings in this
method was to have a convenient way to convert to an integer
from the input data interpreted as big-endian.
In Python3 we have `int.from_bytes(..., 'big')` for that
purpose, hence there is no need for that anymore and we can
simply operate on bytes only.
2022-02-14 12:48:43 +01:00
Kristaps Kaupe
b75f4c89ec
RPC: Return external_signer in getwalletinfo 2022-02-10 03:23:47 +02:00
fanquake
87b5b002ad
Merge bitcoin/bitcoin#24259: test: Remove unused valgrind suppressions
fa4b61911d test: Remove unused valgrind suppressions (MarcoFalke)
faccb2d7fe test: Exclude broken feature_init for now (MarcoFalke)
fa086d891b test: Properly skip feature_syscall_sandbox in valgrind (MarcoFalke)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK fa4b61911d

Tree-SHA512: 5be1a8f288182d386531a033ae7258f753dd655dfa1746a52b65622a0359c2b7143a25b49c0747538308eed606a691847d2f59a5a0382b7751b8de7172adf0d3
2022-02-08 13:19:49 +00:00
MarcoFalke
eca694a4e7
Merge bitcoin/bitcoin#24239: test: fix ceildiv division by using integers
d1fab9d5d2 test: Call ceildiv helper with integer (Martin Zumsande)

Pull request description:

  On master,

  `assert_fee_amount(Decimal("0.00000993"), 217, Decimal("0.00004531"))` passes
  `assert_fee_amount(Decimal("0.00000993"), Decimal("217"), Decimal("0.00004531"))` fails.

  the reason is that the // operator in  `ceildiv(a,b) = -(-a//b)`  has a different behavior for Decimals, see [doc](https://docs.python.org/3/library/decimal.html#decimal-objects).

  `wallet_send.py` calls this function with Decimals, and I think this is the reason for the failure reported in the OP of #24151 (`wallet_send.py --legacy-wallet` line 332, the numbers used in the example above are from there). However, the other failures reported there cannot be explained by this, so this is just a partial fix.

ACKs for top commit:
  ryanofsky:
    Code review ACK d1fab9d5d2. Tracking down this problem was a good find, and code seems safer and easier to understand now

Tree-SHA512: 5bf0568cd1a0824f6b1a15a03580b6e9391b4f51112a97c1d00469d255bf6dda45c49a36fa567a5ba9b9973efe1d9cdd480db91965c9f4c2aa963629a8a32cba
2022-02-07 17:08:26 +01:00
MarcoFalke
9392e1350c
Merge bitcoin/bitcoin#24195: test: Fix failfast option for functional test runner
a036358994 test: Repair failfast option for test runner (Martin Zumsande)

Pull request description:

  Fixes #23990

  After #23799, the `--failfast` option in the test runner for the functional tests stopped working, because a second outer loop was introduced, which would have needed a `break` too for the test runner to fail immediately. This also led to the errors reported in #23990.

  This provides a straightforward fix for that.
  There is also #23995 which is a larger refactor, but that hasn't been updated in a while to fix the failfast issue.

ACKs for top commit:
  pg156:
    Tested ACK a036358994. I agree adding the `all_passed` flag to break out of the outer loop when needed makes sense. The "failfast" option works after this change.

Tree-SHA512: 3e2f775e36c13d180d32a05cd1cfe0883274e8615cdbbd4e069a9899e9b9ea1091066cf085e93f1c5326bd8ecc6ff524e0dad7c638f60dfdb169fefcdb26ee52
2022-02-07 16:57:50 +01:00
Martin Zumsande
d1fab9d5d2 test: Call ceildiv helper with integer
It returns an incorrect result when called with a Decimal,
for which the "//" operator works differently.
Also drop unnecessary call to satoshi_round.
2022-02-07 15:35:43 +01:00
MarcoFalke
fa086d891b
test: Properly skip feature_syscall_sandbox in valgrind
Follow up to commit fa9c26ab3a
2022-02-04 16:33:05 +01:00
Kiminuo
41d7166c8a
refactor: replace boost::filesystem with std::filesystem
Warning: Replacing fs::system_complete calls with fs::absolute calls
in this commit may cause minor changes in behaviour because fs::absolute
no longer strips trailing slashes; however these changes are believed to
be safe.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2022-02-03 18:35:52 +08:00
MarcoFalke
a41976ab77
Merge bitcoin/bitcoin#24223: test: use MiniWallet for interface_rest.py
438e6f4c33 test: speedup interface_rest.py by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
11b9684dfd test: use MiniWallet for rest_interface.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (interface_rest.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078.

  Note that the original test sent funds from one node to another and checked node's balances, but the state of a node's wallet is not relevant to any of the REST endpoints, i.e. the replacement is quite stright-forward. In an additional commit, the test is further sped up by using the good ol' immediate tx relay trick (parameter `-whitelist=noban@127.0.0.1`).

ACKs for top commit:
  brunoerg:
    ACK 438e6f4c33

Tree-SHA512: eac351c5fb7e043c36c193d51545f20f563be9aaa04f3429a2bfb452ae4aa72294d2552800d6cac55c9a3ec2b4f30bcda2abcd74736dec3ed75e7d83c5af437f
2022-02-02 09:07:54 +01:00
Andrew Chow
02e1d8d06f
Merge bitcoin/bitcoin#24083: Revert "Add to spends only transcations from me"
3ee6d0788e test: add more wallet conflicts assertions (S3RK)
3b98bf9c43 Revert "Add to spends only transcations from me" (S3RK)

Pull request description:

  This reverts commit d04566415e from #22929.

  This commit was based on invalid assumption that `mapTxSpends` should contain only outgoing txs and broke wallet conflicts feature.

ACKs for top commit:
  achow101:
    ACK 3ee6d0788e

Tree-SHA512: bf5a77ced6bac57d5eb85771d9189c53e1edc295d179ed5a1bdce18e365794a9101b4cecf35387b27f67260db3b47f7214e7876e490494529b748cceeb95632d
2022-02-01 14:46:11 -05:00
brunoerg
a5b66738f1 test: add wtxid in expected_fields for wallet_basic 2022-02-01 08:44:51 -03:00
Sebastian Falbesoner
438e6f4c33 test: speedup interface_rest.py by whitelisting peers (immediate tx relay)
By whitelisting the peers via -whitelist, the inventory is transmissioned
immediately rather than on average every 5 seconds, speeding up the test by at
least a factor of two:

before:
$ time ./interface_rest.py
...
0m14.82s real     0m01.44s user     0m01.19s system

with this commit:
$ time ./interface_rest.py
...
0m05.67s real     0m01.07s user     0m01.35s system
2022-01-31 22:56:34 +01:00
Sebastian Falbesoner
11b9684dfd test: use MiniWallet for rest_interface.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-01-31 22:47:40 +01:00
Martin Zumsande
bfcd60f5d5 test: activate all index types in feature_init.py 2022-01-31 21:21:39 +01:00
MarcoFalke
fa7b07571f
test: Fix feature_init intermittent issues 2022-01-31 10:10:25 +01:00
MarcoFalke
fa4595deb3
test: Remove random line number feature from feature_init.py
This is needed for the next commit.

Also, it doesn't really test anything novel. wait_for_debug_log is
inherently racy, so will randomly terminate at the exact point or later.
So the randomization is already sufficiently covered by the existing
test.
2022-01-31 10:09:11 +01:00
MarcoFalke
cccc1e70b8
Enforce Taproot script flags whenever WITNESS is set 2022-01-29 14:48:37 +01:00
Martin Zumsande
a036358994 test: Repair failfast option for test runner 2022-01-28 15:46:51 +01:00
MarcoFalke
d4e92d8436
Merge bitcoin/bitcoin#23508: Add getdeploymentinfo RPC
a380922891 Release notes for getdeploymentinfo rpc (Anthony Towns)
240cad09ba rpc: getdeploymentinfo: include signalling info (Anthony Towns)
376c0c6dae rpc: getdeploymentinfo: include block hash/height (Anthony Towns)
a7469bcd35 rpc: getdeploymentinfo: change stats to always refer to current period (Anthony Towns)
7f15c1841b rpc: getdeploymentinfo: allow specifying a blockhash other than tip (Anthony Towns)
fd826130a0 rpc: move softfork info from getblockchaininfo to getdeploymentinfo (Anthony Towns)

Pull request description:

  The aim of this PR is to improve the ability to monitor soft fork status. It first moves the softfork section from getblockchaininfo into a new RPC named getdeploymentinfo, which is then also able to query the status of forks at an arbitrary block rather than only at the tip. In addition, bip9 status is changed to indicate the status of the given block, rather than just for the next block, and an additional field is included to indicate whether each block in the signalling period signaled.

ACKs for top commit:
  laanwj:
    Code review and lightly tested ACK a380922891
  Sjors:
    tACK a380922891
  fjahr:
    tACK a380922891

Tree-SHA512: 7417d733b47629f229c5128586569909250481a3e94356c52fe67a03fd42cd81745246e384b98c4115fb61587714c879e4bc3e5f5c74407d9f8f6773472a33cb
2022-01-28 08:46:03 +01:00
Andrew Chow
3d223712d3
Merge bitcoin/bitcoin#16795: rpc: have raw transaction decoding infer output descriptors
6498ba151b transaction decoding infer output descriptors (Gregory Sanders)

Pull request description:

  Following discussion in #16725 this is complementary data to expose. All outputs are inferred.

ACKs for top commit:
  achow101:
    ACK 6498ba151b
  meshcollider:
    utACK 6498ba151b

Tree-SHA512: 36664117ddbe46d5fdde7ed6541ef2c9d8dfb7a3636b97f363bf1c325096fe00d9d2acea2d1917ea19fdb82f1ea296c12e440c5c703d6a9bfc1a02fba028bcd8
2022-01-26 17:52:51 -05:00
MarcoFalke
2935bd9d67
Merge bitcoin/bitcoin#24113: test, bugfix: fix intermittent failure in getrawtransaction_tests
449dffc610 test, bugfix: fix intermittent failure in getrawtransaction_tests (Jon Atack)

Pull request description:

  Easiest to review with `colorMoved = dimmed-zebra` and `colorMovedWs = allow-indentation-change`.

  Resolves #23991

Top commit has no ACKs.

Tree-SHA512: a50c597857e45c52dae1b6fc5988ea077c09f2b32ffe3605f0c64bd780f5dd2635755cb26c4981233bf925c5ea9f7310163349a549239dc85657558f253ffb7b
2022-01-26 11:04:17 +01:00
MarcoFalke
dd405add6e
Merge bitcoin/bitcoin#24154: test: add functional test for -maxtipage parameter
75656adfd2 test: add functional test for `-maxtipage` parameter (Sebastian Falbesoner)

Pull request description:

  This PR adds a missing test for the `-maxtipage` parameter which controls what is the allowed maximum tip age for leaving IBD:
  792d0d8d51/src/init.cpp (L540)

  Relevant code path in the `CChainState::IsInitialBlockDownload` method:
  792d0d8d51/src/validation.cpp (L1479-L1480)

  The test is pretty simple and should be self-explanatory.

ACKs for top commit:
  MarcoFalke:
    review ACK 75656adfd2

Tree-SHA512: 0a10dca13cb18c29e64fc8412f4c8f2bcaff1bab8645bd85266c242ba88ce036a150c03cbbe9810c3bb44649810af0aa9cb3584dbae886a7bdb16b72150d08de
2022-01-26 07:36:07 +01:00
Gregory Sanders
6498ba151b transaction decoding infer output descriptors 2022-01-26 09:56:51 +08:00
Andrew Chow
e30b6ea194
Merge bitcoin/bitcoin#24067: wallet: Actually treat (un)confirmed txs as (un)confirmed
fac8165443 Remove unused checkFinalTx (MarcoFalke)
fa272eab44 wallet: Avoid dropping confirmed coins (MarcoFalke)
888841ea8d interfaces: Remove unused is_final (MarcoFalke)
dddd05e7a3 qt: Treat unconfirmed txs as unconfirmed (MarcoFalke)

Pull request description:

  The wallet has several issues:

  ## Unconfirmed txs in the GUI

  The GUI clumsily attempts to guess if unconfirmed txs are locked until a future time. This is currently based on the locktime only, not nSequence, thus wrong. Fix this by removing the clumsy code and treat all unconfirmed txs as unconfirmed. The GUI already prints whether a tx is in the mempool, in which case the user knows that the tx wasn't locked until a future time. If the tx is not in the mempool, it might be better to report the exact reject reason from the mempool instead of using incorrect heuristics.

  ## Confirmed txs in the wallet

  The wallet drops coins that it incorrectly assumes to be locked until a future time, even if they are already confirmed in the chain. This is because the wallet is using the wrong time (adjusted network time) instead of MTP, due to the `-1` default argument of `CheckFinalTx`.

  The issues are fixed in separate commits and there is even a test.

ACKs for top commit:
  achow101:
    ACK fac8165443
  prayank23:
    reACK fac8165443
  glozow:
    code review ACK fac8165443, I understand now how this fixes both issues.

Tree-SHA512: 210afb855f4c6d903fee49eba6b1a9735d699cf0168b669eabb38178e53b3a522258b7cc669f52489c6cd3e38bf358afde12eef3ba2e2f2ffaeb06b8f652ccd0
2022-01-25 16:17:51 -05:00
MarcoFalke
39d9bbe4ac
Merge bitcoin/bitcoin#23706: rpc: getblockfrompeer followups
923312fbf6 rpc: use peer_id, block_hash for FetchBlock (Sjors Provoost)
34d5399211 rpc: more detailed errors for getblockfrompeer (Sjors Provoost)
60243cac72 rpc: turn already downloaded into error in getblockfrompeer (Sjors Provoost)
809d66bb65 rpc: clarify getblockfrompeer behavior when called multiple times (Sjors Provoost)
0e3d7c5ee1 refactor: drop redundant hash argument from FetchBlock (Sjors Provoost)
8d1a3e6498 rpc: allow empty JSON object result (Sjors Provoost)
bfbf91d0b2 test: fancier Python for getblockfrompeer (Sjors Provoost)

Pull request description:

  Followups from #20295.

ACKs for top commit:
  jonatack:
    ACK 923312fbf6 📦
  fjahr:
    tested ACK 923312fbf6

Tree-SHA512: da9eca76e302e249409c9d7f0d16cca668ed981e2ab6ca2d1743dad0d830b94b1bc5ffb9028a00764b863201945c273cc8f4409a4c9ca3817830007dffa2bc20
2022-01-25 18:48:41 +01:00
laanwj
b94d0c7af1
Merge bitcoin/bitcoin#23201: wallet: Allow users to specify input weights when funding a transaction
3866272c45 tests: Test specifying input weights (Andrew Chow)
6fa762a372 rpc, wallet: Allow users to specify input weights (Andrew Chow)
808068e90e wallet: Allow user specified input size to override (Andrew Chow)
4060c50d7e wallet: add input weights to CCoinControl (Andrew Chow)

Pull request description:

  When funding a transaction with external inputs, instead of providing solving data, a user may want to just provide the maximum signed size of that input. This is particularly useful in cases where the input is nonstandard as our dummy signer is unable to handle those inputs.

  The input weight can be provided to any input regardless of whether it belongs to the wallet and the provided weight will always be used regardless of any calculated input weight. This allows the user to override the calculated input weight which may overestimate in some circumstances due to missing information (e.g. if the private key is not known, a maximum size signature will be used, but the actual signer may be doing additional work which reduces the size of the signature).

  For `send` and `walletcreatefundedpsbt`, the input weight is specified in a `weight` field in an input object. For `fundrawtransaction`,  a new `input_weights` field is added to the `options` object. This is an array of objects consisting of a txid, vout, and weight.

  Closes #23187

ACKs for top commit:
  instagibbs:
    reACK 3866272c45
  glozow:
    reACK 3866272 via range-diff
  t-bast:
    ACK 3866272c45

Tree-SHA512: 2c8b471ee537c62a51389b7c4e86b5ac1c3a223b444195042be8117b3c83e29c0619463610b950cbbd1648d3ed01ecc5bb0b3c4f39640680da9157763b9b9f9f
2022-01-25 17:51:05 +01:00
Sebastian Falbesoner
75656adfd2 test: add functional test for -maxtipage parameter 2022-01-25 16:20:00 +01:00
MarcoFalke
fa272eab44 wallet: Avoid dropping confirmed coins 2022-01-25 10:15:12 +01:00
fanquake
0147278e37
Merge bitcoin/bitcoin#21464: Mempool Update Cut-Through Optimization
c5b36b1c1b Mempool Update Cut-Through Optimization (Jeremy Rubin)
c49daf9885 [TESTS] Increase limitancestorcount in tournament RPC test to showcase improved algorithm (Jeremy Rubin)

Pull request description:

  Often when we're updating mempool entries we update entries that we ultimately end up removing the updated entries shortly thereafter. This patch makes it so that we filter for such entries a bit earlier in processing, which yields a mild improvement for these cases, and is negligible overhead otherwise.

  There's potential for a better -- but more sophisticated -- algorithm that can be used taking advantage of epochs, but I figured it is better to do something that is simple and works first and upgrade it later as the other epoch mempool work proceeds as it makes the patches for the epoch algorithm simpler to understand, so you can consider this as preparatory work. It could either go in now if it is not controversial, or we could wait until the other patch is ready to go.

ACKs for top commit:
  instagibbs:
    reACK c5b36b1
  sipa:
    utACK c5b36b1c1b
  mzumsande:
    Code Review ACK c5b36b1c1b

Tree-SHA512: 78b16864f77a637d8a68a65e23c019a9757d8b2243486728ef601d212ae482f6084cf8e69d810958c356f1803178046e4697207ba40d6d10529ca57de647fae6
2022-01-25 11:20:18 +08:00
Andrew Chow
3866272c45 tests: Test specifying input weights
Added tests to rpc_fundrawtransaction, wallet_send, and rpc_psbt that
test that external inputs can be spent when input weight is provided.
Also tested that the input weight overrides any calculated weight.

Additionally, rpc_psbt's external inputs test is cleaned up a bit to be
more similar to rpc_fundrawtransaction's and avoid potential pitfalls
due to non-deterministic coin selection behavior.
2022-01-24 11:29:38 -05:00
Jon Atack
449dffc610
test, bugfix: fix intermittent failure in getrawtransaction_tests 2022-01-20 21:15:04 +01:00
S3RK
3ee6d0788e test: add more wallet conflicts assertions
verify wallet conflicts from the receiver's side
2022-01-20 10:13:03 +01:00
fanquake
a541e5d519
Merge bitcoin/bitcoin#24104: fs: Make compatible with boost 1.78
dc5d6b0d47 fs: Make compatible with boost 1.78 (Andrew Chow)

Pull request description:

  Boost 1.78 removed `operator+` in a way that breaks our usage of it in a subclass. A [proposed workaround](https://github.com/boostorg/filesystem/issues/223#issuecomment-1000230207) for this is to cast the argument to `boost::filesystem::path`, and this is backwards compatible with older versions of boost.

  Additionally, it appears that `fs::canonical` no longer removes trailing slashes. This was causing a test to fail. The solution is to explicitly remove the trailing separator in the one place that `fs::canonical` is used.

  Lastly, `fs::create_directories` now has an error message saying `create_directories` instead of `create_directory`. This caused wallet_multiwallet.py to fail. The error message check has been updated to be able accept either string.

  Fixes #23846

ACKs for top commit:
  ryanofsky:
    Code review ACK dc5d6b0d47
  vincenzopalazzo:
    ACK dc5d6b0d47

Tree-SHA512: d4d8e7b49b8dfbf0ced9bfe9a2b3827841227fc755fc799f19159076b0ccf882432cc8b6ad93cdeda98fb58b942b9ba50a9e0a6b4f6b1e0097e80f1074ae5682
2022-01-20 13:06:04 +08:00
Andrew Chow
dc5d6b0d47 fs: Make compatible with boost 1.78 2022-01-19 16:25:20 -05:00
MarcoFalke
5c3bfee46a
Merge bitcoin/bitcoin#24054: test: rest /tx with an invalid/unknown txid
bd52684508 test: rest /tx with an invalid/unknown txid (brunoerg)

Pull request description:

  This PR adds test coverage to the endpoint `/tx` (rest) passing an invalid and an unknown txid to test its return.
  Invalid -> should return status code 400 (bad request)
  Unknown -> should return status code 404 (not found)

ACKs for top commit:
  kallewoof:
    ACK bd52684508

Tree-SHA512: a7fbb63f30d06fc0855133a36e8317c7930ba13aa2b4a2dd1fc35079d59eacace72e1ffe7ae1b3e067066fe51792415940d72d923e83a659a0d5965e4110b32a
2022-01-19 08:42:42 +01:00
Anthony Towns
240cad09ba rpc: getdeploymentinfo: include signalling info 2022-01-15 04:37:56 +10:00
Anthony Towns
376c0c6dae rpc: getdeploymentinfo: include block hash/height 2022-01-15 04:37:56 +10:00
Anthony Towns
a7469bcd35 rpc: getdeploymentinfo: change stats to always refer to current period
On a period boundary, getdeploymentinfo (and previously getblockchaininfo)
would report the status and statistics for the next block rather than
the current block. Change this to always report the status/statistics
of the current block, but add status-next to report the status for the
next block.
2022-01-15 04:37:56 +10:00
Anthony Towns
fd826130a0 rpc: move softfork info from getblockchaininfo to getdeploymentinfo 2022-01-15 04:37:56 +10:00
brunoerg
bd52684508 test: rest /tx with an invalid/unknown txid 2022-01-13 17:00:18 -03:00
MarcoFalke
807169e10b
Merge bitcoin/bitcoin#24035: test: use MiniWallet for mempool_accept.py
aa8a65e4a8 test: use MiniWallet for mempool_accept.py (Sebastian Falbesoner)
b24f6c6855 test: MiniWallet: support default `from_node` for creating txs (Sebastian Falbesoner)
f30041c914 test: create txs with current `nVersion` (2) by default (Sebastian Falbesoner)
2f79786822 test: refactor: add constant for sequence number `SEQUENCE_FINAL` (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (mempool_accept.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078.

  It also includes some other minor changes that came up while working on the replacement:
  * [commit 1/4] replace magic number 0xffffffff for a tx's nSequence with a new constant `SEQUENCE_FINAL`
  * [commit 2/4] create `CTransaction` instances with the current nVersion=2 by default, in order to use BIP68 for tests
  * [commit 3/4] support default `from_node` parameter for creating txs (this is a stripped down version of PR #24025)

ACKs for top commit:
  MarcoFalke:
    re-ACK aa8a65e4a8 📊

Tree-SHA512: 34cd085ea4147ad5bd3f3321c84204064ceb95f382664c7fe29062c1bbc79d9d9465c5e46d35e11c416f2f3cd46030c90a09b518c829c73ae40d060be5e4c9cb
2022-01-13 18:42:24 +01:00
Sebastian Falbesoner
aa8a65e4a8 test: use MiniWallet for mempool_accept.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-01-13 18:32:07 +01:00
Pieter Wuille
b5f33ac1f8 Simplify wallet_taproot.py functional test 2022-01-12 11:09:41 -05:00
Pieter Wuille
eb0667ea96 Add tests for (sorted)multi_a derivation/signing 2022-01-12 11:09:41 -05:00
sogoagain
9d3e95d77c [bugfix] prevent UnicodeDecodeError errors when opening log file in feature_init.py
open log file as a raw byte stream in feature_init.py
(fixes #23989)
2022-01-12 02:06:15 +09:00
Sebastian Falbesoner
b24f6c6855 test: MiniWallet: support default from_node for creating txs
If no `from_node` parameter is passed explicitely to the
`create_self_transfer` method, the test node passed in the course
of creating the MiniWallet instance is used.  This seems to
be the main use-case in most of the current functional
tests, i.e. in many instances the calls can be shortened.
2022-01-11 16:22:33 +01:00
Sebastian Falbesoner
f30041c914 test: create txs with current nVersion (2) by default
This enables testing of BIP68 without the need of explicitly
setting nVersion to 2. This is e.g. useful for transactions
created with MiniWallet.
2022-01-11 16:22:27 +01:00
Sebastian Falbesoner
2f79786822 test: refactor: add constant for sequence number SEQUENCE_FINAL 2022-01-11 15:28:28 +01:00
fanquake
be72ae25a8
Merge bitcoin/bitcoin#23996: test: set ban after mocking time
c9374af102 test: set ban after mocking time (brunoerg)

Pull request description:

  Fixes #23988

  Set ban after mocking time to avoid intermittent failures related to the assertion of ban_duration and time_remaining.

  See: https://cirrus-ci.com/task/6754020390862848?logs=ci#L4652

ACKs for top commit:
  mzumsande:
    Tested ACK c9374af102
  vincenzopalazzo:
    ACK c9374af102

Tree-SHA512: fac3ac91a045bb46334d7c568f6a53a3b0a45b306914a54ea13bcc845734eaaad1ff295ff3ab158037fd9d08df77344058331336110b8f7888832b16b0589be5
2022-01-07 07:49:13 +08:00
brunoerg
c9374af102 test: set ban after mocking time 2022-01-06 15:33:43 -03:00
MarcoFalke
e941507500
Merge bitcoin/bitcoin#23963: test: run feature_pruning.py without wallet compiled
0754e9c01b test: run feature_pruning.py without wallet compiled (Sebastian Falbesoner)

Pull request description:

  Only one small part of the pruning test (sub-test `wallet_test`) is wallet-related, hence we can run all other parts without wallet compiled.

ACKs for top commit:
  MarcoFalke:
    cr ACK 0754e9c01b

Tree-SHA512: 856856903d21d64953ed0102cc2a96f55975c4b7d8e93e57b82c266024967160df64df2b6068be089efc05e883e8d6d12e7327053420d4c640b9d8cc5bcb1c58
2022-01-06 15:02:46 +01:00
MarcoFalke
799fd7a488
Merge bitcoin/bitcoin#23075: test: Fee estimation functional test cleanups
60ae1161a4 qa: replace assert with test framework assertion helpers in fee estimation test (Antoine Poinsot)
e50213967b qa: fee estimation with RBF test cleanups (Antoine Poinsot)
15f5fd62af qa: don't mine non standard txs in fee estimation test (Antoine Poinsot)
eae52dd6ab qa: pass scriptsig directly to txins constructor in fee estimation test (Antoine Poinsot)
1fc03155e5 qa: split coins in a single tx in fee estimation test (Antoine Poinsot)
cc204b8be7 qa: use a single p2sh script in fee estimation test (Antoine Poinsot)
19dd91a9be qa: remove a redundant condition in fee estimation test (Antoine Poinsot)

Pull request description:

  Some cleanups that i noticed would be desirable while working on  #23074 and #22539, which are intentionally not based on it. Mainly simplifications and a slight speedup.

  - Use a single tx to create the `2**9` coins instead of creating `2**8` 2-outputs transactions
  - Use a single P2SH script
  - Avoid the use of non-standard transactions
  - Misc style nits (happy to take more)

ACKs for top commit:
  pg156:
    I ACK all commits up to 60ae1161a4 (except 1fc03155e5, where I have a question more for my own learning than actually questioning the PR). I built and ran the test successfully. I agree after the changes, the behavior is kept the same and the code is shorter and easier to reason.
  glozow:
    utACK 60ae1161a4

Tree-SHA512: 57ae2294eb68961ced30f32448c4a530ba1cdee17881594eecb97e1b9ba8927d58c25022b847eb07fb67d676bf436108c416c2f2174864d258fcca5b528b8bbd
2022-01-06 13:27:12 +01:00
laanwj
801aaac2b3
Merge bitcoin/bitcoin#23834: wallettool: Check that the dumpfile checksum is the correct size
ac617cc141 wallettool: Check that the dumpfile checksum is the correct size (Andrew Chow)

Pull request description:

  After parsing the checksum, make sure that it is the size that we expect it to be.

  This issue was reported by Pedro Baptista.

ACKs for top commit:
  laanwj:
    Code review ACK ac617cc141

Tree-SHA512: 8135b3fb1f4f6b6c91cfbac7d1d3421f1f6c664a742c92940f68eae857f92ce49d042cc3aa5c2df6ef182825271483d65efc7543ec7a8ff047fd7c08666c8899
2022-01-05 19:16:28 +01:00
laanwj
121d47afe3
Merge bitcoin/bitcoin#23799: test: Let test_runner.py start multiple jobs per timeslot
975097f424 Let test_runner.py start multiple jobs per timeslot (Pieter Wuille)

Pull request description:

  test_runner.py currently only checks every 0.5s whether any job has finished, and if so, starts at most one new job. At higher parallellism it becomes increasingly likely that multiple jobs have finished at the same time. Fix this by always noticing *all* finished jobs every timeslot, and starting as many new ones.

ACKs for top commit:
  laanwj:
    Code review and lightly tested ACK 975097f424
  prayank23:
    ACK 975097f424

Tree-SHA512: b70c51f05efcde9bc25475c192b86e86b4c399495b42dee20576af3e6b99e8298be8b9e82146abdabbaedb24a13ee158a7c8947518b16fc4f33a3b434935b550
2022-01-05 17:19:54 +01:00
MarcoFalke
e00e990606
Merge bitcoin/bitcoin#23978: test: use MiniWallet for mining_basic.py
7746606cfa test: use MiniWallet for mining_basic.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (mining_basic.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078.

ACKs for top commit:
  brunoerg:
    crACK 7746606cfa

Tree-SHA512: 4455b8b764413b0fc3ef388e3c5d5758f9e6b6d3193ac660269a9ba1c988022e6b7bc148549c2167942ea472c5aaddd2b6b3b9d99790d0654b089af975b86e11
2022-01-05 09:35:22 +01:00
fanquake
96eb0093d0
test: wait rather than assert presence of file in startupnotify test
Should fix #23967.
2022-01-05 09:25:43 +08:00
Sebastian Falbesoner
7746606cfa test: use MiniWallet for mining_basic.py
This test can now be run even with the Bitcoin Core wallet disabled.
2022-01-04 19:39:23 +01:00
Sebastian Falbesoner
0754e9c01b test: run feature_pruning.py without wallet compiled
Only one small part of the pruning test (sub-test `wallet_test`) is
wallet-related, hence we can run all other parts without wallet compiled.
2022-01-03 22:13:46 +01:00
MarcoFalke
d69af93223
Merge bitcoin/bitcoin#23737: test: make feature_init more robust
8904f17ea7 test: enable txindex in feature_init (James O'Beirne)
93db6d8422 test: feature_init: retain debug.log and improve detection (James O'Beirne)
24fcf6e435 test: feature_init: tweak all .ldb files of a certain type (James O'Beirne)
a8ffbc01db test: add TestNode.wait_for_debug_log (James O'Beirne)
a2fb62b632 test: introduce TestNode.debug_log_bytes (James O'Beirne)

Pull request description:

  Hopefully fixes #23646.

  This makes a few changes to feature_init (along the lines of the discussion here: https://github.com/bitcoin/bitcoin/pull/23289#discussion_r766615073). The changes are detailed in the specific commits, but a summary might be:
  - more robust log-line detection (at the expense of maybe being slower)
  - retain all debug.log content
  - perturb .ldb files in a more complete way

Top commit has no ACKs.

Tree-SHA512: 9c8900998fdb96d3cc92bd30c4eeda2a47e9787e6432546d458c17f24eb9e2b6b3eb573db225172888ca71d7a62260f497f83827597b982b9e35c5452cb6e514
2022-01-03 08:51:09 +01:00
MarcoFalke
31f385c138
Merge bitcoin/bitcoin#23532: test: add functional test for -startupnotify
126853214a test: add functional test for -startupnotify (Bruno Garcia)

Pull request description:

  This PR adds a functional test for -startupnotify. It basically starts the node passing a command on -startupnotify to create a file on tmp and then, we check if the file has been successfully created.

ACKs for top commit:
  theStack:
    Tested ACK 126853214a
  kristapsk:
    re-ACK 126853214a

Tree-SHA512: 5bf3e46124ee5c9d609c9993e6465d5a71a8d2275dcf07c8ce0549f013f7f8863d483b46b7164152f566468a689371ccb87f01cf118c3c9cac5b2be673b61a5c
2022-01-03 08:47:02 +01:00
fanquake
9d099b02d8
Merge bitcoin/bitcoin#23836: rest: Expose block filters follow-ups
4523d28b6b [test] compare filter and header with the result of the getblockfilter RPC (Niklas Gögge)
3a2464f216 [rest] drop superfluous rpc serializations flags for block filters (Niklas Gögge)
064abd14a5 [rest] add a more verbose error message for invalid header counts (Niklas Gögge)
83b8f3a896 [refactor] various style fix-ups (Niklas Gögge)

Pull request description:

  This PR addresses unresolved review comments from [#17631](https://github.com/bitcoin/bitcoin/pull/17631).
  This includes:
  * various style fix-ups
  * returning a more verbose error message for invalid header counts
  * removing superfluous rpc serializations flags for block filters
  * improving the test to include comparing the block filters returned from the rest  with the ones returned from the `getblockfilter` RPC.

ACKs for top commit:
  jnewbery:
    ACK 4523d28b6b
  brunoerg:
    tACK 4523d28b6b

Tree-SHA512: 634e6b2ae3e1d5f31675a50cfe11a5e74bf5a51b9e7b512d9e18879bf2ed424fc0ac6ec633023f192e3ad12cf0c73b0b51de79dd7ec00844dba3e1493d823b8c
2022-01-02 08:37:12 +08:00
MarcoFalke
fa4c599145
rpc: Quote user supplied descriptor in error msg 2022-01-01 00:26:08 +13:00
MarcoFalke
623745ca74
Merge bitcoin/bitcoin#23912: Insert and bump copyright headers
1362d6173f scripted-diff: Insert missed copyright headers (Hennadii Stepanov)
f47dda2c58 scripted-diff: Bump copyright headers (Hennadii Stepanov)
c29105efdc script: Fix copyright_header.py (Hennadii Stepanov)

Pull request description:

  This PR is an alternative to #23903.

  It bumps the existing copyright headers as we did every year, and adds the missed copyright headers.

  A small fix has been applied to the `copyright_header.py` in order to prevent such weird bumping as `2021` --> `2021-2017`.

ACKs for top commit:
  MarcoFalke:
    ACK 1362d6173f

Tree-SHA512: 204d970fe8c51546b26b8f03fe4297db8a9bef5101df851540b7b9eddbd3a09677ee81fdd882c60937d732407f42c9883165bd978272200cff8f90190f075905
2021-12-31 12:08:49 +01:00
Hennadii Stepanov
f47dda2c58
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
* 2020: fa0074e2d8
* 2019: aaaaad6ac9
2021-12-30 19:36:57 +02:00
MarcoFalke
f3d6a5ce77
Merge bitcoin/bitcoin#23879: test: check ban_duration and time_remaining after setting ban
da349f131a test: check ban_duration and time_remaining after setting ban (brunoerg)

Pull request description:

  This PR adds functional test coverage for `ban_duration` and `time_remaining` introduced in #21602

ACKs for top commit:
  shaavan:
    ACK da349f131a
  theStack:
    Tested ACK da349f131a

Tree-SHA512: 51e63f3a36adb1c81e4d49426486af2cd9c8c4319f94e06a47fa7da8100a8b53c029d28d4a4771bdbf4e0a2bfb4ddd3740b9974bd08d8ff06f2a0fc2b6d8a6b5
2021-12-30 11:10:46 +01:00
James O'Beirne
8904f17ea7
test: enable txindex in feature_init
Now that #23365 is merged.
2021-12-29 13:21:24 -05:00
James O'Beirne
93db6d8422
test: feature_init: retain debug.log and improve detection
This test sporadically fails due to the Python test missing log lines
for reasons that are poorly understood. The problem is made worse by the
fact that this test does not retain the log files from iteration to
iteration.

Change the test to do logline detection in a more robust manner (by
using `re.search` on the whole log content) in a way that is comparable
to the existing `assert_debug_log` utility, and retain all debug.log
content from case to case.
2021-12-29 13:04:36 -05:00
James O'Beirne
24fcf6e435
test: feature_init: tweak all .ldb files of a certain type
This part of the test sporadically fails on CI infrastructure. Instead
of perturbing a single .ldb file of each type, move all .ldb files of a
given type to ensure a bad startup.
2021-12-29 13:04:35 -05:00
James O'Beirne
a8ffbc01db
test: add TestNode.wait_for_debug_log 2021-12-29 13:04:34 -05:00
James O'Beirne
a2fb62b632
test: introduce TestNode.debug_log_bytes 2021-12-29 13:04:29 -05:00
brunoerg
da349f131a test: check ban_duration and time_remaining after setting ban 2021-12-27 11:39:07 -03:00
Sebastian Falbesoner
d2efb66458 test: use MiniWallet for p2p_compactblocks.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-12-27 14:39:22 +01:00
Andrew Chow
ac617cc141 wallettool: Check that the dumpfile checksum is the correct size
After parsing the checksum, make sure that it is the size that we expect
it to be.
2021-12-26 12:33:23 -05:00
Sebastian Falbesoner
e844115dea test: use MiniWallet for rpc_scantxoutset.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-12-26 15:52:45 +01:00
Sebastian Falbesoner
983ca0456c test: introduce address_to_scriptpubkey helper
Works only with legacy addresses (Base58Check) right now.
2021-12-26 12:10:56 +01:00
Sebastian Falbesoner
e704d4d26f test: introduce getnewdestination helper for generating various address types
This serves as a replacement for the getnewaddress RPC if no wallet is
available. In addition to the address, it also returns the corresponding
public key and output script (scriptPubKey).
2021-12-26 12:10:52 +01:00
Sjors Provoost
34d5399211
rpc: more detailed errors for getblockfrompeer 2021-12-24 16:29:03 +01:00
Sjors Provoost
60243cac72
rpc: turn already downloaded into error in getblockfrompeer 2021-12-24 16:29:03 +01:00
Bruno Garcia
126853214a test: add functional test for -startupnotify 2021-12-24 08:41:43 -03:00
MarcoFalke
fafe4dea16
test: Fix pep8 of touched file
Can be reviewed with --word-diff-regex=. --ignore-all-space
2021-12-24 12:04:25 +01:00
MarcoFalke
fa0ac9d7e3
test: Fix rpc_scantxoutset intermittent issue 2021-12-24 12:03:21 +01:00
Niklas Gögge
4523d28b6b [test] compare filter and header with the result of the getblockfilter RPC 2021-12-22 20:49:47 +01:00
Niklas Gögge
064abd14a5 [rest] add a more verbose error message for invalid header counts 2021-12-22 20:47:53 +01:00
Sebastian Falbesoner
c27bba9672 test: check for invalid listtransactions RPC parameters 2021-12-22 20:44:44 +01:00
Andrew Chow
3ac38058ce
Merge bitcoin/bitcoin#23789: wallet: Strictly match tx change type to improve privacy
fada6c65d2 wallet: Strictly match tx change type to improve privacy (MarcoFalke)

Pull request description:

  Currently the change type will only match a destination by accident, making it easier to determine the change.

  Fix that by strictly matching one of the destinations.

ACKs for top commit:
  S3RK:
    Concept & Approach ACK fada6c6. Also did light code review .
  achow101:
    ACK fada6c65d2
  prayank23:
    tACK fada6c65d2
  w0xlt:
    tACK fada6c6

Tree-SHA512: 2b072c3c32debac7b0bef07a6df9a8f1a631e0f7d556b859973f18894ca490225582dc13e4588b29fa205ffbcd30fb632d5313b304d10ad17a26adc3f7684471
2021-12-20 14:48:22 -05:00
MarcoFalke
9ac064d245
Merge bitcoin/bitcoin#23796: test: check that pruneblockchain RPC fails for future block or timestamp
140a49ce5e test: check that pruneblockchain RPC fails for future block or timestamp (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `pruneblockchain` RPC for the case that a future block or timestamp is passed:
  8c0bd871fc/src/rpc/blockchain.cpp (L1101)
  8c0bd871fc/src/rpc/blockchain.cpp (L1111)
  Note that the test method `manual_test` gets called twice, once each with `use_timestamp` set to True/False, respectively. Depending on that, the helper function `height` either converts the passed block height to the timestamp of that block, or just returns it without modification.

  The other tests for failures in this RPC are also changed to be more detailled ("Cannot prune blocks because node is not in prune mode", "Negative block height"), as I don't think there is any value in just checking a sub-string. If there is ever an error with the same sub-string is introduced, it's not clear which error is exactly checked with the test, so it makes sense to be as specific as possible.

ACKs for top commit:
  brunoerg:
    tACK 140a49ce5e

Tree-SHA512: bee3cee9f35c2a63a1839d7ec1f83e354d9d3c0c2ca32d300dca2de8b755d555f769ba2b80ac37d31df6ee7e2b8eaefb8134c4727a7144e47c0f5e34f2cc5822
2021-12-20 13:32:57 +01:00
MarcoFalke
1c41fb9181
Merge bitcoin/bitcoin#23812: test: fix intermittent failures in p2p_timeouts.py
0a1b6fa5a1 test: fix intermittent timeouts in p2p_timeouts.py (Martin Zumsande)

Pull request description:

  Fixes  #23800 by making sure that all peers are connected (i.e. `m_connected` is set) before the mocktime is bumped.
  We can't wait for verack here, but we can wait for a debug log entry ("Added connection peer=2") instead.

  In the failed CI runs (e.g. https://cirrus-ci.com/task/5600553806856192?logs=ci#L7469)  different peers were added at different mocktimes.

ACKs for top commit:
  naumenkogs:
    ACK 0a1b6fa5a1
  theStack:
    Concept and approach ACK 0a1b6fa5a1

Tree-SHA512: 1a3c8a9a79339d4adc6ecb1731eb0d0eadb2e5024ad3c6779b4696691f85d6c3304ef8689746d0332150a4cf04489ca4b2ff3eeb0bb76feec28c1e4bb9dbca19
2021-12-20 09:13:20 +01:00
MarcoFalke
70d6a09f5c
Merge bitcoin/bitcoin#17631: Expose block filters over REST
2b64fa3251 Update REST docs with new accessors (Matt Corallo)
ef7c8228fd Expose block filters over REST. (Matt Corallo)

Pull request description:

  This adds a new rest endpoint:
  /rest/blockfilter/filtertype/requesttype/blockhash (eg
  /rest/blockfilter/basic/header/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f.hex)
  which exposes either the filter "header" or the filter data itself.
  Most of the code is cribbed from the equivalent RPC.

  You can test it at 000000005b.hex

ACKs for top commit:
  dergoegge:
    ACK 2b64fa3251 - Adding blockfilters to the REST interface is analogous to serving other public data such as transactions or blocks.

Tree-SHA512: d487bc694266375c94d6fcf2e9d788a8a42a3b94e8d3290e46335a64cbcde55084ce5ea6119b79a4065888d94d7c3ae25a59a901fa46e3711f0eb296add12696
2021-12-20 09:00:34 +01:00
Martin Zumsande
0a1b6fa5a1 test: fix intermittent timeouts in p2p_timeouts.py
by checking that all nodes are added before the mocktime is bumped.
Fixes #23800
2021-12-18 11:40:55 +01:00
MarcoFalke
fada6c65d2
wallet: Strictly match tx change type to improve privacy 2021-12-17 07:59:39 +01:00
Pieter Wuille
975097f424 Let test_runner.py start multiple jobs per timeslot 2021-12-16 15:19:52 -05:00
Sebastian Falbesoner
140a49ce5e test: check that pruneblockchain RPC fails for future block or timestamp 2021-12-16 16:43:27 +01:00
MarcoFalke
6b212cbbb9
Merge bitcoin/bitcoin#23720: test: Refactor addr_relay.py addr generation, increase mocktime
261dddb924 test: Combine addr generation helper functions (Martin Zumsande)
aeeccd9aa6 test: Fix intermittent issue in p2p_addr_relay.py (Martin Zumsande)

Pull request description:

  Fixes #22449 by increasing the mocktime jump (just as in 6168eb06b2), which prevents failures due to rare Poisson timer events, or at least makes them a lot more unlikely.

  The second commit combines the addr generation helper functions `setup_addr_msg` and `setup_rand_addr_msg`. It also changes the way `addr.time` is filled to random, because before, if too many addresses (>600) were created in a batch, they would stop being relayed because their timestamp would be too far in the future.

ACKs for top commit:
  josibake:
    reACK 261dddb924
  jnewbery:
    utACK 261dddb924

Tree-SHA512: d0eca887de4bc85092730284cf612193d2c12b0a3d624a2bfa5fef4a5890d3b6375c564333c5927425958e4b6ec86b8854b18b2233c7b6f1691d9ddc397948a9
2021-12-16 15:07:45 +01:00
Martin Zumsande
261dddb924 test: Combine addr generation helper functions
This combines the addr generation helper functions setup_addr_msg
and setup_rand_addr_msg.
It also changes the way addr.time is filled to random, because before,
if too many addresses (>600) were created in a batch, they would stop
being relayed because their timestamp would be too far in the future.
2021-12-16 12:43:38 +00:00
Martin Zumsande
aeeccd9aa6 test: Fix intermittent issue in p2p_addr_relay.py
by increasing the mocktime bump for m_next_addr_send, which is on a
Poisson timer, and explain why. Closes #22449
2021-12-16 12:43:38 +00:00
MarcoFalke
df6e961c41
Merge bitcoin/bitcoin#23777: doc: follow-ups from #23365 (index backwards search)
e4a8d561ed doc: add explanations for assert in index and magic numbers in test (Martin Zumsande)

Pull request description:

  This adds two explanations suggested in the review of #23365,  that I didn't manage to address before that PR was merged:

  https://github.com/bitcoin/bitcoin/pull/23365#discussion_r763981042
  https://github.com/bitcoin/bitcoin/pull/23365#discussion_r763982639

ACKs for top commit:
  jnewbery:
    ACK e4a8d561ed

Tree-SHA512: 0500c8abb37bb3e3694463ad5e74b2e1483615ccf1d7529b0d5faa694652ada17d242dc7fda6d995733766c627d54178a2c8fa21a570cdf13292f64ff5425b56
2021-12-16 08:55:06 +01:00
MarcoFalke
a30642926a
Merge bitcoin/bitcoin#23721: wallet, refactor: Move restorewallet() logic to the wallet section
62fa61fa4a refactor: remove the wallet folder if the restore fails (w0xlt)
abbb7eccef refactor: Move restorewallet() RPC logic to the wallet section (w0xlt)
4807f73f48 refactor: Implement restorewallet() logic in the wallet section (w0xlt)

Pull request description:

  Currently `restorewallet()` logic is written in the RPC layer and it can´t be reused by GUI. So it moves this to the wallet section and then, GUI can access it.

  This is necessary to implement the "Restore Wallet" menu item in the GUI (which is already implemented  in https://github.com/bitcoin-core/gui/pull/471 ).

  This commit also simplifies error handling and adds a new behavior: if the restore fails, the invalid wallet folder is removed.

ACKs for top commit:
  achow101:
    ACK 62fa61fa4a
  shaavan:
    crACK 62fa61fa4a

Tree-SHA512: 7ccfbad5943f38616ba0c2dd443c97a4b5bc1f6612dbf5a9e7a0263100aba36671fae929a2e7688442667be394645f44484af137a4802f204a33c4689eb27c39
2021-12-16 08:42:44 +01:00
MarcoFalke
98c362a4d3
Merge bitcoin/bitcoin#23782: test: include two more interruptions points
618f4d2890 test: re-organized array according to order of logs and included 2 more interruption events (seaona)
71115a5e23 test: include two more interruptions points (seaona)

Pull request description:

  This PR aims to introduce 2 more interruption points in the process of initialization, in order to make the` feature_init `testcase more complete. These are the following:

  -` Checking all blk files are present`
  -` init message: Starting network threads`

  It is a small improvement for increasing the coverage of potential interruptions, and making sure that the node can restart successfully after these interruptions.

ACKs for top commit:
  jamesob:
    ACK 618f4d2890, pending CI
  jarolrod:
    ACK 618f4d2890

Tree-SHA512: 9d709734e298e955709094bb97478ca7f18859874f1ba026f7c9014d87205aea63f6cf2093ebee600eaf82d3245adb11e77fae24a1ae48b69efefd57f3def921
2021-12-16 07:47:40 +01:00
MarcoFalke
24cec4b5c0
test: Fix intermittent test failure in feature_backwards_compatibility 2021-12-16 12:51:08 +07:00
Sjors Provoost
d8b705f1ca
test: previous releases: add v22.0 2021-12-16 12:41:45 +07:00
Sjors Provoost
40849eebd9
test: bump sandbox argument minimum version
The -sandbox argument is not present in the v22.0 release. Changing the minimum version to 229900 ensures it's used when testing the master branch.

If the argument is backported, the minimum version can be adjusted to e.g. 220100.
2021-12-16 12:41:44 +07:00
Sjors Provoost
8a57a06a50
test: previous releases: add v0.21.0 2021-12-16 12:41:44 +07:00
Sjors Provoost
8cba75f5fd
test: v0.20.1 backwards compatibility
The file checksums were added in an earlier commit. Since the DMG
file is never downloaded, we drop that checksum.
2021-12-16 12:41:44 +07:00
Sjors Provoost
0e4b695b6a
test: backwards compatibility: misc fixes
This cleanup may slightly impact test coverage.

Reduce code repition by looping over the list of nodes.

Reduce brittleness by refering to nodes by name rather than index.

Add helper method nodes_wallet_dir.
2021-12-16 12:41:44 +07:00
w0xlt
62fa61fa4a refactor: remove the wallet folder if the restore fails 2021-12-15 18:41:40 -03:00
w0xlt
abbb7eccef refactor: Move restorewallet() RPC logic to the wallet section
It also simplifies restorewallet() and loadwallet() RPC error handling.
2021-12-15 18:41:40 -03:00
seaona
618f4d2890 test: re-organized array according to order of logs and included 2 more interruption events 2021-12-15 16:17:54 +01:00
seaona
71115a5e23 test: include two more interruptions points 2021-12-15 13:32:32 +01:00
MarcoFalke
60b5795133
Merge bitcoin/bitcoin#23758: net: Use type-safe mockable time for peer connection time
fad943821e scripted-diff: Rename touched member variables (MarcoFalke)
fa663a4c0d Use mockable time for peer connection time (MarcoFalke)
fad7ead146 refactor: Use type-safe std::chrono in net (MarcoFalke)

Pull request description:

  Benefits:
  * Type-safe
  * Mockable
  * Allows to revert a temporary test workaround

ACKs for top commit:
  naumenkogs:
    ACK fad943821e
  shaavan:
    ACK fad943821e

Tree-SHA512: af9bdfc695ab727b100c6810a7289d29b02b0ea9fa4fee9cc1f3eeefb52c8c465ea2734bae0c1c63b3b0d6264ba2c493268bc970ef6916570eb166de77829d82
2021-12-15 13:07:34 +01:00
Martin Zumsande
e4a8d561ed doc: add explanations for assert in index and magic numbers in test
These were suggested in review of #23365

Co-authored-by: John Newbery <john@johnnewbery.com>
2021-12-15 10:05:11 +00:00
Andrew Chow
efa115aa4a tests: Add missing assert_equal import to p2p_add_connections.py 2021-12-14 12:28:25 -05:00
MarcoFalke
9635760ce8
Merge bitcoin/bitcoin#22777: net processing: don't request tx relay on feeler connections
eaf6be0114 [net processing] Do not request transaction relay from feeler connections (John Newbery)
0220b834b1 [test] Add testing for outbound feeler connections (John Newbery)

Pull request description:

  Feelers are short-lived connections used to test the viability of peers. The bitcoind node will periodically open feeler connections to addresses in its addrman, wait for a `version` message from the peer, and then close the connection.

  Currently, we set `fRelay` to `1` in the `version` message for feeler connections, indicating that we want the peer to relay transactions to us. However, we close the connection immediately on receipt of the `version` message, and so never process any incoming transaction announcements. This PR changes that behaviour to instead set `fRelay` to `0` indicating that we do not wish to receive transaction announcements from the peer.

  This PR also extends the `addconnection` RPC to allow creating outbound feeler connections from the node to the test framework, and a test to verify that the node sets `fRelay` to `0` in the `version` message to feeler connections.

ACKs for top commit:
  naumenkogs:
    ACK eaf6be0114
  MarcoFalke:
    review ACK eaf6be0114 🏃

Tree-SHA512: 1c56837dbd0a396fe404a5e39f7459864d15f666664d6b35ad109628b13158e077e417e586bf48946a23bd5cbe63716cb4bf22cdf8781b74dfce6047b87b465a
2021-12-14 17:57:10 +01:00
MarcoFalke
5dd28e5cff
Merge bitcoin/bitcoin#23755: rpc: Quote user supplied strings in error messages
fa24a3df87 rpc: Quote user supplied strings in error messages (MarcoFalke)

Pull request description:

  I can't see a downside doing this and this fixes a fuzzing crash

  Background:

  This is a follow-up to commit 926fc2a0d4, which introduced the "starts_with-hack". Maybe an alternative to the hack would be to assign a unique error code to internal bugs? However, I think this can be done in an separate pull request and the changes here make sense even on their own.

ACKs for top commit:
  fanquake:
    ACK fa24a3df87 - to fix the fuzzers.

Tree-SHA512: d998626406a64396a037a6d1fce22fce3dadb7567c2f9638e450ebe8fb8ae77d134e15dd02555326732208f698d77b0028bc62be9ceee9c43282b61fe95fccbd
2021-12-13 17:13:42 +01:00
MarcoFalke
c8fac372c2
Merge bitcoin/bitcoin#23648: test: Use byte unit 'M' for -maxuploadtarget functional test
41b9f7d062 test: Use byte unit 'M' for -maxuploadtarget functional test (Douglas Chimento)

Pull request description:

ACKs for top commit:
  shaavan:
    ACK 41b9f7d062
  stratospher:
    ACK 41b9f7d.

Tree-SHA512: 25b46347c671e8d6fd8878e7fee40e773bb03641e53e41e8a79a286fe4a0cf71c0c0986d6d7418fcb656c07f7216cc50a7ee4366f9213c32b01ae74326031f80
2021-12-13 16:31:42 +01:00
MarcoFalke
fa24a3df87
rpc: Quote user supplied strings in error messages 2021-12-13 15:18:35 +01:00
MarcoFalke
bf66e258a8
Merge bitcoin/bitcoin#23365: index: Fix backwards search for bestblock
9600ea0145 test: Add edge case of pruning up to index height (Martin Zumsande)
698c524698 index: Fix backwards search for bestblock (Martin Zumsande)

Pull request description:

  This PR attempts to fix an intermittent Init issue encountered during the stress testing of #23289, which relates to the pruning-compatible filter reconstruction logic introduced in #15946.

  The problem would occur when the node starts with `-txindex=1` but `ThreadSync` is interrupted after it sets `m_best_block_index` to Genesis, and before it gets do any further work.
  In that case, during the next restart of the node, an Init error would be thrown because  `BaseIndex::Init()` tries to backtrack from the tip to the last block which has been successfully indexed (here: Genesis), but the backtracking logic didn't work properly in this case:
  The loop
  `while (block_to_test && block->pprev && (block->pprev->nStatus & BLOCK_HAVE_DATA))` checks if a predecessor exists **before** performing the check `block_to_test == block` and then possbily setting `prune_violation = false`
  If `block_to_test` and `block` are the Genesis block this check will not be reached because `block->pprev` does not exist.

  To reproduce this bug on regtest:
  1) start a node with a fresh datadir using `-txindex=1` (or any other index)
  2) stop and restart without any index
  3) mine a block
  3) stop and restart again with the index enabled
  ->InitError `Error: txindex best block of the index goes beyond pruned data. (...)`

  Fix this by requiring that we have the data for the block of the current iteration `block` (instead of requiring it for the predecessor `block->pprev`)
  That way, the check for `block_to_test == block` is also reached when `block_to_test` is the Genesis block.
  No longer requiring the data of `block->pprev` also means that we can now prune up to `m_best_block_index` height without requiring a reindex (one block more than before). I added this edge case to `feature_blockfilterindex_prune.py`, the new version should fail on master.

ACKs for top commit:
  ryanofsky:
    Partial code review ACK 9600ea0145 for the code change, not the test changes. (Test changes are indirect and little over my head.) It seems obvious that previous code `prune_violation = true, while (block->pprev)` would incorrectly detect a prune violation at the genesis block, and the fix here make sense and looks correct.

Tree-SHA512: c717f372cee8fd49718b1b18bfe237aa6ba3ff4468588c10e1272d7a2ef3981d10af4e57de51dec295e2ca72d441bc6c2812f7990011a94d7f818775e3ff1a38
2021-12-13 13:38:02 +01:00
MarcoFalke
fa663a4c0d
Use mockable time for peer connection time
This allows to revert the temporary commit
0bfb9208df (test: fix test failures in
test/functional/p2p_timeouts.py).
2021-12-13 13:32:05 +01:00
Antoine Poinsot
a9256dc340
rpc: output all hash preimages in 'decodepsbt' 2021-12-11 18:32:17 +01:00
MarcoFalke
ac92ab6da5
Merge bitcoin/bitcoin#23113: Add warnings to createmultisig and addmultisig if using uncompressed keys
d5cab1a96d Add createmultisig and addmultisigaddress warnings release note (Samuel Dobson)
e46fc935aa Add warnings field to addmultisigaddress to warn about uncompressed keys (Samuel Dobson)
d1a9742623 Add warnings field to createmultisig to warn about uncompressed keys (Samuel Dobson)

Pull request description:

  Fixes #21368

  Currently, if there are any uncompressed keys when calling `AddAndGetMultisigDestination`, it will just default to a legacy address regardless of the chosen `address_type`. Rather than keeping this silent behaviour which may be confusing to users, we explicitly add a `warnings` field which will warn the user why their address format is different.

ACKs for top commit:
  achow101:
    ACK d5cab1a96d

Tree-SHA512: c2ac7f7689251bd4fcd8c26506f053921fbaf34c7a26a74e82ebc7f82cc0bd25407fd7954bf98365dcafa51fa45dcdbee6214320580ca69509690c3555e71cc0
2021-12-11 09:41:25 +01:00
Jon Atack
0bfb9208df test: fix test failures in test/functional/p2p_timeouts.py 2021-12-11 01:20:32 +01:00
Douglas Chimento
41b9f7d062
test: Use byte unit 'M' for -maxuploadtarget functional test 2021-12-10 20:41:34 +02:00
Andrew Chow
d8043ddf64 Add global xpub test vectors from BIP 2021-12-10 08:29:47 -05:00
Andrew Chow
94065cc6c5 Test for proprietary field 2021-12-10 08:29:47 -05:00
Andrew Chow
c3eb416b88 Implement PSBT versions 2021-12-10 08:29:26 -05:00
MarcoFalke
011d6e429b
Merge bitcoin/bitcoin#22514: psbt: Actually use SIGHASH_DEFAULT for PSBT signing
c0405ee27f rpc: Document that DEFAULT is for Taproot, ALL for everything else (Andrew Chow)
d3992669df psbt: Actually use SIGHASH_DEFAULT (Andrew Chow)
eb9a1a2c59 psbt: Make sighash_type std::optional<int> (Andrew Chow)

Pull request description:

  Make the behavior align with the help text by actually using SIGHASH_DEFAULT as the default sighash for signing PSBTs.

ACKs for top commit:
  Sjors:
    re-utACK c0405ee27f

Tree-SHA512: 5199fb41de416b2f10ac451f824e7c94b428ba11fdb9e50f0027c692e959ce5813a340c34a4e52d7aa128e12008303d80939a693eff36a869720e45442119828
2021-12-10 10:17:36 +01:00
MarcoFalke
9f7661c0c4
Merge bitcoin/bitcoin#19499: p2p: Make timeout mockable and type safe, speed up test
fadc0c80ae p2p: Make timeout mockable and type safe, speed up test (MarcoFalke)
fa6d5a238d scripted-diff: Rename m_last_send and m_last_recv (MarcoFalke)

Pull request description:

  Use type-safe time for better code readability/maintainability and mockable time for better testability. This speeds up the p2p_timeout test.

  This is also a bugfix for intermittent test issues like: https://cirrus-ci.com/task/4769904156999680?command=ci#L2836

  Fixes #20654

ACKs for top commit:
  laanwj:
    Code review ACK fadc0c80ae
  naumenkogs:
    ACK fadc0c80ae

Tree-SHA512: 28c6544c97f188c8a0fbc80411c74ab74ffd055885322c325aa3d1c404b29c3fd70a737e86083eecae58ef394db1cb56bc122d06cff63742aa89a8e868730c64
2021-12-10 10:02:12 +01:00
fanquake
09ad512369
Merge bitcoin/bitcoin#23628: Check descriptors returned by external signers
5493e92501 Check descriptors returned by external signers (sstone)

Pull request description:

  Check that descriptors returned by external signers have been parsed properly when creating a new wallet.
  See https://github.com/bitcoin/bitcoin/issues/23627 for context.

  The problem is that parsing an invalid descriptor will return `null` which is not checked for in `CWallet::SetupDescriptorScriptPubKeyMans()`.

  I'm not completely sure what the best fix is since there several strategies for dealing with errors in the current codebase but the proposed fix is very simple and consistent with other validation checks in `CWallet::SetupDescriptorScriptPubKeyMans()`.

ACKs for top commit:
  jamesob:
    Code review ACK 5493e92501
  achow101:
    ACK 5493e92501

Tree-SHA512: 63259f4aa519405a86c554b6813efdb741314bdaa18bf005b70ea8bb92a27abc6e2b65f7c584641dc257fc78a6499f42b51b5310c243e611c4663430dccf3d04
2021-12-10 09:17:35 +08:00
Sebastian Falbesoner
61fb410c0d test: add feature_coinstatsindex.py --descriptors to test_runner.py 2021-12-09 16:40:52 +01:00
Sebastian Falbesoner
50b044a88e test: fix test feature_coinstatsindex.py for descriptor wallets 2021-12-09 16:40:35 +01:00
Antoine Poinsot
60ae1161a4
qa: replace assert with test framework assertion helpers in fee estimation test 2021-12-09 15:11:42 +01:00
Antoine Poinsot
e50213967b
qa: fee estimation with RBF test cleanups
Followups to #22539

Co-Authored-By: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2021-12-09 15:11:35 +01:00
Antoine Poinsot
15f5fd62af
qa: don't mine non standard txs in fee estimation test
We don't need dust outputs anymore.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-12-09 14:59:46 +01:00
Antoine Poinsot
eae52dd6ab
qa: pass scriptsig directly to txins constructor in fee estimation test
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-12-09 14:59:46 +01:00
Antoine Poinsot
1fc03155e5
qa: split coins in a single tx in fee estimation test
This simplifies the code, and slightly speeds up the test.

Running `./test/functional/test_runner.py -j15 $(printf 'feature_fee_estimation %.0s' {1..15})`
on master 3 times gives:

- Before:
    ALL                       | ✓ Passed  | 788 s (accumulated)
    ALL                       | ✓ Passed  | 818 s (accumulated)
    ALL                       | ✓ Passed  | 873 s (accumulated)

- After:
    ALL                       | ✓ Passed  | 763 s (accumulated)
    ALL                       | ✓ Passed  | 798 s (accumulated)
    ALL                       | ✓ Passed  | 731 s (accumulated)

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-12-09 14:59:44 +01:00
Antoine Poinsot
cc204b8be7
qa: use a single p2sh script in fee estimation test
Using 2 different scripts is unnecessary complication

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-12-09 14:59:16 +01:00
MarcoFalke
fa1b63c018
test: Replace hashlib.new with named constructor 2021-12-09 14:36:06 +01:00
Antoine Poinsot
19dd91a9be
qa: remove a redundant condition in fee estimation test
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-12-09 13:04:07 +01:00
sstone
5493e92501 Check descriptors returned by external signers
Check that descriptors returned by external signers have been parsed properly when creating a new wallet.
2021-12-09 11:17:04 +01:00
Sjors Provoost
bfbf91d0b2
test: fancier Python for getblockfrompeer
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2021-12-09 15:49:37 +07:00
MarcoFalke
529ed33362
Merge bitcoin/bitcoin#23715: test: feature_rbf.py: check specified wallet type availability
84bc35d7a5 test: feature_rbf.py: check specified wallet type availability (Sebastian Falbesoner)

Pull request description:

  The test currently leads to a failure if in general wallet support is compiled, but the library for the specified type (BDB/SQLite) is not, i.e.  if started with the `--legacy-wallet` parameter, but bitcoind is compiled without BDB support, see e.g. https://github.com/bitcoin/bitcoin/pull/23682#issuecomment-989044207

  Fix this by checking if the specified wallet type (BDB for legacy wallet, SQLite for descriptor wallet) is available.

  Also move the helper `is_specified_wallet_compiled()` to the
  test framework's class BitcoinTestFramework first, so it can be reused.

  Should further pave the way for #23682. On my local instance without BDB compiled, all targets in test_runner pass now.

ACKs for top commit:
  mzumsande:
    ACK 84bc35d7a5, changes loook good for me and I confirmed that this fixes the error encountered on master when running the test `--without-bdb`.

Tree-SHA512: 1575c03c793c8e0ac195d0914eff75d02604301c8fb77d0fdb7c0b245561569c0c7db387ef4de499044b68ab6e14b4b78b955f6e74c84197bcaed95f439c9824
2021-12-09 09:24:22 +01:00
MarcoFalke
68ca8677e0
Merge bitcoin/bitcoin#23716: test: replace hashlib.ripemd160 with an own implementation
5b559dc7ec Swap out hashlib.ripemd160 for own implementation (Pieter Wuille)
ad3e9e1f21 Add pure Python RIPEMD-160 (Pieter Wuille)

Pull request description:

  Closes #23710.

ACKs for top commit:
  jamesob:
    ACK 5b559dc7ec, pending CI

Tree-SHA512: dcd4ea2027eac572f7ab0da434b081b9a5d6b78675e559258a446b4d254b29d93c4d2cc12da4a28303543d6d99f5f2246fde4052e84af81d18e04399b137b39e
2021-12-09 09:18:30 +01:00
Jeremy Rubin
c49daf9885 [TESTS] Increase limitancestorcount in tournament RPC test to showcase improved algorithm 2021-12-08 23:07:56 -08:00
Pieter Wuille
5b559dc7ec Swap out hashlib.ripemd160 for own implementation 2021-12-08 14:29:41 -05:00
Pieter Wuille
ad3e9e1f21 Add pure Python RIPEMD-160 2021-12-08 14:29:34 -05:00
Sebastian Falbesoner
84bc35d7a5 test: feature_rbf.py: check specified wallet type availability
The test currently leads to a failure if in general wallet
support is compiled, but the library for the specified type
(BDB/SQLite) is not, i.e.  if started with the
`--legacy-wallet` parameter, but bitcoind is compiled
without BDB support.

Fix this by checking if the specified wallet type (BDB for
legacy wallet, SQLite for descriptor wallet) is available.

Also move the helper `is_specified_wallet_compiled()` to the
test framework's class BitcoinTestFramework first, so it can
be reused.
2021-12-08 19:57:43 +01:00
MarcoFalke
faa0833c43
doc: Normalize RPC description whitespace 2021-12-08 19:43:24 +01:00
Sebastian Falbesoner
b57bf25cfe test: interface_bitcoin_cli.py: check specified wallet type availability
Currently the test performs the wallet-relevant parts if
_any_ wallet type support is compiled in, independently of
whether the test is run with legacy or descriptor wallet
specified. This leads to a failure if the test is started
with the `--legacy-wallet` parameter, but bitcoind is
compiled without BDB support.

Fix this by checking if the specified wallet type (BDB for
legacy wallet, SQLite for descriptor wallet) is available.
2021-12-08 17:37:25 +01:00
Andrew Chow
d3992669df psbt: Actually use SIGHASH_DEFAULT
Make the behavior align with the help text by actually using
SIGHASH_DEFAULT as the default sighash for signing PSBTs.
2021-12-08 09:43:30 -05:00
MarcoFalke
f6013265b7
Merge bitcoin/bitcoin#20295: rpc: getblockfrompeer
dce8c4c381 rpc: getblockfrompeer (Sjors Provoost)
b884ababc2 rpc: move Ensure* helpers to server_util.h (Sjors Provoost)

Pull request description:

  This adds an RPC method to fetch a block directly from a peer. This can used to fetch stale blocks with lower proof of work that are normally ignored by the node (`headers-only` in `getchaintips`).

  Usage:
  ```
  bitcoin-cli getblockfrompeer HASH peer_n
  ```

  Closes #20155

  Limitations:
  * you have to specify which peer to fetch the block from
  * the node must already have the header

ACKs for top commit:
  jnewbery:
    ACK dce8c4c381
  fjahr:
     re-ACK dce8c4c381

Tree-SHA512: 843ba2b7a308f640770d624d0aa3265fdc5c6ea48e8db32269b96a082b7420f7953d1d8d1ef2e6529392c7172dded9d15639fbc9c24e7bfa5cfb79e13a5498c8
2021-12-08 10:39:37 +01:00
Samuel Dobson
e46fc935aa Add warnings field to addmultisigaddress to warn about uncompressed keys 2021-12-08 17:14:40 +13:00
Samuel Dobson
d1a9742623 Add warnings field to createmultisig to warn about uncompressed keys 2021-12-08 17:11:46 +13:00
MarcoFalke
63c63b5533
Merge bitcoin/bitcoin#14707: [RPC] Include coinbase transactions in receivedby RPCs
1dcba996d3 Coinbase receivedby rpcs release notes (Andrew Toth)
b5696750a9 Test including coinbase transactions in receivedby wallet rpcs (Andrew Toth)
bce20c34d6 Include coinbase transactions in receivedby wallet rpcs (Andrew Toth)

Pull request description:

  The current `*receivedby*` RPCs filter out coinbase transactions. This doesn't seem correct since an output to your address in a coinbase transaction *is* receiving those coins.

  This PR corrects this behaviour. Also, a new option `include_immature_coinbase` is added (default=`false`) that includes immature coinbase transactions when set to true.

  However, since this is potentially a breaking change this PR introduces a hidden configuration option `-deprecatedrpc=exclude_coinbase`. This can be set to revert to previous behaviour. If no reports of broken workflow are received, then this option can be removed in a future release.

  Fixes https://github.com/bitcoin/bitcoin/issues/14654.

ACKs for top commit:
  jnewbery:
    reACK 1dcba996d3

Tree-SHA512: bfc43b81279fea5b6770a4620b196f6bc7c818d221b228623e9f535ec75a2406bc440e3df911608a3680f11ab64c5a4103917162114f5ff7c4ca8ab07bb9d3df
2021-12-07 20:52:13 +01:00
Andrew Toth
b5696750a9 Test including coinbase transactions in receivedby wallet rpcs 2021-12-07 10:48:37 -05:00
MarcoFalke
4fd0ce75c5
Merge bitcoin/bitcoin#22689: rpc: deprecate top-level fee fields in getmempool RPCs
2f9515f37a rpc: move fees object to match help (josibake)
07ade7db8f doc: add release note for fee field deprecation (josibake)
2ee406ce3e test: add functional test for deprecatedrpc=fees (josibake)
35d928c632 rpc: deprecate fee fields from mempool entries (josibake)

Pull request description:

  per #22682 , top level fee fields for mempool entries have been deprecated since 0.17 but are still returned. this PR properly deprecates them so that they are no longer returned unless `-deprecatedrpc=fees` is passed.

  the first commit takes care of deprecation and also updates `test/functional/mempool_packages.py` to only use the `fees` object. the second commit adds a new functional test for `-deprecatedrpc=fees`

  closes #22682

  ## questions for the reviewer

  * `-deprecatedrpc=fees` made the most sense to me, but happy to change if there is a name that makes more sense
  * #22682 seems to indicate that after some period of time, the fields will be removed all together. if we have a rough idea of when this will be, i can add a `TODO: fully remove in vXX` comment to `entryToJSON`

  ## testing
  to get started on testing, compile, run the tests, and start your node with the deprecated rpcs flag:

  ```bash
  ./src/bitcoind -daemon -deprecatedrpc=fees
  ```
  you should see entries with the deprecated fields like so:
  ```json
  {
    "<txid>": {
      "fees": {
        "base": 0.00000671,
        "modified": 0.00000671,
        "ancestor": 0.00000671,
        "descendant": 0.00000671
      },
      "fee": 0.00000671,
      "modifiedfee": 0.00000671,
      "descendantfees": 671,
      "ancestorfees": 671,
      "vsize": 144,
      "weight": 573,
     ...
    },
  ```
  you can also check `getmempoolentry` using any of the txid's from the output above.

  next start the node without the deprecated flag, repeat the commands from above and verify that the deprecated fields are no longer present at the top level, but present in the "fees" object

ACKs for top commit:
  jnewbery:
    reACK 2f9515f37a
  glozow:
    utACK 2f9515f37a

Tree-SHA512: b175f4d39d26d96dc5bae26717d3ccfa5842d98ab402065880bfdcf4921b14ca692a8919fe4e9969acbb5c4d6e6d07dd6462a7e0a0a7342556279b381e1a004e
2021-12-07 15:26:06 +01:00
MarcoFalke
89ea2b3809
Merge bitcoin/bitcoin#20583: rpc: Add missing BlockUntilSyncedToCurrentChain to wallet RPCs
fa5362a9a0 rpc: Add missing BlockUntilSyncedToCurrentChain to wallet RPCs (MarcoFalke)

Pull request description:

  Wallet RPCs that allow a rescan based on block-timestamp or block-height
  need to sync with the active chain first, because the user might assume
  the wallet is up-to-date with the latest block they got reported via a
  blockchain RPC.

ACKs for top commit:
  meshcollider:
    utACK fa5362a9a0

Tree-SHA512: d4831f1f08f854f9a49fc969de86c438f856e41c2163c801a6ff36dc2f6299cb342b44663279c524a8b7ca9a50895db1243cd7d49bed79277ada857213f20a26
2021-12-07 09:24:34 +01:00
MarcoFalke
08dcc5912d
Merge bitcoin/bitcoin#23688: test: remove unneeded sync_all() calls in wallet_listtransactions.py
0ba98eda28 test: remove unneeded sync_all() calls in wallet_listtransactions.py (Sebastian Falbesoner)

Pull request description:

  This is a small follow-up to #23659. The `self.sync_all()` calls after generating blocks can be removed, since that happens automatically per default by the test framework's generate function (if no explicit sync_fun is passed).
  On the course of touching the file, imports are sorted and the grammar of a log message is fixed.

ACKs for top commit:
  fanquake:
    ACK 0ba98eda28 - thanks for following up.
  shaavan:
    ACK 0ba98eda28

Tree-SHA512: 451e733865dcb1e424d90289c8c89272837a9af6fd4b77d6c60728c84524d9c792d684b7e601b02a0efda67231183c42dd9040d96214ac7d9473b2808cabe73f
2021-12-07 08:57:46 +01:00
Sebastian Falbesoner
0ba98eda28 test: remove unneeded sync_all() calls in wallet_listtransactions.py 2021-12-06 23:36:31 +01:00
Sebastian Falbesoner
035767f54a test: add interface_bitcoin_cli.py --descriptors to test_runner.py 2021-12-06 20:13:10 +01:00
Sebastian Falbesoner
e4fa28a322 test: fix test interface_bitcoin_cli.py for descriptor wallets 2021-12-06 20:13:06 +01:00
W. J. van der Laan
695ba2fe54
Merge bitcoin/bitcoin#23486: rpc: Only allow specific types to be P2(W)SH wrapped in decodescript
99993425af rpc: Only allow specific types to be P2(W)SH wrapped in decodescript (MarcoFalke)

Pull request description:

  It seems confusing to return a P2SH wrapping address that is eventually either policy- or consensus-unspendable.

ACKs for top commit:
  laanwj:
    Code review re-ACK 99993425af

Tree-SHA512: 3cd530442acee7c295d244995f0f17b2cae7212f1e0970bb5807621f8ff8e4308a3236b385d77087cd493d32ee524813d8edd15e91d937ef9a800094b7bc4946
2021-12-06 19:26:04 +01:00
MarcoFalke
32d9f3770a
Merge bitcoin/bitcoin#23596: test: fix wallet_transactiontime_rescan.py --descriptors and add to test runner
e4a54af6b8  test: add wallet_transactiontime_rescan.py --descriptors to test_runner.py (Sebastian Falbesoner)
b60e02e993 test: fix test wallet_transactiontime_rescan.py for descriptor wallets (Sebastian Falbesoner)
a905ed1a61 test: refactor: use `set_node_times` helper in wallet_transactiontime_rescan.py (Sebastian Falbesoner)

Pull request description:

  The functional test wallet_transactiontime_rescan.py currently fails on master branch, if descriptor wallets are used (argument `--descriptors`). This is due to the fact that in this case, the test framework maps the importaddress RPC calls to the importdescriptors RPC (rescan=False -> timestamp='now'), which always rescans blocks of the past 2 hours, based on the current MTP timestamp. In order to avoid importing the last address (wo3), we generate 10 more blocks with advanced time, to ensure that the balance after importing is zero:
  681b25e3cd/test/functional/wallet_transactiontime_rescan.py (L125-L134)

  Calling this test with descriptor wallets is also added to test runner. Fixes #23562.

ACKs for top commit:
  Sjors:
    tACK e4a54af
  brunoerg:
    tACK e4a54af6b8

Tree-SHA512: 9fd8e298d48dd7947b1218d61a1a66c1241b3dbb14451b0ec7cd30caa74ee540e7ee5a7bd10d421b9e3b6e549fa5c3e85bd02496436128b433b328118642f600
2021-12-06 12:28:25 +01:00
W. J. van der Laan
22feb7fee9
Merge bitcoin/bitcoin#23577: Follow-ups to Bech32 error detection
a4fe70171b Make Bech32 LocateErrors return error list rather than using out-arg (Samuel Dobson)
2fa4fd1961 Use std::iota instead of manually pushing range (Samuel Dobson)
405c96fc9f Use bounds-checked array lookups in Bech32 error detection code (Samuel Dobson)
28d9c2857f Simplify encoding of e in GF(1024) tables to (1,0) (Samuel Dobson)
14358a029d Replace GF1024 tables and syndrome constants with compile-time generated constexprs. (Samuel Dobson)
63f7b69779 Update release note for bech32 error detection (Samuel Dobson)
c8b9a224e7 Report encoding type in bech32 error message (Samuel Dobson)
92f0cafdca Improve Bech32 boost tests (Samuel Dobson)
bb4d3e9b97 Address review comments for Bech32 error validation (Samuel Dobson)

Pull request description:

  A number of follow-ups and improvements to the bech32 error location code, introduced in #16807.

  Notably, this removes the hardcoded GF1024 tables in favour of constexpr table generation.

ACKs for top commit:
  laanwj:
    Re-ACK a4fe70171b

Tree-SHA512: 6312373c20ebd6636f5797304876fa0d70fa777de2f6c507245f51a652b3d1224ebc55b236c9e11e6956c1e88e65faadab51d53587078efccb451455aa2e2276
2021-12-06 12:18:17 +01:00
MarcoFalke
fadc0c80ae
p2p: Make timeout mockable and type safe, speed up test 2021-12-06 10:47:52 +01:00
MarcoFalke
99993425af
rpc: Only allow specific types to be P2(W)SH wrapped in decodescript 2021-12-06 10:33:12 +01:00
MarcoFalke
d20d6ac545
Merge bitcoin/bitcoin#23672: tests: Add data-driven testcases to rpc_decodescript.py
b35942e500 tests: Add data-driven testcases to rpc_decodescript.py (Dimitri)

Pull request description:

  closes #23641

Top commit has no ACKs.

Tree-SHA512: 2f494c78ad085d523fae15befaadb9e0fc382b5310e3a95395ecf06a90968b15b6d232f7564098ed0a68419e27aa2e5260fe691cf2ce84af9fb6b65634e54d77
2021-12-06 10:09:30 +01:00
Fabian Jahr
c055f6b216
test: Remove false coinstatsindex test 2021-12-05 22:42:29 +01:00
Dimitri
b35942e500 tests: Add data-driven testcases to rpc_decodescript.py 2021-12-05 22:07:04 +01:00
MarcoFalke
fa5362a9a0
rpc: Add missing BlockUntilSyncedToCurrentChain to wallet RPCs
Wallet RPCs that allow a rescan based on block-timestamp or block-height
need to sync with the active chain first, because the user might assume
the wallet is up-to-date with the latest block they got reported via a
blockchain RPC.
2021-12-03 11:13:00 +01:00
James O'Beirne
4e1cb904ba
test: fix: remove outdated TestNode.generate calls
Currently failing on CI. After this change the test itself still fails,
but at least it's apparently for a non-incidental reason.
2021-12-02 17:07:08 -05:00
W. J. van der Laan
bce58bbb3d
Merge bitcoin/bitcoin#22929: wallet: Automatically add receiving destinations to the address book
3d71d16d1e test: listtranscations with externally generated addresses (S3RK)
d04566415e Add to spends only transcations from me (S3RK)
9f3a622b1c Automatically add labels to detected receiving addresses (S3RK)
c1b99c088c Return used destinations from ScriptPubKeyMan::MarkUnusedAddresses (S3RK)
03840c2064 Add CWallet::IsInternalScriptPubKeyMan (S3RK)
456e350926 wallet: resolve ambiguity of two ScriptPubKey managers providing same script (S3RK)

Pull request description:

  This PR fixes certain use-cases when **send-to-self** transactions are missing from `listtransactions` output.

  1. When a receiving address is generated externally to the wallet
  (e.g. same wallet running on two nodes, or by 3rd party from xpub)
  2. When restoring backup with lost metadata, but keypool gap is not exceeded yet

  When the block is connected or tx added to mempool we already mark used keys. This PR extends this logic to determine whether the destination is a receiving one and if yes add it to the address book with empty label.

  Works both for legacy and descriptors wallets.
  - For legacy it uses the internal flag from the keypool entry. Caveat: because we don't know which script type would be used we add all possible destinations for such keys.
  - For descriptor wallets it uses internal flag for the script pub key manager. Caveat: it only works for active descriptors.

  fixes #19856
  fixes #20293

ACKs for top commit:
  laanwj:
    Code review ACK 3d71d16d1e

Tree-SHA512: 03fafd5548ead0c4ffe9ebcc9eb2849f1d2fa7270fda4166419b86877d4e57dcf04460e465fbb9c90b42031f3c05d1b83f1b67a9f82c2a42980825ed1e7b52e6
2021-12-02 19:37:33 +01:00
MarcoFalke
26a1147ce5
Merge bitcoin/bitcoin#23636: Remove GetAdjustedTime from init.cpp
fa551b3bdd Remove GetAdjustedTime from init.cpp (MarcoFalke)
fa815f8473 Replace addrman.h include with forward decl in net.h (MarcoFalke)

Pull request description:

  It seems confusing to call `GetAdjustedTime` there, because no offset could have been retrieved from the network at this point. Even if connman was started, `timedata` needs at least 5 peer connections to calculate an offset.

  Fix the confusion by replacing `GetAdjustedTime` with `GetTime`, which does not change behavior.

  Also:
  * Replace magic number with `MAX_FUTURE_BLOCK_TIME` to clarify the context
  * Add test, which passes both on current master and this pull request
  * An unrelated refactoring commit, happy to drop

ACKs for top commit:
  dongcarl:
    Code Review ACK fa551b3bdd, noticed the exact same thing here: e073634c37
  mzumsande:
    Code Review ACK fa551b3bdd
  jnewbery:
    Code review ACK fa551b3bdd
  shaavan:
    ACK fa551b3bdd
  theStack:
    Code-review ACK fa551b3bdd

Tree-SHA512: 15807a0e943e3e8d8c5250c8f6d7b56afb26002b1e290bf93636a2c747f27e78f01f1de04ce1a83d6339e27284c69c43e077a8467545c4078746f4c1ecb1164d
2021-12-02 15:24:55 +01:00
W. J. van der Laan
6acda4b00b
Merge bitcoin/bitcoin#23155: rpc: various fixups for dumptxoutset
ffd09281fe rpc: various fixups for dumptxoutset (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11) (parent PR: #15606)

  ---

  A few fixes to make this RPC actually useful when generating snapshots.

  - Generate an assumeutxo hash and display it (sort of a bugfix)
  - Add nchaintx to output (necessary for use in chainparams entry)
  - Add path of serialized UTXO file to output

ACKs for top commit:
  laanwj:
    Code review ACK ffd09281fe

Tree-SHA512: b0b5fd5138dea0e21258b1b18ab75bf3fd1628522cc1dbafa81af9cb9fa96562a1c39124fdb31057f256bfc560f462f907e9fe5e209b577b3f57afae2b7be826
2021-12-02 10:35:02 +01:00
Sjors Provoost
dce8c4c381
rpc: getblockfrompeer
Co-authored-by: John Newbery <john@johnnewbery.com>
2021-12-02 13:16:18 +07:00
fanquake
9a09d307e9
rpc: remove info about mallocinfo needing glibc 2.10+
We require glibc 2.18+.
2021-12-01 19:18:26 +08:00
W. J. van der Laan
aef8c7cf82
Merge bitcoin/bitcoin#23289: test: add stress tests for initialization
d9803f7a0a test: add stress tests for initialization (James O'Beirne)
23f85616a8 test: add node.chain_path and node.debug_log_path (James O'Beirne)

Pull request description:

  In the course of coming up with a test plan for #23280, I thought it would be neat to include a Python snippet showing how I tested the initialization process. I quickly realized I was reinventing the functional test framework... so here's a new test.

  This change bangs init around like the Fonz hitting a jukebox. It adds some interesting (read: lazy and random) coverage for the initialization process by
  - interrupting init with SIGTERM after certain log statements,
  - interrupting init at random points, and
  - starting init with some essential data missing (block files, block indices, etc.) to test init error paths.

  As far as I can tell, some of these code paths are uncovered otherwise (namely the startup errors).

  ---

  Incidentally, I think I may have uncovered some kind of a bug or race condition with indexing initialization based on an intermittent failure in this testcase. This test sometimes fails after shutting down immediately after `loadblk` thread start:
  ```
  2021-10-15T21:14:51.295000Z TestFramework (INFO): Starting node and will exit after line 'loadblk thread start'
    36   │ 2021-10-15T21:14:51.296000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up
    37   │ 2021-10-15T21:14:51.493000Z TestFramework (INFO): terminating node after 110 log lines seen
    38   │ 2021-10-15T21:14:51.625000Z TestFramework (INFO): Starting node and will exit after line 'txindex thread start'
    39   │ 2021-10-15T21:14:51.625000Z TestFramework.node0 (DEBUG): bitcoind started, waiting for RPC to come up
    ------> [[ FAILURE HERE ]] 2021-10-15T21:15:21.626000Z TestFramework (WARNING): missed line {bail_line}; bailing now after {num_lines} lines
  ```
  and then fails to start up afterwards. Combined logs showing `Error: txindex best block of the index goes beyond pruned data`, when the node under test is not pruned:

  ```
    node0 2021-10-15T21:16:51.848439Z [shutoff] [validationinterface.cpp:244] [ChainStateFlushed] Enqueuing ChainStateFlushed: block hash=1014bc4ff4917602ae53d10e9dfe230af4b7d52a6cdaa8a47798b9c288180907
     node0 2021-10-15T21:16:51.848954Z [shutoff] [init.cpp:302] [Shutdown] Shutdown: done
     test  2021-10-15T21:16:51.882000Z TestFramework (ERROR): Unexpected exception caught during testing
       Traceback (most recent call last):
         File "/home/james/src/bitcoin/test/functional/test_framework/test_framework.py", line 132, in main
   self.run_test()
         File "/home/james/src/bitcoin/./test/functional/stress_init.py", line 87, in run_test
   check_clean_start()
         File "/home/james/src/bitcoin/./test/functional/stress_init.py", line 60, in check_clean_start
   node.wait_for_rpc_connection()
         File "/home/james/src/bitcoin/test/functional/test_framework/test_node.py", line 224, in wait_for_rpc_connection
   raise FailedToStartError(self._node_msg(
       test_framework.test_node.FailedToStartError: [node 0] bitcoind exited with status 1 during initialization
     test  2021-10-15T21:16:51.882000Z TestFramework (DEBUG): Closing down network thread
     test  2021-10-15T21:16:51.933000Z TestFramework (INFO): Stopping nodes
     test  2021-10-15T21:16:51.933000Z TestFramework.node0 (DEBUG): Stopping node

     node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
     node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
     node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
     node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
     node0 stderr Error: txindex best block of the index goes beyond pruned data. Please disable the index or reindex (which will download the whole blockchain again)
  ```

ACKs for top commit:
  laanwj:
    Code review ACK d9803f7a0a

Tree-SHA512: 4d80dc399daf199a6222e81e47d12d830dc7af07355eddbb7f52479a676a645b8d3d45093ff54a9295f01a163b2f4fe0e038e83fc269969e03d4cfda69eaf111
2021-11-30 20:50:11 +01:00
W. J. van der Laan
63c0d0e937
Merge bitcoin/bitcoin#21327: net_processing: ignore transactions while in IBD
6aed8b7e9b [test] tx processing before and after ibd (glozow)
b9e105b664 [net_processing] ignore all transactions during ibd (glozow)

Pull request description:

  This is basically a mini, IBD-only version of #21224

  Incoming transactions aren't really relevant until we're caught up. That's why we send a giant feefilter and don't send tx getdatas, but we also shouldn't process them if peers send them anyway. Simply ignore them.

ACKs for top commit:
  jnewbery:
    reACK 6aed8b7e9b
  laanwj:
    Code review ACK 6aed8b7e9b

Tree-SHA512: 8e1616bf355f9d0b180bdbc5461f24c757dc5d7bc7bf651470f3b0bffcca5d5e68287106255b5cede2d96b42bce448a0f8c0649de35a530c5e079f7c89c70a35
2021-11-30 19:09:15 +01:00
MarcoFalke
fa551b3bdd
Remove GetAdjustedTime from init.cpp 2021-11-30 17:19:49 +01:00
James O'Beirne
ffd09281fe
rpc: various fixups for dumptxoutset
- Actually generate an assumeutxo hash and display it
- Add nchaintx to output (necessary for use in chainparams entry)
- Add path of serialized UTXO file to output
2021-11-30 11:19:26 -05:00
W. J. van der Laan
383d350bd5
Merge bitcoin/bitcoin#22513: rpc: Allow walletprocesspsbt to sign without finalizing
a99ed89865 psbt: sign without finalizing (Andrew Chow)

Pull request description:

  It can be useful to sign an input with `walletprocesspsbt` but not finalize that input if it is complete. This PR adds another option to `walletprocesspsbt` to be able to do that. We will still finalize by default.

  This does not materially change the PSBT workflow since `finalizepsbt` needs to be called in order to extract the tx for broadcast.

ACKs for top commit:
  meshcollider:
    utACK a99ed89865
  Sjors:
    utACK a99ed89

Tree-SHA512: c88e5d3222109c5f4e763b1b9d97ce4655f68f2985a4509caab2d4e7f5bac5047328fd69696e82a330f5c5a333e0312568ae293515689b77a4747ca2f17caca6
2021-11-29 17:20:20 +01:00
W. J. van der Laan
a574f4ad39
Merge bitcoin/bitcoin#23392: test: move check_node_connections to util
988024fe37 test: add check_node_connections in util (brunoerg)

Pull request description:

  This function (`check_node_connections`) exists in `feature_anchors.py` and `p2p_add_connections.py` and does the same thing in both, this PR moves this function to util to avoid repetition and facilitate maintenance.

ACKs for top commit:
  laanwj:
    Code review ACK 988024fe37

Tree-SHA512: bf86c5659933539c72cb91ad587552b45c918be74d36fb429e78f3b954f01ed0855a85dd49aea35b432fbd18227c05eb3fec8b99c139c3509c39b19bccf6b7fd
2021-11-29 13:51:27 +01:00
Sebastian Falbesoner
b79dbe86a9 test: add feature_rbf.py --descriptors to test_runner.py 2021-11-26 17:16:08 +01:00
Sebastian Falbesoner
166f8ec28e test: always rescan after importing private keys in init_wallet helper
This fixes the functional test feature_rbf.py for descriptor wallets.
2021-11-26 17:10:55 +01:00
Martin Zumsande
9600ea0145 test: Add edge case of pruning up to index height 2021-11-26 16:27:32 +01:00
Sebastian Falbesoner
e4a54af6b8 test: add wallet_transactiontime_rescan.py --descriptors to test_runner.py 2021-11-25 17:34:34 +01:00
Sebastian Falbesoner
b60e02e993 test: fix test wallet_transactiontime_rescan.py for descriptor wallets 2021-11-25 17:31:22 +01:00
Sebastian Falbesoner
a905ed1a61 test: refactor: use set_node_times helper in wallet_transactiontime_rescan.py 2021-11-25 17:11:32 +01:00
MarcoFalke
064c729a96
Merge bitcoin/bitcoin#23512: policy: Treat taproot as always active
fa3e0da06b policy: Treat taproot as always active (MarcoFalke)

Pull request description:

  Now that taproot is active, it can be treated as if it was always active for policy for the next major release. This simplifies the code and changes two things:

  * Importing `tr` descriptors can be done before the chain is fully synced. This is fine, because the wallet will already generate `tr` descriptors by default (regardless of the taproot status) after commit 47fe7445e7.
  * Valid taproot spends won't be rejected from the mempool before taproot is active. This is strictly speaking a bugfix after commit 47fe7445e7, since the wallet may generate taproot spends before the chain is fully synced. For example, a slow node or a purposefully offline node. Currently, the wallet needs the mempool to account for change. See https://github.com/bitcoin/bitcoin/issues/11887.

  A similar change was done for segwit v0 in https://github.com/bitcoin/bitcoin/pull/13120 .

  This effectively reverts commit c5ec0367d7.

ACKs for top commit:
  mjdietzx:
    Code Review ACK fa3e0da06b
  achow101:
    ACK fa3e0da06b
  sipa:
    utACK fa3e0da06b
  gruve-p:
    ACK fa3e0da06b
  gunar:
    Code Review + tACK fa3e0da06
  rajarshimaitra:
    code review + tACK fa3e0da06b

Tree-SHA512: c6dc7a4e6c345bdec33f256847dc63906ab1696aa683ab9b32a79e715613950884ac3a1a7a44e95f31bb28e58dd64679a616175f7e152b21f5550f3337c8e622
2021-11-25 08:16:19 +01:00
Vasil Dimov
e53a8505db
net: respect -onlynet= when making outbound connections
Do not make outbound connections to hosts which belong to a network
which is restricted by `-onlynet`.

This applies to hosts that are automatically chosen to connect to and to
anchors.

This does not apply to hosts given to `-connect`, `-addnode`,
`addnode` RPC, dns seeds, `-seednodes`.

Fixes https://github.com/bitcoin/bitcoin/issues/13378
Fixes https://github.com/bitcoin/bitcoin/issues/22647
Supersedes https://github.com/bitcoin/bitcoin/pull/22651
2021-11-24 12:44:05 +01:00
Samuel Dobson
c8b9a224e7 Report encoding type in bech32 error message 2021-11-23 15:48:59 +13:00
Samuel Dobson
bb4d3e9b97 Address review comments for Bech32 error validation 2021-11-23 15:48:59 +13:00
W. J. van der Laan
95d19f8c1a
Merge bitcoin/bitcoin#16807: Let validateaddress locate error in Bech32 address
88cc481092 Modify copyright header on Bech32 code (Samuel Dobson)
5599813b80 Add lots of comments to Bech32 (Samuel Dobson)
2eb5792ec7 Add release notes for validateaddress Bech32 error detection (MeshCollider)
42d6a029e5 Refactor and add more tests for validateaddress (Samuel Dobson)
c4979f77c1 Add boost tests for bech32 error detection (MeshCollider)
02a7bdee42 Add error_locations to validateaddress RPC (Samuel Dobson)
b62b67e06c Add Bech32 error location function (Samuel Dobson)
0b06e720c0 More detailed error checking for base58 addresses (Samuel Dobson)

Pull request description:

  Addresses (partially) #16779 - no GUI change in this PR

  Adds a LocateError function the bech32 library, which is then called by `validateaddress` RPC, (and then eventually from a GUI tool too, future work). I think modifying validateaddress is nicer than adding a separate RPC for this.
  Includes tests.

  Based on https://github.com/sipa/bech32/blob/master/ecc/javascript/bech32_ecc.js
  Credit to sipa for that code

ACKs for top commit:
  laanwj:
    Code review and manually tested ACK 88cc481092
  ryanofsky:
    Code review ACK 88cc481092 with caveat that I only checked the new `LocateErrors` code to try to verify it didn't have unsafe or unexpected operations or loop forever or crash. Did not try to verify behavior corresponds to the spec. In the worst case bugs here should just affect error messages not actual decoding of addresses so this seemed ok.
  w0xlt:
    tACK 88cc481

Tree-SHA512: 9c7fe9745bc7527f80a30bd4c1e3034e16b96a02cc7f6c268f91bfad08a6965a8064fe44230aa3f87e4fa3c938f662ff4446bc682c83cb48c1a3f95cf4186688
2021-11-22 13:26:01 +01:00
MarcoFalke
3a36ec83d0
Merge bitcoin/bitcoin#23521: test: refactor: dedup code by taking use of create_block parameters
e57c0eb865 test: refactor: replace OP_1/OP_TRUE magic numbers by constants (Sebastian Falbesoner)
df5d783aef test: refactor: take use of `create_block` txlist parameter (Sebastian Falbesoner)
ae9df4ef93 test: refactor: take use of `create_block` version parameter (or use default) (Sebastian Falbesoner)

Pull request description:

  The helper `create_block` offers two parameters `version` and `txlist` which set the `nVersion` field / extend the `vtx` array of the block, respectively. By taking use of those, we can remove a lot of code, including the recalculation of the merkle root. Both passing txs in string and `CTransaction` format is supported, i.e. we also save potential calls to `tx_from_hex`.
  The PR also contains another commit which replaces magic numbers for OP_TRUE/OP_1 (0x51) with the proper constant from the `script` module.

  Instances setting the block version of 4 explicitely after calling `create_block` are removed, as this is the default since #16333 got merged (see https://github.com/bitcoin/bitcoin/pull/23521#discussion_r751173671).

ACKs for top commit:
  stratospher:
    tested ACK e57c0eb.

Tree-SHA512: a56965168d36b40b84e7f334b00472b82c31e8482c9e2651c97a791abd7fee3b40ca15e943a7acafa3acf172066fdace38bb13240084b789fd6ff4f6e510e23a
2021-11-22 10:55:30 +01:00
MarcoFalke
47fe7445e7
Merge bitcoin/bitcoin#22364: wallet: Make a tr() descriptor by default
4868c9f1b3 Extract Taproot internal keyid with GetKeyFromDestination (Andrew Chow)
d8abbe119c Mention bech32m in -addresstype and -changetype help (Andrew Chow)
8fb57845ee Create a tr() descriptor bech32m DescriptorScriptPubKeyMan by default (Andrew Chow)
54b3699862 Store pubkeys in TRDescriptor::MakeScripts (Andrew Chow)

Pull request description:

  Make a `tr()` descriptor by default in descriptor wallets so that users will be able to make and use segwit v1 bech32m addresses.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 4868c9f1b3
  Sjors:
    re-utACK 4868c9f1b3
  gruve-p:
    ACK 4868c9f1b3
  meshcollider:
    Concept + code review ACK 4868c9f1b3

Tree-SHA512: e5896e665b8d559f1d759b6582d1bb24f70d4698a57307684339d9fdcdac28ae9bc17bc946a7efec9cb35c130a95ffc36e3961a335124ec4535d77b8d00e9631
2021-11-22 10:01:17 +01:00
MarcoFalke
368831371d
Merge bitcoin/bitcoin#23558: test: run rpc-generateblock.py even with wallet disabled
1e86ff7941 test: run rpc-generateblock.py even with wallet disabled (Darius Parvin)

Pull request description:

  Run rpc_generateblock.py test even when the wallet is disabled, as discussed in #20078.

  This PR picked up from where #20808 left off, started by @ nginocchio. Since then, there have been many improvements to `MiniWallet`, making this PR more straightforward.

  L23 makes use of `MiniWallet.rescan_utxos()` to add the pre-mined block utxos (when `self.setup_clean_chain` is not set to `True`), rather than generating new blocks during the test.

ACKs for top commit:
  mjdietzx:
    Tested ACK 1e86ff7941

Tree-SHA512: 4285f61516dd53a08004eeea26d58f45b4c1c67f5ca4c94ff1bc9fc7e50f486de2e033a8b4aaf67cb4c33d73aa929362e18dc75d5c7951cbf58120b5fb1de555
2021-11-21 08:28:09 +01:00
Darius Parvin
1e86ff7941 test: run rpc-generateblock.py even with wallet disabled 2021-11-20 00:57:35 -08:00
brunoerg
988024fe37 test: add check_node_connections in util 2021-11-18 13:40:43 -03:00
Sebastian Falbesoner
83f6c0f9ef test: add decodescript RPC test for P2TR output type 2021-11-18 01:07:26 +01:00
Sebastian Falbesoner
099c6957de test: check for decodescript RPC 'type' results 2021-11-18 00:11:10 +01:00
Sebastian Falbesoner
0d43525c61 test: add logging to rpc_decodescript.py
Also remove the enumerations ("1)", "2)"...) from the test
cases as those potentially hinder maintainability; e.g. if a
new case in inserted in-between, all the remaining
enumerations would need to be adapted.
2021-11-17 23:26:28 +01:00
Sebastian Falbesoner
e57c0eb865 test: refactor: replace OP_1/OP_TRUE magic numbers by constants 2021-11-17 15:45:34 +01:00
Sebastian Falbesoner
df5d783aef test: refactor: take use of create_block txlist parameter
Passing a list of transactions `txlist` to `create_block` appends
them to the block, hence we don't need to do that manually anymore.
The merkle root calculation can also be removed, since this is done
in the end of the helper.
2021-11-17 15:45:30 +01:00
Sebastian Falbesoner
ae9df4ef93 test: refactor: take use of create_block version parameter (or use default) 2021-11-17 15:44:09 +01:00
W. J. van der Laan
d94dc69ee4
Merge bitcoin/bitcoin#23501: test: various feature_nulldummy.py improvements
f1ed30451f test: refactor: simplify `block_submit` in feature_nulldummy.py (Sebastian Falbesoner)
5ba9f1ff59 test: refactor: rename NULLDUMMY-invalidation helper (Sebastian Falbesoner)
e1d4a128e8 test: simplify and document NULLDUMMY-invalidation helper (Sebastian Falbesoner)

Pull request description:

  This PR improves the functional test `feature_nulldummy.py` by simplifying the helpers `trueDummy` (renamed to `invalidate_nulldummy_tx`) and `block_submit`. Details can be found in the commit messages.

ACKs for top commit:
  laanwj:
    Code review ACK f1ed30451f

Tree-SHA512: ad227b31936f53c5dbded823643bced296d86f40b90f2c144a9857db3d00544f9ad5bbce4c7e84b6ece25e78e95c19aafb1d8fb31e610dcd5cbf3da63190de85
2021-11-17 10:46:00 +01:00
Andrew Chow
8fb57845ee Create a tr() descriptor bech32m DescriptorScriptPubKeyMan by default 2021-11-16 12:20:13 -05:00
MarcoFalke
fa3e0da06b
policy: Treat taproot as always active 2021-11-16 08:20:33 +01:00
fanquake
ffdab41f94
Merge bitcoin/bitcoin#23474: test: scripted-diff cleanups after generate* changes
fac23c2114 scripted-diff: Bump copyright headers (MarcoFalke)
fa974f1f14 scripted-diff: Remove redundant sync_all and sync_blocks (MarcoFalke)
fad13991ae test: Properly set sync_fun in NodeNetworkLimitedTest (MarcoFalke)
faeff57709 test: Use 4 spaces for indentation (MarcoFalke)

Pull request description:

  Some cleanups after commit 94db963de5

ACKs for top commit:
  fanquake:
    ACK fac23c2114

Tree-SHA512: 5acfd5bb9679b41969d0fc6fc85801ccadcd6530ea692bac6352668e06fc7a9b0e1db3fd6fba435e84afe983d2eb07bd0a47c8364462bb7110004bd3d102b698
2021-11-16 11:22:06 +08:00
fanquake
41e6909c07
Merge bitcoin/bitcoin#23516: test: Force --nosandbox when --valgrind
fa9c26ab3a test: Force --nosandbox when --valgrind (MarcoFalke)

Pull request description:

  The two options are mutually exclusive and will result in a test failure. Fix that.

  Can be tested with:

  ```
  $ ./test/functional/wallet_disable.py --valgrind

ACKs for top commit:
  fanquake:
    ACK fa9c26ab3a

Tree-SHA512: 7d1c36c1b6627ca041757eb0515a0d6cc962a56d783ee4f5647a2ddc2d104491f0586a8ea0b8acebe0a203190f4f5567b349123dfd4c181bcc63361174a8ab63
2021-11-16 07:30:03 +08:00