Commit graph

4559 commits

Author SHA1 Message Date
MarcoFalke
d565d9b56d
Merge bitcoin/bitcoin#23312: tests: reduce feature_segwit.py usage of the legacy wallet
e9ade032f3 tests: Add feature_segwit.py --descriptors to test_runner.py (Andrew Chow)
ae6cbcc909 tests: restrict feature_segwit legacy wallet import tests (Andrew Chow)
1d13c44a4c tests: Use descriptors for feature_segwit multisig setup (Andrew Chow)

Pull request description:

  `feature_segwit.py` has tests for some legacy wallet behavior, but otherwise does not really need the legacy wallet. Those parts now require `--legacy-wallet` to be provided (as with other legacy wallet tests). Other parts of the test are changed to work with descriptor wallets as well.

ACKs for top commit:
  lsilva01:
    tACK e9ade03

Tree-SHA512: 4ae76a4d2a8d318e7f8ad4c984748e3cdd67ed00359fcd798a08492ed9399b1d01be88c9ebea3d6c996fbe190cf41708a15c98f088f0cb5c47d6d00fff258944
2021-10-25 15:32:27 +02:00
MarcoFalke
beb45b8b14
Merge bitcoin/bitcoin#23311: wallet: Use PACKAGE_NAME to mention our software
da791c7f66 wallet: Use PACKAGE_NAME to mention our software (Hennadii Stepanov)

Pull request description:

  This PR replaces "bitcoin" and "bitcoind" with `PACKAGE_NAME` in wallet log and error messages.

ACKs for top commit:
  jonatack:
    ACK da791c7f66
  lsilva01:
    Tested ACK da791c7 on Ubuntu 20.04.
  brunoerg:
    tACK da791c7f66
  stratospher:
    Tested ACK da791c7
  shaavan:
    ACK da791c7f66

Tree-SHA512: c613446d9c8c3f85e6f5fec77382c9bc17746a853c89e72e1a3a79bf355d7bd9e455bbde8f9e02a894d225a67029c732cdc68ec8c58ac8237dde27d39dae8be7
2021-10-25 15:24:00 +02:00
MarcoFalke
49e40f5704
Merge bitcoin/bitcoin#22711: test: check for specific block reject reasons in p2p_segwit.py
45827fd718 test: check for block reject reasons in p2p_segwit.py [2/2] (Sebastian Falbesoner)
4eb532ff8b test: check for block reject reasons in p2p_segwit.py [1/2] (Sebastian Falbesoner)
b1488c4dce test: fix reference to block processing test in p2p_segwit.py (Sebastian Falbesoner)

Pull request description:

  In the test `p2p_segwit.py`, there are many instances where we send a segwit block to a node with the expectation that it is rejected. For this purpose, the helper function `test_witness_block` exists which allows also to check for a specific reject `reason` that is asserted in the debug log:
  502d22ceed/test/functional/p2p_segwit.py (L119-L120)

  This PR aims to increase the precision of the tests by adding the expected reject reasons to all `test_witness_block` call instances (found via `grep`ing after `test_witness_block(.*accepted=False`). For some blocks that are rejected due to failed script verification, the exact reason is only shown in the debug log if parallel script verification is disabled. For this reason, the addition of the reasons is split up in two commits:
  * first, all instances are tackled that are not subject to script verification, i.e. it doesn't matter whether parallel script verification is enabled or not (e.g. `bad-blk-weight`, `bad-witness-merkle-match`...)
  * then, we explicitely set `-par=1` to only use one script thread, and add the remaining reasons (`non-mandatory-script-verify-flag` with the more specific reason in parantheses)

ACKs for top commit:
  stratospher:
    tested ACK 45827fd.

Tree-SHA512: 00f31867f11d48b38a42b1e79a1303bda1c797ccf3d8c73e6107d70b062604d51ee2a3f2251e7f068dfafdaf09469d27dfee438d9bc9ebaef7febc4b6ef90a95
2021-10-25 14:53:40 +02:00
W. J. van der Laan
a1d55ced09
Merge bitcoin/bitcoin#23139: rpc: fix "trusted" field in TransactionDescriptionString(), add coverage
66f6efc70a rpc: improve TransactionDescriptionString() "generated" help (Jon Atack)
296cfa312f test: add listtransactions/listsinceblock "trusted" coverage (Jon Atack)
d95913fc43 rpc: fix "trusted" description in TransactionDescriptionString (Jon Atack)

Pull request description:

  The RPC gettransaction, listtransactions, and listsinceblock helps returned by `TransactionDescriptionString()` inform the user that the `trusted` boolean field is only present if the transaction is trusted and safe to spend from.

  The field is in fact returned by `WalletTxToJSON()` when the transaction has 0 confirmations (or negative confirmations, if conflicted), and it can be true or false.

  This patch fixes the help, adds test coverage, and touches up the help for the neighboring `generate` field.

ACKs for top commit:
  rajarshimaitra:
    tACK 66f6efc70a
  theStack:
    Tested ACK 66f6efc70a

Tree-SHA512: 4c2127765b82780e07bbdbf519d27163d414d9f15598e01e02210f210e6009be344c84951d7274e747b1386991d4c3b082cd25aebe885fb8cf0b92d57178f68e
2021-10-22 16:26:48 +02:00
MarcoFalke
02feae54a5
Merge bitcoin/bitcoin#23002: Make descriptor wallets by default
9c1052a521 wallet: Default new wallets to descriptor wallets (Andrew Chow)
f19ad40463 rpc, wallet: Descriptor wallets are no longer experimental (Andrew Chow)

Pull request description:

  Changes the default wallet type from legacy to descriptors. Descriptor wallets will now by the default type. Additionally, descriptor wallets will no longer be marked as experimental.

  This follows the timeline proposed in #20160

ACKs for top commit:
  lsilva01:
    Tested ACK 9c1052a521 on Ubuntu 20.04
  prayank23:
    tACK 9c1052a521
  meshcollider:
    Code review ACK 9c1052a521

Tree-SHA512: 834e6fec88e0c18673af7ebe135bd5333694d1be502164eb93a90e3e76c27974165aa4e59426945100c88e4eca07356e16886ef5b05cf789683ecb23fc71a12a
2021-10-22 13:31:10 +02:00
MarcoFalke
12ff8993bc
Merge bitcoin/bitcoin#23331: test: Replace log with assert_equal in wallet_abandonconflict
fa6c62f34b test: Replace log with assert_equal in wallet_abandonconflict (MarcoFalke)

Pull request description:

  This will make the test fail as soon as the bug is fixed, forcing it to
  be updated. Otherwise, the bug might be fixed (or made worse)
  accidentally, leaving the test in an inconsistent state.

ACKs for top commit:
  theStack:
    Concept and code-review ACK fa6c62f34b
  brunoerg:
    tACK fa6c62f34b

Tree-SHA512: 416f72380164bf3f93332a5cfa81a8e61d8ada3714ef6815889ed3cf2d16c09411760dee4acf19629227e565b765b3dea491a0b23efd38eb370254cfadf7c441
2021-10-21 22:00:49 +02:00
W. J. van der Laan
8a083bc5b5
Merge bitcoin/bitcoin#23218: p2p: Use mocktime for ping timeout
fadf1186c8 p2p: Use mocktime for ping timeout (MarcoFalke)

Pull request description:

  It is slightly confusing to use mocktime for some times, but not others.

  Start fixing that by making the ping timeout use mocktime.

  The only downside would be that tests that use mocktime disconnect peers after this patch. However, I don't think this is an issue, as the inactivity check is already disabled for all functional tests after commit 6d76b57ca0. Only one unit test needed the inactivity check disabled as part of this patch.

  A nice side effect of this patch is that the `p2p_ping` functional test now runs 4 seconds faster.

ACKs for top commit:
  laanwj:
    Code review ACK fadf1186c8

Tree-SHA512: e9e7b21040a89d9d574b3038f85a67e6336de6cd6e41aa286769cd03cada6e75a94ec01700e052e56d822ef85d7813cc06bf7e67b81543eff8917a16cdccf942
2021-10-21 19:44:38 +02:00
MarcoFalke
fa6c62f34b
test: Replace log with assert_equal in wallet_abandonconflict
This will make the test fail as soon as the bug is fixed, forcing it to
be updated. Otherwise, the bug might be fixed (or made worse)
accidentally, leaving the test in an inconsistent state.
2021-10-21 11:15:15 +02:00
MarcoFalke
88fc7950f8
Merge bitcoin/bitcoin#23267: test: bip125-replaceable in listsinceblock
b7884dd1b6 test: bip125-replaceable in listsinceblock (brunoerg)

Pull request description:

  This PR adds test coverage for bip125-replaceable in listsinceblock. I added this test into wallet_listtransactions.py instead of putting it into wallet_listsinceblock.py to utilize the scenario already created in wallet_listtransactions.py and avoid repetition.

ACKs for top commit:
  theStack:
    ACK b7884dd1b6
  promag:
    ACK b7884dd1b6.
  stratospher:
    tested ACK b7884dd. Verified the bip125-replaceable status of some transactions with listsinceblock.
  lsilva01:
    tACK b7884dd  on Ubuntu 20.04

Tree-SHA512: 510dfe5a6f9d68e5a656514d356dc8fe99324296ed8caa78f0eb4b6c6906cf70b1fb50bde80aa6f61d726b2fa1d4ce1fe48c635ce24285588e56ceff92291617
2021-10-21 10:21:30 +02:00
MarcoFalke
548ad5ef7f
Merge bitcoin/bitcoin#23281: test: check that bumpfee RPC fails for txs with descendants in mempool
4ac8c89ad9 test: check that bumpfee RPC fails for txs with descendants in mempool (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the bumpfee RPC error _"Transaction has descendants in the mempool"_,

  6419bdfeb1/src/wallet/feebumper.cpp (L29-L32)

  which is thrown if the bumped tx has descendants in the mempool and is _not_ connected to the bitcoin wallet (for those, the error "Transaction has descendants in the Wallet" is thrown a few lines above). To achieve that, the test framework's MiniWallet is used.

ACKs for top commit:
  brunoerg:
    tACK 4ac8c89ad9
  promag:
    Code review ACK 4ac8c89ad9. Nice stuff!
  lsilva01:
    Tested ACK 4ac8c89 cad756b10c9dee2d9e1405 on Ubuntu 20.04.
  stratospher:
    tested ACK 4ac8c89.

Tree-SHA512: 83e99f9dd2b140c0c0597c0c36c9c948fa334871be40e58a5e004440698d9685661c69bb83ab937d30f692545a3799705f991b31904f2ef31a2fbc3ae1179fa8
2021-10-21 10:14:47 +02:00
MarcoFalke
9aa4ddb410
Merge bitcoin/bitcoin#23287: test: get and decode tx with a single gettransaction RPC call
130ee48108 test: get and decode tx with a single `gettransaction` RPC call (Sebastian Falbesoner)

Pull request description:

  Rather than subsequently calling `gettransaction` and `decoderawtransaction` to get the decoded information  for a specific tx-id, we can simply use the verbose version of `gettransaction`, which returns this in a 'decoded' key. I.e.

  ```
  node.decoderawtransaction(node.gettransaction(txid)['hex'])
  ```

  can simply be replaced by:

  ```
  node.gettransaction(txid=txid, verbose=True)['decoded']
  ```

  Rationale: shorter code, shorter test logs, less RPC calls.

ACKs for top commit:
  stratospher:
    tested ACK 130ee48
  amadeuszpawlik:
    tACK 130ee48108
  lsilva01:
    Tested ACK 130ee48 on Ubuntu 20.04.
  shaavan:
    ACK 130ee48108

Tree-SHA512: cf0bd26e1e21b8022fb8062857906e0706f0ee32d3277f985c461e2519405afe445ab005f5f763fb268c7b4d6e48b2d47eda7af8621b3bce67cece8dfc9bc153
2021-10-21 10:07:06 +02:00
fanquake
69986de383
Merge bitcoin/bitcoin#22839: log: improve addrman logging
b65a25a846 log: improve addrman logging (Martin Zumsande)

Pull request description:

  The addrman helper functions `GetNewBucket()` and `GetTriedBucket()`
  1) log into the wrong category (`BCLog::NET` instead of `BCLog::ADDRMAN`)
  2)  log too unspecifically - especially `GetTriedBucket()` gets called from many  different places (e.g. `Check_()`, `Serialize()`), it seems sufficient to me logging these when moving an address from new to tried. Running a node with `-checkaddrman=1`and net logging currently results in a lot of repetitive log entries.

  This PR moves these log entries to `Add_()` and `Good_()` and also adds logging for `Select_()` (allowing statistics about New/Tried success probabilities), `GetAddr_()`, `ClearNew()` and `MakeTried()`.

ACKs for top commit:
  jnewbery:
    ACK b65a25a846
  vasild:
    ACK b65a25a846

Tree-SHA512: 90ab0f64eb44b7388a198efccb613577b74989fea73194bda7de8bfbd50bdb19127cb12f5ec645c7859afdb89290614a79e255f3af0a63a58d4f21aa8fe7b696
2021-10-21 08:35:30 +08:00
Hennadii Stepanov
da791c7f66
wallet: Use PACKAGE_NAME to mention our software 2021-10-20 21:09:14 +03:00
James O'Beirne
23f85616a8
test: add node.chain_path and node.debug_log_path
To allow easier access to the node's datadir and debug logs.
2021-10-20 11:24:56 -04:00
lsilva01
7b3c9e4ee8 Make explicit the node param in init_wallet() 2021-10-20 00:30:28 -03:00
Andrew Chow
e9ade032f3 tests: Add feature_segwit.py --descriptors to test_runner.py 2021-10-19 18:43:18 -04:00
Andrew Chow
ae6cbcc909 tests: restrict feature_segwit legacy wallet import tests
A portion of feature_segwit deals with the legacy wallet IsMine and
import behavior. This is now hidden behind --legacy-wallet
2021-10-19 18:41:46 -04:00
Andrew Chow
1d13c44a4c tests: Use descriptors for feature_segwit multisig setup
When setting up the multisig addresses in feature_segwit.py, use
descriptors rather than addmultisigaddress.
2021-10-19 18:41:05 -04:00
W. J. van der Laan
986003aff9
Merge bitcoin/bitcoin#22918: rpc: Add level 3 verbosity to getblock RPC call (#21245 modified)
5c34507ecb core_write: Rename calculate_fee to have_undo for clarity (fyquah)
8edf6204a8 release-notes: Add release note about getblock verbosity level 3. (fyquah)
459104b2aa rest: Add test for prevout fields in getblock (fyquah)
4330af6f72 rpc: Add test for level 3 verbosity getblock rpc call. (fyquah)
51dbc167e9 rpc: Add level 3 verbosity to getblock RPC call. (fyquah)
3cc95345ca rpc: Replace boolean argument for tx details with enum class. (fyquah)

Pull request description:

  Author of #21245 expressed [time issues](https://github.com/bitcoin/bitcoin/pull/21245#issuecomment-902332088) in the original PR. Given that #21245 has received a lot of review*, I have decided to open this new pull request with [modifications required to get ACK from luke-jr ](https://github.com/bitcoin/bitcoin/pull/21245#issuecomment-905150806) and a few nits of mine.

  ### Original PR description

  > Display the prevout in transaction inputs when calling getblock level 3 verbosity. This PR affects the existing `/rest/block` API by adding a `prevout` fields to tx inputs. This is mentioned in the change to the release notes.
  >
  > I added some functional tests that
  >
  >     * checks that the RPC call still works when TxUndo can't be found
  >
  >     * Doesn't display the "value" or "scriptPubKey" of the previous output when at a lower verbosity level
  >
  >
  > This "completes" the issue #18771

  ### Possible improvements

  * b0bf4f255f - I can include even this commit to this PR if deemed useful or I can leave it for a follow-up PR. See https://github.com/bitcoin/bitcoin/pull/21245#issuecomment-894853784 for more context.

  ### Examples

  Examples of the `getblock` output with various verbose levels. Note that `000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5` contains only 2 transactions.

  #### Verbose level 0

  ```bash
  ./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 0
  ```

  ##### Verbose level 1

  ```bash
  ./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 1
  ```

  ##### Verbose level 2

  ```bash
  ./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 2
  ```

  ##### Verbose level 3

  ```bash
  ./bitcoin-cli -testnet getblock 000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5 3
  ```

  #### REST

  ```bash
  curl -H "content-type:text/plain;" http://127.0.0.1:18332/rest/block/000000000000001f682b188971cc1a121546be4e9d5baf22934fdc7f538288d5.json
  ```

  <sub>* ... and my everyday obsessive checking of my email inbox whether the PR moves forward.</sub>

  Edit laanwj: Removed at symbol from message, and large example output to prevent it from all ending up in the commit message.

ACKs for top commit:
  0xB10C:
    ACK 5c34507ecb
  meshcollider:
    utACK 5c34507ecb
  theStack:
    ACK 5c34507ecb 👘
  promag:
    Concept ACK 5c34507ecb

Tree-SHA512: bbff120d8fd76e617b723b102b0c606e0d8eb27f21c631d5f4cdab0892137c4bc7c65b1df144993405f942c91be47a26e80480102af55bff22621c19f518aea3
2021-10-19 15:47:53 +02:00
MarcoFalke
fa44b071fd
test: Remove unused node from mining_prioritisetransaction 2021-10-19 11:02:10 +02:00
Martin Zumsande
b65a25a846 log: improve addrman logging 2021-10-19 00:20:28 +02:00
Hennadii Stepanov
ffdd94d753
test: Fix wallet_multisig_descriptor_psbt.py 2021-10-18 23:14:56 +03:00
W. J. van der Laan
ff65b696f3
Merge bitcoin/bitcoin#22067: Test and document a basic M-of-N multisig using descriptor wallets and PSBTs
9de0d94508 doc: add disclaimer highlighting shortcomings of the basic multisig example (Michael Dietz)
f9479e4626 test, doc: basic M-of-N multisig minor cleanup and clarifications (Michael Dietz)
e05cd0546a doc: add another signing flow for multisig with descriptor wallets and PSBTs (Michael Dietz)
17dd657300 doc: M-of-N multisig using descriptor wallets and PSBTs, as well as a signing flow (Michael Dietz)
1f20501efc test: add functional test for multisig flow with descriptor wallets and PSBTs (Michael Dietz)

Pull request description:

  Aims to resolve issue https://github.com/bitcoin/bitcoin/issues/21278. I try to follow the steps laanwj outlined there exactly, with the exception of using `combinepsbt` instead of `joinpsbts`. I wrote a functional test to make sure it works as expected before doing the docs, and figured it would also be a good source of documentation. So I kept the test as simple as possible and didn't go crazy with edge-cases and various checks. I do have a lot more test-cases I've written that I will follow up with (either in a separate PR or another commit - lmk if you have a preference), but I want to do it in a way that doesn't bloat this test so it remains useful as a quickstart (unless that's a bad idea)?

ACKs for top commit:
  S3RK:
    Code review ACK 9de0d94. Rspigler's argument convinced me that we should leave the workflow with two wallets. I assume using multisig with external signers is a popular use-case and it's important to keep compatibility.
  laanwj:
    Code and documentation review ACK 9de0d94508

Tree-SHA512: 6c76e787c21f09d8be5eaa11f3ca3eaa4868497824050562bdfb2095c73b90f5e8987a8775119891d6bfde586e3f31ad1b13e4b67b0802e1d23ef050227a1211
2021-10-18 16:17:45 +02:00
MarcoFalke
2e82af46e2
Merge bitcoin/bitcoin#23207: test: Delete generate* calls from TestNode
fac62e6ff5 test: Delete generate* calls from TestNode (MarcoFalke)
fac7f6102f test: Use generate* node RPC, not wallet RPC (MarcoFalke)
faac1cda6e test: Use generate* from TestFramework, not TestNode (MarcoFalke)

Pull request description:

  Deleting the methods is needed for #22567 to pave the way to make it easier to implicitly call the `sync_all` member function.

  Without the methods being deleted, nothing prevents developers from adding calls to it. As history showed, developers *will* add calls to it. For example, see commit eb02dbba3c from today or the first commit in this pull request.

ACKs for top commit:
  stratospher:
    Tested ACK fac62e6.
  brunoerg:
    tACK fac62e6ff5
  promag:
    Code review ACK fac62e6ff5.

Tree-SHA512: 6d4dea8f95ead954acfef2e6a5d98897ce0c2d02265c5b137bb149d0265543bd51d7e8403e1945b9af75df5524ca50064fe1d2a432b25c8abc71bbb28ed6ed53
2021-10-18 13:06:03 +02:00
fanquake
ef596923a8
Merge bitcoin/bitcoin#23080: test: check abandoned tx in listsinceblock
bda620aecd test: check abandoned tx in listsinceblock (brunoerg)

Pull request description:

  This PR tests if the abandoned transaction is correct in listsinceblock return (wallet_abandonconflict.py).

ACKs for top commit:
  jonatack:
    ACK bda620aecd
  theStack:
    ACK bda620aecd
  stratospher:
    Tested ACK bda620a. This PR verifies whether the transaction txAB1 has been abandoned in listsinceblock and is a nice addition to the test!

Tree-SHA512: e4dce344cf621de7a8b5bd8660d252419772a293080fc881f6f448b6df85c6b1c8f0df619e855a40b6393f53c836f0d7fadbd3916c20cccd3a95830b8b502991
2021-10-18 12:02:55 +08:00
Sebastian Falbesoner
130ee48108 test: get and decode tx with a single gettransaction RPC call
Rather than subsequently calling `gettransaction` and
`decoderawtransaction` to get the decoded information  for a specific
tx-id, we can simply use the verbose version of `gettransaction`, which
returns this in a 'decoded' key. I.e.

node.decoderawtransaction(node.gettransaction(txid)['hex'])

can be replaced by:

node.gettransaction(txid=txid, verbose=True)['decoded']
2021-10-17 18:10:36 +02:00
Matt Corallo
ef7c8228fd Expose block filters over REST.
This adds a new rest endpoint:
/rest/blockfilter/filtertype/requesttype/blockhash (eg
/rest/blockfilter/basic/header/000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f.hex)
which exposes either the filter "header" or the filter data itself.
Most of the code is cribbed from the equivalent RPC.
2021-10-17 02:56:56 +00:00
josibake
a46f71bb70
lint: enable mypy checking for missing imports
Achieve this by adding some ignore, and making data/ importable.

Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2021-10-16 09:14:37 +08:00
Sebastian Falbesoner
4ac8c89ad9 test: check that bumpfee RPC fails for txs with descendants in mempool
This commit adds missing test coverage for the bumpfee RPC
error "Transaction has descendants in the mempool", which is
thrown if the bumped tx has descendants in the mempool and is
_not_ connected to the bitcoin wallet. To achieve that, the
test framework's MiniWallet is used.
2021-10-15 02:38:18 +02:00
W. J. van der Laan
6419bdfeb1
Merge bitcoin/bitcoin#23093: Add ability to flush keypool and always flush when upgrading non-HD to HD
6531599f42 test: Add check that newkeypool flushes change addresses too (Samuel Dobson)
84fa19c77a Add release notes for keypool flush changes (Samuel Dobson)
f9603ee4e0 Add test for flushing keypool with newkeypool (Samuel Dobson)
6f6f7bb36c Make legacy wallet upgrades from non-HD to HD always flush the keypool (Samuel Dobson)
2434b10781 Fix outdated keypool size default (Samuel Dobson)
22cc797ca5 Add newkeypool RPC to flush the keypool (Samuel Dobson)

Pull request description:

  This PR makes two main changes:
  1) Adds a new RPC `newkeypool` which will entirely flush and refill the keypool.
  2) When upgradewallet is called on old, non-HD wallets upgrading them to HD, we now always flush the keypool and generate a new one, to immediately start using the HD generated keys.

  This PR is motivated by a number of users with old, pre-compressed-key wallets upgrading them and being confused about why they still can't generate p2sh-segwit or bech32 addresses -- this is due to uncompressed keys remaining in the keypool post-upgrade and being illegal in these newer address formats. There is currently no easy way to flush the keypool other than to call `getnewaddress` a hundred/thousand times or an ugly hack of using a `sethdseed` call.

ACKs for top commit:
  laanwj:
    re-ACK 6531599f42
  meshcollider:
    Added new commit 6531599f42 to avoid invalidating previous ACKs.
  instagibbs:
    ACK 6531599f42

Tree-SHA512: 50c79c5d42dd27ab0ecdbfdc4071fdaa1b2dbb2f9195ed325b007106ff19226419ce57fe5b1539c0c24101b12f5e034bbcfb7bbb0451b766cb1071295383d774
2021-10-14 18:05:58 +02:00
Samuel Dobson
ec4e43c21c
Merge #23235: Reduce unnecessary default logging
b5950dd59c validation: put coins cache write log into bench debug log (Anthony Towns)
31b2b802b5 blockstorage: use debug log category (Anthony Towns)
da94ebc2fa validation: move header validation error logging to VALIDATION debug category (Anthony Towns)
1d7d835ec3 validation: include block hash when reporting prev block not found errors (Anthony Towns)

Pull request description:

  Moves the following log messages into debug log categories:

   * "AcceptBlockHeader: ..." to validation
   * "Prune: deleted blk/rev" to new blockstorage log category
   * "Leaving block file" moves from validation to blockstorage
   * "write coins cache to disk" to bench

  Also adds the hash of the block to the log message when AcceptBlockHeader is rejecting because of problems with the prev block.

ACKs for top commit:
  practicalswift:
    cr ACK b5950dd59c
  Empact:
    Code review ACK b5950dd59c
  laanwj:
    Code review ACK b5950dd59c
  promag:
    Code review ACK b5950dd59c.
  meshcollider:
    Code review ACK b5950dd59c

Tree-SHA512: a73fdbfe8d36da48a3e89c2d5e0b6a3c5045d280c1a57f61c38d0d21f4f198aece4bd85155be3439e179d5dabdb523bf15fa0395e0e3ceff19c878ba3112c840
2021-10-14 18:40:59 +13:00
Samuel Dobson
6531599f42 test: Add check that newkeypool flushes change addresses too 2021-10-14 16:52:59 +13:00
brunoerg
b7884dd1b6 test: bip125-replaceable in listsinceblock 2021-10-12 12:10:48 -03:00
MarcoFalke
fa8d492894
rest: Return error when header count is not integral 2021-10-12 09:10:19 +02:00
Samuel Dobson
42d6a029e5 Refactor and add more tests for validateaddress 2021-10-12 12:03:14 +13:00
Samuel Dobson
0b06e720c0 More detailed error checking for base58 addresses 2021-10-12 12:03:14 +13:00
Andrew Chow
80dc829be7 tests: Calculate fees more similarly to CFeeRate::GetFee
Because of floating point precision issues, not all of the rounding done
is always correct. To fix this, the fee calculation for
assert_fee_amount is changed to better reflect how CFeeRate::GetFee does
it.

First the feerate is converted to an int representing sat/kvb. Then this
is multiplied by the transaction size, divivided by 1000, and rounded up
to the nearest sat. The result is then converted back to BTC (divided by
1e8) and then rounded down to the nearest sat to avoid precision errors.
2021-10-11 12:29:33 -04:00
Anthony Towns
31b2b802b5 blockstorage: use debug log category 2021-10-11 21:45:49 +10:00
MarcoFalke
1790a8ddac
Merge bitcoin/bitcoin#22794: test: Verify if wallet is compiled in rpc_invalid_address_message.py test
c2fbdca549 Add BECH32_INVALID_VERSION test (lsilva01)
b142f79ddb skip test_getaddressinfo() if wallet is disabled (lsilva01)

Pull request description:

  Most of  `test/functional/rpc_invalid_address_message.py` does not requires wallet.
  But if the project is compiled in disable-wallet mode, the entire test will be skipped.

  This PR changes the test to run the RPC tests first and then checks if the wallet is compiled.

ACKs for top commit:
  stratospher:
    tested ACK c2fbdca

Tree-SHA512: 11fa2fedf4a15aa45e3f12490df8e22290a867d5de594247211499533c32289c68c0b60bd42dbf8305e43dbcc042789e7139317ef5c9f8cf386f2d84c91b9ac2
2021-10-11 11:27:37 +02:00
Andrew Chow
ce2cc44afd tests: Test for assertion when feerate is rounded down
When calculating a txs absolute fee, if the fee is rounded down to the
nearest satoshi, it is possible for the coin selection algorithms to
undercalculate the fee needed. This can lead to an assertion error in
some situations. One such scenario is added to
rpc_fundrawtransaction.py.
2021-10-08 13:53:48 -04:00
Andrew Chow
0fbaef9676 fees: Always round up fee calculated from a feerate
When calculating the fee for a given tx size from a fee rate, we should
always round up to the next satoshi. Otherwise, if we round down (via
truncation), the calculated fee may result in a fee with a feerate
slightly less than targeted.

This is particularly important for coin selection as a slightly lower
feerate than expected can result in a variety of issues.
2021-10-08 13:53:48 -04:00
W. J. van der Laan
991753e4d5
Merge bitcoin/bitcoin#23118: test: refactor: introduce script_util helper for creating P2PK scripts
429b49378e test: introduce script_util helper for creating P2PK scripts (Sebastian Falbesoner)

Pull request description:

  This PR is a follow-up to #22363, which took use of already existing `script_util` helpers to get rid of manual CScript for the P2{PKH,SH,WPKH,WSH} output types, in order to increase readability and maintainability of the test code. Here the same is done for P2PK scripts by introducing a helper `key_to_p2pk_script` and using it. Note that the helper only accepts ECDSA pubkeys (i.e. ones with a size of 33 or 65 bytes), hence it can't be used for scripts in the form of [x-only-pubkey, OP_CHECKSIG].

ACKs for top commit:
  brunoerg:
    Code review ACK 429b49378e
  laanwj:
    Code review ACK 429b49378e
  rajarshimaitra:
    Concept + tACK 429b49378e

Tree-SHA512: 984aea01eba5f38a328d69905d90a3a36f0a02419ca3e5baf3c8095895fb094e3780c7da16fad5851db3847bdb05ce8cda244ab09b79b8aa9602dfb3c5e0414c
2021-10-07 15:41:57 +02:00
W. J. van der Laan
6f0cbc75be
Merge bitcoin/bitcoin#22539: Re-include RBF replacement txs in fee estimation
3b613722f6 Add release notes for fee est with replacement txs (Antoine Poinsot)
4556406562 qa: test fee estimation with replacement transactions (Antoine Poinsot)
053415b297 qa: split run_test into smaller parts (Antoine Poinsot)
06c5ce9714 Re-include RBF replacement txs in fee estimation (Antoine Poinsot)

Pull request description:

  This effectively reverts #9519.

  RBF is now largely in use on the network (signaled for by around 20% of
  all transactions on average) and replacement logic is implemented in
  most end-user wallets. The rate of replaced transactions is also
  expected to rise as fee-bumping techniques are being developed for
  pre-signed transaction ("L2") protocols.

ACKs for top commit:
  prayank23:
    reACK 3b613722f6
  Zero-1729:
    re-ACK 3b613722f6
  benthecarman:
    reACK 3b613722f6
  glozow:
    ACK 3b613722f6
  theStack:
    re-ACK 3b613722f6 🍪

Tree-SHA512: a6146d15c80ff4ba9249314b0ef953a66a15673e61b8f98979642814f1b169b5695e330e3ee069fa9a7e4d1f8aa10e1dcb7f9aa79181cea5a4c4dbcaf5483023
2021-10-07 13:47:36 +02:00
MarcoFalke
fadf1186c8
p2p: Use mocktime for ping timeout 2021-10-07 13:22:02 +02:00
MarcoFalke
c0b6c96eee
Merge bitcoin/bitcoin#23146: Test transactions conflicted by double spend in listtransactions
502f50da12 Test transactions conflicted by double spend in listtransactions (Jon Atack)

Pull request description:

  Test the properties of transactions conflicted by a double spend as returned by RPC listtransactions in the abandoned, confirmations, trusted and walletconflicts fields. These fields are also returned by RPCs listsinceblock and gettransactions.

ACKs for top commit:
  brunoerg:
    tACK 502f50da12
  rajarshimaitra:
    Concept + tACK 502f50da12

Tree-SHA512: 28968f4a5f1960ea45b2e6f5b20fe25c1b51f66944062dcddea52ea970ad21c74d583793d091b84e8a5e506d6aecc1f0435c5b918213975b22c38e02bba19aa1
2021-10-07 09:59:03 +02:00
MarcoFalke
b4437d7dfe
Merge bitcoin/bitcoin#23210: test: Replace satoshi_round with int() or Decimal()
fa2ac5881e test: Replace satoshi_round with int() or Decimal() (MarcoFalke)

Pull request description:

  satoshi_round will round down. To make the code easier to parse use
  Decimal() where possible, which does not round. Or use int(), which
  explicitly rounds down.

ACKs for top commit:
  lsilva01:
    Tested ACK fa2ac5881e on Ubuntu 20.04.

Tree-SHA512: 17795d906aa7652933d43e510e993cdd9cf8926da1febf1c42d463048cb38c92dc518ec08736efe29c0189ffd532b108bc7a715f32b4c2ee58b215df65352eb9
2021-10-07 09:09:11 +02:00
MarcoFalke
fa2ac5881e
test: Replace satoshi_round with int() or Decimal()
satoshi_round will round down. To make the code easier to parse use
Decimal() where possible, which does not round. Or use int(), which
explicitly rounds down.
2021-10-06 15:16:59 +02:00
MarcoFalke
fa3ab7b5b1
test: Avoid RPC roundtrip in MiniWallet get_descriptor() 2021-10-06 14:54:26 +02:00
MarcoFalke
fac62e6ff5
test: Delete generate* calls from TestNode 2021-10-06 13:41:08 +02:00
MarcoFalke
fac7f6102f
test: Use generate* node RPC, not wallet RPC 2021-10-06 13:39:44 +02:00
MarcoFalke
faac1cda6e
test: Use generate* from TestFramework, not TestNode 2021-10-06 13:39:39 +02:00
S3RK
3d71d16d1e test: listtranscations with externally generated addresses 2021-10-06 10:01:53 +02:00
Samuel Dobson
75a9305d45 Fix intermittent test failures due to missing sync_all 2021-10-06 12:19:04 +13:00
Samuel Dobson
eb02dbba3c Use self.generate not node.generate throughout tests 2021-10-06 12:18:33 +13:00
Jon Atack
22b44fc696
p2p: improve checkaddrman logging with duration in milliseconds
and update the function name to CheckAddrman (drop "Force") for
nicer log output as it is prefixed to each of these log messages:

2021-09-21T18:42:50Z [opencon] CheckAddrman: new 64864, tried 1690, total 66554 started
2021-09-21T18:42:50Z [opencon] CheckAddrman: completed (76.21ms)

The existing Doxygen documentation on the function already makes
clear that it is unaffected by m_consistency_check_ratio.
2021-10-05 18:34:22 +02:00
MarcoFalke
c4fc899442
Merge bitcoin/bitcoin#22950: [p2p] Pimpl AddrMan to abstract implementation details
021f86953e [style] Run changed files through clang formatter. (Amiti Uttarwar)
375750387e scripted-diff: Rename CAddrInfo to AddrInfo (Amiti Uttarwar)
dd8f7f2500 scripted-diff: Rename CAddrMan to AddrMan (Amiti Uttarwar)
3c263d3f63 [includes] Fix up included files (Amiti Uttarwar)
29727c2aa1 [doc] Update comments (Amiti Uttarwar)
14f9e000d0 [refactor] Update GetAddr_() function signature (Amiti Uttarwar)
40acd6fc9a [move-only] Move constants to test-only header (Amiti Uttarwar)
7cf41bbb38 [addrman] Change CAddrInfo access (Amiti Uttarwar)
e3f1ea659c [move-only] Move CAddrInfo to test-only header file (Amiti Uttarwar)
7cba9d5618 [net, addrman] Remove external dependencies on CAddrInfo objects (Amiti Uttarwar)
8af5b54f97 [addrman] Introduce CAddrMan::Impl to encapsulate addrman implementation. (Amiti Uttarwar)
f2e5f38f09 [move-only] Match ordering of CAddrMan declarations and definitions (Amiti Uttarwar)
5faa7dd6d8 [move-only] Move CAddrMan function definitions to cpp (Amiti Uttarwar)

Pull request description:

  Introduce the pimpl pattern for AddrMan to separate the implementation details from the externally used object representation. This reduces compile-time dependencies and conceptually clarifies AddrMan's interface from the implementation specifics.

  Since the unit & fuzz tests currently rely on accessing AddrMan internals, this PR introduces addrman_impl.h, which is exclusively imported by addrman.cpp and test files.

ACKs for top commit:
  jnewbery:
    ACK 021f86953e
  GeneFerneau:
    utACK [021f869](021f86953e)
  mzumsande:
    ACK 021f86953e
  rajarshimaitra:
    Concept + Code Review ACK 021f86953e
  theuni:
    ACK 021f86953e

Tree-SHA512: aa70cb77927a35c85230163c0cf6d3872382d79048b0fb79341493caa46f8e91498cb787d8b06aba4da17b2f921f2230e73f3d66385519794fff86a831b3a71d
2021-10-05 16:48:33 +02:00
fyquah
459104b2aa rest: Add test for prevout fields in getblock 2021-10-05 10:42:34 +02:00
fyquah
4330af6f72 rpc: Add test for level 3 verbosity getblock rpc call. 2021-10-05 10:42:34 +02:00
W. J. van der Laan
9e530c6352
Merge bitcoin/bitcoin#20487: Add syscall sandboxing using seccomp-bpf (Linux secure computing mode)
4747da3a5b Add syscall sandboxing (seccomp-bpf) (practicalswift)

Pull request description:

  Add experimental syscall sandboxing using seccomp-bpf (Linux secure computing mode).

  Enable filtering of system calls using seccomp-bpf: allow only explicitly allowlisted (expected) syscalls to be called.

  The syscall sandboxing implemented in this PR is an experimental feature currently available only under Linux x86-64.

  To enable the experimental syscall sandbox the `-sandbox=<mode>` option must be passed to `bitcoind`:

  ```
    -sandbox=<mode>
         Use the experimental syscall sandbox in the specified mode
         (-sandbox=log-and-abort or -sandbox=abort). Allow only expected
         syscalls to be used by bitcoind. Note that this is an
         experimental new feature that may cause bitcoind to exit or crash
         unexpectedly: use with caution. In the "log-and-abort" mode the
         invocation of an unexpected syscall results in a debug handler
         being invoked which will log the incident and terminate the
         program (without executing the unexpected syscall). In the
         "abort" mode the invocation of an unexpected syscall results in
         the entire process being killed immediately by the kernel without
         executing the unexpected syscall.
  ```

  The allowed syscalls are defined on a per thread basis.

  I've used this feature since summer 2020 and I find it to be a helpful testing/debugging addition which makes it much easier to reason about the actual capabilities required of each type of thread in Bitcoin Core.

  ---

  Quick start guide:

  ```
  $ ./configure
  $ src/bitcoind -regtest -debug=util -sandbox=log-and-abort
  …
  2021-06-09T12:34:56Z Experimental syscall sandbox enabled (-sandbox=log-and-abort): bitcoind will terminate if an unexpected (not allowlisted) syscall is invoked.
  …
  2021-06-09T12:34:56Z Syscall filter installed for thread "addcon"
  2021-06-09T12:34:56Z Syscall filter installed for thread "dnsseed"
  2021-06-09T12:34:56Z Syscall filter installed for thread "net"
  2021-06-09T12:34:56Z Syscall filter installed for thread "msghand"
  2021-06-09T12:34:56Z Syscall filter installed for thread "opencon"
  2021-06-09T12:34:56Z Syscall filter installed for thread "init"
  …
  # A simulated execve call to show the sandbox in action:
  2021-06-09T12:34:56Z ERROR: The syscall "execve" (syscall number 59) is not allowed by the syscall sandbox in thread "msghand". Please report.
  …
  Aborted (core dumped)
  $
  ```

  ---

  [About seccomp and seccomp-bpf](https://en.wikipedia.org/wiki/Seccomp):

  > In computer security, seccomp (short for secure computing mode) is a facility in the Linux kernel. seccomp allows a process to make a one-way transition into a "secure" state where it cannot make any system calls except exit(), sigreturn(), and read() and write() to already-open file descriptors. Should it attempt any other system calls, the kernel will terminate the process with SIGKILL or SIGSYS. In this sense, it does not virtualize the system's resources but isolates the process from them entirely.
  >
  > […]
  >
  > seccomp-bpf is an extension to seccomp that allows filtering of system calls using a configurable policy implemented using Berkeley Packet Filter rules. It is used by OpenSSH and vsftpd as well as the Google Chrome/Chromium web browsers on Chrome OS and Linux. (In this regard seccomp-bpf achieves similar functionality, but with more flexibility and higher performance, to the older systrace—which seems to be no longer supported for Linux.)

ACKs for top commit:
  laanwj:
    Code review and lightly tested ACK 4747da3a5b

Tree-SHA512: e1c28e323eb4409a46157b7cc0fc29a057ba58d1ee2de268962e2ade28ebd4421b5c2536c64a3af6e9bd3f54016600fec88d016adb49864b63edea51ad838e17
2021-10-04 22:45:43 +02:00
MarcoFalke
c6f710ec98
Merge bitcoin/bitcoin#23052: test: use miniwallet in test_rpc function in feature_rbf.py
74c0d81b46 test: use miniwallet in test_rpc() function in feature_rbf.py (Shubhankar)

Pull request description:

  This PR aims to further increase MiniWallet usage in the functional test feature_rbf.py by using it in the test_rpc(...) . Sub test needs wallet to get a utxo for creating raw txn, and to get address for output, address can be hardcoded and mini wallet can be used for utxo. fund raw transaction is a wallet rpc and should be tested only when bitcoin core is compiled with wallet

ACKs for top commit:
  laanwj:
    Code review ACK 74c0d81b46
  theStack:
    tACK 74c0d81b46

Tree-SHA512: a98cc0df0fe70ee113a293cd4fd659c5bece47c17937d368193b65b150b331a2c694cdeb2792386f1a6deefb8d6750ed83ffa0ae6f9d13fa2b7625cd80bc692a
2021-10-04 11:11:45 +02:00
Samuel Dobson
573b4621cc
Merge bitcoin/bitcoin#17211: Allow fundrawtransaction and walletcreatefundedpsbt to take external inputs
928af61cdb allow send rpc take external inputs and solving data (Andrew Chow)
e39b5a5e7a Tests for funding with external inputs (Andrew Chow)
38f5642ccc allow fundtx rpcs to work with external inputs (Andrew Chow)
d5cfb864ae Allow Coin Selection be able to take external inputs (Andrew Chow)
a00eb388e8 Allow CInputCoin to also be constructed with COutPoint and CTxOut (Andrew Chow)

Pull request description:

  Currently `fundrawtransaction` and `walletcreatefundedpsbt` both do not allow external inputs as the wallet does not have the information necessary to estimate their fees.

  This PR adds an additional argument to both those RPCs which allows the user to specify solving data. This way, the wallet can use that solving data to estimate the size of those inputs. The solving data can be public keys, scripts, or descriptors.

ACKs for top commit:
  prayank23:
    reACK 928af61cdb
  meshcollider:
    Re-utACK 928af61cdb
  instagibbs:
    crACK 928af61cdb
  yanmaani:
    utACK 928af61.

Tree-SHA512: bc7a6ef8961a7f4971ea5985d75e2d6dc50c2a90b44c664a1c4b0f1be5c1c97823516358fdaab35771a4701dbefc0862127b1d0d4bfd02b4f20d2befa4434700
2021-10-04 22:08:46 +13:00
katesalazar
2198f79e87 Add an argparse abbreviated mode to --failfast
Short options should only be a single character. If not, they can't be
concatenated in a single "-word" (from review by luke-jr).

F is chosen instead of f, because f could be reserved to the nested
wallet_hd.py (test_framework/test_framework.py) arguments parser.
2021-10-04 08:08:52 +02:00
Andrew Chow
928af61cdb allow send rpc take external inputs and solving data 2021-10-03 13:24:14 -04:00
Andrew Chow
e39b5a5e7a Tests for funding with external inputs 2021-10-03 13:24:14 -04:00
Jon Atack
502f50da12
Test transactions conflicted by double spend in listtransactions
Test the properties of transactions conflicted by a double
spend as returned by RPC listtransactions in the "abandoned",
"confirmations", "trusted" and "walletconflicts" fields.

These fields are also returned by RPCs listsinceblock and
gettransactions.
2021-10-02 15:27:40 +02:00
practicalswift
4747da3a5b Add syscall sandboxing (seccomp-bpf) 2021-10-01 13:51:10 +00:00
brunoerg
bda620aecd test: check abandoned tx in listsinceblock 2021-10-01 09:51:14 -03:00
MarcoFalke
35a31d5f7e
Merge bitcoin/bitcoin#23136: test: update fee rate assertion helper in the functional test framework
b658d7d5c5 test: update assert_fee_amount() in test_framework/util.py (Jon Atack)

Pull request description:

  Follow-up to 42e1b5d979 (#12486).
  - update call to `round()` with our utility function `satoshi_round()` to avoid intermittent test failures
  - rename `fee_per_kB` to `feerate_BTC_kvB` for precision
  - store division result in `feerate_BTC_vB`

  Possibly resolves #19418.

ACKs for top commit:
  meshcollider:
    utACK b658d7d5c5

Tree-SHA512: f124ded98c913f98782dc047a85a05d3fdf5f0585041fa81129be562138f6261ec1bd9ee2af89729028277e75b591b0a7ad50244016c2b2fa935c6e400523183
2021-10-01 10:54:23 +02:00
MarcoFalke
4e1de1fc59
Merge bitcoin/bitcoin#22340: p2p: Use legacy relaying to download blocks in blocks-only mode
18c5b23a0f [test] Test that -blocksonly nodes still serve compact blocks. (Niklas Gögge)
a79ad65fc2 [test] Test that getdata(CMPCT) is still sent on regular low bandwidth connections. (Niklas Gögge)
5e231c116b [test] Test that -blocksonly nodes do not send getdata(CMPCT) on a low bandwidth connection. (Niklas Gögge)
5bf6587457 [test] Test that -blocksonly nodes do not request high bandwidth mode. (Niklas Gögge)
0dc8bf5b92 [net processing] Dont request compact blocks in blocks-only mode (Niklas Gögge)

Pull request description:

  A blocks-only node does not participate in transaction relay to reduce its own bandwidth usage and therefore does not have a mempool. The use of compact blocks is not beneficial to such a node since it will always have to download full blocks.

  In both high- and low-bandwidth relaying the `cmpctblock` message is sent. This represent a bandwidth overhead for blocks-only nodes because the `cmpctblock` message is several times larger in the average case than the equivalent `headers` or `inv` announcement.

  ![compact blocks](https://raw.githubusercontent.com/bitcoin/bips/master/bip-0152/protocol-flow.png)

  >**Example:**
  >A block with 2000 txs results in a `cmpctblock` with 2000*6 bytes in short ids. This is several times larger than the equivalent 82 bytes for a `headers` message or 37 bytes for an `inv`.

  ## Approach

  This PR makes blocks-only nodes always use the legacy relaying to download new blocks.
  It does so by making blocks-only nodes never initiate a high-bandwidth block relay connection by disabling the sending of `sendcmpct(1)`. Additionally a blocks-only node will never request a compact block using `getdata(CMPCT)`.

  A blocks-only node will continue to serve compact blocks to its peers in both high- and low-bandwidth mode.

ACKs for top commit:
  naumenkogs:
    ACK 18c5b23a0f
  rajarshimaitra:
    tACK 18c5b23a0f
  jnewbery:
    reACK 18c5b23a0f
  theStack:
    re-ACK 18c5b23a0f 🥛

Tree-SHA512: 0c78804aa397513d41f97fe314efb815efcd852d452dd903df9d4749280cd3faaa010fa9b51d7d5168b8a77e08c8ab0a491ecdbdb3202f2e9cd5137cddc74624
2021-10-01 08:16:54 +02:00
W. J. van der Laan
571bb94dfb
Merge bitcoin/bitcoin#23123: Remove -rescan startup parameter
dc3ec74d67 Add rescan removal release note (Samuel Dobson)
bccd1d942d Remove -rescan startup parameter (Samuel Dobson)
f963b0fa8c Corrupt wallet tx shouldn't trigger rescan of all wallets (Samuel Dobson)
6c006495ef Remove outdated dummy wallet -salvagewallet arg (Samuel Dobson)

Pull request description:

  Remove the `-rescan` startup parameter.

  Rescans can be run with the `rescanblockchain` RPC.

  Rescans are still done on wallet-load if needed due to corruption, for example.

ACKs for top commit:
  achow101:
    ACK dc3ec74d67
  laanwj:
    re-ACK dc3ec74d67

Tree-SHA512: 608360d0e7d73737fd3ef408b01b33d97a75eebccd70c6d1b47a32fecb99b9105b520b111b225beb10611c09aa840a2b6d2b6e6e54be5d0362829e757289de5c
2021-09-30 20:49:40 +02:00
Jon Atack
b658d7d5c5
test: update assert_fee_amount() in test_framework/util.py
- update call to round() with satoshi_round() to avoid intermittent test failures
- rename fee_per_kB to feerate_BTC_kvB for precision
- store division result in feerate_BTC_vB
2021-09-30 16:38:55 +02:00
W. J. van der Laan
1cf7fb9fd6
Merge bitcoin/bitcoin#23104: log: Avoid breaking single log lines over multiple lines in the log file
2222c04e1b log: Adjust coin selection log string (MarcoFalke)
fa6c1e850f test: Fix typos in tests (MarcoFalke)
faeae2980f log: Avoid broken DEBUG_LOCKORDER log (MarcoFalke)
faffaa85cd log: Avoid broken SELECTCOINS log (MarcoFalke)

Pull request description:

  Follow up to commit d8b4b3077f

ACKs for top commit:
  laanwj:
    re-ACK 2222c04e1b
  practicalswift:
    cr ACK 2222c04e1b

Tree-SHA512: e0daf76815a1b7c4898ceffedeaf7ede093223abf709874f9a0d78c8e41551c14e8b56d055c8fdf06ec698df64e67dfc168bbd8716131b23648d1d1294fa6636
2021-09-30 14:42:11 +02:00
Samuel Dobson
bccd1d942d Remove -rescan startup parameter 2021-09-30 12:06:27 +13:00
Jon Atack
296cfa312f
test: add listtransactions/listsinceblock "trusted" coverage 2021-09-29 22:36:57 +02:00
Jon Atack
d95913fc43
rpc: fix "trusted" description in TransactionDescriptionString
The helps for RPCs gettransaction, listtransactions, and
listsinceblock returned by TransactionDescriptionString()
state that the "trusted" boolean field is only present if the
transaction is trusted and safe to spend from.

The "trusted" boolean field is in fact returned by
WalletTxToJSON() when the transaction has 0 confirmations,
or negative confirmations, if conflicted, and it can be
true or false.

This commit updates TransactionDescriptionString() to a
more accurate description for "trusted" and updates the
existing line of test coverage to fail more helpfully.
2021-09-29 22:35:43 +02:00
MarcoFalke
fa6c1e850f
test: Fix typos in tests 2021-09-29 18:47:45 +02:00
Antoine Poinsot
4556406562
qa: test fee estimation with replacement transactions
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-09-29 17:24:28 +02:00
Antoine Poinsot
053415b297
qa: split run_test into smaller parts
Let's not have run_test get into a giant function as we add more tests

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-09-29 17:15:18 +02:00
Shubhankar
74c0d81b46 test: use miniwallet in test_rpc() function in feature_rbf.py 2021-09-29 20:43:45 +05:30
Sebastian Falbesoner
429b49378e test: introduce script_util helper for creating P2PK scripts 2021-09-29 14:09:14 +02:00
MarcoFalke
d648bbb0a7
Merge bitcoin/bitcoin#23124: Fix feature_segwit.py failure due to witness
b207971465 Fix feature_segwit failure due to witness (Samuel Dobson)

Pull request description:

  Fixes #23116

  The failure is due to sometimes spending segwit outputs, which add an additional 1 sigop in the witness, added to the 2 (*4) in the outputs.

ACKs for top commit:
  brunoerg:
    tACK b207971465

Tree-SHA512: 7e3c09d162e6941a3028514e95ae3aab2a3f858e2f056229c5faff5e37ece897dc63e3f8c0e07fad9fc4621b74a6dee5ff777e910b2b009a41bb6e72a09217bd
2021-09-29 13:36:19 +02:00
MarcoFalke
33e31f8df9
Merge bitcoin/bitcoin#23079: test: use MiniWallet for p2p_filter.py
cfdb6baa22 test: use MiniWallet for p2p_filter.py (Sebastian Falbesoner)
6fc2cd3f09 test: introduce helper to create random P2WPKH scriptPubKeys (Sebastian Falbesoner)
aa26797f69 test: MiniWallet: add `send_to` method to create arbitrary txouts (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (p2p_filter.py) to be run even with the Bitcoin Core wallet disabled by using the MiniWallet instead, as proposed in #20078.
  For this purpose, a MiniWallet method `send_to` is introduced first, which allows to create arbitrary outputs (scriptPubKey/amount). Note that the implementation for this is already present in feature_rbf.py (recently added in PR #22998), i.e. it is simply moved to the MiniWallet interface.

ACKs for top commit:
  laanwj:
    Code review ACK cfdb6baa22

Tree-SHA512: 13b063631f0d7af065b7757cfe8b47c9be6cb9850ac5db2968a2bba4f5a18cdc9f89173a9b03971545356225082042f5fdbe49d3036027d18e8b7eb042d04f5e
2021-09-29 09:13:07 +02:00
MarcoFalke
7d4ea7c7ef
Merge bitcoin/bitcoin#23120: test: Remove unused and confusing main parameter from script_util
fa54efda9b test: pep-8 touched test (MarcoFalke)
fa46768059 test: Remove unused and confusing main parameter from script_util (MarcoFalke)

Pull request description:

ACKs for top commit:
  fanquake:
    ACK fa54efda9b

Tree-SHA512: 124e888e16c92bb24ab4d6f68a768d295500a48f8c6c0b4f069493effcd761f80be78dd93b31edbb529ebe4c8aaca764aaff48d1e3b23659dde722981dc787a5
2021-09-29 08:08:25 +02:00
Samuel Dobson
b207971465 Fix feature_segwit failure due to witness 2021-09-29 17:41:35 +13:00
Amiti Uttarwar
dd8f7f2500 scripted-diff: Rename CAddrMan to AddrMan
-BEGIN VERIFY SCRIPT-
git grep -l CAddrMan src/ test/ | xargs sed -i 's/CAddrMan/AddrMan/g'
-END VERIFY SCRIPT-
2021-09-28 22:21:10 -04:00
Andrew Chow
a99ed89865 psbt: sign without finalizing
We don't always want to finalize after signing, so make it possible to
do that.
2021-09-28 19:13:42 -04:00
Samuel Dobson
6a5381a06b
Merge bitcoin/bitcoin#20591: wallet, bugfix: fix ComputeTimeSmart function during rescanning process.
240ea294d5 doc: update doxygen documention of ComputeTimeSmart() and AddToWalletIfInvolvingMe() regarding rescanning_old_block parameter (BitcoinTsunami)
d6eb39af21 test: add functional test to check transaction time determination during block rescanning (BitcoinTsunami)
07b44f16e7 wallet: fix ComputeTimeSmart algorithm to use blocktime during old block rescanning (BitcoinTsunami)

Pull request description:

  The function ComputeTimeSmart in wallet.cpp assume that transaction are discovered in the right order.
  Moreover the 'smarttime' determination algorithm is coded with realtime scenario in mind and not rescanning of old block.

  The functional test demonstrate that if the user import a wallet, then rescan only recent history, and then rescan the entire history, the older transaction discovered would have an incorrect time determination.
  In the context of rescanning old block, the only time value that as a meaning is the blocktime.

  That's why I've fixed the problem with a simple separation between rescanning of old block and realtime time determination. The fix is written to have no impact on every realtime scenario and only impact the behaviour during a rescanning process.
  This PR Fixes #20181.

  To be fair, I don't think that this bug could be triggered with the wallet GUI, because it always proceed with a proper rescan.
  But RPC API provide the possibility to trigger it. I've discovered it, because Specter desktop v0.10.0 was impacted. (https://github.com/cryptoadvance/specter-desktop/issues/680).

ACKs for top commit:
  jonatack:
    ACK 240ea294d5 per `git diff b92d552 240ea29`, re-verified rebase to latest master + debug build clean + new test passes on the branch and fails on master, only change since my review a few hours ago is incorporation of latest review suggestions
  meshcollider:
    re-utACK 240ea294d5

Tree-SHA512: 514b02e41d011ddfa325f5e8080b93800e1ea4ed5853fa420670a6ac700e8b463000dbea65f8ced8565cfb950c7f51b69154034dcb111e67aca3b964a0061494
2021-09-29 11:18:23 +13:00
Samuel Dobson
b55232a337
Merge bitcoin/bitcoin#22722: rpc: update estimatesmartfee to return max of CBlockPolicyEstimator::estimateSmartFee, mempoolMinFee and minRelayTxFee
ea31caf6b4 update estimatesmartfee rpc to return max of estimateSmartFee, mempoolMinFee and minRelayTxFee. (pranabp-bit)

Pull request description:

  This PR is in response to the issue [#19699](https://github.com/bitcoin/bitcoin/issues/19699).

  Based on the discussion in the comments of PR [#22673](https://github.com/bitcoin/bitcoin/pull/22673) changes have been made in the `estimatesmartfee` itself such that it takes into account `mempoolMinFee` and `relayMinFee` . Hence it provides a fee estimate that is most likely to be paid by the user in an actual transaction, preventing issues such as [#16072](https://github.com/bitcoin/bitcoin/issues/16072).

  The test file test/functional/feature_fee_estimation.py has also been updated to check this functionality.

ACKs for top commit:
  meshcollider:
    re-utACK ea31caf6b4

Tree-SHA512: 8f36153a07cbd552c5c13d11d9c6e987a7a555ea4cc83f2573c0c92dd97c706d90c30a7248671437c2f3a836d3272f8fad53d15a5fa6efaa0409ae8009b0a18d
2021-09-29 10:55:29 +13:00
Samuel Dobson
d6492d4ed0
Merge bitcoin/bitcoin#22650: Remove -deprecatedrpc=addresses flag and corresponding code/logic
43cd6b8af9 doc: add release notes for removal of the -deprecatedrpc=addresses flag (Michael Dietz)
2b1fdc2c6c refactor: minor styling, prefer snake case and same line if (Michael Dietz)
d64deac7b8 refactor: share logic between ScriptPubKeyToUniv and ScriptToUniv (Michael Dietz)
8721638daa rpc: remove deprecated addresses and reqSigs from rpc outputs (Michael Dietz)

Pull request description:

  Resolves #21797 now that we've branched-off to v23 ("addresses" and "reqSigs" deprecated) "ExtractDestinations" should be removed.

   `-deprecatedrpc=addresses` was initially added in this PR #20286 (which resolved the original issue #20102).

  Some chunks of code and logic are no longer used/necessary with the removal of this, and therefore some minor refactoring is done in this PR as well (separated commits)

ACKs for top commit:
  MarcoFalke:
    re-ACK 43cd6b8af9 🐉
  meshcollider:
    Code review ACK 43cd6b8af9
  jonatack:
    ACK 43cd6b8af9 per `git range-diff a9d0cec 92dc5e9 43cd6b8`, also rebased to latest master, debug built + quick re-review of each commit to bring back context, and ran tests locally at the final commit

Tree-SHA512: fba83495e396d3c06f0dcf49292f14f4aa6b68fa758f0503941fade1a6e7271cda8378e2734af1faea550d1b43c85a36c52ebcc9dec0732936f9233b4b97901c
2021-09-29 10:41:30 +13:00
Niklas Gögge
18c5b23a0f [test] Test that -blocksonly nodes still serve compact blocks. 2021-09-28 22:11:30 +02:00
Niklas Gögge
a79ad65fc2 [test] Test that getdata(CMPCT) is still sent on regular low bandwidth connections.
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-09-28 22:11:30 +02:00
Niklas Gögge
5e231c116b [test] Test that -blocksonly nodes do not send getdata(CMPCT) on a low bandwidth connection.
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-09-28 22:11:30 +02:00
Niklas Gögge
5bf6587457 [test] Test that -blocksonly nodes do not request high bandwidth mode.
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-09-28 22:11:30 +02:00
BitcoinTsunami
d6eb39af21 test: add functional test to check transaction time determination during block rescanning 2021-09-28 21:49:22 +02:00
MarcoFalke
fa54efda9b
test: pep-8 touched test
Can be reviewed with "--word-diff-regex=.".
2021-09-28 15:48:55 +02:00
MarcoFalke
fa46768059
test: Remove unused and confusing main parameter from script_util
Bitcoin script opcodes are equal on all chains (main and test) anyway.

Can be reviewed with "--word-diff-regex=.".
2021-09-28 15:46:57 +02:00
W. J. van der Laan
efa227f5df
Merge bitcoin/bitcoin#23097: Run specified functional tests with all matching flags
b8909b0746 Run functional tests with all possible flags (Samuel Dobson)

Pull request description:

  Functional tests which use flags like `--descriptors` or `--legacy-wallet` won't run if only the base script is given to `test_runner.py` because it doesn't match any script in the list exactly. It would be easier if it would just run both options.

  For example, instead of:
  ```
  test_runner.py 'wallet_basic.py --legacy-wallet' 'wallet_basic.py --descriptors'
  ```

  We can now just run:
  ```
  test_runner.py wallet_basic
  ```

  Also useful for `--usecli`, the IPv4/IPv6/nonloopback `rpc_bind.py` variations, etc.

ACKs for top commit:
  laanwj:
    Code review ACK b8909b0746
  MarcoFalke:
    review ACK b8909b0746

Tree-SHA512: d367037cb170e705551726d47fe4569ebc3ceadece280dd3edbb3ecb41e19f3263d6d272b407316ed6011164e850df4321fb340b1b183b34497c9f7cc439f4d8
2021-09-28 15:32:23 +02:00
pranabp-bit
ea31caf6b4 update estimatesmartfee rpc to return max of estimateSmartFee, mempoolMinFee and minRelayTxFee.
This will provide better estimates which would be closer to fee paid in actual
transactions.
The test has also been changed such that when the node is restarted with a
high mempoolMinFee, the estimatesmartfee still returns a feeRate greater
than or equal to the mempoolMinFee, minRelayTxFee.(just like the feeRate of actual transactions)
2021-09-28 18:36:38 +05:30
MarcoFalke
fa8f3ba131
test: pep-8 2021-09-28 10:52:33 +02:00
MarcoFalke
fac5708afc
test: Use assert_equal over assert for easier debugging 2021-09-28 10:48:51 +02:00
MarcoFalke
a9d0cec499
Merge bitcoin/bitcoin#23106: Ensure wallet is unlocked before signing PSBT with walletprocesspsbt and GUI
7e3ee4cdd0 GUI: Ask user to unlock wallet before signing psbt (Samuel Dobson)
0f3acecf33 Add test that walletprocesspsbt requires unlocked wallet when signing (Samuel Dobson)
0e895212bb Ensure wallet is unlocked before signing in walletprocesspsbt (Samuel Dobson)

Pull request description:

  If signing a PSBT, we need to ensure the wallet is unlocked.

  Fixes #22874, fixes bitcoin-core/gui#312

ACKs for top commit:
  achow101:
    ACK 7e3ee4cdd0
  lsilva01:
    Code Review ACK 7e3ee4cdd0
  benthecarman:
    ACK 7e3ee4cdd0

Tree-SHA512: 6726a873582747900ab454ea21153a92be86808a4c1517dc2856b389876a2da9e8df1ffa9b567b6bd017038342c3544ecf5ca3c97744e7debe0a5ee65563687d
2021-09-28 09:49:44 +02:00
Samuel Dobson
0f3acecf33 Add test that walletprocesspsbt requires unlocked wallet when signing 2021-09-28 13:27:07 +13:00
Sebastian Falbesoner
cfdb6baa22 test: use MiniWallet for p2p_filter.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-09-27 13:55:25 +02:00
Sebastian Falbesoner
6fc2cd3f09 test: introduce helper to create random P2WPKH scriptPubKeys 2021-09-27 13:55:25 +02:00
Sebastian Falbesoner
aa26797f69 test: MiniWallet: add send_to method to create arbitrary txouts
With this new method, outputs to an arbitrary scriptPubKey/amount can
be created. Note that the implementation was already present in the
test feature_rbf.py and is just moved to the MiniWallet interface, in
order to enable other tests to also use it.
2021-09-27 13:55:25 +02:00
merge-script
632be5514c
Merge bitcoin/bitcoin#23061: Fix (inverse) meaning of -persistmempool
faa9c19a4b doc: Add 23061 release notes (MarcoFalke)
faff17bbde Fix (inverse) meaning of -persistmempool (MarcoFalke)

Pull request description:

  Passing `-persistmempool` is currently treated as `-nopersistmempool`

ACKs for top commit:
  jnewbery:
    reACK faa9c19a4b
  hebasto:
    ACK faa9c19a4b, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f34a89a07745dabe340eb845b2a348b79c093e9056f7a21c17e1ba2e278177c9b4cf30e8095791fd645a7f90eb34850b2eee0c869b4f6ec02bf749c73b0e52ee
2021-09-27 10:12:14 +02:00
merge-script
58c25bdcea
Merge bitcoin/bitcoin#23092: test: Remove Windows workaround in authproxy
fad02274ba test: Remove Windows workaround in authproxy (MarcoFalke)

Pull request description:

  Might no longer be needed after https://github.com/bitcoin/bitcoin/pull/23089

ACKs for top commit:
  hebasto:
    ACK fad02274ba, passed 3 consequential runs on my [personal CI](https://cirrus-ci.com/task/4897456077930496):

Tree-SHA512: 3c408e068ad7590dc0e5922c0b4cd3bf927e22fe624b13b8ab38db848342d310c9c934f358638fd5234c7b5f10f95d3bddc676deb925dcbba54945e2e8229275
2021-09-27 09:26:21 +02:00
merge-script
ba3f0b2c48
Merge bitcoin/bitcoin#23084: test: avoid non-determinism in asmap-addrman test
5825b34783 test: avoid non-determinism in asmap-addrman test (Jon Atack)

Pull request description:

  This is the same approach as for the addpeeraddress test in `test/functional/rpc_net.py` in commit 869f1368.

  The probability of collision when adding an addrman entry is expected to be 1/2^16 = 1/65536 for an address from a different /16.  This change hopes to avoid these collisions by adding 1 tried entry before adding 1 new table one, instead of 2 tried entries followed by 2 new entries, which appears to have caused a collision in the CI.

  To verify the regression test still fails when expected:

  - `git checkout 181a120 && git cherry-pick ef242f5`
  - recompile bitcoind
  - git checkout this branch and run `test/functional/feature_asmap.py`. Expected output:

  ```
  AssertionError: Unexpected stderr bitcoind: ./addrman.h:739: void CAddrMan::Check() const: Assertion `false' failed. !=
  ```

  Closes #23078.

  Co-authored-by: Martin Zumsande <mzumsande@gmail.com>

ACKs for top commit:
  mzumsande:
    Re-ACK 5825b34783

Tree-SHA512: eb6624a196fa5617c84aad860c8f91e8a8f60fc9fe2d7168facc298ee38db5e93b7e988e11c2ac1b399dc2c6b8fad360fb10bdbf10e598a1878300388475a200
2021-09-27 09:20:25 +02:00
W. J. van der Laan
09cb5ec6c8
Merge bitcoin/bitcoin#23065: Allow UTXO locks to be written to wallet DB
d96b000e94 Make GUI UTXO lock/unlock persistent (Samuel Dobson)
077154fe69 Add release note for lockunspent change (Samuel Dobson)
719ae927dc Update lockunspent tests for lock persistence (Samuel Dobson)
f13fc16295 Allow lockunspent to store the lock in the wallet DB (Samuel Dobson)
c52789365e Allow locked UTXOs to be store in the wallet database (Samuel Dobson)

Pull request description:

  Addresses and closes #22368

  As per that issue (and its predecessor #14907), there seems to be some interest in allowing unspent outputs to be locked persistently. This PR does so by adding a flag to lockunspent to store the change in the wallet database. Defaults to false, so there is no change in default behaviour.

  Edit: GUI commit changes default behaviour. UTXOs locked/unlocked via the GUI are now persistent.

ACKs for top commit:
  achow101:
    ACK d96b000e94
  kristapsk:
    ACK d96b000e94
  lsilva01:
    Tested ACK d96b000e94 on Ubuntu 20.04
  prayank23:
    ACK d96b000e94

Tree-SHA512: 957a5bbfe7f763036796906ccb1598feb6c14c5975838be1ba24a198840bf59e83233165cb112cebae909b6b25bf27275a4d7fa425923ef6c788ff671d7a89a8
2021-09-26 11:30:18 +02:00
Samuel Dobson
f9603ee4e0 Add test for flushing keypool with newkeypool 2021-09-26 15:35:54 +13:00
Samuel Dobson
6f6f7bb36c Make legacy wallet upgrades from non-HD to HD always flush the keypool 2021-09-26 15:35:54 +13:00
Samuel Dobson
b8909b0746 Run functional tests with all possible flags 2021-09-26 13:58:19 +13:00
Samuel Dobson
719ae927dc Update lockunspent tests for lock persistence 2021-09-25 23:50:06 +12:00
merge-script
16ccb3a1cd
Merge bitcoin/bitcoin#23086: test: Add -testactivationheight tests to rpc_blockchain
fa4ca8d579 test: Add -testactivationheight tests to rpc_blockchain (MarcoFalke)

Pull request description:

  Suggested: https://github.com/bitcoin/bitcoin/pull/22818#discussion_r712513991

ACKs for top commit:
  laanwj:
    Code review ACK fa4ca8d579
  theStack:
    Concept and code-review ACK fa4ca8d579

Tree-SHA512: 41304db1a15c0c705a9cc2808c9f1d7831a321a8a7948a28ec5d3ee1ed3da6a0ce67cd50c99a33aaed86830c59608eb6ffadbeaba67d95245c490f9b6c277912
2021-09-25 09:39:42 +02:00
MarcoFalke
fad02274ba
test: Remove Windows workaround in authproxy
This reverts commit fab9899204.
2021-09-25 09:33:16 +02:00
merge-script
442e32e117
Merge bitcoin/bitcoin#22817: test: Avoid race after connect_nodes
fa04f26aa7 test: Avoid race after connect_nodes (MarcoFalke)

Pull request description:

  Wait until the connection is fully established on both sides (verack). Fixes #22714

ACKs for top commit:
  kiminuo:
    utACK fa04f26aa7

Tree-SHA512: bc2c44b44b688086ff84046924cf5251dd625584e93ce8fa17de27023855b32f3bb55109b846abbcec775e2836c7f3c5a81d6b4aff7c4ac065b9aefa044c1883
2021-09-25 08:50:52 +02:00
Jon Atack
5825b34783
test: avoid non-determinism in asmap-addrman test
This is the same approach as for the addpeeraddress test in
`test/functional/rpc_net.py` in commit 869f1368.

The probability of collision when adding an addrman entry is
expected to be 1/2^16 = 1/65536 for an address from a different /16.

This change hopes to avoid these collisions by adding 1 tried entry
before adding 1 new table one, instead of 2 tried entries followed
by 2 new entries, which appears to have caused a collision in the CI.

To verify the regression test stills fails when expected:

- git checkout 181a120 && git cherry-pick ef242f5
- recompile bitcoind
- git checkout this branch and run test/functional/feature_asmap.py. Expected output:

```
AssertionError: Unexpected stderr bitcoind: ./addrman.h:739: void CAddrMan::Check() const: Assertion `false' failed. !=
```

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2021-09-24 22:00:24 +02:00
Michael Dietz
8721638daa
rpc: remove deprecated addresses and reqSigs from rpc outputs 2021-09-24 14:22:49 -05:00
Sebastian Falbesoner
45827fd718 test: check for block reject reasons in p2p_segwit.py [2/2]
This commit adds specific expected reject reasons for segwit blocks
sent to the node, that are only showing up if one script threads
is used. For this reason, the node is started with the parameter
`-par=1`.
2021-09-24 17:38:03 +02:00
Sebastian Falbesoner
4eb532ff8b test: check for block reject reasons in p2p_segwit.py [1/2]
This commit adds specific expected reject reasons for segwit blocks
sent to the node, that are independent of whether multiple script threads
are activated or not.
2021-09-24 17:36:46 +02:00
Sebastian Falbesoner
b1488c4dce test: fix reference to block processing test in p2p_segwit.py
The block test was renamed from `p2p-fullblocks.py` to
`feature_block.py` in commit ca6523d0c8 (PR #11774).
2021-09-24 17:36:45 +02:00
W. J. van der Laan
01b5cfb951
Merge bitcoin/bitcoin#23047: test: Use MiniWallet in mempool_persist
faae0988d6 test: Check other fields are loaded correctly as well (MarcoFalke)
fa4db92617 test: Remove unused self.connect_nodes (MarcoFalke)
fafb7b7a89 test: pep8 (MarcoFalke)
fa32cb2467 test: Use MiniWallet in mempool_persist (MarcoFalke)
faca688a85 test: Add MiniWallet get_descriptor function (MarcoFalke)

Pull request description:

ACKs for top commit:
  laanwj:
    Code review ACK faae0988d6

Tree-SHA512: 6124f16ee1f3f416c50dc07aebe8846ff7e2b7c8e5dd84f9517cb5f1df021b9e57ed7c7e17bc099a37c663cd93f6d417c5e0622c0b359956403d53e705eb5549
2021-09-24 17:09:44 +02:00
MarcoFalke
fa4ca8d579
test: Add -testactivationheight tests to rpc_blockchain 2021-09-24 15:32:00 +02:00
merge-script
8e9801bfc4
Merge bitcoin/bitcoin#22818: test: Activate all regtest softforks at height 1, unless overridden
fa4db8671b test: Activate all regtest softforks at height 1, unless overridden (MarcoFalke)
faad1e5ffd Introduce -testactivationheight=name@height setting (MarcoFalke)
fadb2ef2fa test: Add extra_args argument to TestChain100Setup constructor (MarcoFalke)
faa46986aa test: Remove version argument from build_next_block in p2p_segwit test (MarcoFalke)
fa086ef539 test: Remove unused ~TestChain100Setup (MarcoFalke)

Pull request description:

  All softforks that are active at the tip of mainnet, should also be active from genesis in regtest. Otherwise their rules might not be enforced in user testing, thus making their testing less useful.

  To still allow tests to check pre-softfork rules, a runtime argument can change the activation height.

ACKs for top commit:
  laanwj:
    Code review ACK fa4db8671b
  theStack:
    re-ACK fa4db8671b

Tree-SHA512: 6397d46ff56ebc48c007a4cda633904d6ac085bc76b4ecf83097c546c7eec93ac0c44b88083b2611b9091c8d1fb8ee1e314065de078ef15e922c015de7ade8bf
2021-09-24 14:04:51 +02:00
fanquake
86c3b84388
Merge bitcoin/bitcoin#23036: test: use test_framework.p2p P2P_SERVICES constant in functional tests
b69a106bcd test: use test_framework.p2p P2P_SERVICES in functional tests (Jon Atack)

Pull request description:

  `P2P_SERVICES` is defined in `test/functional/test_framework/p2p.py`, so we can use it as a single definition for our functional tests. It may also be a tiny bit more efficient to use the constant rather than calculating `NODE_NETWORK | NODE_WITNESS` every time we need it in the tests.

ACKs for top commit:
  laanwj:
    Code review ACK b69a106bcd
  klementtan:
    crACK b69a106bcd
  fanquake:
    ACK b69a106bcd - didn't look at the formatting changes.

Tree-SHA512: f83e593663a69182986325d9ba2b4b787b87896d6648973f4f802f191a2573201b9e7d7e10e69662ef1965fa63268845726ed1aa5742a2e38dcccf4aebc6a961
2021-09-23 17:13:02 +08:00
John Newbery
eaf6be0114 [net processing] Do not request transaction relay from feeler connections
Add a test to verify that feeler connections do not request transaction relay.
2021-09-22 16:12:16 +01:00
John Newbery
0220b834b1 [test] Add testing for outbound feeler connections
Extend the addconnection RPC method to allow creating outbound
feeler connections. Extend the test framework to accept those
feeler connections.
2021-09-22 16:12:14 +01:00
MarcoFalke
faff17bbde
Fix (inverse) meaning of -persistmempool 2021-09-22 11:29:44 +02:00
lsilva01
c2fbdca549 Add BECH32_INVALID_VERSION test 2021-09-21 19:25:21 -03:00
lsilva01
b142f79ddb skip test_getaddressinfo() if wallet is disabled 2021-09-21 19:25:21 -03:00
merge-script
a8a272ac32
Merge bitcoin/bitcoin#22734: addrman: Avoid crash on corrupt data, Force Check after deserialize
fa3669f72f fuzz: Move all addrman fuzz targets to one file (MarcoFalke)
fa7a883f5a addrman: Replace assert with throw on corrupt data (MarcoFalke)
fa298971e6 Refactor: Turn the internal addrman check helper into a forced check (MarcoFalke)
fae5c633dc move-only: Move CAddrMan::Check to cpp file (MarcoFalke)

Pull request description:

  Assert should only be used for program internal logic errors, not to sanitize external user input.

  The assert was introduced via the debug-only runtime option `-checkaddrman` in commit 803ef70fd9, thus won't need a backport.

  Also, it doesn't really make sense to continue when the deserialized addrman doesn't pass the sanity check.

  For example, if `nLastSuccess` is negative, it would  later result in integer overflows. Thus, this patch fixes #22931.

  Also,
  Fixes #22503
  Fixes #22504
  Fixes #22519

  Closes #22498

  Steps to test:

  ```
  mkdir -p /tmp/test_235/regtest/
  echo 'H4sIAAAAAAAAA/u1f+stZmUGYgELgwPRakfBKBgFo2AUjIJRMApGwSgYBaNgFIyCUTBswdyGpFnLjUKjP9e0bvjYusl6b+L2e7Vs2dd6N//Pua0/xQUALJAn93IQAAA=' | base64 --decode | zcat > /tmp/test_235/regtest/peers.dat
  ./src/qt/bitcoin-qt -regtest -datadir=/tmp/test_235/ -checkaddrman=1 -printtoconsole | grep -A2 'Loading P2P addresses'
  ```

  Output before:
  ```
  2021-09-10T11:28:37Z init message: Loading P2P addresses…
  2021-09-10T11:28:37Z ADDRMAN CONSISTENCY CHECK FAILED!!! err=-16
  bitcoin-qt: addrman.cpp:765: void CAddrMan::Check() const: Assertion `false' failed.

  (program crashes)
  ```

  Output after:
  ```
  2021-09-10T11:26:00Z init message: Loading P2P addresses…
  2021-09-10T11:26:00Z Error: Invalid or corrupt peers.dat (Corrupt data. Consistency check failed with code -16: iostream error). If you believe this is a bug, please report it to https://github.com/bitcoin/bitcoin/issues. As a workaround, you can move the file ("/tmp/test_235/regtest/peers.dat") out of the way (rename, move, or delete) to have a new one created on the next start.

  (program exits)
  ```

ACKs for top commit:
  naumenkogs:
    ACK fa3669f72f
  jnewbery:
    Code review ACK fa3669f72f
  vasild:
    ACK fa3669f72f

Tree-SHA512: 687e4a4765bbc66495152fa7a49d28ee84b405dc5370ba87b4016b5593e45f54c4ce5cae579e4d433e0e082d20fc263969fa602679c911accef0adb2d6213bd6
2021-09-21 18:21:00 +02:00
merge-script
ae674a0198
Merge bitcoin/bitcoin#22998: test: use MiniWallet for make_utxo helper in feature_rbf.py
f680d27155 test: use MiniWallet for make_utxo helper in feature_rbf.py (Sebastian Falbesoner)
0f27524602 test: scale amounts in test_doublespend_tree down by factor 10 (Sebastian Falbesoner)
d1e2481274 test: scale amounts in test_doublespend_chain down by factor 10 (Sebastian Falbesoner)

Pull request description:

  This PR aims to further increase MiniWallet usage in the functional test feature_rbf.py by using it in the `make_utxo(...)` helper, which is the only part that needs a wallet for most sub-tests. In order to do that, the amounts for the utxos have to be scaled down in two sub-tests first (`test_doublespend_chain` and `test_doublespend_tree`, see first two commits), since we need amounts passed to `make_utxo` than can be funded by only one input. For creating UTXOs with a value of 50 BTC, we'd need to implement a method for consolidating multiple utxos into one first, which seems to be overkill.

  Note that after this PR's change, there is only one sub-test left (`test_rpc`) that needs the wallet compiled into bitcoind.

ACKs for top commit:
  MarcoFalke:
    review ACK f680d27155 🦐

Tree-SHA512: 46c8c245086a9e79855c4ede2f8f412333cf2658136805196b203b3567c89398d77fcb80715c0bb72fdc84331cc67544b2fdc259193a3adcb2fc36e147c26fce
2021-09-21 15:20:07 +02:00
MarcoFalke
fa7a883f5a
addrman: Replace assert with throw on corrupt data
Assert should only be used for program internal logic errors, not to
sanitize external user input.
2021-09-21 10:09:45 +02:00
merge-script
223ad2fd0d
Merge bitcoin/bitcoin#22831: test: add addpeeraddress "tried", test addrman checks on restart with asmap
cdaab90662 Add test for addrman consistency check on restart with asmap (Jon Atack)
869f136816 Add test for rpc addpeeraddress with "tried" argument (Jon Atack)
ef242f5213 Allow passing "tried" to rpc addpeeraddress to call CAddrMan::Good() (Jon Atack)

Pull request description:

  This pull adds a `tried` argument to RPC addpeeraddress and a regression test for the recent addrman/asmap changes and issue.

  PR #22697 introduced a reproducible bug in commit 181a1207 that fails addrman consistency checks and causes it to significantly lose peer entries when the `-asmap` configuration option is used.

  The issue occurs upon bitcoind restart due to an initialization order change in `src/init.cpp` in that commit, whereby CAddrman asmap is set after deserializing `peers.dat`, rather than before.

  Issue reported on the `#bitcoin-core-dev` IRC channel starting at https://www.erisian.com.au/bitcoin-core-dev/log-2021-08-23.html#l-263.

  ```
  addrman lost 22813 new and 2 tried addresses due to collisions or invalid addresses
  ADDRMAN CONSISTENCY CHECK FAILED!!! err=-17 bitcoind: ./addrman.h:707: void CAddrMan::Check() const: Assertion `false' failed. Aborted
  ```

  How to reproduce:

  - `git checkout 181a1207`, build, and launch bitcoind with the `-asmap` and `-checkaddrman=1` configuration options enabled
  - restart bitcoind
  - bitcoind aborts on the second call to the addrman consistency checks in `CAddrMan::Check()`

  How to test this pull:

  - `git checkout 181a1207`, cherry pick the first commit of this branch, build, git checkout this branch, run `test/functional/rpc_net.py`, which should pass, and then run `test/functional/feature_asmap.py`, which should fail with the following output:

  ```
  AssertionError: Unexpected stderr bitcoind: ./addrman.h:739: void CAddrMan::Check() const: Assertion `false' failed.
  ```

ACKs for top commit:
  jnewbery:
    utACK cdaab90662
  mzumsande:
    re-ACK cdaab90662 (based on code review of diff to d586817)
  vasild:
    ACK cdaab90662

Tree-SHA512: 0251a18fea629b62486fc907d7ab0e96c6df6fadb9e4d62cff018bc681afb6ac31e0e7258809c0a88f91e4a36c4fb0b16ed294ce47ef30585217de89c3342399
2021-09-21 09:34:28 +02:00
merge-script
0c1a39390f
Merge bitcoin/bitcoin#23041: test: Add addrman deserialization error tests
faa81f9486 test: Add addrman deserialization error tests (MarcoFalke)

Pull request description:

  Add missing test coverage

ACKs for top commit:
  jonatack:
    Light code review ACK faa81f9486 and ran the test

Tree-SHA512: 8b254ba912c83473125faaf7df02a33a99840b40460bdce1486991a01de9ba6371c053354318f09b69fdc18c823bca3f2f7d341db0f8950e22d8435acbaa9cf5
2021-09-21 09:28:45 +02:00
merge-script
89447a63b9
Merge bitcoin/bitcoin#23017: test: Replace MiniWallet scan_blocks with rescan_utxos
fa7e3f1fc1 test: Replace MiniWallet scan_blocks with rescan_utxos (MarcoFalke)

Pull request description:

  This avoids having to fiddle with the `start` and `num` parameters and instead use the `scantxoutset` RPC functionality via `rescan_utxos`.

ACKs for top commit:
  Shubhankar-Gambhir:
    ACK fa7e3f1, all tests were succesfull
  theStack:
    re-ACK fa7e3f1fc1

Tree-SHA512: 6f47d2acac9f180b2b0f8f04797e74ecb1fc180f6b164c67813a3a1f97acea54baed74e5e0a3512e3babf76b105c09e1ba4cad818c83c7cb2beb7377b4c96954
2021-09-21 09:22:30 +02:00
fanquake
1260b7e483
Merge bitcoin/bitcoin#23001: doc: Enable TLS in links in documentation
9bdda50151 Enable TLS in links in documentation (Jeremy Rand)

Pull request description:

  This PR enables TLS in several documentation links, which improves security.

ACKs for top commit:
  fanquake:
    ACK 9bdda50151

Tree-SHA512: 9d04d8771a9daf3c3b9914ff324e2eabfdf3ff5ae7f7dc92b84a1f3527010ceb860e73873a8f24d6051763eb472d9ea324ccbd6129a40318a520ca88c05f0586
2021-09-21 14:47:05 +08:00
W. J. van der Laan
488e745560
Merge bitcoin/bitcoin#12677: RPC: Add ancestor{count,size,fees} to listunspent output
6cb60f3e6d doc/release-notes: Add new listunspent fields (Luke Dashjr)
0be2f17ef5 QA: Add tests for listunspent ancestor{count,size,fees} to mempool_packages (Luke Dashjr)
6966e80f45 RPC: Add ancestor{count,size,fees} to listunspent output (Luke Dashjr)
3f77dfdaf0 Expose ancestorsize and ancestorfees via getTransactionAncestry (Luke Dashjr)

Pull request description:

  Requested by a user

ACKs for top commit:
  prayank23:
    reACK 6cb60f3e6d
  fjahr:
    Code review re-ACK 6cb60f3e6d
  kiminuo:
    ACK [6cb60f3](6cb60f3e6d)
  achow101:
    Code Review ACK 6cb60f3e6d
  naumenkogs:
    ACK 6cb60f3e6d
  darosior:
    utACK 6cb60f3e6d

Tree-SHA512: 5d16e5799558691e5853ab7ea2cc85514cb45da3ce69134d855c71845beef32ec6af5ab28d4462683e9800c8ea126f162773a9d3d5660edac08fd8edbfeda173
2021-09-20 19:25:43 +02:00
W. J. van der Laan
d809d8bf12
Merge bitcoin/bitcoin#22959: cli: Display all proxies in -getinfo
7c3712fa32 cli: Display all proxies in -getinfo (klementtan)

Pull request description:

  **Changes**: Display all proxies in `-getinfo`

  **Motivation**:

  * Currently `-getinfo` only return the proxy of the first network in `getnetworkinfo`.
  * This PR will display all unique proxies in `getnetworkinfo` as suggested in https://github.com/bitcoin/bitcoin/issues/17314#issue-514543978
       >List all proxies, at least if they're different from the IPv4 one

  ![image](https://user-images.githubusercontent.com/49265907/133991832-a1f38b36-2975-4ce2-a427-e4ffab23383e.png)

  **Testing**:

  You can verify this change by starting bitcoind with
  ```shell
  ./src/bitcoind -signet --proxy=127.0.0.1:9050 --i2psam=127.0.0.1:7656
  ```

  Execute `-getinfo`
  ```shell
  ./src/bitcoin-cli -signet -getinfo
  ```

ACKs for top commit:
  laanwj:
    Tested ACK 7c3712fa32
  prayank23:
    utACK 7c3712fa32

Tree-SHA512: 9eae97866220227f30ca4585f52799fa66fc1135047d869c4aabe598aee1a9414cb9e1c4a8d19165e52d65005f3c6d4bcc37463ace0ddb44389dfbcd4ca74095
2021-09-20 17:48:31 +02:00
MarcoFalke
faae0988d6
test: Check other fields are loaded correctly as well 2021-09-20 15:49:27 +02:00
MarcoFalke
fa4db92617
test: Remove unused self.connect_nodes
The nodes are stopped in the next line, no need to connect them
2021-09-20 15:49:23 +02:00
MarcoFalke
fafb7b7a89
test: pep8 2021-09-20 15:48:47 +02:00
MarcoFalke
fa32cb2467
test: Use MiniWallet in mempool_persist 2021-09-20 15:48:32 +02:00
MarcoFalke
faca688a85
test: Add MiniWallet get_descriptor function 2021-09-20 15:48:12 +02:00
klementtan
7c3712fa32
cli: Display all proxies in -getinfo 2021-09-20 18:52:04 +08:00
MarcoFalke
faa81f9486
test: Add addrman deserialization error tests 2021-09-20 09:07:44 +02:00
MarcoFalke
fa7e3f1fc1
test: Replace MiniWallet scan_blocks with rescan_utxos 2021-09-20 08:31:04 +02:00
Sebastian Falbesoner
ebe49b5b7c test: fix confusing off-by-one nValue in feature_coinstatsindex.py
Due to evil floating-point arithmetic, the creation of one of the
transaction outputs in feature_coinstatsindex.py leads to it's nValue
being off by one satoshi: the Python expression `int(21.99 * COIN)`
doesn't yield 2199000000 as expected, but 2198999999.

This makes the test more confusing than necessary (w.r.t. the expected
`gettxoutsetinfo` values), and could also cause problems if the value
is ever changed. Fix by using a `Decimal` type for specifying the
value in BTC, rather than using a bare floating-point.
2021-09-19 21:23:24 +02:00
Jon Atack
b69a106bcd
test: use test_framework.p2p P2P_SERVICES in functional tests 2021-09-19 14:20:48 +02:00
fanquake
de2af19dc8
Merge bitcoin/bitcoin#22987: qa: Fix "RuntimeError: Event loop is closed" on Windows
357f0c7233 ci: Enable more functional tests on Windows MSVC task (Hennadii Stepanov)
f55932678f qa: Fix "RuntimeError: Event loop is closed" on Windows (Hennadii Stepanov)

Pull request description:

  On master (2161a05855), running functional tests that use the P2P interface ends with an error:
  ```
  RuntimeError: Event loop is closed
  ```

  This PR fixes this bug, and enables more functional tests on Windows MSVC CI task.

  More details about bugfix:
  - [What’s New In Python 3.7](https://docs.python.org/3/whatsnew/3.7.html#asyncio)
  - https://bugs.python.org/issue33792
  - actual [change](https://docs.python.org/3.8/library/asyncio-policy.html#asyncio.WindowsSelectorEventLoopPolicy) done in Python 3.8

  Excluded tests, that are listed in the `EXCLUDE_TESTS` environment variable, need more thorough investigation to be enabled.

ACKs for top commit:
  MarcoFalke:
    review ACK 357f0c7233 🌆

Tree-SHA512: d0ba85be81d55c934959ce7402a9c726598125e9751a1de179d16759d0e8b8a915de879c3a62c12d3564c5e0d9649ebd86963744449626efaa42d9eaa99ad3d0
2021-09-18 16:49:19 +08:00
Andrew Chow
9c1052a521 wallet: Default new wallets to descriptor wallets 2021-09-17 13:32:06 -04:00
Jeremy Rand
9bdda50151
Enable TLS in links in documentation 2021-09-16 22:00:20 +00:00
Luke Dashjr
0be2f17ef5 QA: Add tests for listunspent ancestor{count,size,fees} to mempool_packages 2021-09-16 20:34:49 +00:00
MarcoFalke
fa4db8671b
test: Activate all regtest softforks at height 1, unless overridden 2021-09-16 18:53:04 +02:00
MarcoFalke
faad1e5ffd
Introduce -testactivationheight=name@height setting 2021-09-16 18:52:59 +02:00
MarcoFalke
faa46986aa
test: Remove version argument from build_next_block in p2p_segwit test
The block version does not have any effect on the segwit consensus rules
or block relay logic.

Same for feature_dersig.
2021-09-16 18:52:19 +02:00
Sebastian Falbesoner
f680d27155 test: use MiniWallet for make_utxo helper in feature_rbf.py 2021-09-16 16:48:55 +02:00
W. J. van der Laan
58e02395ba
Merge bitcoin/bitcoin#22955: p2p: Rename fBlocksOnly, Add test
fa66a7d732 p2p: Rename fBlocksOnly, Add test (MarcoFalke)
fac66d0a39 test: Simplify p2p_blocksonly test with new miniwallet rescan_utxos method (MarcoFalke)

Pull request description:

  `fBlocksOnly` has several issues:
  * The name is confusing
  * It is untested

  Fix both.

ACKs for top commit:
  laanwj:
    Code review ACK fa66a7d732

Tree-SHA512: 4218f455eeb37297f74603d7d44895288605844ae828a40dfb7a70215f1a058ac5ad945a22732f5ebcad3ad375d54ba360bea69ea79639a30d4c88b042448f0f
2021-09-16 16:38:14 +02:00
W. J. van der Laan
6d76b57ca0
Merge bitcoin/bitcoin#22960: test: Set peertimeout in write_config
fad4f44645 test: Set peertimeout in write_config (MarcoFalke)

Pull request description:

  This avoids having to remember to set it whenever mocktime is used with
  peer connections. Also, it might help avoiding disconnects when
  attaching a debugger to a running test.

ACKs for top commit:
  laanwj:
    Concept and code review ACK fad4f44645

Tree-SHA512: 00c742571c0524c1b3f55e0217433ef7aa2dccccc12650caab98b4cf9231669f37fc589c7475f28d5725ffe2436c76205920eaece4a47fd27dc8872421a48e5c
2021-09-16 16:00:41 +02:00
Sebastian Falbesoner
0f27524602 test: scale amounts in test_doublespend_tree down by factor 10
This is done in order to prepare the make_utxo helper to use MiniWallet,
which only supports creating transactions with single inputs, i.e. we
need to create amounts small enough to be funded by coinbase transactions
(50 BTC).
2021-09-16 14:32:43 +02:00
Sebastian Falbesoner
d1e2481274 test: scale amounts in test_doublespend_chain down by factor 10
This is done in order to prepare the make_utxo helper to use MiniWallet,
which only supports creating transactions with single inputs, i.e. we
need to create amounts small enough to be funded by coinbase transactions
(50 BTC).
2021-09-16 14:22:39 +02:00
Hennadii Stepanov
f55932678f
qa: Fix "RuntimeError: Event loop is closed" on Windows 2021-09-15 20:33:28 +03:00
Jon Atack
cdaab90662
Add test for addrman consistency check on restart with asmap
PR #22697 introduced a reproducible issue in commit 181a1207 that causes the
addrman tried table to fail consistency checks and significantly lose peer
entries when the `-asmap` configuration option is used.

The issue occurs on bitcoind restart due to an initialization order change
in `src/init.cpp` in that commit whereby CAddrman asmap is set after
deserializing `peers.dat`, rather than before.

Issue reported on the `#bitcoin-core-dev` IRC channel starting at
https://www.erisian.com.au/bitcoin-core-dev/log-2021-08-23.html#l-263.

```
addrman lost 22813 new and 2 tried addresses due to collisions or invalid addresses
ADDRMAN CONSISTENCY CHECK FAILED!!! err=-17
bitcoind: ./addrman.h:707: void CAddrMan::Check() const: Assertion `false' failed. Aborted
```

How to reproduce:
- `git checkout 181a1207` and recompile
- launch bitcoind with `-asmap` and `-checkaddrman=1` config options
- restart bitcoind
- bitcoind aborts on second call to `CAddrMan::Check()`

This commit adds a regression test to reproduce the case; it passes or fails
with the same error.

Co-authored-by: John Newbery <john@johnnewbery.com>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2021-09-15 16:38:03 +02:00
Jon Atack
869f136816
Add test for rpc addpeeraddress with "tried" argument
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
Co-authored-by: John Newbery <john@johnnewbery.com>
Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2021-09-15 16:25:49 +02:00
Sebastian Falbesoner
2600db6c36 test: fix misleading fee unit in mempool_limit.py
The helper `send_large_txs` in its current interface has a fee_rate
parameter, implying that it would create a transaction with exactly that
rate. Unfortunately, this fee rate is only passed to MiniWallet's
`create_self_transfer` method, which can't know that we append several
tx outputs after, increasing the tx's vsize and decreasing it's fee rate
accordingly.

In our case, the fee rate is off by several orders of magnitude, as the
tx's vsize changes changes from 96 to 67552 vbytes (>700x), i.e. the
value passed to this function is neither really a fee rate nor an
absolute fee, but something in-between, which is very confusing.

Clarify the interface by passing an absolute fee that is deducted in the end
(and verified, via testmempoolaccept) and also describe how we come up with the
value passed.
2021-09-14 15:51:21 +02:00
merge-script
2b264971ad
Merge bitcoin/bitcoin#22543: test: Use MiniWallet in mempool_limit.py
08634e82c6 fix typos in logging messages (ShubhamPalriwala)
d447ded6ba replace: self.nodes[0] with node (ShubhamPalriwala)
dddca3899c test: use MiniWallet in mempool_limit.py (ShubhamPalriwala)

Pull request description:

  This is a PR proposed in #20078

  This PR enables running another non-wallet functional test even when the wallet is disabled thanks to the MiniWallet, i.e. it can be run even when bitcoin-core is compiled with --disable-wallet.

  It also includes changes in wallet.py in the form of a new method, `create_large_transactions()` for the MiniWallet to create large transactions.

  Efforts for this feature started in #20874 but were not continued and that PR was closed hence I picked this up.

  To test this PR locally, compile and build bitcoin-core without the wallet and run:
  ```
  $ test/functional/mempool_limit.py
  ```

ACKs for top commit:
  amitiuttarwar:
    ACK 08634e8, only git changes since last push (and one new line).
  Zero-1729:
    ACK 08634e82c6 🧉

Tree-SHA512: 0f744ad26bf7a5a784aac1ed5077b59c95a36d1ff3ad0087ffd10ac8d5979f7362c63c20c2ce2bfa650fda02dfbcd60b1fceee049a2465c8d221cce51c20369f
2021-09-14 11:10:08 +02:00
ShubhamPalriwala
08634e82c6 fix typos in logging messages 2021-09-14 00:58:25 +05:30
ShubhamPalriwala
d447ded6ba replace: self.nodes[0] with node 2021-09-14 00:55:45 +05:30
ShubhamPalriwala
dddca3899c test: use MiniWallet in mempool_limit.py
Co-authored-by: ShubhamPalriwala <spalriwalau@gmail.com>
Co-authored-by: stackman27 <sishirg27@gmail.com>
Signed-off-by: ShubhamPalriwala <spalriwalau@gmail.com>
2021-09-14 00:52:11 +05:30
MarcoFalke
fad4f44645
test: Set peertimeout in write_config
This avoids having to remember to set it whenever mocktime is used with
peer connections. Also, it might help avoiding disconnects when
attaching a debugger to a running test.
2021-09-13 09:41:58 +02:00
MarcoFalke
fa66a7d732
p2p: Rename fBlocksOnly, Add test
The new name describes better what the bool does and also limits the confusion of the three different concepts:
* fBlocksOnly (This bool to skip tx invs)
* -blocksonly (A setting to ignore incoming txs)
* block-relay-only (A connection type in the block-relay-only P2P graph)
2021-09-12 12:53:50 +02:00
MarcoFalke
fac66d0a39
test: Simplify p2p_blocksonly test with new miniwallet rescan_utxos method 2021-09-12 12:13:58 +02:00
merge-script
053a5fc7d9
Merge bitcoin/bitcoin#22762: Raise InitError when peers.dat is invalid or corrupted
fa55c3dc1b Raise InitError when peers.dat is invalid or corrupted (MarcoFalke)
fa4e2ccfd8 Inline ReadPeerAddresses (MarcoFalke)
fa5aeec80c Move LoadAddrman from init to addrdb (MarcoFalke)

Pull request description:

  peers.dat is silently erased when it can not be parsed or when it appears corrupted. Fix that by notifying the user. This might help in the following examples:

  * The user provided the database, but picked the wrong one.
  * A future version of Bitcoin Core wrote the file and it can't be read.
  * The file was corrupted by a logic bug in Bitcoin Core.
  * The file was corrupted by a disk failure.

ACKs for top commit:
  jonatack:
    Code review re-ACK fa55c3dc1b per `git range-diff eb1f570 fa59c6d fa55c3` and verified the new tests fail on master, except "Check mocked addrman is valid", as expected
  prayank23:
    tACK fa55c3dc1b
  vasild:
    ACK fa55c3dc1b

Tree-SHA512: 78264a78ee570a3c3262cf9c8542b5ffaffa5f52da1eef66c8c381f346989272967cfe1769c573502d9d7d3f7ad68c3ac3b2ec734185d2e4e7595b7122b14196
2021-09-10 11:41:20 +02:00
merge-script
60881158c8
Merge bitcoin/bitcoin#22907: test: Avoid intermittent test failure in feature_csv_activation.py
fa676dbac8 test: pep-8 whitespace (MarcoFalke)
faed284eab test: Avoid intermittent test failure in feature_csv_activation.py (MarcoFalke)

Pull request description:

  Otherwise there will be disconnects if the test runs longer than the default peertimeout (60s):

  ```
   node0 2021-09-05T20:28:30.973116Z (mocktime: 2021-09-01T07:17:29Z) [net] [net.cpp:1323] [InactivityCheck] socket receive timeout: 393061s peer=0
  ```

  Fix that by skipping `InactivityCheck` via a large `-peertimeout`.

ACKs for top commit:
  fanquake:
    ACK fa676dbac8

Tree-SHA512: 061c0585a805aa2f8e55c4beedd4b8498a2951f33d60aa3632dda0a284db3a627d14a23dbd57e8a66c69a1612f39418e3a755c8ca97f6ae1105c0d70f0d1a801
2021-09-10 10:02:03 +02:00
merge-script
fac7181091
Merge bitcoin/bitcoin#22582: test: a test to check descendant limits
fa7db1cbf7 [test] checks descendants limtis for second generation Package descendants (ritickgoenka)

Pull request description:

  This PR adds a new functional test to test the new descendant limits for packages that were proposed in #21800.
   ```
  +----------------------+
  |                      |
  |         M1           |
  |        ^  ^          |
  |       M2   ^         |
  |      .      ^        |
  |     .        ^       |
  |    .          ^      |
  |   .            ^     |
  |  M24            ^    |
  |                  ^   |
  |                  P1  |
  |                  ^   |
  |                  P2  |
  |                      |
  +----------------------+
  ```

  This test is for checking a transaction to fail its descendant count limits because of a combination of mempool descendants, package direct descendants, and package indirect descendants.

  In this test,  P1 has M1 as a mempool ancestor, P2 has no in-mempool ancestors, but when combined P2 has M1 as an ancestor and M1 exceeds descendant_limits (23 in-mempool descendants + 2 in-package descendants, a total of 26 including itself)

ACKs for top commit:
  ryanofsky:
    Code review ACK fa7db1cbf7. Only were suggested changes since last review: simplifying test and dropping P3 transaction as John suggested, and adding assert_equal I suggested
  glozow:
    ACK fa7db1cbf7
  jnewbery:
    ACK fa7db1cbf7

Tree-SHA512: d1eb993550ac8ce31cbe42e17c6522a213ede66970d5d9391f31a116477ab5889fefa6ff2df6ceadd63a28c1be1ad893b0e8e449247e9ade2ca61dc636508d68
2021-09-09 15:42:12 +02:00
W. J. van der Laan
b05d3e76e7
Merge bitcoin/bitcoin#22079: zmq: Add support to listen on IPv6 addresses
e6998838e5 doc: Add IPv6 address to zmq example (nthumann)
8abe5703a9 test: Add IPv6 test to zmq (nthumann)
ded449b726 zmq: Enable IPv6 on listening socket (nthumann)

Pull request description:

  This PR adds support for listening on IPv6 addresses with bitcoinds ZMQ interface, just like the RPC server.
  Currently, it is not possible to specify an IPv6 address, as the `ZMQ_IPV6` [socket option](http://api.zeromq.org/master:zmq-setsockopt#toc27) is not set and therefore the ZMQ initialization fails, if one does so. The absence of this option has also been noted [here](https://github.com/bitcoin/bitcoin/issues/15198#issuecomment-617378512).
  With this PR one can e.g. set `-zmqpubhashblock=tcp://[::1]:28333` to listen on the IPv6 loopback address.

ACKs for top commit:
  laanwj:
    Code review ACK e6998838e5
  theStack:
    Tested ACK e6998838e5 🌱

Tree-SHA512: 43c3043d8d5c79794d475926259c1be975b694db4fcc1f7750a9a28e242f0fa1b531735a63ea5777498003aa5834f6243f39742d0f3941f2f37593d0c7890700
2021-09-09 15:37:13 +02:00
merge-script
a5d00d4baf
Merge bitcoin/bitcoin#22788: scripted-diff: Use generate* from TestFramework
fa0b916971 scripted-diff: Use generate* from TestFramework (MarcoFalke)

Pull request description:

  This is needed for #22567.

  By using the newly added `generate*` member functions of the test framework, it paves the way to make it easier to implicitly call `sync_all` after block generation to avoid intermittent issues.

ACKs for top commit:
  jonatack:
    ACK fa0b916971

Tree-SHA512: e74a324b60250a87c08847cdfd7b6ce3e1d89b891659fd168f6dd7dc0aa718d0edd28285374a613f462f34f4ef8e12c90ad44fb58721c91b2ea691406ad22c2a
2021-09-09 14:02:45 +02:00
MarcoFalke
fa676dbac8
test: pep-8 whitespace 2021-09-09 13:58:46 +02:00
MarcoFalke
faed284eab
test: Avoid intermittent test failure in feature_csv_activation.py 2021-09-09 13:58:00 +02:00
MarcoFalke
fa55c3dc1b
Raise InitError when peers.dat is invalid or corrupted 2021-09-09 09:20:43 +02:00
MarcoFalke
fadecbd9a4
test: Fix tests on Windows 2021-09-08 18:58:31 +02:00
fanquake
7d7d5e8efd
Merge bitcoin/bitcoin#22879: addrman: Fix format string in deserialize error
fab0b55cf0 addrman: Fix format string in deserialize error (MarcoFalke)
facce4ca44 test: Remove useless overwrite (MarcoFalke)

Pull request description:

  The format string is evaluated differently on modern compilers (clang 10 and later, as well as gcc 10 and later).

  Work around the behaviour change in compilers by pinning the underlying type of the format arguments.

  Can be tested by observing a failing test when running against master compiled with clang 10 or gcc 10 (or later).

ACKs for top commit:
  jonatack:
    ACK fab0b55cf0 verified the test fails on master as expected only at line 61 (assertion fixed by the code change); the last two test additions pass as expected
  mzumsande:
    ACK fab0b55cf0

Tree-SHA512: 07462901435107f3bc79098fd7d06446bfe8fe065fffdd35adfcba8f1dd3c499575006557afe7bc74b79d690c5ef7b58e3e031e908161be5529cf237e3b30609
2021-09-08 15:17:42 +08:00
MarcoFalke
fa92777448
Merge bitcoin/bitcoin#22736: log, sync: change lock contention from preprocessor directive to log category
7e69873283 sync: remove DEBUG_LOCKCONTENTION preprocessor directives (Jon Atack)
9b08006bc5 log, sync: improve lock contention logging and add time duration (Jon Atack)
3f4c6b87f1 log, timer: add timing macro in usec LOG_TIME_MICROS_WITH_CATEGORY (Jon Atack)
b7a17444e0 log, sync: add LOCK logging category, apply it to lock contention (Jon Atack)

Pull request description:

  To enable lock contention logging, `DEBUG_LOCKCONTENTION` has to be defined at compilation. Once built, the logging is not limited to a category and is high frequency, verbose and in all-caps. With these factors combined, it seems likely to be rarely used.

  This patch:
  - adds a `lock` logging category
  - adds a timing macro in microseconds, `LOG_TIME_MICROS_WITH_CATEGORY`
  - updates `BCLog::LogMsg()` to omit irrelevant decimals for microseconds and skip unneeded code and math
  - improves the lock contention logging, drops the all-caps, and displays the duration in microseconds
  - removes the conditional compilation directives
  - allows lock contentions to be logged on startup with `-debug=lock` or at run time with `bitcoin-cli logging '["lock"]'`

  ```
  $ bitcoind -signet -debug=lock
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1920 started
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1920 completed (4μs)
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1302 started
  2021-09-01T12:40:01Z LockContention: cs_vNodes, net.cpp:1302 completed (4μs)
  2021-09-01T12:40:02Z LockContention: cs_vNodes, net.cpp:2242 started
  2021-09-01T12:40:02Z LockContention: cs_vNodes, net.cpp:2242 completed (20μs)
  2021-09-01T12:43:04Z LockContention: ::cs_main, validation.cpp:4980 started
  2021-09-01T12:43:04Z LockContention: ::cs_main, validation.cpp:4980 completed (3μs)

  $ bitcoin-cli -signet logging
    "lock": true,

  $ bitcoin-cli -signet logging [] '["lock"]'
    "lock": false,

  $ bitcoin-cli -signet logging '["lock"]'
    "lock": true,
  ```

  I've tested this with Clang 13 and GCC 10.2.1, on Debian, with and without `--enable-debug`.

ACKs for top commit:
  hebasto:
    re-ACK 7e69873283, added a contention duration to the log message since my [previous](https://github.com/bitcoin/bitcoin/pull/22736#pullrequestreview-743764606) review.
  theStack:
    re-ACK 7e69873283 🔏 ⏲️

Tree-SHA512: c4b5eb88d3a2c051acaa842b3055ce30efde1f114f61da6e55fcaa27476c1c33a60bc419f7f5ccda532e1bdbe70815222ec2b2b6d9226f29c8e94e598aacfee7
2021-09-06 10:31:12 +02:00
MarcoFalke
fab0b55cf0
addrman: Fix format string in deserialize error
Also add a regression test.
2021-09-05 10:26:03 +02:00
Michael Dietz
f9479e4626
test, doc: basic M-of-N multisig minor cleanup and clarifications
wallet_multisig_descriptor_psbt.py is refactored in this commit. While
behavior doesn't change we do cleanup the way wallets are accessed
throughout the test as this is done a lot for the various signers
and their multisigs. We also get rid of some shallow methods and
instead inline them for improved readability.

descriptors.md is improved to be more explicit about which wallet
(ie the signer or multisig) is required for each step.
2021-09-03 13:49:03 -05:00
MarcoFalke
facce4ca44
test: Remove useless overwrite 2021-09-03 16:14:57 +02:00
Samuel Dobson
d5d0a5c604
Merge bitcoin/bitcoin#17526: Add Single Random Draw as an additional coin selection algorithm
3633b667ff Use SelectCoinsSRD if it has less waste (Andrew Chow)
8bf789b4b4 Add SelectCoinsSRD function (Andrew Chow)
2ad3b5d2ad tests: wallet_basic lock needed unspents (Andrew Chow)
b77885f13e tests: wallet_txn explicilty specify inputs (Andrew Chow)
59ba7d2861 tests: rpc_fundrawtx better test for UTXO inclusion with include_unsafe (Andrew Chow)
a165bfbe44 tests: rpc_fundrawtx use specific inputs for unavailable change test (Andrew Chow)
df765a484d tests: rpc_fundrawtx lock to UTXO types (Andrew Chow)

Pull request description:

  To ease in the use of SRD as our fallback mechanism, this PR adds it as a secondary fallback algorithm in addition to the knapsack solver. Since #22009, the solution with the least waste will be chosen. This pattern is continued with SRD simply being another solution whose waste is compared.

ACKs for top commit:
  glozow:
    reACK 3633b66 via `git range-diff  981b9d1...3633b66`, thanks for taking the suggestions
  laanwj:
    Concept and code review ACK 3633b667ff

Tree-SHA512: 895659f553fea2230990136565bdf18b1328de8b0ce47f06b64bb4d69301f6dd68cb38debe5c24fb6de1317b735fc020a987c541f00bbea65229de47e53adf92
2021-09-03 22:45:48 +12:00
W. J. van der Laan
eabc2c482a
Merge bitcoin/bitcoin#23102: test: Add missing re.escape() to feature_addrman test
fa01f22e6e test: Add missing re.escape() to feature_addrman test (MarcoFalke)

Pull request description:

  Needed to run the test on windows

ACKs for top commit:
  laanwj:
    Code review ACK fa01f22e6e
  hebasto:
    ACK fa01f22e6e, passed 2 consequential runs in my [personal CI](https://cirrus-ci.com/task/6522304080379904).

Tree-SHA512: d7ca4fb882cc6693989ddf6fc092db3259a0619cb8f87293c588484b9c62e6755e9fb1bb2c1ab85fcc8f0349d9bc155ba515e16674c0f6f56236e7fbb14655a8
2021-09-27 14:12:16 +02:00
MarcoFalke
fa01f22e6e
test: Add missing re.escape() to feature_addrman test 2021-09-27 09:52:41 +02:00
Andrew Chow
2ad3b5d2ad tests: wallet_basic lock needed unspents
To avoid accidentally spending UTXOs that are needed later in the test,
lock those UTXOs after they're creation.
2021-09-23 13:33:25 -04:00
Andrew Chow
b77885f13e tests: wallet_txn explicilty specify inputs
Instead of relying on coin selection to deterministically choose
the correct inputs to use, just specify them explicitly and use
the raw transaction RPCs.
2021-09-23 13:33:25 -04:00
Andrew Chow
59ba7d2861 tests: rpc_fundrawtx better test for UTXO inclusion with include_unsafe
Don't assume that specific inputs are going to be used when they aren't
specified explicitly.

Also fixes a bug in the include_unsafe test where after the inputs
confirm, include_unsafe should be set to False rather than True.
2021-09-23 13:32:41 -04:00
Andrew Chow
a165bfbe44 tests: rpc_fundrawtx use specific inputs for unavailable change test
For the test that checks that there is no error when change is
unavailable but change is also not needed, use specific UTXOs so that
SRD does not cause this to fail when it chooses random inputs.
2021-09-09 15:07:43 -04:00
Andrew Chow
df765a484d tests: rpc_fundrawtx lock to UTXO types
For some of the tests within rpc_fundrawtx, there is the expectation
that two independent calls to coin selection RPCs will use the same type
of UTXO. This is not necessarily guaranteed, so to make sure it is, use
lockunspent prior to those tests.
2021-09-09 15:07:42 -04:00
Samuel Dobson
629c4ab2e3
Merge bitcoin/bitcoin#22100: refactor: Clean up new wallet spend, receive files added #21207
b11a195ef4 refactor: Detach wallet transaction methods (followup for move-only) (Russell Yanofsky)

Pull request description:

  This makes `CWallet` and `CWalletTx` methods in `spend.cpp` and `receive.cpp` files into standalone functions.

  It's a followup to [#21207 MOVEONLY: CWallet transaction code out of wallet.cpp/.h](https://github.com/bitcoin/bitcoin/pull/21207), which moved code from `wallet.cpp` to new `spend.cpp` and `receive.cpp` files.

  There are no changes in behavior. This is just making methods into functions and removing circular dependencies created by #21207. There are no comment or documentation changes, either. Removed comments from `transaction.h` are just migrated to `spend.h`, `receive.h`, and `wallet.h`.

  ---

  This commit was split off from #21206 so there are a few earlier review comments there

ACKs for top commit:
  achow101:
    ACK b11a195ef4
  Sjors:
    utACK b11a195ef4
  meshcollider:
    light ACK b11a195ef4

Tree-SHA512: 75ce818d3f03b728b14b12e2d21bd20b7be73978601989cb37ff98254393300d1bb7823281449cd3d9e40756d67d42bd9a46bbdafd2e8baa95aaf2cb1c84549f
2021-09-03 21:21:58 +12:00
Hennadii Stepanov
c0a5fceee9
test: Add test for erase orphan tx conflicted by block 2021-09-03 08:10:58 +03:00
Hennadii Stepanov
fa45bb2119
test: Add test for erase orphan tx included by block 2021-09-03 08:03:54 +03:00
Hennadii Stepanov
5c049780c8
test: Add test for erase orphan tx from peer 2021-09-03 07:53:41 +03:00
fanquake
b997dd211e
Merge bitcoin/bitcoin#22809: test: Check that non-signaling BIP125 tx can be replaced via parent
fa2e9de59f test: Check that non-signaling BIP125 tx can be replaced via parent (MarcoFalke)

Pull request description:

  While `optout_child_tx` in the `test_no_inherited_signaling` test is reported as "bip125-replaceable", it is not *directly* replaceable. For example by bumping the fee of `optout_child_tx`. However, it is still replaceable *indirectly* via it's BIP-125 signalling parent.

  Clarify this by extending the test.

ACKs for top commit:
  mjdietzx:
    Tested ACK fa2e9de59f
  josibake:
    ACK fa2e9de59f

Tree-SHA512: b3608beae743dcb6152df4d2cfe1c0af6b4404ba3837f73e1d1431bd7c637f0c7fab0379aaab2218d5cd63e71070a079c0595ec031056058e8d3c933c2bae0a9
2021-09-02 17:26:53 +08:00
MarcoFalke
fa0b916971
scripted-diff: Use generate* from TestFramework
-BEGIN VERIFY SCRIPT-
 sed --regexp-extended -i \
     's/((self\.)?(nodes\[[^]]+\]|[a-z_]*(wallet|node)[0-9a-z_]*))\.(generate(|toaddress|block|todescriptor)(\(|, ))/self.\5\1, /g' \
     $(git grep -l generate ./test | grep -v 'test_framework/' | grep -v 'feature_rbf')
-END VERIFY SCRIPT-
2021-09-02 10:34:35 +02:00
MarcoFalke
7e75400bb5
Merge bitcoin/bitcoin#22437: test, refactor: add GetTransaction() coverage, improve rpc_rawtransaction
387355bb94 test, refactor: rpc_rawtransaction PEP8 (Jon Atack)
7d5cec2e49 refactor: separate the rpc_rawtransaction tests into functions (Jon Atack)
409779df95 move-only: regroup similar rpc_rawtransaction tests together (Jon Atack)
d861040dd2 test: remove no longer needed (ASCII art) comments (Jon Atack)
14398b30d6 test: add and harmonize getrawtransaction logging (Jon Atack)
85d8869cf8 test: run 2nd getrawtransaction section with/without -txindex (Jon Atack)
0097740773 refactor: txid to constant in rpc_rawtransaction to isolate tests (Jon Atack)
8c19d1329f refactor: dedup/reorg createrawtransaction sequence number tests (Jon Atack)
7f073594c9 Test src/node/transaction::GetTransaction() without -txindex (Jon Atack)

Pull request description:

  Following up on https://github.com/bitcoin/bitcoin/pull/22383#pullrequestreview-698583510, this pull adds missing `src/node/transaction::GetTransaction()` test coverage for combinations of `-txindex` and `blockhash` and does some refactoring of the test file.

ACKs for top commit:
  mjdietzx:
    reACK 387355bb94
  josibake:
    reACK 387355bb94
  MarcoFalke:
    Approach ACK 387355bb94 🔆

Tree-SHA512: b47c4ff87d69c61434e5729c954b338bc13744eddaba0879ca9f5f42243ba2cb4640d94c5f74de9f2735a8bf5e66b3d1c3bd3b7c26cd7324da7d3270ce87c6fd
2021-09-01 18:16:20 +02:00
Russell Yanofsky
b11a195ef4 refactor: Detach wallet transaction methods (followup for move-only)
Followup to commit "MOVEONLY: CWallet transaction code out of
wallet.cpp/.h" that detaches and renames some CWalletTx methods, making
into them into standalone functions or CWallet methods instead.

There are no changes in behavior and no code changes that aren't purely
mechanical. It just gives spend and receive functions more consistent
names and removes the circular dependencies added by the earlier
MOVEONLY commit.

There are also no comment or documentation changes. Removed comments
from transaction.h are just migrated to spend.h, receive.h, and
wallet.h.
2021-09-01 02:22:58 -05:00
Samuel Dobson
70676e40d8
Merge bitcoin/bitcoin#22009: wallet: Decide which coin selection solution to use based on waste metric
86beee0579 Use waste metric for deciding which selection to use (Andrew Chow)
b3df0caf7c tests: Test GetSelectionWaste (Andrew Chow)
4f5ad43b1e Add waste metric calculation function (Andrew Chow)
935b3ddf72 scripted-diff: tests: Use KnapsackSolver directly (Andrew Chow)
6a023a6f90 tests: Add KnapsackGroupOutputs helper function (Andrew Chow)
d5069fc1aa tests: Use SelectCoinsBnB directly instead of AttemptSelection (Andrew Chow)
54de7b4746 Allow the long term feerate to be configured, default of 10 sat/vb (Andrew Chow)

Pull request description:

  Branch and Bound introduced a metric that we call waste. This metric is used as part of bounding the search tree, but it can be generalized to all coin selection solutions, including those with change. As such, this PR introduces the waste metric at a higher level so that we can run both of our coin selection algorithms (BnB and KnapsackSolver) and choose the one which has the least waste. In the event that both find a solution with the same change, we choose the one that spends more inputs.

  Also this PR sets the long term feerate to 10 sat/vb rather than using the 1008 block estimate. This allows the long term feerate to be the feerate that we switch between consolidating and optimizing for fees. This also removes a bug where the long term feerate would incorrectly be set to the fallback fee. While this doesn't matter prior to this PR, it does have an effect following this. The long term feerate can be configured by the user through a new `-consolidatefeerate` option.

ACKs for top commit:
  Xekyo:
    reACK 86beee0 via git range-diff fe47558...86beee0
  meshcollider:
    re-utACK 86beee0579

Tree-SHA512: 54b154b346538eca68ae2a3b83a033b495c1605c14f842bfc43ded2256b110983ce674c647fe753cf0305b1b178403d8d60d6d4203c7a712bec784be52e90d42
2021-09-01 16:59:13 +12:00
Jon Atack
387355bb94
test, refactor: rpc_rawtransaction PEP8 2021-08-31 21:59:51 +02:00
Jon Atack
7d5cec2e49
refactor: separate the rpc_rawtransaction tests into functions 2021-08-31 21:59:38 +02:00
Jon Atack
409779df95
move-only: regroup similar rpc_rawtransaction tests together 2021-08-31 21:57:27 +02:00
Jon Atack
d861040dd2
test: remove no longer needed (ASCII art) comments 2021-08-31 21:55:15 +02:00
Jon Atack
14398b30d6
test: add and harmonize getrawtransaction logging 2021-08-31 21:55:00 +02:00
Jon Atack
85d8869cf8
test: run 2nd getrawtransaction section with/without -txindex
(and make the 'string "Flase"' test clearer as requested by reviewers)
2021-08-31 20:51:26 +02:00
Jon Atack
0097740773
refactor: txid to constant in rpc_rawtransaction to isolate tests 2021-08-31 20:51:23 +02:00
Jon Atack
8c19d1329f
refactor: dedup/reorg createrawtransaction sequence number tests 2021-08-31 20:51:20 +02:00
Jon Atack
7f073594c9
Test src/node/transaction::GetTransaction() without -txindex 2021-08-31 20:51:17 +02:00
MarcoFalke
fa2e9de59f
test: Check that non-signaling BIP125 tx can be replaced via parent 2021-08-30 17:01:46 +02:00
Andrew Chow
86beee0579 Use waste metric for deciding which selection to use
Instead of always choosing BnB if it finds a solution, always do both
BnB and KnapsackSolver and choose the one which has the least waste.
2021-08-27 12:46:19 -04:00
MarcoFalke
fa04f26aa7
test: Avoid race after connect_nodes 2021-08-27 10:18:28 +02:00
fanquake
adccbb380b
Merge bitcoin/bitcoin#21862: test: Set regtest.BIP65Height = 111 to speed up tests
faf7e485e9 Set regtest.BIP65Height = 111 to speed up tests (MarcoFalke)

Pull request description:

  No need to waste time by forcing creation of more than 1000 blocks to get the benefits of being able to test BIP 65. Also, reducing the height makes it more likely that (third-party) tests are conforming to BIP 65, which is enforced on mainnet for all new blocks.

ACKs for top commit:
  theStack:
    re-ACK faf7e485e9 📍
  Zero-1729:
    re-ACK faf7e485e9
  kristapsk:
    ACK faf7e485e9

Tree-SHA512: 79a8263e7233838666b9b636b496a8b9eb12398c779f9434677e1d62816732c0a7c7b3e73965be1fb0038d35e05e5a90e665bd74e9610104127dfc4ea38169bf
2021-08-27 07:53:25 +08:00
ritickgoenka
fa7db1cbf7 [test] checks descendants limtis for second generation Package descendants 2021-08-26 23:41:08 +05:30
MarcoFalke
0492b56e38
Merge bitcoin/bitcoin#22738: test: fix failure in feature_nulldummy.py on single-core machines
7720d4f650 test: fix failure in feature_nulldummy.py on single-core machines (Sebastian Falbesoner)
646b3885f7 test: refactor: use named args for block_submit in feature_nulldummy.py (Sebastian Falbesoner)

Pull request description:

  On single-core machines, executing the test `feature_nulldummy.py` results in the following assertion error:

  ```
      ...
      2021-08-18T15:37:58.805000Z TestFramework (INFO): Test 4: Non-NULLDUMMY base multisig transaction is invalid after activation
      2021-08-18T15:37:58.814000Z TestFramework (ERROR): Assertion failed
      Traceback (most recent call last):
        File "[...]/test/functional/test_framework/test_framework.py", line 131, in main
          self.run_test()
        File "[...]/test/functional/feature_nulldummy.py", line 107, in run_test
          self.block_submit(self.nodes[0], [test4tx], accept=False)
        File "[...]/test/functional/feature_nulldummy.py", line 134, in block_submit
          assert_equal(None if accept else 'block-validation-failed', node.submitblock(block.serialize().hex()))
        File "[...]/test/functional/test_framework/util.py", line 49, in assert_equal
          raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
      AssertionError: not(block-validation-failed == non-mandatory-script-verify-flag (Dummy CHECKMULTISIG argument must be zero))
      2021-08-18T15:37:58.866000Z TestFramework (INFO): Stopping nodes
      ...
  ```

  There are hardly any single-core machines around anymore, but the behaviour can be reproduced on a multi-core machine by patching the function `GetNumCores()` to return 1 on the master branch and running `feature_nulldummy.py`:

  ```diff
  diff --git a/src/util/system.cpp b/src/util/system.cpp
  index 30d410381..149b512fc 100644
  --- a/src/util/system.cpp
  +++ b/src/util/system.cpp
  @@ -1338,7 +1338,7 @@ bool SetupNetworking()

   int GetNumCores()
   {
  -    return std:🧵:hardware_concurrency();
  +    return 1;
   }
  ```
  As solution, parallel script verification is disabled (`-par=1`) and the exact reject reason is checked, which also increases the precision of the test (the possibility that the block is rejected because of another unintended reason is ruled out). See also related PR #22711 which applies the same approach for the p2p segwit test. The PR also includes a refactoring commit which changes the calls to `self.block_submit()` to use named arguments and removes the default value for parameter `accept` (i.e. explicitely passing `accept=...` is mandatory), with the aim to increase the test readability.

ACKs for top commit:
  josibake:
    ACK 7720d4f650
  Saviour1001:
    Tested ACK <code>[7720d4f](7720d4f650)</code>

Tree-SHA512: 8a31ebab3e2ab38e555d7a23139b3324a134a0dedc5b879a2419348ae858323882dbbfcbbf88b68e4f8d7eea8cfe43ee19da1d0d2a36c93ae7878c4980cac31d
2021-08-26 11:12:03 +02:00
MarcoFalke
faf7e485e9
Set regtest.BIP65Height = 111 to speed up tests 2021-08-26 11:08:24 +02:00
fanquake
f046131481
Merge bitcoin/bitcoin#22797: test, doc: refer to the correct variable names in p2p_invalid_tx.py
0d9fdd329e test, doc: refer to the correct variable names in p2p_invalid_tx.py (aitorjs)

Pull request description:

  _tx_orphan_no_fee_ and _tx_orphan_invalid_ don't exist as transactions.

  Have been replaced by _tx_orphan_2_no_fee_ and _tx_orphan_2_invalid_ respectively.

  **Motivation**: Comments are more accurate and easy understandable under the tests context (I think).

ACKs for top commit:
  kristapsk:
    utACK 0d9fdd329e
  theStack:
    ACK 0d9fdd329e 📃

Tree-SHA512: a4cafd931e51fe2a67085e10e9c61178c864c14982664d112b76327e040af08cd1de04eca4a8ae980fad57ba7078017ce02fc60e7658f38380e8172c2ae28b77
2021-08-26 09:03:59 +08:00
aitorjs
0d9fdd329e test, doc: refer to the correct variable names in p2p_invalid_tx.py 2021-08-25 22:55:10 +02:00
fanquake
eb09c26724
Merge bitcoin/bitcoin#22741: test: Add generate* calls to test framework
fab2e23b57 Use generate* from TestFramework (MarcoFalke)
faf7e92804 test: Add generate* calls to test framework (MarcoFalke)

Pull request description:

  This is needed for #22567.

  By making the calls to `generate*` member function of the test framework, it paves the way to make it easier to implicitly call the `sync_all` member function.

ACKs for top commit:
  jnewbery:
    utACK fab2e23b57

Tree-SHA512: 7a7be6be71f0602119689df45d63a1adec309f323eac2330ee0f200676001afe825605859bd02c6a8a8dcf85d925dc1bc37370ef1ceb8ad1d85a66eec0dbfff9
2021-08-24 16:22:30 +08:00
Sebastian Falbesoner
7720d4f650 test: fix failure in feature_nulldummy.py on single-core machines
On single-core machines, executing the test feature_nulldummy.py results in
the following assertion error:

...
2021-08-18T15:37:58.805000Z TestFramework (INFO): Test 4: Non-NULLDUMMY base multisig transaction is invalid after activation
2021-08-18T15:37:58.814000Z TestFramework (ERROR): Assertion failed
Traceback (most recent call last):
  File "[...]/test/functional/test_framework/test_framework.py", line 131, in main
    self.run_test()
  File "[...]/test/functional/feature_nulldummy.py", line 107, in run_test
    self.block_submit(self.nodes[0], [test4tx], accept=False)
  File "[...]/test/functional/feature_nulldummy.py", line 134, in block_submit
    assert_equal(None if accept else 'block-validation-failed', node.submitblock(block.serialize().hex()))
  File "[...]/test/functional/test_framework/util.py", line 49, in assert_equal
    raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
AssertionError: not(block-validation-failed == non-mandatory-script-verify-flag (Dummy CHECKMULTISIG argument must be zero))
2021-08-18T15:37:58.866000Z TestFramework (INFO): Stopping nodes
...

The behaviour can be reproduced on a multi-core machine by simply changing the
function GetNumCores() (in src/util/system.cpp) to return 1:

int GetNumCores()
{
    return 1;
}
2021-08-23 18:11:37 +02:00
Sebastian Falbesoner
646b3885f7 test: refactor: use named args for block_submit in feature_nulldummy.py 2021-08-23 18:11:37 +02:00
MarcoFalke
f6f7a12462
Merge bitcoin/bitcoin#22622: util: Check if specified config file cannot be opened
127b4608e9 test: Check if specified config file cannot be opened (nthumann)
6bb54708e6 util: Check if specified config file cannot be opened (nthumann)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/issues/22612.
  When running e.g. `./src/bitcoind -datadir=/tmp/bitcoin -regtest -conf=/tmp/bitcoin/regtest/bitcoin.conf` and the specified config cannot be opened (doesn't exist, permission denied, ...), the initialization silently uses the default config.

  As voidburn already noted:
  > I can't think of a situation in which a config file is specified explicitly (in the startup options, as per service unit linked above), but inaccessible, where the fail condition should be to keep booting using defaults instead.

  With this patch applied, the initialization will fail immediately, if the specified config file cannot be opened. If no config file is explicitly specified, the behavior is unchanged. This not only affects `bitcoind`, but also `bitcoin-cli` and `bitcoin-qt`.

  In the example below the datadir is accessible, but the config file is not due to insufficient permissions:
  ```
  $ ./src/bitcoind -datadir=/tmp/bitcoin -regtest --debug=1 -conf=/tmp/bitcoin/regtest/bitcoin.conf
  Error: Error reading configuration file: specified config file "/tmp/bitcoin/regtest/bitcoin.conf" could not be opened.
  ```

ACKs for top commit:
  0xB10C:
    ACK 127b4608e9
  Zero-1729:
    tACK 127b4608e9
  theStack:
    Tested ACK 127b4608e9

Tree-SHA512: 4fe487921485426f1d1da8d256c388af517b984b639d776aec7b159b3e23b669824093d3bdd31139d9415ed5f5de405b3e6a51b110c8ab471f12b9c99ac67cc1
2021-08-23 12:58:01 +02:00
MarcoFalke
489beb3984
Merge bitcoin/bitcoin#22641: test: Split rpc_signmessage test for disabled wallet
a3b559c970 test: added test for disabled wallet (Shubhankar Gambhir)

Pull request description:

  This PR enables a part of the non-wallet functional test  (rpc_signmessage.py) to be run even with the Bitcoin Core wallet disabled, it is inspired by #20078.

  Divided tests in rpc_signmessage.py into 2 files wallet_signmessagewithaddress.py and rpc_signmessagewithprivkey.py, latter one can run even when wallet is disabled that provides extra test which was not performed earlier.

  * we need bitcoincore wallet to run  rpc_signmessage.py, but it is olny required for signing messages with address and not for signing messages wih private key, so latter one can be in a seperate test which can run without wallet
  * verifying message doesn't require wallet, so it can be used in both tests without any problem
  * 2 tests are named as wallet_signmessagewithaddress.py and rpc_signmessagewithprivkey.py to provide clarity of what they are testing.

ACKs for top commit:
  vasild:
    ACK a3b559c970
  theStack:
    Code-review ACK a3b559c970

Tree-SHA512: 1bfca3baf3123a02f0a2389e55e141d64430c3bed40ff5a5fb97ef2c66e2853c46e4b2dff62b948eb94dc574cb89d061769330f0535e2d5d1be76b60101136ac
2021-08-23 09:34:28 +02:00
MarcoFalke
fab2e23b57
Use generate* from TestFramework
The changes in feature_rbf can be reviewed with
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2021-08-19 19:02:11 +02:00
MarcoFalke
faf7e92804
test: Add generate* calls to test framework 2021-08-19 15:30:07 +02:00
MarcoFalke
f5a406f003
Merge bitcoin/bitcoin#22633: refactor: Replace remaining binascii method calls
021daedfa1 refactor: replace remaining binascii method calls (Zero-1729)

Pull request description:

  This PR removes the remaining `binascii` method calls outside `test/functional` and `test_framework`, as pointed out here  https://github.com/bitcoin/bitcoin/pull/22619#pullrequestreview-722153458.

  Follow-up to #22593 and #22619
  Closes #22605

ACKs for top commit:
  josibake:
    re-ACK 021daedfa1
  theStack:
    re-ACK 021daedfa1

Tree-SHA512: 2ae9fee8917112c91a5406f219ca70f24cd8902b903db5a61fc2de85ad640d669a772f5c05970be0fcee6ef1cdd32fae2ca5d1ec6dc9798b43352c8160ddde6f
2021-08-21 16:29:55 +02:00
MarcoFalke
4fc15d1566
Merge bitcoin/bitcoin#22707: test: refactor use of getrawmempool in functional tests for efficiency
47c48b5f35 test: only use verbose for getrawmempool when necessary in functional tests (Michael Dietz)
77349713b1 test: use getmempoolentry instead of getrawmempool in functional tests when appropriate (Michael Dietz)
86dbd54ae8 test: improve mempool_updatefrom efficiency by using getmempoolentry for specific txns (Michael Dietz)

Pull request description:

  I don't think this changes the intention of the test. But it does shave ~30 seconds off the time it takes to run. From what I've seen our CI `macOS 11 native [gui] [no depends]` runs `mempool_updatefrom.py` in ~135 seconds. After this PR it should run in ~105 seconds

  I noticed this improvement should probably be made when testing performance/runtimes of https://github.com/bitcoin/bitcoin/pull/22698. But I wanted to separate this out from that PR so the affects of each is decoupled

  Edit: The major change in this PR is improving mempool_updatefrom.py's runtime as this is a very long running test. Then made the same efficiency improvements across all the functional tests as it made since to do that here

ACKs for top commit:
  theStack:
    Tested ACK 47c48b5f35

Tree-SHA512: 40f553715f3d4649dc18c2738554eafaca9ea800c4b028c099217896cc1c466ff457ae814d59cf8564c782a8964d8fac3eda60c1b6ffb08bbee1439b2d34434b
2021-08-20 17:39:17 +02:00
MarcoFalke
999f8b24cc
Merge bitcoin/bitcoin#22568: test: add addr-fetch peer connection state and timeout coverage
f8d8eb5fda test: add addr-fetch timeout connection coverage in p2p_addrfetch.py (Jon Atack)
9321086af7 test: add assert_getpeerinfo method and coverage in p2p_addrfetch.py (Jon Atack)

Pull request description:

  This patch adds additional addr-fetch peer connection state and timeout coverage as a follow-up to #22096.

ACKs for top commit:
  Saviour1001:
    Tested ACK <code>[f8d8eb5](f8d8eb5fda)</code>
  mzumsande:
    Code review ACK f8d8eb5fda

Tree-SHA512: 9a13a705d1da6b308d6dcbc6930575205e2e88bfe9f2e7cb4e0c4c40d26538430e6b02c6c772d0cee64e534777348291469a139f99afbf9d4f93f31b9e7b0818
2021-08-19 21:56:15 +02:00
S3RK
8dcbbbea64 test: fix bug in 22686 2021-08-19 10:09:36 +02:00
Jon Atack
b7a17444e0
log, sync: add LOCK logging category, apply it to lock contention 2021-08-18 19:57:15 +02:00
fanquake
68faa87881
test: use f-strings in mining_*.py tests 2021-08-18 12:39:20 +08:00
fanquake
c2a5d560df
test: use f-strings in interface_*.py tests 2021-08-18 12:39:20 +08:00
fanquake
86d958262d
test: use f-strings in feature_proxy.py 2021-08-18 12:39:20 +08:00
fanquake
31bdb33dcb
test: use f-strings in feature_segwit.py 2021-08-18 12:39:20 +08:00
fanquake
b166d54c3c
test: use f-strings in feature_versionbits_warning.py 2021-08-18 12:39:19 +08:00
fanquake
cf6d66bf94
test: use f-strings in feature_settings.py 2021-08-18 12:39:19 +08:00
fanquake
6651d77f22
test: use f-strings in feature_pruning.py 2021-08-18 12:39:19 +08:00
fanquake
961f5813ba
test: use f-strings in feature_notifications.py 2021-08-18 12:39:19 +08:00
fanquake
1a546e6f6c
test: use f-strings in feature_minchainwork.py 2021-08-18 12:39:19 +08:00
fanquake
6679eceacc
test: use f-strings in feature_logging.py 2021-08-18 12:39:18 +08:00
fanquake
fb633933ab
test: use f-strings in feature_loadblock.py 2021-08-18 12:39:18 +08:00
fanquake
e9ca8b254d
test: use f-strings in feature_help.py 2021-08-18 12:39:18 +08:00
fanquake
ff7e330999
test: use f-strings in feature_filelock.py 2021-08-18 12:39:18 +08:00
fanquake
d5a6adc5e4
test: use f-strings in feature_fee_estimation.py 2021-08-18 12:39:18 +08:00
fanquake
a2de33cbdc
test: use f-strings in feature_dersig.py 2021-08-18 12:39:17 +08:00
fanquake
a2502cc63f
test: use f-strings in feature_dbcrash.py 2021-08-18 12:39:17 +08:00
fanquake
3e2f84e7a9
test: use f-strings in feature_csv_activation.py 2021-08-18 12:39:17 +08:00
fanquake
e2f1fd8ee9
test: use f-strings in feature_config_args.py 2021-08-18 12:39:17 +08:00
fanquake
36d33d32b1
test: use f-strings in feature_cltv.py 2021-08-18 12:39:17 +08:00
fanquake
dca173cc04
test: use f-strings in feature_blocksdir.py 2021-08-18 12:39:17 +08:00
fanquake
5453e87062
test: use f-strings in feature_backwards_compatibility.py 2021-08-18 12:39:08 +08:00
fanquake
6f3d5ad67a
test: use f-strings in feature_asmap.py 2021-08-18 09:52:25 +08:00
Zero-1729
021daedfa1
refactor: replace remaining binascii method calls 2021-08-16 19:30:56 +01:00
MarcoFalke
b935abb9eb
Merge bitcoin/bitcoin#22684: test: check for invalid -prune parameters
e2ff385e13 test: check for invalid `-prune` parameters (Sebastian Falbesoner)

Pull request description:

  This small PR adds missing test coverage for invalid `-prune` parameter values / combinations:

  77e23ca945/src/init.cpp (L926-L928)

  77e23ca945/src/init.cpp (L935-L937)

  77e23ca945/src/init.cpp (L844-L849)

  Not sure if the tests fit into `feature_config_args.py` or should rather be moved into `feature_pruning.py`; the latter though seems to be run less often due to being very memory-hungry.

ACKs for top commit:
  laanwj:
    Code review ACK e2ff385e13

Tree-SHA512: bb0db98090058ecac9f8a01301634e9dba9a65fd56b6a0b770f88da28c4f01e240e22b1225f0d231e28bdd4b5b51bff0e6853cccc46ed0190e91b84f7954a9db
2021-08-16 18:15:21 +02:00
Sebastian Falbesoner
e2ff385e13 test: check for invalid -prune parameters 2021-08-16 16:36:48 +02:00
Michael Dietz
47c48b5f35
test: only use verbose for getrawmempool when necessary in functional tests 2021-08-16 18:34:14 +05:00
Michael Dietz
77349713b1
test: use getmempoolentry instead of getrawmempool in functional tests when appropriate 2021-08-16 18:29:07 +05:00
Michael Dietz
86dbd54ae8
test: improve mempool_updatefrom efficiency by using getmempoolentry for specific txns 2021-08-16 17:56:28 +05:00
Michael Dietz
e05cd0546a
doc: add another signing flow for multisig with descriptor wallets and PSBTs 2021-08-16 10:43:07 +05:00
Michael Dietz
1f20501efc
test: add functional test for multisig flow with descriptor wallets and PSBTs 2021-08-16 10:43:07 +05:00
fanquake
820129aee9
Merge bitcoin/bitcoin#22686: wallet: Use GetSelectionAmount in ApproximateBestSubset
92885c4f69 test: Test for ApproximateBestSubset edge case with too little fees (Andrew Chow)
d9262324e8 wallet: Assert that enough was selected to cover the fees (Andrew Chow)
2de222c401 wallet: Use GetSelectionAmount for target value calculations (Andrew Chow)

Pull request description:

  The `m_value` used for the target calculation in `ApproximateBestSubset` is incorrect, it should be `GetSelectionAmount`. This causes a bug that is only apparent when the minimum relay fee is set to be very high.

  A test case is added for this, in addition to an assert in `CreateTransactionInternal` that would have also caught this issue if someone were able to hit the edge case.

  Fixes #22670

ACKs for top commit:
  instagibbs:
    utACK 92885c4f69

Tree-SHA512: bd61fa61ffb60873e097737eebea3afe8a42296ba429de9038b3a4706763b34de9409de6cdbab21ff7f51f4787b503f840873182d9c4a1d6e12a54b017953547
2021-08-16 11:23:53 +08:00
Samuel Dobson
502d22ceed
Merge bitcoin/bitcoin#22541: Add a new RPC command: restorewallet
5fe8100ff3 Change the wallet_backup.py test to use the restorewallet RPC command instead of restoring wallets manually. (lsilva01)
ae23faba6f Add a new RPC command: restorewallet (lsilva01)

Pull request description:

  As far as I know, there is no command to restore the wallet from a backup file.
  The only way to do this is to replace the `wallet.dat` of a newly created wallet with the backup file, which is hardly an intuitive way.

  This PR implements the `restorewallet` RPC command which restores the wallet from the backup file.

  To test:
  First create a backup file:
  `$ bitcoin-cli -rpcwallet="wallet-01" backupwallet /home/Backups/wallet-01.bak`

  Then restore it in another wallet:
  `$ bitcoin-cli  restorewallet "restored-wallet-01" /home/Backups/wallet-01.bak`

ACKs for top commit:
  achow101:
    re-ACK 5fe8100ff3
  prayank23:
    tACK 5fe8100ff3
  meshcollider:
    utACK 5fe8100ff3

Tree-SHA512: 9639df4d8ad32f255f5b868320dc69878bd9aceb3b471b49dfad500b67681e2d354292b5410982fbf18e25a44ed0c06fd4a0dd010e82807c2e00ff32e84047a1
2021-08-15 16:58:05 +12:00
fanquake
4c87665707
Merge bitcoin/bitcoin#22604: p2p, rpc, test: address rate-limiting follow-ups
d930c7f5b0 p2p, rpc, test: address rate-limiting follow-ups (Jon Atack)

Pull request description:

  Incorporates review feedback in #22387.

  Edit, could be considered separately: should a release note (or two) be added for 22.0? e.g. the new getpeerinfo fields in `Updated RPCs` and the rate-limiting itself in `P2P and network changes`.

ACKs for top commit:
  MarcoFalke:
    review ACK d930c7f5b0
  theStack:
    re-ACK d930c7f5b0 🌮
  Zero-1729:
    crACK d930c7f

Tree-SHA512: b2101cad87f59c238603f38bd8e8df7a4d48929794e4de9e0e0ff2afa935a68475c2d369aa669d124a0bec2f50280fb47e8b980bde6ad812db08cf67b71c066a
2021-08-14 13:33:47 +08:00
Andrew Chow
92885c4f69 test: Test for ApproximateBestSubset edge case with too little fees
ApproximateBestSubset had an edge case (due to not using
GetSelectionAmount) where it was possible for it to return success but
fail to select enough to cover transaction fees. A test is added that
could trigger this failure prior to the fix being implemented.
2021-08-13 00:34:47 -04:00
Shubhankar Gambhir
a3b559c970 test: added test for disabled wallet
Divided tests in rpc_signmessage.py into 2 files wallet_signmessagewithaddress.py and
rpc_signmessagewithprivkey.py, latter one can run even when wallet is disabled.
2021-08-12 18:14:56 +05:30
lsilva01
5fe8100ff3 Change the wallet_backup.py test to use the restorewallet RPC command instead of restoring wallets manually. 2021-08-10 22:44:54 -03:00
W. J. van der Laan
0b5344b0d1
Merge bitcoin/bitcoin#22632: test: Set regtest.BIP66Height = 102 to speed up tests
fafe896a0b test: Set regtest.BIP66Height = 102 to speed up tests (MarcoFalke)

Pull request description:

  No need to waste time by forcing creation of more than 1000 blocks to get the benefits of being able to test BIP 66. Also, reducing the height makes it more likely that (third-party) tests are conforming to BIP 66, which is enforced on mainnet for all new blocks.

ACKs for top commit:
  GeneFerneau:
    Concept + code review ACK [fafe896](fafe896a0b)
  0xB10C:
    crACK fafe896a0b
  laanwj:
    ACK fafe896a0b
  Zero-1729:
    tACK fafe896
  kristapsk:
    ACK fafe896a0b. Full functional test suite showed few second speed incrase on my laptop (although I didn't do proper benchmarking with multiple runs, just single `time ./test/functional/test_runner.py` on current master vs this PR).
  theStack:
    Tested ACK fafe896a0b
  hg333:
    tACK fafe896a0b

Tree-SHA512: 4bbee3c8587d612e74a59fde49b6439c1296f2fc27d3a7cf59a35e920f729fdd581c930290bd04def618f81412236676ddb99b4ceb4d80dfb9fd610b128a04b1
2021-08-10 16:18:26 +02:00
fanquake
21438d55d5
Merge bitcoin/bitcoin#21800: mempool/validation: mempool ancestor/descendant limits for packages
accf3d5868 [test] mempool package ancestor/descendant limits (glozow)
2b6b26e57c [test] parameterizable fee for make_chain and create_child_with_parents (glozow)
313c09f7b7 [test] helper function to increase transaction weight (glozow)
f8253d69d6 extract/rename helper functions from rpc_packages.py (glozow)
3cd663a5d3 [policy] ancestor/descendant limits for packages (glozow)
c6e016aa13 [mempool] check ancestor/descendant limits for packages (glozow)
f551841d3e [refactor] pass size/count instead of entry to CalculateAncestorsAndCheckLimits (glozow)
97dd1c729d MOVEONLY: add helper function for calculating ancestors and checking limits (glozow)
f95bbf58aa misc package validation doc improvements (glozow)

Pull request description:

  This PR implements a function to calculate mempool ancestors for a package and enforces ancestor/descendant limits on them as a whole. It reuses a portion of `CalculateMemPoolAncestors()`; there's also a small refactor to move the reused code into a generic helper function. Instead of calculating ancestors and descendants on every single transaction in the package and their ancestors, we use a "worst case" heuristic, treating every transaction in the package as each other's ancestor and descendant. This may overestimate everyone's counts, but is still pretty accurate in the our main package use cases, in which at least one of the transactions in the package is directly related to all the others (e.g. 1 parent + 1 child, multiple parents with 1 child, or chains).

  Note on Terminology: While "package" is often used to describe groups of related transactions _within_ the mempool, here, I only use package to mean the group of not-in-mempool transactions we are currently validating.

  #### Motivation

  It would be a potential DoS vector to allow submission of packages to mempool without a proper guard for mempool ancestors/descendants. In general, the purpose of mempool ancestor/descendant limits is to limit the computational complexity of dealing with families during removals and additions. We want to be able to validate multiple transactions on top of the mempool, but also avoid these scenarios:

  - We underestimate the ancestors/descendants during package validation and end up with extremely complex families in our mempool (potentially a DoS vector).
  - We expend an unreasonable amount of resources calculating everyone's ancestors and descendants during package validation.

ACKs for top commit:
  JeremyRubin:
    utACK accf3d5
  ariard:
    ACK accf3d5.

Tree-SHA512: 0d18ce4b77398fe872e0b7c2cc66d3aac2135e561b64029584339e1f4de2a6a16ebab3dd5784f376e119cbafc4d50168b28d3bd95d0b3d01158714ade2e3624d
2021-08-09 12:23:39 +08:00
Samuel Dobson
8fa03c4ddf
Merge bitcoin/bitcoin#21500: wallet, rpc: add an option to list private descriptors
bb822a7af8 wallet, rpc: add listdescriptors private option (S3RK)

Pull request description:

  Rationale: make it possible to backup your wallet with `listdescriptors` command

  * The default behaviour is still to show public version
  * For private version only the root xprv is returned

  Example use-case:
  ```
  > bitcoin-cli -regtest -named createwallet wallet_name=old descriptors=true
  > bitcoin-cli -regtest -rpcwallet=old listdescriptors true | jq '.descriptors' > descriptors.txt

  > bitcoin-cli -regtest -named createwallet wallet_name=new descriptors=true blank=true
  > bitcoin-cli -regtest -rpcwallet=new importdescriptors "$(cat descriptors.txt)"
  ```

  In case of watch-only wallet without private keys there will be following output:
  ```
  error code: -4
  error message:
  Can't get descriptor string.
  ```

ACKs for top commit:
  achow101:
    re-ACK bb822a7af8
  Rspigler:
    tACK bb822a7af8
  jonatack:
    ACK bb822a7af8 per `git diff 2854ddc bb822a7`
  prayank23:
    tACK bb822a7af8
  meshcollider:
    Code review ACK bb822a7af8

Tree-SHA512: f6dddc72a74e5667071ccd77f8dce578382e8e29e7ed6a0834ac2e114a6d3918b59c2f194f4079b3259e13d9ba3b4f405619940c3ecb7a1a0344615aed47c43d
2021-08-09 14:09:07 +12:00
glozow
accf3d5868 [test] mempool package ancestor/descendant limits 2021-08-06 10:04:59 +01:00
glozow
2b6b26e57c [test] parameterizable fee for make_chain and create_child_with_parents 2021-08-06 10:04:59 +01:00
glozow
313c09f7b7 [test] helper function to increase transaction weight 2021-08-06 10:04:59 +01:00
glozow
f8253d69d6 extract/rename helper functions from rpc_packages.py
MOVEONLY; no change in behavior. Rename because there is another helper
funciton in chain_transaction in test_framework.util.py
2021-08-06 10:04:59 +01:00
MarcoFalke
f4328ebef5
Merge bitcoin/bitcoin#22619: test: refactor: use consistent bytes <-> hex-string conversion in functional test framework
5a1bef60a0 test: refactor: remove binascii from test_framework (Zero-1729)

Pull request description:

  This PR continues the work started in PR #22593, regarding using the `bytes` built-in module. In this PR specifically, instances of `binascii`'s methods `hexlify`, `unhexlify`,  and `a2b_hex` have been replaced with the build-in `bytes` module's `hex` and `fromhex` methods where appropriate to make bytes <-> hex-string conversions consistent across the functional test files and test_framework.

  Additionally, certain changes made are based on the following assumption:

  ```
  bytes.hex(data) == binascii.hexlify(data).decode()
  bytes.hex(data).encode() == binascii.hexlify(data)
  ```

  Ran the functional tests to ensure behaviour is still consistent and changes didn't break existing tests.

  closes #22605

ACKs for top commit:
  theStack:
    Code-review ACK 5a1bef60a0 🔢

Tree-SHA512: 8f28076cf0580a0d02a156f3e1e94c9badd3d41c3fbdfb2b87cd8a761dde2c94faa5f4c448d6747b1ccc9111c3ef1a1d7b42a11c806b241fa0410b7529e2445f
2021-08-05 12:15:01 +02:00
MarcoFalke
fafe896a0b
test: Set regtest.BIP66Height = 102 to speed up tests 2021-08-05 12:08:33 +02:00
Amiti Uttarwar
aa79c91260 [docs] Add release notes for #21528
And fix a typo in the test.
2021-08-04 12:36:22 -07:00
Zero-1729
5a1bef60a0
test: refactor: remove binascii from test_framework 2021-08-04 19:59:13 +01:00
Jon Atack
d930c7f5b0
p2p, rpc, test: address rate-limiting follow-ups 2021-08-04 19:03:51 +02:00
nthumann
127b4608e9
test: Check if specified config file cannot be opened 2021-08-04 14:54:05 +02:00
MarcoFalke
87257d860e
Merge bitcoin/bitcoin#22532: test : improve mempool_accept_wtxid.py
91b05974fc Improve mempool_accept_wtxid.py (naiza)

Pull request description:

  Follow-up to #22253 adding changes suggested in [#22253 (review)](https://github.com/bitcoin/bitcoin/pull/22253#discussion_r666933370)

ACKs for top commit:
  glozow:
    utACK 91b05974fc

Tree-SHA512: 383064138a5b2160d769c9df370470fd585c91682083013a6fa15e14448a4b481bc09b3a0ed6e75554db2c378df6b2263c65f209f973c9e9d577e15814a4be1d
2021-08-03 11:04:50 +02:00
MarcoFalke
ad0fc453cc
Merge bitcoin/bitcoin#16333: test: Set BIP34Height = 2 for regtest
222290f543 test: Set BIP34Height = 2 for regtest (MarcoFalke)
fac90c55be test: Create all blocks with version 4 or higher (MarcoFalke)

Pull request description:

  BIP34 is active on the current tip of mainnet, so all miners must obey it. It would be nice if it also was active in fresh regtest instances from the earliest time possible.

  I changed the BIP34 height to `2`, so that the block at height=1 may be used to mine a duplicate coinbase. (Needed to test mainnet behaviour)

  This pull is done in two commits:

  *  test: Create all blocks with version 4 or higher:
     Now that BIP34 is activated earlier, we need to create blocks with a higher version number. Just bump it to 4 instead of 2 to avoid having to bump it again later.

  *  test: Set BIP34Height = 2 for regtest:
     This fixes the BIP34 implementation in the tests (to match the one of the Core codebase) and updates the tests where needed

ACKs for top commit:
  ajtowns:
    ACK 222290f543
  jonatack:
    ACK 222290f543 tested and reviewed rebased to current master 5e213822f8
  theStack:
    Tested ACK 222290f543

Tree-SHA512: d69c637a62a64b8e87de8c7f0b305823d8f4d115c1852514b923625dbbcf9a4854b5bb3771ff41702ebf47c4c182a4442c6d7c0b9f282c95a34b83e56a73939b
2021-08-03 10:10:43 +02:00
fanquake
10fbb37268
Merge bitcoin/bitcoin#22098: [test, init] DNS seed querying logic
82b6f89819 [style] Small style improvements to DNS parameters (Amiti Uttarwar)
4c89e24f64 [test] Test the delay before querying DNS seeds (Amiti Uttarwar)
6395c8ed56 [test] Test the interactions between -forcednsseed and -dnsseed (Amiti Uttarwar)
6f6b7df6bd [init] Disallow starting up with conflicting paramters for -dnsseed and -forcednsseed (Amiti Uttarwar)
26d0ffe4f2 [test] Test -forcednsseed causes querying DNS seeds (Amiti Uttarwar)
35851450a9 [test] Test the interactions between -connect and -dnsseed (Amiti Uttarwar)
75c05af361 [test] Test logic to query DNS seeds with block-relay-only connections (Amiti Uttarwar)
9c08719778 [test] Introduce test logic to query DNS seeds (Amiti Uttarwar)

Pull request description:

  This PR adds a DNS seed to the regtest chain params to enable testing the DNS seed querying logic of `CConnman::ThreadDNSAddressSeed` and relevant startup parameters. Adds coverage for the changes in #22013 (and then some).

  The main behavioral change to bitcoind is that this PR disallows starting up with conflicting parameters for `-dnsseed` and `-forcednsseed`.

  The tests include:
  * parameter interactions of different combinations of `-connect`, `-dnsseed` and `-forcednsseed`
  * the delay before querying DNS seeds depending on how many addresses are in the addrman
  * the behavior of `-forcednsseed`
  * skipping DNS querying if we have outbound full relay connections & not block-relay-only connections

  Huge props to mzumsande for identifying the timing technique for testing successful connections before running `ThreadDNSAddressSeed` 🙌🏽

ACKs for top commit:
  mzumsande:
    ACK 82b6f89819
  jnewbery:
    reACK 82b6f89819

Tree-SHA512: 9f0c29bfbf99426727e79c0a25606ae09deab91a92e3c5cee7f84c3ca7503a8ac9ab85a85c51841d40b164ef8c991326070f0b2f41d075fb7985df26f6e95d6d
2021-08-03 11:21:15 +08:00
fanquake
06788c6705
Merge bitcoin/bitcoin#21528: [p2p] Reduce addr blackholes
3f7250b328 [test] Use the new endpoint to improve tests (Amiti Uttarwar)
3893da06db [RPC] Add field to getpeerinfo to indicate if addr relay is enabled (Amiti Uttarwar)
0980ca78cd [test] Test that we intentionally select addr relay peers. (Amiti Uttarwar)
c061599e40 [net_processing] Remove RelayAddrsWithPeer function (Amiti Uttarwar)
201e496481 [net_processing] Introduce new field to indicate if addr relay is enabled (Amiti Uttarwar)
1d1ef2db7e [net_processing] Defer initializing m_addr_known (Amiti Uttarwar)
6653fa3328 [test] Update p2p_addr_relay test to prepare (Amiti Uttarwar)
2fcaec7bbb [net_processing] Introduce SetupAddressRelay (Amiti Uttarwar)

Pull request description:

  This PR builds on the test refactors extracted into #22306 (first 5 commits).

  This PR aims to reduce addr blackholes. When we receive an `addr` message that contains 10 or less addresses, we forward them to 1-2 peers. This is the main technique we use for self advertisements, so sending to peers that wouldn't relay would effectively "blackhole" the trickle. Although we cannot prevent this in a malicious case, we can improve it for the normal, honest cases, and reduce the overall likelihood of occurrence. Two known cases where peers would not participate in addr relay are if they have connected to you as a block-relay-only connection, or if they are a light client.

  This implementation defers initialization of `m_addr_known` until it is needed, then uses its presence to decide if the peer is participating in addr relay. For outbound (not block-relay-only) peers, we initialize the filter before sending the initial self announcement when processing their `version` message. For inbound peers, we initialize the filter if/when we get an addr related message (`ADDR`, `ADDRV2`, `GETADDR`). We do NOT initialize the filter based on a `SENDADDRV2` message.

  To communicate about these changes beyond bitcoin core & to (try to) ensure that no other software would be disrupted, I have:
  - Posted to the [mailing list](https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-April/018784.html)
  - Researched other open source clients to confirm compatibility, opened issues in all the projects & documented in https://github.com/bitcoin/bitcoin/pull/21528#issuecomment-809906430. Many have confirmed that this change would not be problematic.
  - Raised as topic during [bitcoin-core-dev meeting](https://www.erisian.com.au/bitcoin-core-dev/log-2021-03-25.html#l-954)
  - Raised as topic during [bitcoin p2p meeting](https://www.erisian.com.au/bitcoin-core-dev/log-2021-04-20.html#l-439)

ACKs for top commit:
  jnewbery:
    reACK 3f7250b328
  glozow:
    ACK 3f7250b328
  ajtowns:
    utACK 3f7250b328

Tree-SHA512: 29069282af684c1cd37d107c395fdd432dcccb11626f3c2dabfe92fdc4c85e74c7c4056fbdfa88017fec240506639b72ac6c311f8ce7c583112eb15f47e421af
2021-08-03 09:47:51 +08:00
naiza
91b05974fc Improve mempool_accept_wtxid.py
Improve mempool_accept_wtxid.py

Improve mempool_accept_wtxid.py

Improve mempool_accept_wtxid.py

Improve mempool_accept_wtxid.py
2021-08-03 06:49:22 +05:30
MarcoFalke
b620b2d58a
Merge bitcoin/bitcoin#22378: test: remove confusing MAX_BLOCK_BASE_SIZE
607076d01b test: remove confusing `MAX_BLOCK_BASE_SIZE` (Sebastian Falbesoner)
4af97c74ed test: introduce `get_weight()` helper for CBlock (Sebastian Falbesoner)
a084ebe133 test: introduce `get_weight()` helper for CTransaction (Sebastian Falbesoner)

Pull request description:

  This is a very late follow-up PR to #10618, which removed the constant `MAX_BLOCK_BASE_SIZE` from the core implementation about four years ago (see also #10608 in why it was considered confusing and superfluous).
  Since there is also no point in still keeping it in the functional test framework, the PR switches to weight-based accounting on the relevant test code parts and use `MAX_BLOCK_WEIGHT` instead for the block limit checks. To prepare that, the first two commits introduce `get_weight()` helpers for the classes CTransaction and CBlock, respectively.

ACKs for top commit:
  MarcoFalke:
    review ACK 607076d01b 🚴

Tree-SHA512: d59aa0b6b3dfd0a849b8063e66de275d252f705f99e25cd3bf6daec028b47d946777ee5b42a060f5283cb18e917ac073119c2c0e11bbc21211f69ef0a6ed335a
2021-08-02 15:51:48 +02:00
Sebastian Falbesoner
ca6c154ef1 test: refactor: remove hex_str_to_bytes helper
Use the built-in class method bytes.fromhex() instead,
which is available since Python 3.0.
2021-08-01 19:26:51 +02:00
MarcoFalke
f2e41d1109
Merge bitcoin/bitcoin#22429: test: refactor: fix segwit terminology (s/witness_program/witness_script/)
8a2b58db9e test: fix segwit terminology (s/witness_program/witness_script/) (Sebastian Falbesoner)

Pull request description:

  This PR fixes wrong uses of the term "witness program", which according to [BIP141](https://github.com/bitcoin/bips/blob/master/bip-0141.mediawiki#Witness_program)  is defined as follows:
  > A scriptPubKey (or redeemScript as defined in BIP16/P2SH) that consists of a 1-byte push opcode (for 0 to 16) followed by a data push between 2 and 40 bytes gets a new special meaning. The value of the first push is called the "version byte". **The following byte vector pushed is called the "witness program".**

  In most cases where "witness program" is used in tests (concerns comments, variable names and in one instance even a function name) what we really want to denote is the "witness script". Thanks to [MarcoFalke for pointing this out in a review comment](https://github.com/bitcoin/bitcoin/pull/22363#discussion_r666794261)!

  Some historical background: At the time when the P2P segwit tests were first introduced (commit 330b0f31ee, PR #8149), the term "witness program" was not used consistently in BIP141: https://bitcoin.stackexchange.com/questions/46451/what-is-the-precise-definition-of-witness-program
  This was fixed in PR https://github.com/bitcoin/bips/pull/416 later.

  So in some way, this PR can be seen as a very late follow-up to the BIP141 fix that also reflects these changes in the tests.

ACKs for top commit:
  josibake:
    tACK 8a2b58db9e

Tree-SHA512: f36bb9e53d1b54b86bfa87ec12f33e3ebca64b5f59d97e9662fe35ba12c25e1c9a4f93a5425d0eaa3879dce9e50368d345555b927bfab76945511f873396892b
2021-08-01 16:59:19 +02:00
Amiti Uttarwar
4c89e24f64 [test] Test the delay before querying DNS seeds
When starting up with a populated addrman, ThreadDNSAddressSeed adds a delay
during which time the node may be able to connect to some peers. This commit
tests the delay changes based on the number of addresses in the addrman.
2021-07-30 11:15:49 -07:00
Amiti Uttarwar
6395c8ed56 [test] Test the interactions between -forcednsseed and -dnsseed
Test that passing conflicting parameters for the two causes a startup error.
This logic also impacts -connect, which soft sets -dnsseed, so add a test for
that too.
2021-07-30 11:15:49 -07:00
Amiti Uttarwar
26d0ffe4f2 [test] Test -forcednsseed causes querying DNS seeds 2021-07-30 11:15:49 -07:00
Amiti Uttarwar
35851450a9 [test] Test the interactions between -connect and -dnsseed 2021-07-30 11:15:49 -07:00
Amiti Uttarwar
75c05af361 [test] Test logic to query DNS seeds with block-relay-only connections
When a node is able to properly shutdown, it will persist its block-relay-only
connections to the addrman. On startup, it will attempt to reconnect to these
anchors. Since block-relay-only connections do not participate in ADDR relay,
succesful connections are insufficient to skip querying the DNS seeds.

This test fails prior to the changes in #22013.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2021-07-30 11:15:49 -07:00
Amiti Uttarwar
9c08719778 [test] Introduce test logic to query DNS seeds
This commit introduces a DNS seed to the regest chain params in order to add
coverage to the DNS querying logic.

The first test checks that we do not query DNS seeds if we are able to
succesfully connect to 2 outbound connections. Since we participate in ADDR
relay with those connections, including sending a GETADDR message during the
VERSION handshake, querying the DNS seeds is unnecessary.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2021-07-30 11:15:49 -07:00
MarcoFalke
da1c0c64fd
Merge bitcoin/bitcoin#22330: test: use MiniWallet for simple doublespend sub-test in feature_rbf.py
aa02c64540 test: use MiniWallet for simple doublespend test in feature_rbf.py (Sebastian Falbesoner)
a3f6397c73 test: feature_rbf.py: make MiniWallet instance available for all sub-tests (Sebastian Falbesoner)
84c874794c test: remove unneeded initialization code in feature_rbf.py (Sebastian Falbesoner)

Pull request description:

  This PR's goal is to prepare the functional test `feature_rbf.py` for more MiniWallet usage. It first gets rid of unused initialization code (I guess that was needed at times when the nodes were still in IBD at the start of tests?), then makes the MiniWallet instance introduced in #22210 available for all sub-tests, and finally, uses that instance in the first sub-test `test_simple_doublespend`.

  Note that the same idea of replacing the `make_utxo` calls with MiniWallet can be also applied to other sub-tests too; this just serves as a first proof-of-concept.

ACKs for top commit:
  MarcoFalke:
    re-ACK aa02c64540 🌯

Tree-SHA512: 2902dd15d493d83b0790028c92d14fbd99ca05ace704c7011fb38261ce6517aeb810ef9f360fcb701d95887975b6a2911cfe538858d38fceb2c1c2a40afdbe6b
2021-07-30 14:01:02 +02:00
MarcoFalke
78f040a6b3
Merge bitcoin/bitcoin#22490: test: Disable automatic connections per default in the functional tests
8ca51af1ec test: Disable automatic connections by default (Martin Zumsande)

Pull request description:

  A node normally doesn't make automatic connections to peers in the functional tests because neither DNS seeds nor hardcoded peers are available on regtest. However, when random entries are inserted into addrman as part of a functional test (e.g. while testing addr relay), `ThreadOpenConnections` will periodically try to connect to them, resulting in log entries such as:
  `[opencon] [net.cpp:400] [ConnectNode] trying connection 18.166.1.1:8333 lastseen=0.0hrs`

  I don't think it's desirable that functional tests try to connect to random computers on the internet, aside from the possibility that at some point in time someone out there might actually answer in a way to ruin a test.

  This PR fixes this problem by disabling  `ThreadOpenConnections` by adding `-connect=0` to the default args, and adding exceptions only when needed for the test to pass.

ACKs for top commit:
  tryphe:
    Concept ACK, light code review ACK 8ca51af1ec

Tree-SHA512: bcfb2de610e6c35a97a2bd7ad6267e968b1ac7529638d99276993cd5bc93ce9919d54e22d6dc84e1b02ecd626ab6554e201693552ea065c29794eece38c43f7d
2021-07-30 13:26:50 +02:00
MarcoFalke
b6c3fceed6
Merge bitcoin/bitcoin#22520: test: improve rpc_blockchain.py tests and assert on time and mediantime
ef5e9304cd test: update logging and docstring in rpc_blockchain.py (Jon Atack)
d548dc71e4 test: replace magic values by constants in rpc_blockchain.py (Jon Atack)
78c361086f test: assert on mediantime in getblockheader and getblockchaininfo (Jon Atack)
0a9129c588 test: assert on the value of getblockchaininfo#time (Jon Atack)

Pull request description:

  Follow-up to #22407 improving test coverage per https://github.com/bitcoin/bitcoin/pull/22407#pullrequestreview-702077013.

ACKs for top commit:
  tryphe:
    untested ACK ef5e9304cd

Tree-SHA512: f746d56f430331bc6a2ea7ecd27b21b06275927966aacf1f1127d8d5fdfd930583cabe72e23df3adb2e005da904fc05dc573b8e5eaa2f86e0e193b89a17a5734
2021-07-30 11:44:25 +02:00
Amiti Uttarwar
3f7250b328 [test] Use the new endpoint to improve tests 2021-07-29 17:43:01 -07:00
Amiti Uttarwar
0980ca78cd [test] Test that we intentionally select addr relay peers.
This test checks that we only relay addresses with inbound peers who have sent
us an addr related message. Uses a combination of GETADDR and ADDR to verify
when peers are eligible.
2021-07-29 17:43:01 -07:00
Amiti Uttarwar
1d1ef2db7e [net_processing] Defer initializing m_addr_known
Use SetupAddressRelay to only initialize `m_addr_known` as needed. For outbound
peers, we initialize the filter before sending our self announcement (not
applicable for block-relay-only connections). For inbound peers, we initialize
the filter when we get an addr related message (ADDR, ADDRV2, GETADDR).

These changes intend to mitigate address blackholes. Since an inbound peer has
to send us an addr related message to become eligible as a candidate for addr
relay, this should reduce our likelihood of sending them self-announcements.
2021-07-29 17:40:21 -07:00
Amiti Uttarwar
6653fa3328 [test] Update p2p_addr_relay test to prepare
Use an init param to make clear whether a getaddr message should be sent when
the P2PConnection receives a version message. These changes are in preparation
for upcoming commits that modify the behavior of a bitcoind node and the test
framework.
2021-07-29 17:40:21 -07:00
MarcoFalke
24fb69dca4
Merge bitcoin/bitcoin#22139: test: add type annotations to util.get_rpc_proxy
fbeb8c43bc test: add type annotations to util.get_rpc_proxy (fanquake)

Pull request description:

  Split out from #22092 while we address the functional test failure.

ACKs for top commit:
  instagibbs:
    ACK fbeb8c43bc

Tree-SHA512: 031ef8703202ae5271787719fc3fea8693574b2eb937ccf852875de95798d7fa3c39a8db7c91993d0c946b45d9b4d6de570bd1102e0344348784723bd84803a8
2021-07-29 09:52:17 +02:00
Sebastian Falbesoner
aa02c64540 test: use MiniWallet for simple doublespend test in feature_rbf.py 2021-07-28 18:15:53 +02:00
Jon Atack
f8d8eb5fda
test: add addr-fetch timeout connection coverage in p2p_addrfetch.py 2021-07-28 18:02:02 +02:00
Jon Atack
9321086af7
test: add assert_getpeerinfo method and coverage in p2p_addrfetch.py 2021-07-28 18:00:09 +02:00
MarcoFalke
67b9416540
Merge bitcoin/bitcoin#21562: [net processing] Various tidying up of PeerManagerImpl ctor
fde1bf4f61 [net processing] Default initialize m_recent_confirmed_transactions (John Newbery)
37dcd12d53 scripted-diff: Rename recentRejects (John Newbery)
cd9902ac50 [net processing] Default initialize recentRejects (John Newbery)
a28bfd1d4c [net processing] Default initialize m_stale_tip_check_time (John Newbery)
9190b01d8d [net processing] Add Orphanage empty consistency check (John Newbery)

Pull request description:

  - Use default initialization of PeerManagerImpl members where possible
  - Remove unique_ptr indirection where it's not needed

ACKs for top commit:
  MarcoFalke:
    ACK fde1bf4f61 👞
  theStack:
    re-ACK fde1bf4f61

Tree-SHA512: 7ddedcc972df8e933e1fbe5c88b8ea17df89e1e58fc769518512c5540e49dc8eddb3f47e78d1329a6fc5644d2c1d11c981f681fd633f5218bfa4b3e6a86f3d7b
2021-07-28 16:31:41 +02:00
W. J. van der Laan
31fef69c03
Merge bitcoin/bitcoin#22047: index, rpc: Coinstatsindex follow-ups
779e638ca9 coinstats: Add comments for new coinstatsindex values (Fabian Jahr)
5b3d4e724f Index: Improve logging in coinstatsindex (Fabian Jahr)
d4356d4e48 rpc: Block until synced if coinstatsindex is used in gettxoutsetinfo (Fabian Jahr)
a5f6791139 rpc: Add missing gettxoutsetinfo help docs (Fabian Jahr)
01386bfd88 Index: Return early from failed coinstatsindex init (Fabian Jahr)
1e3842385b index: Use batch writing in coinstatsindex WriteBlock (Fabian Jahr)
fb65dde147 scripted-diff: Fix coinstats data member names (Fabian Jahr)
8ea8c927ac index: Avoid unnecessary type casts in coinstatsindex (Fabian Jahr)

Pull request description:

  This is a collection of smaller follow-ups to #19521, addressing several post-merge review comments.

ACKs for top commit:
  Sjors:
    re-utACK 779e638ca9
  jonatack:
    re-ACK 779e638ca9 diff since last review involves doc changes only; rebased to current master and verified clean debug build/no silent conflicts, unit tests, and feature_coinstatsindex functional test
  laanwj:
    Code review ACK 779e638ca9
  Talkless:
    re-utACK 779e638ca9 after cosmetic changes.

Tree-SHA512: cb0d038d230c582d7fe3041c89b1e04d39971fab3739d540c609cf826754c6c513b12ded08ac92180aec7a9d7a70114ece50357bd1a902de4adaae9f30b8d699
2021-07-28 15:19:34 +02:00
MarcoFalke
5e213822f8
Merge bitcoin/bitcoin#22530: log: sort logging categories alphabetically
d596dba987 test: assert logging categories are sorted in rpc and help (Jon Atack)
17bbff3b88 log, refactor: use guard clause in LogCategoriesList() (Jon Atack)
7c57297319 log: sort LogCategoriesList and LogCategoriesString alphabetically (Jon Atack)
f720cfa824 test: verify number of categories returned by logging RPC (Jon Atack)

Pull request description:

  Sorting the logging categories seems more user-friendly with the number of categories we now have, allowing CLI users to more quickly find a particular category.

  before
  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: net, tor, mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman, selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej, libevent, coindb, qt, leveldb, validation, i2p, ipc

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: net, tor,
         mempool, http, bench, zmq, walletdb, rpc, estimatefee, addrman,
         selectcoins, reindex, cmpctblock, rand, prune, proxy, mempoolrej,
         libevent, coindb, qt, leveldb, validation, i2p, ipc.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "net": false,
    "tor": true,
    "mempool": false,
    "http": false,
    "bench": false,
    "zmq": false,
    "walletdb": false,
    "rpc": false,
    "estimatefee": false,
    "addrman": false,
    "selectcoins": false,
    "reindex": false,
    "cmpctblock": false,
    "rand": false,
    "prune": false,
    "proxy": true,
    "mempoolrej": false,
    "libevent": false,
    "coindb": false,
    "qt": false,
    "leveldb": false,
    "validation": false,
    "i2p": true,
    "ipc": false
  }
  ```

  after

  ```
  $ bitcoin-cli help logging
  ...
  The valid logging categories are: addrman, bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb, libevent, mempool, mempoolrej, net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor, validation, walletdb, zmq

  $ bitcoind -h | grep -A8 "debug=<category>"
    -debug=<category>
         ...
         output all debugging information. <category> can be: addrman,
         bench, cmpctblock, coindb, estimatefee, http, i2p, ipc, leveldb,
         libevent, mempool, mempoolrej, net, proxy, prune, qt, rand,
         reindex, rpc, selectcoins, tor, validation, walletdb, zmq.

  $ bitcoin-cli logging [] '["addrman"]'
  {
    "addrman": false,
    "bench": false,
    "cmpctblock": false,
    "coindb": false,
    "estimatefee": false,
    "http": false,
    "i2p": false,
    "ipc": false,
    "leveldb": false,
    "libevent": false,
    "mempool": false,
    "mempoolrej": false,
    "net": false,
    "proxy": false,
    "prune": false,
    "qt": false,
    "rand": false,
    "reindex": false,
    "rpc": false,
    "selectcoins": false,
    "tor": false,
    "validation": false,
    "walletdb": false,
    "zmq": false
  }
  ```

ACKs for top commit:
  theStack:
    re-ACK d596dba987

Tree-SHA512: d546257f562b0a288d1b19a028f1a510aaf21bd21da058e7c84653d305ea8662ecb4647ebefd2b97411f845fe5b0b841d40d3fe6814eefcb8ce82df341dfce22
2021-07-28 14:31:11 +02:00
MarcoFalke
93878d2ab5
Merge bitcoin/bitcoin#22423: test: wallet_listtransactions improvements (speedup, cleanup, logging)
a006d7d730 test: add logging to wallet_listtransactions (Sebastian Falbesoner)
47915b1187 test: remove unneeded/redundant code in wallet_listtransactions (Sebastian Falbesoner)
fb6c6a7938 test: speedup wallet_listtransactions by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  This PR improves the test `wallet_listtransactions.py` in three ways:
  * speeds up runtime by a factor of 2-3x by using the good ol' immediate tx relay trick (`-whitelist=noban@127.0.0.1`)
  * removes unneeded/redundant code
  * adds log messages, mostly by turning comments into `self.log.info(...)` calls

ACKs for top commit:
  jonatack:
    ACK a006d7d730
  kristapsk:
    ACK a006d7d730

Tree-SHA512: a91a19f5ebc4d05f0b96c5419683c4c57ac0ef44b64eeb8dd550bd72296fd3a2857a3ba83f755fe4b0b3bd06439973f226070b5d0ce2dee58344dae78cb50290
2021-07-28 14:15:40 +02:00
Jon Atack
d596dba987
test: assert logging categories are sorted in rpc and help 2021-07-28 12:36:28 +02:00
Sebastian Falbesoner
a3f6397c73 test: feature_rbf.py: make MiniWallet instance available for all sub-tests
also document on why we start scanning blocks at height 76
2021-07-27 23:09:48 +02:00
fanquake
fbeb8c43bc
test: add type annotations to util.get_rpc_proxy
Remove proxy.url assignment:
error: "AuthServiceProxy" has no attribute "url"
2021-07-27 11:26:14 +08:00
Sebastian Falbesoner
12f094ec21 test: use constants for CSV/CLTV activation heights in rpc_signrawtransaction 2021-07-27 00:14:57 +02:00
Sebastian Falbesoner
746f203f19 test: introduce generate_to_height helper, use in rpc_signrawtransaction
This will speed up the test a bit and avoid potential .generate() RPC
timeouts (in sub-test `test_signing_with_cltv()`) on slower machines.
2021-07-27 00:14:52 +02:00
Martin Zumsande
8ca51af1ec test: Disable automatic connections by default
This prevents the node from trying to connect to random IPs on the internet
while running the functional tests. Exceptions are added when required for
the test to pass.
2021-07-26 19:11:13 +02:00
Sebastian Falbesoner
e3237b1cd0 test: check that CSV/CLTV are active in rpc_signrawtransaction
Without this check, the tests would also pass if the CSV and
CLTV activation heights are not reached yet (e.g. if the .generate()
calls before are removed), as the operations OP_CSV and OP_CLTV
simply behave as NOPs.
Also fixes a comment in the sub-test `test_signing_with_cltv()`.
2021-07-25 23:44:29 +02:00
Fabian Jahr
d4356d4e48
rpc: Block until synced if coinstatsindex is used in gettxoutsetinfo
During initial sync after startup the gettxoutsetinfo RPC will still return an error while catching up. However, after the initial sync the index will not error immediately anymore when it's in the process of syncing to the tip while being called. Instead it will block until synced and then return the response.
2021-07-25 20:59:49 +02:00
Jon Atack
f720cfa824
test: verify number of categories returned by logging RPC 2021-07-22 17:44:25 +02:00
W. J. van der Laan
5d83e7d714
Merge bitcoin/bitcoin#21090: Default to NODE_WITNESS in nLocalServices
a806647d26 [validation] Always include merkle root in coinbase commitment (Dhruv Mehta)
189128c220 [validation] Set witness script flag with p2sh for blocks (Dhruv Mehta)
ac82b99db7 [p2p] remove redundant NODE_WITNESS checks (Dhruv Mehta)
6f8b198b82 [p2p] remove unused segwitheight=-1 option (Dhruv Mehta)
eba5b1cd64 [test] remove or move tests using `-segwitheight=-1` (Dhruv Mehta)

Pull request description:

  Builds on #21009 and makes progress on remaining items in #17862

  Removing `RewindBlockIndex()` in #21009 allows the following:

  - removal of tests using `segwitheight=-1` in `p2p_segwit.py`.
  - move `test_upgrade_after_activation()` out of `p2p_segwit.py` reducing runtime
  - in turn, that allows us to drop support for `-segwitheight=-1`, which is only supported for that test.
  - that allows us to always set `NODE_WITNESS` in our local services. The only reason we don't do that is to support `-segwitheight=-1`.
  - that in turn allows us to drop all of the `GetLocalServices() & NODE_WITNESS` checks inside `net_processing.cpp`, since our local services would always include `NODE_WITNESS`

ACKs for top commit:
  mzumsande:
    Code-Review ACK a806647d26
  laanwj:
    Code review ACK a806647d26, nice cleanup
  jnewbery:
    utACK a806647d26
  theStack:
    ACK a806647d26

Tree-SHA512: 73e1a69d1d7eca1f5c38558ec6672decd0b60b16c2ef6134df6f6af71bb159e6eea160f9bb5ab0eb6723c6632d29509811e29469d0d87abbe9b69a2890fbc73e
2021-07-22 17:36:38 +02:00
Jon Atack
ef5e9304cd
test: update logging and docstring in rpc_blockchain.py 2021-07-22 01:43:45 +02:00
Jon Atack
d548dc71e4
test: replace magic values by constants in rpc_blockchain.py 2021-07-22 01:43:34 +02:00
Jon Atack
78c361086f
test: assert on mediantime in getblockheader and getblockchaininfo 2021-07-21 18:01:09 +02:00
Jon Atack
0a9129c588
test: assert on the value of getblockchaininfo#time 2021-07-21 17:38:19 +02:00
Klement Tan
a37e29d32f
cli: Implement human readable -getinfo. 2021-07-21 19:27:04 +08:00
MarcoFalke
40fed336b2
Merge bitcoin/bitcoin#22510: test: add test for RPC error 'Transaction already in block chain'
2ebf2fe0e4 test: check for RPC error 'Transaction already in block chain' (-27) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the RPC error "Transaction already in block chain" (error code `RPC_VERIFY_ALREADY_IN_CHAIN` = `RPC_TRANSACTION_ALREADY_IN_CHAIN` = -27), which is thrown in the function `BroadcastTransaction` (src/node/transaction.cpp).

ACKs for top commit:
  kristapsk:
    ACK 2ebf2fe0e4 (ran linter, looked at changes and ran modified test and checked code in `src/node/transaction.cpp`)
  darosior:
    ACK 2ebf2fe0e4

Tree-SHA512: 8bfbd3ff3da0cb3b8745f69b8ca2377f85fa99f0270750840b60e6ae43b5645c5c59b236993e8b2ad0444ec4171484e4f1ee23fa7e81b79d4222bcb623666fa5
2021-07-21 11:10:58 +02:00
MarcoFalke
458d6ac23b
Merge bitcoin/bitcoin#22407: rpc: Return block time in getblockchaininfo
20edf4bcf6 rpc: Return block time in getblockchaininfo (João Barbosa)

Pull request description:

  Return tip time in `getblockchaininfo`, for some use cases this can save a call to `getblock`.

ACKs for top commit:
  naumenkogs:
    ACK 20edf4bcf6
  theStack:
    re-ACK 20edf4bcf6
  0xB10C:
    ACK 20edf4bcf6
  kristapsk:
    ACK 20edf4bcf6
  Zero-1729:
    re-ACK 20edf4bcf6

Tree-SHA512: 29a920cfff1ef53e0af601c3f93f8f9171f3be47fc84b0fa293cb865b824976e8c1510b17b27d17daf0b8e658dd77d9dc388373395f0919fc4a23cd5019642d5
2021-07-21 09:47:35 +02:00
Sebastian Falbesoner
2ebf2fe0e4 test: check for RPC error 'Transaction already in block chain' (-27) 2021-07-20 22:50:14 +02:00
MarcoFalke
539023ab41
Merge bitcoin/bitcoin#22492: wallet: Reorder locks in dumpwallet to avoid lock order assertion
9b85a5e2f7 tests: Test for dumpwallet lock order issue (Andrew Chow)
25d99e6511 Reorder dumpwallet so that cs_main functions go first (Andrew Chow)

Pull request description:

  When a wallet is loaded which has an unconfirmed transaction in the mempool, it will end up establishing the lock order of cs_wallet -> cs_main -> cs_KeyStore. If `dumpwallet` is used on this wallet, then a lock order of cs_wallet -> cs_KeyStore -> cs_main will be used, which causes a lock order assertion. This PR fixes this by reordering `dumpwallet` and `GetKeyBirthTimes` (only used by `dumpwallet`). Specifically, in both functions, the function calls which lock cs_main are done prior to locking cs_KeyStore. This avoids the lock order issue.

  Additionally, I have added a test case to `wallet_dump.py`. Of course testing this requires `--enable-debug`.

  Fixes #22489

ACKs for top commit:
  MarcoFalke:
    review ACK 9b85a5e2f7 🎰
  ryanofsky:
    Code review ACK 9b85a5e2f7. Nice to reduce lock scope, and good test!
  prayank23:
    tACK 9b85a5e2f7
  lsilva01:
    Tested ACK 9b85a5e2f7 under the same conditions reported in issue #22489 and the `dumpwallet` command completed successfully.

Tree-SHA512: d370a8f415ad64ee6a538ff419155837bcdbb167e3831b06572562289239028c6b46d80b23d227286afe875d9351f3377574ed831549ea426fb926af0e19c755
2021-07-20 15:04:07 +02:00
fanquake
8ed8164e6f
Merge bitcoin/bitcoin#22261: [p2p/mempool] Two small fixes to node broadcast logic
5a77abd4e6 [style] Clean up BroadcastTransaction() (John Newbery)
7282d4c036 [test] Allow rebroadcast for same-txid-different-wtxid transactions (glozow)
cd48372b67 [mempool] Allow rebroadcast for same-txid-different-wtxid transactions (John Newbery)
847b6ed48d [test] Test transactions are not re-added to unbroadcast set (Duncan Dean)
2837a9f1ea [mempool] Only add a transaction to the unbroadcast set when it's added to the mempool (John Newbery)

Pull request description:

  1. Only add a transaction to the unbroadcast set when it's added to the mempool

      Currently, if BroadcastTransaction() is called to rebroadcast a
      transaction (e.g. by ResendWalletTransactions()), then we add the
      transaction to the unbroadcast set. That transaction has already been
      broadcast in the past, so peers are unlikely to request it again,
      meaning RemoveUnbroadcastTx() won't be called and it won't be removed
      from m_unbroadcast_txids.

      Net processing will therefore continue to attempt rebroadcast for the
      transaction every 10-15 minutes. This will most likely continue until
      the node connects to a new peer which hasn't yet seen the transaction
      (or perhaps indefinitely).

      Fix by only adding the transaction to the broadcast set when it's added to the mempool.

  2. Allow rebroadcast for same-txid-different-wtxid transactions

      There is some slightly unexpected behaviour when:

      - there is already transaction in the mempool (the "mempool tx")
      - BroadcastTransaction() is called for a transaction with the same txid
        as the mempool transaction but a different witness (the "new tx")

      Prior to this commit, if BroadcastTransaction() is called with
      relay=true, then it'll call RelayTransaction() using the txid/wtxid of
      the new tx, not the txid/wtxid of the mempool tx. For wtxid relay peers,
      in SendMessages(), the wtxid of the new tx will be taken from
      setInventoryTxToSend, but will then be filtered out from the vector of
      wtxids to announce, since m_mempool.info() won't find the transaction
      (the mempool contains the mempool tx, which has a different wtxid from
      the new tx).

      Fix this by calling RelayTransaction() with the wtxid of the mempool
      transaction in this case.

  The third commit is a comment/whitespace only change to tidy up the BroadcastTransaction() function.

ACKs for top commit:
  duncandean:
    reACK 5a77abd
  naumenkogs:
    ACK 5a77abd4e6
  theStack:
    re-ACK 5a77abd4e6
  lsilva01:
    re-ACK 5a77abd4e6

Tree-SHA512: d1a46d32a9f975220e5b432ff6633fac9be01ea41925b4958395b8d641680500dc44476b12d18852e5b674d2d87e4d0160b4483e45d3d149176bdff9f4dc8516
2021-07-20 20:57:58 +08:00
fanquake
e4487fd5bb
Merge bitcoin/bitcoin#22096: p2p: AddrFetch - don't disconnect on self-announcements
5730a43703 test: Add functional test for AddrFetch connections (Martin Zumsande)
c34ad3309f net, rpc: Enable AddrFetch connections for functional testing (Martin Zumsande)
533500d907 p2p: Add timeout for AddrFetch peers (Martin Zumsande)
b6c5d1e450 p2p: AddrFetch - don't disconnect on self-announcements (Martin Zumsande)

Pull request description:

  AddrFetch connections (old name: oneshots) are intended to be short-lived connections on which we ask a peer for addresses via `getaddr` and disconnect after receiving them.

  This is done by disconnecting after receiving the first `addr`. However, it is no longer working as intended, because nowadays, the first `addr` a typical bitcoin core node sends is its self-announcement.
  So we'll disconnect before the peer gets a chance to answer our `getaddr`.

  I checked that this affects both `-seednode` peers specified manually, and DNS seeds when AddrFetch is used as a fallback if DNS doesn't work for us.

  The current behavior of getting peers via AddrFetch when starting with an empty addrman would be to connect to the peer, receive its self-announcement and add it to addrman, disconnect, reconnect to the same peer again as a full outbound (no other addresses in addrman) and then receive more `addr`. This is silly and not in line with AddrFetch peer being intended to be short-lived peers. 

  Fix this by only disconnecting after receiving an `addr` message of size > 1.

  [Edit] As per review discussion, this PR now also adds a timeout after which we disconnect if we haven't received any suitable `addr`, and a functional test.

ACKs for top commit:
  amitiuttarwar:
    reACK 5730a43703
  naumenkogs:
    ACK 5730a43703
  jnewbery:
    ACK 5730a43703

Tree-SHA512: 8a81234f37e827705138eb254223f7f3b3bf44a06cb02126fc7990b0d231b9bd8f07d38d185cc30d55bf35548a6fdc286b69602498d875b937e7c58332158bf9
2021-07-20 20:27:21 +08:00
John Newbery
37dcd12d53 scripted-diff: Rename recentRejects
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:\<$1\>:$2:g" $(git grep -l "\<$1\>" ./src ./test); }

ren recentRejects m_recent_rejects
-END VERIFY SCRIPT-
2021-07-20 13:14:32 +01:00
João Barbosa
20edf4bcf6 rpc: Return block time in getblockchaininfo 2021-07-20 10:43:26 +01:00
Andrew Chow
9b85a5e2f7 tests: Test for dumpwallet lock order issue
Adds a test for the condition which can trigger a lock order assertion.
Specifically, there must be an unconfirmed transaction in the mempool
which belongs to the wallet being loaded. This will establish the order
of cs_wallet -> cs_main -> cs_KeyStore. Then dumpwallet is called on
that wallet. Previously, this would have used a lock order of cs_wallet
-> cs_KeyStore -> cs_main, but this should be fixed now. The test
ensures that.
2021-07-19 12:25:11 -04:00
W. J. van der Laan
d3474b8df2
Merge bitcoin/bitcoin#22387: Rate limit the processing of rumoured addresses
a4bcd687c9 Improve tests using statistics (John Newbery)
f424d601e1 Add logging and addr rate limiting statistics (Pieter Wuille)
b4ece8a1cd Functional tests for addr rate limiting (Pieter Wuille)
5648138f59 Randomize the order of addr processing (Pieter Wuille)
0d64b8f709 Rate limit the processing of incoming addr messages (Pieter Wuille)

Pull request description:

  The rate at which IP addresses are rumoured (through ADDR and ADDRV2 messages) on the network seems to vary from 0 for some non-participating nodes, to 0.005-0.025 addr/s for recent Bitcoin Core nodes. However, the current codebase will happily accept and process an effectively unbounded rate from attackers. There are measures to limit the influence attackers can have on the addrman database (bucket restrictions based on source IPs), but still - there is no need to permit them to feed us addresses at a rate that's orders of magnitude larger than what is common on the network today, especially as it will cause us to spam our peers too.

  This PR implements a [token bucket](https://en.wikipedia.org/wiki/Token_bucket) based rate limiter, allowing an average of 0.1 addr/s per connection, with bursts up to 1000 addresses at once. Whitelisted peers as well as responses to GETADDR requests are exempt from the limit. New connections start with 1 token, so as to not interfere with the common practice of peers' self-announcement.

ACKs for top commit:
  laanwj:
    ACK a4bcd687c9
  vasild:
    ACK a4bcd687c9
  jnewbery:
    ACK a4bcd687c9
  jonatack:
    ACK a4bcd687c9

Tree-SHA512: b757de76ad78a53035b622944c4213b29b3b55d3d98bf23585afa84bfba10808299d858649f92269a16abfa75eb4366ea047eae3216f7e2f6d3c455782a16bea
2021-07-19 12:42:07 +02:00
Andrew Chow
5012a7912e Test that descriptor wallet upgrade does nothing 2021-07-16 15:34:56 -04:00
John Newbery
a4bcd687c9 Improve tests using statistics 2021-07-15 16:31:47 -07:00
Sebastian Falbesoner
a006d7d730 test: add logging to wallet_listtransactions
Co-authored-by: Jon Atack <jon@atack.com>
2021-07-16 01:15:49 +02:00
Sebastian Falbesoner
47915b1187 test: remove unneeded/redundant code in wallet_listtransactions
-> remove unneeded get-out-of IBD generate()
(The test framework already sets up the nodes to be out of IBD
 in setup_nodes(), if setup_clean_chain is not set to True)

-> remove duplicate code line assigning an utxo
2021-07-16 01:15:48 +02:00
Sebastian Falbesoner
fb6c6a7938 test: speedup wallet_listtransactions by whitelisting peers (immediate tx relay)
By whitelisting the peers via -whitelist, the inventory is transmissioned
immediately rather than on average every 5 seconds, speeding up the test by at
least a factor of two:

before:
$ time ./wallet_listtransactions.py
...
0m40.25s real     0m01.74s user     0m01.70s system

with this PR:
$ time ./wallet_listtransactions.py
...
0m14.93s real     0m01.68s user     0m01.87s system

This commit also moves the wallet_listtransactions tests into the < 30s group.
2021-07-16 01:15:45 +02:00
Pieter Wuille
b4ece8a1cd Functional tests for addr rate limiting 2021-07-15 13:03:17 -07:00
Pieter Wuille
5648138f59 Randomize the order of addr processing 2021-07-15 12:59:23 -07:00
Pieter Wuille
0d64b8f709 Rate limit the processing of incoming addr messages
While limitations on the influence of attackers on addrman already
exist (affected buckets are restricted to a subset based on incoming
IP / network group), there is no reason to permit them to let them
feed us addresses at more than a multiple of the normal network
rate.

This commit introduces a "token bucket" rate limiter for the
processing of addresses in incoming ADDR and ADDRV2 messages.
Every connection gets an associated token bucket. Processing an
address in an ADDR or ADDRV2 message from non-whitelisted peers
consumes a token from the bucket. If the bucket is empty, the
address is ignored (it is not forwarded or processed). The token
counter increases at a rate of 0.1 tokens per second, and will
accrue up to a maximum of 1000 tokens (the maximum we accept in a
single ADDR or ADDRV2). When a GETADDR is sent to a peer, it
immediately gets 1000 additional tokens, as we actively desire many
addresses from such peers (this may temporarily cause the token
count to exceed 1000).

The rate limit of 0.1 addr/s was chosen based on observation of
honest nodes on the network. Activity in general from most nodes
is either 0, or up to a maximum around 0.025 addr/s for recent
Bitcoin Core nodes. A few (self-identified, through subver) crawler
nodes occasionally exceed 0.1 addr/s.
2021-07-15 12:52:38 -07:00
Sebastian Falbesoner
84c874794c test: remove unneeded initialization code in feature_rbf.py 2021-07-15 21:15:26 +02:00
W. J. van der Laan
a88fa1a555
Merge bitcoin/bitcoin#22211: net: relay I2P addresses even if not reachable (by us)
7593b06bd1 test: ensure I2P addresses are relayed (Vasil Dimov)
e7468139a1 test: make CAddress in functional tests comparable (Vasil Dimov)
33e211d2a4 test: implement ser/unser of I2P addresses in functional tests (Vasil Dimov)
86742811ce test: use NODE_* constants instead of magic numbers (Vasil Dimov)
ba45f02708 net: relay I2P addresses even if not reachable (by us) (Vasil Dimov)

Pull request description:

  Nodes that can reach the I2P network (have set `-i2psam=`) will relay
  I2P addresses even without this patch. However, nodes that can't reach
  the I2P network will not. This was done as a precaution in
  https://github.com/bitcoin/bitcoin/pull/20119 before anybody could
  connect to I2P because then, for sure, it would have been useless.

  Now, however, we have I2P support and a bunch of I2P nodes, so get all
  nodes on the network to relay I2P addresses to help with propagation,
  similarly to what we do with Tor addresses.

ACKs for top commit:
  jonatack:
    ACK 7593b06bd1
  naumenkogs:
    ACK 7593b06bd1.
  laanwj:
    Code review ACK 7593b06bd1
  kristapsk:
    ACK 7593b06bd1. Code looks correct, tested that functional test suite passes and also that `test/functional/p2p_addrv2_replay.py` fails if I undo changes in `IsRelayable()`.

Tree-SHA512: c9feec4a9546cc06bc2fec6d74f999a3c0abd3d15b7c421c21fcf2d610eb94611489e33d61bdcd5a4f42041a6d84aa892f7ae293b0d4f755309a8560b113b735
2021-07-15 16:53:34 +02:00
fanquake
e2c4ac7cfb
Merge bitcoin/bitcoin#22447: test: whitelist rpc_rawtransaction peers to speed up tests
a3d6ec5bb5 test: move rpc_rawtransaction tests to < 30s group (Jon Atack)
5a1ed96077 test: whitelist rpc_rawtransaction peers to speed up tests (Jon Atack)

Pull request description:

  Speed up the somewhat slow `rpc_rawtransaction.py` test by more than 3x (from 45-55 seconds to 15 seconds on a laptop running 2 x 2.5GHz).

ACKs for top commit:
  mjdietzx:
    ACK a3d6ec5bb5
  kristapsk:
    ACK a3d6ec5bb5
  theStack:
    ACK a3d6ec5bb5 🐎
  brunoerg:
    tACK a3d6ec5bb5

Tree-SHA512: f1d105594c9b5b257a7096b631a6fa5aeb50e330a351f75c2d6ffa7dd73abdb6e1f596a78c16d204a9bac3fe506e0519f9ad96bb8477ab6424c8e18125ccb659
2021-07-15 14:31:19 +08:00
Jon Atack
a3d6ec5bb5
test: move rpc_rawtransaction tests to < 30s group 2021-07-14 16:08:21 +02:00
Jon Atack
5a1ed96077
test: whitelist rpc_rawtransaction peers to speed up tests
in my testing from 45-55 seconds to 15.
2021-07-14 16:03:15 +02:00
Hennadii Stepanov
0c845e3f89
test: Fix wallet_listdescriptors.py if bdb is not compiled 2021-07-14 16:26:08 +03:00
MarcoFalke
531c2b7c04
Merge bitcoin/bitcoin#20354: test: Add feature_taproot.py --previous_release
fa80e10d94 test: Add feature_taproot.py --previous_release (MarcoFalke)
85ccffa266 test: move releases download incantation to README (Sjors Provoost)
29d6b1da2a test: previous releases: add v0.20.1 (Sjors Provoost)

Pull request description:

  Disabling the new consensus code at runtime is fine, but potentially fragile and incomplete. Fix that by giving the option to run with a version that has been compiled without any taproot code.

ACKs for top commit:
  Sjors:
    tACK fa80e10
  NelsonGaldeman:
    tACK fa80e10d94

Tree-SHA512: 1a1feef823f08c05268759645a8974e1b2d39a024258f5e6acecbe25097aae3fa9302c27262978b40f1aa8e7b525b60c0047199010f2a5d6017dd6434b4066f0
2021-07-14 10:57:06 +02:00
W. J. van der Laan
d8f1e1327f
Merge bitcoin/bitcoin#22112: Force port 0 in I2P
4101ec9d2e doc: mention that we enforce port=0 in I2P (Vasil Dimov)
e0a2b390c1 addrman: reset I2P ports to 0 when loading from disk (Vasil Dimov)
41cda9d075 test: ensure I2P ports are handled as expected (Vasil Dimov)
4f432bd738 net: do not connect to I2P hosts on port!=0 (Vasil Dimov)
1f096f091e net: distinguish default port per network (Vasil Dimov)
aeac3bce3e net: change I2P seeds' ports to 0 (Vasil Dimov)
38f900290c net: change assumed I2P port to 0 (Vasil Dimov)

Pull request description:

  _This is an alternative to https://github.com/bitcoin/bitcoin/pull/21514, inspired by https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-815049933. They are mutually exclusive. Just one of them should be merged._

  Change assumed ports for I2P to 0 (instead of the default 8333) as this is closer to what actually happens underneath with SAM 3.1 (https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-812632520, https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-816564719).

  Don't connect to I2P peers with advertised port != 0 (we don't specify a port to our SAM 3.1 proxy and it always connects to port = 0).

  Note, this change:
  * Keeps I2P addresses with port != 0 in addrman and relays them to others via P2P gossip. There may be non-bitcoin-core-22.0 peers using SAM 3.2 and for them such addresses may be useful.
  * Silently refuses to connect to I2P hosts with port != 0. This is ok for automatically chosen peers from addrman. Not so ok for peers provided via `-addnode` or `-connect` - a user who specifies `foo.b32.i2p:1234` (non zero port) may wonder why "nothing is happening".

  Fixes #21389

ACKs for top commit:
  laanwj:
    Code review ACK 4101ec9d2e
  jonatack:
    re-ACK 4101ec9d2e per `git range-diff efff9c3 0b0ee03 4101ec9`, built with DDEBUG_ADDRMAN, did fairly extensive testing on mainnet both with and without a peers.dat / -dnsseeds=0 to test boostrapping.

Tree-SHA512: 0e3c019e1dc05e54f559275859d3450e0c735596d179e30b66811aad9d5b5fabe3dcc44571e8f7b99f9fe16453eee393d6e153454dd873b9ff14907d4e6354fe
2021-07-13 14:52:41 +02:00
W. J. van der Laan
842e2a9c54
Merge bitcoin/bitcoin#20234: net: don't bind on 0.0.0.0 if binds are restricted to Tor
2feec3ce31 net: don't bind on 0.0.0.0 if binds are restricted to Tor (Vasil Dimov)

Pull request description:

  The semantic of `-bind` is to restrict the binding only to some address.
  If not specified, then the user does not care and we bind to `0.0.0.0`.
  If specified then we should honor the restriction and bind only to the
  specified address.

  Before this change, if no `-bind` is given then we would bind to
  `0.0.0.0:8333` and to `127.0.0.1:8334` (incoming Tor) which is ok -
  the user does not care to restrict the binding.

  However, if only `-bind=addr:port=onion` is given (without ordinary
  `-bind=`) then we would bind to `addr:port` _and_ to `0.0.0.0:8333` in
  addition.

  Change the above to not do the additional bind: if only
  `-bind=addr:port=onion` is given (without ordinary `-bind=`) then bind
  to `addr:port` (only) and consider incoming connections to that as Tor
  and do not advertise it. I.e. a Tor-only node.

ACKs for top commit:
  laanwj:
    Code review ACK 2feec3ce31
  jonatack:
    utACK 2feec3ce31 per `git diff a004833 2feec3c`
  hebasto:
    ACK 2feec3ce31, tested on Linux Mint 20.1 (x86_64):

Tree-SHA512: a04483af601706da928958b92dc560f9cfcc78ab0bb9d74414636eed1c6f29ed538ce1fb5a17d41ed82c9c9a45ca94899d0966e7ef93da809c9bcdcdb1d1f040
2021-07-12 10:08:22 +02:00
Martin Zumsande
5730a43703 test: Add functional test for AddrFetch connections
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-07-12 02:16:54 +02:00
Martin Zumsande
c34ad3309f net, rpc: Enable AddrFetch connections for functional testing
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-07-12 02:16:45 +02:00
Sebastian Falbesoner
8a2b58db9e test: fix segwit terminology (s/witness_program/witness_script/) 2021-07-11 16:01:18 +02:00
S3RK
bb822a7af8 wallet, rpc: add listdescriptors private option 2021-07-10 15:20:52 +02:00
glozow
7282d4c036 [test] Allow rebroadcast for same-txid-different-wtxid transactions
Co-authored-by: John Newbery <john@johnnewbery.com>
2021-07-09 18:21:34 +01:00
Duncan Dean
847b6ed48d [test] Test transactions are not re-added to unbroadcast set 2021-07-09 17:24:08 +01:00
W. J. van der Laan
8ab0c77299
Merge bitcoin/bitcoin#22253: validation: distinguish between same tx and same-nonwitness-data tx in mempool
b7a8cd9963 [test] submit same txid different wtxid as mempool tx (glozow)
fdb48163bf [validation] distinguish same txid different wtxid in mempool (glozow)

Pull request description:

  On master, if you submit a transaction with the same txid but different witness to the mempool, it thinks the transactions are the same. Users submitting through `BroadcastTransaction()` (i.e. `sendrawtransaction` or the wallet) don't get notified that there's a different transaction in the mempool, although it doesn't crash. Users submitting through `testmempoolaccept()` will get a "txn-already-in-mempool" error.

  This PR simply distinguishes between `txn-already-in-mempool` and `txn-same-nonwitness-data-in-mempool`, without handling them differently: `sendrawtransaction` still will not throw, but `testmempoolaccept` will give you a different error.

  I believe the intention of #19645 is to allow full swaps of transactions that have different witnesses but identical nonwitness data. Returning a different error message + adding a test was suggested: https://github.com/bitcoin/bitcoin/pull/19645#issuecomment-705109193 so this is that PR.

ACKs for top commit:
  naumenkogs:
    ACK b7a8cd9963
  jnewbery:
    Code review ACK b7a8cd9963
  theStack:
    Code-review ACK b7a8cd9963
  darosior:
    re-utACK b7a8cd9963

Tree-SHA512: 9c6591edaf8727ba5b4675977adb8cbdef7288584003b6cd659828032dc92d2ae915800a8ef8b6fdffe112c1b660df72297a3dcf2e2e3e1f959c6cb3678c63ee
2021-07-09 17:34:46 +02:00
Vasil Dimov
41cda9d075
test: ensure I2P ports are handled as expected 2021-07-09 11:19:37 +02:00
MarcoFalke
d1e4c56309
Merge bitcoin/bitcoin#22363: test: refactor: use script_util helpers for creating P2{PKH,SH,WPKH,WSH} scripts
905d672b74 test: use script_util helpers for creating P2W{PKH,SH} scripts (Sebastian Falbesoner)
285a65ccfd test: use script_util helpers for creating P2SH scripts (Sebastian Falbesoner)
b57b633b94 test: use script_util helpers for creating P2PKH scripts (Sebastian Falbesoner)
61b6a017a9 test: wallet util: fix multisig P2SH-P2WSH script creation (Sebastian Falbesoner)

Pull request description:

  PR #18788 (commit 08067aebfd) introduced functions to generate output scripts for various types. This PR replaces all manual CScript creations in the P2PKH, P2SH, P2WPKH, P2WSH formats with those helpers in order to increase readability and maintainability over the functional test codebase. The first commit fixes a bug in the wallet_util helper module w.r.t. to P2SH-P2WSH script creation (the result is not used in any test so far, hence it can still be seen as refactoring).

  The following table shows a summary of the output script patterns tackled in this PR:

  | Type | master branch  | PR branch |
  | ---------- | ------------- | ------------- |
  | P2PKH | `CScript([OP_DUP, OP_HASH160, hash160(key), OP_EQUALVERIFY, OP_CHECKSIG])` | `key_to_p2pkh_script(key)`  |
  |             | `CScript([OP_DUP, OP_HASH160, keyhash, OP_EQUALVERIFY, OP_CHECKSIG])` | `keyhash_to_p2pkh_script(keyhash)` |
  | P2SH  | `CScript([OP_HASH160, hash160(script), OP_EQUAL])`  | `script_to_p2sh_script(script)` |
  | P2WPKH | `CScript([OP_0, hash160(key)])` | `key_to_p2wpkh_script(key)` |
  | P2WSH | `CScript([OP_0, sha256(script)])` | `script_to_p2wsh_script(script)` |

  Note that the `key_to_...` helpers can't be used if an invalid key size (not 33 or 65 bytes) is passed, which is the case in some rare instances where the scripts still have to be created manually.

  Possible follow-up ideas:
  * further simplify by identifying P2SH-wrapped scripts and using `key_to_p2sh_p2wpkh_script()` and `script_to_p2sh_p2wsh_script()` helpers
  * introduce and use `key_to_p2pk_script()` helper for P2PK scripts

ACKs for top commit:
  rajarshimaitra:
    tACK 905d672b74
  LarryRuane:
    tACK 905d672b74
  0xB10C:
    ACK 905d672b74
  MarcoFalke:
    review ACK 905d672b74 🕹

Tree-SHA512: 7ccfe69699bc81168ac122b03536720013355c1b2fbb088355b616015318644c4d1cd27e20c4f56c89ad083ae609add4bc838cf6316794d0edb0ce9cf7fa0fd8
2021-07-09 11:17:58 +02:00
glozow
b7a8cd9963 [test] submit same txid different wtxid as mempool tx
Co-authored-by: Antoine Riard <ariard@student.42.fr>
Co-authored-by: Antoine Riard <antoine.riard@gmail.com>
2021-07-08 09:31:45 +01:00
Dhruv Mehta
eba5b1cd64 [test] remove or move tests using -segwitheight=-1 2021-07-07 22:12:04 -07:00
Vasil Dimov
2feec3ce31
net: don't bind on 0.0.0.0 if binds are restricted to Tor
The semantic of `-bind` is to restrict the binding only to some address.
If not specified, then the user does not care and we bind to `0.0.0.0`.
If specified then we should honor the restriction and bind only to the
specified address.

Before this change, if no `-bind` is given then we would bind to
`0.0.0.0:8333` and to `127.0.0.1:8334` (incoming Tor) which is ok -
the user does not care to restrict the binding.

However, if only `-bind=addr:port=onion` is given (without ordinary
`-bind=`) then we would bind to `addr:port` _and_ to `0.0.0.0:8333` in
addition.

Change the above to not do the additional bind: if only
`-bind=addr:port=onion` is given (without ordinary `-bind=`) then bind
to `addr:port` (only) and consider incoming connections to that as Tor
and do not advertise it. I.e. a Tor-only node.
2021-07-07 15:46:38 +02:00
Sebastian Falbesoner
1f449586a9 test: add bad-txns-prevout-null test to mempool_accept.py 2021-07-05 23:06:06 +02:00
Sebastian Falbesoner
aa0a5bb70d test: add bad-txns-prevout-null test case to invalid_txs.py
This reject reason is triggered for non-coinbase transactions with
a coinbase-like outpoint, i.e. hash=0, n=0xffffffff.

Note that the invalid tx templates are currently used in the
functional tests feature_block.py and p2p_invalid_tx.py.
2021-07-05 23:00:41 +02:00
Sebastian Falbesoner
905d672b74 test: use script_util helpers for creating P2W{PKH,SH} scripts 2021-07-05 20:40:52 +02:00
Sebastian Falbesoner
285a65ccfd test: use script_util helpers for creating P2SH scripts 2021-07-05 20:40:47 +02:00
Sebastian Falbesoner
607076d01b test: remove confusing MAX_BLOCK_BASE_SIZE
The constant `MAX_BLOCK_BASE_SIZE` has been removed from the
core implementation years ago due to being confusing and
superfluous, as it is implied by the block weight limit (see
PRs #10618 and #10608). Since there is also no point in
still keeping it in the functional test framework, we switch
to weight-based accounting on the relevant test code parts
and use `MAX_BLOCK_WEIGHT` instead for the block limit
checks.
2021-07-03 17:34:41 +02:00
MarcoFalke
a926d6dfd2
Merge bitcoin/bitcoin#22310: test: Add functional test for replacement relay fee check
c4ddee64c7 test: Add test for replacement relay fee check (Antoine Riard)

Pull request description:

  This PR adds rename the `reject_reason` of our implementation of BIP125 rule 4 and adds missing functional test coverage. Note, `insufficient fee` is already the `reject_reason` of few others `PreChecks` replacement checks and as such might be confusing.

  > The replacement transaction must also pay for its own bandwidth at or above the rate set by the node's minimum relay fee setting. For example, if the minimum relay fee is 1 satoshi/byte and the replacement transaction is 500 bytes total, then the replacement must pay a fee at least 500 satoshis higher than the sum of the originals.

  ```
          // Finally in addition to paying more fees than the conflicts the
          // new transaction must pay for its own bandwidth.
          CAmount nDeltaFees = nModifiedFees - nConflictingFees;
          if (nDeltaFees < ::incrementalRelayFee.GetFee(nSize))
          {
              return state.Invalid(TxValidationResult::TX_MEMPOOL_POLICY, "insufficient fee",
                      strprintf("rejecting replacement %s, not enough additional fees to relay; %s < %s",
                          hash.ToString(),
                          FormatMoney(nDeltaFees),
                          FormatMoney(::incrementalRelayFee.GetFee(nSize))));
          }
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK c4ddee64c7
  glozow:
    ACK c4ddee6, one small suggestion if you retouch.

Tree-SHA512: 7c5d1065db6e6fe57a9f083bf051a7a55eb9892de3a2888679d4a6853491608c93b6e35887ef383a9988d14713fa13a0b1d6134b7354af5fd54765f0d4e98568
2021-07-01 18:36:17 +02:00
fanquake
045bb06ebd
Merge bitcoin/bitcoin#19651: wallet: importdescriptors update existing
3efaf83c75 wallet: deactivate descriptor (S3RK)
6737d9655b test: wallet importdescriptors update existing (S3RK)
586f1d53d6 wallet: maintain SPK consistency on internal flag change (S3RK)
f1b7db1474 wallet: don't mute exceptions in importdescriptors (S3RK)
bf68ebc1cd wallet: allow to import same descriptor twice (S3RK)

Pull request description:

  Rationale: allow updating existing descriptors with `importdescriptors` command.

  Currently if you run same `importdescriptors` command twice with a descriptor containing private key you will get very confusing error — `Missing required fields`. What happens is that Wallet tries to write imported private key to the disk, but it exists already so we get `DB_KEYEXIST (-30995)` from BerkelyDB. Please note, that we set `DB_NOOVERWRITE` (I guess not to lose some keys accidentally). The exception is caught in `catch (...)` in rpcdump.cpp with a generic error.

  With this PR if a descriptor is already present than we will update its activeness, internalness, label, range and next_index.
  For the range only expansion is allowed (range start can only decrease, range end increase).

ACKs for top commit:
  achow101:
    re-ACK 3efaf83c75
  meshcollider:
    Code review ACK 3efaf83c75
  jonatack:
    Light ACK 3efaf83c75 per `git range-diff a000cb0 5d96704 3efaf83` and as a sanity check, re-debug-built on debian with gcc 10.2.1 and clang 11, ran wallet_importdescriptors.py

Tree-SHA512: 122c4b621d64ec8a3b625f3aed9f01a2b5cbaf2029ad0325b5ff38d67fff5cd35324335fabe2dd5169548b01b267c81be6ae0f5c834342f3d5f6eeed515c4843
2021-07-01 10:06:56 +08:00
Sebastian Falbesoner
4af97c74ed test: introduce get_weight() helper for CBlock 2021-07-01 02:32:31 +02:00
Sebastian Falbesoner
a084ebe133 test: introduce get_weight() helper for CTransaction 2021-07-01 02:32:25 +02:00
Antoine Riard
c4ddee64c7 test: Add test for replacement relay fee check 2021-06-30 18:47:08 -04:00
S3RK
3efaf83c75 wallet: deactivate descriptor 2021-06-28 21:44:50 +02:00
S3RK
6737d9655b test: wallet importdescriptors update existing 2021-06-28 21:44:50 +02:00
S3RK
bf68ebc1cd wallet: allow to import same descriptor twice 2021-06-28 21:37:37 +02:00
Sebastian Falbesoner
b57b633b94 test: use script_util helpers for creating P2PKH scripts 2021-06-28 20:14:07 +02:00
Sebastian Falbesoner
61b6a017a9 test: wallet util: fix multisig P2SH-P2WSH script creation 2021-06-28 20:14:01 +02:00
Andrew Chow
e6cf0ed92d wallet, rpc: listdescriptors does not need unlocked
With the last hardened xpub cache, we don't neeed to have the wallet be
unlocked for listdescriptors.
2021-06-24 14:08:46 -04:00
W. J. van der Laan
0553d75268
Merge bitcoin/bitcoin#22154: Add OutputType::BECH32M and related wallet support for fetching bech32m addresses
754f134a50 wallet: Add error message to GetReservedDestination (Andrew Chow)
87a0e7a3b7 Disallow bech32m addresses for legacy wallet things (Andrew Chow)
6dbe4d1072 Use BECH32M for tr() desc, WitV1Taproot, and WitUnknown CTxDests (Andrew Chow)
699dfcd8ad Opportunistically use bech32m change addresses if available (Andrew Chow)
0262536c34 Add OutputType::BECH32M (Andrew Chow)
177c15d2f7 Limit LegacyScriptPubKeyMan address types (Andrew Chow)

Pull request description:

  Currently bech32m addresses are classfied as bech32. Because bech32m is incompatible with bech32, we need to define a new `OutputType` for it so that it can be handled correctly. This PR adds `OutputType::BECH32M`, updates all of the relevant `OutputType` classifications, and handle requests for bech32m addresses. There is now a `bech32m` address type string that can be used.

  * `tr()` descriptors now report their output type as `OutputType::BECH32M`. `WtinessV1Taproot` and `WitnessUnknown` are also classified as `OutputType::BECH32M`.
  * Bech32m addresses are completely disabled for legacy wallets. They cannot be imported (explicitly disallowed in `importaddress` and `importmulti`), will not be created when getting all destinations for a pubkey, and will not be added with `addmultisigaddress`. Additional protections have been added to `LegacyScriptPubKeyMan` to disallow attempting to retrieve bech32m addresses.
  * Since Taproot multisigs are not implemented yet, `createmultisig` will also disallow the bech32m address type.
  * As Taproot is not yet active, `DescriptorScriptPubKeyMan` cannot and will not create a `tr()` descriptor. Protections have been added to make sure this cannot occur.
  * The change address type detection algorithm has been updated to return `bech32m` when there is a segwit v1+ output script and the wallet has a bech32m `ScriptPubKeyMan`, falling back to bech32 if one is not available.

ACKs for top commit:
  laanwj:
    re-review ACK 754f134a50
  Sjors:
    re-utACK 754f134: only change is switching to `bech32m` in two `wallet_taproot.py` test cases.
  fjahr:
    re-ACK 754f134a50
  jonatack:
    ACK 754f134a50

Tree-SHA512: 6ea90867d3631d0d438e2b08ce6ed930f37d01323224661e8e38f183ea5ee2ab65b5891394a3612c7382a1aff907b457616c6725665a10c320174017b998ca9f
2021-06-24 14:20:28 +02:00
MarcoFalke
e59ea87954
Merge bitcoin/bitcoin#22311: test: Add missing syncwithvalidationinterfacequeue in p2p_blockfilters
fadddd13ee test: Add missing syncwithvalidationinterfacequeue (MarcoFalke)
faa211fc6e test: Misc cleanup (MarcoFalke)
fa1668bf50 test: Run pep-8 (MarcoFalke)
facd97ae0f scripted-diff: Renames (MarcoFalke)

Pull request description:

  The index on the block filters is running in the background on the validation interface. To avoid intermittent test failures, it needs to be synced.

  Also other cleanups.

ACKs for top commit:
  lsilva01:
    Tested ACK fadddd13ee on Ubuntu 20.04

Tree-SHA512: d858405db426a2f9d5620059dd88bcead4e3fba3ccc6bd8023f624b768fbcfa2203246fb0b2db620490321730d990f0e78063b21a26988c969cb126d4bd697bd
2021-06-24 12:54:00 +02:00
MarcoFalke
d6a59166a1
Merge bitcoin/bitcoin#22257: test: refactor: various (de)serialization helpers cleanups/improvements
bdb8b9a347 test: doc: improve doc for `from_hex` helper (mention `to_hex` alternative) (Sebastian Falbesoner)
1914054208 scripted-diff: test: rename `FromHex` to `from_hex` (Sebastian Falbesoner)
a79396fe5f test: remove `ToHex` helper, use .serialize().hex() instead (Sebastian Falbesoner)
2ce7b47958 test: introduce `tx_from_hex` helper for tx deserialization (Sebastian Falbesoner)

Pull request description:

  There are still many functional tests that perform conversions from a hex-string to a message object (deserialization) manually. This PR identifies all those instances and replaces them with a newly introduced helper `tx_from_hex`.

  Instances were found via
  * `git grep "deserialize.*BytesIO"`

  and some of them manually, when it were not one-liners.

  Further, the helper `ToHex` was removed and simply replaced by `.serialize().hex()`, since now both variants are in use (sometimes even within the same test) and using the helper doesn't really have an advantage in readability. (see discussion https://github.com/bitcoin/bitcoin/pull/22257#discussion_r652404782)

ACKs for top commit:
  MarcoFalke:
    review re-ACK bdb8b9a347 😁

Tree-SHA512: e25d7dc85918de1d6755a5cea65471b07a743204c20ad1c2f71ff07ef48cc1b9ad3fe5f515c1efaba2b2e3d89384e7980380c5d81895f9826e2046808cd3266e
2021-06-24 12:47:13 +02:00
MarcoFalke
bfa885898a
Merge bitcoin/bitcoin#22306: [test] Improvements to p2p_addr_relay.py
6168eb06b2 [test] Prevent intermittent issue (Amiti Uttarwar)
1d8193e2a2 [test] Remove GetAddrStore class (Amiti Uttarwar)
ef2f149bf2 [test] Update GetAddrStore callers to use AddrReceiver (Amiti Uttarwar)
e8c67ea19a [test] Add functionality to AddrReceiver (Amiti Uttarwar)
09dc073cff [test] Allow AddrReceiver to be used more generally (Amiti Uttarwar)

Pull request description:

  A test refactor broken out from #21528 & a fix to #22243.

  This PR:
  1. consolidates the two helper classes into one, with the intent of making the test logic more clear & usable as we add more subtests to the file

  2. hopefully fixes the test flakiness by bumping up the mocktime interval to ensure `m_next_addr_send` timer triggers

ACKs for top commit:
  mzumsande:
    Code-Review ACK 6168eb06b2
  lsilva01:
    Tested ACK 6168eb06b2 on Ubuntu 20.04
  brunoerg:
    tACK 6168eb06b2

Tree-SHA512: 248324f9d37e0e5ffe4acc437cd72ad9a2960abc868a97c6040a36e6ea8b59029127ac4f63fcf67d981a5bb4dbf2334bb2c23c541fae8e910d5523884bcedcba
2021-06-24 12:10:44 +02:00
Samuel Dobson
567670bec5
Merge bitcoin/bitcoin#22166: Add support for inferring tr() descriptors
d637a9b397 Taproot descriptor inference (Pieter Wuille)
c7388e5ada Report address as solvable based on inferred descriptor (Pieter Wuille)
29e5dd1a5b consensus refactor: extract ComputeTapleafHash, ComputeTaprootMerkleRoot (Pieter Wuille)

Pull request description:

  Includes:
  * First commit from #21365, adding TaprootSpendData in SigningProvider
  * A refactor to expose ComputeTapleafHash and ComputeTaprootMerkleRoot from script/interpreter
  * A tiny change to make `getaddressinfo` report tr() descriptors as solvable (so that inferred descriptors are shown), despite not having signing code for them.
  * Logic to infer the script tree back from TaprootSpendData, and then use that to infer descriptors.

ACKs for top commit:
  achow101:
    re-ACK d637a9b397
  Sjors:
    re-utACK d637a9b
  meshcollider:
    Code review ACK d637a9b397

Tree-SHA512: 5ab9b95da662382d8549004be4a1297a577d7caca6b068f875c7c9343723931d03fa9cbf133de11f83b74e4851490ce820fb80413c77b9e8495a5f812e505d86
2021-06-23 22:42:30 +12:00
MarcoFalke
d6e0d78c31
Merge bitcoin/bitcoin#20966: banman: save the banlist in a JSON format on disk
bb719a08db style: remove () from assert in rpc_setban.py (Vasil Dimov)
24b10ebda3 doc: fix grammar in doc/files.md (Vasil Dimov)
dd4e957dcd test: ensure banlist can be read from disk after restart (Vasil Dimov)
d197977ae2 banman: save the banlist in a JSON format on disk (Vasil Dimov)

Pull request description:

  Save the banlist in `banlist.json` instead of `banlist.dat`.

  This makes it possible to store Tor v3 entries in the banlist on disk
  (and any other addresses that cannot be serialized in addrv1 format).

  Only read `banlist.dat` if it exists and `banlist.json` does not exist (first start after an upgrade).

  Supersedes https://github.com/bitcoin/bitcoin/pull/20904
  Resolves https://github.com/bitcoin/bitcoin/issues/19748

ACKs for top commit:
  jonatack:
    Code review re-ACK bb719a08db per `git range-diff 6a67366 4b52c72 bb719a0`
  achow101:
    Code Review ACK bb719a08db

Tree-SHA512: fc135c3a1fe20bcf5d008ce6bea251b4135e56c78bf8f750b4bd8144c095b81ffe165133cdc7e4715875eec7e7c4e13ad9f5d2450b21102af063d7c8abf716b6
2021-06-23 10:01:56 +02:00
MarcoFalke
03aa59a4e5
Merge bitcoin/bitcoin#22313: test: Add missing sync_all to feature_coinstatsindex
fafd9165e9 test: Add missing sync_all to feature_coinstatsindex (MarcoFalke)

Pull request description:

  Sync the blocks before invalidating them to ensure all nodes are on the right tip. Otherwise nodes[0] might stay on the "stale" block and the test fails (intermittently)

ACKs for top commit:
  jamesob:
    crACK fafd9165e9

Tree-SHA512: ca567b97b839b56c91d52831eaac18d8c843d376be90c9fd8b49d2eb4a46b801a1d2402996d5dfe2bef3e2c9bd75d19ed443e3f42cc4679c5f20043ba556efc8
2021-06-23 08:07:19 +02:00
Andrew Chow
754f134a50 wallet: Add error message to GetReservedDestination
Adds an error output parameter to all GetReservedDestination functions
so that callers can get the actual reason that a change address could
not be fetched. This more closely matches GetNewDestination. This allows
for more granular error messages, such as one that indicates that
bech32m addresses cannot be generated yet.
2021-06-22 21:57:04 -04:00
Andrew Chow
87a0e7a3b7 Disallow bech32m addresses for legacy wallet things
We don't want the legacy wallet to ever have bech32m addresses so don't
allow importing them. This includes addmultisigaddress as that is a
legacy wallet only RPC

Additionally, bech32m multisigs are not available yet, so disallow them
in createmultisig.
2021-06-22 21:57:04 -04:00
Andrew Chow
6dbe4d1072 Use BECH32M for tr() desc, WitV1Taproot, and WitUnknown CTxDests
The tr() descriptor, WitnessV1Taproot CTxDestination, and
WitnessUnknown CTxDestination are OutputType::BECH32M so they should
report as such.
2021-06-22 21:57:04 -04:00
MarcoFalke
fafd9165e9
test: Add missing sync_all to feature_coinstatsindex 2021-06-22 20:10:33 +02:00
MarcoFalke
fadddd13ee
test: Add missing syncwithvalidationinterfacequeue 2021-06-22 19:42:51 +02:00
MarcoFalke
faa211fc6e
test: Misc cleanup
* Replace wait_until with assert_equal where possible
* Use send_and_ping helper where possible
2021-06-22 19:42:35 +02:00
MarcoFalke
fa1668bf50
test: Run pep-8
Can be reviewed with --word-diff-regex=.
2021-06-22 19:42:25 +02:00
MarcoFalke
facd97ae0f
scripted-diff: Renames
-BEGIN VERIFY SCRIPT-

 ren() { sed -i "s/\<$1\>/$2/g" test/functional/p2p_blockfilters.py ; }

 # Rename from "node" to "peer" to avoid confusion with self.nodes
 ren node0 peer_0
 ren node1 peer_1

 # Remove the confusing "C" prefix
 ren CFiltersClient FiltersClient

-END VERIFY SCRIPT-
2021-06-22 19:41:28 +02:00
MarcoFalke
fa27baa9c8
Revert "test: Add temporary logging to debug #20975"
This reverts commit faa94961d6.
2021-06-22 10:04:06 +02:00
MarcoFalke
672870ab7b
Merge bitcoin/bitcoin#22201: test: Fix TestShell to allow running in Jupyter Notebook
168b6c317c add dummy file param to fix jupyter (Josiah Baker)

Pull request description:

  this fixes argparse to use `parse_known_args`. previously, if an unknown argument was passed, argparse would fail with an `unrecognized arguments: %s` error.

  ## why
  the documentation mentions being able to run `TestShell` in a REPL interpreter or a jupyter notebook. when i tried to run inside a jupyter notebook, i got the following error:

  ![image](https://user-images.githubusercontent.com/7444140/121382910-57554880-c947-11eb-94f2-49da8679528c.png)

  this was due to the notebook passing the filename of the notebook as an argument. this is a known problem with notebooks and argparse, documented here: https://stackoverflow.com/questions/48796169/how-to-fix-ipykernel-launcher-py-error-unrecognized-arguments-in-jupyter

  ## testing
  to test, make sure you have jupyter notebooks installed. you can do this by running:
  ```
  pip install notebook
  ```
  or following instructions from [here](https://jupyterlab.readthedocs.io/en/stable/getting_started/installation.html).

  once installed, start a notebook (`jupyter notebook`), launch a python3 kernel and run the following snippet:

  ```python
  import sys

  # make sure this is the path for your system
  sys.path.insert(0, "/path/to/bitcoin/test/functional")
  from test_framework.test_shell import TestShell

  test = TestShell().setup(num_nodes=2, setup_clean_chain=True)
  ```

  you should see the following output, without errors:
  ![image](https://user-images.githubusercontent.com/7444140/121383301-a307f200-c947-11eb-83b6-6c50b2cada25.png)

  if you are unfamiliar with notebooks, here is a short guide on using them: https://jupyter.readthedocs.io/en/latest/running.html

ACKs for top commit:
  MarcoFalke:
    review ACK 168b6c317c
  jamesob:
    crACK 168b6c317c
  practicalswift:
    cr ACK 168b6c317c

Tree-SHA512: 4fee1563bf64a1cf9009934182412446cde03badf2f19553b78ad2cb3ceb0e5e085a5db41ed440473494ac047f04641311ecbba3948761c6553d0ca4b54937b4
2021-06-22 08:11:16 +02:00
Amiti Uttarwar
6168eb06b2 [test] Prevent intermittent issue
Since m_next_addr_send is on a Poisson distribution, increase the mocktime bump
to ensure we don't experience flakiness in the tests. Closes #22243.
2021-06-21 18:05:54 -07:00
Amiti Uttarwar
1d8193e2a2 [test] Remove GetAddrStore class 2021-06-21 18:05:48 -07:00
Amiti Uttarwar
ef2f149bf2 [test] Update GetAddrStore callers to use AddrReceiver 2021-06-21 18:05:43 -07:00
Amiti Uttarwar
e8c67ea19a [test] Add functionality to AddrReceiver
Add two simple helper functions to `AddrReceiver` to support callers currently
using `GetAddrStore` [used in next commit].
2021-06-21 18:05:27 -07:00
Amiti Uttarwar
09dc073cff [test] Allow AddrReceiver to be used more generally
The `on_addr` functionality of `AddrReceiver` tests logic specific to how the
addr messages are set up in the test bodies. To allow other callers to also use
`AddrReceiver`, only apply the assertion logic if the caller indicates
desirability by setting `test_addr_contents` to true when initializing the
class.
2021-06-21 10:25:06 -07:00
Josiah Baker
168b6c317c add dummy file param to fix jupyter
testshell in jupyter was failing due to an extra arg.
this adds a dummy -f param, which allows TestShell to
be used in a command line or jupyter environment
2021-06-21 17:12:16 +02:00
MarcoFalke
74013641e0
Merge bitcoin/bitcoin#22089: test: MiniWallet: fix fee calculation for P2PK and check tx vsize
d6d2ab9845 test: MiniWallet: fix fee calculation for P2PK and check tx vsize (Sebastian Falbesoner)
ce024b1c0e test: MiniWallet: force P2PK signature to have fixed size (71 bytes) (Sebastian Falbesoner)

Pull request description:

  This PR is a follow-up to #21945. It aims to both fix the fee calculation for P2PK mode transactions and enable its vsize check. Currently, the latter assumes a fixed tx length, which is fine for anyone-can-spend txs but doesn't apply to P2PK output spends due to varying DER signature size; the vsize check is therefore disabled for P2PK mode on master branch.

  Creating one million DER signatures with MiniWallet shows the following distribution of sizes (smart people with better math skills probably could deduce the ratios without trying, but hey):

  | DER signature size [bytes]  | #occurences (ratio) |
  | ------------- | ------------- |
  | 71  | 498893 (49.89%) |
  | 70 | 497244 (49.72%) |
  | 69 | 3837 (0.38%) |
  | 68 | 22 (0.0022%) |

  Note that even smaller signatures are possible (for smaller R and S values with leading zero bytes), it's just that the probability decreases exponentially.     Instead of choosing a large vsize check range and hoping that smaller signatures are never created (potentially leading to flaky tests), the proposed solution is ~~to limit the signature size to the two most common sizes 71 and 70 (>99.6% probability) and then accordingly only check for two vsize values; the value to be used for fee calculation is a decimal right between the two possible sizes (167.5 vbytes) and for the vsize check it's rounded down/up integer values are used.~~ to simply grind the signature to a fixed size of 71 bytes (49.89% probability, i.e. on average each call to `sign_tx()`, on average two ECC signing operations are needed).

  ~~The idea of grinding signatures to a fixed size (similar to https://github.com/bitcoin/bitcoin/pull/13666 which grinds to low-R values) would be counter-productive, as the signature creation in the test suite is quite expensive and this would significantly slow down tests that calculate hundreds of signatures (like e.g. feature_csv_activation.py).~~

  For more about transaction sizes on different input/output types, see the following interesting article: https://medium.com/coinmonks/on-bitcoin-transaction-sizes-97e31bc9d816

ACKs for top commit:
  MarcoFalke:
    Concept ACK d6d2ab9845

Tree-SHA512: 011c70ee0e4adf9ba12902e4b6c411db9ae96bdd8bc810bf1d67713367998e28ea328394503371fc1f5087a819547ddaea56c073b28db893ae1c0031d7927f32
2021-06-21 16:11:13 +02:00
W. J. van der Laan
6556da77d7
Merge bitcoin/bitcoin#21056: rpc: Add a -rpcwaittimeout parameter to limit time spent waiting
b9e76f1bf0 rpc: Add test for -rpcwaittimeout (Christian Decker)
f76cb10d7d rpc: Prefix rpcwaittimeout error with details on its nature (Christian Decker)
c490e17ef6 doc: Add release notes for the `-rpcwaittimeout` cli parameter (Christian Decker)
a7fcc8eb59 rpc: Add a `-rpcwaittimeout` parameter to limit time spent waiting (Christian Decker)

Pull request description:

  Adds a new numeric `-rpcwaittimeout` that can be used to limit the
  time we spend waiting on the RPC server to appear. This is used by
  downstream projects to provide a bit of slack when `bitcoind`s RPC
  interface is not available right away.

  This makes the `-rpcwait` argument more useful, since we can now limit
  how long we'll ultimately wait, before potentially giving up and reporting
  an error to the caller. It was discussed in the context of the BTCPayServer
  wanting to have c-lightning wait for the RPC interface to become available
  but still have the option of giving up eventually ([4355]).

  I checked with laanwj whether this is already possible ([comment]), and
  whether this would be a welcome change. Initially I intended to repurpose
  the (optional) argument to `-rpcwait`, however I decided against it since it
  would potentially break existing configurations, using things like `rpcwait=1`,
  or `rpcwait=true` (the former would have an unintended short timeout, when
  old behavior was to wait indefinitely).

  ~Due to its simplicity I didn't implement a test for it yet, but if that's desired I
  can provide one.~ Test was added during reviews.

  [4355]: https://github.com/ElementsProject/lightning/issues/4355
  [comment]: https://github.com/ElementsProject/lightning/issues/4355#issuecomment-768288261

ACKs for top commit:
  laanwj:
    Code review ACK b9e76f1bf0
  promag:
    ACK b9e76f1bf0.

Tree-SHA512: 3cd6728038ec7ca7c35c2e7ccb213bfbe963f99a49bb48bbc1e511c4dd23d9957c04f9af1f8ec57120e47b26eaf580b46817b099d5fc5083c98da7aa92db8638
2021-06-21 15:54:56 +02:00
Vasil Dimov
bb719a08db
style: remove () from assert in rpc_setban.py 2021-06-21 15:29:28 +02:00
Vasil Dimov
dd4e957dcd
test: ensure banlist can be read from disk after restart
With `banlist.dat` (being written in addrv1 format) if we would try to
write a Tor v3 subnet, it would serialize as a dummy-all-0s IPv6
address and subsequently, when deserialized will not result in the same
subnet.

This problem does not exist with `banlist.json` where the data is saved
in textual, human-readable form.
2021-06-21 15:27:03 +02:00
Sebastian Falbesoner
bdb8b9a347 test: doc: improve doc for from_hex helper (mention to_hex alternative) 2021-06-21 14:36:02 +02:00
Sebastian Falbesoner
1914054208 scripted-diff: test: rename FromHex to from_hex
-BEGIN VERIFY SCRIPT-
sed -i 's/\<FromHex\>/from_hex/g' $(git grep -l FromHex)
-END VERIFY SCRIPT-

Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2021-06-21 14:33:59 +02:00
Sebastian Falbesoner
a79396fe5f test: remove ToHex helper, use .serialize().hex() instead 2021-06-21 14:30:03 +02:00
Sebastian Falbesoner
2ce7b47958 test: introduce tx_from_hex helper for tx deserialization
`FromHex` is mostly used for transactions, so we introduce a
shortcut `tx_from_hex` for `FromHex(CTransaction, hex_str)`.
2021-06-21 14:28:05 +02:00
W. J. van der Laan
f6a25bea82
Merge bitcoin/bitcoin#22147: p2p: Protect last outbound HB compact block peer
30aee2dfe6 tests: Add test for compact block HB selection (Pieter Wuille)
6efbcec4de Protect last outbound HB compact block peer (Suhas Daftuar)

Pull request description:

  If all our high-bandwidth compact block serving peers (BIP 152) stall block
  download, then we can be denied a block for (potentially) a long time. As
  inbound connections are much more likely to be adversarial than outbound
  connections, mitigate this risk by never removing our last outbound HB peer if
  it would be replaced by an inbound.

ACKs for top commit:
  achow101:
    ACK 30aee2dfe6
  ariard:
    Code ACK 30aee2dfe
  jonatack:
    ACK 30aee2dfe6

Tree-SHA512: 5c6c9326e3667b97e0864c371ae2174d2be9054dad479f4366127b9cd3ac60ffa01ec9707b16ef29cac122db6916cf56fd9985733390017134ace483278921d5
2021-06-21 08:18:55 +02:00
nthumann
8abe5703a9
test: Add IPv6 test to zmq 2021-06-20 16:56:08 +02:00
MarcoFalke
e172ea8804
Merge bitcoin/bitcoin#22210: test: Use MiniWallet in test_no_inherited_signaling RBF test
fa7d71f270 test: Run pep-8 on touched test (MarcoFalke)
fab7e99c2a test: Use MiniWallet in test_no_inherited_signaling RBF test (MarcoFalke)
fab871f649 test: Remove unused generate() from test (MarcoFalke)
faff3f35b7 test: Add txin.sequence option to MiniWallet (MarcoFalke)

Pull request description:

  This comes with nice benefits:
  * Less code and complexity
  * Test can be run without wallet compiled in

  Also add some additional checks for `getmempoolentry` (#22209) and other cleanups 🎨

ACKs for top commit:
  mjdietzx:
    Tested ACK fa7d71f270 thanks for the explanations, nicely done
  theStack:
    ACK fa7d71f270 🍷

Tree-SHA512: 0e9b8fe985779d8d7034d256deed627125bb374b6ae2972c461b3a220739a51061c6147ad69339bee16282f82716c7f3f8a7a89c693ceb1e47ea50709272332a
2021-06-19 08:47:52 +02:00
MarcoFalke
222290f543
test: Set BIP34Height = 2 for regtest 2021-06-18 20:58:03 +02:00
Pieter Wuille
d637a9b397 Taproot descriptor inference 2021-06-18 11:28:47 -07:00
MarcoFalke
da1e6d5911
Merge bitcoin/bitcoin#14604: tests: Add test and refactor feature_block.py
55311197c4 Added new test for future blocks reacceptance (sanket1729)
511a5af462 Fixed inconsistencies between code and comments (sanket1729)

Pull request description:

  This Commit does 3 things:
  1) Adds a test case for checking reacceptance a previously rejected block which
  was too far in the future.
  ~~2) clean up uses of rehash or calc_sha256 where it was not needed~~
  3) While constructing block 44, this commit makes the code consistent with the expected figure in
  the comment just above it by adding a transaction to the block.
  4) Fix comment describing `sign_tx()` function

ACKs for top commit:
  duncandean:
    reACK 5531119
  brunoerg:
    reACK 55311197c4

Tree-SHA512: d40c72fcdbb0b2a0715adc58441eeea08147ee2ec5e371a4ccc824ebfdc6450698bd40aaeecb7ea7bfdb3cd1b264dd821b890276fff8b8d89b7225cdd9d6b546
2021-06-18 18:08:49 +02:00
MarcoFalke
fac90c55be
test: Create all blocks with version 4 or higher 2021-06-18 17:53:53 +02:00
MarcoFalke
0844084c13
Merge bitcoin/bitcoin#22249: test: kill process group to avoid dangling processes when using --failfast
451b96f7d2 test: kill process group to avoid dangling processes (S3RK)

Pull request description:

  This is an alternative to #19281

  This PR fixes a problem when after test failure with `--failfast` option there could be dangling nodes. The nodes will continue to occupy rpc/p2p ports on the machine and will cause further test failures.

  If there are any dangling nodes left at the end of the test run we kill the whole process group.
  Pros: the operations is immediate and won't lead to CI timeout
  Cons: the test_runner process is also killed and exit code is 137

  Example output:
  ```
  ...
  Early exiting after test failure

  TEST                           | STATUS    | DURATION

  rpc_decodescript.py            | ✓ Passed  | 2 s
  rpc_deprecated.py              | ✓ Passed  | 2 s
  rpc_deriveaddresses.py         | ✓ Passed  | 2 s
  rpc_dumptxoutset.py            | ✖ Failed  | 2 s

  ALL                            | ✖ Failed  | 8 s (accumulated)
  Runtime: 4 s

  Killed: 9
  > echo $?
  137
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 451b96f7d2
  aitorjs:
    ACK 451b96f7d2. Manual testing with and without **--failfast**.

Tree-SHA512: 87e510a1411b9e7571e63cf7ffc8b9a8935daf9112ffc0f069d6c406ba87743ec439808181f7e13cb97bb200fad528589786c47f0b43cf3a2ef0d06a23cb86dd
2021-06-18 14:22:51 +02:00
Samuel Dobson
5c2e2afe99
Merge bitcoin/bitcoin#21365: Basic Taproot signing support for descriptor wallets
458a345b05 Add support for SIGHASH_DEFAULT in RPCs, and make it default (Pieter Wuille)
c0f0c8eccb tests: check spending of P2TR (Pieter Wuille)
a2380127e9 Basic Taproot signing logic in script/sign.cpp (Pieter Wuille)
49487bc3b6 Make GetInputUTXO safer: verify non-witness UTXO match (Pieter Wuille)
fd3f6890f3 Construct and use PrecomputedTransactionData in PSBT signing (Pieter Wuille)
5cb6502ac5 Construct and use PrecomputedTransactionData in SignTransaction (Pieter Wuille)
5d2e22437b Don't nuke witness data when signing fails (Pieter Wuille)
ce9353164b Permit full precomputation in PrecomputedTransactionData (Pieter Wuille)
e841fb503d Add precomputed txdata support to MutableTransactionSignatureCreator (Pieter Wuille)
a91d532338 Add CKey::SignSchnorr function for BIP 340/341 signing (Pieter Wuille)
e77a2839b5 Use HandleMissingData also in CheckSchnorrSignature (Pieter Wuille)
dbb0ce9fbf Add TaprootSpendData data structure, equivalent to script map for P2[W]SH (Pieter Wuille)

Pull request description:

  Builds on top of #22051, adding signing support after derivation support.

  Nothing is changed in descriptor features. Signing works for key path and script path spending, through the normal sending functions, and PSBT-based RPCs. However, PSBT usability is rather low as no extensions have been defined to convey Taproot-specific information, so all script information must be known to the signing wallet.

ACKs for top commit:
  achow101:
    re-ACK 458a345b05
  fjahr:
    Code review ACK 458a345b05
  Sjors:
    ACK 458a345b05

Tree-SHA512: 30ed212cf7754763a4a81624ebc084c51727b8322711ac0b390369213c1a891d367ed8b123882ac08c99595320c11ec57ee42304ff22a69afdc3d1a0d55cc711
2021-06-18 09:12:44 +12:00
MarcoFalke
dd24567a24
Merge bitcoin/bitcoin#22120: test: p2p_invalid_block: Check that a block rejected due to too-new tim…
754e802274 test: check rejected future block later accepted (Luke Dashjr)

Pull request description:

  (Luke) was unsure if the code sufficiently avoided caching a
  time-too-new rejection, so wrote this test to check it.  It looks like
  despite only exempting BLOCK_MUTATED, it is still okay because header
  failures never cache block invalidity.  This test will help ensure that
  if this ever changes, BLOCK_TIME_FUTURE gets excluded at the same time.

  This PR re-opens https://github.com/bitcoin/bitcoin/pull/17872 which went stale and addresses the nits raised by reviewers there.

ACKs for top commit:
  MarcoFalke:
    review ACK 754e802274

Tree-SHA512: a2bbc8fffb523cf2831e1ecb05f20868e30106a38cc2e369e4973fa549cca06675a668df16f76c49cc4ce3a22925404255e5c53c4232d63ba1b9fca878509aa0
2021-06-17 09:05:52 +02:00
sanket1729
55311197c4 Added new test for future blocks reacceptance
Adds a test case for checking reacceptance a previously rejected block
that was too far in the future.
2021-06-16 16:43:20 -07:00
sanket1729
511a5af462 Fixed inconsistencies between code and comments
1) Makes the code for block 44 consistent with  the expected figure in
the comment above it by adding a transaction to the block
2) Fixed comment describing sign_tx() function
2021-06-16 16:43:20 -07:00
Luke Dashjr
754e802274
test: check rejected future block later accepted
(Luke) was unsure if the code sufficiently avoided caching a
time-too-new rejection, so wrote this test to check it.  It looks like
despite only exempting BLOCK_MUTATED, it is still okay because header
failures never cache block invalidity.  This test will help ensure that
if this ever changes, BLOCK_TIME_FUTURE gets excluded at the same time.

Co-authored-by: Will Clark <will8clark@gmail.com>
2021-06-15 21:35:29 +01:00
S3RK
451b96f7d2 test: kill process group to avoid dangling processes 2021-06-15 09:37:58 +02:00
MarcoFalke
964d91b193
Merge bitcoin/bitcoin#22130: test: refactor: dedup utility function chain_transaction()
01eedf3821 test: doc: improve doc for chain_transaction() helper (Sebastian Falbesoner)
6e63e366d6 test: refactor: dedup utility function chain_transaction() (Sebastian Falbesoner)

Pull request description:

  Both tests `mempool_packages.py` and `mempool_package_onemore.py` define a utility function `chain_transaction` with a similar implementation. This PR deduplicates it by moving it into the util package and keeping the more general properties:
  * pass a list of parent_txids/vouts instead of single values
  * always mark the BIP125-replaceable flag for txs, created via `createrawtransaction` (this is needed by the `mempool_package_onemore.py` test, but doesn't hurt the other one)

  This is a low-hanging fruit; as a potential follow-up one could probably also deduplicate the function `chain_transaction` in `rpc_packages.py`, which looks a bit different, as it also takes the parent locking script into account and doesn't send the tx.

ACKs for top commit:
  mjdietzx:
    reACK 01eedf3821
  klementtan:
    Code review ACK 01eedf3821
  MarcoFalke:
    review ACK 01eedf3821 🙅

Tree-SHA512: ac7105d02c23f53d76d4ec9dc8de1074dd8faefeecd44b107921b78665279498966152fed312ecbe252a1c34a9643d531166329a4fea0e773df3bb75d43092b0
2021-06-13 18:27:32 +02:00
MarcoFalke
9c1ec689f3
Merge bitcoin/bitcoin#22102: Remove Warning: from warning message printed for unknown new rules
6d7e46ce23 Remove `Warning:` (Prayank)

Pull request description:

  Reason: I noticed that `Warning` is printed 2 times in `-getinfo` while reviewing https://github.com/bitcoin/bitcoin/pull/21832#issuecomment-851004943

  Same string is used for GUI, log and stderr. If we need to add `Warning:` in GUI or other place we can always prepend to this string.

  CLI:

  ```
  Warnings: Unknown new rules activated (versionbit 28)

  ```

  GUI:

  ![image](https://user-images.githubusercontent.com/13405205/120110401-e36ab180-c18a-11eb-8031-4d52287dc263.png)

ACKs for top commit:
  MarcoFalke:
    review ACK 6d7e46ce23

Tree-SHA512: 25760cf6d850f6c2216d651fa46574d2d21a9d58f4577ecb58f9566ea8cd07bfcd6679bb8a1f53575e441b02d295306f492c0c99a48c909e60e5d977ec1861f6
2021-06-13 09:21:46 +02:00
Pieter Wuille
c0f0c8eccb tests: check spending of P2TR 2021-06-12 12:38:15 -07:00
W. J. van der Laan
b0e5fbf6fa
Merge bitcoin/bitcoin#22156: Allow tr() import only when Taproot is active
fbf485c9b2 Allow tr() import only when Taproot is active (Andrew Chow)

Pull request description:

  To avoid issues around fund loss, only allow descriptor wallets to import `tr()` descriptors after taproot has activated.

ACKs for top commit:
  sipa:
    utACK fbf485c9b2
  fjahr:
    Code review ACK fbf485c9b2
  laanwj:
    Code review ACK fbf485c9b2
  prayank23:
    utACK fbf485c9b2

Tree-SHA512: 83c43376515eea523dbc89bc5a0fde53e54aec492e49a40c2a33d80fc94aac459e232ae07b024b4bd75b58078c8d090bc7a2d69541c5d3d4834d2f4cfc9c8208
2021-06-12 17:22:41 +02:00
Vasil Dimov
7593b06bd1
test: ensure I2P addresses are relayed
This test would fail if `CNetAddr::IsRelayable()` returns `false` for
I2P addresses, given that this test node does not have I2P connectivity.
2021-06-11 19:17:59 +02:00
Vasil Dimov
e7468139a1
test: make CAddress in functional tests comparable
This way we can compare CAddress objects using `==` or even
arrays of CAddress using `array1 == array2`.
2021-06-11 19:16:07 +02:00
Vasil Dimov
33e211d2a4
test: implement ser/unser of I2P addresses in functional tests 2021-06-11 19:16:06 +02:00
Vasil Dimov
86742811ce
test: use NODE_* constants instead of magic numbers
We just assigned `NODE_NETWORK | NODE_WITNESS` to `nServices` a few
lines above. Use that for verifying correctness instead of `9`.
2021-06-11 13:52:42 +02:00
MarcoFalke
6985038046
Merge bitcoin/bitcoin#22118: test: check anchors.dat when node starts for the first time
ef99d03c2b test: check anchors.dat when node starts for the first time (bruno)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598406712, https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598406187, https://github.com/bitcoin/bitcoin/pull/21338#discussion_r598405613.

ACKs for top commit:
  laanwj:
    Code review ACK ef99d03c2b

Tree-SHA512: 505f1f34fbc0c72a92968883be0f1c5f169a4ba3aa8a56e1ce8bc5e514f49e3a17ce51fd40be0073dc4bc06eaeda36dfe90ace843c181170e34643226afd78ef
2021-06-11 08:23:02 +02:00
Andrew Chow
fbf485c9b2 Allow tr() import only when Taproot is active
To avoid issues around fund loss, only allow descriptor wallets
to import tr() descriptors after taproot has activated.
2021-06-10 15:45:47 -04:00
Pieter Wuille
30aee2dfe6 tests: Add test for compact block HB selection 2021-06-10 10:22:13 -04:00
MarcoFalke
fa7d71f270
test: Run pep-8 on touched test
Can be reviewed with --ignore-all-space
2021-06-10 13:39:24 +02:00
MarcoFalke
fab7e99c2a
test: Use MiniWallet in test_no_inherited_signaling RBF test 2021-06-10 13:38:56 +02:00
MarcoFalke
fab871f649
test: Remove unused generate() from test
This is already done by the test framework in setup_nodes()
2021-06-10 13:38:18 +02:00
MarcoFalke
faff3f35b7
test: Add txin.sequence option to MiniWallet 2021-06-10 13:38:10 +02:00
fanquake
ef8f2966ac
Merge bitcoin/bitcoin#22084: package testmempoolaccept followups
ee862d6efb MOVEONLY: context-free package policies (glozow)
5cac95cd15 disallow_mempool_conflicts -> allow_bip125_replacement and check earlier (glozow)
e8ecc621be [refactor] comment/naming improvements (glozow)
7d91442461 [rpc] reserve space in txns (glozow)
6c5f19d9c4 [package] static_assert max package size >= max tx size (glozow)

Pull request description:

  various followups from #20833

ACKs for top commit:
  jnewbery:
    utACK ee862d6efb
  ariard:
    Code Review ACK ee862d6

Tree-SHA512: 96ecb41f7bbced84d4253070f5274b7267607bfe4033e2bb0d2f55ec778cc41e811130b6321131e0418b5835894e510a4be8a0f822bc9d68d9224418359ac837
2021-06-10 19:09:54 +08:00
Sebastian Falbesoner
d6d2ab9845 test: MiniWallet: fix fee calculation for P2PK and check tx vsize 2021-06-08 19:38:43 +02:00
Sebastian Falbesoner
ce024b1c0e test: MiniWallet: force P2PK signature to have fixed size (71 bytes)
In order to enable exact fee calculation for transactions that spend
P2PK outputs in the MiniWallet, we enforce the created signatures to
have a fixed length (>49.89% probability) by default. With that it is
easier to check the created transactions vsize and avoid flaky tests
that would appear whenever the signatures R- or S-values are smaller
(due to leading zero bytes).

Note that to get the total scriptSig size one has to add another
2 bytes, as there is also the OP_PUSHx instruction on the front and
the sighash type byte on the back, leading to a final scriptSig size
of 73 bytes.
2021-06-08 19:38:17 +02:00
MarcoFalke
82bc7faec8
Merge bitcoin/bitcoin#21946: Document and test lack of inherited signaling in RBF policy
2eb0eeda39 validation: document lack of inherited signaling in RBF policy (Antoine Riard)
906b6d9da6 test: Extend feature_rbf.py with no inherited signaling (Antoine Riard)

Pull request description:

  Contrary to BIP125 or other full-node implementation (e.g btcd), Bitcoin Core's mempool policy doesn't implement inherited signaling.

  This PR documents our mempool behavior on this and add a test demonstrating the case.

ACKs for top commit:
  jonatack:
    ACK 2eb0eeda39
  benthecarman:
    ACK 2eb0eeda39

Tree-SHA512: d41453d3b49bae3c1eb532a968f43bc047084913bd285929d4d9cba142777ff2be38163d912e28dfc635f4ecf446de68effad799c6e71be52f81e83410c712fb
2021-06-08 17:00:28 +02:00
Daniel Kraft
7a681d61b0 Add sync_blocks in wallet_orphanedreward.py.
Add an explicit sync_blocks call in wallet_orphanedreward.py, which was
missing and could lead to intermittent failures of the test due to
race conditions.

This will presumably fix https://github.com/bitcoin/bitcoin/issues/22181.
2021-06-08 09:52:10 +02:00
Sebastian Falbesoner
01eedf3821 test: doc: improve doc for chain_transaction() helper
Change to docstring format and describe the functions
purpose, its parameters and return value in more detail.
2021-06-08 00:21:37 +02:00
Sebastian Falbesoner
6e63e366d6 test: refactor: dedup utility function chain_transaction() 2021-06-08 00:21:37 +02:00
Prayank
6d7e46ce23 Remove Warning:
+ Remove `Warning:` from warning message printed for unknown new rules
+ Change warning message in test

Author:    Prayank <prayank@tutanota.de>
2021-06-07 20:19:18 +05:30
W. J. van der Laan
3c393ef9e1
Merge bitcoin/bitcoin#22149: test: Add temporary logging to debug #20975
faa94961d6 test: Add temporary logging to debug #20975 (MarcoFalke)

Pull request description:

  to be reverted after a fix

ACKs for top commit:
  laanwj:
    Code review ACK faa94961d6

Tree-SHA512: 1f3103fcf4cad0af54e26c4d257bd824b128b5f2d2b81c302e861a829fd55d6a099fa476b79b30a71fe98975ae604b9e3ff31fd48a51d442389a9bd515e60ba0
2021-06-07 16:03:45 +02:00
fanquake
260b1d74fe
Merge bitcoin/bitcoin#22092: test: convert documentation into type annotations
68ace23fa3 test: convert docs into type annotations in test_framework/test_node.py (fanquake)
8bfcba36db test: convert docs into type annotations in test_framework/wallet.py (fanquake)
b043ca8e8b test: convert docs into type annotations in test_framework/util.py (fanquake)

Pull request description:

  Rather than having function types exist as documentation, make them type annotations, which enables more `mypy` checking.

ACKs for top commit:
  instagibbs:
    utACK 68ace23fa3

Tree-SHA512: b705f26b48baabde07b9b2c0a8d547b4dcca291b16eaf5ac70462bb3a1f9e9c2783d93a9d4290889d0cbb3f7db3671446754411a1f498b265d336f6ff33478df
2021-06-07 13:05:56 +08:00
MarcoFalke
898dd9e262
Merge bitcoin/bitcoin#22150: test: Remove unused node from feature_nulldummy
fa2b6c62cd test: Remove unused node from feature_nulldummy (MarcoFalke)

Pull request description:

  This is confusing and might even slow down the test.

  This reverts a change that was added a year ago in d438d60 and then the need for it was removed by 95d5d5e six months ago.

Top commit has no ACKs.

Tree-SHA512: 9a86792e9a634cf7bbd4e7a21b1acdfc3baba1b1962fe2b9b73848436d10351d2326dca01313c097ba2342dde7207add73e731d053c0bfa888a5d8f2b233a7cf
2021-06-05 08:41:37 +02:00
Martin Zumsande
ca3a77068b test: Fix p2p_leak.py intermittent failure by lowering timeout 2021-06-04 22:42:04 +02:00
MarcoFalke
fa2b6c62cd
test: Remove unused node from feature_nulldummy 2021-06-04 15:18:44 +02:00
MarcoFalke
faa94961d6
test: Add temporary logging to debug #20975 2021-06-04 15:05:27 +02:00
MarcoFalke
3ac5209662
Merge bitcoin/bitcoin#18795: Test: wallet issue with orphaned rewards
e4356f6a6c Testcase for wallet issue with orphaned rewards. (Daniel Kraft)

Pull request description:

  This adds a new test case demonstrating the wallet issue when block rewards are orphaned (#14148).

ACKs for top commit:
  LarryRuane:
    ACK e4356f6a6c
  leonardojobim:
    reACK e4356f6a6c .

Tree-SHA512: e9a2310ee1b3d52cfa302f431ed3d272bbc1b9195439ff318d9eb1006c0b28968dbe840e1600b6ff185e5d7ea57e4dcc837cef16051b5537445e10bc363b8c22
2021-06-04 13:54:34 +02:00
W. J. van der Laan
c7dd9ff71b
Merge bitcoin/bitcoin#22051: Basic Taproot derivation support for descriptors
2667366aaa tests: check derivation of P2TR (Pieter Wuille)
7cedafc541 Add tr() descriptor (derivation only, no signing) (Pieter Wuille)
90fcac365e Add TaprootBuilder class (Pieter Wuille)
5f6cc8daa8 Add XOnlyPubKey::CreateTapTweak (Pieter Wuille)
2fbfb1becb Make consensus checking of tweaks in pubkey.* Taproot-specific (Pieter Wuille)
a4bf84039c Separate WitnessV1Taproot variant in CTxDestination (Pieter Wuille)
41839bdb89 Avoid dependence on CTxDestination index order (Pieter Wuille)
31df02a070 Change Solver() output for WITNESS_V1_TAPROOT (Pieter Wuille)
4b1cc08f9f Make XOnlyPubKey act like byte container (Pieter Wuille)

Pull request description:

  This is a subset of #21365, to aide review.

  This adds support `tr(KEY)` or `tr(KEY,SCRIPT)` or `tr(KEY,{{S1,{{S2,S3},...}},...})` descriptors, describing Taproot outputs with specified internal key, and optionally any number of scripts, in nested groups of 2 inside `{`/`}` if there are more than one. While it permits importing `tr(KEY)`, anything beyond that is just laying foundations for more features later.

  Missing:
  * Signing support (see #21365)
  * Support for more interesting scripts inside the tree (only `pk(KEY)` is supported for now). In particular, a multisig policy based on the new `OP_CHECKSIGADD` opcode would be very useful.
  * Inferring `tr()` descriptors from outputs (given sufficient information).
  * `getaddressinfo` support.
  * MuSig support. Standardizing that is still an ongoing effort, and is generally kind of useless without corresponding PSBT support.
  * Convenient ways of constructing descriptors without spendable internal key (especially ones that arent't trivially recognizable as such).

ACKs for top commit:
  Sjors:
    utACK 2667366 (based on https://github.com/bitcoin/bitcoin/pull/21365#issuecomment-846945215 review, plus the new functional test)
  achow101:
    Code Review ACK 2667366aaa
  lsilva01:
    Tested ACK 2667366aaa
  meshcollider:
    utACK 2667366aaa

Tree-SHA512: 61046fef22c561228338cb178422f0b782ef6587ec8208d3ce2bd07afcff29a664b54b35c6b01226eb70b6540b43f6dd245043d09aa6cb6db1381b6042667e75
2021-06-03 21:58:41 +02:00
W. J. van der Laan
07ededa30c
Merge bitcoin/bitcoin#22050: p2p: remove tor v2 support
5d82a57db4 contrib: remove torv2 seed nodes (Jon Atack)
5f7e086dac contrib: update generate-seeds.py to ignore torv2 addresses (Jon Atack)
8be56f0f8e p2p, refactor: extract OnionToString() from CNetAddr::ToStringIp() (Jon Atack)
5f9d3c09b4 p2p: remove torv2 from CNetAddr::ToStringIP() (Jon Atack)
3d39042144 p2p: remove torv2 in SetIP() and ADDR_TORV2_SIZE constant (Jon Atack)
cff5ec477a p2p: remove pre-addrv2 onions from SerializeV1Array() (Jon Atack)
4192a74413 p2p: ignore torv2-in-ipv6 addresses in SetLegacyIPv6() (Jon Atack)
1d631e956f p2p: remove BIP155Network::TORV2 from GetBIP155Network() (Jon Atack)
7d1769bc45 p2p: remove torv2 from SetNetFromBIP155Network() (Jon Atack)
eba9a94b9f fuzz: rename CNetAddr/CService deserialize targets (Jon Atack)
c56a1c9b18 p2p: drop onions from IsAddrV1Compatible(), no longer relay torv2 (Jon Atack)
f8e94002fc p2p: remove torv2/ADDR_TORV2_SIZE from SetTor() (Jon Atack)
0f1c58ae87 test: update feature_proxy to torv3 (Jon Atack)

Pull request description:

  ![image](https://user-images.githubusercontent.com/2415484/120018909-4d425a00-bfd7-11eb-83c9-95a3dac97926.jpeg)

  This patch removes support in Bitcoin Core for Tor v2 onions, which are already removed from the release of Tor 0.4.6.

  - no longer serialize/deserialize and relay Tor v2 addresses
  - ignore incoming Tor v2 addresses
  - remove Tor v2 addresses from the addrman and peers.dat on node launch
  - update generate-seeds.py to ignore Tor v2 addresses
  - remove Tor v2 hard-coded seeds

  Tested with tor-0.4.6.1-alpha (no v2 support) and 0.4.5.7 (v2 support). With the latest Tor (no v2 support), this removes all the warnings like those reported with current master in https://github.com/bitcoin/bitcoin/issues/21351

  ```
  <bitcoind debug log>
  Socks5() connect to […].onion:8333 failed: general failure

  <tor log>
  Invalid hostname [scrubbed]; rejecting
  ```

  and the addrman no longer has Tor v2 addresses on launching bitcoind.
  ```rake
  $ ./src/bitcoin-cli -addrinfo
  {
    "addresses_known": {
      "ipv4": 44483,
      "ipv6": 8467,
      "torv2": 0,
      "torv3": 2296,
      "i2p": 6,
      "total": 55252
    }
  }
  ```
  After recompiling back to current master and restarting with either of the two Tor versions (0.4.5.7 or 0.4.6.1), -addrinfo initially returns 0 Tor v2 addresses and then begins finding them again.

  Ran nodes on this patch over the past week on mainnet/testnet/signet/regtest after building with DEBUG_ADDRMAN.

  Verified that this patch bootstraps an onlynet=onion node from the Tor v3 hardcoded fixed seeds on mainnet and testnet and connects to blocks and v3 onion peers: `rm ~/.bitcoin/testnet3/peers.dat ; ./src/bitcoind -testnet -dnsseed=0 -onlynet=onion`

  ![Screenshot from 2021-05-28 00-26-17](https://user-images.githubusercontent.com/2415484/119905021-ea02ea00-bf3a-11eb-875f-27ef57640c49.png)

  Tested using `addnode`, `getaddednodeinfo`,`addpeeraddress`, `disconnectnode` and `-addrinfo` that a currently valid, connectable Tor v2 peer can no longer be added:

  ![Screenshot from 2021-05-30 11-32-05](https://user-images.githubusercontent.com/2415484/120099282-29435d80-c12a-11eb-81b6-5084244d7d2a.png)

  Thanks to Vasil Dimov, Carl Dong, and Wladimir J. van der Laan for their work on BIP155 and Tor v3 that got us here.

ACKs for top commit:
  laanwj:
    Code review ACK 5d82a57db4

Tree-SHA512: 590ff3d2f6ef682608596facb4b01f44fef69716d2ab3552ae1655aa225f4bf104f9ee08d6769abb9982a8031de93340df553279ce1f5023771f9f2b651178bb
2021-06-03 18:43:55 +02:00
Christian Decker
b9e76f1bf0 rpc: Add test for -rpcwaittimeout
Suggested-by: Jon Atack <@jonatack>
2021-06-03 14:49:15 +02:00
fanquake
68ace23fa3
test: convert docs into type annotations in test_framework/test_node.py 2021-06-03 20:45:03 +08:00
fanquake
8bfcba36db
test: convert docs into type annotations in test_framework/wallet.py 2021-06-03 20:45:03 +08:00
fanquake
b043ca8e8b
test: convert docs into type annotations in test_framework/util.py 2021-06-03 20:45:03 +08:00
Daniel Kraft
e4356f6a6c Testcase for wallet issue with orphaned rewards.
This adds a new test case demonstrating the wallet issue when block
rewards are orphaned (https://github.com/bitcoin/bitcoin/issues/14148).
2021-06-02 13:53:50 +02:00
glozow
5cac95cd15 disallow_mempool_conflicts -> allow_bip125_replacement and check earlier 2021-06-02 09:52:50 +01:00
MarcoFalke
7e83e74e7f
Merge bitcoin/bitcoin#21178: test: run mempool_reorg.py even with wallet disabled
a3f0cbf82d test: run mempool_reorg.py even with wallet disabled (Darius Parvin)

Pull request description:

  Run mempool_reorg.py test even when the wallet is disabled, as discussed in #20078.

  As part of this PR I created a new method in `MiniWallet`, `create_self_transfer`, to return a raw tx (without broadcasting it) and its associated utxo.

ACKs for top commit:
  MarcoFalke:
    cr ACK a3f0cbf82d

Tree-SHA512: 316a38faffadcb87499c1d6eca21e9696cef65362bbffcf621788a9b771bb1fa2971b1c7835cbd34b952d7612ad83afbca824cd8be39ecd6b994e8963027f991
2021-06-01 15:05:45 +02:00
Sebastian Falbesoner
7e32fde912 test: feature_cltv.py: don't return tx copies in modification functions
The functions cltv_modify_tx(), cltv_invalidate() and cltv_validate()
all modify the passed transaction in-place, i.e. there is no need
to return a copy.
2021-05-31 19:47:30 +02:00
Sebastian Falbesoner
9ab2ce0a66 test: drop unused node parameters in feature_cltv.py
The following functions had a parameter "node" that is unsused
and hence can be removed:
    - cltv_modify_tx()
    - cltv_invalidate()
    - cltv_validate()
2021-05-31 19:34:00 +02:00
Sebastian Falbesoner
0c2139a3f1 test: fix typo in feature_cltv.py (s/ctlv/cltv/) 2021-05-31 19:18:04 +02:00
Darius Parvin
a3f0cbf82d test: run mempool_reorg.py even with wallet disabled
- run mempool_reorg.py even when the wallet is not compiled
- add `locktime` argument to `create_self_transfer` and `send_self_transfer`
- use more logs instead of comments
2021-05-31 09:28:10 -07:00
MarcoFalke
c5ee0cc11a
Merge bitcoin/bitcoin#21989: test: Use COINBASE_MATURITY in functional tests
bfa9309ad6 Use COINBASE_MATURITY constant in functional tests. (Kiminuo)
525448df9d Move COINBASE_MATURITY from `feature_nulldummy` test to `blocktools`. (Kiminuo)

Pull request description:

  `COINBASE_MATURITY` constant was added to `feature_nulldummy` test in #21373. This PR moves the constant to `blocktools.py` file and uses the constant in more tests as suggested [here](https://github.com/bitcoin/bitcoin/pull/21373#discussion_r605418462).

  Edit: Goal of this PR is to replace integer constants with `COINBASE_MATURITY` but not necessarily in *all* cases because that would mean to read and fully understand all tests. That's out of my time constraints. Any reports where `COINBASE_MATURITY` should be used are welcome though!

ACKs for top commit:
  theStack:
    ACK bfa9309ad6 🌇

Tree-SHA512: 01f04645f05a39028681f355cf3d42dd63ea3303f76d93c430e0fdce441934358a2d847a54e6068d61932f1b75e1d406f51859b057b3e4b569f7083915cb317f
2021-05-31 11:26:25 +02:00
Kiminuo
bfa9309ad6 Use COINBASE_MATURITY constant in functional tests. 2021-05-31 07:32:28 +02:00
MarcoFalke
d7a6bba949
Merge bitcoin/bitcoin#22103: test: Fix IPv6 check on BSD systems
2be3572506 test: Fix IPv6 check on BSD systems (nthumann)

Pull request description:

  I noticed that `test_ipv6_local()` always returns `False` on macOS or FreeBSD, even though IPv6 is working perfectly fine. This causes `test/functional/rpc_bind.py --ipv6` and `test/functional/feature_proxy.py` to skip their run.
  Apparently, there's a check if the port number is `0` (see [here](64881da478/sys/netinet6/udp6_usrreq.c (L248)) or [here](8f02f2a044/bsd/netinet6/udp6_usrreq.c (L282))), while Linux has no problem with this.
  This is fixed by specifying any other port number than `0`, e.g. `1`. Still, because of `SOCK_DGRAM`, no actual connection is made.

ACKs for top commit:
  fanquake:
    ACK 2be3572506 - nice improvement. I checked that with this change ipv6 related tests in `feature_proxy.py` are being run.
  theStack:
    ACK 2be3572506

Tree-SHA512: 8417c2d3cf71050529f3fa409a03872040fe5d249eae4172f276e62156e505a20d375b963712a186c9ad7967d8a497b5900d327c74a9693f68c33063871d4691
2021-05-31 07:24:54 +02:00
nthumann
2be3572506
test: Fix IPv6 check on BSD systems 2021-05-30 23:47:50 +02:00
bruno
ef99d03c2b test: check anchors.dat when node starts for the first time 2021-05-28 16:50:07 -03:00
W. J. van der Laan
7257e50dba
Merge bitcoin/bitcoin#20833: rpc/validation: enable packages through testmempoolaccept
13650fe2e5 [policy] detect unsorted packages (glozow)
9ef643e21b [doc] add release note for package testmempoolaccept (glozow)
c4259f4b7e [test] functional test for packages in RPCs (glozow)
9ede34a6f2 [rpc] allow multiple txns in testmempoolaccept (glozow)
ae8e6df709 [policy] limit package sizes (glozow)
c9e1a26d1f [fuzz] add ProcessNewPackage call in tx_pool fuzzer (glozow)
363e3d916c [test] unit tests for ProcessNewPackage (glozow)
cd9a11ac96 [test] make submit optional in CreateValidMempoolTransaction (glozow)
2ef187941d [validation] package validation for test accepts (glozow)
578148ded6 [validation] explicit Success/Failure ctors for MempoolAcceptResult (glozow)
b88d77aec5 [policy] Define packages (glozow)
249f43f3cc [refactor] add option to disable RBF (glozow)
897e348f59 [coins/mempool] extend CCoinsViewMemPool to track temporary coins (glozow)
42cf8b25df [validation] make CheckSequenceLocks context-free (glozow)

Pull request description:

  This PR enables validation dry-runs of packages through the `testmempoolaccept` RPC. The expectation is that the results returned from `testmempoolaccept` are what you'd get from test-then-submitting each transaction individually, in that order (this means the package is expected to be sorted in topological order, for now at least). The validation is also atomic: in the case of failure, it immediately halts and may return "unfinished" `MempoolAcceptResult`s for transactions that weren't fully validated. The API for 1 transaction stays the same.

  **Motivation:**
  - This allows you to test validity for transaction chains (e.g. with multiple spending paths and where you don't want to broadcast yet); closes #18480.
  - It's also a first step towards package validation in a minimally invasive way.
  - The RPC commit happens to close #21074 by clarifying the "allowed" key.

  There are a few added restrictions on the packages, mostly to simplify the logic for areas that aren't critical to main package use cases:
  - No package can have conflicts, i.e. none of them can spend the same inputs, even if it would be a valid BIP125 replacement.
  - The package cannot conflict with the mempool, i.e. RBF is disabled.
  - The total count of the package cannot exceed 25 (the default descendant count limit), and total size cannot exceed 101KvB (the default descendant size limit).

  If you're looking for review comments and github isn't loading them, I have a gist compiling some topics of discussion [here](https://gist.github.com/glozow/c3acaf161c95bba491fce31585b2aaf7)

ACKs for top commit:
  laanwj:
    Code review re-ACK 13650fe2e5
  jnewbery:
    Code review ACK 13650fe2e5
  ariard:
    ACK 13650fe

Tree-SHA512: 8c5cbfa91a6c714e1c8710bb281d5ff1c5af36741872a7c5df6b24874d6272b4a09f816cb8a4c7de33ef8e1c2a2c252c0df5105b7802f70bc6ff821ed7cc1a2f
2021-05-27 22:40:24 +02:00
Jon Atack
0f1c58ae87
test: update feature_proxy to torv3 2021-05-27 19:47:52 +02:00
fanquake
ecddd12482
Merge bitcoin/bitcoin#18418: wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100
e6fe1c37d0 rpc: Improve avoidpartialspends and avoid_reuse documentation (Fabian Jahr)
8f073076b1 wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100 (Fabian Jahr)

Pull request description:

  Follow-up to #17824.

  This increases OUTPUT_GROUP_MAX_ENTRIES to 100 which means that OutputGroups will now be up to 100 outputs large, up from previously 10. The main motivation for this change is that during the PR review club on #17824 [several participants signaled](https://bitcoincore.reviews/17824.html#l-339) that 100 might be a better value here.

  I think fees should be manageable for users but more importantly, users should know what they can expect when using the wallet with this configuration, so I also tried to clarify the documentation on `-avoidpartialspends` and `avoid_reuse` a bit. If there are other additional ways how or docs where users can be made aware of the potential consequences of using these parameters, please let me know. Another small upside is that [there seem to be a high number of batching transactions with 100 and 200 inputs](https://miro.medium.com/max/3628/1*sZ5eaBSbsJsHx-J9iztq2g.png)([source](https://medium.com/@hasufly/an-analysis-of-batching-in-bitcoin-9bdf81a394e0)) giving these transactions a bit of a larger anonymity set, although that is probably a very weak argument.

ACKs for top commit:
  jnewbery:
    ACK e6fe1c37d0
  Xekyo:
    retACK e6fe1c37d0
  rajarshimaitra:
    tACK `e6fe1c3`
  achow101:
    ACK e6fe1c37d0
  glozow:
    code review ACK e6fe1c37d0

Tree-SHA512: 79685c58bafa64ed8303b0ecd616fce50fc9a2b758aa79833e4ad9f15760e09ab60c007bc16ab4cbc4222e644cfd154f1fa494b0f3a5d86faede7af33a6f2826
2021-05-26 19:32:51 +08:00
Sebastian Falbesoner
3e05a57297 test: use MiniWallet (P2PK mode) for feature_dersig.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-25 14:39:45 +02:00
MarcoFalke
40f7a2891f
Merge bitcoin/bitcoin#22043: rpc, test: addpeeraddress test coverage, code simplify/constness
b36e0cd1b9 rpc: simplify addpeeraddress and improve code constness (Jon Atack)
6b1926cf1e test: addpeeraddress functional test coverage (Jon Atack)

Pull request description:

  - Add functional test coverage for rpc addpeeraddress
  - Simplify addpeeraddress and improve code constness

ACKs for top commit:
  klementtan:
    ACK [`b36e0cd`](b36e0cd1b9)
  MarcoFalke:
    review ACK b36e0cd1b9 💭

Tree-SHA512: 01773fb70f23db5abf46806bb27804e48feff27272b2e6582bd5b886e9715088eb2d84755106bce2ad6f88e21582f7f071a30a89d5b17286d899c3dd8553b4fc
2021-05-25 09:40:00 +02:00
Pieter Wuille
2667366aaa tests: check derivation of P2TR 2021-05-24 14:59:34 -07:00
glozow
13650fe2e5 [policy] detect unsorted packages 2021-05-24 15:48:32 +01:00
glozow
c4259f4b7e [test] functional test for packages in RPCs 2021-05-24 15:48:32 +01:00
glozow
9ede34a6f2 [rpc] allow multiple txns in testmempoolaccept
Only allow "packages" with no conflicts, sorted in order of dependency,
and no more than 25 for now.  Note that these groups of transactions
don't necessarily need to adhere to some strict definition of a package
or have any dependency relationships. Clients are free to pass in a
batch of 25 unrelated transactions if they want to.
2021-05-24 15:45:01 +01:00
Sebastian Falbesoner
6cebac598e test: MiniWallet: introduce enum type for output mode
For the MiniWallet constructor, the two boolean parameters
"raw_script" and "use_p2pk" are replaced by a single parameter of the
newly introduced type MiniWalletMode (derived by enum.Enum), which can
hold the following values:
	- ADDRESS_OP_TRUE
	- RAW_OP_TRUE
	- RAW_P2PK
2021-05-24 16:10:49 +02:00
MarcoFalke
778b920179
Merge bitcoin/bitcoin#21945: test: add P2PK support to MiniWallet
4bea301692 test: use P2PK-MiniWallet for feature_csv_activation.py (Sebastian Falbesoner)
dc7eb64e83 test: MiniWallet: add P2PK support (Sebastian Falbesoner)

Pull request description:

  This PR adds support for creating and spending transactions with raw pubkey (P2PK) outputs to MiniWallet, [as suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/21900#discussion_r629524841). Using that  mode in the test `feature_csv_activation.py`, all txs submitted to the mempool follow the standard policy, i.e. `-acceptnonstdtxn=1` can be removed.

  Possible follow-ups:
  * Improve MiniWallet constructor Interface; an enum-like parameter instead of two booleans would probably be better
  * Look at other tests that could benefit from P2PK (e.g. feature_cltv.py?)
  * Check vsize also for P2PK txs (vsize varies due to signature, i.e. a range has to be asserted)

ACKs for top commit:
  laanwj:
    Code review ACK 4bea301692

Tree-SHA512: 9b428e6b7cfde59a8c7955d5096cea88af1384a5f49723f00052e9884d819d952d20a5ab39bb02f9d8b6073769c44462aa265d84a33e33da33c2d21670c488a6
2021-05-24 08:42:59 +02:00
Jon Atack
6b1926cf1e
test: addpeeraddress functional test coverage 2021-05-23 10:39:08 +02:00
Kiminuo
525448df9d Move COINBASE_MATURITY from feature_nulldummy test to blocktools. 2021-05-21 14:33:40 +02:00
MarcoFalke
fad0867d6a
Cleanup -includeconf error message
Remove the erroneous trailing newline '\n'. Also, print only the first
value to remove needless redundancy in the error message.
2021-05-21 10:54:12 +02:00
MarcoFalke
fa9f711c37
Fix crash when parsing command line with -noincludeconf=0 2021-05-21 10:53:09 +02:00
Jon Atack
3f89c0e990
test: improve getnodeaddresses coverage, test by network 2021-05-19 13:06:04 +02:00
Fabian Jahr
8f073076b1
wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100 2021-05-18 02:09:18 +02:00
Sebastian Falbesoner
4bea301692 test: use P2PK-MiniWallet for feature_csv_activation.py
Using the MiniWallet in P2PK mode, all transactions submitted to the
mempool are following the standard policy now, i.e. the node command
line parameter '-acceptnonstdtxn=1' is not needed anymore.
2021-05-17 20:12:34 +02:00
Sebastian Falbesoner
dc7eb64e83 test: MiniWallet: add P2PK support 2021-05-17 19:50:31 +02:00
Antoine Riard
906b6d9da6 test: Extend feature_rbf.py with no inherited signaling 2021-05-14 14:12:29 -04:00
MarcoFalke
faad68fcd4
index: Avoid async shutdown on init error 2021-05-12 10:47:35 +02:00
MarcoFalke
d8ae29ec8f
Merge bitcoin/bitcoin#21900: test: use MiniWallet for feature_csv_activation.py
bd7f27d16d refactor: feature_csv_activation.py: move tx helper functions to methods (Sebastian Falbesoner)
2eca46b0aa test: use MiniWallet for feature_csv_activation.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (feature_csv_activation.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078.

  Short reviewers guideline:
  - Since we exclusively work with anyone-can-spend outputs here (raw scriptPubKey = OP_TRUE), signing is not needed anymore. The function `sign_transaction` and its calls are removed, after changing a tx (e.g. its scriptSig or nVersion) a simple `.rehash()` call is sufficient. Also, generating an address `self.nodeaddress` (and with that, passing it to the the various test tx creation/sending helper methods) is not needed anymore and removed.
  - The test repeatedly uses the same input for creating different txs (e.g. with different txversions 1 and 2). To let `MiniWallet` create a tx with a specific input, we have to call `.get_utxo()` before which also marks the UTXO as spent. The method is changed to also support keeping the UTXO in its internal list (`mark_as_spent=False`). With the behaviour on master, the second call to `.get_utxo()` with the same input would fail.
  - To keep the diff in the first commit short, the `miniwallet` is set as a global variable, to avoid passing it on every tx creation/spending helper. The global is eliminated in the second (refactoring) commit, where all the helpers are moved to the test class as methods. By that, we can use `self.nodes[0]` directly in the helpers and don't have to pass it again and again. I think there could still be a lot of improvements/refactoring done in the test, but that should hopefully serve as a good basis.

ACKs for top commit:
  laanwj:
    Code review ACK bd7f27d16d
  MarcoFalke:
    review ACK bd7f27d16d 🐕

Tree-SHA512: 24fb6a0f7702bae40d5271d197119827067d4b597e954d182e4c1aa5d0fa870368eb3ffed469b26713fa8ff8eb3ecc06abc80b2449cd68156d5559e7ae8a2b11
2021-05-10 17:50:21 +02:00
W. J. van der Laan
32692d2681
Merge bitcoin/bitcoin#21359: rpc: include_unsafe option for fundrawtransaction
11d6459b6e rpc: include_unsafe option for fundrawtransaction (t-bast)

Pull request description:

  Allow RPC users to opt-in to unsafe inputs when funding a raw transaction.

  Applications that need to manage a complex RBF flow (such as lightning nodes using anchor outputs) are very limited if they can only use safe inputs.

  I also added this option to `send` and `walletcreatefundedpsbt` who internally delegate to `fundrawtransaction`.

  Fixes #21299

ACKs for top commit:
  laanwj:
    Code review ACK 11d6459b6e

Tree-SHA512: 5e542a4febcfd6f41cf784678ff02ec9282eae2082c274983f72c5ea87b7ebbe1bd5fdc6a020d7a9d5996157754eb4966b8aeb6c1ceebf0b1519f735579b8bac
2021-05-10 16:05:23 +02:00
MarcoFalke
adf7843410
Merge bitcoin/bitcoin#21786: wallet: ensure sat/vB feerates are in range (mantissa of 3)
847288df07 test: fee rate values that cannot be represented as sat/vB (Jon Atack)
06a90fa038 rpc: for sat/vB fee rates, limit ParseFixedPoint decimals to 3 (Jon Atack)
0742c7840f rpc: enable passing decimals to AmountFromValue, add doxygen (Jon Atack)
8ce3ef57a3 test: ParseFixedPoint with 3 decimals for sat/vB fee rates (Jon Atack)
b503327597 test: type error and out of range fee rates where missing (Jon Atack)
c5fd4344f7 test: explicit fee rates with invalid amounts (Jon Atack)
ea6f76b66e test: improve zero-value explicit fee rate coverage (Jon Atack)

Pull request description:

  - Improve/close gaps in existing test coverage before making the change
  - Enable passing `decimals` to `ParseFixedPoint()` when calling `AmountFromValue()`
  - Limit explicit fee rates in sat/vB passed in by users to 3 decimals, and raise otherwise
  - Add regression test coverage

  Closes #20534.

ACKs for top commit:
  MarcoFalke:
    review ACK 847288df07 🔷

Tree-SHA512: c539d07ae9b21c0d6c8ea460beb9c8dad5559445518aace560abc3c05c588907bae189b6fd7602b3b397de4a42356136c3ec6f960d3dcf2d5d16377aef4ab5a2
2021-05-10 09:02:48 +02:00
Sebastian Falbesoner
bd7f27d16d refactor: feature_csv_activation.py: move tx helper functions to methods
This allows to get rid of the global miniwallet variable and to specify
the used node self.nodes[0] at only one place, instead of passing it to
every tx creation/send method again and again.

Can be reviewed with --ignore-all-space --color-moved=dimmed-zebra
2021-05-10 01:31:33 +02:00
Sebastian Falbesoner
2eca46b0aa test: use MiniWallet for feature_csv_activation.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-10 01:31:27 +02:00
Jon Atack
847288df07
test: fee rate values that cannot be represented as sat/vB 2021-05-09 12:50:04 +02:00
Jon Atack
06a90fa038
rpc: for sat/vB fee rates, limit ParseFixedPoint decimals to 3 2021-05-09 12:50:02 +02:00
Jon Atack
b503327597
test: type error and out of range fee rates where missing 2021-05-09 12:49:56 +02:00
Jon Atack
c5fd4344f7
test: explicit fee rates with invalid amounts 2021-05-09 12:49:47 +02:00
MarcoFalke
5925f1e652
Merge bitcoin/bitcoin#21872: net: Sanitize message type for logging
09205b33aa net: Clarify message header validation errors (W. J. van der Laan)
955eee7680 net: Sanitize message type for logging (W. J. van der Laan)

Pull request description:

  - Use `SanitizeString` when logging message errors to make sure that the message type is sanitized. I have checked all logging in `net.cpp`.

  - For the `MESSAGESTART` error don't inspect and log header details at all: receiving invalid start bytes makes it likely that the packet isn't even formatted as valid P2P message. Logging the four unexpected start bytes (as hex) should be enough.

  - Update `p2p_invalid_messages.py` test to check this.

  - Improve error messages in a second commit.

  Issue reported by gmaxwell.

ACKs for top commit:
  MarcoFalke:
    re-ACK 09205b33aa only change is log message fixup 🔂
  practicalswift:
    re-ACK 09205b33aa

Tree-SHA512: 8fe5326af135cfcf39ea953d9074a8c966b9b85a810b06a2c45b8a745cf115de4f321e72fc769709d6bbecfc5953aab83176db6735b04c0bc6796f59272cadce
2021-05-09 07:01:26 +02:00
W. J. van der Laan
09205b33aa net: Clarify message header validation errors
Make the errors less shouty and more descriptive.
2021-05-07 11:28:21 +02:00
W. J. van der Laan
955eee7680 net: Sanitize message type for logging
- Use `SanitizeString` when logging message errors to make sure that the
message type is sanitized.

- For the `MESSAGESTART` error don't inspect and log header details at
all: receiving invalid start bytes makes it likely that the packet isn't
even formatted as valid P2P message. Logging the four unexpected start
bytes should be enough.

- Update `p2p_invalid_messages.py` test to check this.

Issue reported by gmaxwell.
2021-05-06 17:30:52 +02:00
MarcoFalke
06d573f053
Merge bitcoin/bitcoin#21867: test: use MiniWallet for p2p_blocksonly.py
9f767e8438 test: use MiniWallet for p2p_blocksonly.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (p2p_blocksonly.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078.

  Note that MiniWallet creates segwit transactions by default, i.e. txid and wtxid are not identical and we have to return both from `check_p2p_tx_violation(...)`: wtxid is needed to match an expected `"received getdata for: wtx ..."` debug output, whereas the txid is needed to wait for a certain tx via `wait_for_tx(...)`.

ACKs for top commit:
  jonatack:
    ACK 9f767e8438 tested with `--disable-wallet`

Tree-SHA512: f08001f02c3c310ccdf713af0ba17304368a36414f412749908bbe8c03ad1e902190b8768b79f3b4909855762f285e7ab1b627cc4f45c90b42bb097a43cb4318
2021-05-06 16:52:55 +02:00
Jon Atack
ea6f76b66e
test: improve zero-value explicit fee rate coverage 2021-05-06 09:06:27 +02:00
Sebastian Falbesoner
9f767e8438 test: use MiniWallet for p2p_blocksonly.py
This test can now be run even with the Bitcoin Core wallet disabled.
2021-05-06 02:30:05 +02:00
MarcoFalke
128b98fce3
Merge bitcoin/bitcoin#21681: validation: fix ActivateSnapshot to use hardcoded nChainTx
91d93aac4e validation: remove nchaintx from assumeutxo metadata (James O'Beirne)
931684b24a validation: fix ActivateSnapshot to use hardcoded nChainTx (James O'Beirne)

Pull request description:

  This fixes an oversight from the move of nChainTx from the user-supplied
  snapshot metadata into the hardcoded assumeutxo chainparams.

  Since the nChainTx is now unused in the metadata, it should be removed
  in a future commit.

  See: https://github.com/bitcoin/bitcoin/pull/19806#discussion_r612165410

ACKs for top commit:
  Sjors:
    utACK 91d93aac4e
  ryanofsky:
    Code review ACK 91d93aac4e. No change to previous commit, just new commit removing now unused utxo snapshot field and updating tests.

Tree-SHA512: 445bdd738faf007451f40bbcf360dd1fb4675e17a4c96546e6818c12e33dd336dadd95cf8d4b5f8df1d6ccfbc4bf5496864bb5528e416cea894857b6b732140c
2021-05-05 18:34:16 +02:00
W. J. van der Laan
1b9a5236e9
Merge bitcoin/bitcoin#21740: test: add new python linter to check file names and permissions
46b025e00d test: add new python linter to check file names and permissions (windsok)
6f6bb3ebc7 test: fix file permissions on various scripts (windsok)

Pull request description:

  Adds a new python linter test which tests for correct filenames and file permissions in the repository.

  Replaces the existing tests in the `test/lint/lint-filenames.sh` and `test/lint/lint-shebang.sh` linter tests, as well as adding some new and increased testing. This increased coverage is intended to catch issues such as in #21728 and https://github.com/bitcoin/bitcoin/pull/16807/files#r345547050

  Summary of tests:
  * Checks every file in the repository against an allowed regexp to make sure only lowercase or uppercase alphanumerics (a-zA-Z0-9), underscores (_), hyphens (-), at (@) and dots (.) are used in repository filenames.

  * Checks only source files (*.cpp, *.h, *.py, *.sh) against a stricter allowed regexp to make sure only lowercase alphanumerics (a-z0-9), underscores (_), hyphens (-) and dots (.) are used in source code filenames. Additionally there is an exception regexp for directories or files which are excepted from matching this regexp (This should replicate the existing `test/lint/lint-filenames.sh` test)

  * Checks all files in the repository match an allowed executable or non-executable file permission octal. Additionally checks that for executable files, the file contains a shebang line.

  * Checks that for executable `.py` and `.sh` files, the shebang line used matches an allowable list of shebangs (This should replicate the existing `test/lint/lint-shebang.sh` test)

  * Checks every file that contains a shebang line to ensure it has an executable permission

  Additionally updates the permissions on various files to comply with the new tests.

  Fixes #21729

ACKs for top commit:
  practicalswift:
    cr re-ACK 46b025e00d: patch still looks correct
  kiminuo:
    code review ACK 46b025e00d if `contrib/gitian-descriptors/assign_DISTNAME` permission change is deemed OK.
  laanwj:
    Code review ACK 46b025e00d

Tree-SHA512: 1c8201a2cee0d9cbce15652b68cec9a6458a8b493fcd5392f98560aca0b1a12e668baab65a47100f116f626dadc3f591deb47f7368468c6a46c6c712c2533455
2021-05-05 17:14:22 +02:00
MarcoFalke
a0f46028fc
Merge bitcoin/bitcoin#21814: test: Fix feature_config_args.py intermittent issue
fab1eb65b1 test: Fix feature_config_args.py intermittent issue (MarcoFalke)

Pull request description:

  Fix #21448

ACKs for top commit:
  laanwj:
    Code review ACK fab1eb65b1

Tree-SHA512: cad9f684f43aa801d0c1cb5f1684ffa624df1216be225cea46b1389ba2b67cbd6159ffb786fe144bf1ca865623dd9a10289d4293cfabb678bdb243d4ea00734d
2021-05-05 16:46:24 +02:00
W. J. van der Laan
fb43d7cb1e
Merge bitcoin/bitcoin#21787: test: fix off-by-ones in rpc_fundrawtransaction assertions
f09e6b2585 test: fix off-by-ones in rpc_fundrawtransaction (Jon Atack)

Pull request description:

  The variables in these assertions should be the same within each line.

ACKs for top commit:
  laanwj:
    ACK f09e6b2585
  theStack:
    ACK f09e6b2585

Tree-SHA512: 7ac754eaadd8cb00a725afa55bccbb8de7547dedac9350d79a9a470918245617e075c56a91adc36fb653bbe8a0a325d59b00443155a7e1a81ebf22e4e4cf56d9
2021-05-05 15:32:28 +02:00
MarcoFalke
779aaa7f03
Merge bitcoin/bitcoin#21754: test: Run feature_cltv with MiniWallet
fa066f1b66 test: Run feature_cltv with MiniWallet (MarcoFalke)
fa5591d38c test: Hide tx rehash in helper (MarcoFalke)
fa5f938cfe test: Remove new_tx reference (MarcoFalke)

Pull request description:

  Allows to run the test even with no wallet compiled in

ACKs for top commit:
  theStack:
    ACK fa066f1b66 💽

Tree-SHA512: 3f659a178ba3ee0baffd70fddf8b8a68e5551d85626c7f254b234d7f75e6a16430a32a7952037db358b579f045b4d296b46156f72e5d226f3e80334dc635ca10
2021-05-05 08:47:28 +02:00
fanquake
60132382a7
Merge bitcoin/bitcoin#20867: Support up to 20 keys for multisig under Segwit context
ebd4be43cc doc: add release notes for 20867 (Antoine Poinsot)
5aa50ab9cc rpc/util: multisig: only check redeemScript size is <= 520 for P2SH (Antoine Poinsot)
063df9e897 test/functional: standardness sanity checks for P2(W)SH multisig (Antoine Poinsot)
ae0429d3af script: allow up to 20 keys in wsh() descriptors (Antoine Poinsot)
9fc68faf35 script: match multisigs with up to MAX_PUBKEYS_PER_MULTISIG keys (Antoine Poinsot)

Pull request description:

  As described in https://github.com/bitcoin/bitcoin/issues/20620 multisigs are currently limited to 16 keys in descriptors and RPC helpers, even for P2WSH and P2SH-P2WSH.

  This adds support for multisig with up to 20 keys (which are already standard) for Segwit v0 context for descriptors (`wsh()`, `sh(wsh())`) and RPC helpers.

  Fixes https://github.com/bitcoin/bitcoin/issues/20620

ACKs for top commit:
  meshcollider:
    re-utACK ebd4be43cc
  instagibbs:
    re-ACK ebd4be43cc

Tree-SHA512: 36141f10a8288010d17d5c4fe8d24878bcd4533b88a8aba3a44fa8f74ceb3182d70fee01427e0ab7f53ce7fab46c88c1cd3ac3b18ab8a10bd4a6b8b74ed79e46
2021-05-03 12:44:23 +08:00
fanquake
59869704c0
Merge bitcoin/bitcoin#21818: doc: fixup -coinstatsindex help, update bitcoin.conf and files.md
54133c59b8 doc: add indexes/coinstats/db/ to files.md (Jon Atack)
5d1050f516 doc: fix -coinstatsindex help, and test/rpc touchups (Jon Atack)
e041ee0a80 doc: add coinstatsindex to bitcoin.conf (Jon Atack)

Pull request description:

ACKs for top commit:
  Sjors:
    utACK 54133c59b8
  MarcoFalke:
    cr ACK 54133c59b8
  clarkmoody:
    utACK 54133c5

Tree-SHA512: 1a7f3e89873b7dc79ec71d5d39e9e3e4977ce43cc4bee208ad55291bef1bb319a9d1c34ed84a87d6a803db983bdfd0af4d9f396cec0bec86b1701ebbb6f34378
2021-05-02 13:50:38 +08:00
Jon Atack
5d1050f516
doc: fix -coinstatsindex help, and test/rpc touchups 2021-05-01 13:57:26 +02:00
MarcoFalke
d22e7ee933
Merge bitcoin/bitcoin#21822: test: resolve bug in interface_bitcoin_cli.py
c5bb142817 test: resolve bug in test/functional/interface_bitcoin_cli.py - Test -getinfo with -rpcwallet=unloaded wallet returns no balances (klementtan)

Pull request description:

  I think there is a bug in this test case where the new value of `cli_get_info` is not asserted.

ACKs for top commit:
  jonatack:
    ACK c5bb142817

Tree-SHA512: 50c0c2c8fe63c95f951dee892fbacedf92208f47efe5ed481fbb255f15137c799d9200fa3ff31a442df0691248d7ff04d899842722c3032cd7f35553622ba38c
2021-05-01 13:12:37 +02:00
klementtan
c5bb142817 test: resolve bug in test/functional/interface_bitcoin_cli.py - Test -getinfo with -rpcwallet=unloaded wallet returns no balances 2021-05-01 17:58:01 +08:00
t-bast
11d6459b6e
rpc: include_unsafe option for fundrawtransaction
Allow RPC users to opt-in to unsafe inputs when funding a raw transaction.

Applications that need to manage a complex RBF flow (such as lightning
nodes using anchor outputs) are very limited if they can only use safe inputs.

Fixes #21299
2021-04-30 18:53:47 +02:00
W. J. van der Laan
2b45cf0bcd
Merge bitcoin/bitcoin#19521: Coinstats Index
5f96d7d22d rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height (Fabian Jahr)
23fe50436b test: Add test for coinstatsindex behavior in reorgs (Fabian Jahr)
90c966b0f3 rpc: Allow gettxoutsetinfo and getblockstats for stale blocks (Fabian Jahr)
b9362392ae index, rpc: Add use_index option for gettxoutsetinfo (Fabian Jahr)
bb7788b121 test: Test coinstatsindex robustness across restarts (Fabian Jahr)
e0938c2909 test: Add tests for block_info in gettxoutsetinfo (Fabian Jahr)
2501576ecc rpc, index: Add verbose amounts tracking to Coinstats index (Fabian Jahr)
655d929836 test: add coinstatsindex getindexinfo coverage, improve current tests (Jon Atack)
ca01bb8d68 rpc: Add Coinstats index to getindexinfo (Fabian Jahr)
57a026c30f test: Add unit test for Coinstats index (Fabian Jahr)
6a4c0c09ab test: Add functional test for Coinstats index (Fabian Jahr)
3f166ecc12 rpc: gettxoutsetinfo can be requested for specific blockheights (Fabian Jahr)
3c914d58ff index: Coinstats index can be activated with command line flag (Fabian Jahr)
dd58a4de21 index: Add Coinstats index (Fabian Jahr)
a8a46c4b3c refactor: Simplify ApplyStats and ApplyHash (Fabian Jahr)
9c8a265fd2 refactor: Pass hash_type to CoinsStats in stats object (Fabian Jahr)
2e2648a902 crypto: Make MuHash Remove method efficient (Fabian Jahr)

Pull request description:

  This is part of the coinstats index project tracked in #18000

  While the review of the new UTXO set hash algorithm (MuHash) takes longer recently #19328 was merged which added the possibility to run `gettxoutsetinfo` with a specific hash type. As the first type it added `hash_type=none` which skips the hashing of the UTXO set altogether. This alone did not make `gettxoutsetinfo` much faster but it allows the use of an index for the remaining coin statistics even before a new hashing algorithm has been added. Credit to Sjors for the idea to take this intermediate step.

  Features summary:
  - Users can start their node with the option `-coinstatsindex` which syncs the index in the background
  - After the index is synced the user can  use `gettxoutsetinfo` with `hash_type=none` or `hash_type=muhash` and will get the response instantly out of the index
  - The user can specify a height or block hash when calling `gettxoutsetinfo` to see coin statistics at a specific block height

ACKs for top commit:
  Sjors:
    re-tACK 5f96d7d22d
  jonatack:
    Code review re-ACK 5f96d7d22d per `git range-diff 13d27b4 07201d3 5f96d7d`
  promag:
    Tested ACK 5f96d7d22d. Light code review ACK 5f96d7d22d.

Tree-SHA512: cbca78bee8e9605c19da4fbcd184625fb280200718396c694a56c7daab6f44ad23ca9fb5456d09f245d8b8d9659fdc2b3f3ce5e953c1c6cf4003dbc74c0463c2
2021-04-30 17:27:19 +02:00
MarcoFalke
fab1eb65b1
test: Fix feature_config_args.py intermittent issue 2021-04-30 10:49:49 +02:00
MarcoFalke
fad6269916
test: Assert that exit code indicates failure 2021-04-29 13:03:12 +02:00
MarcoFalke
faecb72c3c
test: Fix intermittent issue in p2p_segwit.py 2021-04-29 13:02:33 +02:00
John Newbery
a33bdb52d1 [tests] Speed up p2p_segwit.py
Never sleep for more than 5 seconds when waiting for an
inv-getdata exchange to time out.
2021-04-29 09:17:56 +01:00
MarcoFalke
fa066f1b66
test: Run feature_cltv with MiniWallet 2021-04-29 07:18:28 +02:00
MarcoFalke
fa5591d38c
test: Hide tx rehash in helper
This avoids code-bloat and accidentally forgetting to call it
2021-04-29 07:17:36 +02:00
MarcoFalke
fa5f938cfe
test: Remove new_tx reference
new_tx is a reference to tx. To avoid confusion and code-bloat, remove it
2021-04-29 07:17:32 +02:00
MarcoFalke
fb66dbe786
Merge bitcoin/bitcoin#21762: test: Speed up mempool_spend_coinbase.py
fa40eb5b6b test: Speed up mempool_spend_coinbase.py (MarcoFalke)
fa29382ab2 test: Fix test cache issue (MarcoFalke)
fa085b470a test: Create MiniWallet.create_self_transfer (MarcoFalke)
fa1bedb494 test: Add MiniWallet.sendrawtransaction (MarcoFalke)

Pull request description:

  Locally the test will run 4 seconds faster with `--valgrind` (18s vs 14s)

ACKs for top commit:
  mjdietzx:
    crACK fa40eb5b6b

Tree-SHA512: ecfb60dda5ca5d7e6367bb9c6210390d95ebf6396ce657728901d118b75bb90c98f9351df3b01004d00682234448d6c6a13338d12097f7dced2cf7f1bd84d924
2021-04-29 07:13:49 +02:00
fanquake
7f37a1d560
Merge bitcoin/bitcoin#21785: test: Fix intermittent issue in p2p_addr_relay.py
faa51ef4d3 test: Fix intermittent issue in p2p_addr_relay.py (MarcoFalke)
fa37116c82 test: Use self.send_addr_msg (MarcoFalke)

Pull request description:

  Fixes https://github.com/bitcoin/bitcoin/pull/21707/files#r621048952

ACKs for top commit:
  mzumsande:
    ACK faa51ef4d3
  amitiuttarwar:
    ACK faa51ef4d3

Tree-SHA512: b1fbc1a4a13b020890b465083a82762981de17dd710388c651353aefc28dee23d39803c2a1dadbf88287e92598901afcb01b488115b0fecd746ab6d73775415b
2021-04-29 10:08:27 +08:00
Antoine Poinsot
063df9e897
test/functional: standardness sanity checks for P2(W)SH multisig
Note that it also test for sortedmulti(), which the previous commit didn't.

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2021-04-28 10:00:30 +02:00
MarcoFalke
faa51ef4d3
test: Fix intermittent issue in p2p_addr_relay.py 2021-04-28 08:16:29 +02:00
MarcoFalke
fa37116c82
test: Use self.send_addr_msg 2021-04-28 08:13:05 +02:00
Jon Atack
f09e6b2585
test: fix off-by-ones in rpc_fundrawtransaction 2021-04-27 17:13:38 +02:00
W. J. van der Laan
ac219dcbcc
Merge bitcoin/bitcoin#19160: multiprocess: Add basic spawn and IPC support
84934bf70e multiprocess: Add echoipc RPC method and test (Russell Yanofsky)
7d76cf667e multiprocess: Add comments and documentation (Russell Yanofsky)
ddf7ecc8df multiprocess: Add bitcoin-node process spawning support (Russell Yanofsky)
10afdf0280 multiprocess: Add Ipc interface implementation (Russell Yanofsky)
745c9cebd5 multiprocess: Add Ipc and Init interface definitions (Russell Yanofsky)
5d62d7f6cd Update libmultiprocess library (Russell Yanofsky)

Pull request description:

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).

  ---

  This PR adds basic process spawning and IPC method call support to `bitcoin-node` executables built with `--enable-multiprocess`[*].

  These changes are used in https://github.com/bitcoin/bitcoin/pull/10102 to let node, gui, and wallet functionality run in different processes, and extended in https://github.com/bitcoin/bitcoin/pull/19460 and https://github.com/bitcoin/bitcoin/pull/19461 after that to allow gui and wallet processes to be started and stopped independently and connect to the node over a socket.

  These changes can also be used to implement new functionality outside the `bitcoin-node` process like external indexes or pluggable transports (https://github.com/bitcoin/bitcoin/pull/18988). The `Ipc::spawnProcess` and `Ipc::serveProcess` methods added here are entry points for spawning a child process and serving a parent process, and being able to make bidirectional, multithreaded method calls between the processes. A simple example of this is implemented in commit "Add echoipc RPC method and test."

  Changes in this PR aside from the echo test were originally part of #10102, but have been split and moved here for easier review, and so they can be used for other applications like external plugins.

  Additional notes about this PR can be found at https://bitcoincore.reviews/19160

  [*] Note: the `--enable-multiprocess` feature is still experimental, and not enabled by default, and not yet supported on windows. More information can be found in [doc/multiprocess.md](https://github.com/bitcoin/bitcoin/blob/master/doc/multiprocess.md)

ACKs for top commit:
  fjahr:
    re-ACK 84934bf70e
  ariard:
    ACK 84934bf. Changes since last ACK fixes the silent merge conflict about `EnsureAnyNodeContext()`. Rebuilt and checked again debug command `echoipc`.

Tree-SHA512: 52a948b5e18a26d7d7a09b83003eaae9b1ed2981978c36c959fe9a55abf70ae6a627c4ff913a3428be17400a3dace30c58b5057fa75c319662c3be98f19810c6
2021-04-27 11:59:26 +02:00
W. J. van der Laan
19a56d1519
Merge bitcoin/bitcoin#21009: Remove RewindBlockIndex logic
d831e711ca [validation] RewindBlockIndex no longer needed (Dhruv Mehta)

Pull request description:

  Closes #17862

  Context from [original comment](https://github.com/bitcoin/bitcoin/issues/17862#issuecomment-744285188) (minor edits):

  `RewindBlockIndex()` is a mechanism to allow nodes to be upgraded after segwit activation, while still keeping their chainstate/datadir in a consistent state. It works as follows:

  - A pre-segwit (i.e. v0.13.0 or older) node is running.
  -  Segwit activates. The pre-segwit node remains sync'ed to the tip, but is not enforcing the new segwit rules.
  - The user upgrades the node to a segwit-aware version (v0.13.1 or newer).
  - On startup, in `AppInitMain()`, `RewindBlockIndex()` is called. This walks the chain backwards from the tip, disconnecting and erasing blocks that from after segwit activation that weren't validated with segwit rules.
  - those blocks are then redownloaded (with witness data) and validated with segwit rules.

  This logic probably isn't required any more since:

  - Segwit activated at height 481824, when the block chain was 130GB and the total number of txs was 250 million. Today, we're at height 667704, the blockchain is over 315GB and the total number of txs is over 600 million. Even if 20% of that added data is witness data (a high estimate), then around 150GB of transactions would need to be rewound to get back to segwit activation height. It'd probably be faster to simply validate from genesis, especially since we won't be validating any scripts before the assumevalid block. It's also unclear whether rewinding 150GB of transactions would even work. It's certainly never been tested.
  - Bitcoin Core v0.13 is hardly used any more. https://luke.dashjr.org/programs/bitcoin/files/charts/software.html shows less than 50 nodes running it. The software was EOL on Aug 1st 2018. It's very unlikely that anyone is running 0.13 and will want to upgrade to 0.22.

  This PR introduces `NeedsRedownload()` which merely checks for insufficiently validated segwit blocks and requests that the user restarts the node with `-reindex`. Reindexing the block files upon restart will make the node rebuild chain state and block index from the `blk*.dat` files on disk. The node won't be able to index the blocks with `BLOCK_OPT_WITNESS`, so they will be missing from the chain and be re-downloaded, with witness data.

  Removing this code allows the following (done in follow-up #21090):

  - removal of tests using `segwitheight=-1` in `p2p_segwit.py`.
  - in turn, that allows us to drop support for `-segwitheight=-1`, which is only supported for that test.
  - that allows us to always set `NODE_WITNESS` in our local services. The only reason we don't do that is to support `-segwitheight=-1`.
  - that in turn allows us to drop all of the `GetLocalServices() & NODE_WITNESS` checks inside `net_processing.cpp`, since our local services would always include `NODE_WITNESS`

ACKs for top commit:
  jnewbery:
    utACK d831e711ca
  jamesob:
    ACK d831e711ca
  laanwj:
    Cursory code review ACK d831e711ca. Agree with the direction of the change, thanks for simplifying the logic here.
  glozow:
    utACK d831e711ca

Tree-SHA512: 3eddf5121ccd081ad7f15a5c6478ef867083edc8ba0bf1ee759e87bc070ee3d2f0698a3feba8db8dc087987c8452887b6f72cff05b3e178f41cb10a515fb8053
2021-04-27 10:14:52 +02:00
James O'Beirne
91d93aac4e
validation: remove nchaintx from assumeutxo metadata
This value is no longer used and is instead specified statically
in chainparams. This change means that previously generated
snapshots will no longer be usable.
2021-04-26 13:22:37 -04:00
MarcoFalke
683dda2a70
Merge bitcoin/bitcoin#21777: test: Fix feature_notifications.py intermittent issue
fa4aec2b26 test: Fix feature_notifications.py intermittent issue (MarcoFalke)

Pull request description:

  Fixes #21683

Top commit has no ACKs.

Tree-SHA512: 256806d82877477f4b3d795658f61127c0de4eff07216f6071f40a8ec1f5d43f3c587f35dd436d480dc261ef6646ac5547db104d22f3fcfeeb61bbdbe04bcc31
2021-04-26 18:04:22 +02:00
MarcoFalke
0a0a95b9d6
Merge bitcoin/bitcoin#21707: test: Extend functional tests for addr relay
a732ee353c [test] Add tests for addr relay in -blocksonly mode (Amiti Uttarwar)
a6694eaed8 [test] Add address relay tests involving outbound peers (Martin Zumsande)
8188b77c17 [test] Add tests for getaddr behavior (Martin Zumsande)
d2dbfe6ff1 [test] Extract sending an addr message into a helper (Amiti Uttarwar)
c991943399 [test] Refactor the addr relay test to prepare for new tests (Amiti Uttarwar)

Pull request description:

  This extends the functional test `p2p_addr_relay.py`.
  It adds test coverage for address relay involving outbound peers, tests for both outgoing and incoming `GETADDR` requests and tests for `-blocksonly` mode.

  The initial refactors and some of the new tests were taken from Amiti Uttarwar's PR #21528 - they are general test improvements not directly tied to the change proposed there.

ACKs for top commit:
  amitiuttarwar:
    re-ACK a732ee353c, small diff based on code review
  MarcoFalke:
    Concept ACK a732ee353c 🌊

Tree-SHA512: e80d52683808ddd6b948a5134239f002f3fecf61b60e187877b07be6251721fde847104e495c75a1a5133a09c0b41a9255a0bec82932c0b304b516fa89bce33e
2021-04-26 09:14:34 +02:00
MarcoFalke
fa4aec2b26
test: Fix feature_notifications.py intermittent issue 2021-04-26 08:22:46 +02:00
windsok
6f6bb3ebc7 test: fix file permissions on various scripts
Updates permissions on files to comply with the new test added in the following commit
2021-04-23 17:13:28 -07:00
Russell Yanofsky
84934bf70e multiprocess: Add echoipc RPC method and test
Add simple interfaces::Echo IPC interface with one method that just takes and
returns a string, to test multiprocess framework and provide an example of how
it can be used to spawn and call between processes.
2021-04-23 03:02:50 -05:00
MarcoFalke
fa40eb5b6b
test: Speed up mempool_spend_coinbase.py 2021-04-23 09:56:02 +02:00
MarcoFalke
fa29382ab2
test: Fix test cache issue
The documentation did not match the implementation, no coins
were mined to the OP_TRUE address.
2021-04-23 09:48:56 +02:00
MarcoFalke
fa085b470a
test: Create MiniWallet.create_self_transfer 2021-04-23 09:39:21 +02:00
MarcoFalke
fa1bedb494
test: Add MiniWallet.sendrawtransaction
Can be reviewed with --ignore-all-space --color-moved=dimmed-zebra
2021-04-22 20:02:31 +02:00
MarcoFalke
4b5659c6b1
Merge bitcoin/bitcoin#19801: test: check for all possible OP_CLTV fail reasons in feature_cltv.py (BIP 65)
b01cd9471f test: check that _all_ invalid-CLTV txs are rejected after BIP65 activation (Sebastian Falbesoner)
dbc1981474 test: check that _all_ invalid-CLTV txs are allowed in a block pre-BIP65 (Sebastian Falbesoner)
8d0ce50c48 test: prepare cltv_invalidate to test all failure reasons in feature_cltv.py (Sebastian Falbesoner)
ce994e1202 test: add tx modfication helper function in feature_cltv.py (Sebastian Falbesoner)

Pull request description:

  The functional test for [BIP65](https://github.com/bitcoin/bips/blob/master/bip-0065.mediawiki) / `OP_CHECKLOCKTIMEVERIFY` (`feature_cltv.py`) currently only tests one out of five conditions that lead to failure of the op-code -- by prepending the script `OP_1NEGATE OP_CHECKLOCKTIMEVERIFY OP_DROP` to a tx's first input's scriptSig, the case of "_the top item on the stack is less than 0_" is checked:

  f8462a6d27/test/functional/feature_cltv.py (L26-L35)

  This PR adds the other cases (5 in total) by taking an integer argument to the function `cltv_invalidate` that is called in a loop instead of only once per testing scenario. Here is the full list of failure conditions and how they are tested (note that the scriptSig should still be valid before activation of BIP65, when `OP_CLTV` is simply a no-op):
  * _the stack is empty_
  ➡️  prepending `OP_CHECKLOCKTIMEVERIFY` to scriptSig
  * _the top item on the stack is less than 0_
  ➡️  prepending `OP_1NEGATE OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  * _the lock-time type (height vs. timestamp) of the top stack item and the nLockTime field are not the same_
  ➡️  prepending `OPNum(1000) OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  ➡️ setting tx.vin[0].nSequence=0 and tx.nCheckTimeLock=1296688602 (genesis block timestamp)
  * _the top stack item is greater than the transaction's nLockTime field_
  ➡️  prepending `OPNum(1000) OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  ➡️ setting tx.vin[0].nSequence=0 and tx.nCheckTimeLock=500
  * _the nSequence field of the txin is 0xffffffff_
  ➡️  prepending `OPNum(500) OP_CHECKLOCKTIMEVERIFY OP_DROP` to scriptSig
  ➡️ setting tx.vin[0].nSequence=0xffffffff and tx.nCheckTimeLock=500

  The first commit creates a helper function for the tx modification and also includes some tidying up like turning single-line to multi-line Python imports where necessary and cleaning up some PEP8 warnings. The second commit prepares the invalidation function `cltv_invalidate` and the third and the fourth use it and check for the expected reject reason strings ("Operation not valid with the current stack size", "Negative locktime" and "Locktime requirement not satisfied").

ACKs for top commit:
  MarcoFalke:
    review ACK b01cd9471f 🐣

Tree-SHA512: dd82ae86e2bc4f3ab9bb1cfc9f04e4431b2b59c8aaf2a9f4b28654a1577e003fb43c500f99d76ff57e96262168e1cad7c1a0d71158e4b01063737e8f4be1e07d
2021-04-22 12:58:23 +02:00
Dhruv Mehta
d831e711ca [validation] RewindBlockIndex no longer needed
Instead of rewinding blocks, we request that the user restarts with
-reindex
2021-04-21 16:09:14 -07:00
Amiti Uttarwar
a732ee353c [test] Add tests for addr relay in -blocksonly mode
Co-Authored-By: Martin Zumsande <mzumsande@gmail.com>
2021-04-21 22:34:10 +02:00
Martin Zumsande
a6694eaed8 [test] Add address relay tests involving outbound peers
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-04-21 22:34:03 +02:00
Martin Zumsande
8188b77c17 [test] Add tests for getaddr behavior
Co-authored-by: Amiti Uttarwar <amiti@uttarwar.org>
2021-04-21 22:16:39 +02:00
Amiti Uttarwar
d2dbfe6ff1 [test] Extract sending an addr message into a helper
Also reduces mocktime to prevent idle disconnects
Co-Authored-By: Martin Zumsande <mzumsande@gmail.com>
2021-04-21 22:16:39 +02:00
Amiti Uttarwar
c991943399 [test] Refactor the addr relay test to prepare for new tests
Moves setting up the addr message into a repeatable function, and breaks up the
existing tests into separate functions for legibility.
2021-04-21 22:14:16 +02:00
Fabian Jahr
5f96d7d22d
rpc: gettxoutsetinfo rejects hash_serialized_2 for specific height 2021-04-19 21:11:37 +02:00
Fabian Jahr
23fe50436b
test: Add test for coinstatsindex behavior in reorgs
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
2021-04-19 21:11:37 +02:00
Fabian Jahr
b9362392ae
index, rpc: Add use_index option for gettxoutsetinfo 2021-04-19 21:11:33 +02:00
Fabian Jahr
bb7788b121
test: Test coinstatsindex robustness across restarts 2021-04-19 20:31:02 +02:00
Fabian Jahr
e0938c2909
test: Add tests for block_info in gettxoutsetinfo
This additional data will automatically be returned if the coinstats index is used.
2021-04-19 20:31:02 +02:00
Fabian Jahr
2501576ecc
rpc, index: Add verbose amounts tracking to Coinstats index 2021-04-19 20:31:02 +02:00
Jon Atack
655d929836
test: add coinstatsindex getindexinfo coverage, improve current tests 2021-04-19 20:31:02 +02:00
Fabian Jahr
6a4c0c09ab
test: Add functional test for Coinstats index 2021-04-19 20:31:01 +02:00
MarcoFalke
83c715415a
Merge #20857: test: update documentation in feature_csv_activation.py
9053b88b1c update docstring in feature_csv_activation.py (Pierre K)

Pull request description:

  These changes in the test documentation reflect the changes introduced in #17921.

ACKs for top commit:
  MarcoFalke:
    review ACK  9053b88

Tree-SHA512: 17fb954baded8dab1c869dd48b76b516150bae616c792c573e4114d4adfdd40195745c56570aa3050cc0015ee496acd7ec178df8ba14831dd22f9722fda84da2
2021-04-19 09:47:57 +02:00
Pierre K
9053b88b1c update docstring in feature_csv_activation.py
These changes in the test documentation reflect the changes introduced in #17921
2021-04-18 22:05:14 +02:00
klementtan
a411494261 rpc: Improve getblock error message for invalid data type. 2021-04-18 12:30:58 +08:00
João Barbosa
44dab423eb qa: Test default include_mempool value of gettxout 2021-04-16 23:44:49 +01:00
fanquake
2cd834e6c0
Merge #21377: Speedy trial support for versionbits
ffe33dfbd4 chainparams: drop versionbits threshold to 90% for mainnnet and signet (Anthony Towns)
f054f6bcd2 versionbits: simplify state transitions (Anthony Towns)
55ac5f568a versionbits: Add explicit NEVER_ACTIVE deployments (Anthony Towns)
dd07e6da48 fuzz: test versionbits delayed activation (Anthony Towns)
dd85d5411c tests: test versionbits delayed activation (Anthony Towns)
73d4a70639 versionbits: Add support for delayed activation (Anthony Towns)
9e6b65f6fa tests: clean up versionbits test (Anthony Towns)
5932744450 tests: test ComputeBlockVersion for all deployments (Anthony Towns)
63879f0a47 tests: pull ComputeBlockVersion test into its own function (Anthony Towns)

Pull request description:

  BIP9-based implementation of "speedy trial" activation specification, see https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2021-March/018583.html

  Edge cases are tested by fuzzing added in #21380.

ACKs for top commit:
  instagibbs:
    tACK ffe33dfbd4
  jnewbery:
    utACK ffe33dfbd4
  MarcoFalke:
    review ACK ffe33dfbd4 💈
  achow101:
    re-ACK ffe33dfbd4
  gmaxwell:
    ACK ffe33dfbd4
  benthecarman:
    ACK ffe33dfbd4
  Sjors:
    ACK ffe33dfbd4
  jonatack:
    Initial approach ACK ffe33dfbd4 after a first pass of review, building and testing each commit, mostly looking at the changes and diffs. Will do a more high-level review iteration. A few minor comments follow to pick/choose/ignore.
  ariard:
    Code Review ACK ffe33df

Tree-SHA512: f79a7146b2450057ee92155cbbbcec12cd64334236d9239c6bd7d31b32eec145a9781c320f178da7b44ababdb8808b84d9d22a40e0851e229ba6d224e3be747c
2021-04-15 10:04:14 +08:00
MarcoFalke
a1f0b8b62e
Merge #21634: tests: Skip SQLite fsyncs while testing
41f891da50 tests: Skip SQLite fsyncs while testing (Andrew Chow)

Pull request description:

  Since we want tests to run quickly, and since tests do a lot more db operations than expected we expect to see in actual usage, we disable sqlite's syncing behavior to make db operations run much faster. This syncing behavior is necessary for normal operation as it helps guarantee that data won't become lost or corrupted, but in tests, we don't care about that.

  Fixes #21628

ACKs for top commit:
  vasild:
    ACK 41f891da50

Tree-SHA512: f36f969a182c622691ae5113573a3250e8d367437e83a1a9d3d2b55dd3a9cdf3c6474169a7bd271007bb9ce47f585aa7a6aeae6eebbaeb02d79409b02f47fd8b
2021-04-13 16:31:12 +02:00
Andrew Chow
41f891da50 tests: Skip SQLite fsyncs while testing
Since we want tests to run quickly, and since tests do a lot more db
operations than expected we expect to see in actual usage, we disable
sqlite's syncing behavior to make db operations run much faster. This
syncing behavior is necessary for normal operation as it helps guarantee
that data won't become lost or corrupted, but in tests, we don't care
about that.
2021-04-12 19:29:03 -04:00
Anthony Towns
73d4a70639 versionbits: Add support for delayed activation 2021-04-12 11:14:49 +10:00
Sjors Provoost
b54b2e7b1a
Move external signer out of wallet module
This commit moves the ExternalSigner class and RPC methods out of the wallet module.

The enumeratesigners RPC can be used without a wallet since #21417.
With additional modifications external signers could be used without a wallet in general, e.g. via signrawtransaction.

The signerdisplayaddress RPC is ranamed to walletdisplayaddress because it requires wallet context.
A future displayaddress RPC call without wallet context could take a descriptor argument.

This commit fixes a rpc_help.py failure when configured with --disable-wallet.
2021-04-08 17:56:00 +02:00
W. J. van der Laan
cb79cabdd9
Merge #21594: rpc: add network field to getnodeaddresses
5c446784b1 rpc: improve getnodeaddresses help (Jon Atack)
1b9189866a rpc: simplify/constify getnodeaddresses code (Jon Atack)
3bb6e7b655 rpc: add network field to rpc getnodeaddresses (Jon Atack)

Pull request description:

  This patch adds a network field to RPC `getnodeaddresses`, which is useful on its own, particularly with the addition of new networks like I2P and others in the future, and which I also found helpful for adding a new CLI command as a follow-up to this pull that calls `getnodeaddresses` and needs to know the network of each address.

  While here, also improve the `getnodeaddresses` code and help.

  ```
  $ bitcoin-cli -signet getnodeaddresses 3
  [
    {
      "time": 1611564659,
      "services": 1033,
      "address": "2600:1702:3c30:734f:8f2e:744b:2a51:dfa5",
      "port": 38333,
      "network": "ipv6"
    },
    {
      "time": 1617531931,
      "services": 1033,
      "address": "153.126.143.201",
      "port": 38333,
      "network": "ipv4"
    },
    {
      "time": 1617473058,
      "services": 1033,
      "address": "nsgyo7begau4yecc46ljfecaykyzszcseapxmtu6adrfagfrrzrlngyd.onion",
      "port": 38333,
      "network": "onion"
    }
  ]

  $ bitcoin-cli help getnodeaddresses
  getnodeaddresses ( count )

  Return known addresses, which can potentially be used to find new nodes in the network.

  Arguments:
  1. count    (numeric, optional, default=1) The maximum number of addresses to return. Specify 0 to return all known addresses.

  Result:
  [                         (json array)
    {                       (json object)
      "time" : xxx,         (numeric) The UNIX epoch time when the node was last seen
      "services" : n,       (numeric) The services offered by the node
      "address" : "str",    (string) The address of the node
      "port" : n,           (numeric) The port number of the node
      "network" : "str"     (string) The network (ipv4, ipv6, onion, i2p) the node connected through
    },
    ...
  ]
  ```
  Future idea: allow passing `getnodeaddresses` a network (or networks) as an argument to return only addresses in that network.

ACKs for top commit:
  laanwj:
    Tested ACK 5c446784b1
  jarolrod:
    re-ACK 5c446784b1
  promag:
    Code review ACK 5c446784b1.

Tree-SHA512: ab0101f50c76d98c3204133b9f2ab6b7b17193ada31455ef706ad11afbf48f472fa3deb33e96028682369b35710ccd07d81863d2fd55c1485f32432f2b75efa8
2021-04-07 18:56:01 +02:00
Jon Atack
3bb6e7b655
rpc: add network field to rpc getnodeaddresses 2021-04-07 12:57:07 +02:00
fanquake
245a5cd560
Merge #21166: Introduce DeferredSignatureChecker and have SignatureExtractorClass subclass it
a97a9298ce Test that signrawtx works when a signed CSV and CLTV inputs are present (Andrew Chow)
6965456c10 Introduce DeferringSignatureChecker and inherit with SignatureExtractor (Andrew Chow)

Pull request description:

  Previously SignatureExtractorChecker took a MutableTransactionSignatureChecker and passed through function calls to that. However not all functions were implemented so not everything passed through as it should have. To solve this, SignatureExctractorChecker now implements all of those functions via a new class - DeferredSignatureChecker. DeferredSignatureChecker is introduced to allow for future signature checkers which use another SignatureChecker but need to be able to do somethings outside of just the signature checking.

  Fixes #21151

ACKs for top commit:
  sipa:
    utACK a97a9298ce
  meshcollider:
    Code review ACK a97a9298ce
  instagibbs:
    utACK a97a9298ce

Tree-SHA512: bca784c75c2fc3fcb74e81f4e3ff516699e8debaa2db81e12843abdfe9cf265dac11db8619751cb9b3e9bbe779805d029fabe5f3cbca5e86bfd72de3664b0b94
2021-04-07 12:47:41 +08:00
fanquake
7aa0d8adf8
Merge #21063: wallet, rpc: update listdescriptors response format
2e5f7def22 wallet, rpc: update listdescriptors response format (Ivan Metlushko)

Pull request description:

  Update `listdescriptors` response format according to [RPC interface guidelines](https://github.com/bitcoin/bitcoin/blob/master/doc/developer-notes.md#rpc-interface-guidelines).

  This is a follow up for #20226

  **Before:**
  ```
  Result:
  [                               (json array) Response is an array of descriptor objects
    {                             (json object)
      "desc" : "str",             (string) Descriptor string representation
      "timestamp" : n,            (numeric) The creation time of the descriptor
      "active" : true|false,      (boolean) Activeness flag
      "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
      "range" : [                 (json array, optional) Defined only for ranged descriptors
        n,                        (numeric) Range start inclusive
        n                         (numeric) Range end inclusive
      ],
      "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
    },
    ...
  ]
  ```

  **After:**
  ```
  Result:
  {                                 (json object)
    "wallet_name" : "str",          (string) Name of wallet this operation was performed on
    "descriptors" : [               (json array) Array of descriptor objects
      {                             (json object)
        "desc" : "str",             (string) Descriptor string representation
        "timestamp" : n,            (numeric) The creation time of the descriptor
        "active" : true|false,      (boolean) Activeness flag
        "internal" : true|false,    (boolean, optional) Whether this is internal or external descriptor; defined only for active descriptors
        "range" : [                 (json array, optional) Defined only for ranged descriptors
          n,                        (numeric) Range start inclusive
          n                         (numeric) Range end inclusive
        ],
        "next" : n                  (numeric, optional) The next index to generate addresses from; defined only for ranged descriptors
      },
      ...
    ]
  }
  ```

ACKs for top commit:
  achow101:
    re-ACK 2e5f7def22
  meshcollider:
    utACK 2e5f7def22
  jonatack:
    re-ACK 2e5f7def22

Tree-SHA512: 49bf73e46e2a61003ce594a4bfc506eb9592ccb799c2909c43a1a527490a4b4009f78dc09f3d47b4e945d3d7bb3cd2632cf48c5ace5feed5066158cc010dddc1
2021-04-02 13:18:35 +08:00
MarcoFalke
6e22b522f9
Merge #21373: test: generate fewer blocks in feature_nulldummy to fix timeouts, speed up
ccd976dd3d test: use 327 fewer blocks in feature_nulldummy (Jon Atack)
68c280f197 test, refactor: abstract the feature_nulldummy blockheight values (Jon Atack)

Pull request description:

  The resolved timeout issue seen in the CI can be reproduced locally by running `test/functional/feature_nulldummy.py --valgrind --loglevel=debug`

  Speeds up the normal test runtime for me from 3.8 to 2.2 seconds (debug build). Thanks to Marco Falke for the approach suggestion.

ACKs for top commit:
  AnthonyRonning:
    reACK ccd976dd3d - ran a few times with the rest of the tests and still passing for me with just the fewer block change.
  MarcoFalke:
    review ACK ccd976dd3d 🏝

Tree-SHA512: 38339dca4276d1972e3a5a5ee436da64e9e58fd3b50b186e34b07ade9523ac4c03f6c3869c5f2a59c23b43c44f87e712f8297a01a8d83202049c081e3eeb4445
2021-04-01 13:17:12 +02:00
Sebastian Falbesoner
b01cd9471f test: check that _all_ invalid-CLTV txs are rejected after BIP65 activation 2021-03-31 21:30:25 +02:00
Sebastian Falbesoner
dbc1981474 test: check that _all_ invalid-CLTV txs are allowed in a block pre-BIP65 2021-03-31 21:09:59 +02:00
Sebastian Falbesoner
8d0ce50c48 test: prepare cltv_invalidate to test all failure reasons in feature_cltv.py
only the "top item on the stack is less than 0" is used in the test right now
2021-03-31 21:09:59 +02:00
Sebastian Falbesoner
ce994e1202 test: add tx modfication helper function in feature_cltv.py
+ reformat python imports
+ fix PEP8 warnings (all except E501 line too long)
2021-03-31 21:09:59 +02:00
MarcoFalke
1c7be9ab90
Merge #20286: rpc: deprecate addresses and reqSigs from rpc outputs
90ae3d8ca6 doc: Add release notes for -deprecatedrpc=addresses and bitcoin-tx (Michael Dietz)
085b3a7299 rpc: deprecate `addresses` and `reqSigs` from rpc outputs (Michael Dietz)

Pull request description:

  Considering the limited applicability of `reqSigs` and the confusing output of `1` in all cases except bare multisig, the `addresses` and `reqSigs` outputs are removed for all rpc commands.

  1) add a new sane "address" field (for outputs that have an identifiable address, which doesn't include bare multisig)
  2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact (with all weird/wrong behavior they have now)
  3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely always.

  Note: Some light refactoring done to allow us to very easily delete a few chunks of code (marked with TODOs) when we remove this deprecated behavior.

  Using `IsDeprecatedRPCEnabled` in core_write.cpp caused some circular dependencies involving core_io

  Circular dependencies were caused by rpc/util unnecessarily importing node/coinstats and node/transaction. Really what rpc/util needs are some fundamental type/helper-function definitions. So this was cleaned up to make more sense.

  This fixes #20102.

ACKs for top commit:
  MarcoFalke:
    re-ACK 90ae3d8ca6 📢

Tree-SHA512: 8ffb617053b5f4a8b055da17c06711fd19632e0037d71c4c8135e50c8cd7a19163989484e4e0f17a6cc48bd597f04ecbfd609aef54b7d1d1e76a784214fcf72a
2021-03-29 15:14:31 +02:00
MarcoFalke
4399dc8142
Merge #21509: p2p: Don't send FEEFILTER in blocksonly mode.
beead33a21 [test] no send feefilters when txrelay is turned off (glozow)
18a9b27dd6 p2p: Don't send FEEFILTER in blocksonly mode (Martin Zumsande)

Pull request description:

  The purpose of FEEFILTER messages (BIP 133) is to inform our peers that we do not want transactions below a specified fee rate.
  In blocksonly mode, we do not want our peer to send us any transactions at all (and will disconnect if a peer still sends a transaction INV or TX). 

  Therefore, I don't think that it makes sense to send FEEFILTER messages every 10 minutes on average in blocksonly mode - this PR disables it.

  Note that on block-relay-only connections, FEEFILTER is already disabled, just not in blocksonly mode.

ACKs for top commit:
  glozow:
    re ACK beead33a21 🙂 thanks for adding the test!
  amitiuttarwar:
    reACK beead33a21
  MarcoFalke:
    review ACK beead33a21
  jnewbery:
    reACK beead33a21

Tree-SHA512: e748cd52fe23d647fa49008b020389956ac508e16ce9fd108d8afb773bff95788298ae229162bd70215d7246fc25c796484966dc05890b0b4ef601f9cd35628b
2021-03-29 11:56:47 +02:00
MarcoFalke
9b48b3ac42
Merge #21390: test: Test improvements for UTXO set hash tests
4f2653a890 test: Use deterministic chain in utxo set hash test (Fabian Jahr)
4973c5175c test: Remove wallet dependency of utxo set hash test (Fabian Jahr)
1a27af1d7b rpc: Improve gettxoutsetinfo help (Fabian Jahr)

Pull request description:

  Follow-ups to #19145:
  - Small improvement on the help text of RPC gettxoutsetinfo
  - Using deterministic blockchain in the test `functional/feature_utxo_set_hash.py`
  - Removing wallet dependency in the test `functional/feature_utxo_set_hash.py`

  Split out of #19521.

ACKs for top commit:
  MarcoFalke:
    review ACK 4f2653a890 👲

Tree-SHA512: 92927b3aa22b6324eb4fc9d346755313dec44d973aa69a0ebf80a8569b5f3a7cf3539721ebdba183737534b9e29b3e33f412515890f0d0b819878032a3bba8f9
2021-03-26 08:52:59 +01:00
glozow
beead33a21 [test] no send feefilters when txrelay is turned off 2021-03-25 21:41:56 +01:00
MarcoFalke
ed49203daa
Merge #21357: test: Unconditionally check for fRelay field in test framework
39a9ec579f Unconditionally check for fRelay field in test framework (Troy Giorshev)

Pull request description:

  picking up #20411 (rebased onto master)

  There is a discrepancy in the implementation of our p2p protocol between
  bitcoind and the testing framework.  The fRelay field is an optional
  field at the end of a version message as of protocol version 70001.
  However, when deserializing a message in bitcoind, we don't check the
  version to see if it should have an fRelay field or not.  Instead, we
  unconditionally attempt to deserialize into the field.

  This commit brings the testing framework in line with the implementation
  in core.

  This matters for a version message with the following fields:

  Version = 60000
  fRelay = 1

  Bitcoind would deserialize this into a version message with
  Version=60000 and fRelay=1, whereas (before this commit) our testing
  framework would deserialize this into a version message with
  Version=60000 and fRelay=0.

ACKs for top commit:
  jnewbery:
    utACK 39a9ec579f

Tree-SHA512: 13a23f1180b7121ba41cb85baa38094b41f4607a7c88b3384775177cb116e76faf5514760624f98a4e8a830767407c46753a7e0285158c33e0c6ce395de8f15c
2021-03-24 19:28:24 +01:00
MarcoFalke
7ce7ce5716
Merge #21338: test: add functional test for anchors.dat
581791c620 test: add functional test for anchors.dat (bruno)

Pull request description:

  This PR adds a functional test for anchors.dat.

  It creates a node and adds 2 outbound block-relay-only connections and 5 inbound connections.
  When the node is down, anchors.dat should contain the 2 addresses from the outbound block-relay-only connections.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 581791c620
  hebasto:
    ACK 581791c620

Tree-SHA512: 77038b09e36ee5ae473a26d6f566c0ed283af258c34df8486706a24f72b05abab621a293ac886d03849bc45bc28be7336137252225b25aff393baa6b5238688c
2021-03-24 08:27:03 +01:00
Troy Giorshev
39a9ec579f Unconditionally check for fRelay field in test framework
There is a discrepancy in the implementation of our p2p protocol between
bitcoind and the testing framework.  The fRelay field is an optional
field at the end of a version message as of protocol version 70001.
However, when deserializing a message in bitcoind, we don't check the
version to see if it should have an fRelay field or not.  Instead we
unconditionally attempt to deserialize into the field.

This commit brings the testing framework in line with the implementation
in core.

This matters for a version message with the following fields:

Version = 60000
fRelay = 1

Bitcoind would deserialize this into a version message with
Version=60000 and fRelay=1, whereas (before this commit) our testing
framework would deserialize this into a version message with
Version=60000 and fRelay=0.
2021-03-23 19:57:17 -04:00
Fabian Jahr
4f2653a890
test: Use deterministic chain in utxo set hash test
Also melts the previously separated test cases into one.
2021-03-23 20:47:24 +01:00
Fabian Jahr
4973c5175c
test: Remove wallet dependency of utxo set hash test 2021-03-23 20:32:50 +01:00
Michael Dietz
085b3a7299
rpc: deprecate addresses and reqSigs from rpc outputs
1) add a new sane "address" field (for outputs that have an
   identifiable address, which doesn't include bare multisig)
2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact
   (with all weird/wrong behavior they have now)
3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely,
   always.
2021-03-23 10:51:43 -04:00
MarcoFalke
4132193617
Merge #21040: wallet: Fix already-loading message grammar
ae9d26a8f0 wallet: Fix already-loading error message grammar (Fotis Koutoupas)

Pull request description:

ACKs for top commit:
  practicalswift:
    cr ACK ae9d26a8f0
  prayank23:
    ACK ae9d26a8f0

Tree-SHA512: 2f58d309dd33954f47e3ed339887b11bfdad4519f89ed3dd9bf3fb0db246d78b89653d553d02350ec84ce68bbb904db9fac00a2aad8d37f48df694d6782f35df
2021-03-21 07:51:11 +01:00
bruno
581791c620 test: add functional test for anchors.dat 2021-03-19 20:30:52 -03:00
MarcoFalke
3530d5d2d8
Merge #18335: bitcoin-cli: print useful error if bitcoind rpc work queue exceeded
8dd5946c0b add functional test (Larry Ruane)
b5a80fa7e4 util: Handle HTTP_SERVICE_UNAVAILABLE in bitcoin-cli (Hennadii Stepanov)

Pull request description:

  If `bitcoind` is processing 16 RPC requests, attempting to submit another request using `bitcoin-cli` produces this less-than-helpful error message: `error: couldn't parse reply from server`. This PR changes the error to: `error: server response: Work queue depth exceeded`.

ACKs for top commit:
  fjahr:
    tACK 8dd5946c0b
  luke-jr:
    utACK 8dd5946c0b (no changes since previous utACK)
  hebasto:
    re-ACK 8dd5946c0b, only suggested changes since my [previous](https://github.com/bitcoin/bitcoin/pull/18335#pullrequestreview-460621350) review.
  darosior:
    ACK 8dd5946c0b

Tree-SHA512: 33e25f6ff05d9b56fae2bdb68b132557bb8e995f5438ac4fbbc53c304c5152a98aa43c43600c31d8a6a2830cbd48bf8ec7d89dce50190b29ec00a43830126913
2021-03-19 20:52:16 +01:00
Wladimir J. van der Laan
8ec881d3b6
Merge #20861: BIP 350: Implement Bech32m and use it for v1+ segwit addresses
03346022d6 naming nits (Fabian Jahr)
2e7c80fb5b Add signet support to gen_key_io_test_vectors.py (Pieter Wuille)
fe5e495c31 Use Bech32m encoding for v1+ segwit addresses (Pieter Wuille)
25b1c6e13d Add Bech32m test vectors (Pieter Wuille)
da2bb6976d Implement Bech32m encoding/decoding (Pieter Wuille)

Pull request description:

  This implements [BIP 350](https://github.com/bitcoin/bips/blob/master/bip-0350.mediawiki):
  * For segwit v1+ addresses, a new checksum algorithm called Bech32m is used.
  * Segwit v0 address keep using Bech32 as specified in [BIP 173](https://github.com/bitcoin/bips/blob/master/bip-0173.mediawiki).

ACKs for top commit:
  Sjors:
    utACK 0334602
  jnewbery:
    utACK 03346022d6
  achow101:
    ACK 0334602
  fjahr:
    re-ACK 0334602
  benthecarman:
    ACK 03346022d6

Tree-SHA512: 4424cfd44869d813d6152fb3ed867b204036736bc2344a039b93700b6f36a43e9110478f138eb81c97c77ab27ecb776dada5ba632cb5a3a9d244924d2540a557
2021-03-18 20:37:21 +01:00
MarcoFalke
5ef16038a1
Merge #21410: test: increase rpc_timeout for fundrawtx test_transaction_too_large
d09120b7d1 test: give fundraw more time for test_transaction_too_large (Jon Atack)

Pull request description:

  to hopefully fix timeouts from a new test added in 48a0319bab of #20536 merged March 8, 2021

  seen locally when running via the test runner

  ```
  File "/home/jon/projects/bitcoin/bitcoin/test/functional/rpc_fundrawtransaction.py", line 927, in test_transaction_too_large
  raise JSONRPCException({
      test_framework.authproxy.JSONRPCException: 'generatetoaddress' RPC took longer than 30.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```
  and in the CI like https://bitcoinbuilds.org/index.php?ansilog=28537952-2c92-46f2-9871-8918e5ba2738.log#l2398
  ```
  File "/home/ubuntu/src/test/functional/rpc_fundrawtransaction.py", line 927, in test_transaction_too_large
  test_framework.authproxy.JSONRPCException: 'generatetoaddress' RPC took longer than 240.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```

Top commit has no ACKs.

Tree-SHA512: f11c923439014fe12420f986c640fd301a26282eb41516957d73b9c751087cbae3d0e316f9ccb49bcb424f488540266f70d3f97948633e77c62bd7935df90452
2021-03-16 19:25:31 +01:00
Pieter Wuille
fe5e495c31 Use Bech32m encoding for v1+ segwit addresses
This also includes updates to the Python test framework implementation,
test vectors, and release notes.
2021-03-16 10:48:36 -07:00
Jon Atack
d09120b7d1
test: give fundraw more time for test_transaction_too_large 2021-03-16 17:18:54 +01:00
Maayan Keshet
06e1fb0b17 Add new format string placeholders for walletnotify to include relevant block information for transactions 2021-03-15 18:45:36 +02:00
MarcoFalke
e9b6400a1e
Merge #21411: test: add logging, reduce blocks, move sync_all in wallet_ groups
c62f9bc0e9 test: use fewer blocks in wallet_groups and move sync call (Jon Atack)
3a16b5ef95 test: add missing logging to wallet_groups.py (Jon Atack)

Pull request description:

  - add logging (particularly useful as the tests are somewhat slow)
  - generate 101 blocks instead of 110
  - move `sync_all` call into the loop, so fewer blocks are synced on each call, to hopefully see fewer CI timeouts as in https://bitcoinbuilds.org/index.php?ansilog=88eee99e-1727-44ed-b778-3b9c75c33928.log

  ```
  L2742     File "/home/ubuntu/src/test/functional/wallet_groups.py", line 162, in run_test
  L2743       self.sync_all()
  test_framework.authproxy.JSONRPCException: 'syncwithvalidationinterfacequeue' RPC took longer than 960.000000 seconds. Consider using larger timeout for calls that take longer to return. (-344)
  ```

ACKs for top commit:
  MarcoFalke:
    cr ACK c62f9bc0e9

Tree-SHA512: 711deafcd589cb8196cb207ff882e0f2ab6b70828a6abad91f83f535974cc430a56b9e8a960fd233d31d610932a0d48b49ee681aae564d145a3040288ecda8f8
2021-03-11 19:09:09 +01:00
Jon Atack
c62f9bc0e9
test: use fewer blocks in wallet_groups and move sync call
- generate 101 blocks instead of 110

- move a sync_all call into the loop so that fewer blocks are
  synced on each call to hopefully see fewer timeouts
2021-03-11 17:15:05 +01:00
Jon Atack
ccd976dd3d
test: use 327 fewer blocks in feature_nulldummy
by generating 100 blocks for coinbase maturity instead of 427.

This speeds up the test and should help avoid timeout errors.
2021-03-10 22:47:59 +01:00
Jon Atack
68c280f197
test, refactor: abstract the feature_nulldummy blockheight values
Refactoring only, no change in test behavior.
2021-03-10 22:38:21 +01:00
Ivan Metlushko
2e5f7def22 wallet, rpc: update listdescriptors response format 2021-03-09 09:04:35 +01:00
Samuel Dobson
a8b0892b74
Merge #20536: wallet: Error with "Transaction too large" if the funded tx will end up being too large after signing
48a0319bab Add a test that selects too large if BnB is used (Andrew Chow)
3e69939b78 Fail if maximum weight is too large (Andrew Chow)
51e2cd322c Have CalculateMaximumSignedTxSize also compute tx weight (Andrew Chow)

Pull request description:

  Currently the `Transaction too large` is calculated on the transaction that is returned from `CreateTransaction`. This does not make sense for when `CreateTransaction` is being used for `fundrawtransaction` as no signing occurs so the final returned transaction is missing signatures. Thus users may successfully fund a transaction but fail to broadcast it after it has been fully signed.

  So instead we should figure out whether the transaction we are funding will be too large after it is signed. We can do this by having `CalculateMaximumSignedTxSize` also return the transaction weight and then comparing that weight against the maximum weight.

ACKs for top commit:
  instagibbs:
    ACK 48a0319bab
  meshcollider:
    utACK 48a0319bab
  Xekyo:
    utACK with nits 48a0319bab

Tree-SHA512: 1700c60b07f67e2d5c591c5ccd131ac9f1861fab3def961c3c9c4b3281ec1063fe8e4f0f7f1038cac72692340856406bcee8fb45c8104d2ad34357a0ec878ac7
2021-03-09 10:42:21 +13:00
Jon Atack
3a16b5ef95
test: add missing logging to wallet_groups.py 2021-03-07 20:25:11 +01:00
MarcoFalke
f7653eb5ae
Merge #21345: test: bring p2p_leak.py up to date
a061a29970 test: bring p2p_leak.py up to date. (Martin Zumsande)

Pull request description:

  After the introduction of wtxidrelay and sendaddrv2 messages during version handshake, extend p2p_leak.py test to reflect this.
  Also, some minor fixes and doc improvements.

  I also added a test that peers not completing the version handshake will be disconnected for timeout, as suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/19723#issuecomment-699540294.

ACKs for top commit:
  brunoerg:
    Tested ACK a061a29970
  theStack:
    Tested ACK a061a29970

Tree-SHA512: 26c601491fa8710fc972d1b8f15da6b387a95b42bbfb629ec4c668769ad3824b6dd6a33d97363bca2171e403d8d1ce08abf3e5c9cab34f98d53e0109b1c7a0e5
2021-03-05 08:18:42 +01:00
Andrew Chow
a97a9298ce Test that signrawtx works when a signed CSV and CLTV inputs are present 2021-03-04 21:54:34 -05:00
fanquake
fbf5d16238
Merge #21246: doc: Correction for VerifyTaprootCommitment comments
6a0a6e7d05 Correction for VerifyTaprootCommitment comments (Russell O'Connor)

Pull request description:

  According to BIP-341, 'p' is called the taproot *internal* key, not inner key.

ACKs for top commit:
  sipa:
    ACK 6a0a6e7d05
  benthecarman:
    ACK 6a0a6e7d05
  theStack:
    ACK 6a0a6e7d05

Tree-SHA512: 94f553476a8404bff4b2d5724a1a54c5f530b987a616cd00a3800095f245c06e3c7a9066c729976f32069a56029406859a70ba523151d333dc1ed874f242bce8
2021-03-05 10:30:33 +08:00
Wladimir J. van der Laan
92b7efcf54
Merge #21148: Split orphan handling from net_processing into txorphanage
5e50e2d1b9 txorphanage: comment improvements (Anthony Towns)
eeeafb324e net_processing: move AddToCompactExtraTransactions into PeerManagerImpl (Anthony Towns)
f8c0688b94 scripted-diff: Update txorphanage naming convention (Anthony Towns)
6bd4963c06 txorphanage: Move functions and data into class (Anthony Towns)
03257b832d txorphanage: Extract EraseOrphansForBlock (Anthony Towns)
3c4c3c2fdd net_processing: drop AddOrphanTx (Anthony Towns)
26d1a6ccd5 denialofservices_tests: check txorphanage's AddTx (Anthony Towns)
1041616d7e txorphanage: Extract OrphanageAddTx (Anthony Towns)
f294da7274 txorphanage: Extract GetOrphanTx (Anthony Towns)
83679ffc60 txorphanage: Extract HaveOrphanTx (Anthony Towns)
ee135c8d5b txorphanage: Extract AddChildrenToWorkSet (Anthony Towns)
38a11c355a txorphanage: Add lock annotations (Anthony Towns)
81dd57e5b1 txorphanage: Pass uint256 by reference instead of value (Anthony Towns)
9d5313df7e move-only: Add txorphanage module (Anthony Towns)

Pull request description:

  Splits orphan handling into its own module and reduces global usage.

ACKs for top commit:
  jnewbery:
    utACK 5e50e2d1b9
  amitiuttarwar:
    utACK 5e50e2d1b9
  glozow:
    re ACK 5e50e2d1b9, comment updates
  laanwj:
    Code review ACK 5e50e2d1b9

Tree-SHA512: 92a959bb5dd414c96f78cb8dcaa68adb85faf16b8b843a2cbe0bb2aa08df13ad6bd9424d29b98f57a82ec29c942fbdbea3011883d00bf0b0feb643e295174e46
2021-03-04 10:16:38 +01:00
MarcoFalke
d099894ec1
Merge #20969: test: check that getblockfilter RPC fails without block filter index
233a886b42 test: check that getblockfilter RPC fails without block filter index (Sebastian Falbesoner)

Pull request description:

  If a node was started without compact block filter index (parameter `--blockfilterindex=0`), the `getblockfilter` RPC call should fail.

ACKs for top commit:
  MarcoFalke:
    review ACK 233a886b42

Tree-SHA512: c8824373fad7d1de2dcb43c1d9541d736b478235be243080d2b7479c2588eac0e5722337ec1307394b331e0002fbcabb368e4955c2dc98dd5fce76d8c089e8a1
2021-03-04 08:37:10 +01:00
Martin Zumsande
a061a29970 test: bring p2p_leak.py up to date.
After the introduction of wtxidrelay and sendaddrv2 messages during
version handshake, extend p2p_leak.py test to reflect this.
Also, some minor fixes and doc improvements.
2021-03-03 00:16:16 +01:00
Wladimir J. van der Laan
b9f41df1ea
Merge #20685: Add I2P support using I2P SAM
a701fcf01f net: Do not skip the I2P network from GetNetworkNames() (Vasil Dimov)
0181e24439 net: recognize I2P from ParseNetwork() so that -onlynet=i2p works (Vasil Dimov)
b905363fa8 net: accept incoming I2P connections from CConnman (Vasil Dimov)
0635233a1e net: make outgoing I2P connections from CConnman (Vasil Dimov)
9559bd1404 net: add I2P to the reachability map (Vasil Dimov)
76c35c60f3 init: introduce I2P connectivity options (Vasil Dimov)
c22daa2ecf net: implement the necessary parts of the I2P SAM protocol (Vasil Dimov)
5bac7e45e1 net: extend Sock with a method to check whether connected (Vasil Dimov)
42c779f503 net: extend Sock with methods for robust send & read until terminator (Vasil Dimov)
ea1845315a net: extend Sock::Wait() to report a timeout (Vasil Dimov)
78fdfbea66 net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions (Vasil Dimov)
34bcfab562 net: move the constant maxWait out of InterruptibleRecv() (Vasil Dimov)
cff65c4a27 net: extend CNetAddr::SetSpecial() to support I2P (Vasil Dimov)
f6c267db3b net: avoid unnecessary GetBindAddress() call (Vasil Dimov)
7c224fdac4 net: isolate the protocol-agnostic part of CConnman::AcceptConnection() (Vasil Dimov)
1f75a653dd net: get the bind address earlier in CConnman::AcceptConnection() (Vasil Dimov)
25605895af net: check for invalid socket earlier in CConnman::AcceptConnection() (Vasil Dimov)
545bc5f81d util: fix WriteBinaryFile() claiming success even if error occurred (Vasil Dimov)
8b6e4b3b23 util: fix ReadBinaryFile() returning partial contents (Vasil Dimov)
4cba2fdafa util: extract {Read,Write}BinaryFile() to its own files (Vasil Dimov)

Pull request description:

  Add I2P support by using the [I2P SAM](https://geti2p.net/en/docs/api/samv3) protocol. Unlike Tor, for incoming connections we get the I2P address of the peer (and they also receive ours when we are the connection initiator).

  Two new options are added:

  ```
    -i2psam=<ip:port>
         I2P SAM proxy to reach I2P peers and accept I2P connections (default:
         none)

    -i2pacceptincoming
         If set and -i2psam is also set then incoming I2P connections are
         accepted via the SAM proxy. If this is not set but -i2psam is set
         then only outgoing connections will be made to the I2P network.
         Ignored if -i2psam is not set. Notice that listening for incoming
         I2P connections is done through the SAM proxy, not by binding to
         a local address and port (default: true)
  ```

  # Overview of the changes

  ## Make `ReadBinary()` and `WriteBinary()` reusable

  We would need to dump the I2P private key to a file and read it back later. Move those two functions out of `torcontrol.cpp`.

  ```
  util: extract {Read,Write}BinaryFile() to its own files
  util: fix ReadBinaryFile() returning partial contents
  util: fix WriteBinaryFile() claiming success even if error occurred
  ```

  ## Split `CConnman::AcceptConnection()`

  Most of `CConnman::AcceptConnection()` is agnostic of how the socket was accepted. The other part of it deals with the details of the `accept(2)` system call. Split those so that the protocol-agnostic part can be reused if we accept a socket by other means.

  ```
  net: check for invalid socket earlier in CConnman::AcceptConnection()
  net: get the bind address earlier in CConnman::AcceptConnection()
  net: isolate the protocol-agnostic part of CConnman::AcceptConnection()
  net: avoid unnecessary GetBindAddress() call
  ```

  ## Implement the I2P [SAM](https://geti2p.net/en/docs/api/samv3) protocol (not all of it)

  Just the parts that would enable us to make outgoing and accept incoming I2P connections.

  ```
  net: extend CNetAddr::SetSpecial() to support I2P
  net: move the constant maxWait out of InterruptibleRecv()
  net: dedup MSG_NOSIGNAL and MSG_DONTWAIT definitions
  net: extend Sock::Wait() to report a timeout
  net: extend Sock with methods for robust send & read until terminator
  net: extend Sock with a method to check whether connected
  net: implement the necessary parts of the I2P SAM protocol
  ```

  ## Use I2P SAM to connect to and accept connections from I2P peers

  Profit from all of the preceding commits.

  ```
  init: introduce I2P connectivity options
  net: add I2P to the reachability map
  net: make outgoing I2P connections from CConnman
  net: accept incoming I2P connections from CConnman
  net: recognize I2P from ParseNetwork() so that -onlynet=i2p works
  net: Do not skip the I2P network from GetNetworkNames()
  ```

ACKs for top commit:
  laanwj:
    re-ACK a701fcf01f
  jonatack:
    re-ACK a701fcf01f reviewed diff per `git range-diff ad89812 2a7bb34 a701fcf`, debug built and launched bitcoind with i2pd v2.35 running a dual I2P+Torv3 service with the I2P config settings listed below (did not test `onlynet=i2p`); operation appears nominal (same as it has been these past weeks), and tested the bitcoind help outputs grepping for `-i i2p` and the rpc getpeerinfo and getnetworkinfo helps

Tree-SHA512: de42090c9c0bf23b43b5839f5b4fc4b3a2657bde1e45c796b5f3c7bf83cb8ec6ca4278f8a89e45108ece92f9b573cafea3b42a06bc09076b40a196c909b6610e
2021-03-02 11:50:13 +01:00
MarcoFalke
cfce346508
Merge #21310: zmq test: fix sync-up by matching notification to generated block
8a8c6383f6 zmq test: fix sync-up by matching notification to generated block (Sebastian Falbesoner)

Pull request description:

  This is a follow-up PR for #21008, fixes #21216.

  In the course of investigating the problem with jnewbery (analyzing the Cirrus log https://cirrus-ci.com/task/4660108304056320), it turned out that the "sync up" procedure of repeatedly generating a block and waiting for a notification with timeout is too brittle in its current form, as the following scenario could happen:

  - generate block A
  - receive notification, timeout happens => repeat procedure
  - generate block B
  - node publishes block A notification
  - receive notification, we receive the one caused by block A (!!!) => sync-up procedure is completed
  - node publishes block B notification
  - the actual test starts
  - on the first notification reception, the one caused by block B is received, rather than the one actually caused by test code => assertion failure

  This change in the PR ensures that after each test block generation, we wait for the notification that is actually caused by that block and ignore others from possibly earlier blocks. The matching is kind of ugly, it assumes that one out of four components in the block is contained in the notification: the block hash, the tx id, the raw block data or the raw transaction data. (Unfortunately we have to support all publisher topics.)

  I'm aware that this is quite a lot of code now only for establishing a robust test setup. OTOH I wouldn't know of a better method right now, suggestions are very welcome.

  Note for potential reviewers: for both reproducing the issue on master branch and verifying on PR branch, one can simply generate two blocks in the sync-up procedure rather than one.

ACKs for top commit:
  MarcoFalke:
    Concept ACK 8a8c6383f6

Tree-SHA512: a2eb78ba06dfd0fda7b1c111b6bbfb5dab4ab08500cc19c7ea02c3239495d5c74cc7d45250a8b3ecc78ca42d97ee6719bf73db8a137839e5e09a3cfcf08ed29e
2021-03-02 11:30:17 +01:00
Vasil Dimov
a701fcf01f
net: Do not skip the I2P network from GetNetworkNames()
So that help texts include "i2p" in:
* `./bitcoind -help` (in `-onlynet` description)
* `getpeerinfo` RPC
* `getnetworkinfo` RPC

Co-authored-by: Jon Atack <jon@atack.com>
2021-03-01 18:19:47 +01:00
Vasil Dimov
76c35c60f3
init: introduce I2P connectivity options
Introduce two new options to reach the I2P network:

* `-i2psam=<ip:port>` point to the I2P SAM proxy. If this is set then
  the I2P network is considered reachable and we can make outgoing
  connections to I2P peers via that proxy. We listen for and accept
  incoming connections from I2P peers if the below is set in addition to
  `-i2psam=<ip:port>`

* `-i2pacceptincoming` if this is set together with `-i2psam=<ip:port>`
  then we accept incoming I2P connections via the I2P SAM proxy.
2021-03-01 18:19:46 +01:00
Russell O'Connor
6a0a6e7d05 Correction for VerifyTaprootCommitment comments
According to BIP-341, 'p' is called the taproot *internal* key, not inner key.
2021-03-01 09:01:48 -05:00
Wladimir J. van der Laan
362e901a17
Merge #18466: rpc: fix invalid parameter error codes for {sign,verify}message RPCs
a5cfb40e27 doc: release note for changed {sign,verify}message error codes (Sebastian Falbesoner)
9e399b9b2d test: check parameter validity in rpc_signmessage.py (Sebastian Falbesoner)
e62f0c71f1 rpc: fix {sign,message}verify RPC errors for invalid address/signature (Sebastian Falbesoner)

Pull request description:

  RPCs that accept address parameters usually return the intended error code `RPC_INVALID_ADDRESS_OR_KEY` (-5) if a passed address is invalid. The two exceptions to the rule are `signmessage` and `verifymessage`, which return `RPC_TYPE_ERROR` (-3) in this case instead. Oddly enough `verifymessage` returns `RPC_INVALID_ADDRESS_OR_KEY` when the _signature_ was malformed, where `RPC_TYPE_ERROR` would be more approriate.

  This PR fixes these inaccuracies and as well adds tests to `rpc_signmessage.py` that check the parameter validity and error codes for the related RPCs `signmessagewithprivkey`, `signmessage` and `verifymessage`.

  master branch:
  ```
  $ ./bitcoin-cli signmessage invalid_addr message
  error code: -3
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage invalid_addr dummy_sig message
  error code: -3
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX invalid_sig message
  error code: -5
  error message:
  Malformed base64 encoding
  ```
  PR branch:
  ```
  $ ./bitcoin-cli signmessage invalid_addr message
  error code: -5
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage invalid_addr dummy_sig message
  error code: -5
  error message:
  Invalid address
  $ ./bitcoin-cli verifymessage 12c6DSiU4Rq3P4ZxziKxzrL5LmMBrzjrJX invalid_sig message
  error code: -3
  error message:
  Malformed base64 encoding
  ```

ACKs for top commit:
  laanwj:
    Code review ACK a5cfb40e27
  meshcollider:
    utACK a5cfb40e27

Tree-SHA512: bae0c4595a2603cea66090f6033785601837b45fd853052312b3a39d8520566c581994b68f693dd247c22586c638c3b7689c849085cce548cc36b9bf0e119d2d
2021-03-01 11:45:42 +01:00
Sebastian Falbesoner
8a8c6383f6 zmq test: fix sync-up by matching notification to generated block
It turned out that the "sync up" procedure of repeatedly generating a
block and waiting for a notification once with timeout is too naive in
its current form, as the following scenario could happen:
    - generate block A
    - receive notification, timeout happens -> repeat procedure
    - generate block B
    - node publishes block A notification
    - receive notification, we receive the one caused by block A
      -> sync-up procedure is completed
    - node publishes block B
    - the actual test starts
    - on the first notification reception, one caused by block B is received,
      rather than the one actually caused by test code, leading to failure

This change ensures that after each test block generation, we wait for
the notification that is actually caused by that block and ignore others
from possibly earlier blocks.

Co-authored-by: Jon Atack <jon@atack.com>
2021-03-01 01:31:41 +01:00
MarcoFalke
fb67caebe2
Merge #21297: test: feature_blockfilterindex_prune.py improvements
88c4b9b761 test: remove unneeded node from feature_blockfilterindex_prune.py (Jon Atack)
ace3f4cbdf test: improve assertions in feature_blockfilterindex_prune.py (Jon Atack)

Pull request description:

  - improves the assertions
  - removes an unneeded node, reducing from two to one, and some unneeded `extra_arg` code

ACKs for top commit:
  MarcoFalke:
    ACK 88c4b9b761
  brunoerg:
    Tested ACK 88c4b9b761

Tree-SHA512: 295700da3a5f583ee02ae2d184db93cce0e13aba69115d5db07f83e96a66b4b850adaff2c3725b6585799565b9ee654b1fee8a6245eaba8c21e1cb5ce524eb2b
2021-02-27 13:11:58 +01:00
Anthony Towns
6bd4963c06 txorphanage: Move functions and data into class
Collects all the orphan handling globals into a single member var in
net_processing, and ensures access is encapuslated into the interface
functions. Also adds doxygen comments for methods.
2021-02-27 01:07:55 +10:00
Wladimir J. van der Laan
8d37841cdf
Merge #21277: wallet: listdescriptors uses normalized descriptor form
a69c3b35f8 wallet: listdescriptors uses normalized descriptor form (Ivan Metlushko)

Pull request description:

  Rationale: show importable descriptors with `listdescriptors` RPC

  It uses #19136 to derive xpub at the last hardened step.

  **Before**:
  ```
  [
      {
        "desc": "wpkh(tpubD6NzVbkrYhZ4YUQRJL49TWw1VR5v3QKUNYaGGMUfJUm19x5ZqQ2hEiPiYbAQvD2nHoPGQGPg3snLPM8sjmYpvx7XQhkmyfk8xhsUwKbXzyh/84'/1'/0'/0/*)#p4cn3erf",
        "timestamp": 1613982591,
        "active": true,
        "internal": false,
        "range": [
          0,
          999
        ],
        "next": 0
      },
      ...
  ]
  ```

  **After**:
  ```
  [
    {
      "desc": "wpkh([d4ade89c/84'/1'/0']tpubDDUEYcVXy6Vh5meHvcXN3sAr4k3fWwLZGpAHbkAHL8EnkDxp4d99CjNhJHfM2fUJicANvAKnCZS6XaVAgwAeKYc1KesGCN5qbQ25qQHrRxM/0/*)#8wq8rcft",
      "timestamp": 1613982591,
      "active": true,
      "internal": false,
      "range": [
        0,
        999
      ],
      "next": 0
    },
    ...
  ]
  ```

ACKs for top commit:
  achow101:
    ACK a69c3b35f8

Tree-SHA512: 4f92e726cb8245aa0b520729cfd272945f0c66830f0555544e0618883aca516318543fa6ab1862058c64b4e4ed54ad1da953e032f4846eef7454122031c1b005
2021-02-26 10:08:02 +01:00
Jon Atack
88c4b9b761
test: remove unneeded node from feature_blockfilterindex_prune.py 2021-02-25 18:05:59 +01:00
Jon Atack
ace3f4cbdf
test: improve assertions in feature_blockfilterindex_prune.py 2021-02-25 15:10:19 +01:00
MarcoFalke
c0e44ee8e4
Merge #21254: test: Avoid connecting to real network when running tests
fa730e9157 test: Avoid connecting to real network when running tests (MarcoFalke)
fa1b713941 test: Assume node is running in subtests (MarcoFalke)

Pull request description:

  Introduced in #19884

ACKs for top commit:
  Sjors:
    ACK fa730e9157

Tree-SHA512: fe132a9ffe2fae1ab16857a3dec9839526fdf74d27a1ae794fbffca8356f639c4b916dc888b260281e9cc793916706c18d1687ebb5a076d4e1c481d218d308d3
2021-02-25 14:44:22 +01:00
MarcoFalke
1b1d8bde1c
Merge #21252: test: Add missing wait for sync to feature_blockfilterindex_prune
fa560cc6c4 test: Intermittent issue in feature_blockfilterindex_prune (MarcoFalke)

Pull request description:

  https://cirrus-ci.com/task/4962244553342976?command=ci#L5131

  The index is built in a background thread, so we have to wait for it.

ACKs for top commit:
  jonatack:
    ACK fa560cc6c4

Tree-SHA512: e7a246fe43a28511581fe34b1f5a85303b1874b2535330afc0405269cce7306984ecc6af389791321e3aa4b224819e89d9b89dd5bc080d60baa20bd007412787
2021-02-25 14:32:46 +01:00
Wladimir J. van der Laan
2059d32edb
Merge #21200: test: Speed up rpc_blockchain.py by removing miniwallet.generate()
faa137eb9e test: Speed up rpc_blockchain.py by removing miniwallet.generate() (MarcoFalke)
fa1fe80c75 test: Change address type from P2PKH to P2WSH in rpc_blockchain (MarcoFalke)
fa4d8f3169 test: Cache 25 mature coins for ADDRESS_BCRT1_P2WSH_OP_TRUE (MarcoFalke)
fad25153f5 test: Remove unused bug workaround (MarcoFalke)
faabce7d07 test: Start only the number of nodes that are needed (MarcoFalke)

Pull request description:

  Speed up various tests:

  * Remove unused nodes, which only consume time on start/stop
  * Remove unused "bug workarounds"
  * Remove the need for `miniwallet.generate()` by adding `miniwallet.scan_blocks()`. (On my system, with valgrind, generating 105 blocks takes 3.31 seconds. Rescanning 5 blocks takes 0.11 seconds.)

ACKs for top commit:
  laanwj:
    Code review ACK faa137eb9e

Tree-SHA512: ead1988d5aaa748ef9f8520af1e0bf812cf1d72e281ad22fbd172b7306d850053040526f8adbcec0b9a971c697a0ee7ee8962684644d65b791663eedd505a025
2021-02-25 10:13:39 +01:00
MarcoFalke
fa80e10d94
test: Add feature_taproot.py --previous_release 2021-02-25 08:51:11 +01:00
Sjors Provoost
85ccffa266 test: move releases download incantation to README 2021-02-25 08:50:34 +01:00
Sjors Provoost
29d6b1da2a test: previous releases: add v0.20.1
Can be reviewed with --ignore-all-space
2021-02-25 08:50:34 +01:00
MarcoFalke
1b045b5eef
Merge #21053: rpc, test: document {previous,next}blockhash as optional
ba7e17e073 rpc, test: document {previous,next}blockhash as optional (Sebastian Falbesoner)

Pull request description:

  This PR updates the result help of the following RPCs w.r.t. the `previousblockhash` and `nextblockhash` fields:
  - getblockheader
  - getblock

  Also adds trivial tests on genesis block (should not contain "previousblockhash") and best block (should not contain "nextblockhash").

Top commit has no ACKs.

Tree-SHA512: ef42c5c773fc436e1b4a67be14e2532e800e1e30e45e54a57431c6abb714d2c069c70d40ea4012d549293b823a1973b3f569484b3273679683b28ed40abf46bb
2021-02-23 18:28:23 +01:00
Sjors Provoost
d4b0107d68
rpc: send: support external signer 2021-02-23 14:34:32 +01:00
Sjors Provoost
245b4457cf
rpc: signerdisplayaddress 2021-02-23 14:34:31 +01:00
Sjors Provoost
7ebc7c0215
wallet: ExternalSigner: add GetDescriptors method 2021-02-23 14:34:31 +01:00
Sjors Provoost
259f52cc33
test: external_signer wallet flag is immutable 2021-02-23 14:34:31 +01:00
Sjors Provoost
2655197e1c
rpc: add external_signer option to createwallet 2021-02-23 14:34:31 +01:00
Sjors Provoost
2700f09c41
rpc: signer: add enumeratesigners to list external signers 2021-02-23 14:34:31 +01:00
Sjors Provoost
07b7c940a7
rpc: add external signer RPC files 2021-02-23 14:34:30 +01:00
Sjors Provoost
f3e6ce78fb
test: add external signer test
Includes a mock to mimick the HWI interace.
2021-02-23 14:34:30 +01:00
Ivan Metlushko
a69c3b35f8 wallet: listdescriptors uses normalized descriptor form 2021-02-23 08:51:01 +01:00
Sjors Provoost
f7eb7ecc67
test: framework: add skip_if_no_external_signer 2021-02-21 16:27:10 +01:00
MarcoFalke
fa730e9157
test: Avoid connecting to real network when running tests
Can be reviewed with --word-diff-regex=.
2021-02-21 11:02:58 +01:00
MarcoFalke
fa1b713941
test: Assume node is running in subtests
Every (sub)test in the framework assumes the node is running, except for
the (sub)tests in this file. Remove that confusion by stopping the node
at the start of every subtest, instead of at the end.
2021-02-21 11:01:53 +01:00
MarcoFalke
fa560cc6c4
test: Intermittent issue in feature_blockfilterindex_prune 2021-02-21 08:08:08 +01:00
Jonas Schnelli
9f3ffa2938
Merge #21230: test: Fix NODE_NETWORK_LIMITED_MIN_BLOCKS disconnection
fa24247d0f test: Fix NODE_NETWORK_LIMITED_MIN_BLOCKS disconnection (MarcoFalke)
fab6995629 test: Make test actually test something (MarcoFalke)
fae8f35df8 test: pep8 touched test (MarcoFalke)

Pull request description:

  Fix several bugs. Also, fix #21227

ACKs for top commit:
  jonasschnelli:
    utACK fa24247d0f - thanks for fixing.
  ryanofsky:
    Code review ACK fa24247d0f with caveat above that I don't really understand the problem or fix. But the cleanups look good and the fix does seem perfectly safe. More description would be welcome!

Tree-SHA512: 67f6ec92f6493aa822ae3fa8a7426a5acdc684044b8bafc0c65b652f63ccce969d0a6f1d1f099d6a91d05f478724869345b70335f2cfcfd00df46aef05cc4f9e
2021-02-19 08:38:01 +01:00
Samuel Dobson
3a2d5bfeb3
Merge #21201: rpc: Disallow sendtoaddress and sendmany when private keys disabled
6bfbc97d71 test: disallow sendtoaddress/sendmany when private keys disabled (Jon Atack)
0997019e76 Disallow sendtoaddress and sendmany when private keys disabled (Andrew Chow)

Pull request description:

  Since `sendtoaddress` and `sendmany` (which use the `SendMoney` function) create and commit a transaction, they should not do anything when the wallet does not have private keys. Otherwise a valid transaction cannot be made.

  Fixes #21104

ACKs for top commit:
  jonatack:
    ACK 6bfbc97d71
  meshcollider:
    utACK 6bfbc97d71
  kristapsk:
    ACK 6bfbc97d71. "Error: Private keys are disabled for this wallet" is definitely a better error message than "Insufficient funds" here. Hopefully change of error code from -6 to -4 doesn't break any software using Bitcoin JSON-RPC API.

Tree-SHA512: f277d6b5252e43942d568614032596f2c0827f00cd0cb71e44ffcb9822bfb15a71730a3e3688f31e59ba4eb7d275250c4e65ad4b6b3e96be6314c56a672432fb
2021-02-19 14:00:48 +13:00
MarcoFalke
fa24247d0f
test: Fix NODE_NETWORK_LIMITED_MIN_BLOCKS disconnection 2021-02-18 20:45:05 +01:00
MarcoFalke
fab6995629
test: Make test actually test something
The context manager was not even created, so previously it did not check the debug log
2021-02-18 20:43:35 +01:00
MarcoFalke
fae8f35df8
test: pep8 touched test 2021-02-18 20:43:32 +01:00
Wladimir J. van der Laan
b805dbb0b9
Merge #19809: log: Prefix log messages with function name and source code location if -logsourcelocations is set
b4511e2e2e log: Prefix log messages with function name if -logsourcelocations is set (practicalswift)

Pull request description:

  Prefix log messages with function name if `-logfunctionnames` is set.

  Yes, exactly like `-logthreadnames` but for function names instead of thread names :)

  This is a small developer ergonomics improvement: I've found this to be a cheap/simple way to correlate log output and originating function.

  For me it beats the ordinary cycle of 1.) try to figure out a regexp matching the static part of the dynamic log message, 2.) `git grep -E 'Using .* MiB out of .* requested for signature cache'`, 3.) `mcedit filename.cpp` (`openemacs filename.cpp` works too!) and 4.) search for log message and scroll up to find the function name :)

  Without any logging parameters:

  ```
  $ src/bitcoind -regtest
  2020-08-25T03:29:04Z Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z block tree size = 1
  2020-08-25T03:29:04Z nBestHeight = 0
  2020-08-25T03:29:04Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z 0 addresses found from DNS seeds
  ```

  With `-logthreadnames` and `-logfunctionnames`:

  ```
  $ src/bitcoind -regtest -logthreadnames -logfunctionnames
  2020-08-25T03:29:04Z [init] [ReportHardwareRand] Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z [init] [InitSignatureCache] Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [InitScriptExecutionCache] Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [LoadChainTip] Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z [init] [AppInitMain] block tree size = 1
  2020-08-25T03:29:04Z [init] [AppInitMain] nBestHeight = 0
  2020-08-25T03:29:04Z [loadblk] [LoadMempool] Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z [dnsseed] [ThreadDNSAddressSeed] 0 addresses found from DNS seeds
  ```

ACKs for top commit:
  laanwj:
    Code review ACK b4511e2e2e
  MarcoFalke:
    review ACK b4511e2e2e 🌃

Tree-SHA512: d100f5364630c323f31d275259864c597f7725e462d5f4bdedcc7033ea616d7fc0d16ef1b2af557e692f4deea73c6773ccfc681589e7bf6ba970b9ec169040c7
2021-02-18 14:37:51 +01:00
Wladimir J. van der Laan
860f916803
Merge #20524: test: Move MIN_VERSION_SUPPORTED to p2p.py
9f21ed4037 [test] Check user agent string from test framework connections (John Newbery)
9ce4c3c4c1 [test] Add P2P_SERVICES to p2p.py (John Newbery)
010542614d [test] Move MY_RELAY to p2p.py (John Newbery)
9b4054cb7a [test] Move MY_SUBVERSION to p2p.py (John Newbery)
7e158a6910 [test] Move MY_VERSION to p2p.py (John Newbery)
652311165c [test] Move MIN_VERSION_SUPPORTED to p2p.py (John Newbery)

Pull request description:

  The messages.py module should contain code and helpers for
  [de]serializing p2p messages. Specific usage of those messages should
  be in p2p.py. This PR moves test framework specific constants to p2p.py.

  It also changes the SUBVERSION constant to be a string instead of a bytes object. That means that it needs to be explicitly converted to a bytes object to serialize into a version message. Failing to do so would cause an easy-to-spot bug. This should avoid silent failures like the one solved in #20522.

ACKs for top commit:
  laanwj:
    Code review ACK 9f21ed4037

Tree-SHA512: 41d46575ac0ec36ad074d6c6a5b9cef50b05eeb8ddd8ed0a8f0d0c4617cc7b8baa6580af5b83a668230ce1ac27bf0e56914d0361a48b1b05fd75e2e60350eeaf
2021-02-18 14:01:57 +01:00
Samuel Dobson
db656db2ed
Merge #19136: wallet: add parent_desc to getaddressinfo
de6b389d5d tests: Test getaddressinfo parent_desc (Andrew Chow)
e4ac869a0a rpc: Add parent descriptor to getaddressinfo output (Andrew Chow)
bbe4a36152 wallet: Add GetDescriptorString to DescriptorScriptPubKeyMan (Andrew Chow)
9be1437c49 descriptors: Add ToNormalizedString and tests (Andrew Chow)

Pull request description:

  Adds `parent_desc` field to the `getaddressinfo` RPC to export a public descriptor. Using the given address, `getaddressinfo` will look up which `DescriptorScriptPubKeyMan` can be used to produce that address. It will then return the descriptor for that `DescriptorScriptPubKeyMan` in the `parent_desc` field. The descriptor will be in a normalized form where the xpub at the last hardened step is derived so that the descriptor can be imported to other wallets. Tests are added to check that the correct descriptor is being returned for the wallet's addresses and that these descriptors can be imported and used in other wallets.

  As part of this PR, a `ToNormalizedString` function is added to the descriptor classes. This really only has an effect on `BIP32PubkeyProvider`s that have hardened derivation steps. Tests are added to check that normalized descriptors are returned.

ACKs for top commit:
  Sjors:
    utACK de6b389d5d
  S3RK:
    Tested ACK de6b389
  jonatack:
    Tested ACK de6b389d5d modulo a few minor comments
  fjahr:
    Code review ACK de6b389d5d
  meshcollider:
    Tested ACK de6b389d5d

Tree-SHA512: a633e4a39f2abbd95afd7488484cfa66fdd2651dac59fe59f2b80a0940a2a4a13acf889c534a6948903d701484a2ba1218e3081feafe0b9a720dccfa9e43ca2b
2021-02-18 21:51:16 +13:00
Jonas Schnelli
9017d55e7c
Merge #15946: Allow maintaining the blockfilterindex when using prune
84716b134e Add "index/blockfilterindex -> validation -> index/blockfilterindex" to expected circular dependencies (Jonas Schnelli)
ab3a0a2fb9 Add functional test for blockfilterindex in prune-mode (Jonas Schnelli)
c286a22f7b Add debug startup parameter -fastprune for more effective pruning tests (Jonas Schnelli)
5e112269c3 Avoid pruning below the blockfilterindex sync height (Jonas Schnelli)
00d57ff768 Avoid accessing nullpointer in BaseIndex::GetSummary() (Jonas Schnelli)
6abe9f5b11 Allow blockfilter in conjunction with prune (Jonas Schnelli)

Pull request description:

  Maintaining the blockfilterindexes in prune mode is possible and may lead to efficient p2p based rescans of wallets (restore backups, import/sweep keys) beyond the prune height (rescans not part of that PR).

  This PR allows running the blockfilterindex(es) in conjunction with pruning.
  * Bitcoind/Qt will shutdown during startup when missing block data has been detected ([re]enable `-blockfilterindex` when we already have pruned)
  * manual block pruning is disabled during blockfilterindex sync
  * auto-pruning is delayed during blockfilterindex sync

  ToDos:
  * [x] Functional tests

ACKs for top commit:
  fjahr:
    Code review ACK 84716b1
  ryanofsky:
    Code review ACK 84716b134e. Only changes since last review were suggested new FindFilesToPrune argument and test.
  benthecarman:
    tACK 84716b134e

Tree-SHA512: 91d832c6c562c463f7ec7655c08956385413a99a896640b9737bda0183607fac530435d03d87c3c0e70c61ccdfe73fe8f3639bc7d26d33ca7e60925ebb97d77a
2021-02-18 09:40:42 +01:00
John Newbery
9f21ed4037 [test] Check user agent string from test framework connections
Add a check that new connections from the test framework to the
node have the correct user agent string. This makes bugs easier
to detect if the user agent string ever changes.
2021-02-17 09:29:44 +00:00
John Newbery
9ce4c3c4c1 [test] Add P2P_SERVICES to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore specify the nServices value in the calling code,
not in the messages.py module.
2021-02-17 09:29:41 +00:00
John Newbery
010542614d [test] Move MY_RELAY to p2p.py
messages.py is for message and primitive data structures. Specifics
about the test framework's p2p implementation should be in p2p.py.

Also rename to P2P_VERSION_RELAY. Also rename msg_version.nRelay to
relay. In Bitcoin Core, this is referred to as fRelay, since it's a
bool, so this field has always been misnamed.
2021-02-17 09:23:32 +00:00
John Newbery
9b4054cb7a [test] Move MY_SUBVERSION to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore move MY_SUBVERSION to p2p.py.

Also rename to P2P_SUBVERSION.
2021-02-17 09:22:37 +00:00
John Newbery
7e158a6910 [test] Move MY_VERSION to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore move MY_VERSION to p2p.py.

Also rename to P2P_VERSION to distinguish it from
other versioning used in Bitcoin/Bitcoin Core.

Also always set the nVersion field in CBlockLocator to 0 and ignore the
field in deserialized messages. The field is not currently used for
anything in Bitcoin Core.
2021-02-17 09:00:53 +00:00
John Newbery
652311165c [test] Move MIN_VERSION_SUPPORTED to p2p.py
The messages.py module should contain code and helpers for
[de]serializing p2p messages. Specific usage of those messages should
be in p2p.py. Therefore move MIN_VERSION_SUPPORTED to p2p.py.

Also rename to MIN_P2P_VERSION_SUPPORTED to distinguish it from
other versioning used in Bitcoin/Bitcoin Core.
2021-02-17 09:00:24 +00:00
MarcoFalke
69f7f50aa5
Merge #20993: test: store subversion (user agent) as string in msg_version
de85af5cce test: store subversion (user agent) as string in msg_version (Sebastian Falbesoner)

Pull request description:

  It seems more natural to treat the "subversion" field (=user agent string, see [BIP 14](https://github.com/bitcoin/bips/blob/master/bip-0014.mediawiki#Proposal)) of a node as pure string rather than a bytestring within the test framework. This is also suggested with the naming prefix in `msg_version.strSubVer`: one probably wouldn't expect a field starting with "str" to be a bytestring that needs further decoding to be useful. This PR moves the encoding/decoding parts to the serialization/deserialization routines so that the user doesn't have to bother with that anymore.

  Note that currently, in the master branch the `msg_version.strSubVer` is never read (only in `msg_version.__repr__`); However, one issue that is solved by this PR came up while testing #19509 (not merged yet): A decoding script for binary message capture files takes use of the functional test framework convert it into JSON format. Bytestrings will be convered to hexstrings, while pure strings will (surprise surprise) end up without modification in the file.

  So without this patch, we get:

  ```
  $ jq . out.json | grep -m5 strSubVer
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
        "strSubVer": "2f5361746f7368693a302e32302e312f"
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
        "strSubVer": "2f5361746f7368693a302e32302e312f"
        "strSubVer": "2f5361746f7368693a32312e39392e302f"
  ```

  After this patch:

  ```
  $ jq . out2.json | grep -m5 strSubVer
        "strSubVer": "/Satoshi:21.99.0/"
        "strSubVer": "/Satoshi:0.20.1/"
        "strSubVer": "/Satoshi:21.99.0/"
        "strSubVer": "/Satoshi:0.20.1/"
        "strSubVer": "/Satoshi:21.99.0/"
  ```

ACKs for top commit:
  jnewbery:
    utACK de85af5cce

Tree-SHA512: ff23642705c858e8387a625537dfec82e6b8a15da6d99b8d12152560e52d243ba17431b602b26f60996d897e00e3f37dcf8dc8a303ffb1d544df29a5937080f9
2021-02-17 09:36:30 +01:00
Jon Atack
6bfbc97d71 test: disallow sendtoaddress/sendmany when private keys disabled 2021-02-16 15:49:28 -05:00
MarcoFalke
3c9d9d21e1
Merge #21008: test: fix zmq test flakiness, improve speed
ef21fb7313 zmq test: speedup test by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
5c6546362d zmq test: fix flakiness by using more robust sync method (Sebastian Falbesoner)
8666033630 zmq test: accept arbitrary sequence start number in ZMQSubscriber (Sebastian Falbesoner)
6014d6e1b5 zmq test: dedup message reception handling in ZMQSubscriber (Sebastian Falbesoner)

Pull request description:

  Fixes #20934 by using the "sync up" method described in https://github.com/bitcoin/bitcoin/issues/20538#issuecomment-738791868.

  After improving robustness with this approach (commits 1-3), it turned out that there were still some fails, but those were unrelated to zmq: Out of 500 runs, 3 times `sync_mempool()` or `sync_blocks()` timed out, which can happen because the trickle relay time has no upper bound -- hence in rare cases, it takes longer than 60s. This is fixed by enabling immediate tx relay on node1 (commit 4), which as a nice side-effect also gives us a rough 2x speedup for the test.

  For further details, also see the explanations in the commit messages.

  There is no guarantee that the test is still not flaky, but it would help if potential reviewers would run the following script locally and report how many runs failed (feel free to do less than 1000 runs, as this takes quite a long if ran with `--valgrind`):
  ```
  #!/bin/sh
  OUTPUT_FILE=./zmq_results
  echo ===== repeated zmq test ===== > $OUTPUT_FILE

  for i in `seq 1000`; do
      echo ------------------------
      echo ----- test run $i -----
      echo ------------------------
      echo --- $i --- >> $OUTPUT_FILE
      ./test/functional/interface_zmq.py --valgrind
      if [ $? -ne 0 ]; then
          echo "FAILED. /o\\" >> $OUTPUT_FILE
      else
          echo "PASSED. \\o/" >> $OUTPUT_FILE
      fi
  done

  echo Failed test runs:
  grep FAILED $OUTPUT_FILE | wc -l
  ```

ACKs for top commit:
  jonatack:
    Light ACK ef21fb7313 with the caveat that I was unable to make the test fail with valgrind both here and on master, so I can't vouch that it actually fixes the CI flakiness. The test does run ~2x faster with this.

Tree-SHA512: 7a1e7592fbbd98e69e1e1294486b91253e589c72b3c6bbb7f587028ec07cca59b7d984e4ebf256c4bc3e8a529ec77d31842f3dd874038aea0b684abfea50306a
2021-02-16 18:56:20 +01:00
MarcoFalke
faa137eb9e
test: Speed up rpc_blockchain.py by removing miniwallet.generate() 2021-02-16 17:47:55 +01:00
MarcoFalke
fa1fe80c75
test: Change address type from P2PKH to P2WSH in rpc_blockchain
This change does not matter for the test, except that it increases
the bogosize due to the increase in the size of the scriptPubKey
2021-02-16 17:47:48 +01:00
MarcoFalke
fa4d8f3169
test: Cache 25 mature coins for ADDRESS_BCRT1_P2WSH_OP_TRUE 2021-02-16 16:49:08 +01:00
MarcoFalke
fad25153f5
test: Remove unused bug workaround 2021-02-16 16:28:40 +01:00
MarcoFalke
faabce7d07
test: Start only the number of nodes that are needed 2021-02-16 16:25:50 +01:00
Jonas Schnelli
ab3a0a2fb9 Add functional test for blockfilterindex in prune-mode 2021-02-16 10:30:37 +01:00
MarcoFalke
489030f2a8
Merge #20965: net, rpc: return NET_UNROUTABLE as not_publicly_routable, automate helps
96635e6177 init: use GetNetworkNames() in -onlynet help (Jon Atack)
0dbde700a6 rpc: use GetNetworkNames() in getnetworkinfo and getpeerinfo helps (Jon Atack)
1c3af37881 net: create GetNetworkNames() (Jon Atack)
b45eae4d53 net: update NET_UNROUTABLE to not_publicly_routable in GetNetworkName() (Jon Atack)

Pull request description:

  per the IRC discussion today at http://www.erisian.com.au/bitcoin-core-dev/log-2021-01-19.html#l-87

  - return a more helpful string name for `Network::NET_UNROUTABLE`: "not_publicly_routable" instead of "unroutable"
  - update the RPC getpeerinfo "network" help, and automate it and the getnetworkinfo "network#name" and the -onlynet help doc generation

ACKs for top commit:
  theStack:
    re-ACK 96635e6177 🌳
  MarcoFalke:
    review ACK 96635e6177 🐗

Tree-SHA512: 511a7d987126b48a7a090739aa7c4964b6186a3ff8f5f7eec9233dd816c6b7a6dc91b3ea6b824aa68f218a8a3ebdc6ffd214e9a88af38f2bf23f3257c4284c3a
2021-02-15 15:31:15 +01:00
MarcoFalke
8d6994f93d
Merge #21100: test: remove unused function xor_bytes
f64adc1eed test: remove unused function xor_bytes (Sebastian Falbesoner)

Pull request description:

  The function `xor_bytes` was introduced in commit 3c226639eb (#19953, BIP340-342 validation), even [code-reviewed](https://github.com/bitcoin/bitcoin/pull/19953/files#r509383731), but actually never used. The [default signing algorithm in BIP340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki#Default_Signing) needs a xor operation, but this step is currently done by a single xor operation on large integer operands:

  ```
  t = (sec ^ int.from_bytes(TaggedHash("BIP0340/aux", aux), 'big')).to_bytes(32, 'big')
  ```

  Alternatively, we could keep the function and as well use it:
  ```diff
  --- a/test/functional/test_framework/key.py
  +++ b/test/functional/test_framework/key.py
  @@ -492,7 +492,7 @@ def sign_schnorr(key, msg, aux=None, flip_p=False, flip_r=False):
       P = SECP256K1.affine(SECP256K1.mul([(SECP256K1_G, sec)]))
       if SECP256K1.has_even_y(P) == flip_p:
           sec = SECP256K1_ORDER - sec
  -    t = (sec ^ int.from_bytes(TaggedHash("BIP0340/aux", aux), 'big')).to_bytes(32, 'big')
  +    t = xor_bytes(sec.to_bytes(32, 'big'), TaggedHash("BIP0340/aux", aux))
       kp = int.from_bytes(TaggedHash("BIP0340/nonce", t + P[0].to_bytes(32, 'big') + msg), 'big') % SECP256K1_ORDER
       assert kp != 0
       R = SECP256K1.affine(SECP256K1.mul([(SECP256K1_G, kp)]))
  ```

ACKs for top commit:
  practicalswift:
    cr ACK f64adc1eed: untested unused code should be removed

Tree-SHA512: e9afae303488f19c6f6f44874d3537ed1c8164a197490e2b4e34853db886b858825b719648fa1a30b95177dcee9cf241f94ee9b835f0a2cae07024ce38a8c090
2021-02-15 12:10:41 +01:00
Dhruv Mehta
d4187e4619 [test] Use mocktime in test_seed_peers()
Test case now takes < 5 seconds instead of > 2 minutes
2021-02-12 09:35:18 -08:00
Dhruv Mehta
015637dd44 [refactor] Correct log message in net.cpp 2021-02-12 09:23:03 -08:00
Wladimir J. van der Laan
e9c037ba64
Merge #19884: p2p: No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty
fe3e993968 [p2p] No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty. Add -fixedseeds arg. (Dhruv Mehta)

Pull request description:

  Closes #19795

  Before PR: If `peers.dat` is empty and `-dnsseed=0`, bitcoind will fallback on to fixed seeds but only after a 60 seconds delay.
  After PR: There's no 60 second delay.

  To reproduce:
  `rm ~/.bitcoin/peers.dat && src/bitcoind -dnsseed=0` without and with patch code

  Other changes in the PR:
  - `-fixedseeds` command line argument added: `-dnsseed=0 -fixedseeds=0 -addnode=X` provides a trusted peer only setup. `-dnsseed=0 -fixedseeds=0` allows for a `addnode` RPC to add a trusted peer without falling back to hardcoded seeds.

ACKs for top commit:
  LarryRuane:
    re-ACK fe3e993968
  laanwj:
    re-ACK fe3e993968

Tree-SHA512: 79449bf4e83a315be6dbac9bdd226de89d2a3f7f76d9c5640a2cb3572866e6b0e8ed67e65674c9824054cf13119dc01c7e1a33848daac6b6c34dbc158b6dba8f
2021-02-12 11:49:34 +01:00
Wladimir J. van der Laan
8d82eddee6
Merge #19145: Add hash_type MUHASH for gettxoutsetinfo
e987ae5a55 test: Add test for deterministic UTXO set hash results (Fabian Jahr)
6ccc8fc067 test: Add test for gettxoutsetinfo RPC with MuHash (Fabian Jahr)
0d3b2f643d rpc: Add hash_type MUHASH to gettxoutsetinfo (Fabian Jahr)
2474645f3b refactor: Separate hash and stats calculation in coinstats (Fabian Jahr)
a1fcceac69 refactor: Improve encapsulation between MuHash3072 and Num3072 (Fabian Jahr)

Pull request description:

  This is another Pr in the series PRs for Coinstatsindex (see overview in #18000). This PR adds the `hash_type` option `muhash` to `gettxoutsetinfo` through which the user can calculate the serialized muhash of the utxo set. This PR does not use the index yet.

ACKs for top commit:
  Sjors:
    tACK e987ae5
  achow101:
    ACK e987ae5a55
  jonatack:
    Tested re-ACK e987ae5a55 per `git diff 3506d90 e987ae5`, reviewed diff, debug built, ran gettxoutsetinfo -signet and help on this branch vs master, at height 23127 both returned `hash_serialized_2` of `2b72d65f3b6efb2311f58374ea2b939abf49684d44f4bafda45faa3b5452a454` and this branch returned `muhash` of `c9f1ff12d345ccf9939c6bbf087e6f7399b6115adee1569287e9c5c43dbb475c`
  ryanofsky:
    Code review ACK e987ae5a55. Looks very good. I left one suggestion to simplify code, but feel free to ignore it here and maybe consider it for later since PR has already had a lot of review.

Tree-SHA512: 9a739ce375e73749fa69a467262b60d3e5314ef384e2d7150b3bbc8e4125cd9fd1db95306623bb9a632fcbaf5d9d2bf2f5cc43bf717d4ff5e2c9c4b52dd9296c
2021-02-12 10:47:41 +01:00
Dhruv Mehta
fe3e993968 [p2p] No delay in adding fixed seeds if -dnsseed=0 and peers.dat is empty. Add -fixedseeds arg. 2021-02-11 16:10:40 -08:00
MarcoFalke
685c16fcb2
Merge #21043: net: Avoid UBSan warning in ProcessMessage(...)
3ddbf22ed1 util: Disallow negative mocktime (MarcoFalke)
f5f2f97168 net: Avoid UBSan warning in ProcessMessage(...) (practicalswift)

Pull request description:

  Avoid UBSan warning in `ProcessMessage(...)`.

  Context: https://github.com/bitcoin/bitcoin/pull/20380#issuecomment-770427182 (thanks Crypt-iQ!)

ACKs for top commit:
  MarcoFalke:
    re-ACK 3ddbf22ed1 only change is adding patch written by me
  ajtowns:
    ACK 3ddbf22ed1 -- code review only

Tree-SHA512: e8d7af0457ca86872b75a4e406c0a93aafd841c2962e244e147e748cc7ca118c56be0fdafe53765f4b291410030b2c3cc8f76f733b37a955d34fc885ab6037b9
2021-02-11 12:40:12 +01:00
Sebastian Falbesoner
ef21fb7313 zmq test: speedup test by whitelisting peers (immediate tx relay)
Speeds up the zmq test roughly by a factor of 2x (~20 sec. instead of
~40 sec.) and also avoids timeouts on the synchronization methods
(sync_mempool() / sync_blocks()) that happened with a slight chance.
This is due to the fact that there is no upper bound on the trickle
relay time, so even the default of 60s is sometimes too low. Fixed by
enabling immediate tx relay on node1.
2021-02-09 23:55:32 +01:00
Sebastian Falbesoner
5c6546362d zmq test: fix flakiness by using more robust sync method
After connecting the subscriber sockets to the node, there is no
guarantee that the node's zmq publisher interfaces are ready yet, which
means that potentially the first expected notification messages could
get lost and the test fails. Currently this is handled by just waiting
for a short period of time (200ms), which works most of the time but is
still problematic, as in some rare cases the setup time takes much
longer, even in the range of multiple seconds.

The solution in this commit approaches the problem by using a more
robust method of syncing up, originally proposed by instagibbs:
    1. Generate a block on the node
    2. Try to receive a notification on all subscribers
    3. If all subscribers get a message within the timeout (1 second),
       we are done, otherwise repeat starting from step 1
2021-02-09 23:55:23 +01:00
Sebastian Falbesoner
8666033630 zmq test: accept arbitrary sequence start number in ZMQSubscriber
The ZMQSubscriber reception methods currently assert that the first
received publisher message has a sequence number of zero. In order to
fix the current test flakiness via "syncing up" to nodes in the setup
phase, we have to cope with the situation that messages get lost and the
first actual received message has a sequence number larger than zero.
2021-02-09 22:54:01 +01:00
Sebastian Falbesoner
6014d6e1b5 zmq test: dedup message reception handling in ZMQSubscriber 2021-02-09 22:54:01 +01:00
Sebastian Falbesoner
f64adc1eed test: remove unused function xor_bytes 2021-02-09 17:58:21 +01:00
Bruno Garcia
c9095b738f test: remove unnecessary assignment in bdb 2021-02-09 10:22:50 -03:00
MarcoFalke
c4214d0e0d
Merge #21117: test: remove assert_blockchain_height
fa0a4d6c60 test: remove assert_blockchain_height (MarcoFalke)

Pull request description:

  This simplifies the code and solves intermittent timeouts caused by commit 0d39b5848a.

  E.g. https://cirrus-ci.com/task/5196092369272832?command=ci#L3126

  ```
   test  2021-02-08T12:27:56.275000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 127, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 180, in run_test
                                         self.assert_blockchain_height(self.nodes[0], 101)
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/feature_assumevalid.py", line 92, in assert_blockchain_height
                                         assert False, "blockchain too short after timeout: %d" % current_height
                                     AssertionError: blockchain too short after timeout: 101

ACKs for top commit:
  glozow:
    ACK fa0a4d6c60

Tree-SHA512: 3859b0c1581c21f03c775f119204cc3a219e5d86346883634886f6da46feaf254b9c6c49c1ec4581ffe409cdf05f6e91ec38c3976c3daf4a9e67f96ddc1e0dce
2021-02-09 07:43:47 +01:00
MarcoFalke
b09ad737ee
Merge #20944: rpc: Return total fee in getmempoolinfo
fa362064e3 rpc: Return total fee in mempool (MarcoFalke)

Pull request description:

  This avoids having to loop over the whole mempool to query each entry's fee

ACKs for top commit:
  achow101:
    ACK fa362064e3
  glozow:
    ACK fa362064e3 🧸
  jnewbery:
    ACK fa362064e3

Tree-SHA512: e2fa1664df39c9e187f9229fc35764ccf436f6f75889c5a206d34fff473fc21efbf2bb143f4ca7895c27659218c22884d0ec4195e7a536a5a96973fc9dd82d08
2021-02-08 20:36:46 +01:00
MarcoFalke
fa0a4d6c60
test: remove assert_blockchain_height 2021-02-08 16:56:46 +01:00
MarcoFalke
b401b09355
Merge #21107: test: remove type: comments in favour of actual annotations
9913419cc9 test: remove type: comments in favour of actual annotations (fanquake)

Pull request description:

  Now that we require Python 3.6+, we should be using variable type
  annotations directly rather than `# type:` comments.

  Also takes care of the discarded value issue in p2p_message_capture.py.
  See: https://github.com/bitcoin/bitcoin/pull/19509/files#r571674446.

ACKs for top commit:
  MarcoFalke:
    review ACK 9913419cc9
  jnewbery:
    Code review ACK 9913419cc9

Tree-SHA512: 63aba5eef6c1320578f66cf8a6d85ac9dbab9d30b0d21e6e966be8216e63606de12321320c2958c67933bf68d10f2e76e9c43928e5989614cea34dde4187aad8
2021-02-08 10:56:54 +01:00
MarcoFalke
c969ab43c3
Merge #21084: test: fix timeout decrease in feature_assumevalid
0d39b5848a test: fix timeout decrease in feature_assumevalid (Bruno Garcia)

Pull request description:

  This PR fixes the timeout decrease in assert_blockchain_height function.

ACKs for top commit:
  practicalswift:
    cr ACK 0d39b5848a: patch looks correct
  theStack:
    ACK 0d39b5848a ⏲️

Tree-SHA512: ae3c83420b4de4ad41f1b20b6e77c3a26a8c5ac4fb136b2645fde119545a413c61312f76a16473141774bc955d30ac4fc86e5ca6cf729f8978a17d0dab520feb
2021-02-08 10:38:52 +01:00
fanquake
9913419cc9
test: remove type: comments in favour of actual annotations
Now that we require Python 3.6+, we should be using variable type
annotations directly rather than # type: comments.

Also takes care of the discarded value issue in p2p_message_capture.py.
See: https://github.com/bitcoin/bitcoin/pull/19509/files#r571674446.
2021-02-08 13:24:44 +08:00
fanquake
cf26ca3911
Merge #21081: test: fix the unreachable code at feature_taproot
5e0cd25e29 fix the unreachable code at feature_taproot (Bruno Garcia)

Pull request description:

  This PR removes the unnecessary return statement at the beginning of the function that makes the rest of the function unreachable.

ACKs for top commit:
  practicalswift:
    cr ACK 5e0cd25e29: patch looks correct!
  sipa:
    ACK 5e0cd25e29.
  theStack:
    Tested ACK 5e0cd25e29 🏔️
  sanket1729:
    tACK 5e0cd25e29. I noted this a while ago while fixing feature_taproot.py for elements. Verified that the extreme ranges of CScriptNum are correct and the overflow case for `CHECKSIGADD` works as intended. Adding 1 to 2^31 - 1 results in an overflow, but the interpreter puts a `vch` of corresponding to 2^31 on stack. Even though it cannot be converted to CscriptNum(restricted to 4 bytes), it's result can still be compared by OP_EQUAL.

Tree-SHA512: fff9be3be94f4b3f3ccf24bf588d96e84d14806f82692dccd31631b0e5c79a7575a96c308cb5a4f610ab02e2f854b899f374437c33ecf6d52055d333f2de9b27
2021-02-08 10:24:37 +08:00
Wladimir J. van der Laan
a6b1bf6439
Merge #20267: Disable and fix tests for when BDB is not compiled
49797c3ccf tests: Disable bdb dump test when no bdb (Andrew Chow)
1194cf9269 Fix wallet_send.py wallet setup to work with descriptors (Andrew Chow)
fbaea7bfe4 Require legacy wallet for wallet_upgradewallet.py (Andrew Chow)
b1b679e0ab Explicitly mark legacy wallet tests as such (Andrew Chow)
09514e1bef Setup wallets for interface_zmq.py (Andrew Chow)
4d03ef9a73 Use MiniWallet in rpc_net.py (Andrew Chow)
4de23824b0 Setup wallets for interface_bitcoin_cli.py (Andrew Chow)
7c71c627d2 Setup wallets with descriptors for feature_notifications (Andrew Chow)
1f1bef8dba Have feature_filelock.py test both bdb and sqlite, depending on compiled (Andrew Chow)
c77975abc0 Disable upgrades tests that require BDB if BDB is not compiled (Andrew Chow)
1f20cac9d4 Disable wallet_descriptor.py bdb format check if BDB is not compiled (Andrew Chow)
3641597d7e tests: Don't make any wallets unless wallet is required (Andrew Chow)
b9b88f57a9 Skip legacy wallet reliant tests if BDB is not compiled (Andrew Chow)
6f36242389 tests: Set descriptors default based on compilation (Andrew Chow)

Pull request description:

  This PR fixes tests for when BDB is not compiled. Tests which rely on or test legacy wallet behavior are disabled and skipped when BDB is not compiled. For the components of some tests that are for legacy wallet things, those parts of the tests are skipped.

  For the majority of tests, changes are made so that they can be run with either legacy wallets or descriptor wallets without materially effecting the test. Most tests only need the wallet for balance and transactions, so the type of wallet is not an important part of those tests. Additionally, some tests are wallet agnostic and modified to instead use the test framework's MiniWallet.

ACKs for top commit:
  laanwj:
    ACK 49797c3ccf
  ryanofsky:
    Code review ACK 49797c3ccf. Only change since last review is dropping last commit. Previous review w/ suggestions for future followup is https://github.com/bitcoin/bitcoin/pull/20267#pullrequestreview-581508843

Tree-SHA512: 69659f8a81fb437ecbca962f4082c12835282dbf1fba7d9952f727a49e01981d749af9b09feda1c8ca737516c7d7a08ef17e782795df3fa69892d5021b41c1ed
2021-02-05 14:26:10 +01:00
Bruno Garcia
0d39b5848a test: fix timeout decrease in feature_assumevalid 2021-02-04 21:43:43 -02:00
Bruno Garcia
5e0cd25e29 fix the unreachable code at feature_taproot 2021-02-04 10:57:37 -02:00
MarcoFalke
ea5a50f92a
Merge #21042: doc, test: Improve setup_clean_chain documentation
590bda79e8 scripted-diff: Remove setup_clean_chain if default is not changed (Fabian Jahr)
98892f39e3 doc: Improve setup_clean_chain documentation (Fabian Jahr)

Pull request description:

  The first commit improves documentation on setup_clean_chain which is misunderstood quite frequently. Most importantly it fixes the TestShell docs which are simply incorrect.

  The second commit removes the instances of `setup_clean_clain` in functional tests where it is not changing the default.

  This used to be part of #19168 which also sought to rename`setup_clean_chain`.

ACKs for top commit:
  jonatack:
    ACK 590bda79e8

Tree-SHA512: a7881186e65d31160b8f84107fb185973b37c6e50f190a85c6e2906a13a7472bb4efa9440bd37fe0a9ac5cd2d1e8559870a7e4380632d9a249eca8980b945f3e
2021-02-04 10:30:06 +01:00
MarcoFalke
4e946ebcf1
Merge #20715: util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet
fa61b9d1a6 util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet (MarcoFalke)
7777105a24 refactor: Move all command dependend checks to ExecuteWalletToolFunc (MarcoFalke)
fa06bce4ac test: Add tests (MarcoFalke)
fac05ccdad wallet: [refactor] Pass ArgsManager to WalletAppInit (MarcoFalke)

Pull request description:

  This not only moves the parsing responsibility out from the wallet tool, but it also makes it easier to implement bitcoin-util #19937

  Fixes: #20902

ACKs for top commit:
  ajtowns:
    ACK fa61b9d1a6
  fjahr:
    Code review ACK fa61b9d1a6

Tree-SHA512: 79622b806e8bf9dcd0dc24a8a6687345710df57720992e83a41cd8d6762a6dc112044ebc58fcf6e8fbf45de29a79b04873c5b8c2494a1eaaf902a2884703e47b
2021-02-04 09:12:05 +01:00
Fotis Koutoupas
ae9d26a8f0
wallet: Fix already-loading error message grammar 2021-02-04 00:19:11 +02:00
MarcoFalke
384e090f93
Merge #19509: Per-Peer Message Capture
bff7c66e67 Add documentation to contrib folder (Troy Giorshev)
381f77be85 Add Message Capture Test (Troy Giorshev)
e4f378a505 Add capture parser (Troy Giorshev)
4d1a582549 Call CaptureMessage at appropriate locations (Troy Giorshev)
f2a77ff97b Add CaptureMessage (Troy Giorshev)
dbf779d5de Clean PushMessage and ProcessMessages (Troy Giorshev)

Pull request description:

  This PR introduces per-peer message capture into Bitcoin Core.  📓

  ## Purpose

  The purpose and scope of this feature is intentionally limited.  It answers a question anyone new to Bitcoin's P2P protocol has had: "Can I see what messages my node is sending and receiving?".

  ## Functionality

  When a new debug-only command line argument `capturemessages` is set, any message that the node receives or sends is captured.  The capture occurs in the MessageHandler thread.  When receiving a message, it is captured as soon as the MessageHandler thread takes the message off of the vProcessMsg queue.  When sending, the message is captured just before the message is pushed onto the vSendMsg queue.

  The message capture is as minimal as possible to reduce the performance impact on the node.  Messages are captured to a new `message_capture` folder in the datadir.  Each node has their own subfolder named with their IP address and port.  Inside, received and sent messages are captured into two binary files, msgs_recv.dat and msgs_sent.dat, like so:

  ```
  message_capture/203.0.113.7:56072/msgs_recv.dat
  message_capture/203.0.113.7:56072/msgs_sent.dat
  ```

  Because the messages are raw binary dumps, included in this PR is a Python parsing tool to convert the binary files into human-readable JSON.  This script has been placed on its own and out of the way in the new `contrib/message-capture` folder.  Its usage is simple and easily discovered by the autogenerated `-h` option.

  ## Future Maintenance

  I sympathize greatly with anyone who says "the best code is no code".

  The future maintenance of this feature will be minimal.  The logic to deserialize the payload of the p2p messages exists in our testing framework.  As long as our testing framework works, so will this tool.

  Additionally, I hope that the simplicity of this tool will mean that it gets used frequently, so that problems will be discovered and solved when they are small.

  ## FAQ

  "Why not just use Wireshark"

  Yes, Wireshark has the ability to filter and decode Bitcoin messages.  However, the purpose of the message capture added in this PR is to assist with debugging, primarily for new developers looking to improve their knowledge of the Bitcoin Protocol.  This drives the design in a different direction than Wireshark, in two different ways.  First, this tool must be convenient and simple to use.  Using an external tool, like Wireshark, requires setup and interpretation of the results.  To a new user who doesn't necessarily know what to expect, this is unnecessary difficulty.  This tool, on the other hand, "just works".  Turn on the command line flag, run your node, run the script, read the JSON.  Second, because this tool is being used for debugging, we want it to be as close to the true behavior of the node as possible.  A lot can happen in the SocketHandler thread that would be missed by Wireshark.

  Additionally, if we are to use Wireshark, we are at the mercy of whoever it maintaining the protocol in Wireshark, both as to it being accurate and recent.  As can be seen by the **many** previous attempts to include Bitcoin in Wireshark (google "bitcoin dissector") this is easier said than done.

  Lastly, I truly believe that this tool will be used significantly more by being included in the codebase.  It's just that much more discoverable.

ACKs for top commit:
  MarcoFalke:
    re-ACK bff7c66e67 only some minor changes: 👚
  jnewbery:
    utACK bff7c66e67
  theStack:
    re-ACK bff7c66e67

Tree-SHA512: e59e3160422269221f70f98720b47842775781c247c064071d546c24fa7a35a0e5534e8baa4b4591a750d7eb16de6b4ecf54cbee6d193b261f4f104e28c15f47
2021-02-02 13:11:28 +01:00
MarcoFalke
3ddbf22ed1 util: Disallow negative mocktime
Signed-off-by: practicalswift <practicalswift@users.noreply.github.com>
2021-02-02 08:43:19 +00:00
Jon Atack
0dbde700a6
rpc: use GetNetworkNames() in getnetworkinfo and getpeerinfo helps 2021-02-02 00:16:04 +01:00
Jon Atack
b45eae4d53
net: update NET_UNROUTABLE to not_publicly_routable in GetNetworkName() 2021-02-02 00:00:39 +01:00
Fabian Jahr
590bda79e8
scripted-diff: Remove setup_clean_chain if default is not changed
-BEGIN VERIFY SCRIPT-
git grep -l "self.setup_clean_chain = False" test/functional/*.py | xargs sed -i "/self.setup_clean_chain = False/d";
-END VERIFY SCRIPT-
2021-02-01 23:13:38 +01:00
Fabian Jahr
98892f39e3
doc: Improve setup_clean_chain documentation 2021-02-01 23:13:35 +01:00
MarcoFalke
4c55f92c76
Merge #20954: test: Declare nodes type in test_framework.py.
5353b0c64d Change type definitions for "chain" and "setup_clean_chain" from type comments to Python 3.6+ types. Additionally, set type for "nodes". (Kiminuo)

Pull request description:

  ### Motivation

  When I wanted to understand better https://github.com/bitcoin/bitcoin/pull/19145/files#diff-4bebbd3b112dc222ea7e75ef051838ceffcee63b9e9234a98a4cc7251d34451b test, I noticed that navigation in PyCharm/VS Code did not work for `nodes` variable. I think this is frustrating, especially for newcomers.

  ### Summary

  * This PR modifies Python 3.5 [type comments](https://mypy.readthedocs.io/en/stable/cheat_sheet_py3.html#variables) to Python 3.6+ types and adds a proper type for `nodes` [instance attribute](https://mypy.readthedocs.io/en/stable/class_basics.html#instance-and-class-attributes).
  * This PR does not change behavior.
  * This PR is intentionally very small, if the concept is accepted, a follow-up PRs can be more ambitious.

  ### End result

  1. Open `test/functional/feature_abortnode.py`
  2. Move your caret to: `self.nodes[0].generate[caret here](3)`
  3. Use "Go to definition" [F12] should work now.

  I have tested this on PyCharm (Windows, Ubuntu) and VS Code (Windows, Ubuntu).

  Note: Some `TestNode` methods (e.g. `self.nodes[0].getblock(...)` ) use `__call__` mechanism and navigation does not work for them even with this PR.

ACKs for top commit:
  laanwj:
    ACK 5353b0c64d
  theStack:
    ACK 5353b0c64d

Tree-SHA512: 821773f052ab9b2889dc357d38c59407a4af09e3b86d7134fcca7d78e5edf3a5ede9bfb37595ea97caf9ebfcbda372bcf73763b7f89b0677670f21b3e396a12b
2021-01-31 09:28:21 +01:00
Fabian Jahr
e987ae5a55
test: Add test for deterministic UTXO set hash results 2021-01-30 20:33:23 +01:00
Fabian Jahr
6ccc8fc067
test: Add test for gettxoutsetinfo RPC with MuHash 2021-01-30 20:33:20 +01:00
Fabian Jahr
0d3b2f643d
rpc: Add hash_type MUHASH to gettxoutsetinfo
Also small style fix in rpc/util.cpp
2021-01-30 17:38:21 +01:00
MarcoFalke
c8b83510f4
Merge #20724: Cleanup of -debug=net log messages
48c8a9b964 net_processing: log txrelay flag from version message (Anthony Towns)
98fab37ca0 net: use peer=N instead of from=N in debug log (Anthony Towns)
12302105bb net_processing: additional debug logging for ignored messages (Anthony Towns)
f7edea3b7c net: make debug logging conditional on -debug=net (Anthony Towns)
a410ae8cb0 net, net_processing: log disconnect reasons with -debug=net (Anthony Towns)

Pull request description:

  A few changes to -debug=net logging:

   * always log when disconnecting a peer
   * only log various connection errors when -debug=net is enabled, since errors from random untrusted peers is completely expected
   * log when ignoring a message due to violating protocol (primarily to make it easier to debug other implementations)
   * use "peer=123" rather than "from 123" to make grepping logs a bit easier
   * log the value of the bip-37 `fRelay` field in version messages both when sending and receiving a version message

ACKs for top commit:
  jnewbery:
    ACK 48c8a9b964
  MarcoFalke:
    re-ACK 48c8a9b964 only change is rebase 🚓
  practicalswift:
    re-ACK 48c8a9b964

Tree-SHA512: 6ac530d883dffc4fd7fe20b1dc5ebb5394374c9b499aa7a253eb4a3a660d8901edd72e5ad21ce4a2bf71df25e8f142087755f9756f3497f564ef453a7e9246c1
2021-01-29 07:44:22 +01:00
Sebastian Falbesoner
ba7e17e073 rpc, test: document {previous,next}blockhash as optional
Affects the following RPCs:
- getblockheader
- getblock

Also adds trivial tests on genesis block (should not contain
"previousblockhash") and best block (should not contain
"nextblockhash").
2021-01-29 01:07:18 +01:00
MarcoFalke
fa362064e3
rpc: Return total fee in mempool
Also, add missing lock annotations
2021-01-28 10:43:22 +01:00
MarcoFalke
fa92912b4b
rpc: Use RPCHelpMan for check-rpc-mappings linter 2021-01-28 08:16:34 +01:00
Anthony Towns
98fab37ca0 net: use peer=N instead of from=N in debug log 2021-01-28 16:04:04 +10:00
Samuel Dobson
9deba2de76
Merge #20226: wallet, rpc: add listdescriptors command
647b81b709 wallet, rpc: add listdescriptors command (Ivan Metlushko)

Pull request description:

  Looking for concept ACKs

  **Rationale**: allow users to inspect the contents of their newly created descriptor wallets.

  Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:
   * add an option to export xprv
   * with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes

  The output is compatible with `importdescriptors` command so it could be easily used for backup/recover purposes.

  **Output example:**
  ```json
  [
    {
      "desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
      "timestamp": 1296688602,
      "active": false,
      "range": [
        0,
        999
      ],
      "next": 0
    }
  ]
  ```

ACKs for top commit:
  jonatack:
    re-ACK 647b81b709 rebased to master, debug builds cleanly, reviewed diff since last review, tested with a descriptor wallet (and with a legacy wallet)
  achow101:
    re-ACK 647b81b

Tree-SHA512: 51a3620bb17c836c52cecb066d4fa9d5ff418af56809046eaee0528c4dc240a4e90fff5711ba96e399c6664e00b9ee8194e33852b1b9e75af18061296e19a8a7
2021-01-28 13:40:18 +13:00
Ivan Metlushko
647b81b709 wallet, rpc: add listdescriptors command 2021-01-27 21:22:13 +01:00
Andrew Chow
49797c3ccf tests: Disable bdb dump test when no bdb 2021-01-27 12:52:46 -05:00
Andrew Chow
1194cf9269 Fix wallet_send.py wallet setup to work with descriptors
Fixes the wallet setup so this test works with descriptor wallets. Also
enabled explicit descriptor and legacy wallet testing in the test
runner.
2021-01-27 12:52:46 -05:00
Andrew Chow
fbaea7bfe4 Require legacy wallet for wallet_upgradewallet.py 2021-01-27 12:52:46 -05:00
Andrew Chow
b1b679e0ab Explicitly mark legacy wallet tests as such
Some tests are intended to test only legacy wallet behavior. With
automatic switching of wallet type, we need to make them explicit
2021-01-27 12:52:46 -05:00
Andrew Chow
09514e1bef Setup wallets for interface_zmq.py 2021-01-27 12:52:46 -05:00
Andrew Chow
4d03ef9a73 Use MiniWallet in rpc_net.py 2021-01-27 12:52:46 -05:00
Andrew Chow
4de23824b0 Setup wallets for interface_bitcoin_cli.py 2021-01-27 12:52:46 -05:00
Andrew Chow
7c71c627d2 Setup wallets with descriptors for feature_notifications 2021-01-27 12:52:46 -05:00
Andrew Chow
1f1bef8dba Have feature_filelock.py test both bdb and sqlite, depending on compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
c77975abc0 Disable upgrades tests that require BDB if BDB is not compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
1f20cac9d4 Disable wallet_descriptor.py bdb format check if BDB is not compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
3641597d7e tests: Don't make any wallets unless wallet is required 2021-01-27 12:52:46 -05:00
Andrew Chow
b9b88f57a9 Skip legacy wallet reliant tests if BDB is not compiled 2021-01-27 12:52:46 -05:00
Andrew Chow
6f36242389 tests: Set descriptors default based on compilation
Determines whether descriptors should be used based on whether the
--descriptor or --legacy-wallet option is set,
and the compiled support. If no option is set and both BDB and SQLite
are available, it defaults to legacy.

This is used to switch descriptor agnostic tests between descriptors and
legacy wallet.
2021-01-27 12:52:46 -05:00
Wladimir J. van der Laan
15a9df0706
Merge #20964: rpc: Add specific error code for "wallet already loaded"
a6739cc868 rpc: Add specific error code for "wallet already loaded" (Wladimir J. van der Laan)

Pull request description:

  Add a separate RPC error code for "wallet already loaded" to avoid having to match on message to detect this.
  Requested by shesek for rust-bitcoinrpc.

  If concept ACKed needs:
  - [ ]  Release note
  - [x]  A functional test (updated the existing test to make it pass, I think this is enough)

ACKs for top commit:
  jonasschnelli:
    Code Review ACK a6739cc868
  promag:
    Code review ACK a6739cc868.

Tree-SHA512: 9091872e6ea148aec733705d6af330f72a02f23b936b892ac28f9023da7430af6332418048adbee6014305b812316391812039e9180f7f3362d11f206c13b7d0
2021-01-27 13:43:31 +01:00
Wladimir J. van der Laan
a6739cc868 rpc: Add specific error code for "wallet already loaded" 2021-01-25 07:55:35 +01:00
Bezdrighin
8f0b64fb51 Better error messages for invalid addresses
This commit addresses #20809.

We add an additional 'error' property in the result of 'validateaddress' in case the address is not valid that gives a short description of why the address in invalid. We also change the error message returned by 'getaddressinfo' in case the address is invalid.
2021-01-24 02:44:53 +01:00
Troy Giorshev
381f77be85 Add Message Capture Test
Add a functional test for CaptureMessage.  This connects and then
disconnects a peer so that the handshake can be used to check if capture
is being done correctly.

Included in a docstring in the test is the following:

From the data file we'll only check the structure.

We won't care about things like:
- Deserializing the payload of the message
    - This is managed by the deserialize methods in
      test_framework.messages
- The order of the messages
    - There's no reason why we can't, say, change the order of the
      messages in the handshake
- Message Type
    - We can add new message types

We're ignoring these because they're simply too brittle to test here.
2021-01-23 16:15:05 -05:00
Troy Giorshev
e4f378a505 Add capture parser
This commit adds contrib/message-capture/message-capture-parser.py, a python
script to be used alongside -capturemessages to parse the captured
messages.

It is complete with arguments and will parse any file given, sorting the
messages in the files when creating the output.  If an output file is
specified with -o or --output, it will dump the messages in json format
to that file, otherwise it will print to stdout.

The small change to the unused msg_generic is to bring it in line with
the other message classes, purely to avoid a bug in the future.
2021-01-23 16:01:39 -05:00
Sebastian Falbesoner
de85af5cce test: store subversion (user agent) as string in msg_version 2021-01-23 15:04:35 +01:00
MarcoFalke
fa61b9d1a6
util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet
Co-Authored-by: Anthony Towns <aj@erisian.com.au>
2021-01-21 19:31:28 +01:00
MarcoFalke
fa06bce4ac
test: Add tests 2021-01-21 19:29:58 +01:00
MarcoFalke
11cbd4bb54
Merge #17556: test: Change feature_config_args.py not to rely on strange regtest=0 behavior
ff44cae279 test: Change feature_config_args.py not to rely on strange regtest=0 behavior (Russell Yanofsky)

Pull request description:

  Update test to simply generate a normal mainnet configuration file instead of using a crazy setup where a regtest=1 config file using an includeconf in the [regtest] section includes another config file that specifies regtest=0, retroactively switching the network to mainnet.

  This setup was fragile and only worked because the triggered InitError happened early enough that none of the ignored [regtest] options mattered (only affecting log output).

  This change was originally made as part of #17493

Top commit has no ACKs.

Tree-SHA512: 3f77305454f04438493dfc2abd78a00434b30869454d1c3f54587b9c1f63239c49c90fb3b4d3a777ad130f2184e0f2dac87cee4cd23c50f1b3496a375943da01
2021-01-21 16:51:19 +01:00
fanquake
3734adba39
Merge #20953: test: dedup zmq test setup code (node restart, topics subscription)
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) (Sebastian Falbesoner)

Pull request description:

  This PR deduplicates common setup code for the ZMQ functional test. The following steps, previously duplicated in each sub-test, are put into a new method `setup_zmq_test(...)`:
  - create subscriber sockets (`zmq.SUB`) for each topic with the specified timeout (default 60s)
  - restart node0 with specified zmq notifications enabled (`-zmqpub...=tcp://127.0.0.1:...`...)
  - if desired, connect node0 with node1 (note done by default)
  - connect all susbcriber sockets to publisher (running on node0)
  - wait a bit (currently 200ms), to _"Relax so that the subscribers are ready before publishing zmq messages"_

  Note that the last point should be repaced by a more robust method, as this test is still flaky, see #20934 (also #20590 and #20538).

ACKs for top commit:
  instagibbs:
    ACK 4efb6c2d3b
  laanwj:
    Code review ACK 4efb6c2d3b

Tree-SHA512: d49626756a9c669f1133f1b73ce273994b58c760ce0d6a4bdaa384f043a74149dc2b9fa66fe990413d9105f9c3b6ea973e099669e8e02f2902a5b84fa995028c
2021-01-21 16:38:06 +08:00
Sebastian Falbesoner
233a886b42 test: check that getblockfilter RPC fails without block filter index
If a node was started without compact block filter index, i.e. parameter
`--blockfilterindex=0`, the `getblockfilter` RPC call should fail.
2021-01-20 01:55:24 +01:00
Wladimir J. van der Laan
bc51b99bd5
Merge #20891: rpc: Remove deprecated bumpfee behavior
ea0a7ec949 Remove deprecated bumpfee behavior (Andrew Chow)

Pull request description:

  Removes the deprecation message, behavior, and test.

  This was marked for removal in 22.0.

ACKs for top commit:
  promag:
    ACK ea0a7ec949, maybe add need release notes tag.

Tree-SHA512: d1626906849f6ee37213c32e5f8c1433ad8fb7beabcd88f5801b1964b322171a2341bdfbd9a3a5ab39b2fd9d9c6a05f73298583423a73cab1275653105c03e8e
2021-01-19 17:33:18 +01:00
Kiminuo
5353b0c64d Change type definitions for "chain" and "setup_clean_chain" from type comments to Python 3.6+ types. Additionally, set type for "nodes". 2021-01-18 09:01:07 +01:00
Sebastian Falbesoner
4efb6c2d3b zmq test: deduplicate test setup code (node restart, topics subscription) 2021-01-17 12:58:46 +01:00
practicalswift
b4511e2e2e log: Prefix log messages with function name if -logsourcelocations is set 2021-01-15 09:57:32 +00:00
MarcoFalke
fa0aa87071
rpc: Return wtxid from testmempoolaccept 2021-01-12 18:43:43 +01:00
MarcoFalke
6af013792f
Merge #19315: [tests] Allow outbound & block-relay-only connections in functional tests.
b4dd2ef800 [test] Test the add_outbound_p2p_connection functionality (Amiti Uttarwar)
602e69e427 [test] P2PBlocksOnly - Test block-relay-only connections. (Amiti Uttarwar)
8bb6beacb1 [test/refactor] P2PBlocksOnly - Extract transaction violation test into helper. (Amiti Uttarwar)
99791e7560 [test/refactor] P2PBlocksOnly - simplify transaction creation using blocktool helper. (Amiti Uttarwar)
3997ab9154 [test] Add test framework support to create outbound connections. (Amiti Uttarwar)
5bc04e8837 [rpc/net] Introduce addconnection to test outbounds & blockrelay (Amiti Uttarwar)

Pull request description:

  The existing functional test framework uses the `addnode` RPC to spin up manual connections between bitcoind nodes. This limits our ability to add integration tests for our networking code, which often executes different code paths for different connection types.

  **This PR enables creating `outbound` & `block-relay-only` P2P connections in the functional tests.** This allows us to increase our p2p test coverage, since we can now verify expectations around these connection types.

  This builds out the [prototype](https://github.com/bitcoin/bitcoin/issues/14210#issuecomment-527421978) proposed by ajtowns in #14210. 🙌🏽

  An overview of this branch:
  - introduces a new test-only RPC function `addconnection` which initiates opening an `outbound` or `block-relay-only` connection. (conceptually similar to `addnode` but for different connection types & restricted to regtest)
  - adds `test_framework` support so a mininode can open an `outbound`/`block-relay-only` connection to a `P2PInterface`/`P2PConnection`.
  - updates `p2p_blocksonly` tests to create a `block-relay-only` connection & verify expectations around transaction relay.
  - introduces `p2p_add_connections` test that checks the behaviors of the newly introduced `add_outbound_p2p_connection` test framework function.

  With these changes, there are many more behaviors that we can add integration tests for. The blocksonly updates is just one example.

  Huge props to ajtowns for conceiving the approach & providing me feedback as I've built out this branch. Also thank you to jnewbery for lots of thoughtful input along the way.

ACKs for top commit:
  troygiorshev:
    reACK b4dd2ef800
  jnewbery:
    utACK b4dd2ef800
  MarcoFalke:
    Approach ACK b4dd2ef800 🍢

Tree-SHA512: d1cba768c19c9c80e6a38b1c340cc86a90701b14772c4a0791c458f9097f6a4574b4a4acc7d13d6790c7b1f1f197e2c3d87996270f177402145f084ef8519a6b
2021-01-11 21:06:57 +01:00
Wladimir J. van der Laan
675af2a515
Merge #20852: net: allow CSubNet of non-IP networks
39b43298d9 test: add test for banning of non-IP addresses (Vasil Dimov)
94d335da7f net: allow CSubNet of non-IP networks (Vasil Dimov)

Pull request description:

  Allow creation of valid `CSubNet` objects of non-IP networks and only
  match the single address they were created from (like /32 for IPv4 or
  /128 for IPv6).

  This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
  and in `BanMan` which assume that creating a subnet from any address
  using the `CSubNet(CNetAddr)` constructor would later match that address
  only. Before this change a non-IP subnet would be invalid and would not
  match any address.

ACKs for top commit:
  jonatack:
    Code review re-ACK 39b43298d9 per `git diff 5e95ce6 39b4329`; only change since last review is improvements to the functional test; verified the test fails on master @ 616eace0 where expected (`assert(self.is_banned(node, tor_addr))` fails and unban unfails)
  laanwj:
    code review ACK 39b43298d9

Tree-SHA512: 3239b26d0f2fa2d1388b4fdbc1d05ce4ac1980be699c6ec46049409baefcb2006b1e72b889871e2210e897f6725c48e873f68457eea7e6e4958ab4f959d20297
2021-01-11 11:27:47 +01:00
MarcoFalke
9c0b76c709
Merge #20876: test: Replace getmempoolentry with testmempoolaccept in MiniWallet
faabc26a61 test: Replace getmempoolentry with testmempoolaccept in MiniWallet (MarcoFalke)

Pull request description:

  This is a refactor to not use the return value of `sendrawtransaction` and `getmempoolentry` with the goal that submitting the tx to the mempool will become optional.

ACKs for top commit:
  mjdietzx:
    ACK faabc26a61

Tree-SHA512: 4260a59e65fed1c807530dad23f1996ba6e881bcce91995f5b498a0be6001f67b3e0251507c8801750fe105410147c68bb2f393ebe678c6a3a4d045e5d72fc19
2021-01-11 08:57:28 +01:00
Vasil Dimov
39b43298d9
test: add test for banning of non-IP addresses
Co-authored-by: Jon Atack <jon@atack.com>
2021-01-10 15:51:25 +01:00
Andrew Chow
ea0a7ec949 Remove deprecated bumpfee behavior 2021-01-08 18:58:58 -05:00
MarcoFalke
faabc26a61
test: Replace getmempoolentry with testmempoolaccept in MiniWallet 2021-01-08 09:07:33 +01:00
MarcoFalke
5082324225
Merge #20688: test: run mempool_compatibility.py even with wallet disabled
a7599c80eb test: run mempool_compatibility.py even with wallet disabled (Michael Dietz)

Pull request description:

  Another functional test rewritten as proposed in https://github.com/bitcoin/bitcoin/issues/20078

ACKs for top commit:
  MarcoFalke:
    review ACK a7599c80eb didn't test

Tree-SHA512: cc7a617e5489ed27bbdbdee85a82fa08525375061f7f4524577a6b8ecb340396adac88419b51f513be22ca53edd0a3bd5d572d9f43ffc2c18550b0ef9069d238
2021-01-08 08:35:01 +01:00
Wladimir J. van der Laan
d7e2401c62
Merge #18077: net: Add NAT-PMP port forwarding support
a191e23b8e doc: Add release notes (Hennadii Stepanov)
ae749d12dd doc: Add libnatpmp stuff (Hennadii Stepanov)
e28f9be87a ci: Add libnatpmp-dev package to some builds (Hennadii Stepanov)
5a0185b6c9 gui: Add NAT-PMP network option (Hennadii Stepanov)
a39f7336a3 net: Add -natpmp command line option (Hennadii Stepanov)
28acffd9d5 net: Add NAT-PMP to port mapping loop (Hennadii Stepanov)
a8d9f275d0 net: Add libnatpmp support (Hennadii Stepanov)
58e8364dcd gui: Apply port mapping changes on dialog exit (Hennadii Stepanov)
cf151cc68c scripted-diff: Rename UPnP stuff (Hennadii Stepanov)
4e91b1e24d net: Add flags for port mapping protocols (Hennadii Stepanov)
8b50d1b5bb net: Keep trying to use UPnP when -upnp=1 (Hennadii Stepanov)
28e2961fd6 refactor: Replace magic number with named constant (Hennadii Stepanov)
02ccf69dd6 refactor: Move port mapping code to its own module (Hennadii Stepanov)

Pull request description:

  Close #11902
  This PR is an alternative to:
  - #12288
  - #15717

  To compile with NAT-PMP support on Ubuntu [`libnatpmp-dev`](https://packages.ubuntu.com/source/bionic/libnatpmp) should be available.

  Log excerpt:
  ```
  2020-02-05T20:12:28Z [mapport] NAT-PMP: public address = 95.164.65.194
  2020-02-05T20:12:28Z [mapport] AddLocal(95.164.65.194:18333,3)
  2020-02-05T20:12:28Z [mapport] NAT-PMP: port mapping successful.
  ```

  See: [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html)

  ---

  Some follow-ups are out of this PR's scope:
  - mention NAT-PMP library in the version message
  - ~integrate NAT-PMP into the GUI~ (already [added](https://github.com/bitcoin/bitcoin/pull/18077#issuecomment-589405068))

ACKs for top commit:
  laanwj:
    Tested and code review ACK a191e23b8e

Tree-SHA512: 10e19267c21bf30f20ff1abfc882d526049f0e790b95e12f109dc2bed7c0aef45de03eaf967f4e667e7509be04f1873a5c508087393d947205f3aab2ad6d7cf1
2021-01-07 19:41:55 +01:00
Michael Dietz
a7599c80eb
test: run mempool_compatibility.py even with wallet disabled 2021-01-07 12:24:24 -06:00
Amiti Uttarwar
b4dd2ef800 [test] Test the add_outbound_p2p_connection functionality
Open max number of full-relay and block-relay-only connections from a
functional test with different sorts of behaviors to ensure it behaves as
expected.
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
602e69e427 [test] P2PBlocksOnly - Test block-relay-only connections.
Ensure we will disconnect if the peer sends us a transaction & we don't
announce transactions to the peer.
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
8bb6beacb1 [test/refactor] P2PBlocksOnly - Extract transaction violation test into helper.
This is in preparation for use in the next commit.
2021-01-07 10:15:56 -08:00
Amiti Uttarwar
99791e7560 [test/refactor] P2PBlocksOnly - simplify transaction creation using blocktool helper. 2021-01-07 10:15:56 -08:00
Amiti Uttarwar
3997ab9154 [test] Add test framework support to create outbound connections.
In the interest of increasing our P2P test coverage, add support to create
full-relay or block-relay-only connections. To support this, a P2P connection
spins up a listening thread & uses a callback to trigger the node initiating
the connection.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2021-01-07 10:15:56 -08:00
Wladimir J. van der Laan
b6a71b80d2
Merge #19055: Add MuHash3072 implementation
9815332d51 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr)
01297fb3ca fuzz: Add MuHash consistency fuzz test (Fabian Jahr)
b111410914 test: Add MuHash3072 fuzz test (Fabian Jahr)
c122527385 bench: Add Muhash benchmarks (Fabian Jahr)
7b1242229d test: Add MuHash3072 unit tests (Fabian Jahr)
adc708c98d crypto: Add MuHash3072 implementation (Fabian Jahr)
0b4d290bf5 crypto: Add Num3072 implementation (Fabian Jahr)
589f958662 build: Check for 128 bit integer support (Fabian Jahr)

Pull request description:

  This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`.

ACKs for top commit:
  laanwj:
    Code review ACK 9815332d51

Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
2021-01-07 17:57:17 +01:00
Hennadii Stepanov
a39f7336a3
net: Add -natpmp command line option 2021-01-07 18:07:09 +02:00
Ikko Ashimine
1112035d32
doc: fix various typos
Co-authored-by: Peter Yordanov <ppyordanov@yahoo.com>
2021-01-04 12:31:31 +08:00
Sawyer Billings
e8640849c7
doc: Use https URLs where possible 2021-01-04 12:23:16 +08:00
MarcoFalke
fa0074e2d8
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-12-31 09:45:41 +01:00
MarcoFalke
e3dd0a56cf
Merge #20755: [rpc] Remove deprecated fields from getpeerinfo
454a4088a8 [doc] Add release notes for removed getpeerinfo fields. (Amiti Uttarwar)
b1a936d4ae [rpc] Remove deprecated "whitelisted" field from getpeerinfo (Amiti Uttarwar)
094c3beaa4 [rpc] Remove deprecated "banscore" field from getpeerinfo (Amiti Uttarwar)
537053336f [rpc] Remove deprecated "addnode" field from getpeerinfo (Amiti Uttarwar)

Pull request description:

  This PR removes support for 3 fields on the `getpeerinfo` RPC that were deprecated in v0.21- `addnode`, `banscore` & `whitelisted`.

ACKs for top commit:
  sipa:
    utACK 454a4088a8
  jnewbery:
    ACK 454a4088a8.

Tree-SHA512: ccc0e90c0763eeb8529cf0c46162dbaca3f7773981b3b52d9925166ea7421aed086795d56b320e16c9340f68862388785f52a9b78314865070917b33180d7cd6
2020-12-28 22:40:39 +01:00
fanquake
0e1b57b4bb
Merge #20763: test: Fix comment typo in BitcoinTestFramework
40fdb2a212 test: Fix Comment Typo in BitcoinTestFramework (Joel Klabo)

Pull request description:

  Missing "override" in comment describing use of set_test_params

ACKs for top commit:
  michaelfolkson:
    ACK 40fdb2a212

Tree-SHA512: bf893a0d5f8dc86a3ec2eaf48cd7c0f0f832f3b3d254b3d99953336db7e294571b1d2c8686030bf8a27cbe67b1a85a54e53ebefb2e57d6d8d6ac864a15dce4e7
2020-12-27 17:51:15 +08:00
Amiti Uttarwar
b1a936d4ae [rpc] Remove deprecated "whitelisted" field from getpeerinfo 2020-12-26 13:30:54 -08:00
Amiti Uttarwar
094c3beaa4 [rpc] Remove deprecated "banscore" field from getpeerinfo 2020-12-26 13:30:08 -08:00
Amiti Uttarwar
537053336f [rpc] Remove deprecated "addnode" field from getpeerinfo 2020-12-26 13:30:08 -08:00
Joel Klabo
40fdb2a212 test: Fix Comment Typo in BitcoinTestFramework
Missing "override" in comment describing use of set_test_params
2020-12-24 10:54:56 -08:00
Andrew Chow
48a0319bab Add a test that selects too large if BnB is used
If BnB is used, the test will fail because the transaction is too large.
2020-12-24 12:39:54 -05:00
MarcoFalke
f656165e9c
Merge #18772: rpc: calculate fees in getblock using BlockUndo data
66d012ad7f test: RPC: getblock fee calculations (Elliott Jin)
bf7d6e31b1 RPC: getblock: tx fee calculation for verbosity 2 via Undo data (Elliott Jin)

Pull request description:

  This change is progress towards #18771 .  It adapts the fee calculation part of #16083 and addresses some feedback.  The additional "verbosity level 3" features are planned for a future PR.

  **Original PR description:**

  > Using block undo data (like in #14802) we can now show fee information for each transaction in a block without the need for additional -txindex and/or a ton of costly lookups. For a start we'll add transaction fee information to getblock verbosity level 2. This comes at a negligible speed penalty (<1%).

ACKs for top commit:
  luke-jr:
    tACK 66d012ad7f
  fjahr:
    tACK 66d012ad7f
  MarcoFalke:
    review ACK 66d012ad7f 🗜

Tree-SHA512: be1fe4b866946a8dc36427f7dc72a20e10860e320a28fa49bc85bd2a93a0d699768179be29fa52e18b2ed8505d3ec272e586753ef2239b4230e0aefd233acaa2
2020-12-24 15:32:10 +01:00
MarcoFalke
cc592a85ea
Merge #20189: test: Switch to BIP341's suggested scheme for outputs without script
812baaa1f8 Switch to BIP341's suggested scheme for outputs without script (Pieter Wuille)

Pull request description:

  BIP341 suggests using Hash<sub>TapTweak</sub>(pubkey) to derive the tweak in case of key-only outputs. The functional test framework currently uses Hash<sub>TapTweak</sub>(pubkey || 0x00...00) instead. Change this.

  There is no technical reason to prefer one over the other, but in case someone looks at it for inspiration, it's better to be consistent with the BIP.

ACKs for top commit:
  laanwj:
    ACK 812baaa1f8
  instagibbs:
    ACK 812baaa1f8

Tree-SHA512: 02576c38776ec786255f49d7edecdb1ed8a9dcf0f547d58c23099588b4c3296edf279b103a6eb80e0f07d3c5ee9743f67d152f5244fd63adc6613b004f6969ed
2020-12-24 07:56:33 +01:00
Fabian Jahr
9815332d51
test: Change MuHash Python implementation to match cpp version again 2020-12-22 01:48:34 +01:00
Wladimir J. van der Laan
cc2a5ef9b2
Merge #20683: test: Fix restart node race
fab46b34f4 test: Fix restart node race (MarcoFalke)

Pull request description:

  It is not allowed to start a node before it has been fully stopped. Otherwise it could lead to intermittent issues due to access issues (e.g. cookie file https://cirrus-ci.com/task/6409665024098304?command=ci#L4793)

  Fix that by waiting for the node to fully stop.

ACKs for top commit:
  laanwj:
    code review ACK fab46b34f4

Tree-SHA512: 7605cac0573a7b04f05ff110d0131e8940d87f7baf6d698505ed16b363d4d15b1e552c5ffd1a187c8fe5639f7e265c3122734c85283275746e46bd789614fd21
2020-12-21 20:20:30 +01:00
MarcoFalke
fada8b019a
test: Add missing assignment in mempool_resurrect.py 2020-12-21 16:02:29 +01:00
MarcoFalke
1077c93a34
Merge #20692: test: run mempool_resurrect.py even with wallet disabled
11a32722f0 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz)

Pull request description:

  Another functional test rewritten as proposed in #20078

  **Request for help:**
  `node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected.

  However, `node.gettransaction(txid)` throws the error:
  ```sh
  Traceback (most recent call last):
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main
      self.run_test()
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test
      assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda>
      assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5)
  ```

  Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly?

ACKs for top commit:
  MarcoFalke:
    ACK 11a32722f0

Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
2020-12-21 15:23:57 +01:00
MarcoFalke
b27104dc52
Merge #20687: wallet: Add missing check for -descriptors wallet tool option
fae32f295c wallet: Add missing check for -descriptors wallet tool option (MarcoFalke)
faf8f61368 test: Add missing check for is_sqlite_compiled (MarcoFalke)
fa7dde1c41 wallet: Pass ArgsManager into ExecuteWalletToolFunc instead of using global (MarcoFalke)

Pull request description:

  Also, fix a test failure when compiled without sqlite

ACKs for top commit:
  ryanofsky:
    Code review ACK fae32f295c. Thanks for implementing the -descriptors check and dealing with the test failure!
  jonatack:
    Code review utACK fae32f295c

Tree-SHA512: 3d7710694085822739a8316e4abc6db270799ca6ff6b0f9e5563ae240da65ae6a9cab7ba2647feae6ba540dac40b55b38ed41c8f6ed0bf02a3d1536284448927
2020-12-18 07:56:15 +01:00
MarcoFalke
9b28bd73a3
Merge #20691: ci, doc: Travis CI features and mentions cleanup
95487b0553 doc: Drop mentions of Travis CI as it is no longer used (Hennadii Stepanov)
09d105ef0f ci: Drop travis_fold feature as Travis CI is no longer used (Hennadii Stepanov)

Pull request description:

  As Travis CI is no longer used, this PR:
  - drops `travis_fold` feature
  - drops mentions of Travis CI in docs

ACKs for top commit:
  MarcoFalke:
    ACK 95487b0553

Tree-SHA512: 2e259bb8b1e37bcefc1251737bb2716f06ddb57c490010b373825c4e70f42ca38efae69a2f63f21f577d7cee3725b94097bdddbd313f8ebf499281cf97c53cef
2020-12-18 07:32:28 +01:00
Michael Dietz
11a32722f0
test: run mempool_resurrect.py even with wallet disabled 2020-12-17 21:37:07 -06:00
Hennadii Stepanov
95487b0553
doc: Drop mentions of Travis CI as it is no longer used 2020-12-18 01:15:53 +02:00
MarcoFalke
fae32f295c
wallet: Add missing check for -descriptors wallet tool option 2020-12-17 20:36:41 +01:00
MarcoFalke
faf8f61368
test: Add missing check for is_sqlite_compiled 2020-12-17 20:33:45 +01:00
MarcoFalke
143bd108ed
Merge #19137: wallettool: Add dump and createfromdump commands
23cac24dd3 tests: Test bitcoin-wallet dump and createfromdump (Andrew Chow)
a88c320041 wallettool: Add createfromdump command (Andrew Chow)
e1e7a90d5f wallettool: Add dump command (Andrew Chow)

Pull request description:

  Adds two commands to the `bitcoin-wallet` tool: `dump` and `createfromdump`. These commands will be useful for a wallet storage migration in the future. It is also generally useful to have a storage agnostic dump like this. These commands are similar to BDB's `db_dump` and `db_load` tools. This can also be useful for manual construction of a wallet file for tests.

  `dump` outputs every key-value pair from the wallet as comma separated hex. Each key-value pair is on its own line with the key and value in hex separated by a comma. This is output to the file specified by the new `-dumpfile` option.

  `createfromdump` takes a file produced by `dump` and creates a new wallet file with exactly the records specified in that file.

  A new option `-dumpfile` is added to the wallet tool. When used with `dump`, the records will be written to the specified file. When used with `createfromdump`, the file is read and the key-value pairs constructed from it. `createfromdump` requires `-dumpfile`.

  A simple round-trip test is added to the `tool_wallet.py`.

  This PR is based on #19334,

ACKs for top commit:
  Sjors:
    re-utACK 23cac24
  MarcoFalke:
    re review ACK 23cac24dd3 only change is rebase and removing useless shared_ptr wrapper 🎼
  ryanofsky:
    Code review ACK 23cac24dd3. Only changes since last review rebase and changing a pointer to a reference

Tree-SHA512: 2d63cf62baca3d16495aa698dc02f7d889c81b41015e9c92c23c275bb4a690fc176d351c3fd7f310bd6b17f5a936cc9be694cbecd702af741b96c0f530e72fa2
2020-12-17 15:18:37 +01:00
MarcoFalke
fab46b34f4
test: Fix restart node race 2020-12-17 15:06:31 +01:00
Wladimir J. van der Laan
5b6f970e3f
Merge #20171: Add functional test test_txid_inv_delay
bc4a230087 Remove redundant p2p lock tacking for tx download functional tests (Antoine Riard)
d3b5eac9a9 Add mutation for functional test test_preferred_inv (Antoine Riard)
06efb3163c Add functional test test_txid_inv_delay (Antoine Riard)
a07910abcd test: Makes wtxidrelay support a generic P2PInterface option (Antoine Riard)

Pull request description:

  This is a simple functional test to increase coverage of #19988, checking that txid announcements from txid-relay peers are delayed by TXID_RELAY_DELAY, assuming we have at least another wtxid-relay peer.

  You can verify new test with the following diff :

  ```
  diff --git a/src/net_processing.cpp b/src/net_processing.cpp
  index f14db379f..2a2805df5 100644
  --- a/src/net_processing.cpp
  +++ b/src/net_processing.cpp
  @@ -773,7 +773,7 @@ void PeerManager::AddTxAnnouncement(const CNode& node, const GenTxid& gtxid, std
       auto delay = std::chrono::microseconds{0};
       const bool preferred = state->fPreferredDownload;
       if (!preferred) delay += NONPREF_PEER_TX_DELAY;
  -    if (!gtxid.IsWtxid() && g_wtxid_relay_peers > 0) delay += TXID_RELAY_DELAY;
  +    //if (!gtxid.IsWtxid() && g_wtxid_relay_peers > 0) delay += TXID_RELAY_DELAY;
       const bool overloaded = !node.HasPermission(PF_RELAY) &&
           m_txrequest.CountInFlight(nodeid) >= MAX_PEER_TX_REQUEST_IN_FLIGHT;
       if (overloaded) delay += OVERLOADED_PEER_TX_DELAY;
  ```

ACKs for top commit:
  laanwj:
    ACK bc4a230087

Tree-SHA512: 150e806bc5289feda94738756ab375c7fdd23c80c12bd417d3112043e26a91a717dc325a01079ebd02a88b90975ead5bd397ec86eb745c7870ebec379a8aa711
2020-12-16 18:45:11 +01:00
Andrew Chow
23cac24dd3 tests: Test bitcoin-wallet dump and createfromdump 2020-12-16 12:33:09 -05:00
MarcoFalke
69f1ee1922
Merge #20365: wallettool: add parameter to create descriptors wallet
173cc9b7be test: walettool create descriptors (Ivan Metlushko)
345e88eecf wallettool: add param to create descriptors wallet (Ivan Metlushko)
6d3af3ab62 wallettool: pass in DatabaseOptions into MakeWallet (Ivan Metlushko)

Pull request description:

  Rationale: expose and promote descriptor wallets in more places; make cli tool more consistent with `createwallet` rpc.

  Add `-descriptors` parameter which is off by default. When specified it will create a new descriptors wallet with sqlite backend, which is consistent with `createwallet` rpc.

  This PR is based on a suggestion from **ryanofsky** https://github.com/bitcoin/bitcoin/pull/19137#discussion_r516779603

  Example:
  ```
  $ ./src/bitcoin-wallet  -wallet=fewty -descriptors create
  Topping up keypool...
  Wallet info
  ===========
  Name: fewty
  Format: sqlite
  Descriptors: yes
  Encrypted: no
  HD (hd seed available): yes
  Keypool Size: 6000
  Transactions: 0
  Address Book: 0
  ```
  ```
  $ ./src/bitcoin-wallet  -wallet=fewty create
  Topping up keypool...
  Wallet info
  ===========
  Name: fewty
  Format: bdb
  Descriptors: no
  Encrypted: no
  HD (hd seed available): yes
  Keypool Size: 2000
  Transactions: 0
  Address Book: 0
  ```

ACKs for top commit:
  achow101:
    ACK 173cc9b7be
  ryanofsky:
    Code review ACK 173cc9b7be. This seems pretty nicely implemented now, with opportunities to clean up more and dedup later
  MarcoFalke:
    Concept ACK 173cc9b7be 🌠

Tree-SHA512: cc32ba336ff709de2707ee15f495b4617908e8700ede8401a58e894f44cda485c544d644023c9a6604d88a62db9d92152383ee2e8abf691688c25cf6e222c622
2020-12-16 17:43:20 +01:00
MarcoFalke
9dbcd37105
Merge #20569: test: Fix intermittent wallet_multiwallet issue with got_loading_error
fab48da908 test: Fix intermittent wallet_multiwallet issue with got_loading_error (MarcoFalke)
fa8e15f7b7 test: pep8 wallet_multiwallet.py (MarcoFalke)

Pull request description:

  Failing the test after 10 iterations without a loading error is problematic because it may take 11 iterations to get a loading error.

  Fix that by running until a loading error occurs, which should happen in almost all runs within the first 10 iterations.

ACKs for top commit:
  ryanofsky:
    Code review ACK fab48da908. This seems like a good workaround. I think more ideally think load and unload RPCs would not have racy status reporting (suggested previously https://github.com/bitcoin/bitcoin/pull/19300#pullrequestreview-435362710 and

Tree-SHA512: 6b80b26d916276efe2a01af93bca7dbf71a3e67db9d3deb15175070719bf7d1325a1410d93e74c0316942e388faa2ba185dc9d3759c82d1c73c3c509b9997f05
2020-12-16 15:47:23 +01:00
MarcoFalke
a023094fc4
Merge #20276: test: run mempool_expiry.py even with wallet disabled
3b064fcb9d test: run mempool_expiry.py even with wallet disabled (Michael Dietz)

Pull request description:

  Run the mempool expiry test even when the wallet was not compiled, as proposed in https://github.com/bitcoin/bitcoin/issues/20078.

ACKs for top commit:
  MarcoFalke:
    ACK 3b064fcb9d

Tree-SHA512: 5860dc021d02bc3752268ec1e859505bec87174953223b34b1af8a8e4ab66d645458fbf9571c0b816a9de891c3ff41314996e580869671fccd6972c093e78154
2020-12-16 14:52:34 +01:00
MarcoFalke
b103fdcb3b
Merge #19763: net: don't try to relay to the address' originator
7fabe0f359 net: don't relay to the address' originator (Vasil Dimov)

Pull request description:

  For each address to be relayed we "randomly" pick 2 nodes to send the
  address to (in `RelayAddress()`). However we do not take into
  consideration that it does not make sense to relay the address back to
  its originator (`CNode::PushAddress()` will do nothing in that case).

  This means that if the originator is among the "randomly" picked nodes,
  then we will relay to one node less than intended.

  Fix this by skipping the originating node when choosing candidates to
  relay to.

ACKs for top commit:
  sdaftuar:
    ACK 7fabe0f359 (this time I looked at the test, and verified the test breaks in expected ways if I break the code).
  jnewbery:
    utACK 7fabe0f359 (only net_processing changes. I haven't reviewed the test changes)
  jonatack:
    re-ACK 7fabe0f359 per `git range-diff b76abae fd897f8 7fabe0f`, change since last review is rebase and more readable Doxygen documentation

Tree-SHA512: c6a9d11c7afc97ab4e8960513f6416648d4a8c0c64b713c145a7482a7b9e54946f81386a3351e3ec0011e5594ba5ccff4d10c6f656bb80680d9f0d0a63366165
2020-12-14 13:37:21 +01:00
MarcoFalke
b18978066d
Merge #20079: p2p: Treat handshake misbehavior like unknown message
faaad1bbac p2p: Ignore version msgs after initial version msg (MarcoFalke)
fad68afcff p2p: Ignore non-version msgs before version msg (MarcoFalke)

Pull request description:

  Handshake misbehaviour doesn't cost us more than any other unknown message, so it seems odd to treat it differently

ACKs for top commit:
  jnewbery:
    utACK faaad1bbac
  practicalswift:
    ACK faaad1bbac: patch looks correct

Tree-SHA512: 9f30c3b5c1f6604fd02cff878f10999956152419a3dd9825f8267cbdeff7d06787418b41c7fde8a00a5e557fe89204546e05d5689042dbf7b07fbb7eb95cddff
2020-12-12 12:32:46 +01:00
Michael Dietz
3b064fcb9d
test: run mempool_expiry.py even with wallet disabled
Test coverage is also extended in this commit to:
Ensure that another transaction in the mempool is not evicted
when other transactions expire. Note: this other transaction does
not have any ancestors in the mempool that expired. Otherwise it
would be evicted from the mempool, and we already test this case.
2020-12-10 10:14:35 -06:00
Vasil Dimov
7fabe0f359
net: don't relay to the address' originator
For each address to be relayed we "randomly" pick 2 nodes to send the
address to (in `RelayAddress()`). However we do not take into
consideration that it does not make sense to relay the address back to
its originator (`CNode::PushAddress()` will do nothing in that case).

This means that if the originator is among the "randomly" picked nodes,
then we will relay to one node less than intended.

Fix this by skipping the originating node when choosing candidates to
relay to.
2020-12-10 14:41:41 +01:00
MarcoFalke
fa918dd537
test: Use Popen.wait instead of RPC in assert_start_raises_init_error 2020-12-10 12:36:41 +01:00
Wladimir J. van der Laan
eb53c03b36
Merge #20595: Improve heuristic hex transaction decoding
0f949cde3d Add regression test for incorrect decoding (Pieter Wuille)
39c42c4420 Improve heuristic hex transaction decoding (Pieter Wuille)

Pull request description:

  The current hex tx decoding logic will refuse to decode valid extended-encoded transactions if the result fails the heuristic sanity check, even when the legacy-encoding fails. Fix this.

  Fixes #20579

ACKs for top commit:
  achow101:
    Code review ACK 0f949cde3d
  jonatack:
    Tested ACK 0f949cde3d
  laanwj:
    Code review ACK 0f949cde3d

Tree-SHA512: bd6dc80d824eb9a87026a623be910cac92173f8ce1c8b040c2246348c3cf0c6d64bcc40127b859e5e4da1efe88cf02a6945f7ebb91079799395145cb09d9c7a5
2020-12-10 11:22:10 +01:00
MarcoFalke
38176dc665
Merge #20573: wallet, bugfix: allow send with string fee_rate amounts
6fa72ceb80 test: add coverage for passing fee rate as a string (Jon Atack)
ce207d6b93 wallet, bugfix: allow send to take string fee rate values (Jon Atack)

Pull request description:

  RPC send currently only accepts fee rates as numbers, which is a user-facing bug. It should accept fee rates as an amount, e.g. a string or a number, as documented in its help and like sendtoaddress, sendmany, fundrawtransaction, walletcreatefundedpsbt, and bumpfee. Provide a fix and regression test coverage.

ACKs for top commit:
  MarcoFalke:
    review ACK 6fa72ceb80
  achow101:
    Code review ACK 6fa72ceb80
  promag:
    Code review ACK 6fa72ceb80.

Tree-SHA512: 735f9269cb1b81953764b5283449c0b154bd62de034225be5bcedc515c84faf767fe8fe0741008679fe412922c847b00d116cb11aab775236b779c847ba87167
2020-12-10 08:46:01 +01:00
MarcoFalke
22f13c1e08
Merge #19776: net, rpc: expose high bandwidth mode state via getpeerinfo
343dc4760f test: add test for high-bandwidth mode states in getpeerinfo (Sebastian Falbesoner)
dab6583307 doc: release note for new getpeerinfo fields "bip152_hb_{from,to}" (Sebastian Falbesoner)
a7ed00f8bb rpc: expose high-bandwidth mode states via getpeerinfo (Sebastian Falbesoner)
30bc8fab68 net: save high-bandwidth mode states in CNodeStats (Sebastian Falbesoner)

Pull request description:

  Fixes #19676, "_For every peer expose through getpeerinfo RPC whether or not we selected them as HB peers, and whether or not they selected us as HB peers._" See [BIP152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki), in particular the [protocol flow diagram](https://github.com/bitcoin/bips/raw/master/bip-0152/protocol-flow.png).  The newly introduced states are changed on the following places in the code:
  * on reception of a `SENDCMPCT` message with valid version, the field `m_highbandwidth_from` is changed depending on the first integer parameter in the message (1=high bandwidth, 0=low bandwidth), i.e. it just mirrors the field `CNodeState.fPreferHeaderAndIDs`.
  * after adding a `SENDCMPCT` message to the send queue, the field `m_highbandwidth_to` is changed depending on how the first integer parameter is set (same as above)

  Note that after receiving `VERACK`, the node also sends `SENDCMPCT`, but that is only to announce the preferred version and never selects high-bandwidth mode, hence there is no need to change the state variables there, which are initialized to `false` anyways.

ACKs for top commit:
  naumenkogs:
    reACK 343dc4760f
  jonatack:
    re-ACK 343dc4760f per `git range-diff 7ea6499 4df1d12 343dc47`

Tree-SHA512: f4999e6a935266812c2259a9b5dc459710037d3c9e938006d282557cc225e56128f72965faffb207fc60c6531fab1206db976dd8729a69e8ca29d4835317b99f
2020-12-10 08:21:36 +01:00
Wladimir J. van der Laan
42ed7f51fa
Merge #20606: Remove unused bits from service flags enum
fa40168ab3 Remove unused bits from service flags enum (MarcoFalke)

Pull request description:

  Remove service bits that haven't been observed on the active network for years and won't ever be observed on the network with this meaning. Keeping this dead assignment in our source code forever doesn't add any value.

  I somehow forgot to do this in commit fa0d0ff6e1.

ACKs for top commit:
  laanwj:
    Code review ACK fa40168ab3
  practicalswift:
    cr ACK fa40168ab3
  fanquake:
    ACK fa40168ab3

Tree-SHA512: 376e5ac05940493cf2209fea60515c843e978c4b476f2524f6bf7a37a646d237c3ddcf6c0fa23641f9ba550f625609703d9b51b4be631a7f2a90e1092b557232
2020-12-09 16:37:56 +01:00
MarcoFalke
90ef622ab5
Merge #20564: Don't send 'sendaddrv2' to pre-70016 software, and send before 'verack'
1583498fb6 Send and require SENDADDRV2 before VERACK (Pieter Wuille)
c5a8919660 Don't send 'sendaddrv2' to pre-70016 software (Pieter Wuille)

Pull request description:

  BIP155 defines addrv2 and sendaddrv2 for all protocol versions, but some implementations reject messages they don't know. As a courtesy, don't send it to nodes with a version before 70016, as no software is known to support BIP155 that doesn't announce at least that protocol version number.

  Also move the sending of sendaddrv2 earlier (before sending verack), as proposed in https://github.com/bitcoin/bips/pull/1043. This has the side effect that local address broadcast of torv3 will work (as it'll only trigger after we know whether or not the peer supports addrv2).

ACKs for top commit:
  MarcoFalke:
    ACK 1583498fb6
  jnewbery:
    ACK 1583498fb6
  jonatack:
    ACK 1583498fb6
  vasild:
    ACK 1583498

Tree-SHA512: 3bd5833fa8c8567b6dedd99e4a9b6bb71c127aa66d5284b217503c86d597dc59aa7382c41f3a4bf561bb658b89db81d1a7703a700eef4ffc17cb916660e23a82
2020-12-09 07:01:57 +01:00
Pieter Wuille
0f949cde3d Add regression test for incorrect decoding 2020-12-08 13:13:40 -08:00
Pieter Wuille
1583498fb6 Send and require SENDADDRV2 before VERACK
See the corresponding BIP change: https://github.com/bitcoin/bips/pull/1043
2020-12-08 09:40:10 -08:00
MarcoFalke
fa40168ab3
Remove unused bits from service flags enum 2020-12-08 18:36:51 +01:00
Wladimir J. van der Laan
d38feb6134
Merge #20535: test: Fix intermittent feature_taproot issue
fa275e1539 test: Fix intermittent feature_taproot issue (MarcoFalke)

Pull request description:

  The nodes might disconnect (e.g. due to "Timeout downloading block" https://cirrus-ci.com/task/5313800947630080?command=ci#L1763) and the test fails to continue.

  Fix that by reconnecting the nodes.

ACKs for top commit:
  laanwj:
    code review ACK fa275e1539

Tree-SHA512: 2871183c8058d8292c9c4ef56ea3d19d5616ca712ebdaabb6609f8c9cd2e16c9ac2ce26aa1e94b346872b7b6fec56b59af151af83de3a5aa08bed01bfcc7187a
2020-12-07 20:37:18 +01:00
Wladimir J. van der Laan
5c4911e7e7
Merge #20568: doc: Use FeeModes doc helper in estimatesmartfee
fa8abdc995 rpc: Use FeeModes doc helper in estimatesmartfee (MarcoFalke)

Pull request description:

  Not sure why this doesn't use the doc helper, probably an oversight?

ACKs for top commit:
  laanwj:
    Code review ACK fa8abdc995

Tree-SHA512: 1f2dc8356e3476ddcf9cafafa7f9865ad95bed1e3067c0edab8e3c483e374bdbdbecc066167554b4a1b479e28f6a52c4ae6a75a70c67ee4e1ff4f3ba36b04001
2020-12-07 14:09:53 +01:00
MarcoFalke
03b1db6114
Merge #18766: Disable fee estimation in blocksonly mode (by removing the fee estimates global)
4e28753f60 feestimator: encapsulate estimation file logic (Antoine Poinsot)
e8ea6ad9c1 init: don't create a CBlockPolicyEstimator if we don't relay transactions (Antoine Poinsot)
86ff2cf202 Remove the remaining fee estimation globals (Antoine Poinsot)
03bfeee957 interface: remove unused estimateSmartFee method from node (Antoine Poinsot)

Pull request description:

  If the `blocksonly` mode is turned on after running with transaction
  relay enabled for a while, the fee estimation will serve outdated data
  to both the internal wallet and to external applications that might be
  feerate-sensitive and make use of `estimatesmartfee` (for example a
  Lightning Network node).

  This has already caused issues (for example https://github.com/bitcoin/bitcoin/issues/16840 (C-lightning), or https://github.com/lightningnetwork/lnd/issues/2562 (LND)) and it seems prudent to fail rather than to give inaccurate values.

  This fixes #16840, and closes #16890 which tried to fix the symptoms (RPC) but not the cause as mentioned by sdaftuar :
  > If this is a substantial problem, then I would think we should take action to protect our own wallet users as well (rather than hide the results of what our fee estimation would do!).

ACKs for top commit:
  MarcoFalke:
    re-ACK 4e28753f60 👋
  jnewbery:
    utACK 4e28753f60

Tree-SHA512: c869cf03b86d8194002970bbc84662dae76874967949b9be0d9a4511a1eabcb1627c38aca3154da9dcece1a4c49ec02bd4f9fcca2ec310986e07904559e63ba8
2020-12-07 12:59:48 +01:00
MarcoFalke
fa8abdc995
rpc: Use FeeModes doc helper in estimatesmartfee
Can be reviewed with --ignore-all-space
2020-12-07 09:28:47 +01:00
Jonas Schnelli
eab63b971d
Merge #19847: rpc, refactor: Avoid duplicate set lookup in gettxoutproof
52fc39917f rpc: Reject empty txids in gettxoutproof (João Barbosa)
73dc19a330 rpc, refactor: Avoid duplicate set lookup in gettxoutproof (João Barbosa)

Pull request description:

ACKs for top commit:
  jonasschnelli:
    code review ACK 52fc39917f

Tree-SHA512: 76b18e5235e8b2d394685515a4a60335666eeb0f6b31c1d397f7db2fbe681bc817b8cd3e8f6708b9dacd6113e4e1d94837072cae27834b8a1a22d2717db8191e
2020-12-07 09:17:08 +01:00
MarcoFalke
64156ad4d1
Merge #19893: test: Remove or explain syncwithvalidationinterfacequeue
fa6af31227 test: Document why syncwithvalidationinterfacequeue is needed in tests (MarcoFalke)
fa135a13b8 Revert "test: Add missing sync_all to wallet_balance test" (MarcoFalke)

Pull request description:

  syncwithvalidationinterfacequeue is a hidden test-only RPC, so it should not be used when it is not needed. Thus, either remove it or explain why it is needed.

ACKs for top commit:
  fjahr:
    Code review ACK fa6af31227

Tree-SHA512: de30db4ab521184091ee5beeab02989138cf7cf05088f766a2fb106151b239310b63d5380cb79e2a072f72c5ae9513aecae8eb9c1c7be713771585c3cb04d63a
2020-12-06 19:13:10 +01:00
Jon Atack
6fa72ceb80
test: add coverage for passing fee rate as a string 2020-12-04 22:35:31 +01:00
Jon Atack
ce207d6b93
wallet, bugfix: allow send to take string fee rate values 2020-12-04 22:12:36 +01:00
João Barbosa
52fc39917f rpc: Reject empty txids in gettxoutproof 2020-12-04 18:38:23 +00:00
MarcoFalke
fab48da908
test: Fix intermittent wallet_multiwallet issue with got_loading_error 2020-12-04 10:32:00 +01:00
MarcoFalke
fa8e15f7b7
test: pep8 wallet_multiwallet.py 2020-12-04 10:27:38 +01:00
Andrew Chow
773c42b265 tests: Test that a fully signed tx given to signrawtx is unchanged
Tests that a fully signed transaction given to
signrawtransactionwithwallet is both unchanged and marked as complete.
This tests for a regression in 0.20 where the transaction would not be
marked as complete.
2020-12-03 15:53:58 -05:00
Antoine Poinsot
e8ea6ad9c1
init: don't create a CBlockPolicyEstimator if we don't relay transactions
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-12-03 12:56:37 +01:00
MarcoFalke
681ce59d0e
Merge #20466: test: Fix intermittent p2p_fingerprint issue
fad7be584f test: Fix intermittent p2p_finerprint issue (MarcoFalke)

Pull request description:

  A single sync_with_ping can't be used to drop a block announcement, as the block might be sent *after* the ping has been responded to.

  Fix that by waiting for the block.

ACKs for top commit:
  theStack:
    ACK fad7be584f

Tree-SHA512: d43ba9d07273486858f65a26326cc6637ef743bf7b400e5048ba7eac266fb1893283e6503dd49f179caa1abab2977315fb70ba9fba34be9a817a74259d8e4034
2020-12-03 10:09:30 +01:00
MarcoFalke
283f22cabb
Merge #20461: rpc: Validate -rpcauth arguments
053b4fbad8 doc: Release note regarding -rpcauth validation (João Barbosa)
46001323b1 rpc: Validate -rpcauth arguments (João Barbosa)
d37c813a43 rpc: Refactor to process -rpcauth once (João Barbosa)

Pull request description:

  Invalid `-rpcauth` arguments are currently silently ignored. This make server initialization fail if any `-rpcauth` is invalid.

ACKs for top commit:
  MarcoFalke:
    review ACK 053b4fbad8
  jonatack:
    ACK 053b4fbad8
  ryanofsky:
    Code review ACK 053b4fbad8. Only changes since last review are moving a variable declaration and adding a comment, release notes, and a `const`.

Tree-SHA512: c99923d4a121f0c9f882b07f5402ea53e9b2d9455ad34468a094ffab1d64df26c82e1279734c0d42bc2e113eae7b581fbc3be52f3ed4a2d7450d11793afcf406
2020-12-02 09:37:37 +01:00
MarcoFalke
607c844f37
Merge #20540: test: Fix wallet_multiwallet issue on windows
fada2dfcac test: Fix wallet_multiwallet issue on windows (MarcoFalke)

Pull request description:

  The error message on windows:

  > 2020-11-30T18:10:47.536032Z ListWalletDir: Error scanning C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20201130_181042\wallet_multiwallet_0\node0\regtest\wallets\self_walletdat_symlink: boost::filesystem::status: The name of the file cannot be resolved by the system: "C:\Users\user\AppData\Local\Temp\test_runner_₿_🏃_20201130_181042\wallet_multiwallet_0\node0\regtest\wallets\self_walletdat_symlink\wallet.dat"

ACKs for top commit:
  promag:
    Code review ACK fada2dfcac. Although it could ignore (don't log) directories that lead to no permission error.
  fanquake:
    ACK fada2dfcac

Tree-SHA512: b475162cc3cd1574209d916605b229a79c8089714295f5e16569b71f958f0007d54dc76833938492d931387784588b11b73e3ef00f963540af42c079417f8d72
2020-12-02 08:31:02 +01:00
MarcoFalke
f17e8ba3a1
Merge #20207: Follow-up extra comments on taproot code and tests
2d8099c713 Mention units of MAX_STANDARD_ policy constants (Pieter Wuille)
84e29c7c01 Mention in validation that IsWitnessStandard tests for P2TR (Pieter Wuille)
f867cbcc26 Clean up assets test minimizer LDFLAGS (Pieter Wuille)
ea0e78677b Document additional IsWitnessStandard behavior (Pieter Wuille)
6040de9a46 Add comments on CPubKey::IsValid (Pieter Wuille)
8dbb7de67c Add comments to VerifyTaprootCommitment (Pieter Wuille)
cdf900cbf2 Document need_vin_vout_mismatch argument to make_spender (Pieter Wuille)
18246ed5f0 Fix and improve taproot_construct comments (Pieter Wuille)

Pull request description:

  Addressing some review comments raised here: https://github.com/bitcoin/bitcoin/pull/19953#pullrequestreview-512238027 and https://github.com/bitcoin/bitcoin/pull/19953#pullrequestreview-513499921

ACKs for top commit:
  jonatack:
    ACK 2d8099c per `git range-diff 5009159 4f10965 2d8099c`
  ariard:
    ACK 2d8099c, only changes are comment light improvements on IsValid/IsWitnessStandard.

Tree-SHA512: c4881546c379ea8efc7ef99a43cbf3b9cd3f9dde5fd97a07ee66f2b593c78aef0bd8784853c5c9c737b66c269241a1048bbbdd6c964a3d872efd8ba0ec410b68
2020-12-01 15:11:51 +01:00
MarcoFalke
fada2dfcac
test: Fix wallet_multiwallet issue on windows 2020-12-01 13:38:23 +01:00
MarcoFalke
fa275e1539
test: Fix intermittent feature_taproot issue 2020-11-30 20:45:09 +01:00
MarcoFalke
7ae86b3c68
Merge #20522: [test] Fix sync issue in disconnect_p2ps
3ebde2143a [test] Fix wait condition in disconnect_p2ps (Amiti Uttarwar)

Pull request description:

  #19315 currently has a [test failure](https://cirrus-ci.com/task/4545582645641216) because of a race. `disconnect_p2ps` is intended to have a `wait_until` clause that prevents this race, but the conditional doesn't match since its comparing two different object types. `MY_SUBVERSION` is defined in messages.py as a byte string, but is compared to the value returned by the RPC. This PR simply converts types to ensure they match, which should prevent the race from occurring.

  HUGE PROPS TO jnewbery for discovering the issue 🔎

ACKs for top commit:
  jnewbery:
    ACK 3ebde2143a
  glozow:
    Code review ACK 3ebde2143a

Tree-SHA512: ca096b80a3e4d757a645f38846d6dc89d6a3d35c3435513a72d278e305faddd4aff9e75a767941b51b2abbf59c82679bac1e9a0140d6f285efe3053e51bcc2a8
2020-11-28 18:06:43 +01:00
MarcoFalke
1ae5758981
Merge #20448: RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint wallet
89bdad5b25 RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint (Luke Dashjr)

Pull request description:

  Allow specifying the `wallet_name` param to `unloadwallet` on RPC wallet endpoints, so long as it matches the endpoint wallet.

ACKs for top commit:
  jonatack:
    ACK 89bdad5b25
  MarcoFalke:
    review ACK 89bdad5b25

Tree-SHA512: efb399c33f7b5596870a26a8680f453ca47aa7a6db4e550f9435d13044f1c4bad0ae11e8f0205213409d08b75c4188c3be782e54aafab1f65b97eb8cf5c252a9
2020-11-28 10:14:45 +01:00
Amiti Uttarwar
3ebde2143a [test] Fix wait condition in disconnect_p2ps
MY_SUBVERSION is defined in messages.py as a byte string, but here we were
comparing this value to the value returned by the RPC. Convert to ensure the
types match.
2020-11-27 14:54:55 -08:00
Pieter Wuille
cdf900cbf2 Document need_vin_vout_mismatch argument to make_spender 2020-11-26 14:56:25 -08:00
Pieter Wuille
18246ed5f0 Fix and improve taproot_construct comments 2020-11-26 14:56:25 -08:00
MarcoFalke
afdfd3c8c1
Merge #20403: wallet: upgradewallet fixes, improvements, test coverage
3eb6f8b2e6 wallet (not for backport): improve upgradewallet error messages (Jon Atack)
ca8cd893bb wallet: fix and improve upgradewallet error responses (Jon Atack)
99d56e3571 wallet: fix and improve upgradewallet result responses (Jon Atack)
2498b04ce8 Don't upgrade to HD split if it is already supported (Andrew Chow)
c46c18b788 wallet: refactor GetClosestWalletFeature() (Jon Atack)

Pull request description:

  This follows up on #18836 and #20282 to fix and improve the as-yet unreleased `upgradewallet` feature and also implement review follow-up in https://github.com/bitcoin/bitcoin/pull/18836#discussion_r519328607.

  This PR fixes 4 upgradewallet issues:

  - this bug: https://github.com/bitcoin/bitcoin/pull/20403#discussion_r526063920
  - it returns nothing in the absence of an RPC error, which isn't reassuring for users
  - it returns the same thing both in the case of a successful upgrade and when no upgrade took place
  - the error message object is currently dead code

  This PR fixes the above and provides:

  ...user feedback to not silently return without upgrading
  ```
  {
    "wallet_name": "disable private keys",
    "previous_version": 169900,
    "current_version": 169900,
    "result": "Already at latest version. Wallet version unchanged."
  }
  ```
  ...better feedback after successfully upgrading
  ```
  {
    "wallet_name": "watch-only",
    "previous_version": 159900,
    "current_version": 169900,
    "result": "Wallet upgraded successfully from version 159900 to version 169900."
  }
  ```
  ...helpful error responses
  ```
  {
    "wallet_name": "blank",
    "previous_version": 169900,
    "current_version": 169900,
    "error": "Cannot downgrade wallet from version 169900 to version 159900. Wallet version unchanged."
  }
  {
    "wallet_name": "blank",
    "previous_version": 130000,
    "current_version": 130000,
    "error": "Cannot upgrade a non HD split wallet from version 130000 to version 169899 without upgrading to support pre-split keypool. Please use version 169900 or no version specified."
  }
  ```
  updated help:
  ```
  upgradewallet ( version )

  Upgrade the wallet. Upgrades to the latest version if no version number is specified.
  New keys may be generated and a new wallet backup will need to be made.
  Arguments:
  1. version    (numeric, optional, default=169900) The version number to upgrade to. Default is the latest wallet version.

  Result:
  {                            (json object)
    "wallet_name" : "str",     (string) Name of wallet this operation was performed on
    "previous_version" : n,    (numeric) Version of wallet before this operation
    "current_version" : n,     (numeric) Version of wallet after this operation
    "result" : "str",          (string, optional) Description of result, if no error
    "error" : "str"            (string, optional) Error message (if there is one)
  }
  ```

ACKs for top commit:
  achow101:
    ACK  3eb6f8b
  MarcoFalke:
    review ACK 3eb6f8b2e6 🛡

Tree-SHA512: b767314069e26b5933b123acfea6aa40708507f504bdb22884da020a4ca1332af38a7072b061e36281533af9f4e236d94d3c129daf6fe5b55241127537038eed
2020-11-25 12:46:27 +01:00
MarcoFalke
efd4cdb81f
Merge #20456: test: Fix intermittent issue in mempool_compatibility
fa05d19bd6 test: Fix intermittent issue in mempool_compatibility (MarcoFalke)

Pull request description:

  Fixes https://cirrus-ci.com/task/5141306890518528?command=ci#L6076

  The version is too old to understand getmempoolinfo()[loaded], so it is never called when starting (See https://cirrus-ci.com/task/5141306890518528?command=ci#L5541)

ACKs for top commit:
  achow101:
    ACK fa05d19bd6

Tree-SHA512: e912d5dff6236d2d4ac608f9f3b3e255cc2b611f36d79bfe4e2a940709833a947c682d7703327887e1753eab30b95eb2615d7e2c21ce4bca4f089e717cbb51c4
2020-11-25 08:19:59 +01:00
MarcoFalke
ca4a784942
Merge #20410: wallet: Do not treat default constructed types as None-type
fa69c2c784 wallet: Do not treat default constructed types as None-type (MarcoFalke)
fac4e136fa refactor: Change pointer to reference because it can not be null (MarcoFalke)

Pull request description:

  Equating `0==None` and `""==None` is confusing, unneeded and undocumented

ACKs for top commit:
  jonatack:
    ACK fa69c2c784
  achow101:
    ACK fa69c2c784
  Sjors:
    tACK fa69c2c784 modulo `unset`

Tree-SHA512: c4c8d0ad80c6697621d356a9545caf28ca2facc82bb2fa8e70eceb52372d25f0685237c73688c4b01da0e75d213c77c0d45011a8bdfe81ea783d85f045786dac
2020-11-25 08:02:19 +01:00
MarcoFalke
3a32b62fa7
Merge #20462: RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC request and wallet_name parameter specify a wallet
b1f59d55d9 RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC endpoint and wallet_name parameter specify a wallet (Luke Dashjr)

Pull request description:

  Just documentation clarifications from #20448

ACKs for top commit:
  MarcoFalke:
    review ACK b1f59d55d9
  jonatack:
    re-ACK b1f59d55d9  per `git diff e8303a0 b1f59d5`

Tree-SHA512: ac068b0aa7ceed49496367fdd9425b59dbba18b56e89b26afc22a6c8ece51f0b92a169cacd55740b1cadab2b32f4f8e8700e609066ab7e59d3b53c7891da585e
2020-11-24 12:07:09 +01:00
Luke Dashjr
89bdad5b25 RPC/Wallet: unloadwallet: Allow specifying wallet_name param matching RPC endpoint 2020-11-24 05:33:18 +00:00
Luke Dashjr
b1f59d55d9 RPC/Wallet: unloadwallet: Clarify docs/error when both the RPC endpoint and wallet_name parameter specify a wallet 2020-11-24 05:31:58 +00:00
João Barbosa
46001323b1 rpc: Validate -rpcauth arguments 2020-11-23 21:02:54 +00:00
MarcoFalke
fad7be584f
test: Fix intermittent p2p_finerprint issue 2020-11-23 20:58:50 +01:00
Sjors Provoost
b87caf10b5
test: add is_bdb_compiled helper 2020-11-23 11:30:55 +01:00
MarcoFalke
fa05d19bd6
test: Fix intermittent issue in mempool_compatibility 2020-11-23 09:27:36 +01:00
MarcoFalke
816132e6eb
Merge #20426: wallet: allow zero-fee fundrawtransaction/walletcreatefundedpsbt and other fixes
9f08780dd7 Use the correct incremental fee constant in bumpfee help (Jon Atack)
3f1e10b2b1 Update feeRate (BTC/kvB) to fee_rate (sat/vB) in wallet_bumpfee (Jon Atack)
1b3d700928 Allow zero-fee fundrawtxn and walletcreatefundedpsbt calls (Jon Atack)

Pull request description:

  - Fixes https://github.com/bitcoin/bitcoin/pull/20305/files#r525406176. A check to raise an error on zero-fee txns was mistakenly extended in a0d4957 from the bumpfee and send{toaddress, many} RPCs to also include fundrawtransaction and walletcreatefundedpsbt. This commit re-overrides zero fee rate checking for these two RPCs, not only for the feeRate (BTC/kvB) arg to return to previous behavior, but also for the new fee_rate (sat/vB) arg. Negative fee rates will still raise "amount out of range" by the MoneyRange check in src/bitcoin-tx.cpp::AmountFromValue.

  - Updates a wallet bumpfee test from feeRate (BTC/kvB) to fee_rate (sat/vB)

  - Fixes https://github.com/bitcoin/bitcoin/pull/20305/files#r525405363 to use the correct incremental fee rate constant in the bumpfee help  (thanks Marco Falke for the catch) and rectifies "1.000 sat/vB sat/vB" in the help to "1.000 sat/vB"

ACKs for top commit:
  MarcoFalke:
    review ACK 9f08780dd7  🐾
  promag:
    Code review ACK 9f08780dd7.
  Xekyo:
    Code review reACK 9f08780dd7.

Tree-SHA512: 413dfb4f23ebaf3d2ef210dd04610a843272e64eabba428699f5de4d646a86ac4911dab66b5e2f5ebea53b76e4be8347ef40824c1592c750d5eaa12579d3cdf6
2020-11-21 07:47:40 +01:00
Jon Atack
3f1e10b2b1
Update feeRate (BTC/kvB) to fee_rate (sat/vB) in wallet_bumpfee
as the feeRate argument should soon be deprecated.

Also loosen one test (and a similar one) that caused a one-off CI failure with:
expected message
'Insufficient total fee 0.00000141, must be at least 0.00001704 (oldFee 0.00000999 + incrementalFee 0.00000705)'
actual message
'Insufficient total fee 0.00000141, must be at least 0.00001712 (oldFee 0.00001007 + incrementalFee 0.00000705)'
2020-11-20 09:40:47 +01:00
Jon Atack
1b3d700928
Allow zero-fee fundrawtxn and walletcreatefundedpsbt calls
A check to raise an error on zero-fee txns was mistakenly extended in commit
a0d4957 from the bumpfee and send{toaddress, many} RPCs to also include
fundrawtransaction and walletcreatefundedpsbt.

This commit overrides zero fee rate checking for these two RPCs, not only for
the feeRate (BTC/kvB) arg to return to previous behavior, but also for the new
fee_rate (sat/vB) arg.
2020-11-20 09:40:44 +01:00
MarcoFalke
3d632c3f49
Merge #20428: tests: shrink feature_taproot transfer of funds tx
7ffac12545 tests: shrink feature_taproot transfer of funds tx (Anthony Towns)

Pull request description:

  When moving funds from node 1 to node 0 for the pre-activation tests, there can be a large number of inputs, potentially resulting in a tx that is larger than standardness rules allow, or that takes a long time to sign. This just takes the top 500 outputs, which is plenty (~90% of the wallet balance).

ACKs for top commit:
  luke-jr:
    utACK 7ffac12545
  MarcoFalke:
    cr ACK 7ffac12545

Tree-SHA512: 68445b4827dddb9a8e8614d61a68f5bbd7152557bf940be0a75741cb49deeff1566198da1a777ac66cb3fed93e64a30bf895875d6cc6ae9e48275e3febb620a6
2020-11-20 09:30:12 +01:00
Anthony Towns
7ffac12545 tests: shrink feature_taproot transfer of funds tx 2020-11-20 07:29:42 +10:00
Jon Atack
3eb6f8b2e6
wallet (not for backport): improve upgradewallet error messages 2020-11-19 20:00:56 +01:00
Jon Atack
ca8cd893bb
wallet: fix and improve upgradewallet error responses 2020-11-19 20:00:53 +01:00
Jon Atack
99d56e3571
wallet: fix and improve upgradewallet result responses 2020-11-19 20:00:50 +01:00
Wladimir J. van der Laan
04670ef81e
Merge #20385: test: run mempool_spend_coinbase.py even with wallet disabled
21f2433601 test: run mempool_spend_coinbase.py even with wallet disabled (Michael Dietz)

Pull request description:

  Run the mempool spend coinbase test even when the wallet was not compiled, as proposed in #20078.

ACKs for top commit:
  laanwj:
    ACK 21f2433601

Tree-SHA512: 301582c04376371cfa8f1ebb2418a4341b42ddcd9ad4f48b58bcf888d867a97bdc409972856b67a8339ac5e60124aefee82a049b4f7fc6bca7a18d7e92e090be
2020-11-19 16:40:02 +01:00
Wladimir J. van der Laan
c4d1e24f54
Merge #20047: test: use wait_for_{block,header} helpers in p2p_fingerprint.py
6b56c1f4d0 test: remove last_{block,header}_equals() in p2p_fingerprint.py (Sebastian Falbesoner)
136d96b71f test: use wait_for_{block,header} helpers in p2p_fingerprint.py (Sebastian Falbesoner)

Pull request description:

  This small PR takes use of the message receiving helper functions `wait_for_block()` and `wait_for_header()` (from module `test_framework.p2p`) in the test `p2p_fingerprint.py`.  It also simplifies the checks for very old stale blocks/headers requests by getting rid of the functions `last_block_equals()` and `last_header_equals()` and rather only testing that not any blocks/headers message is received at all. Unneeded sending of requests are also removed and calls to time.sleep(...) substituted by ping syncs.

ACKs for top commit:
  guggero:
    ACK 6b56c1f4

Tree-SHA512: 9114db70f3804adad4ab658236762d4fa73fef91158c5756dd1af2d24196ea740451b0028667e0c4047f1f89fe1355031921d3dfb973acc1370052a4bc12c2ab
2020-11-19 15:38:17 +01:00
MarcoFalke
fa69c2c784
wallet: Do not treat default constructed types as None-type 2020-11-19 13:48:38 +01:00
MarcoFalke
faaad1bbac
p2p: Ignore version msgs after initial version msg
Sending a version message after the intial version message is peer
misbehavior. Though, it seems arbitrary and confusing to disconnect only
after exactly 100 version messages. Duplicate version messages affect us
no different than any other unknown message. So remove the Misbehaving
and ignore the redundant msgs.
2020-11-19 08:07:16 +01:00
MarcoFalke
fad68afcff
p2p: Ignore non-version msgs before version msg
Sending a non-version message before the initial version message is peer
misbehavior. Though, it seems arbitrary and confusing to disconnect only
after exactly 100 non-version messages. So remove the Misbehaving and
instead rely on the existing disconnect-due-to-handshake-timeout logic.
2020-11-19 08:06:30 +01:00
Andrew Chow
2498b04ce8
Don't upgrade to HD split if it is already supported
It is unnecessary to upgrade to FEATURE_HD_SPLIT if this feature is
already supported by the wallet. Because upgrading to FEATURE_HD_SPLIT
actually requires upgrading to FEATURE_PRE_SPLIT_KEYPOOL, users would
accidentally be upgraded to FEATURE_PRE_SPLIT_KEYPOOL instead of nothing
being done.

Fixes the issue described at
https://github.com/bitcoin/bitcoin/pull/20403#discussion_r526063920
2020-11-19 08:04:05 +01:00
MarcoFalke
4b24c3962f
Merge #19504: Bump minimum python version to 3.6
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them (Anthony Towns)
8ae9d314e9 Bump minimum python version to 3.6 (Anthony Towns)

Pull request description:

  Python 3.5 has reached [end-of-life](https://devguide.python.org/#status-of-python-branches) as of September 2020, and 3.6 has some moderately nice [features](https://docs.python.org/3/whatsnew/3.6.html):

  - `f'x = {x}'` as an alternative to `'x = {}'.format(x)` format strings (cf https://github.com/bitcoin/bitcoin/pull/13718#issuecomment-406591027)
  - underscore separators for large numbers, like `1_234_567`
  - improvements to async
  - improvements to typing module

  Note that 3.6 is not available in xenial (16.04), but is available in bionic (18.04), while focal (20.04) has 3.8. CentOS 7 and 8 have 3.6.8, Debian stable has 3.7.3, and [gentoo and arch already had 3.6 and 3.7 in 2018](https://github.com/bitcoin/bitcoin/pull/14954#issuecomment-447118707).

ACKs for top commit:
  MarcoFalke:
    re-ACK 97c738ff1b

Tree-SHA512: ec7fce68845edde4d61a42de12c065fd49e5217311a6fda1323206f091a0afd50f293645dffc27d420127e4e5deb864e953f1b67eff735a0dfbbedd7899a9d60
2020-11-18 10:24:22 +01:00
MarcoFalke
80e32e120e
Merge #20305: wallet: introduce fee_rate sat/vB param/option
05e82d86b0 wallet: override minfee checks (fOverrideFeeRate) for fee_rate (Jon Atack)
9a670b4f07 wallet: update sendtoaddress, send RPC examples with fee_rate (Jon Atack)
be481b72e2 wallet: use MIN_RELAY_TX_FEE in bumpfee help (Jon Atack)
449b730579 wallet: provide valid values if invalid estimate mode passed (Jon Atack)
6da3afbaee wallet: update remaining rpcwallet fee rate units to BTC/kvB (Jon Atack)
173b5b5fe0 wallet: update fee rate units, use sat/vB for fee_rate error messages (Jon Atack)
7f9835a05a wallet: remove fee rates from conf_target helps (Jon Atack)
b7994c01e9 wallet: add fee_rate unit warnings to bumpfee (Jon Atack)
410e471fa4 wallet: remove redundant bumpfee fee_rate checks (Jon Atack)
a0d4957473 wallet: introduce fee_rate (sat/vB) param/option (Jon Atack)
e21212f01b wallet: remove unneeded WALLET_BTC_KB_TO_SAT_B constant (Jon Atack)
6112cf20d4 wallet: add CFeeRate ctor doxygen documentation (Jon Atack)
3f72791613 wallet: fix bug in RPC send options (Jon Atack)

Pull request description:

  This PR builds on #11413 and #20220 to address #19543.

  - replace overloading the conf_target and estimate_mode params with `fee_rate` in sat/vB in the sendtoaddress, sendmany, send, fundrawtransaction, walletcreatefundedpsbt, and bumpfee RPCs

  - allow non-actionable conf_target value of `0` and estimate_mode value of `""` to be passed to use `fee_rate` as a positional argument, in addition to as a named argument

  - fix a bug in the experimental send RPC described in https://github.com/bitcoin/bitcoin/pull/20220#discussion_r513789526 where args were not being passed correctly into the options values

  - update the feerate error message units for these RPCs from BTC/kB to sat/vB

  - update the test coverage, help docs, doxygen docs, and some of the RPC examples

  - other changes to address the excellent review feedback

  See this wallet meeting log for more context: http://www.erisian.com.au/bitcoin-core-dev/log-2020-11-06.html#l-309

ACKs for top commit:
  achow101:
    re-ACK 05e82d8
  MarcoFalke:
    review ACK 05e82d86b0 did not test and found a few style nits, which can be fixed later 🍯
  Xekyo:
    tACK 05e82d86b0
  Sjors:
    utACK 05e82d86b0

Tree-SHA512: a4ee5f184ada53f1840b2923d25873bda88c5a2ae48e67eeea2417a0b35154798cfdb3c147b05dd56bd6608a784e1b91623bb985ee2ab9ef2baaec22206d0a9c
2020-11-17 13:49:12 +01:00
Michael Dietz
21f2433601 test: run mempool_spend_coinbase.py even with wallet disabled 2020-11-16 09:05:34 -06:00
Wladimir J. van der Laan
c48e788246
Merge #18836: wallet: upgradewallet fixes and additional tests
5f9c0b6360 wallet: Remove -upgradewallet from dummywallet (MarcoFalke)
a314271f08 test: Remove unused wallet.dat (MarcoFalke)
bf7635963c tests: Test specific upgradewallet scenarios and that upgrades work (Andrew Chow)
4b418a9dec test: Add test_framework/bdb.py module for inspecting bdb files (Andrew Chow)
092fc43485 tests: Add a sha256sum_file function to util (Andrew Chow)
0bd995aa19 wallet: upgrade the CHDChain version number when upgrading to split hd (Andrew Chow)
8e32e1c41c wallet: remove nWalletMaxVersion (Andrew Chow)
bd7398cc62 wallet: have ScriptPubKeyMan::Upgrade check against the new version (Andrew Chow)
5f720544f3 wallet: Add GetClosestWalletFeature function (Andrew Chow)
842ae3842d wallet: Add utility method for CanSupportFeature (Andrew Chow)

Pull request description:

  This PR cleans up the wallet upgrade mechanism a bit, fixes some probably bugs, and adds more test cases.

  The `nWalletMaxVersion` member variable has been removed as it made `CanSupportFeature` unintuitive and was causing a couple of bugs. The reason this was introduced originally was to allow a wallet upgrade to only occur when the new feature is first used. While this makes sense for the old `-upgradewallet` option, for an RPC, this does not quite make sense. It's more intuitive for an upgrade to occur if possible if the `upgradewallet` RPC is used as that's an explicit request to upgrade a particular wallet to a newer version. `nWalletMaxVersion` was only relevant for upgrades to `FEATURE_WALLETCRYPT` and `FEATURE_COMPRPUBKEY` both of which are incredibly old features. So for such wallets, the behavior of `upgradewallet` will be that the feature is enabled immediately without the wallet needing to be encrypted at that time (note that `FEATURE_WALLETCRYPT` indicates support for encryption, not that the wallet is encrypted) or for a new key to be generated.

  `CanSupportFeature` would previously indicate whether we could upgrade to `nWalletMaxVersion` not just whether the current wallet version supported a feature. While this property was being used to determine whether we should upgrade to HD and HD chain split, it was also causing a few bugs. Determining whether we should upgrade to HD or HD chain split is resolved by passing into `ScriptPubKeyMan::Upgrade` the version we are upgrading to and checking against that. By removing `nWalletMaxVersion` we also fix a bug where you could upgrade to HD chain split without the pre-split keypool.

  `nWalletMaxVersion` was also the version that was being reported by `getwalletinfo` which meant that the version reported was not always consistent across restarts as it depended on whether `upgradewallet` was used. Additionally to make the wallet versions consistent with actually supported versions, instead of just setting the wallet version to whatever is given to `upgradewallet`, we normalize the version number to the closest supported version number. For example, if given 150000, we would store and report 139900.

  Another bug where CHDChain was not being upgraded to the version supporting HD chain split is also fixed by this PR.

  Lastly several more tests have been added. Some refactoring to the test was made to make these tests easier. These tests check specific upgrading scenarios, such as from non-HD (version 60000) to HD to pre-split keypool. Although not specifically related to `upgradewallet`, `UpgradeKeyMetadata` is now being tested too.

  Part of the new tests is checking that the wallet files are identical before and after failed upgrades. To facilitate this, a utility function `sha256sum_file` has been added. Another part of the tests is to examine the wallet file itself to ensure that the records in the wallet.dat file have been correctly modified. So a new `bdb.py` module has been added to deserialize the BDB db of the wallet.dat file. This format isn't explicitly documented anywhere, but the code and comments in BDB's source code in file `dbinc/db_page.h` describe it. This module just dumps all of the fields into a dict.

ACKs for top commit:
  MarcoFalke:
    approach ACK 5f9c0b6360
  laanwj:
    Code review ACK 5f9c0b6360
  jonatack:
    ACK 5f9c0b6360, approach seems fine, code review, only skimmed the test changes but they look well done, rebased on current master, debug built and verified the `wallet_upgradewallet.py` test runs green both before and after running `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2`

Tree-SHA512: 7c4ebf420850d596a586cb6dd7f2ef39c6477847d12d105fcd362abb07f2a8aa4f7afc5bfd36cbc8b8c72fcdd1de8d2d3f16ad8e8ba736b6f4f31f133fe5feba
2020-11-16 11:03:25 +01:00
Ivan Metlushko
173cc9b7be test: walettool create descriptors 2020-11-13 17:52:28 +07:00
Jon Atack
05e82d86b0
wallet: override minfee checks (fOverrideFeeRate) for fee_rate
in RPCs fundrawtransaction and walletcreatefundedpsbt only.

This provides the existing feeRate (BTC/kvB) behavior in these two RPCs to the
new fee_rate (sat/vB) param also.

See these two GitHub review discussions for more info:
https://github.com/bitcoin/bitcoin/pull/10706/#discussion_r126560525
https://github.com/bitcoin/bitcoin/pull/20305#discussion_r520032533
2020-11-12 11:44:15 +01:00
Jon Atack
449b730579
wallet: provide valid values if invalid estimate mode passed
Co-authored-by: Murch <murch@murch.one>
2020-11-12 11:43:17 +01:00
Jon Atack
173b5b5fe0
wallet: update fee rate units, use sat/vB for fee_rate error messages
and BTC/kvB for feeRate error messages.
2020-11-12 11:43:03 +01:00
Samuel Dobson
c2d8ba6265
Merge #19502: Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks
24d2d3341d QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored (Luke Dashjr)
69f59af54d Bugfix: Wallet: Soft-fail exceptions within ListWalletDir file checks (Luke Dashjr)

Pull request description:

  Previously, an exception would be thrown, which could kill the node in some circumstances.

  Includes test changes to cause failure.

  Review with `?w=1`

ACKs for top commit:
  hebasto:
    re-ACK 24d2d3341d, rebased only since my [previous](https://github.com/bitcoin/bitcoin/pull/19502#pullrequestreview-520552944) review.
  promag:
    Tested ACK 24d2d3341d, test change fails on master.
  meshcollider:
    utACK 24d2d3341d

Tree-SHA512: f701f81b3aa3d3e15cee52ac9e7c31a73c0d8166e56bf077235294507cbcee099829fedc432a1c4b6d8780885f4e37897b44b980b08125771de3c849c000499e
2020-11-12 13:26:38 +13:00
Jon Atack
410e471fa4
wallet: remove redundant bumpfee fee_rate checks
SetFeeEstimateMode() handles these checks now and provides a more actionable
error message.
2020-11-11 15:56:01 +01:00
Jon Atack
a0d4957473
wallet: introduce fee_rate (sat/vB) param/option
Create a fee_rate (sat/vB) RPC param and replace overloading the conf_target and
estimate_mode params in the following 6 RPCs with it:

- sendtoaddress
- sendmany
- send
- fundrawtransaction
- walletcreatefundedpsbt
- bumpfee

In RPC bumpfee, the previously existing fee_rate remains but the unit is changed
from BTC/kvB to sat/vB. This is a breaking change, but it should not be an
overly risky one, as the units change by a factor of 1e5 and any fees specified
in BTC/kvB after this commit will either be too low and raise an error or be 1
sat/vB and can be RBFed.

Update the test coverage for each RPC.

Co-authored-by: Murch <murch@murch.one>
2020-11-11 15:55:59 +01:00
Jon Atack
3f72791613
wallet: fix bug in RPC send options
when empty, options were not being populated by arguments of the same name

found while adding test coverage in 603c0050
2020-11-11 15:55:46 +01:00
MarcoFalke
42f950cb27
Merge #20322: test: Fix intermittent issue in wallet_listsinceblock
444412821e test: Fix intermittent issue in wallet_listsinceblock (MarcoFalke)

Pull request description:

ACKs for top commit:
  Empact:
    Code Review ACK 444412821e

Tree-SHA512: 86d47b1e3c8681dd479654589c894016ac81a3c96a34c3b4a75278b2af85054ea8c6f768e518a5322a4928d82d5e99105bbce0f4fa6a7a18c40e3e0799f9ab54
2020-11-10 09:20:51 +01:00
Wladimir J. van der Laan
1dfe19e284
Merge #20153: wallet: do not import a descriptor with hardened derivations into a watch-only wallet
538be4219a wallet: fix importdescriptor silent fail (Ivan Metlushko)

Pull request description:

  Currently `importdescriptor` command will successfully import a descriptor with hardened derivations into a watch-only wallet while silently failing to expand the descriptor to fill the cache. This leads to a broken wallet state and failure to load such wallet due to missing cache on subsequent restart.

ACKs for top commit:
  laanwj:
    Code review ACK 538be4219a
  achow101:
    ACK 538be4219a
  meshcollider:
    utACK 538be4219a

Tree-SHA512: 4bdd0ab4437d55b3f1a79c3a300a0b186089155c020fe220a73d0cce274de47d90371d88918d39fd795f9fccf8db328f1e322d29a6062f9ce94a1c254398f004
2020-11-09 20:19:00 +01:00
Wladimir J. van der Laan
79a3b59cc7
Merge #20120: net, rpc, test, bugfix: update GetNetworkName, GetNetworksInfo, regression tests
7b5bd3102e test: add getnetworkinfo network name regression tests (Jon Atack)
9a75e1e569 rpc: update GetNetworksInfo() to not return unsupported networks (Jon Atack)
ba8997fb2e net: update GetNetworkName() with all enum Network cases (Jon Atack)

Pull request description:

  Following up on the BIP155 addrv2 changes, and starting with 7be6ff6 in #19845, RPC getnetworkinfo began returning networks with empty names.

  <details><summary><code>getnetworkinfo</code> on current master</summary><p>

  ```
    "networks": [
      {
        "name": "ipv4",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "ipv6",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "onion",
        "limited": false,
        "reachable": true,
        "proxy": "127.0.0.1:9050",
        "proxy_randomize_credentials": true
      },
      {
        "name": "",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      }
    ],
  ```
  </p></details>

  <details><summary><code>getnetworkinfo</code> on this branch</summary><p>

  ```
    "networks": [
      {
        "name": "ipv4",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "ipv6",
        "limited": false,
        "reachable": true,
        "proxy": "",
        "proxy_randomize_credentials": false
      },
      {
        "name": "onion",
        "limited": false,
        "reachable": true,
        "proxy": "127.0.0.1:9050",
        "proxy_randomize_credentials": true
      }
    ],
  ```
  </p></details>

  This patch:
  - updates `GetNetworkName()` to the current Network enum
  - updates `getNetworksInfo()` to ignore as-yet unsupported networks
  - adds regression tests

ACKs for top commit:
  hebasto:
    re-ACK 7b5bd3102e
  vasild:
    ACK 7b5bd3102

Tree-SHA512: 8f12363eb430e6f45e59e3b1d69c2f2eb5ead254ce7a67547d116c3b70138d763157335a3c85b51f684a3b3b502c6aace4f6fa60ac3b988cf7a9475a7423c4d7
2020-11-09 17:07:23 +01:00
Wladimir J. van der Laan
4fd37d0a10
Merge #20292: test: Fix intermittent feature_taproot issue
fab900802d ci: Bump timeout factor (MarcoFalke)
50eb0c2512 Small improvements to the Taproot functional tests (Pieter Wuille)
fac865b72d test: Fix intermittent feature_taproot issue (MarcoFalke)
fa1dea19fc test: Fix deser issue in create_block (MarcoFalke)
fa762a3fd4 test: Remove unused unnamed parameter from block.serialize call (MarcoFalke)

Pull request description:

  This fixes three bugs. Also, fix some unrelated code style issues.

  Please refer to the commit messages for more information.

ACKs for top commit:
  laanwj:
    Code review ACK fab900802d

Tree-SHA512: 4e22c240cf345710f3b21fc63243126b90014b3656d0865ff87156e958dd1442e6572c6c0a5701dbbe503eee931a0ceb66eeeb3553137f3d1f5afd27a9f9cada
2020-11-09 15:47:04 +01:00
Wladimir J. van der Laan
663fd92b28
Merge #20266: wallet: fix change detection of imported internal descriptors
bd93fc9945 Fix change detection of imported internal descriptors (Andrew Chow)

Pull request description:

  Import internal descriptors were having address book entries added which meant they would be detected as non-change. Fix this and add a test for it.

ACKs for top commit:
  laanwj:
    Code review ACK bd93fc9945
  meshcollider:
    utACK bd93fc9945
  promag:
    Code review ACK bd93fc9945.

Tree-SHA512: 8fa9e364be317627ec171eedffdb505976c0e7f1e55bc7e8cfdffa3aeea5db24d231f55166602cd0e97a5ba621acc871de0a765c75d0c65678f83e93c3b657c5
2020-11-09 15:14:45 +01:00
Anthony Towns
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them 2020-11-09 17:53:50 +10:00
Luke Dashjr
24d2d3341d QA: wallet_multiwallet: Check that recursive symlink directory and wallet.dat loops are ignored 2020-11-06 04:35:33 +00:00
Stepan Snigirev
568a1d7261 fix ecdsa verify in test framework 2020-11-05 23:16:55 +01:00
MarcoFalke
444412821e
test: Fix intermittent issue in wallet_listsinceblock 2020-11-05 19:51:57 +01:00
MarcoFalke
fa00ff0399
test: Fix wallet_multiwallet test issue on Windows 2020-11-05 13:50:15 +01:00
fanquake
6954e4d16c
Merge #20283: test: Only try witness deser when checking for witness deser failure
fae45c34d1 test: Only try witness deserialize when checking for witness deserialize failure (MarcoFalke)

Pull request description:

  Witness deserialize will fail always. (This is what the test is checking for)

  Consequently, non-witness deserialize is also tried, and it might succeed accidentally. Avoid that by not trying non-witness deserialize.

  Fixes #20249

ACKs for top commit:
  jnewbery:
    utACK fae45c34d1

Tree-SHA512: 45e65b31603e3ca839776a7ed30e363b32eba20dfb67b7b55bff06715876850d4f6ba22f8ea4911a62e1f8ffff395bf187b23c46ddc766516b97057df000deb3
2020-11-05 14:56:02 +08:00
MarcoFalke
83650e4df5
Merge #20199: wallet: ignore (but warn) on duplicate -wallet parameters
58cfbc38e0 Ignoring (but warn) on duplicate -wallet parameters (Jonas Schnelli)

Pull request description:

  I expect that there are many users with load on startup wallet definitions in `bitcoin.conf` or via startup CLI argument.
  With the new `settings.json` r/w configuration file, users unloading and loading a wallet through the GUI or via the RPC calls might end up with a duplicate `-wallet` entry (one that still remains in bitcoin.conf or CLI) plus the new duplication in `settings.json` due to the unload/load.

  Steps to reproduce
  * create wallet (if via RPC set `load_on_startup` or unloadwallet/loadwallet then set `load_on_startup`).
  * stop bitcoin
  * start bitcoind again with same `--wallet=mywallet`

  I guess it is acceptable to skip duplicates.

ACKs for top commit:
  achow101:
    Tested ACK 58cfbc38e0
  meshcollider:
    Code review ACK 58cfbc38e0
  ryanofsky:
    Code review ACK 58cfbc38e0. Changes since previous review: rebased, tweaked warning message, squashed/fixed test

Tree-SHA512: f94e5a999bdd7dc291f0bc142911b0a8033929350d6f6a35b58c4a06a3c8f83147be0f0c402d4e946dedbbcc85b7e023b672c731b6d7a8984d4780017c961cfb
2020-11-05 07:51:07 +01:00
MarcoFalke
5f9c0b6360 wallet: Remove -upgradewallet from dummywallet 2020-11-04 12:16:57 -05:00
MarcoFalke
a314271f08 test: Remove unused wallet.dat 2020-11-04 12:16:57 -05:00
Andrew Chow
bf7635963c tests: Test specific upgradewallet scenarios and that upgrades work 2020-11-04 12:16:54 -05:00
Andrew Chow
4b418a9dec test: Add test_framework/bdb.py module for inspecting bdb files
For upgrade tests and possibly other tests, it is useful to inspect the
bdb file for the wallet (i.e. the wallet.dat file).
test_framework/bdb.py is an implementation of bdb file deserialization
specific for Bitcoin Core's usage.
2020-11-04 12:15:29 -05:00
Andrew Chow
092fc43485 tests: Add a sha256sum_file function to util 2020-11-04 12:15:25 -05:00
MarcoFalke
fa2ecadd0d
test: Fix intermittent rpc_net issue 2020-11-04 13:21:54 +01:00
Samuel Dobson
5d32009f1a
Merge #20220: wallet, rpc: explicit fee rate follow-ups/fixes for 0.21
0be29000c0 rpc: update conf_target helps for correctness/consistency (Jon Atack)
778b9be406 wallet, rpc: fix send subtract_fee_from_outputs help (Jon Atack)
603c005083 wallet: add rpc send explicit fee rate coverage (Jon Atack)
dd341e602d wallet: add sendtoaddress/sendmany explicit fee rate coverage (Jon Atack)
44e7bfa603 wallet: add walletcreatefundedpsbt explicit fee rate coverage (Jon Atack)
6e1ea4273e test: refactor for walletcreatefundedpsbt fee rate coverage (Jon Atack)
3ac7b0c6f1 wallet: fundrawtx fee rate coverage, fixup ParseConfirmTarget() (Jon Atack)
2d8eba8f84 wallet: combine redundant bumpfee invalid params and args tests (Jon Atack)
1697a40b6f wallet: improve bumpfee error/help, add explicit fee rate coverage (Jon Atack)
fc5721723d wallet: fix SetFeeEstimateMode() error message (Jon Atack)
052427eef1 wallet, bugfix: fix bumpfee with explicit fee rate modes (Jon Atack)

Pull request description:

  Follow-up to #11413 providing a base to build on for #19543:

  - bugfix for `bumpfee` raising a JSON error with explicit feerates, fixes issue #20219
  - adds explicit feerate test coverage for `bumpfee`, `fundrawtransaction`, `walletcreatefundedpsbt`, `send`, `sendtoaddress`, and `sendmany`
  - improves a few related RPC error messages and `ParseConfirmTarget()` / error message
  - fixes/improves the explicit fee rate information in the 6 RPC helps, of which 2 were also missing `conf_target` sat/B units

  This provides a spec and regression coverage for the potential next step of a universal `sat/vB` feerate argument (see #19543), as well as immediate coverage and minimum fixes for 0.21.

ACKs for top commit:
  kallewoof:
    Concept/Tested ACK 0be29000c0
  meshcollider:
    Code review + functional test run ACK 0be29000c0

Tree-SHA512: efd965003e991cba51d4504e2940f06ab3d742e34022e96a673606b44fad85596aa03a8c1809f06df7ebcf21a38e18a891e54392fe3d6fb4d120bbe4ea0cf5e0
2020-11-04 16:35:23 +13:00
Samuel Dobson
17c6fb176a
Merge #20282: wallet: change upgradewallet return type to be an object
2ead31fb1b [wallet] Return object from upgradewallet RPC (Sishir Giri)

Pull request description:

  Change the return type of upgradewallet to be an object for future extensibility.

  Also return any error string returned from the `UpgradeWallet()` function.

ACKs for top commit:
  MarcoFalke:
    ACK 2ead31fb1b
  meshcollider:
    Tested ACK 2ead31fb1b

Tree-SHA512: bcc7432d2f35093ec2463ea19e894fa885b698c0e8d8e4bd2f979bd4d722cbfed53ec589d6280968917893c64649dc9e40800b8d854273b0f9a1380f51afbdb1
2020-11-04 14:51:42 +13:00
Pieter Wuille
50eb0c2512 Small improvements to the Taproot functional tests
The "whitelist" and "connect_nodes" is not needed in feature_taproot.py,
so remove it.

The changes to key.py are required when running the unit tests from the
test folder. Failure on current master:

[test]$ python -m unittest functional/test_framework/key.py
.E
======================================================================
ERROR: test_schnorr_testvectors (functional.test_framework.key.TestFrameworkKey)
Implement the BIP340 test vectors (read from bip340_test_vectors.csv).
----------------------------------------------------------------------
Traceback (most recent call last):
  File "test/functional/test_framework/key.py", line 526, in test_schnorr_testvectors
    with open(os.path.join(sys.path[0], 'test_framework', 'bip340_test_vectors.csv'), newline='', encoding='utf8') as csvfile:
FileNotFoundError: [Errno 2] No such file or directory: 'test/test_framework/bip340_test_vectors.csv'

----------------------------------------------------------------------
Ran 2 tests in 0.775s

FAILED (errors=1)
2020-11-03 12:26:17 +01:00
MarcoFalke
fac865b72d
test: Fix intermittent feature_taproot issue
The transaction is too large to fit into the mempool, so put it into a
block.

https://travis-ci.org/github/bitcoin/bitcoin/jobs/740987240#L7217

test  2020-11-03T01:31:08.645000Z TestFramework (ERROR): JSONRPC error
        Traceback (most recent call last):
          File "./test/functional/test_framework/test_framework.py", line 126, in main
            self.run_test()
          File "./test/functional/feature_taproot.py", line 1448, in run_test
            self.nodes[1].sendtoaddress(address=addr, amount=int(self.nodes[1].getbalance() * 70000000) / 100000000)
          File "./test/functional/test_framework/coverage.py", line 47, in __call__
            return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
          File "./test/functional/test_framework/authproxy.py", line 146, in __call__
            raise JSONRPCException(response['error'], status)
        test_framework.authproxy.JSONRPCException: Transaction too large (-6)
2020-11-03 12:26:17 +01:00
MarcoFalke
fa1dea19fc
test: Fix deser issue in create_block
Without the fix a hex-string can not be parsed:

File "./test/functional/test_framework/blocktools.py", line 82, in create_block
    txo.deserialize(io.BytesIO(tx))
TypeError: a bytes-like object is required, not 'str'

Also, remove io import and repace it with our FromHex() helper
2020-11-03 12:25:55 +01:00
MarcoFalke
fa762a3fd4
test: Remove unused unnamed parameter from block.serialize call
All blocks and transactions are serialized with witness, no need to set
this
2020-11-03 12:23:21 +01:00
Jonas Schnelli
58cfbc38e0 Ignoring (but warn) on duplicate -wallet parameters 2020-11-03 12:06:32 +01:00
Antoine Riard
bc4a230087 Remove redundant p2p lock tacking for tx download functional tests
New functional test coverage of tx download was added by #19988,
but `with p2p_lock` is redundant for some tests with `wait_until`
test helper, already guaranteeing test lock tacking.
2020-11-02 18:29:49 -05:00
Antoine Riard
d3b5eac9a9 Add mutation for functional test test_preferred_inv
Add a booelan arg to test_preferred_inv to cover NONPREF_PEER_TX_DELAY
as applied as a TxRequestTracker parameter in AddTxAnnouncement.
2020-11-02 18:29:49 -05:00
Antoine Riard
06efb3163c Add functional test test_txid_inv_delay
Add a simple functional test to cover TXID_RELAY_DELAY as applied
as a TxRequestTracker parameter in AddTxAnnoucement.
2020-11-02 18:29:49 -05:00
Antoine Riard
a07910abcd test: Makes wtxidrelay support a generic P2PInterface option
Its usage is extended beyond p2p_segwit.py in next commit.
2020-11-02 18:29:48 -05:00
Wladimir J. van der Laan
ef4c7c4e0b
Merge #18788: tests: Update more tests to work with descriptor wallets
c7b7e0a692 tests: Make only desc wallets for wallet_multwallet.py --descriptors (Andrew Chow)
d4b67ad214 Avoid creating legacy wallets in wallet_importdescriptors.py (Andrew Chow)
6c9c12bf87 Update feature_backwards_compatibility for descriptor wallets (Andrew Chow)
9a4c631e1c Update wallet_labels.py to not require descriptors=False (Andrew Chow)
242aed7cc1 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors (Andrew Chow)
388053e172 Disable some tests for tool_wallet when descriptors (Andrew Chow)
47d3243160 Make raw multisig tests legacy wallet only in rpc_rawtransaction.py (Andrew Chow)
59d3da5bce Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py (Andrew Chow)
25bc5dccbf Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py (Andrew Chow)
0bd1860300 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py (Andrew Chow)
08067aebfd Add script equivalent of functions in address.py (Andrew Chow)
86968882a8 Add descriptor wallet output to tool_wallet.py (Andrew Chow)
3457679870 Use separate watchonly wallet for multisig in feature_nulldummy.py (Andrew Chow)
a42652ec10 Move import and watchonly tests to be legacy wallet only in wallet_balance.py (Andrew Chow)
4b871909d6 Use importdescriptors for descriptor wallets in wallet_bumpfee.py (Andrew Chow)
c2711e4230 Avoid dumpprivkey in wallet_listsinceblock.py (Andrew Chow)
553dbf9af4 Make import tests in wallet_listtransactions.py legacy wallet only (Andrew Chow)
dc81418fd0 Use a separate watchonly wallet in rpc_fundrawtransaction.py (Andrew Chow)
a357111047 Update wallet_importprunedfunds to avoid dumpprivkey (Andrew Chow)

Pull request description:

  I went through all the tests and checked whether they passed with descriptor wallets. This partially informed some changes in #16528. Some tests needed changes to work with descriptor wallets. These were primarily due to import and watchonly behavior. There are some tests and test cases that only test legacy wallet behavior so those tests won't be run with descriptor wallets.

  This PR updates more tests to have to the `--descriptors` switch in `test_runner.py`. Additionally a mutually exclusive `--legacy-wallet` option has been added to force legacy wallets. This does nothing currently but will be useful in the future when descriptor wallets are the default. For the tests that rely on legacy wallet behavior, this option is being set so that we don't forget in the future. Those tests are `feature_segwit.py`, `wallet_watchonly.py`, `wallet_implicitsegwit.py`, `wallet_import_with_label.py`, and `wallet_import_with_label.py`.

  If you invert the `--descriptors`/`--legacy-wallet` default so that descriptor wallets are the default, all tests (besides the legacy wallet specific ones) will pass.

ACKs for top commit:
  MarcoFalke:
    review ACK c7b7e0a692 🎿
  laanwj:
    ACK c7b7e0a692

Tree-SHA512: 2f4e87815005d1d0a2543ea7947f7cd7593d8cf5312228ef85f8e096f19739b225769961943049cb44f6f07a35b8de988e2246ab9aca5bb5a0b2e62694d5637d
2020-11-02 18:50:37 +01:00
MarcoFalke
fae45c34d1
test: Only try witness deserialize when checking for witness deserialize failure 2020-11-02 14:19:44 +01:00
MarcoFalke
c5ec0367d7
Merge #20165: Only relay Taproot spends if next block has it active
3d0556d410 Increase feature_taproot inactive test coverage (Pieter Wuille)
525cbd425e Only relay Taproot spends if next block has it active (Pieter Wuille)

Pull request description:

  There should be no change to mempool transaction behavior for witness v1 transactions as long as no activation is defined. Until that point, we should treat the consensus rules as under debate, and for soft-fork safety, that means spends should be treated as non-standard.

  It's possible to go further: don't relay them unless the consensus rules are actually active for the next block. This extends non-relay to the period where a deployment is defined, started, locked in, or failed. I see no downsides to this, and the code change is very simple.

ACKs for top commit:
  Sjors:
    utACK 3d0556d410
  MarcoFalke:
    review ACK 3d0556d410 🏓
  jnewbery:
    utACK 3d0556d410

Tree-SHA512: ca625a2981716b4b44e8f3722718fd25fd04e25bf3ca1684924b8974fca49f7c1d438fdd9dcdfbc091a442002e20d441d42c41a0e2096e74a61068da6c60267a
2020-11-02 10:12:06 +01:00
Sishir Giri
2ead31fb1b [wallet] Return object from upgradewallet RPC 2020-11-02 08:38:38 +00:00
Andrew Chow
c7b7e0a692 tests: Make only desc wallets for wallet_multwallet.py --descriptors 2020-11-01 17:54:19 -05:00
Andrew Chow
d4b67ad214 Avoid creating legacy wallets in wallet_importdescriptors.py 2020-11-01 17:54:19 -05:00
Andrew Chow
6c9c12bf87 Update feature_backwards_compatibility for descriptor wallets 2020-11-01 17:54:19 -05:00
Andrew Chow
9a4c631e1c Update wallet_labels.py to not require descriptors=False 2020-11-01 17:54:19 -05:00
Andrew Chow
242aed7cc1 tests: Add a --legacy-wallet that is mutually exclusive with --descriptors
Although legacy wallet is still the default, for future use, add a
--legacy-wallet option to the test framework. Additional tests for
descriptor wallets have been enabled with the --descriptors option.
Tests that must be legacy wallet only are being started with
--legacy-wallet. Even though this option does not currently do anything,
this will be helpful in the future when descriptor wallets become the
default.
2020-11-01 17:54:19 -05:00
Andrew Chow
388053e172 Disable some tests for tool_wallet when descriptors
Some tests are legacy wallet only (and make legacy wallets) so they
shouldn't be run when doing descriptor wallet tests.
2020-11-01 17:54:19 -05:00
Andrew Chow
47d3243160 Make raw multisig tests legacy wallet only in rpc_rawtransaction.py
The traditional multisig workflow doesn't work with descriptor wallets
so make these tests legacy wallet only.
2020-11-01 17:54:19 -05:00
Andrew Chow
59d3da5bce Do addmultisigaddress tests in legacy wallet mode in wallet_address_types.py
addmultisigaddress is not available in descriptor wallets, so only run
these when testing legacy wallets
2020-11-01 17:54:19 -05:00
Andrew Chow
25bc5dccbf Use importdescriptors when in descriptor wallet mode in wallet_createwallet.py
sethdseed and importmulti are not available for descriptor wallets, so
when doing descriptor wallet tests, use importdescriptors instead.

Also changes some output to match what descriptor wallets will return.
2020-11-01 17:54:19 -05:00
Andrew Chow
0bd1860300 Avoid dumpprivkey and watchonly behavior in rpc_signrawtransaction.py
dumpprivkey and watchonly behavior don't work with descriptor wallets.

Test for multisigs is modified to not rely on watchonly behavior for
those multisigs. This has a side effect of removing listunspent, but
that's not the target of this test, so that's fine.
2020-11-01 17:54:19 -05:00
Andrew Chow
08067aebfd Add script equivalent of functions in address.py 2020-11-01 17:54:19 -05:00
Andrew Chow
86968882a8 Add descriptor wallet output to tool_wallet.py
Descriptor wallets output slightly different information in the wallet
tool, so check that output when in descriptor wallet mode.
2020-11-01 17:54:19 -05:00
Andrew Chow
3457679870 Use separate watchonly wallet for multisig in feature_nulldummy.py
Create and import the multisig into a separate watchonly wallet so that
feature_nulldummy.py works with descriptor wallets.

blocktools.create_raw_transaction is also updated to use multiple nodes
and wallets and to use PSBT so that this test passes.
2020-11-01 17:54:19 -05:00
Andrew Chow
a42652ec10 Move import and watchonly tests to be legacy wallet only in wallet_balance.py
Imports and watchonly behavior are legacy wallet only, so make them only
run when the test is in legacy wallet mode.
2020-11-01 17:54:19 -05:00
Andrew Chow
4b871909d6 Use importdescriptors for descriptor wallets in wallet_bumpfee.py
If using descriptor wallets, use importdescriptors instead of
importmulti.
2020-11-01 17:54:19 -05:00
Andrew Chow
c2711e4230 Avoid dumpprivkey in wallet_listsinceblock.py
Generate a privkey in the test framework instead of using dumpprivkey so
that descriptor wallets work in this test.
2020-11-01 17:54:19 -05:00
Andrew Chow
553dbf9af4 Make import tests in wallet_listtransactions.py legacy wallet only
Existing import* RPCs are disabled for descriptor wallets, so only do
these tests for legacy wallets.
2020-11-01 17:54:19 -05:00
Andrew Chow
dc81418fd0 Use a separate watchonly wallet in rpc_fundrawtransaction.py
Import things into a separate watchonly wallet to work with descriptor
wallets.
2020-11-01 17:54:19 -05:00
Andrew Chow
a357111047 Update wallet_importprunedfunds to avoid dumpprivkey
Removes the use of dumpprivkey so that descriptor wallets can pass on
this. Also does a few descriptor wallet specific changes due to
different IsMine semantics.
2020-11-01 17:54:19 -05:00
Pieter Wuille
3d0556d410 Increase feature_taproot inactive test coverage 2020-10-30 15:52:38 -07:00
Pieter Wuille
525cbd425e Only relay Taproot spends if next block has it active 2020-10-30 15:52:19 -07:00
Andrew Chow
bd93fc9945 Fix change detection of imported internal descriptors 2020-10-29 17:55:13 -04:00
Andrew Chow
7411876c75 Ensure a legacy wallet for BDB format check 2020-10-29 14:59:29 -04:00
Andrew Chow
586640381a Skip --descriptor tests if sqlite is not compiled 2020-10-29 12:34:16 -04:00
MarcoFalke
42b66a6b81
Merge #20186: wallet: Make -wallet setting not create wallets
01476a88a6 wallet: Make -wallet setting not create wallets (Russell Yanofsky)

Pull request description:

  This changes `-wallet` setting to only load existing wallets, not create new ones.

  - Fixes settings.json corner cases reported by sjors & promag: https://github.com/bitcoin-core/gui/issues/95, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

  - Prevents accidental creation of wallets reported most recently by jb55 http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

  - Simplifies behavior after #15454. #15454 took the big step of disabling creation of the default wallet. This PR extends it to avoid creating other wallets as well. With this change, new wallets just aren't created on startup, instead of sometimes being created, sometimes not. #15454 release notes are updated here and are simpler.

  This change should be targeted for 0.21.0. It's a bug fix and simplifies behavior of the #15937 / #19754 / #15454 features added in 0.21.0.

  ---

  This PR is implementing the simplest, most basic alternative listed in https://github.com/bitcoin-core/gui/issues/95#issuecomment-694236940. Other improvements mentioned there can build on top of this.

ACKs for top commit:
  achow101:
    ACK 01476a88a6
  hebasto:
    re-ACK 01476a88a6
  MarcoFalke:
    review ACK 01476a88a6 🏂

Tree-SHA512: 0d50f4e5dfbd04a2efd9fd66c02085a0ed705807bdec1cf5770d0ae8cb6af07080fb81306349937bf66acdb713d03fb35636f6442b650d0820e66cbae09c2f87
2020-10-29 15:01:39 +01:00
Wladimir J. van der Laan
6196cf77e5
Merge #19753: p2p: don't add AlreadyHave transactions to recentRejects
d419fdedbe [net processing] Don't add AlreadyHave txs to recentRejects (Troy Giorshev)

Pull request description:

  If we already have a transaction, don't add it to recentRejects

  Now, we only add a transaction to our recentRejects filter if we didn't already have it, meaning that it is added at most once, as intended.

ACKs for top commit:
  jnewbery:
    Code review ACK d419fdedbe
  laanwj:
    Code review ACK d419fdedbe

Tree-SHA512: cff5c1ba36c4700e2d6ab3eec4a3e51e1bef28fb3cc1bc850c84e06d6e5a9f6c32825207c253cc9cdf596b2eaadb6b5be68b3f8ca752b4ef6c31cf85138e3c99
2020-10-29 11:40:15 +01:00
Jon Atack
603c005083
wallet: add rpc send explicit fee rate coverage 2020-10-29 00:22:08 +01:00
Jon Atack
dd341e602d
wallet: add sendtoaddress/sendmany explicit fee rate coverage 2020-10-29 00:22:04 +01:00
Jon Atack
44e7bfa603
wallet: add walletcreatefundedpsbt explicit fee rate coverage 2020-10-29 00:22:02 +01:00
Jon Atack
6e1ea4273e
test: refactor for walletcreatefundedpsbt fee rate coverage 2020-10-29 00:22:00 +01:00
Jon Atack
3ac7b0c6f1
wallet: fundrawtx fee rate coverage, fixup ParseConfirmTarget() 2020-10-29 00:21:57 +01:00
John Newbery
778cd0d88d [tests] Remove getnettotals/getpeerinfo consistency test
We make no guarantees about consistency between RPC calls.
2020-10-28 10:18:09 +00:00
Amiti Uttarwar
47ff5098ad [test] Clarify setup of node topology.
Since the test framework automatically sets up a connection between the nodes,
the second connect_nodes call was a no-op. Remove the redundant call & add
comments to explain the expected topology.
2020-10-27 19:23:02 -07:00
Amiti Uttarwar
0672522aed [move-only, test]: Match test order with run order 2020-10-27 13:45:25 -07:00
Jon Atack
2d8eba8f84
wallet: combine redundant bumpfee invalid params and args tests
might be best reviewed with:

git show COMMIT_HASH -w --color-moved=dimmed-zebra
2020-10-27 21:33:43 +01:00
Jon Atack
1697a40b6f
wallet: improve bumpfee error/help, add explicit fee rate coverage 2020-10-27 21:33:37 +01:00
Troy Giorshev
d419fdedbe [net processing] Don't add AlreadyHave txs to recentRejects
Now, we only add a transaction to our recentRejects filter if we didn't
already have it, meaning that it is added at most once, as intended.
2020-10-27 06:07:41 -04:00
Wladimir J. van der Laan
83363f7b62
Merge #20167: test: Add test for -blockversion
fa9b48549c test: Add test for -blockversion (MarcoFalke)
fa7fb0e442 test: Default blockversion to 4 in feature_block (MarcoFalke)
fa2b778d0c test: Remove unused -blockversion from tests (MarcoFalke)

Pull request description:

  `-blockversion` is currently untested, as in: The setting could be made a no-op without any tests failing. Fix that by adding an explicit test for it. Also, related minor cleanups.

ACKs for top commit:
  guggero:
    ACK fa9b48549c.

Tree-SHA512: 1b2e792f7ed0ec1db163476ee8a938f8f7cb3691f797c721bbe55fdeed92487c2ff83b55467440096917999406c86430cb3a615383cefb4f621828309ff6a1e7
2020-10-27 10:57:57 +01:00
MarcoFalke
fa9b48549c
test: Add test for -blockversion 2020-10-26 16:31:25 +01:00
Jon Atack
fc5721723d
wallet: fix SetFeeEstimateMode() error message
to clarify for the user the confusing error message that the missing fee rate
needs to be set in the conf_target param/option.
2020-10-25 00:35:38 +02:00
Jon Atack
052427eef1
wallet, bugfix: fix bumpfee with explicit fee rate modes 2020-10-24 22:03:11 +02:00
MarcoFalke
88271184e8
Merge #20112: test: Speed up wallet_resendwallettransactions with mockscheduler RPC
fa299ac273 test: Speed up wallet_resendwallettransactions test with mockscheduler RPC (MarcoFalke)

Pull request description:

  Also fixes #20143

ACKs for top commit:
  guggero:
    ACK fa299ac2

Tree-SHA512: 024ced4aa5f5c266e24fd0583d47b45b19c2a6ae25a06fabeacaa0ac996eec0c45f11cc34b2df17d01759b78ed31a991aa86978aafcc76cb0017382f601bf85a
2020-10-22 14:18:06 +02:00
MarcoFalke
7012db2a6b
Merge #20176: test: Fix intermittent issue in p2p_feefilter
fa5a91a352 test: Fix typo (one tx is enough) in p2p_feefilter (MarcoFalke)
fa3af2c0d3 test: Fix intermittent issue in p2p_feefilter (MarcoFalke)

Pull request description:

  Fixes:

  ```
  Traceback (most recent call last):
    File "test/functional/test_framework/test_framework.py", line 126, in main
      self.run_test()
    File "test/functional/p2p_feefilter.py", line 63, in run_test
      self.test_feefilter()
    File "test/functional/p2p_feefilter.py", line 117, in test_feefilter
      txids = [miniwallet.send_self_transfer(fee_rate=Decimal('0.00020000'), from_node=node1)['wtxid'] for _ in range(3)]
    File "test/functional/p2p_feefilter.py", line 117, in <listcomp>
      txids = [miniwallet.send_self_transfer(fee_rate=Decimal('0.00020000'), from_node=node1)['wtxid'] for _ in range(3)]
    File "test/functional/test_framework/wallet.py", line 63, in send_self_transfer
      txid = from_node.sendrawtransaction(tx_hex)
    File "test/functional/test_framework/coverage.py", line 47, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "test/functional/test_framework/authproxy.py", line 146, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: bad-txns-inputs-missingorspent (-25)

ACKs for top commit:
  guggero:
    ACK fa5a91a3

Tree-SHA512: 51d885753f72e1c91c4580709c15bdab60ff8c9d6f9bcb6db78a560e7e4dd7f76ce23add3303b374174afa3f11f74aa61db189a90c68d7f7655b15e64f51ed96
2020-10-22 11:02:33 +02:00
MarcoFalke
1cb4e339f9
Merge #20039: test: Convert amounts from float to decimal
5aadd4be18 Convert amounts from float to decimal (Prayank)

Pull request description:

  > decimal is preferred in accounting applications

  https://docs.python.org/3.8/library/decimal.html

  Decimal type saves an exact value so better than using float.

  ~~3 variables declared with type as 'Decimal' in [test/functional/mempool_accept.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/mempool_accept.py): fee, fee_expected, output_amount~~

  ~~Not required to convert to string anymore for using the above variables as decimal~~

  + fee, fee_expected, output_amount
  ~~+ 8 decimal places~~
  + Using value of coin['amount'] as decimal and removed 'int'
  + Removed unnecessary parentheses
  + Remove str() and use quotes

  Fixes https://github.com/bitcoin/bitcoin/issues/20011

ACKs for top commit:
  guggero:
    ACK 5aadd4be18

Tree-SHA512: 5877cf3837e5b65bec0fc8909de141a720bfa02a747513e21d20f3c41ec0cfecc524d2c347a96596b0a1a97900da2acf08b799f26b11d537e4dcddc6ce45f38e
2020-10-22 10:13:44 +02:00
MarcoFalke
fa5f46600f
test: Fix rpc_net intermittent issue 2020-10-21 21:07:28 +02:00
Prayank
5aadd4be18 Convert amounts from float to decimal
+ fee, fee_expected, output_amount
+ Using value of coin['amount'] as decimal and removed 'int'
+ Removed unnecessary parentheses
+ Remove str() and use quotes
2020-10-21 21:21:39 +05:30
Sebastian Falbesoner
6b56c1f4d0 test: remove last_{block,header}_equals() in p2p_fingerprint.py
Testing that requests to very old blocks / block headers fail can simply be
done by checking that the node doesn't respond with any "blocks" / "headers"
message at all.
Also removes unnecessary sending of block/header requests and replaces
time.sleep(3) with node0.sync_with_ping().
2020-10-21 15:31:24 +02:00
Russell Yanofsky
01476a88a6 wallet: Make -wallet setting not create wallets
This changes -wallet setting to only load existing wallets, not create new ones.

- Fixes settings.json corner cases reported by sjors & promag:
  https://github.com/bitcoin-core/gui/issues/95,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

- Prevents accidental creation of wallets reported most recently by jb55
  http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

- Simplifies behavior after #15454. #15454 took the big step of disabling
  creation of the default wallet. This PR extends it to avoid creating other
  wallets as well. With this change, new wallets just aren't created on
  startup, instead of sometimes being created, sometimes not. #15454 release
  notes are updated here and are simpler.

This change should be targeted for 0.21.0. It's a bug fix and simplifies
behavior of the #15937 / #19754 / #15454 features added in 0.21.0.
2020-10-21 08:48:43 -04:00
MarcoFalke
b46f37ba5e
Merge #20198: Show name, format and if uses descriptors in bitcoin-wallet tool
fa4074b395 Show name, format and if uses descriptors in bitcoin-wallet tool (Jonas Schnelli)

Pull request description:

ACKs for top commit:
  MarcoFalke:
    ACK fa4074b395
  jonatack:
    re-ACK fa4074b395

Tree-SHA512: cf6ee96ff21532fc4b0ba7a0fdfdc1fa485c9b1495447350fe65cd0bd919e0e0280613933265cdee069b8c29ccf015ac374535a70cac3d4fb89f4d08b3a03519
2020-10-21 14:48:43 +02:00
Jonas Schnelli
fa4074b395 Show name, format and if uses descriptors in bitcoin-wallet tool 2020-10-21 13:28:15 +02:00
Elliott Jin
3c7d9ab8c8 test: Move (dis)?connect_nodes globals into TestFramework as helpers 2020-10-20 00:43:00 -07:00
Prayank
4b16c61461 scripted-diff: test: Replace uses of (dis)?connect_nodes global
-BEGIN VERIFY SCRIPT-

 # max-depth=0 excludes test/functional/test_framework/...
 FILES=$(git grep -l --max-depth 0 "connect_nodes" test/functional)

 # Replace (dis)?connect_nodes(self.nodes[a], b) with self.(dis)?connect_nodes(a, b)
 sed -i 's/\b\(dis\)\?connect_nodes(self\.nodes\[\(.*\)\]/self.\1connect_nodes(\2/g' $FILES

 # Remove imports in the middle of a line
 sed -i 's/\(dis\)\?connect_nodes, //g' $FILES
 sed -i 's/, \(dis\)\?connect_nodes//g' $FILES

 # Remove imports on a line by themselves
 sed -i '/^\s*\(dis\)\?connect_nodes,\?$/d' $FILES
 sed -i '/^from test_framework\.util import connect_nodes$/d' $FILES

-END VERIFY SCRIPT-

Co-authored-by: Elliott Jin <elliott.jin@gmail.com>
2020-10-20 00:42:00 -07:00
Elliott Jin
be386840d4 test: Replace use of (dis)?connect_nodes globals
A later scripted-diff commit replaces the majority of uses, which all
follow this pattern:

    (dis)?connect_nodes(self.nodes[a], b)

This commit replaces the few "special cases".
2020-10-20 00:27:45 -07:00
Samuel Dobson
f5bd46a4cc
Merge #20125: rpc, wallet: Expose database format in getwalletinfo
624bab00dd test: add coverage for getwalletinfo format field (Jon Atack)
5e737a0092 rpc, wallet: Expose database format in getwalletinfo (João Barbosa)

Pull request description:

  Support for sqlite based wallets was added in #19077. This PR adds the `format` key in `getwalletinfo` response, that can be `bdb` or  `sqlite`.

ACKs for top commit:
  jonatack:
    Tested ACK 624bab00dd
  laanwj:
    Code review ACK 624bab00dd.
  MarcoFalke:
    doesn't hurt ACK 624bab00dd
  hebasto:
    ACK 624bab00dd, tested on Linux Mint 20 (x86_64).
  meshcollider:
    utACK 624bab00dd

Tree-SHA512: a81f8530f040f6381d33e073a65f281993eccfa717424ab6e651c1203cbaf27794dcb7175570459e7fdaa211565bc060d0a3ecbe70d2b6f9c49b8d5071e4441c
2020-10-20 12:35:33 +13:00
Pieter Wuille
812baaa1f8 Switch to BIP341's suggested scheme for outputs without script 2020-10-19 13:50:36 -07:00
MarcoFalke
4769942d90
Merge #19624: Warn on unknown rw_settings
fa48405ef8 Warn on unknown rw_settings (MarcoFalke)

Pull request description:

  Log a warning to debug log if unknown settings are encountered. This should probably only ever happen when the software is upgraded.

  Something similar is already done for the command line and config file. See:

  * test: Add test for unknown args #16234 (commit fa7dd88b71)

ACKs for top commit:
  ryanofsky:
    Code review ACK fa48405ef8. Looks good and I could see this being helpful for debugging. Thanks for taking suggestions

Tree-SHA512: cec7d88adf84fa0a842f56b26245157736eb50df433db951e622ea07fd145b899822b24cdab1d8b36c066415ce4f0ef09b493fa8a8d691532822a59c573aafa7
2020-10-19 11:30:49 +02:00
MarcoFalke
fa5a91a352
test: Fix typo (one tx is enough) in p2p_feefilter 2020-10-19 09:42:40 +02:00
MarcoFalke
faab86f6c8
test: Fix intermittent issue in wallet_send 2020-10-18 11:02:59 +02:00
MarcoFalke
faca3734c0
test: Fix intermittent issue in wallet_import_rescan 2020-10-18 10:10:20 +02:00
MarcoFalke
80c8a02f1b
Merge #20159: test: mining_getblocktemplate_longpoll.py improvements (use MiniWallet, add logging)
b128b56672 test: add logging for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner)
8ee3536b2b test: remove unused helpers random_transaction(), make_change() and gather_inputs() (Sebastian Falbesoner)
fddce7e199 test: use MiniWallet for mining_getblocktemplate_longpoll.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (mining_getblocktemplate_longpoll.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. Also adds missing log messages for the subtests.

  This was the only functional test that used the `random_transaction` helper in `test_framework/util.py`, hence it is removed, together with other helpers (`make_change` and `gather_inputs`) that were again only used by `random_transaction`.

ACKs for top commit:
  MarcoFalke:
    ACK b128b56672

Tree-SHA512: 09a5fa7b0f5976a47040f7027236d7ec0426d5a4829a082221c4b5fae294470230e89ae3df0bca0eea26833162c03980517f5cc88761ad251c3df4c4a49bca46
2020-10-17 17:57:23 +02:00
MarcoFalke
fa3af2c0d3
test: Fix intermittent issue in p2p_feefilter 2020-10-17 17:48:03 +02:00
Sebastian Falbesoner
b128b56672 test: add logging for mining_getblocktemplate_longpoll.py 2020-10-16 15:41:00 +02:00
Sebastian Falbesoner
8ee3536b2b test: remove unused helpers random_transaction(), make_change() and gather_inputs() 2020-10-16 15:40:54 +02:00
MarcoFalke
fa7fb0e442
test: Default blockversion to 4 in feature_block
There is one tests that checks version=1 blocks are rejected. For all
other tests the version doesn't matter as long as it is large enough.
2020-10-16 13:28:17 +02:00
Sebastian Falbesoner
fddce7e199 test: use MiniWallet for mining_getblocktemplate_longpoll.py
This test can now be run even with the Bitcoin Core wallet disabled.
2020-10-16 13:05:31 +02:00
MarcoFalke
fa2b778d0c
test: Remove unused -blockversion from tests 2020-10-16 13:01:07 +02:00
MarcoFalke
cb21d864c5
Merge #19401: QA: Use GBT to get block versions correct
d438d609cd QA: Use GBT to get block versions correct (Luke Dashjr)
1df2cd1c8f QA: blocktools: Accept block template to create_block (Luke Dashjr)

Pull request description:

  The goal here is to decouple unrelated tests from the details of block versions.

  Currently, these tests are forcing specific versions of blocks for no real reason.

ACKs for top commit:
  fjahr:
    re-ACK d438d609cd
  benthecarman:
    ACK d438d60

Tree-SHA512: 523b1cd4dac8d65c88432e126ce7f60df96ca4b94f7ecc8e83ba4ffbade23e2afe7055fdf586ce3c195a533f2004e63fff83add4267b39473a581c9f1c6d5340
2020-10-16 11:43:21 +02:00
Pieter Wuille
1d22300b99 Address functional test nits 2020-10-15 15:39:09 -07:00
Jon Atack
7b5bd3102e
test: add getnetworkinfo network name regression tests 2020-10-15 19:21:43 +02:00
Wladimir J. van der Laan
0d22482353
Merge #20002: net, rpc, cli: expose peer network in getpeerinfo; simplify/improve -netinfo
6272604bef refactor: enable -netinfo to add future networks (i2p, cjdns) (Jon Atack)
82fd40216c refactor: promote some -netinfo localvars to class members (Jon Atack)
5133fab37e cli: simplify -netinfo using getpeerinfo network field (Jon Atack)
4938a109ad rpc, test: expose CNodeStats network in RPC getpeerinfo (Jon Atack)
6df7882029 net: add peer network to CNodeStats (Jon Atack)

Pull request description:

  This PR:

  - builds on #19991 and #19998
  - exposes peer networks via a new getpeerinfo `network` field ("ipv4", "ipv6", or "onion"), and adds functional tests
  - updates -netinfo to use getpeerinfo `network` rather than detecting the peer networks client-side
  - refactors -netinfo to easily add future networks

ACKs for top commit:
  laanwj:
    ACK 6272604bef

Tree-SHA512: 28883487585135ceaaf84ce09131f2336e3193407f2e3df0960e3f4ac340f500ab94ffecb9d06a4c49bc05e3cca4f914ea4379860bea0bd5df2f834f74616015
2020-10-15 17:44:38 +02:00
Jon Atack
624bab00dd
test: add coverage for getwalletinfo format field 2020-10-15 16:46:34 +02:00
Ivan Metlushko
538be4219a wallet: fix importdescriptor silent fail 2020-10-15 18:02:58 +07:00
MarcoFalke
560dea9b26
Merge #19770: RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions")
5b57dc5458 RPC: getpeerinfo: Wrap long help line for bytesrecv_per_msg (Luke Dashjr)
d681a28219 RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") (Luke Dashjr)

Pull request description:

  If we were going to continue support for "whitelisted", we should have probably made it true if any permission flag was set, rather than only if "default permissions" were used.

  This corrects the description, and deprecates it.

ACKs for top commit:
  laanwj:
    ACK 5b57dc5458

Tree-SHA512: a2e2137f8be8110357c1b2fef2c923fa8c7c4a49b0b2b3a2d78aedf12f8ed5cc7e140018a21b37e6ec7770ed4007542aeef7ad4558973901b107e8e0f81d6003
2020-10-15 11:49:34 +02:00
Wladimir J. van der Laan
3caee16946
Merge #19953: Implement BIP 340-342 validation (Schnorr/taproot/tapscript)
0e2a5e448f tests: dumping and minimizing of script assets data (Pieter Wuille)
4567ba034c tests: add generic qa-asset-based script verification unit test (Pieter Wuille)
f06e6d0345 tests: functional tests for Schnorr/Taproot/Tapscript (Pieter Wuille)
3c226639eb tests: add BIP340 Schnorr signature support to test framework (Pieter Wuille)
206fb180ec --- [TAPROOT] Tests --- (Pieter Wuille)
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342) (Pieter Wuille)
e9a021d7e6 Make Taproot spends standard + policy limits (Pieter Wuille)
865d2c37e2 --- [TAPROOT] Regtest activation and policy --- (Pieter Wuille)
72422ce396 Implement Tapscript script validation rules (BIP 342) (Johnson Lau)
330de894a9 Use ScriptExecutionData to pass through annex hash (Pieter Wuille)
8bbed4b7ac Implement Taproot validation (BIP 341) (Pieter Wuille)
0664f5fe1f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340) (Pieter Wuille)
5de246ca81 Implement Taproot signature hashing (BIP 341) (Johnson Lau)
9eb590894f Add TaggedHash function (BIP 340) (Pieter Wuille)
450d2b2371 --- [TAPROOT] BIP340/341/342 consensus rules --- (Pieter Wuille)
5d62e3a68b refactor: keep spent outputs in PrecomputedTransactionData (Pieter Wuille)
8bd2b4e784 refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script (Pieter Wuille)
107b57df9f scripted-diff: put ECDSA in name of signature functions (Pieter Wuille)
f8c099e220 --- [TAPROOT] Refactors --- (Pieter Wuille)

Pull request description:

  This is an implementation of the Schnorr/taproot consensus rules proposed by BIPs [340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), [341](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), and [342](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki).

  See the list of commits [below](https://github.com/bitcoin/bitcoin/pull/19953#issuecomment-691815830). No signing or wallet support of any kind is included, as testing is done entirely through the Python test framework.

  This is a successor to https://github.com/bitcoin/bitcoin/pull/17977 (see discussion following [this comment](https://github.com/bitcoin/bitcoin/pull/17977#issuecomment-682285983)), and will have further changes squashed/rebased. The history of this PR can be found in #19997.

ACKs for top commit:
  instagibbs:
    reACK 0e2a5e448f
  benthecarman:
    reACK 0e2a5e4
  kallewoof:
    reACK 0e2a5e448f
  jonasnick:
    ACK 0e2a5e448f almost only looked at bip340/libsecp related code
  jonatack:
    ACK 0e2a5e448f modulo the last four commits (tests) that I plan to finish reviewing tomorrow
  fjahr:
    reACK 0e2a5e448f
  achow101:
    ACK 0e2a5e448f

Tree-SHA512: 1b00314450a2938a22bccbb4e177230cf08bd365d72055f9d526891f334b364c997e260c10bc19ca78440b6767712c9feea7faad9a1045dd51a5b96f7ca8146e
2020-10-15 10:22:35 +02:00
Samuel Dobson
8ed37f6c84
Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04 Run dumpwallet for legacy wallets only in  wallet_backup.py (Andrew Chow)
6c6639ac9f Include sqlite3 in documentation (Andrew Chow)
f023b7cac0 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866 Set and check the sqlite user version (Andrew Chow)
9d3d2d263c Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8e walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225 Determine wallet file type based on file magic (Andrew Chow)
6045f77003 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4e Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7 Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2 Add SetupSQLStatements (Andrew Chow)
6636a2608a Implement SQLiteBatch::Close (Andrew Chow)
93825352a3 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372b Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe125 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df82580 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e Add libsqlite3 (Andrew Chow)

Pull request description:

  This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.

  For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

  We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.

  I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.

ACKs for top commit:
  Sjors:
    re-utACK c4a29d0a90
  promag:
    Tested ACK c4a29d0a90.
  fjahr:
    reACK c4a29d0a90
  S3RK:
    Re-review ACK c4a29d0a90
  meshcollider:
    re-utACK c4a29d0a90
  hebasto:
    re-ACK c4a29d0a90, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
  ryanofsky:
    Code review ACK c4a29d0a90. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
  jonatack:
    ACK c4a29d0a90, debug builds and test runs after rebase to latest master @ c2c4dbaebd, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.

Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
2020-10-15 20:12:29 +13:00
Wladimir J. van der Laan
c2c4dbaebd
Merge #19988: Overhaul transaction request logic
fd9a0060f0 Report and verify expirations (Pieter Wuille)
86f50ed10f Delete limitedmap as it is unused now (Pieter Wuille)
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents (Pieter Wuille)
173a1d2d3f Expedite removal of tx requests that are no longer needed (Pieter Wuille)
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers (Pieter Wuille)
242d16477d Change transaction request logic to use txrequest (Pieter Wuille)
5b03121d60 Add txrequest fuzz tests (Pieter Wuille)
3c7fe0e5a0 Add txrequest unit tests (Pieter Wuille)
da3b8fde03 Add txrequest module (Pieter Wuille)

Pull request description:

  This replaces the transaction request logic with an encapsulated class that maintains all the state surrounding it. By keeping it stand alone, it can be easily tested (using included unit tests and fuzz tests).

  The major changes are:

  * Announcements from outbound (and whitelisted) peers are now always preferred over those from inbound peers. This used to be the case for the first request (by delaying the first request from inbound peers), and a bias afters. The 2s delay for requests from inbound peers still exists, but after that, if viable outbound peers remain for any given transaction, they will always be tried first.
  * No more hard cap of 100 in flight transactions per peer, as there is less need for it (memory usage is linear in the number of announcements, but independent from the number in flight, and CPU usage isn't affected by it). Furthermore, if only one peer announces a transaction, and it has over 100 in flight already, we still want to request it from them. The cap is replaced with a rule that announcements from such overloaded peers get an additional 2s delay (possibly combined with the existing 2s delays for inbound connections, and for txid peers when wtxid peers are available).
  * The limit of 100000 tracked announcements is reduced to 5000; this was excessive. This can be bypassed using the PF_RELAY permission (to accommodate locally dumping a batch of many transactions).

  This replaces #19184, rebased on #18044 and with many small changes.

ACKs for top commit:
  ariard:
    Code Review ACK fd9a006. I've reviewed the new TxRequestTracker, its integration in net_processing, unit/functional/fuzzing test coverage. I looked more for soundness of new specification rather than functional consistency with old transaction request logic.
  MarcoFalke:
    Approach ACK fd9a0060f0 🏹
  naumenkogs:
    Code Review ACK fd9a006. I've reviewed everything, mostly to see how this stuff works at the lower level (less documentation-wise, more implementation-wise), and to try breaking it with unexpected sequences of events.
  jnewbery:
    utACK fd9a0060f0
  jonatack:
    WIP light ACK fd9a0060f0 have read the code, verified that each commit is hygienic, e.g. debug build clean and tests green, and have been running a node on and off with this branch and grepping the net debug log. Am still unpacking the discussion hidden by GitHub by fetching it via the API and connecting the dots, storing notes and suggestions in a local branch; at this point none are blockers.
  ryanofsky:
    Light code review ACK fd9a0060f0, looking at txrequest implementation, unit test implementation, and net_processing integration, just trying to understand how it works and looking for anything potentially confusing in the implementation. Didn't look at functional tests or catch up on review discussion. Just a sanity check review focused on:

Tree-SHA512: ea7b52710371498b59d9c9cfb5230dd544fe9c6cb699e69178dea641646104f38a0b5ec7f5f0dbf1eb579b7ec25a31ea420593eff3b7556433daf92d4b0f0dd7
2020-10-14 18:36:59 +02:00
Russell Yanofsky
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets 2020-10-14 11:28:18 -04:00
Andrew Chow
310b0fde04 Run dumpwallet for legacy wallets only in wallet_backup.py
Descriptor wallets don't support dumpwallet, so make the tests that do
dumpwallet legacy wallet only.
2020-10-14 11:28:18 -04:00
Luke Dashjr
d681a28219 RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") 2020-10-14 14:16:42 +00:00
Jon Atack
4938a109ad
rpc, test: expose CNodeStats network in RPC getpeerinfo 2020-10-14 14:56:03 +02:00
MarcoFalke
fa299ac273
test: Speed up wallet_resendwallettransactions test with mockscheduler RPC 2020-10-13 15:38:18 +02:00
MarcoFalke
cd6e193d4c
Merge #20126: test: p2p_leak_tx.py improvements (use MiniWallet, add p2p_lock acquires)
5b77f8098d test: add p2p_lock acquires in p2p_leak_tx.py (Sebastian Falbesoner)
cc8c6823b4 test: use MiniWallet for p2p_leak_tx.py (Sebastian Falbesoner)

Pull request description:

  This PR enables one more of the non-wallet functional tests (p2p_leak_tx.py) to be run even with the Bitcoin Core wallet disabled by using the new MiniWallet instead, as proposed in #20078. It also adds missing p2p_lock acquires that need to be held while modifying internal p2p Interface state (in this case the `last_message` dictionary) to avoid data races.

ACKs for top commit:
  laanwj:
    Code review ACK 5b77f8098d

Tree-SHA512: 6661bc6e3491a9af4bf040f379e5955c525136397e99d3eadde92e247580d0d87efff750e6d3b1f6d9a4e578144a433a982f574ef056b44dd6bca33873a1bae6
2020-10-13 14:27:30 +02:00
Pieter Wuille
0e2a5e448f tests: dumping and minimizing of script assets data
This adds a --dumptests flag to the feature_taproot.py test, to dump all its
generated test cases to files, in a format compatible with the
script_assets_test unit test. A fuzzer for said format is added as well, whose
primary purpose is coverage-based minimization of those dumps.
2020-10-12 17:18:47 -07:00
Pieter Wuille
f06e6d0345 tests: functional tests for Schnorr/Taproot/Tapscript
A large functional test is added that automatically generates random transactions which
exercise various aspects of the new rules, and verifies they are accepted into the mempool
(when appropriate), and correctly accepted/rejected in (Python-constructed) blocks.

Includes sighashing code and many tests by Johnson Lau.
Includes a test by Matthew Zipkin.
Includes several tests and improvements by Greg Sanders.
2020-10-12 17:18:47 -07:00
Pieter Wuille
3c226639eb tests: add BIP340 Schnorr signature support to test framework
Add a pure Python implementation of BIP340 signing and verification, tested against
the BIP's test vectors.
2020-10-12 17:18:47 -07:00
Pieter Wuille
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342)
Define a versionbits-based activation for the new consensus rules on regtest.
No activation or activation mechanism is defined for testnet or mainnet.
2020-10-12 17:18:47 -07:00
Pieter Wuille
e9a021d7e6 Make Taproot spends standard + policy limits
This adds a `TxoutType::WITNESS_V1_TAPROOT` for P2TR outputs, and permits spending
them in standardness rules. No corresponding `CTxDestination` is added for it,
as that isn't needed until we want wallet integration. The taproot validation flags
are also enabled for mempool transactions, and standardness rules are added
(stack item size limit, no annexes).
2020-10-12 17:18:47 -07:00
Pieter Wuille
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers
Maintaining up to 100000 INVs per peer is excessive, as that is far more
than fits in a typical mempool.

Also disable the "overload" penalty for PF_RELAY peers.
2020-10-12 12:14:53 -07:00
Pieter Wuille
242d16477d Change transaction request logic to use txrequest
This removes most transaction request logic from net_processing, and
replaces it with calls to a global TxRequestTracker object.

The major changes are:

* Announcements from outbound (and whitelisted) peers are now always
  preferred over those from inbound peers. This used to be the case for the
  first request (by delaying the first request from inbound peers), and
  a bias afters. The 2s delay for requests from inbound peers still exists,
  but after that, if viable outbound peers remain for any given transaction,
  they will always be tried first.

* No more hard cap of 100 in flight transactions per peer, as there is less
  need for it (memory usage is linear in the number of announcements, but
  independent from the number in flight, and CPU usage isn't affected by it).
  Furthermore, if only one peer announces a transaction, and it has over 100
  in flight and requestable already, we still want to request it from them.
  The cap is replaced with an additional 2s delay (possibly combined with the
  existing 2s delays for inbound connections, and for txid peers when wtxid
  peers are available).

Includes functional tests written by Marco Falke and Antoine Riard.
2020-10-12 12:14:11 -07:00
Sebastian Falbesoner
5b77f8098d test: add p2p_lock acquires in p2p_leak_tx.py 2020-10-12 00:22:16 +02:00
Sebastian Falbesoner
cc8c6823b4 test: use MiniWallet for p2p_leak_tx.py
This test can now be run even with the Bitcoin Core wallet disabled.
2020-10-11 23:57:56 +02:00
fanquake
0b2abaa666
Merge #19954: Complete the BIP155 implementation and upgrade to TORv3
dcf0cb4776 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d9 net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fd Support bypassing range check in ReadCompactSize (Pieter Wuille)

Pull request description:

  This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:

  `net: CAddress & CAddrMan: (un)serialize as ADDRv2`
  `net: advertise support for ADDRv2 via new message`

  plus one more commit:

  `tor: make a TORv3 hidden service instead of TORv2`

ACKs for top commit:
  jonatack:
    re-ACK dcf0cb4776 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
  sipa:
    ACK dcf0cb4776
  hebasto:
    re-ACK dcf0cb4776, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
  laanwj:
    Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb4776 merged on master (12a1c3ad1a).
  ariard:
    Code Review ACK dcf0cb4

Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
2020-10-11 08:51:57 +08:00
Elliott Jin
66d012ad7f test: RPC: getblock fee calculations 2020-10-09 08:50:49 -07:00
Vasil Dimov
353a3fdaad
net: advertise support for ADDRv2 via new message
Introduce a new message `sendaddrv2` to signal support for ADDRv2.
Send the new message immediately after sending the `VERACK` message.

Add support for receiving and parsing ADDRv2 messages.

Send ADDRv2 messages (instead of ADDR) to a peer if he has
advertised support for it.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:50 +02:00
Andrew Chow
de6b389d5d tests: Test getaddressinfo parent_desc 2020-10-09 09:04:18 -04:00
MarcoFalke
392c6f4fb2
Merge #20101: rpc: change no wallet loaded message to be clearer
907f142fc7 rpc: change no wallet loaded message to be clearer (Andrew Chow)

Pull request description:

  Changes the no wallet is loaded rpc error message to be clearer that no wallet is loaded and how the user can load or create a wallet. Also changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as that makes more sense.

ACKs for top commit:
  MarcoFalke:
    review ACK 907f142fc7
  kristapsk:
    ACK 907f142fc7. In addition to standard tests, just in case tested that this doesn't break anything with JoinMarket.
  meshcollider:
    utACK 907f142fc7

Tree-SHA512: 4b413e6ab5430ec75a79de9db6583f2f3f38ccdf71aa373d8386a56e64f07f92200c8107c8c82c92c7c431d739615977c208b771a24c5960fa8676789b5497a2
2020-10-08 15:07:23 +02:00
MarcoFalke
fa6af31227
test: Document why syncwithvalidationinterfacequeue is needed in tests 2020-10-08 14:23:51 +02:00
MarcoFalke
fa135a13b8
Revert "test: Add missing sync_all to wallet_balance test"
This reverts commit fa815255c7.

The underlying bug has been fixed in commit f77b1de16f.
2020-10-08 14:23:37 +02:00
Pieter Wuille
ec3916f40a Use mockable time everywhere in net_processing 2020-10-08 01:04:29 -07:00
Andrew Chow
907f142fc7 rpc: change no wallet loaded message to be clearer
Changes the no wallet is loaded rpc error message to be clearer that no
wallet is loaded and how the user can load or create a wallet. Also
changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as
that makes more sense.
2020-10-07 21:36:44 -04:00
fanquake
db88db4727
Merge #19339: validation: re-delegate absurd fee checking from mempool to clients
b048b275d9 [validation] Remove absurdfee from accepttomempool (John Newbery)
932564b9cf scripted-diff: update max-fee-exceeded error message to include RPC (gzhao408)
8f1290c601 [rpc/node] check for high fee before ATMP in clients (gzhao408)

Pull request description:

  Picked up from #15810. Add separate fee-checking logic for clients that need to enforce max fee rates, then remove the `absurdFee` logic from ATMP.

  ATMP's `nAbsurdFee` argument is used to enforce user-specific behavior (it is not policy since it isn't applied consistently: it is only ever used in RPC and wallet, and set to 0 everywhere else internally). It should be removed from `AcceptToMemoryPool` because (1) validation results/mempool behavior should not be user-specific and (2) enforcing a max fee rate should be the responsibility of the client instead of the mempool.

  Note: this PR does not intend to _remove_ protection from high fees, just re-delegate the responsibility to clients.

ACKs for top commit:
  jnewbery:
    utACK b048b275d9
  LarryRuane:
    re-ACK b048b275d9
  MarcoFalke:
    re-ACK b048b275d9 , only change is squashing one commit 🏦
  instagibbs:
    utACK b048b275d9

Tree-SHA512: 57c17ba16d230a4cae2896dd6a64c924f307757824e35784bf96da7b10aff2d8ea910710edf35e981035623a155f8766209a92a0fdb856549fde78bc3eaae4d2
2020-10-07 10:58:30 +08:00
gzhao408
932564b9cf scripted-diff: update max-fee-exceeded error message to include RPC
-BEGIN VERIFY SCRIPT-
sed -i 's/Fee exceeds maximum configured by \-\maxtxfee/Fee exceeds maximum configured by user (e.g. -maxtxfee, maxfeerate)/g' src/util/error.cpp test/functional/rpc_fundrawtransaction.py test/functional/rpc_psbt.py test/functional/rpc_rawtransaction.py test/functional/wallet_bumpfee.py test/functional/wallet_create_tx.py
-END VERIFY SCRIPT-
2020-10-05 04:55:01 -07:00
gzhao408
8f1290c601 [rpc/node] check for high fee before ATMP in clients
Check absurd fee in BroadcastTransaction and RPC,
return TransactionError::MAX_FEE_EXCEEDED instead
of TxValidationResult::TX_NOT_STANDARD because this
is client preference, not a node-wide policy.
2020-10-05 04:54:05 -07:00
MarcoFalke
875e1ccc9f
Merge #19947: Test: Cover "change_type" option of "walletcreatefundedpsbt" RPC
a56e9f5670 test: Assert exclusive PSBT funding options (Oliver Gugger)
64bc5efd39 test: Assert PSBT change type (Oliver Gugger)

Pull request description:

  Increases test coverage of the `walletcreatefundedpsbt` RPC.

  Tests the following combinations:
   - Make sure the global option `-changetype` is used as the default value for the `change_type` option if not specified.
   - Make sure the global option `-changetype` can be overwritten by explicitly setting the `change_type` option of the `walletcreatefundedpsbt` RPC call.
   - Make sure the options `change_type` and `changeAddress` are mutually exclusive.

ACKs for top commit:
  achow101:
    ACK a56e9f5670

Tree-SHA512: bf0fb20c890887b7228ad9277fdb32f367ba772eed6efbe2b4f471f808d4d435110256601e8ebd9bea57026d9f22f3cc3c26a009b017e3da6d8fc6896313def5
2020-10-05 09:45:11 +02:00
fanquake
54fc96ffa7
Merge #19956: rpc: Improve invalid vout value rpc error message
f471a3be00 scripted diff: Improve invalid vout value rpc error message (Nima Yazdanmehr)

Pull request description:

  Since the `vout` value can start at `0`, the error message for *negative* values can be improved to something like: `vout cannot be negative`.

ACKs for top commit:
  fanquake:
    ACK f471a3be00
  promag:
    Code review ACK f471a3be00.

Tree-SHA512: fbdee3d0ddd5b58eb93934a1217b44e125a9ad39e672b1f35c7609c6c5fcf45ae1b731d3d6135b7225d98792dbfc34a50907b8c41274a5b029d7b5c59f886560
2020-10-03 11:13:21 +08:00
Oliver Gugger
a56e9f5670
test: Assert exclusive PSBT funding options
Make sure the options "change_type" and "changeAddress" of the
walletcreatefundedpsbt RPC cannot both be specified a the same time.
2020-10-02 17:15:52 +02:00
MarcoFalke
171cd05ae3
Merge #20034: test: Get rid of default wallet hacks
c1585bca8d test: Get rid of default wallet hacks (Russell Yanofsky)
ed3acda33b test, refactor: add default_wallet_name and wallet_data_filename variables (Russell Yanofsky)

Pull request description:

  Changes:

  - Get rid of setup_nodes (`-wallet`, `-nowallet`, `-disablewallet`) argument rewriting
  - Get rid of hardcoded wallet `""` names and `-wallet=""` args

  Motivation:

  - Simplify test framework behavior so it's easier to write new tests without having arguments mangled by the framework
  - Make tests more readable, replacing unexplained `""` string literals with `default_wallet_name` references
  - Make it trivial to update default wallet name and wallet data filename for sqlite #19077 testing
  - Stop relying on `-wallet` arguments to create wallets, so it is easy to change `-wallet` option in the future to only load existing wallets not create new ones (to avoid accidental wallet creation, and encourage use of wallet encryption and descriptor features)

ACKs for top commit:
  MarcoFalke:
    ACK c1585bca8d, only effective change is adding documentation 🎵

Tree-SHA512: f62dec7cbdacb5f330aa0e1eec89ab4d065540d91495bbedcb375eda1c080b45ce9edb310ce253c44c4839f1b4cc2c7df9816c58402d5d43f94a437e301ea8bc
2020-10-02 17:07:35 +02:00
Oliver Gugger
64bc5efd39
test: Assert PSBT change type
Make sure the wallet's default change type is respected by default when
funding a PSBT but can be overwritten by the "change_type" option.
2020-10-02 15:48:45 +02:00
Wladimir J. van der Laan
a0185d90a7
Merge #18309: zmq: Add support to listen on multiple interfaces
e66870c5a4 zmq: Append address to notify log output (nthumann)
241803da21 test: Add zmq test to support multiple interfaces (nthumann)
a0b2e5cb6a doc: Add release notes to support multiple interfaces (nthumann)
b1c3f180ec doc: Adjust ZMQ usage to support multiple interfaces (nthumann)
347c94f551 zmq: Add support to listen on multiple interfaces (Nicolas Thumann)

Pull request description:

  This PR adds support for ZeroMQ to listen on multiple interfaces, just like the RPC server.
  Currently, if you specify more than one e.g. `zmqpubhashblock` paramter, only the first one will be used. Therefore a user may be forced to listen on all interfaces (e.g. `zmqpubhashblock=0.0.0.0:28332`), which can result in an increased attack surface.
  With this PR a user can specify multiple interfaces to listen on, e.g.
  `-zmqpubhashblock=tcp://127.0.0.1:28332 -zmqpubhashblock=tcp://192.168.1.123:28332`.

ACKs for top commit:
  laanwj:
    Code review ACK e66870c5a4
  instagibbs:
    reACK e66870c5a4

Tree-SHA512: f38ab4a6ff00dc821e5f4842508cefadb701e70bb3893992c1b32049be20247c8aa9476a1f886050c5f17fe7f2ce99ee30193ce2c81a7482a5a51f8fc22300c7
2020-10-01 17:43:34 +02:00
MarcoFalke
9fc2f011ba
Merge #20048: chainparams: do not log signet startup messages for other chains
6fccad7f71 signet: do not log signet startup messages for other chains (Jon Atack)

Pull request description:

  The following signet startup messages are printed to the debug log immediately on node startup for all chains. This behavior occurs on master as a side effect after the merge of #20014. This PR removes the first message and moves the signet derived magic logging to `init.cpp`.
  ```
  $ ./src/bitcoind
  2020-09-30T14:25:15Z Using default signet network
  2020-09-30T14:25:15Z Signet derived magic (message start): 0a03cf40
  2020-09-30T14:25:15Z Bitcoin Core version v0.20.99.0 ...
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 6fccad7f71
  kallewoof:
    utACK 6fccad7f71
  hebasto:
    ACK 6fccad7f71

Tree-SHA512: 33821dce89b24caf7b7c1ecb41e572ecfb26e6958a1316d359ff240e6ef97c4a1f2cf1b4b974596b252815f9df23960ce385c132ebdbc855bbe6123c3b0b003a
2020-10-01 13:29:05 +02:00
Jon Atack
6fccad7f71
signet: do not log signet startup messages for other chains
and move signet network magic logging from chainparams.cpp to init.cpp
2020-10-01 11:25:42 +02:00
MarcoFalke
40aab35e98
Merge #19253: Tests: tidy up address.py and segwit_addr.py
825fcae484 [tests] Replace bytes literals with hex literals (John Newbery)
64eca45100 [tests] Fix pep8 style violations in address.py (John Newbery)
b230f8b3f3 [tests] Correct docstring for address.py (John Newbery)
ea70e6a2ca [tests] Tidy up imports in address.py (John Newbery)
7f639df0b8 [tests] Remove unused optional verify_checksum parameter (John Newbery)
011e784f74 [tests] Rename segwit encode and decode functions (John Newbery)
e4557133f5 [tests] Move bech32 unit tests to test framework (John Newbery)

Pull request description:

  Lots of small fixes:

  - moving unit tests to test_framework implementation files
  - renaming functions to be clearer
  - removing multiple imports
  - removing unreadable byte literals from the code
  - fixing pep8 violations
  - correcting out-of-date docstring

ACKs for top commit:
  jonatack:
    re-ACK 825fcae484 per `git range-diff a0a422c 7edcdcd 825fcae` and verified `wallet_address_types.py` and `wallet_basic.py --descriptors` (the failure on one travis job) are green locally.
  MarcoFalke:
    ACK 825fcae484
  fanquake:
    ACK 825fcae484 - looks ok to me.

Tree-SHA512: aea509c27c1bcb94bef11205b6a79836c39c62249672815efc9822f411bc2e2336ceb3d72b3b861c3f4054a08e16edb28c6edd3aa5eff72eec1d60ea6ca82dc4
2020-10-01 09:43:11 +02:00
nthumann
241803da21
test: Add zmq test to support multiple interfaces 2020-10-01 00:33:38 +02:00
Nima Yazdanmehr
f471a3be00
scripted diff: Improve invalid vout value rpc error message
-BEGIN VERIFY SCRIPT-
r() { sed -i 's/vout must be positive/vout cannot be negative/g' $1 }
r $(git grep -l 'vout must be positive')
-END VERIFY SCRIPT-
2020-09-30 20:43:05 +03:30
Sebastian Falbesoner
136d96b71f test: use wait_for_{block,header} helpers in p2p_fingerprint.py 2020-09-30 16:48:04 +02:00
MarcoFalke
faf60dee34
doc: Remove double-whitespace from help string, other whitespace fixups 2020-09-30 09:28:33 +02:00
MarcoFalke
1769828684
Merge #19501: send* RPCs in the wallet returns the "fee reason"
69cf5d4eeb [test] Make sure send rpc returns fee reason (Sishir Giri)
d5863c0b3e [send] Make send RPCs return fee reason (Sishir Giri)

Pull request description:

  Whenever a wallet funds a transaction, the fee reason is reported to the user only if the verbose is set to true. I added an extra parameter to `CreateTransaction` function in wallet.cpp. Then I implemented the fee reason return logic in `SendMoney`  in rpcwallet.cpp, followed by verbose parameter in `sendtoaddress` and `sendmany` functions. I also added a fee reason test case in walletbasic.py.

  link to the issue: https://github.com/MarcoFalke/bitcoin-core/issues/22#issue-616251578

ACKs for top commit:
  instagibbs:
    ACK 69cf5d4eeb
  meshcollider:
    utACK 69cf5d4eeb

Tree-SHA512: 2e3af32dcfbd5511ba95f8bc8edca7acfe709a8430ff03e43172e5d0af3dfa4b2f57906978e7f272d878043b9ed8c6004674cf47d7496b005d5f612e9a58aa0e
2020-09-30 09:01:23 +02:00
Wladimir J. van der Laan
8aa6178961
Merge #20003: net: Exit with error message if -proxy is specified without arguments (instead of continuing without proxy server)
9b4fa0af40 net: Print error message if -proxy is specified without arguments (instead of continuing without proxy server) (practicalswift)

Pull request description:

  Exit with error message if `-proxy` is specified without arguments (instead of continuing without proxy server).

  Continuing without a proxy server when the end-user has specified `-proxy` may result in accidental loss of privacy. (The end-user might think he/she is using a proxy when he/she is not.)

  Before this patch:

  ```
  $ src/bitcoind -proxy
  …
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -listen=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -upnp=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -proxy set -> setting -discover=0
  2020-09-23T00:24:33Z InitParameterInteraction: parameter interaction: -listen=0 -> setting -listenonion=0
  …
  2020-09-23T00:24:33Z init message: Starting network threads...
  ```

  `bitcoind` is now running *without* a proxy server (`GetProxy(…, …) == false`, `HaveNameProxy() == false`, etc.).

  Note that the "-proxy set" log messages above which the end-user might interpret as "good, my traffic is now routed via the proxy".

  After this patch:

  ```
  $ src/bitcoind -proxy
  Error: No proxy server specified. Use -proxy=<ip> or -proxy=<ip:port>.
  $ echo $?
  1
  ```

ACKs for top commit:
  laanwj:
    re-ACK 9b4fa0af40
  kristapsk:
    ACK 9b4fa0af40, I have tested the code.
  hebasto:
    re-ACK 9b4fa0af40

Tree-SHA512: 4ba7a011991699a54b5bb87ec68367c681231bf5dcd36f8c89ff9ddc2e8d29df453817b7e362597e652ad6b341a22b7274be0fd78d435e5f0fd8058e5221c4ce
2020-09-29 15:17:28 +02:00
Russell Yanofsky
c1585bca8d test: Get rid of default wallet hacks
- Get rid of hardcoded wallet "" names and -wallet="" args
- Get rid of setup_nodes (-wallet, -nowallet, -disablewallet) argument rewriting

Motivation:

- Simplify test framework behavior so it's easier to write new tests without
  having arguments mangled by the framework
- Make tests more readable, replacing unexplained "" string literals with
  default_wallet_name references
- Make it trivial to update default wallet name and wallet data filename for
  sqlite #19077 testing
- Stop relying on -wallet arguments to create wallets, so it is easy to change
  -wallet option in the future to only load existing wallets not create new
  ones (to avoid accidental wallet creation, and encourage use of wallet
  encryption and descriptor features)
2020-09-29 04:35:01 -04:00
Russell Yanofsky
ed3acda33b test, refactor: add default_wallet_name and wallet_data_filename variables
No changes in behavior
2020-09-29 04:35:01 -04:00
fanquake
6af9b31bfc
Merge #19107: p2p: Move all header verification into the network layer, extend logging
deb52711a1 Remove header checks out of net_processing (Troy Giorshev)
52d4ae46ab Give V1TransportDeserializer CChainParams& member (Troy Giorshev)
5bceef6b12 Change CMessageHeader Constructor (Troy Giorshev)
1ca20c1af8 Add doxygen comment for ReceiveMsgBytes (Troy Giorshev)
890b1d7c2b Move checksum check from net_processing to net (Troy Giorshev)
2716647ebf Give V1TransportDeserializer an m_node_id member (Troy Giorshev)

Pull request description:

  Inspired by #15206 and #15197, this PR moves all message header verification from the message processing layer and into the network/transport layer.

  In the previous PRs there is a change in behavior, where we would disconnect from peers upon a single failed checksum check.  In various discussions there was concern over whether this was the right choice, and some expressed a desire to see how this would look if it was made to be a pure refactor.

  For more context, see https://bitcoincore.reviews/15206.html#l-81.

  This PR improves the separation between the p2p layers, helping improvements like [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and #18989.

ACKs for top commit:
  ryanofsky:
    Code review ACK deb52711a1 just rebase due to conflict on adjacent line
  jnewbery:
    Code review ACK deb52711a1.

Tree-SHA512: 1a3b7ae883b020cfee1bef968813e04df651ffdad9dd961a826bd80654f2c98676ce7f4721038a1b78d8790e4cebe8060419e3d8affc97ce2b9b4e4b72e6fa9f
2020-09-29 16:14:40 +08:00
fanquake
e36aa351a3
Merge #19969: Send RPC bug fix and touch-ups
f7b331ea85 rpc: add brackets to ConstructTransaction (Sjors Provoost)
d813d26f06 [rpc] send: various touch-ups (Sjors Provoost)
0fc1c685e1 [rpc] send: fix parsing replaceable option (Sjors Provoost)
efc9b85e6f Mark send RPC experimental (Sjors Provoost)

Pull request description:

  Followup based on #16378 nits. It also fixes an argument parsing error (uncaught because the test wasn't sufficiently thorough).

  I marked the RPC as experimental so we can tweak it a bit over the next release cycle.

ACKs for top commit:
  meshcollider:
    utACK f7b331ea85
  fjahr:
    utACK f7b331ea85
  kallewoof:
    ACK f7b331ea85

Tree-SHA512: 82dd8ac76a6558872db3f5249d4d6440469400aaa339153bc627d1ee673a91ecfadecb486bc1939ba87ebbd80e26ff29698e93e358599f3d26fde0e526892afe
2020-09-29 15:14:08 +08:00
Sebastian Falbesoner
343dc4760f test: add test for high-bandwidth mode states in getpeerinfo 2020-09-29 00:42:06 +02:00
Sishir Giri
69cf5d4eeb [test] Make sure send rpc returns fee reason 2020-09-28 15:04:56 -07:00
MarcoFalke
faa94cb167
test: Check that invalid peer traffic is accounted for 2020-09-28 10:14:00 +02:00
MarcoFalke
fae243f0cb
test: Remove confusing cast to same type (int to int) 2020-09-28 10:13:51 +02:00
kanon
a5a6965157
[Trivial] python help message 2020-09-27 17:31:42 -04:00
Sishir Giri
d5863c0b3e [send] Make send RPCs return fee reason 2020-09-26 17:57:26 -07:00
MarcoFalke
055abfbc5a
Merge #20023: test: remove unused constants in functional tests
92e28fa8b2 test: remove unused constants in functional tests (Sebastian Falbesoner)

Pull request description:

  This mini-PR gets rid of constants in functional tests that are not used anymore. Found by [vulture ](https://pypi.org/project/vulture/)via the following script that has been lying around here locally for quite some time (I think it was once proposed by practicalswift, but I don't remember the concrete topic/PR):
  ```
  #!/bin/sh
  for F in $(git ls-files -- "*.py"); do vulture "$F" | grep "unused variable"; done
  ```

ACKs for top commit:
  practicalswift:
    ACK 92e28fa8b2: patch looks correct.

Tree-SHA512: 16516abc8014207bcefdf0545dffaecff1fbba66f45b54c02371dcfd1f18194855c6b72598c11b5407009561eafe8048d47af3471f0efb1795d52477d5a0232e
2020-09-26 17:45:59 +02:00
MarcoFalke
4f45ea1f73
Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19 [doc] Release notes (Amiti Uttarwar)
50f94b34a3 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b50 [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca4 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19.
  sipa:
    utACK a512925e19
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19 🌇
  promag:
    Code review ACK a512925e19.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2020-09-26 17:24:54 +02:00
Sebastian Falbesoner
92e28fa8b2 test: remove unused constants in functional tests 2020-09-26 12:50:10 +02:00
Oliver Gugger
0fcaf73199
test: use explicit p2p objects where available
To make the intent of the tests easier to understand, we reference the
p2p connection objects by their explicit names instead of the p2ps array.
2020-09-26 10:48:54 +02:00
MarcoFalke
78f912c901
Merge #19804: test/refactor: reference p2p objects explicitly and remove confusing Test_Node.p2p property
10d61505fe [test] remove confusing p2p property (gzhao408)
549d30faf0 scripted-diff: replace p2p with p2ps[0] in p2p_invalid_tx (gzhao408)
7a0de46aea [doc] sample code for test framework p2p objects (gzhao408)
784f757994 [refactor] clarify tests by referencing p2p objects directly (gzhao408)

Pull request description:

  The `TestNode` has a `p2p` property which is an alias for `p2ps[0]`.

  I think this should be removed because it can be confusing and misleading (to both the test writer and reviewer), especially if a TestNode has multiple p2ps connected (which is the case for many tests).
  Another example is when a test has multiple subtests that connect 1 p2p and use the `p2p` property to reference it. If the subtests don't completely clean up after themselves, the subtests may affect one another.

  The best way to refer to a connected p2p is use the object returned by `add_p2p_connection` like this:
  ```py
  p2p_conn = node.add_p2p_connection(P2PInterface())
  ```
  A good example is [p2p_invalid_locator.py](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_invalid_locator.py), which cleans up after itself (waits in both `wait_for_disconnect` and in `disconnect_p2ps`) but wouldn't need so much complexity if it just referenced the connections directly.

  If there is only one connected, it's not really that tedious to just use `node.p2ps[0]` instead of `node.p2p` (and it can always be aliased inside the test itself).

ACKs for top commit:
  robot-dreams:
    utACK 10d61505fe
  jnewbery:
    utACK 10d61505fe
  guggero:
    Concept ACK 10d61505.

Tree-SHA512: 5965548929794ec660dae03467640cb2156d7d826cefd26d3a126472cbc2494b855c1d26bbb7b412281fbdc92b9798b9765a85c27bc1a97f7798f27f64db6f13
2020-09-25 14:18:31 +02:00
practicalswift
9b4fa0af40 net: Print error message if -proxy is specified without arguments (instead of continuing without proxy server) 2020-09-23 15:41:49 +00:00
Wladimir J. van der Laan
9e217f5a6f
Merge #19572: ZMQ: Create "sequence" notifier, enabling client-side mempool tracking
759d94e70f Update zmq notification documentation and sample consumer (Gregory Sanders)
68c3c7e1bd Add functional tests for zmq sequence topic and mempool sequence logic (Gregory Sanders)
e76fc2b84d Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas (Gregory Sanders)
1b615e61bf zmq test: Actually make reorg occur (Gregory Sanders)

Pull request description:

  This PR creates a new ZMQ notifier that gives a "total hash history" of block (dis)connection, mempool addition/substraction, all in one pipeline. It also exposes a "mempool sequence number" to both this notifier and `getrawmempool` results, which allows the consumer to use the results together without confusion about ordering of results and without excessive `getrawmempool` polling.

  See the functional test `interfaces_zmq.py::test_mempool_sync` which shows the proposed user flow for the client-side tracking of mempool contents and confirmations.

  Inspired by https://github.com/bitcoin/bitcoin/pull/19462#issuecomment-656140421
  Alternative to https://github.com/bitcoin/bitcoin/pull/19462 due to noted deficiencies in current zmq notification streams.

  Also fixes a legacy zmq test that didn't actually trigger a reorg because of identical blocks being generated on each side of the split(oops)

ACKs for top commit:
  laanwj:
    Code review ACK 759d94e70f

Tree-SHA512: 9daf0d7d996190f3a68ff40340a687519323d7a6c51dcb26be457fbc013217ea7b62fbd0700b74b654433d2e370704feb61e5584399290692464fcfcb72ce3b7
2020-09-23 13:55:24 +02:00
MarcoFalke
8219893825
Merge #19993: refactor: Signet fixups
facaf9e61f doc: Document signet BIP (MarcoFalke)
faf0a26711 doc: Update comments for new chain settings (-signet and -chain=signet) (MarcoFalke)
fae0548686 fuzz: Remove needless guard (MarcoFalke)
77771a03df refactor: Remove SignetTxs::m_valid and use optional instead (MarcoFalke)
fa2ad5dae1 test: Run signet test even when wallet was not compiled (MarcoFalke)

Pull request description:

  Some doc and test fixups for #18267

ACKs for top commit:
  ajtowns:
    ACK facaf9e61f -- code review only
  dr-orlovsky:
    Reviewed & ACK facaf9e61f
  kallewoof:
    Code review ACK facaf9e61f

Tree-SHA512: 8085027c488d84bb4bddccba78bd2d4c5af0d8e2644ee72265f1f30fa8c83f61a961d9da2c796f2940e69682291cbee7b1028b6a6ce123ad9134c0ebbf4723b0
2020-09-23 09:02:00 +02:00
Troy Giorshev
deb52711a1 Remove header checks out of net_processing
This moves header size and netmagic checking out of net_processing and
into net.  This check now runs in ReadHeader, so that net can exit early
out of receiving bytes from the peer.  IsValid is now slimmed down, so
it no longer needs a MessageStartChars& parameter.

Additionally this removes the rest of the m_valid_* members from
CNetMessage.
2020-09-22 22:05:18 -04:00
MarcoFalke
b1291b2e8f
Merge #19963: Clarify blocksonly whitelistforcerelay test
e15344889a Clarify blocksonly whitelistforcerelay test (t-bast)

Pull request description:

  As discussed in https://github.com/bitcoin/bitcoin/issues/19943, this test may be a bit misleading to newcomers.

  We underscore the fact that our peer needs to run a modified version of Bitcoin Core to actually relay transactions to a `blocksonly` node and benefit from the `whitelistforcerelay` parameter.

ACKs for top commit:
  naumenkogs:
    ACK e15344889a

Tree-SHA512: cc3526ac26c40a2d878b0ad863008663040683fd21092fcdc93866c2b0a79db8c2d29767d1f70bf56195092fca2aa2961cdbee52b5f0b1ae757cece9cd206301
2020-09-22 22:45:06 +02:00
MarcoFalke
fa2ad5dae1
test: Run signet test even when wallet was not compiled 2020-09-22 22:28:36 +02:00
Gregory Sanders
68c3c7e1bd Add functional tests for zmq sequence topic and mempool sequence logic 2020-09-22 11:34:30 -04:00
Gregory Sanders
1b615e61bf zmq test: Actually make reorg occur 2020-09-22 11:17:50 -04:00
Amiti Uttarwar
50f94b34a3 [rpc] Deprecate getpeerinfo addnode field
This field is now redundant since the connection type field will indicate
MANUAL for addnode connections.
2020-09-21 19:01:29 -07:00
Amiti Uttarwar
df091b9b50 [refactor] Rename test file to allow any getpeerinfo deprecations.
Simple rename/restructure to allow for upcoming test additions.
2020-09-21 19:01:29 -07:00
Amiti Uttarwar
395acfa83a [rpc] Add connection type to getpeerinfo RPC, update tests 2020-09-21 19:01:29 -07:00
Wladimir J. van der Laan
8c5f68118c
Merge #18267: BIP-325: Signet [consensus]
8258c4c007 test: some sanity checks for consensus logic (Anthony Towns)
e47ad375bf test: basic signet tests (Karl-Johan Alm)
4c189abdc4 test: add small signet fuzzer (practicalswift)
ec9b25d046 test: signet network selection tests (Karl-Johan Alm)
3efe298dcc signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bca4e qt: update QT to support signet network (Karl-Johan Alm)
a8de47a1c9 consensus: add signet validation (Karl-Johan Alm)
e8990f1214 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b7cd add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7dad validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of #16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c007 per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c007
  MarcoFalke:
    Approach ACK 8258c4c007 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
2020-09-21 22:33:00 +02:00
Wladimir J. van der Laan
c0c409dcd3
Merge #19697: Improvements on ADDR caching
0d04784af1 Refactor the functional test (Gleb Naumenko)
83ad65f31b Address nits in ADDR caching (Gleb Naumenko)
81b00f8780 Add indexing ADDR cache by local socket addr (Gleb Naumenko)
42ec558542 Justify the choice of ADDR cache lifetime (Gleb Naumenko)

Pull request description:

  This is a follow-up on #18991 which does 3 things:
  - improves privacy of a node listening to multiple addresses via adding cache index by local socket addr (suggested [here](https://github.com/bitcoin/bitcoin/pull/18991#issuecomment-668219345))
  - documents on the choice of 24h cache lifetime
  - addresses nits from #18991

ACKs for top commit:
  jnewbery:
    utACK 0d04784af1
  vasild:
    ACK 0d04784
  jonatack:
    Code review ACK 0d04784

Tree-SHA512: bb65a34dd1ce2811186d3e4469bc33e8399cebaaa494ce13041c7cff23275870e4176a719f7a72f8d779c49f8b2344bf4fa1aeb3ea4e2626d5ae76514f00a750
2020-09-21 19:36:57 +02:00
t-bast
e15344889a
Clarify blocksonly whitelistforcerelay test
As discussed in https://github.com/bitcoin/bitcoin/issues/19943, this
test may be a bit misleading to newcomers.

We underscore the fact that our peer needs to run a modified version of
Bitcoin Core to actually relay transactions to a `blocksonly` node and
benefit from the `whitelistforcerelay` parameter.
2020-09-21 10:16:09 +02:00
MarcoFalke
b99a1633b2
Merge #19781: test: add parameterized constructor for msg_sendcmpct()
638441928a test: add parameterized constructor for msg_sendcmpct() (Sebastian Falbesoner)

Pull request description:

  While working on the test for #19776 I noticed that creating a `sendcmpct` message is quite cumbersome -- due to the lack of a parameterized constructor, one needs to create an empty (that is, initialized with default values) object and then set the two fields one by one. This PR replaces the default constructor with a parameterized constructor and uses it in the test `p2p_compactblocks.py`, reducing LOC. No need to pollute the namespace with temporary throw-away message objects anymore.

ACKs for top commit:
  guggero:
    Code review ACK 638441928a.
  epson121:
    Code review ACK 638441928a

Tree-SHA512: 3b58d276d714b73abc6cc98d1d52dec5f6026b33f03faaeb7dcbc5d83ac377555179f98b159b2b9ecc8957999c35a1dc082e3c69299c5fde4e35f1bd0587ce9d
2020-09-20 11:13:56 +02:00
fanquake
c30f79d418
Merge #19940: rpc: Return fee and vsize from testmempoolaccept
23c35bf005 [test] add get_vsize util for more programmatic testing (gzhao408)
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept (codeShark149)

Pull request description:

  From #19093 and resolves #19057.

  Difference from #19093: return `vsize` and `fees` object (similar to `getmempoolentry`) when the test accept is successful. Updates release-notes.md.

ACKs for top commit:
  jnewbery:
    utACK 23c35bf005
  fjahr:
    utACK 23c35bf
  instagibbs:
    reACK 23c35bf005

Tree-SHA512: dcb81b7b817a4684e9076bc5d427a6f2d549d2edc66544e718260c4b5f8f1d5ae1d47b754175e9f0c8a3bd8371ce116c2dca0583588d513a7d733d5d614f2b04
2020-09-19 15:04:03 +08:00
Sjors Provoost
a5f5374b43
test: create default wallet in extended tests
This was omitted from #15454
2020-09-18 17:54:42 +02:00
Karl-Johan Alm
e47ad375bf
test: basic signet tests 2020-09-18 10:19:43 +09:00
Samuel Dobson
652c45fdbb
Merge #15454: Remove the automatic creation and loading of the default wallet
d26f0648f1 Tell users how to load or create a wallet when no wallet is loaded (Andrew Chow)
1bee1e6269 Do not create default wallet (Andrew Chow)

Pull request description:

  Instead of automatically creating and loading a default wallet, users should instead explicitly create their wallet or load it on start.

  Builds on #19754 which provides the `load_on_startup` behavior for the GUI.

ACKs for top commit:
  jnewbery:
    Manual test and very light code review ACK d26f0648f1
  ryanofsky:
    Code review ACK d26f0648f1. Just suggested changes to first commit (reusing MakeWalletDatabase and adding release notes), no changes to second commit
  jonatack:
    ACK d26f0648f1 light code review, debug build, ran tests, did manual testing with testnet, rebased on master, on linux debian.

Tree-SHA512: 091d785aef64736f7df661c576e815a87f3d029cfa32f3a75ba86fc25795f10b022ab3ae15c5b61a10b8cee16f5650f15cd79cbd6127e5e3ccbef631966d3c30
2020-09-18 12:03:55 +12:00
Sjors Provoost
d813d26f06
[rpc] send: various touch-ups 2020-09-17 21:10:56 +02:00
Sjors Provoost
0fc1c685e1
[rpc] send: fix parsing replaceable option 2020-09-17 20:59:09 +02:00
Wladimir J. van der Laan
a518b1c26b
Merge #19936: Test: batch rpc with params
e1fdd2963b Test batch rpc with params (Gregory Sanders)

Pull request description:

  Useful as an example and test case.

ACKs for top commit:
  laanwj:
    ACK e1fdd2963b
  theStack:
    ACK e1fdd2963b

Tree-SHA512: 2d2ba8960916342b264a14624857d6dd10005be12efafb3e970b82656f721c8f3700ebc9b8809de1b2f887d482b772043504aeaeebc7f2e1c8203f076a451526
2020-09-16 16:25:00 +02:00
gzhao408
23c35bf005 [test] add get_vsize util for more programmatic testing 2020-09-16 07:19:58 -07:00
codeShark149
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept
Return fee and vsize if tx would pass ATMP.
2020-09-15 18:01:32 -07:00
Samuel Dobson
ffaac6e614
Merge #16378: The ultimate send RPC
92326d8976 [rpc] add send method (Sjors Provoost)
2c2a1445dc [rpc] add snake case aliases for transaction methods (Sjors Provoost)
1bc8d0fd59 [rpc] walletcreatefundedpsbt: allow inputs to be null (Sjors Provoost)

Pull request description:

  `walletcreatefundedpsbt` has some interesting features that `sendtoaddress` and `sendmany` don't have:
  * manual coin selection
  * outputting a PSBT (it was controversial to add this, see #18201)
  * create a transaction without adding to wallet (which leads to broadcasting, unless `-walletbroadcast=0`)

  At the same time `walletcreatefundedpsbt` can't broadcast a transaction, which is inconvenient for simple use cases.

  This PR introduces a new `send` RPC method which creates a PSBT, signs it if possible and adds it to the wallet by default. If it can't sign all inputs, it outputs a PSBT. If `add_to_wallet` is set to `false` it will return the transaction in both PSBT and hex format.

  Because it uses a PSBT internally, it will much easier to add hardware wallet support to this method (see #16546).

  For `bitcoin-cli` users, it tries to keep the simplest use case easy to use:

  ```sh
  bitcoin-cli -regtest send '{"ADDRESS": 0.1}' 1 sat/b
  ```

  This paves the way for deprecating `sendtoaddress` and `sendmany` though there's no rush. The only missing feature compared to these older methods is adding labels to a destination address.

  Depends on:
  - [x] #16377 (`[rpc] don't automatically append inputs in walletcreatefundedpsbt`)
  - [x] #11413 (`[wallet] [rpc] sendtoaddress/sendmany: Add explicit feerate option`)
  - [x] #18244 (`[rpc] have lockUnspents also lock manually selected coins`)

ACKs for top commit:
  meshcollider:
    Light re-utACK 92326d8976
  achow101:
    ACK 92326d8976 Reviewed code and test, ran tests.
  kallewoof:
    utACK 92326d8976

Tree-SHA512: 7552ef1b193d4c06e381c44932fdb0d54f64383e4c7d6b988f49d059c7d4bba45ce6aa7813e03df86360ad9dad6f3010eb76ee7da480551742d5fd98c2251c0f
2020-09-15 14:49:08 +12:00
Gregory Sanders
e1fdd2963b Test batch rpc with params 2020-09-13 13:43:03 -04:00
Luke Dashjr
d438d609cd QA: Use GBT to get block versions correct 2020-09-12 18:24:26 +00:00
Luke Dashjr
1df2cd1c8f QA: blocktools: Accept block template to create_block 2020-09-12 18:24:26 +00:00
gzhao408
10d61505fe [test] remove confusing p2p property 2020-09-10 07:39:14 -07:00
gzhao408
549d30faf0 scripted-diff: replace p2p with p2ps[0] in p2p_invalid_tx
-BEGIN VERIFY SCRIPT-
sed -i 's/\.p2p\./.p2ps[0]./g' test/functional/p2p_invalid_tx.py
-END VERIFY SCRIPT-
2020-09-10 07:39:01 -07:00
gzhao408
7a0de46aea [doc] sample code for test framework p2p objects 2020-09-10 07:38:28 -07:00
gzhao408
784f757994 [refactor] clarify tests by referencing p2p objects directly
Use object returned from add_p2p_connection to refer to
p2ps. Add a test class attribute if it needs to be used across
many methods. Don't use the p2p property.
2020-09-10 07:37:14 -07:00
Sjors Provoost
92326d8976
[rpc] add send method 2020-09-10 13:44:53 +02:00
João Barbosa
faf251d854
test: gettxoutproof duplicate txid 2020-09-09 11:27:35 +02:00
MarcoFalke
faf5eb45c4
test: Test empty array in gettxoutproof 2020-09-09 11:27:33 +02:00
MarcoFalke
fa56e866e8
test: Run rpc_txoutproof.py even with wallet disabled 2020-09-09 11:27:28 +02:00
MarcoFalke
faba790bd4
test: MiniWallet: Default fee_rate in send_self_transfer, Pass in utxo_to_spend
Adds two new features to MiniWallet:

* The fee rate is irrelevant sometimes, so just set an arbitrary default
* The utxo to spend needs to be selected manually sometimes
2020-09-09 10:39:05 +02:00
MarcoFalke
fa65a11d0c
test: bugfix: Actually pick largest utxo 2020-09-09 10:38:50 +02:00
MarcoFalke
564e1ab0f3
Merge #19800: test: Mockwallet
fa188c9c59 test: Use MiniWalet in p2p_feefilter (MarcoFalke)
fa39c62eb7 test: inline hashToHex (MarcoFalke)

Pull request description:

  This introduces a minimalistic test wallet, which can be used as a drop in replacement for the Bitcoin Core wallet to create dummy transactions with a given fee rate.

ACKs for top commit:
  jnewbery:
    utACK fa188c9c59

Tree-SHA512: 0aad9cb14eea4f0055bd6a47cc8c8f82a16941b152598c3bf1e083aae84cca4ffa23f0b854a362a68be1b917deba1b5ec7c0207b63b0805d747ba9a7d1d82efe
2020-09-09 09:06:22 +02:00
Andrew Chow
1bee1e6269 Do not create default wallet
No longer create a default wallet. The default wallet will still be
loaded if it exists and not other wallets were specified (anywhere,
including settings.json, bitcoin.conf, and command line).

Tests are updated to be started with -wallet= if they need the default
wallet.

Added test to wallet_startup.py testing that no default wallet is
created and that it is loaded if it exists and no other wallets were
specified.
2020-09-08 21:02:53 -04:00
Sjors Provoost
2c2a1445dc
[rpc] add snake case aliases for transaction methods 2020-09-07 20:33:16 +02:00
Sjors Provoost
1bc8d0fd59
[rpc] walletcreatefundedpsbt: allow inputs to be null
This is of neglible use here, but it allows new RPC methods to take outputs as their first argument and make inputs optional.
2020-09-07 20:33:16 +02:00
MarcoFalke
fa188c9c59
test: Use MiniWalet in p2p_feefilter 2020-09-07 15:06:24 +02:00
Samuel Dobson
56d47e19ed
Merge #19619: Remove wallet.dat path handling from wallet.cpp, rpcwallet.cpp
7bf6dfbb48 wallet: Remove path checking code from bitcoin-wallet tool (Russell Yanofsky)
77d5bb72b8 wallet: Remove path checking code from createwallet RPC (Russell Yanofsky)
a987438e9d wallet: Remove path checking code from loadwallet RPC (Russell Yanofsky)
8b5e7297c0 refactor: Pass wallet database into CWallet::Create (Russell Yanofsky)
3c815cfe54 wallet: Remove Verify and IsLoaded methods (Russell Yanofsky)
0d94e60625 refactor: Use DatabaseStatus and DatabaseOptions types (Russell Yanofsky)
b5b414151a wallet: Add MakeDatabase function (Russell Yanofsky)
288b4ffb6b Remove WalletLocation class (Russell Yanofsky)

Pull request description:

  Get rid of file path handling in wallet application code and move it down to database layer.

  There is no change in behavior except for some changed error messages.

  Motivation for this change is to make code more understandable, but also to prepare for adding SQLite support in #19077 so SQLite implementation can be contained at the database layer and wallet loading code does not need to become more complicated.

ACKs for top commit:
  achow101:
    ACK 7bf6dfbb48
  meshcollider:
    Code re-review and functional test run ACK 7bf6dfbb48

Tree-SHA512: 23ad18324c9e8947f0cf88a3734c2e9fb25536b2cb4d552cf5d1a4ade320fbffb73bb2d1b3a99585c11630aa7092e0fcfc2dd4fe65b91e3a54161433a5cd13cb
2020-09-07 11:45:36 +12:00
Benoit Verret
637d8bce74 Change FILE_CHAR_BLOCKLIST to FILE_CHARS_DISALLOWED
Blocklist is ambiguous. It could mean a list of blocks.

Example: "blocknotify" in the same file refers to Bitcoin blocks.
2020-09-06 12:15:04 -04:00
Fabian Jahr
56b018ca7f
test: Fix flaky wallet_basic test
Co-authored-by: João Barbosa <joao.paulo.barbosa@gmail.com>
2020-09-05 19:55:59 +02:00
Wladimir J. van der Laan
23d3ae7acc
Merge #19405: rpc, cli: add network in/out connections to getnetworkinfo and -getinfo
581b343d5b Add in/out connections to cli -getinfo (Jon Atack)
d9cc13e88d UNIX_EPOCH_TIME fixup in rpc getnettotals (Jon Atack)
1ab49b81cf Add in/out connections to rpc getnetworkinfo (Jon Atack)

Pull request description:

  This is basic info that is present in the GUI that I've been wishing to have exposed via the RPC and CLI without needing a bash workaround or script. For human users it would also be useful to have it in `-getinfo`.

  `bitcoin-cli getnetworkinfo`
  ```
    "connections": 15,
    "connections_in": 6,
    "connections_out": 9,
  ```

  `bitcoin-cli -getinfo`
  ```
    "connections": {
      "in": 6,
      "out": 9,
      "total": 15
    },
  ```

  Update the tests, RPC help, and release notes for the changes. Also fixup the `getnettotals` timemillis help while touching `rpc/net.cpp`.

  -----

  Reviewers can manually test this PR by [building from source](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests), launching bitcoind, and then running `bitcoin-cli -getinfo`, `bitcoin-cli getnetworkinfo`, `bitcoin-cli help getnetworkinfo`, and `bitcoin-cli help getnettotals` (for the UNIX epoch time change).

ACKs for top commit:
  eriknylund:
    > tACK [581b343](581b343d5b) on master at [a0a422c](a0a422c34c), ran unit & functional tests and and confirmed changes on an existing datadir ✌️
  benthecarman:
    tACK `581b343`
  willcl-ark:
    tACK for 581b343d5b, this time rebased onto master at 862fde88be.
  shesek:
    tACK `581b343`. This provides what I needed, thanks!
  n-thumann:
    tACK 581b343 on master at a0a422c, ran unit & functional tests and and confirmed changes on an existing datadir ✌️

Tree-SHA512: 08dd3ac8fefae401bd8253ff3ac027603c528eeccba53cedcb127771316173a7052fce44af8fa33ac98ebc4cf2a2b11cdefd949995d55e9b9a5942b876d00dc5
2020-09-04 15:09:37 +02:00
Russell Yanofsky
7bf6dfbb48 wallet: Remove path checking code from bitcoin-wallet tool
This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
77d5bb72b8 wallet: Remove path checking code from createwallet RPC
This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
a987438e9d wallet: Remove path checking code from loadwallet RPC
This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
Russell Yanofsky
3c815cfe54 wallet: Remove Verify and IsLoaded methods
Checks are now consolidated in MakeBerkeleyDatabase function instead of
happening in higher level code.

This commit does not change behavior except for error messages which now
include more complete information.
2020-09-03 12:24:32 -04:00
John Newbery
825fcae484 [tests] Replace bytes literals with hex literals
It's almost impossible to read bytes literals in code, so replace them
with the hex string literal and then convert them to a bytes object
using bytes.fromhex().
2020-09-03 16:47:49 +01:00
John Newbery
64eca45100 [tests] Fix pep8 style violations in address.py 2020-09-03 16:47:49 +01:00
John Newbery
b230f8b3f3 [tests] Correct docstring for address.py 2020-09-03 16:47:49 +01:00
John Newbery
ea70e6a2ca [tests] Tidy up imports in address.py
No need to import twice from util.py
2020-09-03 16:47:49 +01:00
John Newbery
7f639df0b8 [tests] Remove unused optional verify_checksum parameter
This optional parameter is never used, so remove it.
2020-09-03 16:47:49 +01:00
John Newbery
011e784f74 [tests] Rename segwit encode and decode functions
These functions can be exported to other modules,
so be explicit that they're encoding and decoding
segwit addresses
2020-09-03 16:47:49 +01:00
John Newbery
e4557133f5 [tests] Move bech32 unit tests to test framework 2020-09-03 16:47:46 +01:00
Wladimir J. van der Laan
bd60a9a8ed
Merge #19818: p2p: change CInv::type from int to uint32_t, fix UBSan warning
7984c39be1 test framework: serialize/deserialize inv type as unsigned int (Jon Atack)
407175e0c2 p2p: change CInv::type from int to uint32_t (Jon Atack)

Pull request description:

  Fixes UBSan implicit-integer-sign-change issue per https://github.com/bitcoin/bitcoin/pull/19610#issuecomment-680686460.

  Credit to Crypt-iQ for finding and reporting the issue and to vasild for the original review suggestion in https://github.com/bitcoin/bitcoin/pull/19590#pullrequestreview-455788826.

  Closes #19678.

ACKs for top commit:
  laanwj:
    ACK 7984c39be1
  MarcoFalke:
    ACK 7984c39be1 🌻
  vasild:
    ACK 7984c39be

Tree-SHA512: 59f3a75f40ce066ca6f0bb1927197254238302b4073af1574bdbfe6ed580876437be804be4e47d51467d604f0d9e3a5875159f7f2edbb2351fdb2bb9465100b5
2020-09-03 17:23:52 +02:00
Wladimir J. van der Laan
4053de04e2
Merge #19859: qa: Fixes failing functional test by changing version
6de9429087 qa: Changes v0.17.1 to v0.17.2 (nthumann)

Pull request description:

  As of 0374e821bd v0.17.2 is downloaded instead of v0.17.1 for functional testing. This causes `test/functional/feature_backwards_compatibility.py` to fail, because it [requires](https://github.com/bitcoin/bitcoin/blob/master/test/functional/feature_backwards_compatibility.py#L57) v0.17.1.

  Steps to reproduce:
  Run `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.1 v0.16.3 v0.15.2`. It cannot be downloaded at all because the sha256sum is missing [here](c1e0c2ad3b/test/get_previous_releases.py (L23)).
  Or adjust the command and run `test/get_previous_releases.py -b v0.19.1 v0.18.1 v0.17.2 v0.16.3 v0.15.2`, then run `test/functional/test_runner.py feature_backwards_compatibility`. It´ll fail because the test is missing v0.17.1.

  This PR changes v0.17.1 to v0.17.2 in this test and in a few comments.

ACKs for top commit:
  laanwj:
    ACK 6de9429087
  fanquake:
    ACK 6de9429087 - looks correct. Surprised this wasn't caught/part of #19813. In future you could add any explanations & extra info as part of your commit message as well (even though PR descriptions are included as part of the merge).

Tree-SHA512: bbe50c4fd5c1aedd6dc1cdc3d93ef9005db1c67adca3f263b6b0d869c40b495a3221e706c9389fedea4748e31911dbd591062f60ce9836e58099fbdd9515b4d9
2020-09-03 13:38:21 +02:00
fanquake
136fe4c5e9
Merge #19816: test: Rename wait until helper to wait_until_helper
fa1cd9e1dd test: Remove unused lock arg from BitcoinTestFramework.wait_until (MarcoFalke)
fad2794e93 test: Rename wait until helper to wait_until_helper (MarcoFalke)
facb41bf1d test: Remove unused p2p_lock in VersionBitsWarningTest (MarcoFalke)

Pull request description:

  This avoids confusion with the `wait_until` member functions, which should be preferred because they take the appropriate locks and scale the timeout appropriately on their own.

ACKs for top commit:
  laanwj:
    Code review ACK fa1cd9e1dd
  hebasto:
    ACK fa1cd9e1dd, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 319d400085606a4c738e314824037f72998e6657d8622b363726842aba968744f23c56d27275dfe506b8cbbb6e97fc39ca1d325db05d4d67df0e8b35f2244d5c
2020-09-03 12:07:53 +08:00
Russell Yanofsky
ff44cae279 test: Change feature_config_args.py not to rely on strange regtest=0 behavior
Update test to simply generate a normal mainnet configuration file instead of
using a crazy setup where a regtest=1 config file using an includeconf in the
[regtest] section includes another config file that specifies regtest=0,
retroactively switching the network to mainnet.

This setup was fragile and only worked because the triggered InitError happened
early enough that none of the ignored [regtest] options mattered (only
affecting log output).
2020-09-02 08:14:34 -05:00
nthumann
6de9429087
qa: Changes v0.17.1 to v0.17.2 2020-09-02 13:51:36 +02:00
Wladimir J. van der Laan
505b39e72b
Merge #19610: p2p: refactor AlreadyHave(), CInv::type, INV/TX processing
fb56d37612 p2p: ensure inv is GenMsgTx before ToGenTxid in inv processing (John Newbery)
aa3621385e test: use CInv::MSG_WITNESS_TX flag in p2p_segwit (Jon Atack)
24ee4f01ea p2p: make gtxid(.hash) and fAlreadyHave localvars const (Jon Atack)
b1c855453b p2p: use CInv block message helpers in net_processing.cpp (Jon Atack)
acd6642167 [net processing] Change AlreadyHaveTx() to take a GenTxid (John Newbery)
5fdfb80b86 [net processing] Change AlreadyHaveBlock() to take block_hash argument (John Newbery)
430e183b89 [net processing] Remove mempool argument from AlreadyHaveBlock() (John Newbery)
42ca5618ca [net processing] Split AlreadyHave() into separate block and tx functions (John Newbery)
39f1dc9445 p2p: remove nFetchFlags from NetMsgType TX and INV processing (Jon Atack)
471714e1f0 p2p: add CInv block message helper methods (Jon Atack)

Pull request description:

  Building on #19590 and the recent `wtxid` and `GenTxid` changes, this is a refactoring and cleanup PR to simplify and improve some of the net processing code.

  Some of the diffs are best reviewed with `-w` to ignore spacing.

  Co-authored by John Newbery.

ACKs for top commit:
  laanwj:
    Code review ACK fb56d37612
  jnewbery:
    utACK fb56d37612
  vasild:
    ACK fb56d3761

Tree-SHA512: ba39b58e6aaf850880a842fe5f6295e9f1870906ef690206acfc17140aae2ac854981e1066dbcd4238062478762fbd040ef772fdc2c50eea6869997c583e6a6d
2020-09-02 13:45:40 +02:00
Gleb Naumenko
0d04784af1 Refactor the functional test 2020-09-02 10:33:18 +03:00
Wladimir J. van der Laan
48c1083632
Merge #19105: Add Muhash3072 implementation in Python
36ec9801a4 test: Add chacha20 test vectors in muhash (Fabian Jahr)
0e2b400fea test: Add basic Python/C++ Muhash implementation parity unit test (Fabian Jahr)
b85543cb73 test: Add Python MuHash3072 implementation to test framework (Pieter Wuille)
ab30cece0e test: Move modinv to util and add unit test (Fabian Jahr)

Pull request description:

  This is the second in a [series of pull requests](https://github.com/bitcoin/bitcoin/pull/18000) to implement an Index for UTXO set statistics.

  This pull request adds a Python implementation of Muhash3072, a homomorphic hashing algorithm to be used for hashing the UTXO set. The Python implementation can then be used to compare behavior with the C++ version.

ACKs for top commit:
  jnewbery:
    utACK 36ec9801a
  laanwj:
    Code review ACK 36ec9801a4

Tree-SHA512: a3519c6e11031174f1ae71ecd8bcc7f3be42d7fc9c84c77f2fbea7cfc5ad54fcbe10b55116ad8d9a52ac5d675640eefed3bf260c58a02f2bf3bc0d8ec208baa6
2020-09-01 17:12:20 +02:00
fanquake
a1d14f522c
Merge #19671: wallet: Remove -zapwallettxes
3340dbadd3 Remove -zapwallettxes (Andrew Chow)

Pull request description:

  It's not clear what use there is to keeping `-zapwallettxes` given that it's intended usage has been superseded by `abandontransaction`. So this removes it outright.

  Alternative to #19700

ACKs for top commit:
  meshcollider:
    utACK 3340dbadd3
  fanquake:
    ACK 3340dbadd3 - remaining manpage references will get cleaned up pre-release.

Tree-SHA512: 3e58e1ef6f4f94894d012b93e88baba3fb9c2ad75b8349403f9ce95b80b50b0b4f443cb623cf76c355930db109f491b3442be3aa02972e841450ce52cf545fc8
2020-09-01 09:26:28 +08:00
Wladimir J. van der Laan
e796fdd4cb
Merge #19507: Expand functional zmq transaction tests
7356292e1d Have zmq reorg test cover mempool txns (Gregory Sanders)
a0f4f9c983 Add zmq test for transaction pub during reorg (Gregory Sanders)
2399a0600c Add test case for mempool->block zmq notification (Gregory Sanders)
e70512a83c Make ordering of zmq consumption irrelevant to functional test (Gregory Sanders)

Pull request description:

  Tests written to better define what messages are sent when. Also did a bit of refactoring to make sure the exact notification channel ordering doesn't matter.

  Confusions below aside, I believe having these more descriptive tests helps describe what behavior we expect from ZMQ notificaitons.

  Remaining confusion:
  1) Notification patterns seem to vary wildly with the inclusion of mempool transactions being reorg'ed. See difference between "Add zmq test for transaction pub during reorg" and "Have zmq reorg test cover mempool txns" commits for specifics.
  2) Why does a reorg'ed transaction get announced 3 times? From what I understand it can get announced once for disconnected block, once for mempool entry. What's the third? It occurs a 4th time when included in a block(not added in test)

ACKs for top commit:
  laanwj:
    code review ACK 7356292e1d
  promag:
    Code review ACK 7356292e1d.

Tree-SHA512: 573662429523fd6a1af23dd907117320bc68cb51a93fba9483c9a2160bdce51fb590fcd97bcd2b2751d543d5c1148efa4e22e1c3901144f882b990ed2b450038
2020-08-31 20:46:27 +02:00
Andrew Chow
3340dbadd3 Remove -zapwallettxes
-zapwallettxes is made a hidden option to inform users that it is
removed and they should be using abandontransaction to do the stuck
transaction thing.
2020-08-31 12:39:19 -04:00
MarcoFalke
89a8299a14
Merge #19717: rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining,zmq,rpcdump)
fa3d9ce325 rpc: Assert that RPCArg names are equal to CRPCCommand ones (rpcdump) (MarcoFalke)
fa32c1d5ec rpc: Assert that RPCArg names are equal to CRPCCommand ones (zmq) (MarcoFalke)
faaa46dc20 rpc: Assert that RPCArg names are equal to CRPCCommand ones (mining) (MarcoFalke)
fa93bc14c7 rpc: Remove unused return type from appendCommand (MarcoFalke)

Pull request description:

  This is split out from #18531 to just touch the RPC methods in misc. Description from the main pr:

  ### Motivation

  RPCArg names in the rpc help are currently only used for documentation. However, in the future they could be used to teach the server the named arguments. Named arguments are currently registered by the `CRPCCommand`s and duplicate the RPCArg names from the documentation. This redundancy is fragile, and has lead to errors in the past (despite having linters to catch those kind of errors). See section "bugs found" for a list of bugs that have been found as a result of the changes here.

  ### Changes

  The changes here add an assert in the `CRPCCommand` constructor that the RPCArg names are identical to the ones in the `CRPCCommand`.

  ### Future work

  > Here or follow up, makes sense to also assert type of returned UniValue?

  Sure, but let's not get ahead of ourselves. I am going to submit any further works as follow-ups, including:

  * Removing the CRPCCommand arguments, now that they are asserted to be equal and thus redundant
  * Removing all python regex linters on the args, now that RPCMan can be used to generate any output, including the cli.cpp table
  * Auto-formatting and sanity checking the RPCExamples with RPCMan
  * Checking passed-in json in self-check. Removing redundant checks
  * Checking returned json against documentation to avoid regressions or false documentation
  * Compile the RPC documentation at compile-time to ensure it doesn't change at runtime and is completely static

  ### Bugs found

  * The assert identified issue #18607
  * The changes itself fixed bug #19250

ACKs for top commit:
  fjahr:
    tested ACK fa3d9ce325
  promag:
    Code review ACK fa3d9ce325.

Tree-SHA512: 068ade4b55cc195868d53b7f9a27151d45b440857bb069e261a49d102a49a38fdba5d68868516a1d66a54a73ba34681362f934ded7349e894042bde873b75719
2020-08-31 17:43:35 +02:00
MarcoFalke
c1e0c2ad3b
Merge #19813: util, ci: Hard code previous release tarball checksums
0374e821bd util: Hard code previous release tarball checksums (Hennadii Stepanov)
bd897ce79f scripted-diff: Move previous_release.py to test/get_previous_releases.py (Hennadii Stepanov)

Pull request description:

  #19205 introduced signature verifying for the downloaded `SHA256SUMS.asc`.
  This approach is brittle and does not work in CI environment for many reasons:
  - https://github.com/bitcoin/bitcoin/issues/19812#issuecomment-680760663
  - https://github.com/bitcoin/bitcoin/pull/19013#discussion_r459590779

  This PR:
  - implements **Sjors**' [idea](https://github.com/bitcoin/bitcoin/pull/19205#pullrequestreview-426080048):
  > Alternatively we might as well hard code the checksum for each `tar.gz` release in the source code, here.

  - is an alternative to 5a2c31e528e6bd60635096f233252f3c717f366d (#19013)

  - fixes #19812

  - updates v0.17.1 to v0.17.2

ACKs for top commit:
  MarcoFalke:
    cr ACK 0374e821bd
  Sjors:
    tACK 0374e821bd

Tree-SHA512: cacdcf9f5209eae7da357abb3445585ad2f980920fd5bf75527ce89974d3f531a4cf8b5b35edfc116b23bfdfb45c0437cb14cbc416d76ed2dc5b9e6d33cdad71
2020-08-31 16:18:29 +02:00
Samuel Dobson
f98872f127
Merge #18244: rpc: fundrawtransaction and walletcreatefundedpsbt also lock manually selected coins
6d1f51343c [rpc] fundrawtransaction, walletcreatefundedpsbt lock manually selected coins (Sjors Provoost)

Pull request description:

  When using `fundrawtransaction` and `walletcreatefundedpsbt` with `lockUnspents`, it would only lock automatically selected coins, not manually selected coins. That doesn't make much sense to me if the goal is to prevent accidentally double-spending yourself before you broadcast a transaction.

  Note that when  creating a transaction, manually selected coins are automatic "unlocked" (or more accurately: the lock is ignored). Earlier versions of this PR introduced an error when a locked coin is manually selected, but this idea was abandoned after some discussion. An application that uses this RPC should either rely on automatic coin selection (with `lockUnspents`) or handle lock concurrency itself with manual coin selection. In particular it needs to make sure to avoid/pause calls with automatic coin selection between calling `lockunspent` and the subsequent spending RPC.

  See #7518 for historical background.

ACKs for top commit:
  meshcollider:
    Code review ACK 6d1f51343c
  fjahr:
    Code review ACK 6d1f51343c

Tree-SHA512: 8773c788d92f2656952e1beac147ba9956b8c5132d474e0880e4c89ff53642928b4cbfcd1cb3d17798b9284f02618a8830c93a9f7a4733e5bded96adff1d5d4d
2020-08-31 23:30:53 +12:00
Sebastian Falbesoner
9e399b9b2d test: check parameter validity in rpc_signmessage.py
checks parameter validity and error codes for the related RPCs:
    - signmessagewithprivkey
    - signmessage
    - verifymessage
2020-08-29 10:42:43 +02:00
Hennadii Stepanov
bd897ce79f
scripted-diff: Move previous_release.py to test/get_previous_releases.py
-BEGIN VERIFY SCRIPT-
OLD=contrib/devtools/previous_release.py
NEW=test/get_previous_releases.py
sed -i "s|$OLD|$NEW|g" $(git grep -l $OLD)
git mv $OLD $NEW
-END VERIFY SCRIPT-
2020-08-29 11:26:25 +03:00
Jon Atack
7984c39be1
test framework: serialize/deserialize inv type as unsigned int 2020-08-28 20:12:02 +02:00