Commit graph

5744 commits

Author SHA1 Message Date
Andrew Chow
5bbf735def
Merge bitcoin/bitcoin#27653: test: add unit test coverage for Python ECDSA implementation
96b3f2dbe4 test: add unit test coverage for Python ECDSA implementation (Sebastian Falbesoner)

Pull request description:

  This PR adds missing unit test coverage for the Python ECDSA implementation, which should be useful for detecting potential problems early whenever changes in the test framework's Python implementation of secp256k1 are made (e.g. #26222). Note that right now we don't call `ECPubKey.verify_ecdsa` anywhere in our tests, so we wouldn't notice if it is broken at some point.

  To keep it simple, the already existing unit test for Schnorr signatures is extended to also check ECDSA signatures. For that purpose, the dictionary storing private-key/public-key entries use their legacy types `ECKey/ECPubKey` instead of bare byte-arrays, and for Schnorr signing/verification the necessary conversions (ECKey -> bare private key, ECPubKey -> x-only pubkey) is done later when needed. To avoid code duplication, a helper function `random_bitflip` for damaging signatures is introduced.

  The unit test can be run by either calling it for this single module:
  `$ python3 -m unittest ./test/functional/test_framework/key.py`
  or simply running `$ ./test/functional/test_runner.py` which calls all test framework module's unit tests at the start (see TEST_FRAMEWORK_MODULES list).

ACKs for top commit:
  achow101:
    ACK 96b3f2dbe4
  sipa:
    utACK 96b3f2dbe4
  stratospher:
    tested ACK 96b3f2d.

Tree-SHA512: b993f25b843fa047376addda4ce4b0f15750ffba926528b5cca4c5f99b9af456206f4e8af885d25a017dddddf382ddebf38765819b3d16a3f28810d03b010808
2023-09-29 14:49:58 -04:00
Andrew Chow
9d5150ac47
Merge bitcoin/bitcoin#28540: tests: Fix wallet_resendwallettransactions.py intermittent failure by using manual bumps instead of bumpfee
b5a962564e tests: Use manual bumps instead of bumpfee for resendwallettransactions (Andrew Chow)

Pull request description:

  Bumpfee will try to increase the entire package to the target feerate, which causes repeated bumpfees to quickly shoot up in fees, causing intermittent failures when the fee is too large. We don't care about this property, just that the child is continuously replaced until we observe it's position in mapWallet is before its parent. Instead of using bumpfee, we can create raw transactions which have only pay (just above) the additional incremental relay fee, thus avoiding this problem.

  Fixes #28491

ACKs for top commit:
  kevkevinpal:
    ACK [b5a9625](b5a962564e)
  mzumsande:
    Code review ACK b5a962564e
  pablomartin4btc:
    ACK b5a962564e -> adding the `try_rpc` to avoid (skip) any possible failure around the manual bump fee (if we ever reach it as [explained](https://github.com/bitcoin/bitcoin/pull/28540#issuecomment-1737648048)) makes a lot of sense as the spirit of the test is the tx (child before parent) sort in the `mapWallet` (as also [explained](https://github.com/bitcoin/bitcoin/issues/28491#issuecomment-1736161363)).
  MarcoFalke:
    lgtm ACK b5a962564e

Tree-SHA512: f184f11c73be0c30753181901f51a3b4b9c4135e0c4681e9f4ca94692c49bac15c91683c85266a2124333c8593e9919bfd9102724616faab299740f2eb98741f
2023-09-28 11:22:10 -04:00
Andrew Chow
19a7e608f9
Merge bitcoin/bitcoin#28505: rpc: bumpfee, improve doc for 'reduce_output' arg
b3db8c9d5c rpc: bumpfee, improve doc for 'reduce_output' arg (furszy)

Pull request description:

  Fixes #28180. Resulted from discussions with S3RK, achow101, and Murch.

  The current argument name and description are dangerous as it don't
  describe the case where the user selects the recipient output as the
  change address. This one could end up been increased by the inputs
  minus outputs remainder. Which, when `bumpfee` adds new inputs
  to the transaction, leads the process to send more coins to the
  recipient. Which is not what the user would expect from a
  'reduce_output' param naming.

ACKs for top commit:
  S3RK:
    ACK b3db8c9d5c
  achow101:
    ACK b3db8c9d5c
  murchandamus:
    ACK b3db8c9d5c

Tree-SHA512: 91f607e2f5849041d7c099afdddae11af8bed5b1ac90c9d22921267f272e21b44e107d6968e037f05f958a61fe29e94e5fb44b224fb3606f197f83ec4ba3b1e7
2023-09-27 16:46:27 -04:00
Andrew Chow
b5a962564e tests: Use manual bumps instead of bumpfee for resendwallettransactions
Bumpfee will try to increase the entire package to the target feerate,
which causes repeated bumpfees to quickly shoot up in fees, causing
intermittent failures when the fee is too large. We don't care about
this property, just that the child is continuously replaced until we
observe it's position in mapWallet is before its parent. Instead of
using bumpfee, we can create raw transactions which have only pay the
additional incremental relay fee, thus avoiding this problem.
2023-09-27 11:39:07 -04:00
furszy
b3db8c9d5c
rpc: bumpfee, improve doc for 'reduce_output' arg
The current argument name and description are dangerous as it don't
describe the case where the user selects the recipient output as the
change address. This one could end up been increased by the inputs
minus outputs remainder. Which, when bumpfee adds new inputs
to the transaction, leads the process to send more coins to the
recipient. Which is not what the user would expect from a
'reduce_output' param naming.

Co-authored-by: Murch <murch@murch.one>
2023-09-26 20:17:02 -03:00
Andrew Chow
719cb301e6
Merge bitcoin/bitcoin#28492: RPC: descriptorprocesspsbt returns hex encoded tx if complete
a99e9e655a doc: add release note (ismaelsadeeq)
2b4edf889a test: check `descriptorprocesspsbt` return hex encoded tx (ismaelsadeeq)
c405207a18 rpc: `descriptorprocesspsbt` return hex encoded tx (ismaelsadeeq)

Pull request description:

  Coming from [#28414 comment](https://github.com/bitcoin/bitcoin/pull/28414#pullrequestreview-1618684391) Same thing also for `descriptorprocesspsbt`.

  Before this PR `descriptorprocesspsbt` returns a boolean `complete` which indicates that the psbt is final, users then have to call `finalizepsbt` to get the hex encoded network transaction.

  In this PR if the psbt is complete the return object also has the hex encoded network transaction ready for broadcast with `sendrawtransaction`.

  This save users calling `finalizepsbt` with the descriptor, if it is already complete.

ACKs for top commit:
  achow101:
    ACK a99e9e655a
  pinheadmz:
    ACK a99e9e655a
  ishaanam:
    ACK a99e9e655a

Tree-SHA512: c3f1b1391d4df05216c463127cd593f8703840430a99febb54890bc66fadabf9d9530860605f347ec54c1694019173247a0e7a9eb879d3cbb420f9e8d9839b75
2023-09-23 11:55:38 -04:00
Andrew Chow
2303fd2f43
Merge bitcoin/bitcoin#28471: Fix virtual size limit enforcement in transaction package context
eb8f58f5e4 Add functional test to catch too large vsize packages (Greg Sanders)
1a579f9d01 Handle over-sized (in virtual bytes) packages with no in-mempool ancestors (Greg Sanders)
bc013fe8e3 Bugfix: Pass correct virtual size to CheckPackageLimits (Luke Dashjr)
533660c58a Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT to avoid vbyte confusion (Greg Sanders)

Pull request description:

  (Alternative) Minimal subset of https://github.com/bitcoin/bitcoin/pull/28345 to:

  1) Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT which accounts for additional WU necessary to not exclude default chain limit transactions that would have been accepted individually. Avoids sigops vbyte confusion.
  2) pass correct vsize to chain limit evaluations in package context
  3) stop overly-large packages that have no existing mempool ancestors (also a bugfix by itself if someone sets non-standard chain limits)

  This should fix the known issues while not blocking additional refactoring later.

ACKs for top commit:
  achow101:
    ACK eb8f58f5e4
  ariard:
    Re-Code ACK eb8f58f5e
  glozow:
    reACK eb8f58f5e4

Tree-SHA512: 1b5cca1a526207e25d387fcc29a776a3198c3a013dc2b35c6275b9d5a64db2476c154ebf52e3a1aed0b9924c75613f21a946577aa760de28cadf0c9c7f68dc39
2023-09-21 12:13:52 -04:00
fanquake
cf0711cac3
Merge bitcoin/bitcoin#27934: test: added coverage to estimatefee
d05be124db test: added coverage to estimatefee (kevkevin)

Pull request description:

  Added a assert for an rpc error when we try to estimate fee for the max conf_target

  Line I am adding coverage to https://github.com/bitcoin/bitcoin/blob/master/src/rpc/fees.cpp#LL71C52-L71C52

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d05be124db

Tree-SHA512: dfab075989446e33d1a5ff1a308f1ba1b9f80cce3848fbe4231f69212ceef456a3f2b19365a42123e0397c31893fd9f1fd9973cc00cfbb324386e12ed0e6bccc
2023-09-21 15:50:15 +00:00
Andrew Chow
5027d41988
Merge bitcoin/bitcoin#26366: rpc, test: addnode improv + add test coverage for invalid command
f52cb02f70 doc: make it clear that `node` in `addnode` refers to the node's address (brunoerg)
effd1efefb test: `addnode` with an invalid command should throw an error (brunoerg)
56b27b8487 rpc, refactor: clean-up `addnode` (brunoerg)

Pull request description:

  This PR:

  - Adds test coverage for an invalid `command` in `addnode`.
  - Rename `test_getaddednodeinfo` to `test_addnode_getaddednodeinfo` and its log since this function also tests `addnode` and it doesn't worth to split into 2 ones.
  - Makes it clear in docs that `node` in `addnode` refers to the node's address. It seemed a little weird for me "The node (see getpeerinfo for nodes)", it could mean a lot of things e.g. the node id.
  - Some small improv/clean-up: use `const` where possible, rename some vars, and remove the check for nullance for `command` since it's a non-optional field.

ACKs for top commit:
  achow101:
    ACK f52cb02f70
  jonatack:
    ACK f52cb02f70
  theStack:
    re-ACK f52cb02f70

Tree-SHA512: e4a69e58b784e233463945b4d55a401957f9fe4562c129f59216a44f44fb3221d3449ac578fb35e665ca654c6ade2e741b72c3df78040f7527229c77b6c5b82e
2023-09-21 06:35:16 -04:00
Andrew Chow
8247a8db69
Merge bitcoin/bitcoin#28154: test: refactor: deduplicate segwitv0 ECDSA signing for tx inputs
83d7cfd542 test: refactor: deduplicate segwitv0 ECDSA signing for tx inputs (Sebastian Falbesoner)

Pull request description:

  This PR is a simple follow-up for #28025. It introduces a `signing_input_segwitv0` helper in order to deduplicate the following steps needed to create a segwitv0 ECDSA signature:
  1. calculate the `SegwitV0SignatureHash` with the desired sighash type
  2. create the actual digital signature by calling ECKey.sign_ecdsa on the signature message hash calculated above
  3. put the DER-encoded result (plus sighash byte) at the bottom of the witness stack

ACKs for top commit:
  achow101:
    ACK 83d7cfd542
  pinheadmz:
    code review ACK at 83d7cfd542

Tree-SHA512: b8e55409ddc9ddb14cfc06daeb4730d7750a4632f175f88dcac6ec4d216e71fd4a7eee325a64d6ebba3b33be50bcd30c2de7400f834c01abb67e52840d9823b6
2023-09-20 13:50:15 -04:00
Sebastian Falbesoner
96b3f2dbe4 test: add unit test coverage for Python ECDSA implementation 2023-09-20 18:19:29 +02:00
Greg Sanders
eb8f58f5e4 Add functional test to catch too large vsize packages 2023-09-20 11:33:53 -04:00
Andrew Chow
ff564c75e7
Merge bitcoin/bitcoin#27511: rpc: Add test-only RPC getaddrmaninfo for new/tried table address count
28bac81a34 test: add functional test for getaddrmaninfo (stratospher)
c8eb8dae51 rpc: Introduce getaddrmaninfo for count of addresses stored in new/tried table (stratospher)

Pull request description:

  implements https://github.com/bitcoin/bitcoin/issues/26907. split off from #26988 to keep RPC, CLI discussions separate.

  This PR introduces a new RPC `getaddrmaninfo`which returns the count of addresses in the new/tried table of a node's addrman broken down by network type. This would be useful for users who want to see the distribution of addresses from different networks across new/tried table in the addrman.

  ```jsx
  $ getaddrmaninfo

  Result:
  {                   (json object) json object with network type as keys
    "network" : {     (json object) The network (ipv4, ipv6, onion, i2p, cjdns)
      "new" : n,      (numeric) number of addresses in new table
      "tried" : n,    (numeric) number of addresses in tried table
      "total" : n     (numeric) total number of addresses in both new/tried tables from a network
    },
    ...
  }
  ```

  ### additional context from [original PR](https://github.com/bitcoin/bitcoin/pull/26988)

  1. network coverage tests were skipped because there’s a small chance that addresses from different networks could hash to the same bucket and cause count of different network addresses in the tests to fail. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1137596851.
  2. #26988 uses this RPC in -addrinfo CLI. Slight preference for keeping the RPC hidden since this info will mostly be useful to only super users. see https://github.com/bitcoin/bitcoin/pull/26988#discussion_r1173964808.

ACKs for top commit:
  0xB10C:
    ACK 28bac81a34
  willcl-ark:
    reACK 28bac81a34
  achow101:
    ACK 28bac81a34
  brunoerg:
    reACK 28bac81a34
  theStack:
    Code-review ACK 28bac81a34

Tree-SHA512: 346390167e1ebed7ca5c79328ea452633736aff8b7feefea77460e04d4489059334ae78a3f757f32f5fb7827b309d7186bebab3c3760b3dfb016d564a647371a
2023-09-20 08:25:20 -04:00
Andrew Chow
3966b0a0b6
Merge bitcoin/bitcoin#28472: Remove MemPoolAccept::m_limits to avoid mutating it in package evaluation
ee589d4466 Add regression test for m_limit mutation (Greg Sanders)
275579d8c1 Remove MemPoolAccept::m_limits, only have local copies for carveouts (Greg Sanders)

Pull request description:

  Without remoing it, if we ever call `PreChecks()` multiple times for any reason during any one `MempoolAccept`, subsequent invocations may have incorrect limits, allowing longer/larger chains than should be allowed.

  Currently this is only an issue with `submitpackage`, so this is not exposed on mainnet.

ACKs for top commit:
  achow101:
    ACK ee589d4466
  glozow:
    ACK ee589d4466, nits can be ignored
  ariard:
    Code Review ACK ee589d446

Tree-SHA512: 14cf8edc73e014220def82563f5fb4192d1c2c111829712abf16340bfbfd9a85e2148d723af6fd4995d503dd67232b48dcf8b1711668d25b5aee5eab1bdb578c
2023-09-20 07:49:13 -04:00
Andrew Chow
abe4fedab7
Merge bitcoin/bitcoin#28125: wallet: bugfix, disallow migration of invalid scripts
8e7e3e6149 test: wallet, verify migration doesn't crash for an invalid script (furszy)
1de8a2372a wallet: disallow migration of invalid or not-watched scripts (furszy)

Pull request description:

  Fixing #28057.

  The legacy wallet allows to import any raw script (#28126), without
  checking if it was valid or not. Appending it to the watch-only set.

  This causes a crash in the migration process because we are only
  expecting to find valid scripts inside the legacy spkm.

  These stored scripts internally map to `ISMINE_NO` (same as if they
  weren't stored at all..).

  So we need to check for these special case, and take into account that
  the legacy spkm could be storing invalid not watched scripts.

  Which, in code words, means `IsMineInner()` returning
  `IsMineResult::INVALID` for them.

  Note:
  To verify this, can run the test commit on top of master.
  `wallet_migration.py` will crash without the bugfix commit.

ACKs for top commit:
  achow101:
    ACK 8e7e3e6149

Tree-SHA512: c2070e8ba78037a8f573b05bf6caa672803188f05429adf5b93f9fc1493faedadecdf018dee9ead27c656710558c849c5da8ca5f6f3bc9c23b3c4275d2fb50c7
2023-09-19 13:10:57 -04:00
stratospher
28bac81a34 test: add functional test for getaddrmaninfo 2023-09-19 22:38:56 +05:30
fanquake
53313c49d6
Merge bitcoin/bitcoin#28246: wallet: Use CTxDestination in CRecipient instead of just scriptPubKey
ad0c469d98 wallet: Use CTxDestination in CRecipient rather than scriptPubKey (Andrew Chow)
07d3bdf4eb Add PubKeyDestination for P2PK scripts (Andrew Chow)
1a98a51c66 Allow CNoDestination to represent a raw script (Andrew Chow)
8dd067088d Make WitnessUnknown members private (Andrew Chow)

Pull request description:

  For silent payments, we want to provide a `SilentPaymentsDestination` to be used as the recipient, which requires `CRecipient` to use something other than just the `scriptPubKey` as we cannot know the output script for a silent payment prior to transaction creation. `CTxDestination` seems like the obvious place to add a `SilentPaymentsDestination` as it is our internal representation of an address.

  In order to still allow paying to arbitrary scriptPubKeys (e.g. for data carrier outputs, or the user hand crafted a raw transaction that they have given to `fundrawtransaction`), `CNoDestination` is changed to contain raw scripts.

  Additionally, P2PK scripts are now interpreted as a new `PubKeyDestination` rather than `PKHash`. This results in some things that would have given an address for P2PK scripts to no longer do so. This is arguably more correct.

  `ExtractDestination`'s behavior is slightly changed for the above. It now returns `true` for those destinations that have addresses, so P2PK scripts now result in `false`. Even though it returns false for `CNoDestination`, the script will now be included in that `CNoDestination`.

  Builds on #28244

ACKs for top commit:
  josibake:
    ACK ad0c469d98

Tree-SHA512: ef3f8f3c7284779d9806c77c85b21caf910a79a1f7e7f1b51abcc0d7e074f14e00abf30f625a13075e41d94dad6202c10ddff462c0ee74c2ca4aab585b145a52
2023-09-19 16:48:43 +00:00
Greg Sanders
ee589d4466 Add regression test for m_limit mutation 2023-09-19 09:30:58 -04:00
ismaelsadeeq
2b4edf889a test: check descriptorprocesspsbt return hex encoded tx
Test that if the processed psbt is complete the hex encoded tx
is returned and remove unneccessary rpc call to finalize the
psbt.
2023-09-15 16:48:36 +01:00
Andrew Chow
459272d639
Merge bitcoin/bitcoin#26152: Bump unconfirmed ancestor transactions to target feerate
f18f9ef4d3 Amend bumpfee for inputs with overlapping ancestry (Murch)
2e35e944da Bump unconfirmed parent txs to target feerate (Murch)
3e3e052411 coinselection: Move GetSelectionWaste into SelectionResult (Andrew Chow)
c57889da66 [node] interface to get bump fees (glozow)
c24851be94 Make MiniMinerMempoolEntry fields private (Murch)
ac6030e4d8 Remove unused imports (Murch)
d2f90c31ef Fix calculation of ancestor set feerates in test (Murch)
a1f7d986e0 Match tx names to index in miniminer overlap test (Murch)

Pull request description:

  Includes some commits to address follow-ups from #27021: https://github.com/bitcoin/bitcoin/pull/27021#issuecomment-1554675156

  Reduces the effective value of unconfirmed UTXOs by the fees necessary to bump their ancestor transactions to the same feerate.

  While the individual UTXOs always account for their full ancestry before coin-selection, we can correct potential overestimates with a second pass where we establish the ancestry and bump fee for the whole input set collectively.

  Fixes #9645
  Fixes #9864
  Fixes #15553

ACKs for top commit:
  S3RK:
    ACK f18f9ef4d3
  ismaelsadeeq:
    ACK f18f9ef4d3
  achow101:
    ACK f18f9ef4d3
  brunoerg:
    crACK f18f9ef4d3
  t-bast:
    ACK f18f9ef4d3, I reviewed the latest changes and run e2e tests against eclair, everything looks good 👍

Tree-SHA512: b65180c4243b1f9d13c311ada7a1c9f2f055d530d6c533b78c2068b50b8c29ac1321e89e85675b15515760d4f1b653ebd9da77b37c7be52d9bc565a3538f0aa6
2023-09-14 16:08:37 -04:00
Andrew Chow
541976b42e
Merge bitcoin/bitcoin#27850: test: Add unit & functional test coverage for blockstore
de8f9123af test: cover read-only blockstore (Matthew Zipkin)
5c2185b3b6 ci: enable chattr +i capability inside containers (Matthew Zipkin)
e573f24202 unit test: add coverage for BlockManager (Matthew Zipkin)

Pull request description:

  This PR adds unit and functional tests to cover the behavior described in #2039. In particular, that bitcoind will crash on startup if a reindex is requested but the `blk` files are read-only. Eventually this behavior can be updated with https://github.com/bitcoin/bitcoin/pull/27039. This PR just commits the test coverage from #27039 as suggested in https://github.com/bitcoin/bitcoin/pull/27039#issuecomment-1584915782

ACKs for top commit:
  jonatack:
    ACK de8f9123af modulo suggestions in https://github.com/bitcoin/bitcoin/pull/27850#discussion_r1319010039, tested on macOS, but not on Linux for the Linux-related change in the last push
  achow101:
    ACK de8f9123af
  MarcoFalke:
    lgtm ACK de8f9123af 📶

Tree-SHA512: b9bd684035dcea11c901b649fc39f397a2155a9a8459f3348e67947e387e45312fddeccb52981aef486f8a31deebb5356a7901c1bb94b78f82c24192a369af73
2023-09-14 13:21:14 -04:00
Matthew Zipkin
de8f9123af
test: cover read-only blockstore
Co-authored-by: Andrew Chow <github@achow101.com>
2023-09-14 12:02:01 -04:00
Murch
f18f9ef4d3
Amend bumpfee for inputs with overlapping ancestry
At the end of coin selection reduce the fees by the difference between
the individual bump fee estimates and the collective bump fee estimate.
2023-09-13 15:46:59 -04:00
Murch
2e35e944da
Bump unconfirmed parent txs to target feerate
When a transaction uses an unconfirmed input, preceding this commit it
would not consider the feerate of the parent transaction. Given a parent
transaction with a lower ancestor feerate, this resulted in the new
transaction's ancestor feerate undershooting the target feerate.

This commit changes how we calculate the effective value of unconfirmed UTXOs.
The effective value of unconfirmed UTXOs is decreased by the fee
necessary to bump its ancestry to the target feerate. This also impacts
the calculation of the waste metric: since the estimate for the current
fee is increased by the bump fees, unconfirmed UTXOs current fees appear less
favorable compared to their unchanged long term fees.

This has one caveat: if multiple UTXOs have overlapping ancestries, each
of their individual estimates will account for bumping all ancestors.
2023-09-13 14:33:58 -04:00
fanquake
f1a9fd627b
Merge bitcoin/bitcoin#28251: validation: fix coins disappearing mid-package evaluation
32c1dd1ad6 [test] mempool coins disappearing mid-package evaluation (glozow)
a67f460c3f [refactor] split setup in mempool_limit test (glozow)
d08696120e [test framework] add ability to spend only confirmed utxos (glozow)
3ea71feb11 [validation] don't LimitMempoolSize in any subpackage submissions (glozow)
d227b7234c [validation] return correct result when already-in-mempool tx gets evicted (glozow)
9698b81828 [refactor] back-fill results in AcceptPackage (glozow)
8ad7ad3392 [validation] make PackageMempoolAcceptResult members mutable (glozow)
03b87c11ca [validation] add AcceptSubPackage to delegate Accept* calls and clean up m_view (glozow)
3f01a3dab1 [CCoinsViewMemPool] track non-base coins and allow Reset (glozow)
7d7f7a1189 [policy] check for duplicate txids in package (glozow)

Pull request description:

  While we are evaluating a package, we split it into "subpackages" for evaluation (currently subpackages all have size 1 except the last one). If a subpackage has size 1, we may add a tx to mempool and call `LimitMempoolSize()`, which evicts transactions if the mempool gets full. We handle the case where the just-submitted transaction is evicted immediately, but we don't handle the case in which a transaction from a previous subpackage (either just submitted or already in mempool) is evicted. Mainly, since the coins created by the evicted transaction are cached in `m_view`, we don't realize the UTXO has disappeared until `CheckInputsFromMempoolAndCache` asserts that they exist. Also, the returned `PackageMempoolAcceptResult` reports that the transaction is in mempool even though it isn't anymore.

  Fix this by not calling `LimitMempoolSize()` until the very end, and editing the results map with "mempool full" if things fall out.

  Pointed out by instagibbs in faeed687e5 on top of the v3 PR.

ACKs for top commit:
  instagibbs:
    reACK 32c1dd1ad6

Tree-SHA512: 61e7f69db4712e5e5bfa27d037ab66bdd97f1bf60a8d9ffb96adb1f0609af012c810d681102ee5c7baec7b5fe8cb7c304a60c63ccc445d00d86a2b7f0e7ddb90
2023-09-13 17:51:00 +01:00
glozow
32c1dd1ad6 [test] mempool coins disappearing mid-package evaluation
Test for scenario(s) outlined in PR 28251.
Test what happens when a package transaction spends a mempool coin which
is fetched and then disappears mid-package evaluation due to eviction or
replacement.
2023-09-13 16:14:18 +01:00
glozow
a67f460c3f [refactor] split setup in mempool_limit test
We want to be able to re-use fill_mempool so that none of the tests
affect each other.

Change the logs from info to debug because they are otherwise repeated
many times in the test output.
2023-09-13 16:14:18 +01:00
glozow
d08696120e [test framework] add ability to spend only confirmed utxos
Useful to ensure that the topologies of packages/transactions are as
expected, preventing bugs caused by having unexpected mempool ancestors.
2023-09-13 16:14:18 +01:00
glozow
7d7f7a1189 [policy] check for duplicate txids in package
Duplicates of normal transactions would be found by looking for
conflicting inputs, but this doesn't catch identical empty transactions.
These wouldn't be valid but exiting early is good and AcceptPackage's
result sanity checks assume non-duplicate transactions.
2023-09-13 16:14:17 +01:00
Andrew Chow
8f9c74cb11
Merge bitcoin/bitcoin#28414: wallet rpc: return final tx hex from walletprocesspsbt if complete
2e249b9227 doc: add release note for PR #28414 (Matthew Zipkin)
4614332fc4 test: remove unnecessary finalizepsbt rpc calls (ismaelsadeeq)
e3d484b603 wallet rpc: return final tx hex from walletprocesspsbt if complete (Matthew Zipkin)

Pull request description:

  See https://github.com/bitcoin/bitcoin/pull/28363#discussion_r1315753887

  `walletprocesspsbt` currently returns a base64-encoded PSBT and a boolean indicating if the tx is "complete". If it is complete, the base64 PSBT can be finalized with `finalizepsbt` which returns the hex-encoded transaction suitable for `sendrawtransaction`.

  With this patch, `walletprocesspsbt` return object will ALSO include the broadcast-able hex string if the tx is already final. This saves users the extra step of calling `finalizepsbt` assuming they have already inspected and approve the transaction from earlier steps.

ACKs for top commit:
  ismaelsadeeq:
    re ACK 2e249b9227
  BrandonOdiwuor:
    re ACK 2e249b9
  Randy808:
    Tested ACK 2e249b9227
  achow101:
    ACK 2e249b9227
  ishaanam:
    ACK 2e249b9227

Tree-SHA512: 229c1103265a9b4248f080935a7ad5607c3be3f9a096a9ab6554093b2cd8aa8b4d1fa55b1b97d3925ba208dbc3ccba4e4d37c40e1491db0d27ba3d9fe98f931e
2023-09-12 12:28:13 -04:00
Andrew Chow
07d3bdf4eb Add PubKeyDestination for P2PK scripts
P2PK scripts are not PKHash destinations, they should have their own
type.

This also results in no longer showing a p2pkh address for p2pk outputs.
However for backwards compatibility, ListCoinst will still do this
conversion.
2023-09-12 12:14:31 -04:00
Anthony Towns
971bae9174 rpc: Deprecate rpcserialversion=0 2023-09-11 17:21:53 +10:00
fanquake
c5a63ea56f
Merge bitcoin/bitcoin#27944: test: various USDT functional test cleanups (27831 follow-ups)
9f55773a37 test: refactor: usdt_mempool: store all events (stickies-v)
bc43270450 test: refactor: remove unnecessary nonlocal (stickies-v)
326db63a68 test: log sanity check assertion failures (stickies-v)
f5525ad680 test: store utxocache events (stickies-v)
f1b99ac94f test: refactor: deduplicate handle_utxocache_* logic (stickies-v)
ad90ba36bd test: refactor:  rename inbound to is_inbound (stickies-v)
afc0224cdb test: refactor: remove unnecessary blocks_checked counter (stickies-v)

Pull request description:

  Various cleanups to the USDT functional tests, largely (but not exclusively) follow-ups to https://github.com/bitcoin/bitcoin/pull/27831#pullrequestreview-1491438045. Except for slightly different logging behaviour in "test: store utxocache events" and "test: log sanity check assertion failures", this is a refactor PR, removing unnecessary code and (imo) making it more readable and maintainable.

  The rationale for each change is in the corresponding commit message.

  Note: except for "test: store utxocache events" (which relies on its parent, and I separated into two commits because we may want the parent but not the child), all commits are stand-alone and I'm okay with dropping one/multiple commits if they turn out to be controversial or undesired.

ACKs for top commit:
  0xB10C:
    ACK 9f55773a37. Reviewed the code and ran the USDT interface tests. I stepped through the commits and think all changes are reasonable.

Tree-SHA512: 6c37a0265b6c26d4f9552a056a690b8f86f7304bd33b4419febd8b17369cf6af799cb87c16df35d0c2a1b839ad31de24661d4384eafa88816c2051c522fd3bf5
2023-09-10 14:10:16 +01:00
fanquake
0354206a30
Merge bitcoin/bitcoin#28412: test: remove unused variables in p2p_invalid_block
3eb03803c4 test: remove unused variables in `p2p_invalid_block` (brunoerg)

Pull request description:

ACKs for top commit:
  stickies-v:
    ACK 3eb03803c4

Tree-SHA512: eadae1eb323e5562d1ea0aed43ebf0145f0fdbb6cd6d4646bbf1ca89f384820e7b9cb69f0bb04a949e9f8983a879aee8387d6f7ac3d4e4ea027f8892e656fb98
2023-09-07 16:10:14 +01:00
fanquake
cf421820f5
Merge bitcoin/bitcoin#28409: test: Combine sync_send_with_ping and sync_with_ping
fae0b21e6c test: Combine sync_send_with_ping and sync_with_ping (MarcoFalke)

Pull request description:

  This reduces bloat, complexity, and makes tests less fragile to intermittent failures, see https://github.com/bitcoin/bitcoin/pull/27675#discussion_r1315648343.

  This should not cause any noticeable slowdown, or may even be faster, because active polling will be done at most once.

ACKs for top commit:
  glozow:
    Concept ACK fae0b21e6c
  theStack:
    ACK fae0b21e6c 🏓

Tree-SHA512: 6c543241a7b85458dc7ff6a6203316b80a6227d83d38427e74f53f4c666a882647a8a221e5259071ee7bb5dfd63476fb03c9b558a1ea546734b14728c3c619ba
2023-09-06 17:08:33 +01:00
ismaelsadeeq
4614332fc4
test: remove unnecessary finalizepsbt rpc calls 2023-09-06 11:52:19 -04:00
Matthew Zipkin
e3d484b603
wallet rpc: return final tx hex from walletprocesspsbt if complete 2023-09-05 09:14:32 -04:00
brunoerg
3eb03803c4 test: remove unused variables in p2p_invalid_block 2023-09-05 09:09:45 -03:00
fanquake
ecab855838
Merge bitcoin/bitcoin#28195: blockstorage: Drop legacy -txindex check
fae405556d scripted-diff: Rename CBlockTreeDB -> BlockTreeDB (MarcoFalke)
faf63039cc Fixup style of moved code (MarcoFalke)
fa65111b99 move-only: Move CBlockTreeDB to node/blockstorage (MarcoFalke)
fa8685597e index: Drop legacy -txindex check (MarcoFalke)
fa69148a0a scripted-diff: Use blocks_path where possible (MarcoFalke)

Pull request description:

  The only reason for the check was to print a warning about an increase in storage use. Now that 22.x is EOL and everyone should have migrated (or decided to not care about storage use), remove the check.

  Also, a move-only commit is included. (Rebased from https://github.com/bitcoin/bitcoin/pull/22242)

ACKs for top commit:
  TheCharlatan:
    ACK fae405556d, though I lack historical context to really judge the second commit fa8685597e.
  stickies-v:
    ACK fae405556d

Tree-SHA512: 9da8f48767ae52d8e8e21c09a40c949cc0838794f1856cc5f58a91acd3f00a3bca818c8082242b3fdc9ca5badb09059570bb3870850d3807b75a8e23b5222da1
2023-09-05 11:37:35 +01:00
MarcoFalke
fae0b21e6c
test: Combine sync_send_with_ping and sync_with_ping 2023-09-05 12:11:10 +02:00
fanquake
c004ba48f4
Merge bitcoin/bitcoin#28386: test: remove fixed timeouts from feature_config_args
fbcacd4cf0 test: remove fixed timeouts from feature_config_args (Martin Zumsande)

Pull request description:

  Fixes #28290

  These fixed timeouts aren't affected by the `timeout_factor` option and can therefore cause timeouts in slow environments.
  They are also unnecessary for the test because they measure the wrong thing:
  While there is an internal waiting time of 60s within `ThreadOpenConnections` (beginning only when that thread is started) for fixed seeds querying, the timeouts here don't measure that but the time from startup until a debug log message is encountered, during which many other things happen in init, so they don't make much sense to me in the first place.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK fbcacd4cf0

Tree-SHA512: 7bb3b7db2f9666b1929ffb7773c838ee98b0845569428e5d00ecf5234973d534c4f474e213896c71baabd6096a79347bd21b41a17b130053049714eb8a447c79
2023-09-05 09:59:33 +01:00
fanquake
fb619e1bc3
Merge bitcoin/bitcoin#28396: test: p2p: check that getaddr msgs are only responded once per connection
668aa6af8d test: p2p: check that `getaddr` msgs are only responded once per connection (Sebastian Falbesoner)

Pull request description:

  This simple PR adds missing test coverage for ignoring repeated `getaddr` requests (introduced in #7856, commit 66b07247a7):
  6f03c45f6b/src/net_processing.cpp (L4642-L4648)

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 668aa6af8d
  brunoerg:
    crACK 668aa6af8d

Tree-SHA512: edcdc6501c684fb41911e393f55ded9b044cd2f92918877eca152edd5a4287d1a9d57ae999f1cb42185eae00c3a0af411fcb9bcd5b990ef48849c3834b141584
2023-09-05 09:58:50 +01:00
fanquake
337d6f35a2
Merge bitcoin/bitcoin#28291: rpc: removed StrFormatInternalBug quote delimitation
6e8f6468cb removed StrFormatInternalBug quote delimitation (Reese Russell)

Pull request description:

  This PR rectifies an unnecessary set of quotes delimiting the contents of  ```StrFormatInternalBug```. This is a follow up to MarcoFalke https://github.com/bitcoin/bitcoin/pull/28123#discussion_r1297191493.  The method of action was to remove the escaped quotes that were a part of strprintf. A single functional test case was modified to reflect the new output format.

  ```STR_INTERNAL_BUG``` was applied to https://github.com/bitcoin/bitcoin/pull/28123 in ```std::string RPCArg::ToString(const bool oneline)``` in ```rpc/util.cpp```

  The results can be seen below.

  Previously
  ![image](https://github.com/bitcoin/bitcoin/assets/3104223/53f9ea59-317f-4c62-9fc1-04255eeb4641)

  This PR
  ![image](https://github.com/bitcoin/bitcoin/assets/3104223/5c6a3110-f1f3-4b3c-8e8a-9c8f1c3176e7)

  Additional context can be found here.
  https://github.com/bitcoin/bitcoin/pull/28123#discussion_r1271871716

  Thank you.

ACKs for top commit:
  MarcoFalke:
    review ACK 6e8f6468cb
  stickies-v:
    ACK 6e8f6468cb

Tree-SHA512: 35317e31a527630495b566407e37db9941dab7f81cfaeb1ea3309683c48e4273284645ad615f73e646a137b4f2ae35933603e9182a7dbdd22cac98d038c491dc
2023-09-05 11:41:02 +03:00
Sebastian Falbesoner
668aa6af8d test: p2p: check that getaddr msgs are only responded once per connection 2023-09-03 14:19:41 +02:00
Martin Zumsande
fbcacd4cf0 test: remove fixed timeouts from feature_config_args
They cannot be scaled by the timeout_factor option and can
therefore cause timeouts in slow environments.
They are also not necessary for the test, since they measure time
frome startup until a debug message is encountered, which
is not restricted to 1 minute by any internal logic within bitcoind.
2023-09-01 14:48:55 -04:00
Andrew Chow
5666966dff
Merge bitcoin/bitcoin#28364: log: log wtxids when possible, add TXPACKAGES category
a3b55c94b9 [doc] move comment about AlreadyHaveTx DoS score to the right place (glozow)
3b8c17838a [log] add more logs related to orphan handling (glozow)
51b3275cd1 [log] add category TXPACKAGES for orphanage and package relay (glozow)
a33dde1e41 [log] include wtxid in tx {relay,validation,orphanage} logging (glozow)

Pull request description:

  This was taken from #28031 (see #27463 for project tracking).

  - Log wtxids in addition to txids when possible. This allows us to track the fate of a transaction from inv to mempool accept/reject through logs.
  - Additional orphan-related logging to make testing and debugging easier. Suggested in https://github.com/bitcoin/bitcoin/pull/28031#pullrequestreview-1531022386 and https://github.com/bitcoin/bitcoin/pull/28031#discussion_r1269622220
  - Add `TXPACKAGES` category for logging.
  - Move a nearby comment block that was in the wrong place.

ACKs for top commit:
  instagibbs:
    reACK a3b55c94b9
  achow101:
    ACK a3b55c94b9
  brunoerg:
    crACK a3b55c94b9
  mzumsande:
    Code review ACK a3b55c94b9

Tree-SHA512: 21884ef7c2ea2fd006e715574a9dd3e6cbbe8f82d62c6187fe1d39aad5a834051203fda5f355a06ca40c3e2b9561aec50d7c922a662b1edc96f7b552c9f4b24d
2023-08-31 12:55:38 -04:00
fanquake
1348454d82
Merge bitcoin/bitcoin#28352: test: Support powerpc64le in get_previous_releases.py
faf7e69862 test: Support powerpc64le in get_previous_releases.py (MarcoFalke)

Pull request description:

  To test: `test/get_previous_releases.py -b -t /tmp/prev_releases v22.0`

  On master: `Not sure which binary to download for powerpc64le-unknown-linux-gnu`
  Here: (pass)

ACKs for top commit:
  fanquake:
    ACK faf7e69862

Tree-SHA512: 33d9348f99e0d3924a6a5cba8833ec9e413e80167012b557922f3628069dabd555b02f98a6bfd0eb80e2bbbcdb50865b7bca216e1d080b1546ee4812abda4bc2
2023-08-30 16:16:44 +01:00
glozow
a33dde1e41 [log] include wtxid in tx {relay,validation,orphanage} logging 2023-08-29 16:41:22 +01:00
MarcoFalke
faf7e69862
test: Support powerpc64le in get_previous_releases.py 2023-08-27 22:28:57 +02:00
Jon Atack
6c008a2006 script: replace deprecated pkg_resources with importlib.metadata
in our python linter:

```
./test/lint/lint-python.py:12: DeprecationWarning: pkg_resources is deprecated as an API.
  See https://setuptools.pypa.io/en/latest/pkg_resources.html
  import pkg_resources
```

The importlib.metadata library was added in Python 3.8, which is currently our
minimum-supported Python version.

For more details about importlib.metadata, see https://docs.python.org/3/library/importlib.metadata.html
2023-08-26 16:42:02 -06:00