Commit graph

4624 commits

Author SHA1 Message Date
MarcoFalke
33337eb860
test: Fix TypeError in wait_for_debug_log
Traceback:

print_log = " - " + "\n - ".join(log.splitlines())
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: sequence item 0: expected str instance, bytes found
2023-03-20 11:54:06 +01:00
fanquake
0973018067
Merge bitcoin/bitcoin#27265: test: check that sigop limit also affects ancestor/descendant size (27171 follow-up)
6d24d1ef2b test: check that sigop limit also affects ancestor/descendant size (Sebastian Falbesoner)

Pull request description:

  This is a follow-up to #27171, adding a check that the sigop-limit vsize logic is also respected for {ancestor,descendant}size calculation (as suggested in https://github.com/bitcoin/bitcoin/pull/27171#pullrequestreview-1331143909). For simplicity, we use a one-parent-one-child cluster here and only check for the case that the sigop-limit equivalent size is larger than the serialized vsize.

ACKs for top commit:
  glozow:
    code review ACK 6d24d1ef2b, thanks for taking!

Tree-SHA512: dc65e455d06cfef1f1d6a53b959f99ec1ca3fe51c98dc1ed5826614b5619773d34aff0171c43a0ede4fd45605b2eb7a9278e027196128bb7ad8586b859f1cf70
2023-03-19 12:11:47 +00:00
ishaanam
6e9f8bb050 rpc, tests: in utxoupdatepsbt also look for the transaction in the txindex
Previously only the segwit utxos being spent by the psbt were looked for and
added to the psbt. Now, the full transaction corresponding to each of these
utxos (legacy and segwit) is looked for in the txindex and mempool and added
to the psbt. If txindex is disabled and the transaction is not in the mempool,
then we fall back to getting just the utxo (if segwit) from the utxo set.
2023-03-18 20:58:15 -04:00
fanquake
50171df26c
Merge bitcoin/bitcoin#27212: test: Make the unlikely race in p2p_invalid_messages impossible
fa1eb0ecae test: Make the unlikely race in p2p_invalid_messages impossible (MarcoFalke)

Pull request description:

  After `add_p2p_connection` both sides have the verack processed.
  However the pong from conn in reply to the ping from the node has not
  been processed and recorded in totalbytesrecv.
  Flush the pong from conn by sending a ping from conn.

  This should make the unlikely race impossible.

ACKs for top commit:
  mzumsande:
    ACK fa1eb0ecae
  pinheadmz:
    ACK fa1eb0ecae

Tree-SHA512: 44166587572e8c0c758cac460fcfd5cf403b2883880128b13dc62e7f74ca5cb8f145bb68a903df177ff0e62faa360f913fd409b009d4cd1360f1f4403ade39ae
2023-03-17 14:55:48 +00:00
MarcoFalke
fa1eb0ecae
test: Make the unlikely race in p2p_invalid_messages impossible 2023-03-17 09:25:32 +01:00
Andrew Chow
db03248070
Merge bitcoin/bitcoin#27199: test: fix race condition in encrypted wallet rescan tests
dbeca792a9 test: fix race condition in encrypted wallet rescan tests (ishaanam)

Pull request description:

  This fixes https://github.com/bitcoin/bitcoin/pull/26347#discussion_r1123340738

ACKs for top commit:
  MarcoFalke:
    nice re-ACK dbeca792a9  🚜
  achow101:
    ACK dbeca792a9

Tree-SHA512: 7127254ac0274b5bc8ba0242736e77464acbf1f6e3f6af098b4e47742124c336cd67dffdb385e1e8dbd3a8ae74acd073c99e82fa35c44a615fd7d22b29a0daf7
2023-03-16 17:28:39 -04:00
Greg Sanders
d7cc503843 Fix fund transaction case at 0-value, 0-fee 2023-03-16 14:58:41 -04:00
Andrew Chow
09e86d7a1a
Merge bitcoin/bitcoin#27200: test: psbt: check non-witness UTXO removal for segwit v1 input
3dd2f6461b test: psbt: check non-witness UTXO removal for segwit v1 input (Sebastian Falbesoner)
dd78e3fa43 test: speedup rpc_psbt.py by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)
e194e3e93d test: PSBT: eliminate magic numbers for global unsigned tx key (0) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for dropping non-witness UTXOs from PSBTs for segwit v1+ inputs (see commit 103c6fd279). The formerly [disabled](4600479058) method `test_utxo_conversion` is re-enabled and adapted to spend a Taproot (`bech32m`) instead of a wrapped SegWit (`p2sh-segwit`) output. Note that in contrast to the original test, we have to add the non-witness UTXO manually here using the test framework's PSBT module, since the constructing node knows that the output is segwit v1 and hence doesn't add the non-witness UTXO in the first place (see also [BIP371]( https://github.com/bitcoin/bips/blob/master/bip-0371.mediawiki#user-content-UTXO_Types)).

  I strongly assume that most wallets would behave the same as Bitcoin Core here and wouldn't create PSBTs with non-witness UTXOs for Taproot inputs, but it's still good to test everything works as expected if it's still done and that the non-witness UTXO is simply dropped in that case.

  The first two commits contain a small refactor (magic number elimination in PSBT module) and test speedup of ~2-3x (using whitelisting peers / immediate tx relay).

ACKs for top commit:
  achow101:
    ACK 3dd2f6461b
  instagibbs:
    ACK 3dd2f6461b

Tree-SHA512: b8d7f7ea5d7d21def024b70dfca61991cc96a4193be8857018b4d7cf3ca1465d185619fd4a77623803d9da309aa489c53273e9b7683d970ce12e2399b5b50031
2023-03-16 14:48:10 -04:00
Andrew Chow
ebb15ea75a
Merge bitcoin/bitcoin#26207: rest: add verbose and mempool_sequence query params for mempool/contents
1ff5d61dfd doc: add mempool/contents rest verbose and mempool_sequence args (Andrew Toth)
52a31dccc9 tests: mempool/contents verbose and mempool_sequence query params tests (Andrew Toth)
a518fff0f2 rest: add verbose and mempool_sequence query params for mempool/contents (Andrew Toth)

Pull request description:

  The verbose mempool json response can get very large. This adds an option to return the non-verbose response of just the txids. It is identical to the rpc response so the diff here is minimal. This also adds the mempool_sequence parameter for rpc consistency. Verbose defaults to true to remain backwards compatible.

  It uses query parameters to be compatible with the efforts in https://github.com/bitcoin/bitcoin/issues/25752.

ACKs for top commit:
  achow101:
    ACK 1ff5d61dfd
  stickies-v:
    re-ACK [1ff5d61](1ff5d61dfd)
  pablomartin4btc:
    tested ACK 1ff5d61dfd.

Tree-SHA512: 1bf08a7ffde2e7db14dc746e421feedf17d84c4b3f1141e79e36feb6014811dfde80e1d8dbc476c15ff705de2d3c967b3081dcd80536d76b7edf888f1a92e9d1
2023-03-15 19:39:30 -04:00
ishaanam
dbeca792a9 test: fix race condition in encrypted wallet rescan tests 2023-03-15 17:27:57 -04:00
Sebastian Falbesoner
6d24d1ef2b test: check that sigop limit also affects ancestor/descendant size 2023-03-15 19:39:25 +01:00
glozow
f50fb178c3
Merge bitcoin/bitcoin#27235: Avoid integer overflow in CheckDiskSpace
05eeba2c5f [test] Add manual prune startup test case (dergoegge)
4517419628 [util] Avoid integer overflow in CheckDiskSpace (dergoegge)

Pull request description:

  Starting a fresh node with `-prune=1` causes an integer overflow to happen in `CheckDiskSpace` ([here](f7bdcfc83f/src/init.cpp (L1633-L1648))) because `nPruneTarget` is to the max `uint64_t` value.
  ```
   node1 stderr util/system.cpp:138:51: runtime error: unsigned integer overflow: 52428800 + 18446744073709551615 cannot be represented in type 'unsigned long'
      #0 0x564a482b5088 in CheckDiskSpace(fs::path const&, unsigned long) src/./src/util/system.cpp:138:51
      #1 0x564a4728dc59 in AppInitMain(node::NodeContext&, interfaces::BlockAndHeaderTipInfo*) src/./src/init.cpp:1639:14
      #2 0x564a47256e6a in AppInit(node::NodeContext&, int, char**) src/./src/bitcoind.cpp:221:43
      #3 0x564a47256087 in main src/./src/bitcoind.cpp:265:13
      #4 0x7fcb7cbffd8f  (/lib/x86_64-linux-gnu/libc.so.6+0x29d8f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
      #5 0x7fcb7cbffe3f in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x29e3f) (BuildId: 69389d485a9793dbe873f0ea2c93e02efaa9aa3d)
      #6 0x564a471957f4 in _start (/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/src/bitcoind+0xca07f4) (BuildId: 035cb22302d37317a630900a15a26ecb326d395c)
  SUMMARY: UndefinedBehaviorSanitizer: unsigned-integer-overflow util/system.cpp:138:51 in
  ```

  I think side stepping the overflow for this specific case, is better than adding an exception to the UB suppresions file.

ACKs for top commit:
  MarcoFalke:
    ACK 05eeba2c5f 🥝
  john-moffett:
    ACK 05eeba2c5f

Tree-SHA512: 1d8e6bcb49818139f04b5ab2cbef7f9b422bf0c38a804cd532b6bd0ba4c4fd07f959ba977e59896343f213086c8ecc48180f50d006638dc84649c66ec379d58a
2023-03-13 17:01:48 +00:00
glozow
73a9892bce
Merge bitcoin/bitcoin#26514: Improve address decoding errors
962a0930e6 Improve address decoding errors (Aurèle Oulès)

Pull request description:

  Attempt to fix #21741.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 962a0930e6
  davidgumberg:
    ACK 962a0930e6
  1440000bytes:
    utACK 962a0930e6

Tree-SHA512: 6f216eeaeccf6bfdf0730d38835fdf26c935a5e1fc35006660393a9ad76bf38c85340f0f20e92f87840463d83d891d9714cfad313aab301a16bb8efa4490df06
2023-03-13 16:57:31 +00:00
fanquake
f088949fcf
Merge bitcoin/bitcoin#27221: test: Default timeout factor to 4 under --valgrind
fa27cf4cc7 test: Default timeout factor to 4 under --valgrind (MarcoFalke)

Pull request description:

  valgrind will incur a slowdown of at least 2, so increase the default timeout factor.

  This should reduce the number of reported issues. See also https://github.com/bitcoin/bitcoin/issues/27112#issuecomment-1455762739

ACKs for top commit:
  fanquake:
    ACK fa27cf4cc7 - I still see at least one actual issue when running the functional tests under `--valgrind` (outside the CI system), but will follow up separately with that. Increasing the timeout here seems fine.

Tree-SHA512: 4467559a3bfd98f5735f300f6ed54c68f951191d65a2b1294d71d72cc5d0864964de562d5dfa0a4855fc541ccb269a538b7aeb3d408d2d012a5369513397c395
2023-03-13 16:15:04 +01:00
dergoegge
05eeba2c5f [test] Add manual prune startup test case 2023-03-13 13:10:11 +01:00
furszy
f3221d373a
test: add wallet too-long-mempool-chain error coverage 2023-03-10 11:29:37 -03:00
fanquake
3e7dd4ff33
Merge bitcoin/bitcoin#27171: test: add coverage for sigop limit policy (-bytespersigop setting)
89cd20cbed test: add coverage for sigop limit policy (`-bytespersigop` setting) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `-bytespersigop` option, which determines how pre-taproot signature operations (OP_CHECKSIG{VERIFY}, OP_CHECKMULTIGSIG{VERIFY}) affect fee handling calculations. The setting was introduced in PR #7081 for mitigating the [sigop spam attack](https://bitcointalk.org/index.php?topic=1166928.0); the initial implementation rejected txs exceeding the limit, but was changed in #8365 later to account for higher sizes in the mempool (i.e. exceeding the sigop limit is possible, but has to be compensated by higher fees).

  For each combination of `-bytespersigop` setting and sigops count, the test first creates a P2WSH spending transaction with a witness script that puts sigops in a non-executing branch (OP_FALSE OP_IF OP_CHECKMULTISIG ... OP_CHECKSIG ... OP_ENDIF). This tx is then bumped up to reach exactly the _sig-op limit equivalent vsize_ by padding its datacarrier output. Based on that, increasing the tx's vsize should still reflect a vsize increase in the mempool, while a decrease of the tx's vsize should lead to the mempool treating the tx's vsize to be the _sig-op limit equivalent vsize_, since the limit was exceeded.

  I assume that this parameter is almost never set explicitly by users (also it is not relevant for taproot spends), but it doesn't hurt to have a test for it. See also https://bitcoin.stackexchange.com/a/87958 for another explanation.

ACKs for top commit:
  glozow:
    light review ACK 89cd20cbed
  MarcoFalke:
    nice ACK 89cd20cbed  📁

Tree-SHA512: 06998ce93bf9d5ce6143db2996a43f13990c415f97afe684227ad469349e73952bf4f6c871c1e6349e07606f4d45db64408848873a86a89481cdca5a134e5e60
2023-03-10 14:34:34 +01:00
fanquake
6f5eb7a39e
Merge bitcoin/bitcoin#27226: test: Use self.wait_until over wait_until_helper
faa671591f test: Use self.wait_until over wait_until_helper (MarcoFalke)

Pull request description:

  `wait_until_helper` is a "private" helper, not intended to be used directly, because it doesn't scale the timeout with the timeout factor. Fix this by replacing it with a call to `self.wait_until`, which does the scaling.

ACKs for top commit:
  theStack:
    Code-review ACK faa671591f

Tree-SHA512: 70705f309f83ffd6ea5d090218195d05b868624d909106863372f861138b5a70887070b25beb25044ae1b44250345e45c9cc11191ae7aeca2ad37801a0f62f61
2023-03-10 14:26:06 +01:00
fanquake
6e662a8985
Merge bitcoin/bitcoin#23813: Add test and docs for getblockfrompeer with pruning
fe329dc936 test: Add test for getblockfrompeer on pruned nodes (Fabian Jahr)
cd761e6b2c rpc: Add note on guarantees to getblockfrompeer (Fabian Jahr)

Pull request description:

  These are additions to `getblockfrompeer` that I already [suggested on the original PR](https://github.com/bitcoin/bitcoin/pull/20295#pullrequestreview-817157738).

  The two commits do the following:
  1. Add a test for `getblockfrompeer` usage on pruned nodes. This is important because many use-cases for `getblockfrompeer` are in a context of a pruned node.
  2. Add some information on how long the users of pruned nodes can expect the block to be available after they have used the RPC. I think the behavior is not very intuitive for users and I would not be surprised if users expect the block to be available indefinitely.

ACKs for top commit:
  Sjors:
    re-utACK fe329dc936
  MarcoFalke:
    review ACK fe329dc936 🍉
  stratospher:
    ACK  fe329dc.
  brunoerg:
    re-ACK fe329dc936

Tree-SHA512: a686bd8955d9c3baf365db384e497d6ee1aa9ce2fdb0733fe6150f7e3d94bae19d55bc1b347f1c9f619e749e18b41a52b9f8c0aa2042dd311a968a4b5d251fac
2023-03-10 14:25:00 +01:00
MarcoFalke
fa18504d57
rpc: Add submit option to generateblock 2023-03-10 10:39:14 +01:00
fanquake
710fd571ff
Merge bitcoin/bitcoin#26996: test: Flatten miniwallet array and remove random fee in longpoll
fa0abcdafe test: Flatten miniwallet array and remove random fee in longpoll (MarcoFalke)

Pull request description:

  * Using a single MiniWallet is enough.
  * A random fee isn't needed either.

ACKs for top commit:
  theStack:
    re-ACK fa0abcdafe

Tree-SHA512: 77b99885b3f0d325d067838122114be57ec999ebc82912de6a22c33e2ba28a341c5e053c5bbc424b9922c2616562289a57c7156bd3b431d779182c2e472da59c
2023-03-08 18:32:43 +01:00
Andrew Chow
1ff135ca7f
Merge bitcoin/bitcoin#26194: rpc, wallet: use the same next_index key in listdescriptors and importdescriptors
b082f28101 rpc, wallet: use the same `next_index` in listdescriptors and importdescriptors (w0xlt)

Pull request description:

  Currently `listdescriptors` RPC uses `next` key to represent `WalletDescriptor::next_index` while `importdescriptors` uses `next_index`. This creates two different descriptor formats.

  This  PR changes `listdescriptors` to use the same key as `importdescriptors`.

ACKs for top commit:
  achow101:
    ACK b082f28101
  aureleoules:
    reACK b082f28101

Tree-SHA512: c29ec59051878e614d749ed6dc85e5c14ad00db0e8fcbce3f5066d1aae85ef07ca70f02920299e48d191b7387024fe224b0054c4191a5951cb805106f7b8e37b
2023-03-08 12:15:31 -05:00
MarcoFalke
faa671591f
test: Use self.wait_until over wait_until_helper 2023-03-08 11:31:56 +01:00
MarcoFalke
fa0abcdafe
test: Flatten miniwallet array and remove random fee in longpoll 2023-03-07 17:47:28 +01:00
MarcoFalke
fa27cf4cc7
test: Default timeout factor to 4 under --valgrind 2023-03-07 17:04:27 +01:00
Sebastian Falbesoner
89cd20cbed test: add coverage for sigop limit policy (-bytespersigop setting) 2023-03-07 04:23:33 +01:00
Andrew Chow
86bacd75e7
Merge bitcoin/bitcoin#26742: http: Track active requests and wait for last to finish - 2nd attempt
60978c8080 test: Reduce extended timeout on abortnode test (Fabian Jahr)
660bdbf785 http: Release server before waiting for event base loop exit (João Barbosa)
8c6d007c80 http: Track active requests and wait for last to finish (João Barbosa)

Pull request description:

  This revives #19420. Since promag is not so active at the moment, I can support this to finally get it merged.

  The PR is rebased and comments by jonatack have been addressed.

  Once this is merged, I will also reopen #19434.

ACKs for top commit:
  achow101:
    ACK 60978c8080
  stickies-v:
    re-ACK [60978c8](60978c8080)
  hebasto:
    ACK 60978c8080

Tree-SHA512: eef0fe1081e9331b95cfafc71d82f2398abd1d3439dac5b2fa5c6d9c0a3f63ef19adde1c38c88d3b4e7fb41ce7c097943f1815c10e33d165918ccbdec512fe1c
2023-03-06 19:35:59 -05:00
Sebastian Falbesoner
3dd2f6461b test: psbt: check non-witness UTXO removal for segwit v1 input 2023-03-05 04:07:11 +01:00
Sebastian Falbesoner
dd78e3fa43 test: speedup rpc_psbt.py by whitelisting peers (immediate tx relay)
master branch:
    0m36.86s real     0m03.26s user     0m01.69s system
    0m35.71s real     0m03.78s user     0m01.64s system
    0m45.76s real     0m03.12s user     0m01.27s system

PR branch:
    0m13.04s real     0m02.66s user     0m00.93s system
    0m14.08s real     0m02.81s user     0m00.82s system
    0m14.05s real     0m02.50s user     0m00.93s system
2023-03-05 01:46:57 +01:00
Sebastian Falbesoner
e194e3e93d test: PSBT: eliminate magic numbers for global unsigned tx key (0) 2023-03-04 12:43:38 +01:00
Andrew Chow
bb136aaf2c
Merge bitcoin/bitcoin#26533: prune: scan and unlink already pruned block files on startup
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
77557dda4a prune: scan and unlink already pruned block files on startup (Andrew Toth)

Pull request description:

  There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.
  1. If we call `FindFilesToPrune` or `FindFilesToPruneManual` and crash before `UnlinkPrunedFiles`.
  2. If on Windows there is an open file handle to the file somewhere else when calling `fs::remove` in `UnlinkPrunedFiles` (https://en.cppreference.com/w/cpp/filesystem/remove, https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew#remarks). This could be from another process, or if we are calling `ReadBlockFromDisk`/`ReadRawBlockFromDisk` without having a lock on `cs_main` (which has been allowed since ccd8ef65f9).

  This PR mitigates this by scanning all pruned block files on startup after `LoadBlockIndexDB` and unlinking them again.

ACKs for top commit:
  achow101:
    ACK 3141eab9c6
  pablomartin4btc:
    re-ACK with added functional test 3141eab9c6.
  furszy:
    Code review ACK 3141eab9
  theStack:
    Code-review ACK 3141eab9c6

Tree-SHA512: 6c73bc57838ad1b7e5d441af3c4d6bf4c61c4382e2b86485e57fbb74a61240710c0ceeceb8b4834e610ecfa3175c6955c81ea4b2285fee11ca6383f472979d8d
2023-02-28 09:54:10 -05:00
Andrew Chow
b7702bd546
Merge bitcoin/bitcoin#25943: rpc: Add a parameter to sendrawtransaction which sets a maximum value for unspendable outputs.
7013da07fb Add release note for PR#25943 (David Gumberg)
04f270b435 Add test for unspendable transactions and parameter 'maxburnamount' to sendrawtransaction. (David Gumberg)

Pull request description:

  This PR adds a user configurable, zero by default parameter — `maxburnamount` — to `sendrawtransaction`. This PR makes bitcoin core reject transactions that contain unspendable outputs which exceed `maxburnamount`.  closes #25899.

  As a result of this PR, `sendrawtransaction` will by default block 3 kinds of transactions:

  1. Those that begin with `OP_RETURN` - (datacarriers)
  2. Those whose lengths exceed the script limit.
  3. Those that contain invalid opcodes.

  The user is able to configure a `maxburnamount` that will override this check and allow a user to send a potentially unspendable output into the mempool.

  I see two legitimate use cases for this override:
  1. Users that deliberately use `OP_RETURN` for datacarrier transactions that embed data into the blockchain.
  2.  Users that refuse to update, or are unable to update their bitcoin core client would be able to make use of new opcodes that their client doesn't know about.

ACKs for top commit:
  glozow:
    reACK 7013da07fb
  achow101:
    re-ACK 7013da07fb

Tree-SHA512: f786a796fb71a587d30313c96717fdf47e1106ab4ee0c16d713695e6c31ed6f6732dff6cbc91ca9841d66232166eb058f96028028e75c1507324426309ee4525
2023-02-23 13:57:38 -05:00
Andrew Chow
832fa2d238
Merge bitcoin/bitcoin#25574: validation: Improve error handling when VerifyDB dosn't finish successfully
0af16e7134 doc: add release note for #25574 (Martin Zumsande)
57ef2a4812 validation: report if pruning prevents completion of verification (Martin Zumsande)
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache (Martin Zumsande)
d6f781f1cf validation: return VerifyDBResult::INTERRUPTED if verification was interrupted (Martin Zumsande)
6360b5302d validation: Change return value of VerifyDB to enum type (Martin Zumsande)

Pull request description:

  `VerifyDB()` can fail to complete due to insufficient dbcache at the level 3 checks. This PR improves the error handling in this case in the following ways:
  - The rpc `-verifychain` now returns false if the check can't be completed due to insufficient cache
  - During init, we only log a warning if the default values for `-checkblocks` and `-checklevel` are taken and the check doesn't complete. However, if the user actively specifies one of these args, we return with an InitError if we can't complete the check.

  This PR also changes `-verifychain` RPC to return `false` if the verification didn't finish due to missing block data (pruning) or due to being interrupted by the node being shutdown.

  Previously, this PR also included a fix for a possible assert during verification - this was done in #27009 (now merged).

ACKs for top commit:
  achow101:
    ACK 0af16e7134
  ryanofsky:
    Code review ACK 0af16e7134. Only small suggested changes since the last review, like renaming some of the enum values. I did leave more suggestions, but they are not very important and could be followups
  john-moffett:
    ACK 0af16e7134
  MarcoFalke:
    lgtm re-ACK 0af16e7134 🎚

Tree-SHA512: 84b4f767cf9bfbafef362312757c9bf765b41ae3977f4ece840e40c52a2266b1457832df0cdf70440be0aac2168d9b58fc817238630b0b6812f3836ca950bc0e
2023-02-22 14:19:44 -05:00
fanquake
9f6ef0c156
Merge bitcoin/bitcoin#27143: test: Replace 0xC0 constant
c3b4b5a142 test: Replace 0xC0 constant (roconnor-blockstream)

Pull request description:

  Instead it should be the named constant `LEAF_VERSION_TAPSCRIPT`.

ACKs for top commit:
  instagibbs:
    ACK c3b4b5a142
  theStack:
    ACK c3b4b5a142

Tree-SHA512: c00be584ea2d0e7c01bf5620da0da1f37e5b5298ef95df48d91d137c8c542f5d91be158d45392cf2ba8874bf27bd12924e2eed395773b49d091e3028de3356a2
2023-02-22 18:11:26 +00:00
Andrew Chow
5e55534586
Merge bitcoin/bitcoin#27068: wallet: SecureString to allow null characters
4bbf5ddd44 Detailed error message for passphrases with null chars (John Moffett)
b4bdabc223 doc: Release notes for 27068 (John Moffett)
4b1205ba37 Test case for passphrases with null characters (John Moffett)
00a0861181 Pass all characters to SecureString including nulls (John Moffett)

Pull request description:

  `SecureString` is a `std::string` specialization with a secure allocator. However, in practice it's treated like a C- string (no explicit length and null-terminated). This can cause unexpected and potentially insecure behavior. For instance, if a user enters a passphrase with embedded null characters (which is possible through Qt and the JSON-RPC), it will ignore any characters after the first null, potentially giving the user a false sense of security.

  Instead of assigning to `SecureString` via `std::string::c_str()`, assign it via a `std::string_view` of the original. This explicitly captures the size and still doesn't make any extraneous copies in memory.

  Note to reviewers, the following all compile identically in recent `GCC` (x86-64 and ARM64) with `-O2` (and `-std=c++17`):

  ```C++
  std::string orig_string;
  std::cin >> orig_string;
  SecureString s;
  s.reserve(100);
  // The following all compile identically
  s = orig_string;
  s = std::string_view{orig_string};
  s.assign(std::string_view{orig_string});
  s.assign(orig_string.data(), orig_string.size());
  ```

  So it's largely a matter of preference. However, one thing to keep in mind is that we want to avoid making unnecessary copies of any sensitive data in memory.

  Something like `SecureString s{orig_string};` is still invalid and probably unwanted in our case, since it'd get treated as a short string and optimized away from the secure allocator. I presume that's the reason for the `reserve()` calls.

  Fixes #27067.

ACKs for top commit:
  achow101:
    re-ACK 4bbf5ddd44
  stickies-v:
    re-ACK [4bbf5dd](4bbf5ddd44)
  furszy:
    utACK 4bbf5ddd

Tree-SHA512: 47a96905a82ca674b18076a20a388123beedf70e9de73e42574ea68afbb434734e56021835dd9b148cdbf61709926b487cc95e9021d9bc534a7c93b3e143d2f7
2023-02-22 13:02:16 -05:00
fanquake
c6e65a102c
Merge bitcoin/bitcoin#27137: test: Raise PRNG seed log to INFO
4d84eaec82 Raise PRNG seed log to INFO. (roconnor-blockstream)

Pull request description:

  Some build infrastructure, such as Nix, will delete failed builds by default, keeping only the log (stdout/stderr) of the failed build.

  For flaky tests, it would be very helpful to have the PRNG seed in the default log in order to redo the failed test.

  By simply raising the PRNG seed logging to INFO, we can, by default, record the seed in the log of every build.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4d84eaec82
  theStack:
    ACK 4d84eaec82

Tree-SHA512: 3ccb4a4e7639a3babc3b2a6456a6d0bffc090da34e4545b317f7bfbed4e9950d1b38ea5b2a90c37ccb49b3454bdeff03a6aaf86770b9c4dd14b26320aba50b94
2023-02-22 17:51:39 +00:00
fanquake
63893d5eab
Merge bitcoin/bitcoin#26595: wallet: be able to specify a wallet name and passphrase to migratewallet
9486509be6 wallet, rpc: Update migratewallet help text for encrypted wallets (Andrew Chow)
aaf02b5721 tests: Tests for migrating wallets by name, and providing passphrase (Andrew Chow)
7fd125b27d wallet: Be able to unlock the wallet for migration (Andrew Chow)
6bdbc5ff59 rpc: Allow users to specify wallet name for migratewallet (Andrew Chow)
dbfa345403 wallet: Allow MigrateLegacyToDescriptor to take a wallet name (Andrew Chow)

Pull request description:

  `migratewallet` currently operates on wallets that are already loaded, however this is not necessarily required, and in the future, not possible once the legacy wallet is removed. So we need to also be able to give the wallet name to migrate.

  Additionally, the passphrase is required when migrating a wallet. Since a wallet may not be loaded when we migrate, and as we currently unload wallets when migrating, we need the passphrase to be given to `migratewallet` in order to migrate encrypted wallets.

  Fixes #27048

ACKs for top commit:
  john-moffett:
    reACK 9486509be6
  pinheadmz:
    ACK 9486509be6
  furszy:
    ACK 9486509b

Tree-SHA512: 35e2ba69a148e129a41e20d7fb99c4cab7947b1b7e7c362f4fd06ff8ac6e79e476e07207e063ba5b80e1a33e2343f4b4f1d72d7930ce80c34571c130d2f5cff4
2023-02-22 17:48:23 +00:00
roconnor-blockstream
c3b4b5a142 test: Replace 0xC0 constant
Instead it should be the named constant `LEAF_VERSION_TAPSCRIPT`.
2023-02-22 10:26:07 -05:00
fanquake
0c579203d2
Merge bitcoin/bitcoin#25867: lint: enable E722 do not use bare except
61bb4e783b lint: enable E722 do not use bare except (Leonardo Lazzaro)

Pull request description:

  Improve test code and enable E722 lint check.

   If you want to catch all exceptions that signal program errors, use except Exception: (bare except is equivalent to except BaseException:).

  Reference: https://peps.python.org/pep-0008/#programming-recommendations

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 61bb4e783b

Tree-SHA512: c7497769d5745fa02c78a20f4a0e555d8d3996d64af6faf1ce28e22ac1d8be415b98e967294679007b7bda2a9fd04031a9d140b24201e00257ceadeb5c5d7665
2023-02-22 09:28:09 +00:00
Andrew Chow
aaf02b5721 tests: Tests for migrating wallets by name, and providing passphrase 2023-02-21 15:51:31 -05:00
John Moffett
4b1205ba37 Test case for passphrases with null characters
Add a functional test to make sure the system
properly accepts passphrases with null characters.
2023-02-21 14:40:59 -05:00
Andrew Chow
80f4979322
Merge bitcoin/bitcoin#26347: wallet: ensure the wallet is unlocked when needed for rescanning
6a5b348f2e test: test rescanning encrypted wallets (ishaanam)
493b813e17 wallet: ensure that the passphrase is not deleted from memory when being used to rescan (ishaanam)
66a86ebabb wallet: keep track of when the passphrase is needed when rescanning (ishaanam)

Pull request description:

  Wallet passphrases are needed to top up the keypool of encrypted wallets
  during a rescan. The following RPCs need the passphrase when rescanning:
      - `importdescriptors`
      - `rescanblockchain`

  The following RPCs use the information about whether or not the
  passphrase is being used to ensure that full rescans are able to
  take place (meaning the following RPCs should not be able to run
  if a rescan requiring the wallet to be unlocked  is taking place):
      - `walletlock`
      - `encryptwallet`
      - `walletpassphrasechange`

  `m_relock_mutex` is also introduced so that the passphrase is not
  deleted from memory when the timeout provided in
  `walletpassphrase` is up and the wallet is still rescanning.
  Fixes #25702, #11249

  Thanks to achow101 for coming up with the idea of using a new mutex to solve this issue and for answering related questions.

ACKs for top commit:
  achow101:
    ACK 6a5b348f2e
  hernanmarino:
    ACK 6a5b348f2e
  furszy:
    Tested ACK 6a5b348f

Tree-SHA512: 0b6db692714f6f94594fa47249f5ee24f85713bfa70ac295a7e84b9ca6c07dda65df7b47781a2dc73e5b603a8725343a2f864428ae20d3e126c5b4802abc4ab5
2023-02-21 14:02:49 -05:00
roconnor-blockstream
4d84eaec82
Raise PRNG seed log to INFO.
Some build infrastructure, such as Nix, will delete failed builds by default, keeping only the log of the failed build.

For flaky tests, it would be very helpful to have the PRNG seed in the default log in order to redo the failed test.

By simply raising the PRNG seed logging to INFO, we can, by default, record the seed in the log of every build.
2023-02-21 12:01:13 -05:00
David Gumberg
04f270b435 Add test for unspendable transactions and parameter 'maxburnamount' to sendrawtransaction.
'maxburnamount' sets a maximum value for outputs heuristically deemed unspendable including datacarrier scripts that begin with `OP_RETURN`.
2023-02-20 11:38:52 -07:00
fanquake
94070029fb
Merge bitcoin/bitcoin#27053: wallet: reuse change dest when re-creating TX with avoidpartialspends
14b4921a91 wallet: reuse change dest when recreating TX with avoidpartialspends (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27051

  When the wallet creates a transaction internally, it will also create an alternative that spends using destination groups and see if the fee difference is negligible. If it costs the user the same to send the grouped version, we send it (even if the user has `avoidpartialspends` set to `false` which is default). This patch ensures that the second transaction creation attempt re-uses the change destination selected by the first attempt. Otherwise, the first change address remains reserved, will not be used in the second attempt, and then will never be used by the wallet, leaving gaps in the BIP44 chain.

  If the user had `avoidpartialspends` set to true, there is no second version of the created transaction and the change addresses are not affected.

  I believe this behavior was introduced in https://github.com/bitcoin/bitcoin/pull/14582

ACKs for top commit:
  achow101:
    ACK 14b4921a91

Tree-SHA512: a3d56f251ff4b333fc11325f30d05513e34ab0a2eb703fadd0ad98d167ae074493df1a24068298336c6ed2da6b31aa2befa490bc790bbc260ed357c8f2397659
2023-02-20 17:20:37 +00:00
fanquake
e996219f9a
Merge bitcoin/bitcoin#27113: rpc: Use a FlatSigningProvider in decodescript to allow inferring descriptors for scripts larger than 520 bytes
73ec4b2a83 tests: decodescript can infer descriptors for scripts >520 bytes (Andrew Chow)
7cc7822371 rpc: Use FlatSigningProvider in decodescript (Andrew Chow)

Pull request description:

  `FillableSigningProvider` limits scripts to 520 bytes even though segwit allows scripts to be larger than that. We can avoid this limit by using a `FlatSigningProvider` so that such larger scripts can be decoded.

  Fixes #27111

ACKs for top commit:
  instagibbs:
    ACK 73ec4b2a83

Tree-SHA512: c0e6d21025e2da864471989ac94c54e127d05459b9b048f34a0da8d76d8e372d5472a2e667ba2db74d6286e3e6faa55486ffa9232a068b519afa676394031d5a
2023-02-20 16:41:46 +00:00
fanquake
150cc8ef42
Merge bitcoin/bitcoin#27128: test: fix intermittent issue in p2p_disconnect_ban
1819564c21 test: fix intermittent issue in `p2p_disconnect_ban` (brunoerg)

Pull request description:

  Fixes #26808

  When `node0` calls `disconnectnode` to disconnect `node1`, we should check in `node1` if it worked, because for `node0` the informations in `getpeerinfo` may be updated before really completing the disconnection.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 1819564c21

Tree-SHA512: 53a386fc38e2faa6f6da3536e76857ff4b6f55e2590d73fe857b3fe5d0f3ff92c5c7e4abd50ab4be250cb2106a4d14ad95d4809ea60c6e00ed3ac0e71255b0b0
2023-02-20 16:28:02 +00:00
fanquake
446c8f581c
Merge bitcoin/bitcoin#25950: test: fix test abort for high timeout values (and --timeout-factor 0)
14302a4802 test: fix test abort for high timeout values (and `--timeout-factor 0`) (Sebastian Falbesoner)

Pull request description:

  On master, the functional tests's option `--timeout-factor 0` (which according to the test docs and parameter description should disable the RPC timeouts) currently fails, same as high values like `--timeout-factor 999999`:
  ```
  $ ./test/functional/wallet_basic.py --timeout-factor 0
  2022-08-29T01:26:39.561000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_f24yxzp5
  2022-08-29T01:26:40.262000Z TestFramework (ERROR): Assertion failed
  Traceback (most recent call last):
    File "/home/honey/bitcoin/test/functional/test_framework/test_framework.py", line 549, in start_nodes
      node.wait_for_rpc_connection()
    File "/home/honey/bitcoin/test/functional/test_framework/test_node.py", line 234, in wait_for_rpc_connection
      rpc.getblockcount()
    File "/home/honey/bitcoin/test/functional/test_framework/coverage.py", line 49, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/home/honey/bitcoin/test/functional/test_framework/authproxy.py", line 142, in __call__
      response, status = self._request('POST', self.__url.path, postdata.encode('utf-8'))
    File "/home/honey/bitcoin/test/functional/test_framework/authproxy.py", line 107, in _request
      self.__conn.request(method, path, postdata, headers)
    File "/usr/local/lib/python3.9/http/client.py", line 1285, in request
      self._send_request(method, url, body, headers, encode_chunked)
    File "/usr/local/lib/python3.9/http/client.py", line 1331, in _send_request
      self.endheaders(body, encode_chunked=encode_chunked)
    File "/usr/local/lib/python3.9/http/client.py", line 1280, in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
    File "/usr/local/lib/python3.9/http/client.py", line 1040, in _send_output
      self.send(msg)
    File "/usr/local/lib/python3.9/http/client.py", line 980, in send
      self.connect()
    File "/usr/local/lib/python3.9/http/client.py", line 946, in connect
      self.sock = self._create_connection(
    File "/usr/local/lib/python3.9/socket.py", line 844, in create_connection
      raise err
    File "/usr/local/lib/python3.9/socket.py", line 832, in create_connection
      sock.connect(sa)
  OSError: [Errno 22] Invalid argument
  ```
  This is caused by a high timeout value that Python's HTTP(S) client library can't cope with. Fix this by clamping down the connection's set timeout value in AuthProxy. The change can easily be tested by running an arbitrary test with `--timeout-factor 0` on master (should fail), on this PR (should pass) and on this PR with the clamping value increased by 1 (should fail).

  // EDIT: The behaviour was observed on OpenBSD 7.1 and Python 3.9.12.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 14302a4802

Tree-SHA512: 6469e8ac699f1bb7dea11d5fb8b3ae54d895bb908570587c5631144cd41fe980ca0b1e6d0b7bfa07983307cba15fb26ae92e6766375672bf5be838d8e5422dbc
2023-02-20 16:20:55 +00:00
brunoerg
1819564c21 test: fix intermittent issue in p2p_disconnect_ban
When `node0` calls `disconnectnode` to disconnect `node1`, we should check in `node1` if it worked, because for `node0` the informations in `getpeerinfo` may be updated before really completing the disconnection.
2023-02-20 10:36:35 -03:00
Leonardo Lazzaro
61bb4e783b lint: enable E722 do not use bare except 2023-02-18 11:24:09 +00:00
Sebastian Falbesoner
14302a4802 test: fix test abort for high timeout values (and --timeout-factor 0) 2023-02-17 23:30:59 +01:00
Andrew Chow
f722a9bd13
Merge bitcoin/bitcoin#20018: p2p: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified
2555a3950f p2p: ProcessAddrFetch(-seednode) is unnecessary if -connect is specified (Dhruv Mehta)

Pull request description:

  If the user runs: `bitcoind -connect=X -seednode=Y`, I _think_ it is safe to ignore `-seednode`. A more populated `addrman` (via `getaddr` calls to peers in `-seednode`) is not useful in this configuration: `addrman` entries are used to initiate new outbound connections when slots are open, or to open feeler connections and keep `addrman` from getting stale. This is all done in a part of `ThreadOpenConnections` (below [this line](https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1803)) which is never executed when `-connect` is supplied. With `-connect`, `ThreadOpenConnections` will run [this loop](https://github.com/bitcoin/bitcoin/blob/master/src/net.cpp#L1785) and exit thread execution when interrupted.

  Reviewers may also find it relevant that when `-connect` is used, we [soft disable](https://github.com/bitcoin/bitcoin/blob/master/src/init.cpp#L800) `-dnsseed` in init.cpp perhaps for the same reason i.e. seeding is not useful with `-connect`.

  Running `ProcessAddrFetch` does not seem to have downside except developer confusion AFAICT. I was confused by this and felt it might affect other new bitcoiners too. If there is strong preference to not remove the line, I'd also be happy to just leave a comment there mentioning `ADDR_FETCH`/`-seednode` is irrelevant when used with `-connect`.

  If this change is accepted, the node will still make `getaddr` calls to peers in `-connect` and expand `addrman`. However, disabling those `getaddr` calls would leak information about the node's configuration.

ACKs for top commit:
  mzumsande:
    Code Review ACK 2555a3950f
  achow101:
    ACK 2555a3950f
  vasild:
    ACK 2555a3950f

Tree-SHA512: 9187a0cff58db8edeca7e15379b1c121e7ebe8c38fb82f69e3dae8846ee94c92a329d79025e0f023c7579b2d86e7dbf756e4e30e90a72236bfcd2c00714180b3
2023-02-17 14:21:06 -05:00
Martin Zumsande
57ef2a4812 validation: report if pruning prevents completion of verification
Now the verifychain RPC returns false if the checks didn't
finish because the blocks requested to be queried have been pruned.
2023-02-16 17:58:52 -05:00
Andrew Chow
73966f75f6
Merge bitcoin/bitcoin#25344: New outputs argument for bumpfee/psbtbumpfee
4c8ecccdcd test: add tests for `outputs` argument to `bumpfee`/`psbtbumpfee` (Seibart Nedor)
c0ebb98382 wallet: add `outputs` arguments to `bumpfee` and `psbtbumpfee` (Seibart Nedor)
a804f3cfc0 wallet: extract and reuse RPC argument format definition for outputs (Seibart Nedor)

Pull request description:

  This implements a modification of the proposal in #22007: instead of **adding** outputs to the set of outputs in the original transaction, the outputs given by `outputs` argument **completely replace** the outputs in the original transaction.

  As noted below, this makes it easier to "cancel" a transaction or to reduce the amounts in the outputs, which is not the case with the original proposal in #22007, but it seems from the discussion in this PR that the **replace** behavior is more desirable than **add** one.

ACKs for top commit:
  achow101:
    ACK 4c8ecccdcd
  1440000bytes:
    Code Review ACK 4c8ecccdcd
  ishaanam:
    reACK 4c8ecccdcd

Tree-SHA512: 31361f4a9b79c162bda7929583b0a3fd200e09f4c1a5378b12007576d6b14e02e9e4f0bab8aa209f08f75ac25a1f4805ad16ebff4a0334b07ad2378cc0090103
2023-02-16 13:47:41 -05:00
Andrew Chow
7fd125b27d wallet: Be able to unlock the wallet for migration
Since migration reloads the wallet, the wallet will always be locked
unless the passphrase is given. migratewallet can now take the
passphrase in order to unlock the wallet for migration.
2023-02-16 13:39:03 -05:00
Andrew Chow
73ec4b2a83 tests: decodescript can infer descriptors for scripts >520 bytes 2023-02-16 11:42:28 -05:00
merge-script
437dfe1c26
Merge bitcoin/bitcoin#26714: test: add coverage for unparsable -maxuploadtarget
7a83aa0982 test: add coverage for unparsable `-maxuploadtarget` (brunoerg)

Pull request description:

  This PR adds test coverage for the following error:
  7386da7a0b/src/init.cpp (L1096-L1099)

Top commit has no ACKs.

Tree-SHA512: c115b2b4d2d0eb2316bf9fafd7e0046aa18c9650062779b3a82d6145d188765bff5317f4ca5f79607732fde6d83e1f67756ac20a12c98d060ee68d8acc20c76e
2023-02-16 15:17:54 +01:00
merge-script
3a68e194f5
Merge bitcoin/bitcoin#26586: test: previous releases: add v24.0.1
741908afc1 test: previous releases: add v24.0.1 (Sebastian Falbesoner)

Pull request description:

  The same procedure as every release (see dba1231672 [v23.0] and d8b705f1ca [v22.0]), only a little simpler now: thanks to #25650, the previous release fetch script defaults to downloading/building the necessary tags, i.e. we don't need to extend the tag list in the CI scripts and test/README.md anymore.

ACKs for top commit:
  Sjors:
    tACK 741908afc1

Tree-SHA512: a5426e989bd0bba42aa13e7d4cf60f792bf36bd9a6cdb6ef5799f7574d9a8a20979244627bbd0c6219630367e7fd73bac9e677814bc50233f64592ad035e713e
2023-02-16 15:02:02 +01:00
fanquake
fb82d91a9c
Merge bitcoin/bitcoin#24149: Signing support for Miniscript Descriptors
6c7a17a8e0 psbt: support externally provided preimages for Miniscript satisfaction (Antoine Poinsot)
840a396029 qa: add a "smart" Miniscript fuzz target (Antoine Poinsot)
17e3547241 qa: add a fuzz target generating random nodes from a binary encoding (Antoine Poinsot)
611e12502a qa: functional test Miniscript signing with key and timelocks (Antoine Poinsot)
d57b7f2021 refactor: make descriptors in Miniscript functional test more readable (Antoine Poinsot)
0a8fc9e200 wallet: check solvability using descriptor in AvailableCoins (Antoine Poinsot)
560e62b1e2 script/sign: signing support for Miniscripts with hash preimage challenges (Antoine Poinsot)
a2f81b6a8f script/sign: signing support for Miniscript with timelocks (Antoine Poinsot)
61c6d1a844 script/sign: basic signing support for Miniscript descriptors (Antoine Poinsot)
4242c1c521 Align 'e' property of or_d and andor with website spec (Pieter Wuille)
f5deb41780 Various additional explanations of the satisfaction logic from Pieter (Pieter Wuille)
22c5b00345 miniscript: satisfaction support (Antoine Poinsot)

Pull request description:

  This makes the Miniscript descriptors solvable.

  Note this introduces signing support for much more complex scripts than the wallet was previously able to solve, and the whole tooling isn't provided for a complete Miniscript integration in the wallet. Particularly, the PSBT<->Miniscript integration isn't entirely covered in this PR.

ACKs for top commit:
  achow101:
    ACK 6c7a17a8e0
  sipa:
    utACK 6c7a17a8e0 (to the extent that it's not my own code).

Tree-SHA512: a71ec002aaf66bd429012caa338fc58384067bcd2f453a46e21d381ed1bacc8e57afb9db57c0fb4bf40de43b30808815e9ebc0ae1fbd9e61df0e7b91a17771cc
2023-02-16 10:01:33 +00:00
merge-script
a65d2259f1
Merge bitcoin/bitcoin#27035: test: simplify and speedup mempool_updatefromblock.py by using MiniWallet
dee8549be3 test: simplify and speedup mempool_updatefromblock.py by using MiniWallet (Sebastian Falbesoner)

Pull request description:

  This PR simplifies the functional test mempool_updatefromblock.py by using MiniWallet in order to avoid manual low-level tx creation (signing, outputs selection, fee calculation). Most of the tedious work is done by the method `MiniWallet.send_self_transfer_multi` (calling `create_self_transfer_multi` internally) which supports spending a given set of UTXOs and creating a certain number of outputs.

  As a nice side-effect, the test's performance increases significantly (~3.5x on my system):

  ```
  master
      1m56.80s real     1m50.10s user     0m06.36s system

  PR
      0m32.34s real     0m30.26s user     0m01.41s system
  ```

  The arguments `start_input_txid` and `end_address` have been removed from the `transaction_graph_test` method, as they are currently unused and I don't see them being needed for future tests.

ACKs for top commit:
  brunoerg:
    crACK dee8549be3
  MarcoFalke:
    lgtm ACK dee8549be3 🚏

Tree-SHA512: 9f6da634bdc8c272f9a2af1cddaa364ee371d4e95554463a066249eecebb668d8c6cb123ec8a5404c41b3291010c0c8806a8a01dd227733cec03e73aa93b0103
2023-02-15 16:26:00 +01:00
Matthew Zipkin
14b4921a91
wallet: reuse change dest when recreating TX with avoidpartialspends 2023-02-15 10:14:30 -05:00
ishaanam
6a5b348f2e test: test rescanning encrypted wallets 2023-02-14 23:32:43 -05:00
Andrew Chow
576e16e702
Merge bitcoin/bitcoin#26184: test: p2p: check that headers message with invalid proof-of-work disconnects peer
772671245d test: p2p: check that headers message with invalid proof-of-work disconnects peer (Sebastian Falbesoner)

Pull request description:

  One of the earliest anti-DoS checks done after receiving and deserializing a `headers` message from a peer is verifying whether the proof-of-work is valid (called in method `PeerManagerImpl::ProcessHeadersMessage`):
  f227e153e8/src/net_processing.cpp (L2752-L2762)
  The called method `PeerManagerImpl::CheckHeadersPoW` calls `Misbehaving` with a score of 100, i.e. leading to an immediate disconnect of the peer:
  f227e153e8/src/net_processing.cpp (L2368-L2372)

  This PR adds a simple test for both the misbehaving log and the resulting disconnect. For creating a block header with invalid proof-of-work, we first create one that is accepted by the node (the difficulty field `nBits` is copied from the genesis block) and based on that the nonce is modified until we have block header hash prefix that is too high to fulfill even the minimum difficulty.

ACKs for top commit:
  Sjors:
    ACK 772671245d
  achow101:
    ACK 772671245d
  brunoerg:
    crACK 772671245d
  furszy:
    Code review ACK 77267124 with a non-blocking speedup.

Tree-SHA512: 680aa7939158d1dc672b90aa6554ba2b3a92584b6d3bcb0227776035858429feb8bc66eed18b47de0fe56df7d9b3ddaee231aaeaa360136603b9ad4b19e6ac11
2023-02-14 18:45:35 -05:00
MarcoFalke
fa9ec7b0fe
test: Fix intermittent sync issue in wallet_pruning 2023-02-13 17:32:42 +01:00
Antoine Poinsot
6c7a17a8e0
psbt: support externally provided preimages for Miniscript satisfaction
Co-Authored-By: Andrew Chow <github@achow101.com>
2023-02-13 15:39:25 +01:00
merge-script
a6316590d5
Merge bitcoin/bitcoin#26970: test: fix immediate tx relay in wallet_groups.py
ab4efad51b test: fix immediate tx relay in wallet_groups.py (Sebastian Falbesoner)

Pull request description:

  In the functional test wallet_groups.py we whitelist peers on all nodes (`-whitelist=noban@127.0.0.1`) to enable immediate tx relay for fast mempool synchronization. However, considering that this setting only applies to inbound peers and the default test topology looks like this:
  ```
      node0 <--- node1 <---- node2 <--- ... <-- nodeN
  ```

  txs propagate fast only from lower- to higher-numbered nodes (i.e. "left to right" in the above diagram) and take long from higher- to lower-numbered nodes ("right to left") since in the latter direction we only have outbound peers, where the trickle relay is still active. As a consequence, if a tx is submitted from any node other than node0, the mempool synchronization can take quite long.

  This PR fixes this by simply adding another connection from node0 to the last node, leading to a ~2-3x speedup (5 runs measured via `time ./test/functional/wallet_groups.py` are shown):

  ```
  master:
      0m53.31s real     0m08.22s user     0m05.60s system
      0m32.85s real     0m07.44s user     0m04.08s system
      0m46.40s real     0m09.18s user     0m04.23s system
      0m46.96s real     0m11.10s user     0m05.74s system
      0m57.23s real     0m10.53s user     0m05.59s system

  PR:
      0m19.64s real     0m09.58s user     0m05.50s system
      0m18.05s real     0m07.77s user     0m04.03s system
      0m18.99s real     0m07.90s user     0m04.25s system
      0m17.49s real     0m07.56s user     0m03.92s system
      0m18.11s real     0m07.74s user     0m03.88s system
  ```
  Note that in most tests this is not a problem since txs very often originate from node0.

ACKs for top commit:
  brunoerg:
    utACK ab4efad51b

Tree-SHA512: 12675357e6eb5a18383f2bfe719a184c0790863b37a98749d8e757dd5dc3a36212e16a81f0a192340c11b793eda00db359c7011f46f7c27e3a093af4f5b62147
2023-02-13 11:51:03 +01:00
Antoine Poinsot
611e12502a
qa: functional test Miniscript signing with key and timelocks
We'll need a better integration of the hash preimages PSBT fields to
satisfy Miniscript with such challenges from the RPC.

Thanks to Greg Sanders for his examples and suggestions to improve this
test.
2023-02-11 14:12:13 +01:00
Antoine Poinsot
d57b7f2021
refactor: make descriptors in Miniscript functional test more readable
We'll add more of them in the next commit, let's keep it bearable.
2023-02-11 14:12:13 +01:00
Antoine Poinsot
61c6d1a844
script/sign: basic signing support for Miniscript descriptors
Try to solve a script using the Miniscript satisfier if the legacy
solver fails under P2WSH context. Only solve public key and public key
hash challenges for now.

We don't entirely replace the raw solver and especially rule out trying to
solve CHECKMULTISIG-based multisigs with the Miniscript satisfier since
some features, such as the transaction input combiner, rely on the
specific behaviour of the former.
2023-02-11 14:12:10 +01:00
Fabian Jahr
60978c8080
test: Reduce extended timeout on abortnode test
This was made obsolete by tracking the active requests and explicitly waiting for them to finish before shutdown.
2023-02-10 20:35:02 +01:00
Sebastian Falbesoner
772671245d test: p2p: check that headers message with invalid proof-of-work disconnects peer 2023-02-10 00:01:04 +01:00
721217.xyz
fa6f67837b
test: Fix intermittent sync issue in wallet_pruning 2023-02-09 12:28:56 +01:00
MarcoFalke
51d51d3082
Merge bitcoin/bitcoin#26507: test: remove unused vars in feature_block
741c215b5f test: remove unused vars in `feature_block` (brunoerg)

Pull request description:

  There is no need to assign `self.next_block` to variables if we're not using its return value. Most cases touched here, we're reassigning it right after with the value from `self.update_block`.

Top commit has no ACKs.

Tree-SHA512: 25bbea2a09f38c3a3483fa363f024d2a8edd06a00cccc93cef99e489b9a3485d58bbd6a1ed2dddc00f1cebec7e63aed8ad95701a2645ce20a0db9b69573c20a7
2023-02-08 11:42:22 +01:00
Hennadii Stepanov
c9ba4f9ecb
test: Add test for file system permissions 2023-02-06 11:08:57 +00:00
Antoine Poinsot
dfc9acbf01
rpc: decode Miniscript descriptor when possible in decodescript
The descriptor inference logic would previously always use a dummy
signing provider and would never analyze the witness script of a P2WSH
scriptPubKey.

Note even a valid Miniscript might not always be decodable from Script
without more contextual information (for instance the key preimage for a
pk_h).
2023-02-03 18:15:42 +01:00
Sebastian Falbesoner
dee8549be3 test: simplify and speedup mempool_updatefromblock.py by using MiniWallet 2023-02-03 17:23:26 +01:00
Andrew Chow
fdd363ebd9
Merge bitcoin/bitcoin#26910: wallet: migrate wallet, exit early if no legacy data exist
6d31900e52 wallet: migrate wallet, exit early if no legacy data exist (furszy)

Pull request description:

  The process first creates a backup file then return an error,
  without removing the recently created file, when notices that
  the db is already running sqlite.

ACKs for top commit:
  john-moffett:
    ACK 6d31900e52
  achow101:
    ACK 6d31900e52
  ishaanam:
    crACK 6d31900e52

Tree-SHA512: 9fb52e80de96e129487ab91bef13647bc4570a782003b1e37940e2a00ca26283fd24ad39bdb63a984ae0a56140b518fd0d74aa2fc59ab04405b2c179b7d3c54a
2023-02-01 17:14:13 -05:00
MarcoFalke
357d750cab
Merge bitcoin/bitcoin#26956: test: refactor: introduce replace_in_config helper
b530d9605d test: refactor: introduce `replace_in_config` helper (Sebastian Falbesoner)

Pull request description:

  Currently two functional tests (p2p_permissions.py and wallet_crosschain.py) include quite similar code for substituting strings in a TestNode's bitcoind configuration file, so refactoring that out to a dedicated helper method seems to make sense (probably other tests could need that too in the future).

ACKs for top commit:
  kouloumos:
    ACK b530d9605d

Tree-SHA512: 5ca65a2ef3292460e5720d5c6acf7326335001858e8f71ab054560117f9479dbadb1dacd8c9235f67f3fcfd08dbc761b62704f379cbf619bba8804f16a25bc7b
2023-01-31 10:23:37 +01:00
glozow
b1329b7523
Merge bitcoin/bitcoin#26499: wallet: Abandon descendants of orphaned coinbases
b0fa5989e1 test: Check that orphaned coinbase unconf spend is still abandoned (Andrew Chow)
9addbd7890 wallet: Automatically abandon orphaned coinbases and their children (Andrew Chow)

Pull request description:

  When a block is reorged out of the main chain, any descendants of the coinbase will no longer be valid. Currently they are only marked as inactive, which means that our balance calculations will still include them. In order to be excluded from the balance calculation, they need to either be abandoned or conflicted. This PR goes with the abandoned method.

  Note that even when they are included in balance calculations, coin selection will not select outputs belonging to these transactions because they are not in the mempool.

  Fixes #14148

ACKs for top commit:
  furszy:
    ACK b0fa5989 with a not-blocking nit.
  aureleoules:
    reACK b0fa5989e1
  ishaanam:
    ACK b0fa5989e1

Tree-SHA512: 68f12e7aa8df392d8817dc6ac0becce8dbe83837bfa538f47027e6730e5b2e1b1a090cfcea2dc598398fdb66090e02d321d799f087020d7e1badcf96e598c3ac
2023-01-30 10:09:41 +00:00
Sebastian Falbesoner
b530d9605d test: refactor: introduce replace_in_config helper 2023-01-28 23:49:55 +01:00
Martin Zumsande
6548ba68e8 test: fix intermittent errors in p2p_ibd_stalling.py
Using is_connected instead of num_test_p2p_connections
ensures that python has taken notice that the p2p was
disconnected.
2023-01-27 15:28:21 -05:00
Andrew Chow
835212cd1d
Merge bitcoin/bitcoin#25880: p2p: Make stalling timeout adaptive during IBD
39b93649c4 test: add functional test for IBD stalling logic (Martin Zumsande)
0565951f34 p2p: Make block stalling timeout adaptive (Martin Zumsande)

Pull request description:

  During IBD, there is the following stalling mechanism if we can't proceed with assigning blocks from a 1024 lookahead window because all of these blocks are either already downloaded or in-flight: We'll mark the peer from which we expect the current block that would allow us to advance our tip (and thereby move the 1024 window ahead) as a possible staller. We then give this peer 2 more seconds to deliver a block (`BLOCK_STALLING_TIMEOUT`) and if it doesn't, disconnect it and assign the critical block we need to another peer.

  Now the problem is that this second peer is immediately marked as a potential staller using the same mechanism and given 2 seconds as well - if our own connection is so slow that it simply takes us more than 2 seconds to download this block, that peer will also be disconnected (and so on...), leading to repeated disconnections and no progress in IBD. This has been described in #9213, and I have observed this when doing IBD  on slower connections or with Tor - sometimes there would be several minutes without progress, where all we did was disconnect peers and find new ones.

  The `2s` stalling timeout was introduced in #4468, when blocks weren't full and before Segwit increased the maximum possible physical size of blocks - so I think it made a lot of sense back then.
  But it would be good to revisit this timeout now.

  This PR makes the timout adaptive (idea by sipa):
  If we disconnect a peer for stalling, we now double the timeout for the next peer (up to a maximum of 64s). If we connect a block, we half it again up to the old value of 2 seconds. That way, peers that are comparatively slower will still get disconnected, but long phases of disconnecting all peers shouldn't happen anymore.

  Fixes #9213

ACKs for top commit:
  achow101:
    ACK 39b93649c4
  RandyMcMillan:
    Strong Concept ACK 39b93649c4
  vasild:
    ACK 39b93649c4
  naumenkogs:
    ACK 39b93649c4

Tree-SHA512: 85bc57093b2fb1d28d7409ed8df5a91543909405907bc129de7c6285d0810dd79bc05219e4d5aefcb55c85512b0ad5bed43a4114a17e46c35b9a3f9a983d5754
2023-01-27 01:53:21 -05:00
MarcoFalke
ffc22b7d42
Merge bitcoin/bitcoin#26923: test: refactor: simplify p2p_{tx_download,eviction}.py by using MiniWallet
8609f24be2 test: refactor: simplify p2p_eviction.py by using MiniWallet (Sebastian Falbesoner)
7aa4b32cd4 test: refactor: simplify p2p_tx_download.py by using MiniWallet (Sebastian Falbesoner)

Pull request description:

  Similar to #26892, this PR simplies the functional tests p2p_tx_download.py and p2p_eviction.py by using MiniWallet in order to avoid manual low-level tx creation. For the latter, rather than mining 100 blocks manually, the pre-mined chain of the test framework is used.

  These instances were found via `$ git grep signrawtransactionwithkey ./test/functional`. AFAICT, there are no other instances where MiniWallet could replace tx creation trivially.

ACKs for top commit:
  MarcoFalke:
    review ACK 8609f24be2

Tree-SHA512: dfb0103fe7f0625d125e8e4408baed8bfc1ff579954af17d0ead5277e05f933b2c2d98a0093e8109e947635f1718d5c58d837ab825f26077fac0a40575bd3e6f
2023-01-26 15:54:06 +01:00
Sebastian Falbesoner
ab4efad51b test: fix immediate tx relay in wallet_groups.py 2023-01-26 02:56:34 +01:00
fanquake
ab98673f05
Merge bitcoin/bitcoin#26929: rpc: Throw more user friendly arg type check error (1.5/2)
fafeddfe0e rpc: Throw more user friendly arg type check error (MarcoFalke)

Pull request description:

  The arg type check error doesn't list which arg (position or name) failed. Fix that.

ACKs for top commit:
  stickies-v:
    ACK fafeddfe0e - although I think the functional test isn't in a logical place (but not blocking)

Tree-SHA512: 17425aa145aab5045940ec74fff28f0e3b2b17ae55f91c4bb5cbcdff0ef13732f8e31621d85964dc2c04333ea37dbe528296ac61be27541384b44e37957555c8
2023-01-25 15:25:49 +00:00
MarcoFalke
0486148f75
Merge bitcoin/bitcoin#26829: init: Remove unnecessary sensitive flag from rpcbind
b9d5674541 init: Remove sensitive flag from rpcbind (Andrew Chow)

Pull request description:

  `-rpcbind` is currently flagged as a sensitive option which means that its value will be masked when the command line args are written to the debug.log file. However this is not useful as if `rpcbind` is actually activated, the bound IP addresses will be written to the log anyways. The test `feature_config_args.py` did not catch this contradiction as the test node was not started with `rpcallowip` and so `rpcbind` was not acted upon.

  This also brings `rpcbind` inline with `bind` as that is not flagged as sensitive either.

ACKs for top commit:
  Sjors:
    re-utACK b9d5674541
  willcl-ark:
    ACK b9d5674
  theStack:
    ACK b9d5674541

Tree-SHA512: 50ab5ad2e18ae70649deb1ac429d404b5f5c41f32a4943b2041480580152df22e72d4aae493379d0b23fcb649ab342376a82119760fbf6dfdcda659ffd3e244a
2023-01-25 15:32:41 +01:00
Andrew Chow
b9d5674541 init: Remove sensitive flag from rpcbind 2023-01-23 17:25:02 -05:00
MarcoFalke
fa952fad2f
test: Avoid rpc timeout in p2p_headers_sync_with_minchainwork 2023-01-23 16:35:13 +01:00
MarcoFalke
fa88c043d1
test: Fix intermittent feature_rbf issue 2023-01-21 11:18:12 +01:00
MarcoFalke
fafeddfe0e
rpc: Throw more user friendly arg type check error 2023-01-20 13:26:47 +01:00
Sebastian Falbesoner
8609f24be2 test: refactor: simplify p2p_eviction.py by using MiniWallet
Also, use the pre-mined chain of the test framework rather than
mining 100 blocks manually on each run.
2023-01-19 16:11:27 +01:00
Sebastian Falbesoner
7aa4b32cd4 test: refactor: simplify p2p_tx_download.py by using MiniWallet 2023-01-19 16:08:55 +01:00
MarcoFalke
92dcbe9cc3
Merge bitcoin/bitcoin#23395: util: Add -shutdownnotify option
d96d97ad30  doc: Add release note for shutdownnotify. (klementtan)
0bd73e2c45 util: Add -shutdownnotify option. (klementtan)

Pull request description:

  **Description**: Similar to `-startupnotify`, this PR adds a new option to allow users to specify a command to be executed when Bitcoin Core shuts down.

  **Note**: The `shutdownnotify` commands will not be executed if bitcoind shut down due to *unexpected* reasons (ie `killall -9 bitcoind`).

  ### Testing:
  **Normal shutdown commands**
  ```
  # start bitcoind with shutdownnotify optioin
  ./src/bitcoind -signet -shutdownnotify="touch foo.txt"

  # shutdown bitcoind
  ./src/bitcoin-cli -signet stop

  # check that foo.txt has been created
  ```

  **Final RPC call**
  Commands:
  ```
  $  ./src/bitcoind -signet -nolisten -noconnect -shutdownnotify="./src/bitcoin-cli -signet getblockchaininfo > tmp.txt"
  $ ./src/bitcoin-cli stop
  $ cat tmp.txt
  ```
  <details>
  <summary>Screen Shot</summary>

  ![image](https://user-images.githubusercontent.com/49265907/141186183-cbc6f82c-400d-4a8b-baba-27c0346c2c8a.png)
  </details>

ACKs for top commit:
  achow101:
    ACK d96d97ad30
  1440000bytes:
    ACK d96d97ad30
  theStack:
    re-ACK d96d97ad30

Tree-SHA512: 16f7406fd232e8b97aea5e58854c84755b0c35c88cb3ef9ee123b29a1475a376122b1e100da860cc336d4d657e6046a70e915fdb9b70c9fd097c6eef1b028161
2023-01-19 10:34:54 +01:00
furszy
6d31900e52
wallet: migrate wallet, exit early if no legacy data exist
otherwise the process will create a backup file then return
an error when notices that the db is already running sqlite.
2023-01-18 13:47:31 -03:00
MarcoFalke
fa8fe5b696
scripted-diff: Use new python 3.7 keywords
-BEGIN VERIFY SCRIPT-
 sed -i 's/universal_newlines/text/g' $(git grep -l universal_newlines)
-END VERIFY SCRIPT-
2023-01-18 13:00:34 +01:00
Aurèle Oulès
962a0930e6
Improve address decoding errors 2023-01-17 18:31:51 +01:00
fanquake
89fb354f28
Merge bitcoin/bitcoin#26625: test: Run mempool_packages.py with MiniWallet
fa6b402114 test: Run mempool_packages.py with MiniWallet (MarcoFalke)
fa448c27d2 test: Return fee from MiniWallet (MarcoFalke)
faec09f240 test: Return chain of MiniWallet txs from MiniWallet chain method (MarcoFalke)
faa12d4ccd test: Refactor MiniWallet sign_tx (MarcoFalke)
fa2d82103f test: Return wtxid from create_self_transfer_multi (MarcoFalke)

Pull request description:

  This allows to run the test even when no wallet is compiled in.

  Also, it is a lot nicer to read now.

ACKs for top commit:
  glozow:
    reACK fa6b402

Tree-SHA512: de0338068fd51db01d64ce270f94fd2982a63a6de597325cd1e1f11127e9075bd4aeacace0ed76d09a2db8b962b27237cf11edb4c1fe1a01134d397f8a11bd05
2023-01-17 16:39:45 +00:00
MarcoFalke
fa96f93f05
test: Add test for missing and omitted required arg 2023-01-17 12:31:59 +01:00
Seibart Nedor
4c8ecccdcd test: add tests for outputs argument to bumpfee/psbtbumpfee 2023-01-17 13:28:53 +02:00
MarcoFalke
635f1900d0
Merge bitcoin/bitcoin#26884: test: wallet: add coverage for -spendzeroconfchange setting
603d295199 test: wallet: add coverage for `-spendzeroconfchange` setting (Sebastian Falbesoner)
50112034bc test: remove `-spendzeroconfchange` setting from mempool_limit.py (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `-spendzeroconfchange` setting (in particular the non-default case `=0`). Note that in contrast to the name, the setting does not only apply to change outputs, but in fact to _all_ unconfirmed outputs that we sent to ourselves, i.e. we can trigger the testing path simply with a single recipient address. The first commit removes the setting from the functional test mempool_limit.py, where it doesn't have any effect, since the test was changed to use MiniWallet in commit dddca3899c.

ACKs for top commit:
  brunoerg:
    crACK 603d295199

Tree-SHA512: 15d9c8bd5eb37c6b228bf887eb27debee0a391c82356662785da4553ee2558e611834c3936ef7136812b46f877bab7aa5f3088bbd278b81f296bdda96cc8e1c3
2023-01-17 11:13:50 +01:00