Commit graph

4986 commits

Author SHA1 Message Date
MacroFake
37f5386349
Merge bitcoin/bitcoin#26038: test: invalidating an unknown block throws an error
4b1d5a1053 test: invalidating an unknown block throws an error (brunoerg)

Pull request description:

  While playing with `invalidateblock`, I unintentionally tried to invalidate an unknown block and it threw an error. Looking at the tests I just realized there is no test coverage for this case. This PR adds it.

Top commit has no ACKs.

Tree-SHA512: 25286ead809b3ad022e759127ef3134b271fbe76cb7b50ec2b0c7e2409da8d1b01dc5e80afe73e4564cc9c9c03487a1fe772aea3456988552d2f9c8fb34c730b
2022-09-08 16:51:03 +02:00
brunoerg
4b1d5a1053 test: invalidating an unknown block throws an error 2022-09-08 11:06:35 -03:00
glozow
667401a855 [test] only run feature_rbf.py once
There is no need to run this test twice with --descriptors and
--legacy-wallet, as it doesn't ever use the wallet.
2022-09-08 12:26:41 +01:00
Aurèle Oulès
07b6e74314
test: Display skipped tests reason 2022-09-08 12:41:16 +02:00
MacroFake
2557429d2b
Merge bitcoin/bitcoin#26037: test: Fix wallet_{basic,listsinceblock}.py for BDB-only wallets
9f3a315c6f test: Fix `wallet_listsinceblock.py` for BDB-only wallets (Hennadii Stepanov)
1941ce6cd1 test: Fix `wallet_basic.py` for BDB-only wallets (Hennadii Stepanov)

Pull request description:

  Fixes bitcoin/bitcoin#26029.

ACKs for top commit:
  brunoerg:
    crACK 9f3a315c6f

Tree-SHA512: d31c76e558dedea689ff487644e9f2d2f1df1cc2bb9bb041ede4b272884871167fdb19ccc717394c6ba6af8b8c70e9575b344988e0ce55b241a3a4922d0b7f73
2022-09-08 08:56:09 +02:00
Hennadii Stepanov
9f3a315c6f
test: Fix wallet_listsinceblock.py for BDB-only wallets 2022-09-07 19:31:17 +02:00
Hennadii Stepanov
1941ce6cd1
test: Fix wallet_basic.py for BDB-only wallets 2022-09-07 19:31:17 +02:00
fanquake
37095c7dc4
Merge bitcoin/bitcoin#25678: p2p: skip querying dns seeds if -onlynet disables IPv4 and IPv6
385f5a4c3f p2p: Don't query DNS seeds when both IPv4 and IPv6 are unreachable (Martin Zumsande)
91f0a7fbb7 p2p: add only reachable addresses to addrman (Martin Zumsande)

Pull request description:

  Currently, `-onlynet` does not work well in connection with initial peer discovery, because DNS seeds only resolve to IPv6 and IPv4 adresses:
  With `-onlynet=i2p`, we would load clearnet addresses from DNS seeds into addrman, be content our addrman isn't empty so we don't try to query hardcoded seeds (although these exist for i2p!), and never attempt to make an automatic outbound connection.
  With `-onlynet=onion` and `-proxy` set, we wouldn't load addresses via DNS, but will make AddrFetch connections (through a tor exit node) to a random clearnet peer the DNS seed resolves to (see https://github.com/bitcoin/bitcoin/issues/6808#issuecomment-147652505), thus breaching the `-onlynet` preference of the user - this has been reported in the two issues listed below.

  This PR proposes two changes:
  1.) Don't load addresses that are unreachable (so that we wouldn't connect to them) into addrman. This is already the case for addresses received via p2p addr messages, this PR implements the same for addresses received from DNS seeds and fixed seeds. This means that in the case of `-onlynet=onion`, we wouldn't load fixed seed IPv4 addresses into addrman, only the onion ones.
  2.) Skip trying the DNS seeds if neither IPv4 nor IPv6 are reachable and move directly to adding the hardcoded seeds from networks we can connect to. This is done by soft-setting `-dnsseed` to 0 in this case, unless `-dnsseed=1` was explicitly specified, in which case we abort with an `InitError`.

  Fixes #6808
  Fixes #12344

ACKs for top commit:
  naumenkogs:
    utACK 385f5a4c3f
  vasild:
    ACK 385f5a4c3f

Tree-SHA512: 33a8c29faccb2d9b937b017dba4ef72c10e05e458ccf258f1aed3893bcc37c2e984ec8de998d2ecfa54282abbf44a132e97d98bbcc24a0dcf1871566016a9b91
2022-09-07 18:28:42 +01:00
Martin Zumsande
385f5a4c3f p2p: Don't query DNS seeds when both IPv4 and IPv6 are unreachable
This happens, for example, if the user specified -onlynet=onion or
-onlynet=i2p. DNS seeds only resolve to IPv4 / IPv6 addresses,
making their answers useless to us, since we don't want to make
connections to these.
If, within the DNS seed thread, we'd instead do fallback AddrFetch
connections to one of the clearnet addresses the DNS seed resolves to,
we might get usable addresses from other networks
if lucky, but would be violating our -onlynet user preference
in doing so.

Therefore, in this case it is better to rely on fixed seeds for networks we
want to connect to.

Co-authored-by: Vasil Dimov <vd@FreeBSD.org>
2022-09-06 15:16:35 -04:00
Vasil Dimov
2d0b4e4ff6
init: allow startup with -onlynet=onion -listenonion=1
It does not make sense to specify `-onlynet=onion` without providing a
Tor proxy (even if other `-onlynet=...` are given). This is checked
during startup. However, it was forgotten that a Tor proxy can also be
retrieved from "Tor control" to which we connect if `-listenonion=1`.

So, the full Tor proxy retrieval logic is:
1. get it from `-onion`
2. get it from `-proxy`
3. if `-listenonion=1`, then connect to "Tor control" and get the proxy
   from there (was forgotten before this change)

Fixes https://github.com/bitcoin/bitcoin/issues/24980
2022-09-05 17:52:08 +02:00
glozow
5291933fed
Merge bitcoin/bitcoin#25768: wallet: Properly rebroadcast unconfirmed transaction chains
3405f3eed5 test: Test that an unconfirmed not-in-mempool chain is rebroadcast (Andrew Chow)
10d91c5abe wallet: Deduplicate Resend and ReacceptWalletTransactions (Andrew Chow)

Pull request description:

  Currently `ResendWalletTransactions` (used for normal rebroadcasts) will attempt to rebroadcast all of the transactions in the wallet in the order they are stored in `mapWallet`. This ends up being random as `mapWallet` is a `std::unordered_map`. However `ReacceptWalletTransactions` (used for adding to the mempool on loading) first sorts the txs by wallet insertion order, then submits them. The result is that `ResendWalletTranactions` will fail to rebroadcast child transactions if their txids happen to be lexicographically less than their parent's txid. This PR resolves this issue by combining `ReacceptWalletTransactions` and `ResendWalletTransactions` into a new `ResubmitWalletTransactions` so that the iteration code and basic checks are shared.

  A test has also been added that checks that such transaction chains are rebroadcast correctly.

ACKs for top commit:
  naumenkogs:
    utACK 3405f3eed5
  1440000bytes:
    reACK 3405f3eed5
  furszy:
    Late code review ACK 3405f3ee
  stickies-v:
    ACK 3405f3eed5

Tree-SHA512: 1240d9690ecc2ae8d476286b79e2386f537a90c41dd2b8b8a5a9c2a917aa3af85d6aee019fbbb05e772985a2b197e2788305586d9d5dac78ccba1ee5aa31d77a
2022-09-05 13:54:36 +01:00
MacroFake
e864f2e4af
Merge bitcoin/bitcoin#25976: QA: rpc_blockchain: Test output of getblock verbosity 0, False, and True
f663b43df0 QA: rpc_blockchain: Test output of getblock verbosity 0, False, and True (Luke Dashjr)

Pull request description:

  Currently getblock's "verbosity" is documented as a NUM, though it has a fallback to Boolean for the (deprecated?) "verbose" alias.

  Since we've been doing more generic type-checking on RPC stuff, I think it would be a good idea to actually test the Boolean values work.

  I didn't see an existing test for verbosity=0, so this adds that too.

ACKs for top commit:
  aureleoules:
    ACK f663b43df0.

Tree-SHA512: 321a7795a2f32e469d28879dd323c85cb6b221828030e2a33ad9afd35a648191151a79b04e359b2f58314e43360f81c25f05be07deb42f61efdf556850a7266c
2022-09-05 14:15:29 +02:00
fanquake
0ebd4db32b
Merge bitcoin/bitcoin#25978: test: fix non-determinism in p2p_headers_sync_with_minchainwork.py
88e7807e77 test: fix non-determinism in p2p_headers_sync_with_minchainwork.py (Suhas Daftuar)

Pull request description:

  The test for node3's chaintips (added in PR25960) needs some sort of synchronization in order to be reliable.

ACKs for top commit:
  mzumsande:
    Code Review ACK 88e7807e77
  satsie:
    ACK 88e7807e77

Tree-SHA512: 5607c5b1a95d91e7cf81b695eb356b782cbb303bcc7fd9044e1058c0c0625c5f9e5fe4f4dde9d2bffa27a80d83fc060336720f7becbba505ccfb8a04fcc81705
2022-09-04 22:32:22 +01:00
fanquake
604015ac79
Merge bitcoin/bitcoin#25914: test: Fix intermittent issue in p2p_leak.py
fa2aae597c test: Fix intermittent issue in p2p_leak.py (MacroFake)

Pull request description:

  Diff to reproduce:

  ```diff
  diff --git a/src/net.cpp b/src/net.cpp
  index 865ce2ea97..ccf289d77b 100644
  --- a/src/net.cpp
  +++ b/src/net.cpp
  @@ -1150,6 +1150,7 @@ bool CConnman::InactivityCheck(const CNode& node) const

       if (last_recv.count() == 0 || last_send.count() == 0) {
           LogPrint(BCLog::NET, "socket no message in first %i seconds, %d %d peer=%d\n", count_seconds(m_peer_connect_timeout), last_recv.count() != 0, last_send.count() != 0, node.GetId());
  +        UninterruptibleSleep(6s);
           return true;
       }

  ```

  Example in CI:

  ```
   node0 2022-08-12T09:51:56.015288Z [net] [net.cpp:1152] [InactivityCheck] [net] socket no message in first 3 seconds, 0 0 peer=0
   test  2022-08-12T09:51:57.658000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 133, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/p2p_leak.py", line 155, in run_test
                                         assert not no_version_idle_peer.is_connected
                                     AssertionError
  ```

  https://cirrus-ci.com/task/5346634421764096?logs=ci#L3683

ACKs for top commit:
  satsie:
    ACK fa2aae597c
  luke-jr:
    tACK fa2aae597c

Tree-SHA512: e6ddf5b985f7da365b18b699ff8d0719b71b44e4e6bc5576d4099d1bad2c702495afd85f69f4edba89a883e13756a340946db2e7f4be41b1ac0e3c4f515ca4fd
2022-09-04 10:08:52 +01:00
stickies-v
2186608172
test: apply fixed feerate to avoid variable dynamic fees
Without specifying a feerate, we let the wallet decide on an
appropriate feerate, which can be influenced by various factors
such as what's in the mempool. Since wallet_groups.py fails when
feerates are unstable, we should use a fixed feerate across all nodes.

Closes #25940
2022-09-03 00:49:14 +01:00
Andrew Chow
7281fac2e0
Merge bitcoin/bitcoin#25614: Severity-based logging, step 2
9580480570 Update debug logging section in the developer notes (Jon Atack)
1abaa31aa3 Update -debug and -debugexclude help docs for severity level logging (Jon Atack)
45f9282162 Create BCLog::Level::Trace log severity level (Jon Atack)
2a8712db4f Unit test coverage for -loglevel configuration option (klementtan)
eb7bee5f84 Create -loglevel configuration option (klementtan)
98a1f9c687 Unit test coverage for log severity levels (klementtan)
9c7507bf76 Create BCLog::Logger::LogLevelsString() helper function (klementtan)
8fe3457dbb Update LogAcceptCategory() and unit tests with log severity levels (klementtan)
c2797cfc60 Add BCLog::Logger::SetLogLevel()/SetCategoryLogLevel() for string inputs (klementtan)
f6c0cc0350 Add BCLog::Logger::m_category_log_levels data member and getter/setter (Jon Atack)
2978b387bf Add BCLog::Logger::m_log_level data member and getter/setter (Jon Atack)
f1379aeca9 Simplify BCLog::Level enum class and LogLevelToStr() function (Jon Atack)

Pull request description:

  This is an updated version of https://github.com/bitcoin/bitcoin/pull/25287 and the next steps in parent PR #25203 implementing, with Klement Tan, user-configurable, per-category severity log levels based on an idea by John Newbery and refined in GitHub discussions by Wladimir Van der Laan and Marco Falke.

  - simplify the `BCLog::Level` enum class and the `LogLevelToStr()` function and add documentation
  - update the logging logic to filter logs by log level both globally and per-category
  - add a hidden `-loglevel` help-debug config option to allow testing setting the global or per-category severity level on startup for logging categories enabled with the `-debug` configuration option or the logging RPC (Klement Tan)
  - add a `trace` log severity level selectable by the user; the plan is for the current debug messages to become trace, LogPrint ones to become debug, and LogPrintf ones to become info, warning, or error

  ```
  $ ./src/bitcoind -help-debug | grep -A10 loglevel
    -loglevel=<level>|<category>:<level>
         Set the global or per-category severity level for logging categories
         enabled with the -debug configuration option or the logging RPC:
         info, debug, trace (default=info); warning and error levels are
         always logged. If <category>:<level> is supplied, the setting
         will override the global one and may be specified multiple times
         to set multiple category-specific levels. <category> can be:
         addrman, bench, blockstorage, cmpctblock, coindb, estimatefee,
         http, i2p, ipc, leveldb, libevent, lock, mempool, mempoolrej,
         net, proxy, prune, qt, rand, reindex, rpc, selectcoins, tor,
         util, validation, walletdb, zmq.
  ```

  See the individual commit messages for details.

ACKs for top commit:
  jonatack:
    One final push per `git range-diff a5d5569 ce3c4c9 9580480` (should be trivial to re-ACK) to ensure this pull changes no default behavior in any way for users or the tests/CI in order to be completely v24 compatible, to update the unit test setup in general, and to update the debug logging section in the developer notes.
  klementtan:
    reACK 9580480570
  1440000bytes:
    reACK 9580480570
  vasild:
    ACK 9580480570
  dunxen:
    reACK 9580480
  brunoerg:
    reACK 9580480570

Tree-SHA512: 476a638e0581f40b5d058a9992691722e8b546471ec85e07cbc990798d1197fbffbd02e1b3d081b4978404e07a428378cdc8e159c0004b81f58be7fb01b7cba0
2022-09-01 15:57:56 -04:00
Suhas Daftuar
88e7807e77 test: fix non-determinism in p2p_headers_sync_with_minchainwork.py
The test for node3's chaintips needs some sort of synchronization in order to
be reliable.
2022-09-01 15:45:20 -04:00
Andrew Chow
7921026a24
Merge bitcoin/bitcoin#19602: wallet: Migrate legacy wallets to descriptor wallets
53e7ed075c doc: Release notes and other docs for migration (Andrew Chow)
9c44bfe244 Test migratewallet (Andrew Chow)
0b26e7cdf2 descriptors: addr() and raw() should return false for ToPrivateString (Andrew Chow)
31764c3f87 Add migratewallet RPC (Andrew Chow)
0bf7b38bff Implement MigrateLegacyToDescriptor (Andrew Chow)
e7b16f925a Implement MigrateToSQLite (Andrew Chow)
5b62f095e7 wallet: Refactor SetupDescSPKMs to take CExtKey (Andrew Chow)
22401f17e0 Implement LegacyScriptPubKeyMan::DeleteRecords (Andrew Chow)
35f428fae6 Implement LegacyScriptPubKeyMan::MigrateToDescriptor (Andrew Chow)
ea1ab390e4 scriptpubkeyman: Implement GetScriptPubKeys in Legacy (Andrew Chow)
e664af2976 Apply label to all scriptPubKeys of imported combo() (Andrew Chow)

Pull request description:

  This PR adds a new `migratewallet` RPC which migrates a legacy wallet to a descriptor wallet. Migrated wallets will need a new backup. If a wallet has watchonly stuff in it, a new watchonly descriptor wallet will be created containing those watchonly things. The related transactions, labels, and descriptors for those watchonly things will be removed from the original wallet. Migrated wallets will not have any of the legacy things be available for fetching from `getnewaddress` or `getrawchangeaddress`. Wallets that have private keys enabled will have newly generated descriptors. Wallets with private keys disabled will not have any active `ScriptPubKeyMan`s.

  For the basic HD wallet case of just generated keys, in addition to the standard descriptor wallet descriptors using the master key derived from the pre-existing hd seed, the migration will also create 3 descriptors for each HD chain in: a ranged combo external, a ranged combo internal, and a single key combo for the seed (the seed is a valid key that we can receive coins at!). The migrated wallet will then have newly generated descriptors as the active `ScriptPubKeyMan`s. This is equivalent to creating a new descriptor wallet and importing the 3 descriptors for each HD chain. For wallets containing non-HD keys, each key will have its own combo descriptor.

  There are also tests.

ACKs for top commit:
  Sjors:
    tACK 53e7ed075c
  w0xlt:
    reACK 53e7ed075c

Tree-SHA512: c0c003694ca2e17064922d08e8464278d314e970efb7df874b4fe04ec5d124c7206409ca701c65c099d17779ab2136ae63f1da2a9dba39b45f6d62cf93b5c60a
2022-09-01 15:43:30 -04:00
Luke Dashjr
f663b43df0 QA: rpc_blockchain: Test output of getblock verbosity 0, False, and True 2022-09-01 18:47:45 +00:00
fanquake
6ab84709fc
Merge bitcoin/bitcoin#25960: p2p: Headers-sync followups
94af3e43e2 Fix typo from PR25717 (Suhas Daftuar)
e5982ecdc4 Bypass headers anti-DoS checks for NoBan peers (Suhas Daftuar)
132ed7eaaa Move headerssync logging to BCLog::NET (Suhas Daftuar)

Pull request description:

  Remove BCLog::HEADERSSYNC and move all headerssync logging to BCLog::NET.

  Bypass headers anti-DoS checks for NoBan peers

  Also fix a typo that was introduced in PR25717.

ACKs for top commit:
  Sjors:
    tACK 94af3e43e2
  ajtowns:
    ACK 94af3e43e2
  sipa:
    ACK 94af3e43e2
  naumenkogs:
    ACK 94af3e43e2
  w0xlt:
    ACK 94af3e43e2

Tree-SHA512: 612d594eddace977359bcc8234b2093d273fd50662f4ac70cb90903d28fb831f6e1aecff51a4ef6c0bb0f6fb5d1aa7ff1eb8798fac5ac142783788f3080717dc
2022-09-01 07:45:42 +01:00
Andrew Chow
8343420803
Merge bitcoin/bitcoin#25915: test: Fix wallet_balance intermittent issue
fae5bd9200 test: Fix wallet_balance intermittent issue (MacroFake)

Pull request description:

  Diff to reproduce:

  ```diff
  index d2ed97ca76..25cc2d5734 100755
  --- a/test/functional/wallet_balance.py
  +++ b/test/functional/wallet_balance.py
  @@ -265,7 +265,7 @@ class WalletTest(BitcoinTestFramework):
           self.nodes[0].invalidateblock(block_reorg)
           self.nodes[1].invalidateblock(block_reorg)
           assert_equal(self.nodes[0].getbalance(minconf=0), 0)  # wallet txs not in the mempool are untrusted
  -        self.generatetoaddress(self.nodes[0], 1, ADDRESS_WATCHONLY, sync_fun=self.no_op)
  +        self.generatetoaddress(self.nodes[0], 1, ADDRESS_WATCHONLY)
           assert_equal(self.nodes[0].getbalance(minconf=0), 0)  # wallet txs not in the mempool are untrusted

           # Now confirm tx_orig
  ```

  Example in CI:

  ```
   test  2022-08-24T10:09:22.486000Z TestFramework (ERROR): Assertion failed
                                     Traceback (most recent call last):
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 133, in main
                                         self.run_test()
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/wallet_balance.py", line 269, in run_test
                                         assert_equal(self.nodes[0].getbalance(minconf=0), 0)  # wallet txs not in the mempool are untrusted
                                       File "/tmp/cirrus-ci-build/ci/scratch/build/bitcoin-i686-pc-linux-gnu/test/functional/test_framework/util.py", line 56, in assert_equal
                                         raise AssertionError("not(%s)" % " == ".join(str(arg) for arg in (thing1, thing2) + args))
                                     AssertionError: not(98.85983340 == 0)
  ```

  https://cirrus-ci.com/task/4981266251513856?logs=ci#L3269

ACKs for top commit:
  achow101:
    ACK fae5bd9200
  w0xlt:
    ACK fae5bd9200

Tree-SHA512: 470f366720615c4a9326ec4c581fff569ecce9877f9134bb1975ec3d6f1d13a6403051418a91a80b2a86de617f43e539ec11bbf4f1713d0354d5b0ab98d22437
2022-08-31 11:20:23 -04:00
MacroFake
d16ef40441
Merge bitcoin/bitcoin#25955: test: use sendall when emptying wallet
28ea4c7039 test: simplify splitment with `sendall` in wallet_basic (brunoerg)
923d24583d test: use `sendall` when emptying wallet (brunoerg)

Pull request description:

  In some tests they have used `sendtoaddress` in order to empty a wallet. With the addition of `sendall`, it makes sense to use it for that.

ACKs for top commit:
  achow101:
    ACK 28ea4c7039
  ishaanam:
    utACK 28ea4c7039
  w0xlt:
    ACK 28ea4c7039

Tree-SHA512: 903136d7df5c65d3c02310d5a84241c9fd11070f69d932b4e188b8ad45c38ab5bc1bd5a9242b3e52d2576665ead14be0a03971a9ad8c00431fed442eba4ca48f
2022-08-31 09:03:49 +02:00
brunoerg
28ea4c7039 test: simplify splitment with sendall in wallet_basic
recipients receive equal share of the unspecified amount
2022-08-30 16:28:40 -03:00
Suhas Daftuar
e5982ecdc4 Bypass headers anti-DoS checks for NoBan peers 2022-08-30 14:11:21 -04:00
fanquake
e9035f867a
Merge bitcoin/bitcoin#25717: p2p: Implement anti-DoS headers sync
3add234546 ui: show header pre-synchronization progress (Pieter Wuille)
738421c50f Emit NotifyHeaderTip signals for pre-synchronization progress (Pieter Wuille)
376086fc5a Make validation interface capable of signalling header presync (Pieter Wuille)
93eae27031 Test large reorgs with headerssync logic (Suhas Daftuar)
355547334f Track headers presync progress and log it (Pieter Wuille)
03712dddfb Expose HeadersSyncState::m_current_height in getpeerinfo() (Suhas Daftuar)
150a5486db Test headers sync using minchainwork threshold (Suhas Daftuar)
0b6aa826b5 Add unit test for HeadersSyncState (Suhas Daftuar)
83c6a0c524 Reduce spurious messages during headers sync (Suhas Daftuar)
ed6cddd98e Require callers of AcceptBlockHeader() to perform anti-dos checks (Suhas Daftuar)
551a8d957c Utilize anti-DoS headers download strategy (Suhas Daftuar)
ed470940cd Add functions to construct locators without CChain (Pieter Wuille)
84852bb6bb Add bitdeque, an std::deque<bool> analogue that does bit packing. (Pieter Wuille)
1d4cfa4272 Add function to validate difficulty changes (Suhas Daftuar)

Pull request description:

  New nodes starting up for the first time lack protection against DoS from low-difficulty headers. While checkpoints serve as our protection against headers that fork from the main chain below the known checkpointed values, this protection only applies to nodes that have been able to download the honest chain to the checkpointed heights.

  We can protect all nodes from DoS from low-difficulty headers by adopting a different strategy: before we commit to storing a header in permanent storage, first verify that the header is part of a chain that has sufficiently high work (either `nMinimumChainWork`, or something comparable to our tip). This means that we will download headers from a given peer twice: once to verify the work on the chain, and a second time when permanently storing the headers.

  The p2p protocol doesn't provide an easy way for us to ensure that we receive the same headers during the second download of peer's headers chain. To ensure that a peer doesn't (say) give us the main chain in phase 1 to trick us into permanently storing an alternate, low-work chain in phase 2, we store commitments to the headers during our first download, which we validate in the second download.

  Some parameters must be chosen for commitment size/frequency in phase 1, and validation of commitments in phase 2. In this PR, those parameters are chosen to both (a) minimize the per-peer memory usage that an attacker could utilize, and (b) bound the expected amount of permanent memory that an attacker could get us to use to be well-below the memory growth that we'd get from the honest chain (where we expect 1 new block header every 10 minutes).

  After this PR, we should be able to remove checkpoints from our code, which is a nice philosophical change for us to make as well, as there has been confusion over the years about the role checkpoints play in Bitcoin's consensus algorithm.

  Thanks to Pieter Wuille for collaborating on this design.

ACKs for top commit:
  Sjors:
    re-tACK 3add234546
  mzumsande:
    re-ACK 3add234546
  sipa:
    re-ACK 3add234546
  glozow:
    ACK 3add234546

Tree-SHA512: e7789d65f62f72141b8899eb4a2fb3d0621278394d2d7adaa004675250118f89a4e4cb42777fe56649d744ec445ad95141e10f6def65f0a58b7b35b2e654a875
2022-08-30 15:37:59 +01:00
brunoerg
923d24583d test: use sendall when emptying wallet 2022-08-30 09:52:15 -03:00
Andrew Chow
9c44bfe244 Test migratewallet
Co-Authored-By: furszy <matiasfurszyfer@protonmail.com>
2022-08-29 17:30:38 -04:00
Andrew Chow
3405f3eed5 test: Test that an unconfirmed not-in-mempool chain is rebroadcast
The test checks that parent txs are broadcast before child txs.

The previous behavior is that the rebroadcasting would simply iterate mapWallet. As
mapWallet is a std::unsorted_map, the child can sometimes come before the parent and thus
be rebroadcast in the wrong order and fail the test.
2022-08-29 12:41:50 -04:00
Andrew Chow
10d91c5abe wallet: Deduplicate Resend and ReacceptWalletTransactions
Both of these functions do almost the exact same thing. They can be
deduplicated so that their behavior matches except for the filtering
aspect. As this function will now always be called on wallet loading,
nNextResend will also always be initialized, so
wallet_resendwallettransactions.py is updated to account for that.

This also resolves a bug where ResendWalletTransactions would fail to
rebroadcast txs in insertion order thereby potentially rebroadcasting a
child transaction before its parent and causing the child to not
actually get rebroadcast.

Also names the combined function to ResubmitWalletTransactions as the
function just submits the transactions to the mempool rather than doing
any sending by itself.
2022-08-29 12:38:06 -04:00
Suhas Daftuar
93eae27031 Test large reorgs with headerssync logic 2022-08-29 08:10:35 -04:00
Suhas Daftuar
03712dddfb Expose HeadersSyncState::m_current_height in getpeerinfo() 2022-08-29 08:10:35 -04:00
Suhas Daftuar
150a5486db Test headers sync using minchainwork threshold 2022-08-29 08:10:35 -04:00
Suhas Daftuar
ed6cddd98e Require callers of AcceptBlockHeader() to perform anti-dos checks
In order to prevent memory DoS, we must ensure that we don't accept a new
header into memory until we've performed anti-DoS checks, such as verifying
that the header is part of a sufficiently high work chain. This commit adds a
new argument to AcceptBlockHeader() so that we can ensure that all call-sites
which might cause a new header to be accepted into memory have to grapple with
the question of whether the header is safe to accept, or needs further
validation.

This patch also fixes two places where low-difficulty-headers could have been
processed without such validation (processing an unrequested block from the
network, and processing a compact block).

Credit to Niklas Gögge for noticing this issue, and thanks to Sjors Provoost
for test code.
2022-08-29 08:10:35 -04:00
Suhas Daftuar
551a8d957c Utilize anti-DoS headers download strategy
Avoid permanently storing headers from a peer, unless the headers are part of a
chain with sufficiently high work. This prevents memory attacks using low-work
headers.

Designed and co-authored with Pieter Wuille.
2022-08-29 08:10:35 -04:00
MacroFake
fae5bd9200
test: Fix wallet_balance intermittent issue
Fix it by removing a duplicate balance check on the same node.
2022-08-27 17:24:31 +02:00
Andrew Chow
e191fac4f3
Merge bitcoin/bitcoin#25922: wallet: trigger MaybeResendWalletTxs() every minute
5ef8c2c9fc test: fix typo for MaybeResendWalletTxs (stickies-v)
fbba4a1316 wallet: trigger MaybeResendWalletTxs() every minute (stickies-v)

Pull request description:

  ResendWalletTransactions() only executes every [12-36h (24h average)](1420547ec3/src/wallet/wallet.cpp (L1947)). Triggering it every second is excessive, once per minute should be plenty.

  The goal of this PR is to reduce the amount of (unnecessary) schedule executions by ~60x without meaningfully altering transaction rebroadcast logic/assumptions which would require more significant review.

ACKs for top commit:
  achow101:
    ACK 5ef8c2c9fc
  1440000bytes:
    ACK 5ef8c2c9fc

Tree-SHA512: 4a077e3579b289c11c347eaa0d3601ef2dbb9fee66ab918d56b4a0c2e08222560a0e6be295297a74831836e001a997ecc143adb0c132faaba96a669dac1cd9e6
2022-08-26 17:11:17 -04:00
Andrew Chow
eed2bd37ef
Merge bitcoin/bitcoin#25355: I2P: add support for transient addresses for outbound connections
59aa54f731 i2p: log "SAM session" instead of "session" (Vasil Dimov)
d7ec30b648 doc: add release notes about the I2P transient addresses (Vasil Dimov)
47c0d02f12 doc: document I2P transient addresses usage in doc/i2p.md (Vasil Dimov)
3914e472f5 test: add a test that -i2pacceptincoming=0 creates a transient session (Vasil Dimov)
ae1e97ce86 net: use transient I2P session for outbound if -i2pacceptincoming=0 (Vasil Dimov)
a1580a04f5 net: store an optional I2P session in CNode (Vasil Dimov)
2b781ad66e i2p: add support for creating transient sessions (Vasil Dimov)

Pull request description:

  Add support for generating a transient, one-time I2P address for ourselves when making I2P outbound connection and discard it once the connection is closed.

  Background
  ---
  In I2P connections, the host that receives the connection knows the I2P address of the connection initiator. This is unlike the Tor network where the recipient does not know who is connecting to them, not even the initiator's Tor address.

  Persistent vs transient I2P addresses
  ---
  Even if an I2P node is not accepting incoming connections, they are known to other nodes by their outgoing I2P address. This creates an opportunity to white-list given nodes or treat them differently based on their I2P address. However, this also creates an opportunity to fingerprint or analyze a given node because it always uses the same I2P address when it connects to other nodes. If this is undesirable, then a node operator can use the newly introduced `-i2ptransientout` to generate a transient (disposable), one-time I2P address for each new outgoing connection. That address is never going to be reused again, not even if reconnecting to the same peer later.

ACKs for top commit:
  mzumsande:
    ACK 59aa54f731 (verified via range-diff that just a typo / `unique_ptr` initialisation were fixed)
  achow101:
    re-ACK 59aa54f731
  jonatack:
    utACK 59aa54f731 reviewed range diff, rebased to master, debug build + relevant tests + review at each commit

Tree-SHA512: 2be9b9dd7502b2d44a75e095aaece61700766bff9af0a2846c29ca4e152b0a92bdfa30f61e8e32b6edb1225f74f1a78d19b7bf069f00b8f8173e69705414a93e
2022-08-26 16:33:58 -04:00
Andrew Chow
e664af2976 Apply label to all scriptPubKeys of imported combo() 2022-08-25 16:25:53 -04:00
stickies-v
5ef8c2c9fc
test: fix typo for MaybeResendWalletTxs 2022-08-25 14:29:26 +01:00
stickies-v
fbba4a1316
wallet: trigger MaybeResendWalletTxs() every minute
ResendWalletTransactions() only executes every 12-36h (24h average).
Triggering it every second is excessive, once per minute should be
plenty.
2022-08-25 14:29:25 +01:00
MacroFake
fa2aae597c
test: Fix intermittent issue in p2p_leak.py 2022-08-24 12:51:11 +02:00
MacroFake
3c1e75ef60
Merge bitcoin/bitcoin#25865: test: speedup wallet tests by whitelisting peers (immediate tx relay)
b21e522ce4 test: speedup wallet tests by whitelisting peers (immediate tx relay) (Sebastian Falbesoner)

Pull request description:

  In the course of testing #25297 by running all wallet-related functional tests (see https://github.com/bitcoin/bitcoin/pull/25297#issuecomment-1203365589), I noticed that the run-time of those tests vary a lot between runs, in fact too much for a useful comparison. This PR fixes this by making the tests both more deterministic and also faster, using the good ol' immediate tx relay trick (parameter `-whitelist=noban@127.0.0.1`).

  master branch:
  ```
  wallet_abandonconflict.py --descriptors   | ✓ Passed  | 7 s
  wallet_abandonconflict.py --legacy-wallet | ✓ Passed  | 23 s
  wallet_balance.py --descriptors           | ✓ Passed  | 17 s
  wallet_balance.py --legacy-wallet         | ✓ Passed  | 21 s
  wallet_basic.py --descriptors             | ✓ Passed  | 32 s
  wallet_basic.py --legacy-wallet           | ✓ Passed  | 56 s
  wallet_bumpfee.py --descriptors           | ✓ Passed  | 44 s
  wallet_bumpfee.py --legacy-wallet         | ✓ Passed  | 45 s
  wallet_groups.py --descriptors            | ✓ Passed  | 89 s
  wallet_groups.py --legacy-wallet          | ✓ Passed  | 94 s
  wallet_hd.py --descriptors                | ✓ Passed  | 7 s
  wallet_hd.py --legacy-wallet              | ✓ Passed  | 13 s
  wallet_importdescriptors.py --descriptors | ✓ Passed  | 26 s
  wallet_listreceivedby.py --descriptors    | ✓ Passed  | 28 s
  wallet_listreceivedby.py --legacy-wallet  | ✓ Passed  | 18 s

  ALL                                       | ✓ Passed  | 520 s (accumulated)
  Runtime: 526 s
  ```

  PR branch:
  ```
  wallet_abandonconflict.py --descriptors   | ✓ Passed  | 7 s
  wallet_abandonconflict.py --legacy-wallet | ✓ Passed  | 11 s
  wallet_balance.py --descriptors           | ✓ Passed  | 8 s
  wallet_balance.py --legacy-wallet         | ✓ Passed  | 8 s
  wallet_basic.py --descriptors             | ✓ Passed  | 29 s
  wallet_basic.py --legacy-wallet           | ✓ Passed  | 36 s
  wallet_bumpfee.py --descriptors           | ✓ Passed  | 39 s
  wallet_bumpfee.py --legacy-wallet         | ✓ Passed  | 32 s
  wallet_groups.py --descriptors            | ✓ Passed  | 39 s
  wallet_groups.py --legacy-wallet          | ✓ Passed  | 41 s
  wallet_hd.py --descriptors                | ✓ Passed  | 8 s
  wallet_hd.py --legacy-wallet              | ✓ Passed  | 11 s
  wallet_importdescriptors.py --descriptors | ✓ Passed  | 17 s
  wallet_listreceivedby.py --descriptors    | ✓ Passed  | 7 s
  wallet_listreceivedby.py --legacy-wallet  | ✓ Passed  | 9 s

  ALL                                       | ✓ Passed  | 302 s (accumulated)
  Runtime: 309 s
  ```
  Note that an alternative approach could be to whitelist peers by default for nodes in the functional test framework and only enable the trickle relay for the few tests where it's really needed.

ACKs for top commit:
  naumenkogs:
    utACK b21e522ce4

Tree-SHA512: ac3c8f8f5a401d1b6af60ece9c77e72449f18920c2cb4a1bd65fb4d62cf428779ebf4e1d29009a882977b2252922df4e7183541e0da8de932f8cd479149e8a86
2022-08-24 10:37:25 +02:00
MacroFake
713ea7a418
Merge bitcoin/bitcoin#25906: test: add coverage for invalid parameters for rescanblockchain
d1a0004621 test: add coverage for invalid parameters for `rescanblockchain` (brunoerg)

Pull request description:

  This PR adds test coverage for the following errors:
  2bd9aa5a44/src/wallet/rpc/transactions.cpp (L880-L894)

ACKs for top commit:
  w0xlt:
    reACK d1a0004621

Tree-SHA512: c357fbda3d261e4d06a29d2a5350482db5f97a815adf59abdac1971eb19b69cfd4d54e4d21836851e2e3b116aa2a820ea1437c7aededf86b06df435cca16ac90
2022-08-24 08:51:40 +02:00
brunoerg
d1a0004621 test: add coverage for invalid parameters for rescanblockchain 2022-08-23 17:13:52 -03:00
fanquake
c5f0cbefa3
Merge bitcoin/bitcoin#25775: docs: remove non-signaling mentions of BIP125
1dc03dda05 [doc] remove non-signaling mentions of BIP125 (glozow)
32024d40f0 scripted-diff: remove mention of BIP125 from non-signaling var names (glozow)

Pull request description:

  We have pretty thorough documentation of our RBF policy in doc/policy/mempool-replacements.md. It enumerates each rule with several sentences of rationale. Also, each rule pretty much has its own function (3 and 4 share one), with extensive comments. The doc states explicitly that our rules are similar but differ from BIP125, and contains a record of historical changes to RBF policy.

  We should not use "BIP125" as synonymous with our RBF policy because:
  - Our RBF policy is different from what is specified in BIP125, for example:
      - the BIP does not mention our rule about the replacement feerate being higher (our Rule 6)
      - the BIP uses minimum relay feerate for Rule 4, while we have used incremental relay feerate since #9380
      - the "inherited signaling" question (CVE-2021-31876). Call it discrepancy, ambiguous wording, doc misinterpretation, or implementation details, I would recommend users refer to doc/policy/mempool-replacements.md
      - the signaling policy is configurable, see #25353
  - Our RBF policy may change further
  - We have already marked BIP125 as only "partially implemented" in docs/bips.md since 1fd49eb498
  - See comments from people who are not me recently:
      - https://github.com/bitcoin/bitcoin/pull/25038#discussion_r909507429
      - https://github.com/bitcoin/bitcoin/pull/25575#issuecomment-1179519204

  This PR removes all non-signaling mentions of BIP125 (if people feel strongly, we can remove all mentions of BIP125 period). It may be useful to refer to the concept of "tx opts in to RBF if it has at least one nSequence less than (0xffffffff - 1)" as "BIP125 signaling" because:
  - It is succint.
  - It has already been widely marketed as BIP125 opt-in signaling.
  - Our API uses it when referring to signaling (e.g. getmempoolentry["bip125-replaceable"] and wallet error message "not BIP 125 replaceable"). Changing those is more invasive.
  - If/when we have other ways to signal in the future, we can disambiguate them this way. See #25038 which proposes another way of signaling, and where I pulled these commits from.

  Alternatives:
  - Changing our policy to match BIP125. This doesn't make sense as, for example, we would have to remove the requirement that a replacement tx has a higher feerate (Rule 6).
  - Changing BIP125 to match what we have. This doesn't make sense as it would be a significant change to a BIP years after it was finalized and already used as a spec to implement RBF in other places.
  - Document our policy as a new BIP and give it a number. This might make sense if we don't expect things to change a lot, and can be done as a next step.

ACKs for top commit:
  darosior:
    ACK 1dc03dda05
  ariard:
    ACK 1dc03dda
  t-bast:
    ACK 1dc03dda05

Tree-SHA512: a3adc2039ec5785892d230ec442e50f47f7062717392728152bbbe27ce1c564141f85253143f53cb44e1331cf47476d74f5d2f4b3cd873fc3433d7a0aa783e02
2022-08-22 10:35:26 +01:00
fanquake
607d5a46aa
Merge bitcoin/bitcoin#23202: wallet: allow psbtbumpfee to work with txs with external inputs
c3b099ace0 wallet, tests: Test bumpfee's max input weight calculation (Andrew Chow)
116a620ce7 Make DUMMY_CHECKER availble outside of script/sign.cpp (Andrew Chow)
ff638323d1 test, bumpfee: Check that psbtbumpfee can bump txs with external inputs (Andrew Chow)
1bc8106d4c bumpfee: be able to bump fee of a tx with external inputs (Andrew Chow)
31dd3dc9e5 bumpfee: Clear scriptSigs and scriptWitnesses before calculated max size (Andrew Chow)
a0c3afb898 bumpfee: extract weights of external inputs when bumping fee (Andrew Chow)
612f1e44fe bumpfee: Calculate fee by looking up UTXOs (Andrew Chow)

Pull request description:

  This PR allows `psbtbumpfee` to return a PSBT for transactions that contain external inputs. This does not work for bumping in the GUI nor `bumpfee` because these need private keys available to sign and send the transaction. But `psbtbumpfee` returns a psbt, so it is fine to not be able to sign.

  In order to correctly estimate the size of the inputs for coin selection, the fee bumper will use the size of the inputs of the transaction being bumped. Because the sizes of signatures are not guaranteed, for external inputs, the fee bumper will verify the scripts with a special SignatureChecker which will compute the weight of all of the signatures in that input, and compute their weights if those signatures were maximally sized. This allows the fee bumper to obtain a max size estimate for each external input.

  Builds on #23201 as it relies on the ability to pass weights in to coin selection.

  Closes #23189

ACKs for top commit:
  ishaanam:
    reACK c3b099ace0
  t-bast:
    Re-ran my tests agains c3b099ace0, ACK

Tree-SHA512: 40016ec52d351430977579cfa2694c7e6764f42c9ce09d3a6f1753b767f86053f296d9de988248df033be6d725d67badbf2a5ef82c8ace23c61487729b7691e5
2022-08-22 10:12:19 +01:00
Jon Atack
45f9282162 Create BCLog::Level::Trace log severity level
for verbose log messages for development or debugging only, as bitcoind may run
more slowly, that are more granular/frequent than the Debug log level, i.e. for
very high-frequency, low-level messages to be logged distinctly from
higher-level, less-frequent debug logging that could still be usable in production.

An example would be to log higher-level peer events (connection, disconnection,
misbehavior, eviction) as Debug, versus Trace for low-level, high-volume p2p
messages in the BCLog::NET category. This will enable the user to log only the
former without the latter, in order to focus on high-level peer management events.

With respect to the name, "trace" is suggested as the most granular level
in resources like the following:
- https://sematext.com/blog/logging-levels
- https://howtodoinjava.com/log4j2/logging-levels

Update the test framework and add test coverage.
2022-08-20 11:55:17 +02:00
klementtan
8fe3457dbb Update LogAcceptCategory() and unit tests with log severity levels
Co-authored-by: "Jon Atack <jon@atack.com>"
2022-08-20 11:30:51 +02:00
Andrew Chow
ff638323d1 test, bumpfee: Check that psbtbumpfee can bump txs with external inputs 2022-08-19 14:37:36 -04:00
Andrew Chow
02dea9a47f tests: Use mocktime for wallet encryption timeout 2022-08-19 13:51:39 -04:00
Andrew Chow
ef8e2a5b09 tests: Test that external inputs of txs in wallet is handled correctly 2022-08-18 11:07:22 -04:00
Andrew Chow
a537d7aaa0 wallet: SelectExternal actually external inputs
If an external input's utxo was created by a transaction that the wallet
knows about, then it would not be selected using SelectExternal. This
results in either funding failure or incorrect weight calculation.
2022-08-18 11:00:12 -04:00
Sebastian Falbesoner
b21e522ce4 test: speedup wallet tests by whitelisting peers (immediate tx relay) 2022-08-18 00:15:21 +02:00
Andrew Chow
64f7a1940d
Merge bitcoin/bitcoin#25734: wallet, refactor: #24584 follow-ups
8cd21bb279 refactor: improve readability for AttemptSelection (josibake)
f47ff71761 test: only run test for descriptor wallets (josibake)
0760ce0b9e test: add missing BOOST_ASSERT (josibake)
db09aec937 wallet: switch to new shuffle, erase, push_back (josibake)
b6b50b0f2b scripted-diff: Uppercase function names (josibake)
3f27a2adce refactor: add new helper methods (josibake)
f5649db9d5 refactor: add UNKNOWN OutputType (josibake)

Pull request description:

  This PR is to address follow-ups for #24584, specifically:

  * Remove redundant, hard-to-read code by adding a new `OutputType` and adding shuffle, erase, and push_back methods for `CoinsResult`
  * Add missing `BOOST_ASSERT` to unit test
  * Ensure functional test only runs if using descriptor wallets
  * Improve readability of `AttemptSelection` by removing triple-nested if statement

  Note for reviewers: commit `refactor: add new helper methods` should throw an "unused function warning"; the function is used in the next commit. Also, commit `wallet: switch to new shuffle, erase, push_back` will fail to compile, but this is fixed in the next commit with a scripted-diff. the commits are separate like this (code change then scripted-diff) to improve legibility.

ACKs for top commit:
  achow101:
    ACK 8cd21bb279
  aureleoules:
    ACK 8cd21bb279.
  LarryRuane:
    Concept, code review ACK 8cd21bb279
  furszy:
    utACK 8cd21bb2. Left a small, non-blocking, comment.

Tree-SHA512: a1bbc5962833e3df4f01a4895d8bd748cc4c608c3f296fd94e8afd8797b8d2e94e7bd44d598bd76fa5c9f5536864f396fcd097348fa0bb190a49a86b0917d60e
2022-08-16 20:00:19 -04:00
brunoerg
3e44bee08e test: add coverage for /rest/deploymentinfo 2022-08-16 19:21:51 -03:00
Andrew Chow
c336f813b3
Merge bitcoin/bitcoin#25504: RPC: allow to track coins by parent descriptors
a6b0c1fcc0 doc: add releases notes for 25504 (listsinceblock updates) (Antoine Poinsot)
0fd2d14454 rpc: add an include_change parameter to listsinceblock (Antoine Poinsot)
55f98d087e rpc: output parent wallet descriptors for coins in listunspent (Antoine Poinsot)
b724476158 rpc: output wallet descriptors for received entries in listsinceblock (Antoine Poinsot)
55a82eaf91 wallet: allow to fetch the wallet descriptors for a given Script (Antoine Poinsot)

Pull request description:

  Wallet descriptors are useful for applications using the Bitcoin Core wallet as a backend for tracking coins, as they allow to track coins for multiple descriptors in a single wallet. However there is no information currently given for such applications to link a coin with an imported descriptor, severely limiting the possibilities for such applications of using multiple descriptors in a single wallet. This PR outputs the matching imported descriptor(s) for a given received coin in `listsinceblock` (and friends).

  It comes from a need for an application i'm working on, but i think it's something any software using `bitcoind` to track multiple descriptors in a single wallet would have eventually. For instance i'm thinking about the BDK project. Currently, the way to achieve this is to import raw addresses with labels and to have your application be responsible for wallet things like the gap limit.

  I'll add this to the output of `listunspent` too if this gets a few Concept ACKs.

ACKs for top commit:
  instagibbs:
    ACK a6b0c1fcc0
  achow101:
    re-ACK a6b0c1fcc0

Tree-SHA512: 7a5850e8de98b439ddede2cb72de0208944f8cda67272e8b8037678738d55b7a5272375be808b0f7d15def4904430e089dafdcc037436858ff3292c5f8b75e37
2022-08-16 13:08:05 -04:00
Antoine Poinsot
0fd2d14454
rpc: add an include_change parameter to listsinceblock
It's useful for an external application tracking coins to not be limited
by our change detection. For instance, for a watchonly wallet with two
descriptors a transaction from one to the other would be considered a
change output and not be included in the result (if the address was not
generated by this wallet).
2022-08-16 18:33:05 +02:00
Vasil Dimov
3914e472f5
test: add a test that -i2pacceptincoming=0 creates a transient session
The test is a bit primitive as it checks the Bitcoin Core log and
assumes that if it logs that it creates a transient session, then it
does that indeed.

A more thorough test would be to check that it indeed sends the
`SESSION CREATE ... DESTINATION=TRANSIENT` command and that it uses
the returned I2P address for connecting, even for repeated connections
to the same I2P peer. That would require a mocked SAM server (proxy)
implementation in Python.
2022-08-16 13:02:19 +02:00
fanquake
cf39913e57
Merge bitcoin/bitcoin#25803: refactor: Drop boost/algorithm/string/replace.hpp dependency
fea75ad3ca refactor: Drop `boost/algorithm/string/replace.hpp` dependency (Hennadii Stepanov)
857526e8cb test: Add test case for `ReplaceAll()` function (Hennadii Stepanov)

Pull request description:

  A new implementation of the `ReplaceAll()` seems enough for all of our purposes.

ACKs for top commit:
  adam2k:
    ACK Tested fea75ad3ca
  theStack:
    Code-review ACK fea75ad3ca

Tree-SHA512: dacfffc9d2bd1fb9f034baf8c045b1e8657b766db2f0a7f8ef7e25ee6cd888f315b0124c54aba7a29ae59186b176ef9868a8b709dc995ea215c6b4ce58e174d9
2022-08-16 09:19:28 +01:00
Andrew Chow
22d96d76ab
Merge bitcoin/bitcoin#25720: p2p: Reduce bandwidth during initial headers sync when a block is found
f6a916683d Add functional test for block announcements during initial headers sync (Suhas Daftuar)
05f7f31598 Reduce bandwidth during initial headers sync when a block is found (Suhas Daftuar)

Pull request description:

  On startup, if our headers chain is more than a day behind current time, we'll pick one peer to sync headers with until our best headers chain is caught up (at that point, we'll try to sync headers with all peers).

  However, if an INV for a block is received before our headers chain is caught up, we'll then start to sync headers from each peer announcing the block.  This can result in doing a big headers sync with many (if not all) of our peers simultaneously, which wastes bandwidth.

  This PR would reduce that overhead by picking (at most) one new peer to try syncing headers with whenever a new block is announced, prior to our headers chain being caught up.

ACKs for top commit:
  LarryRuane:
    ACK f6a916683d
  ajtowns:
    ACK f6a916683d
  mzumsande:
    ACK f6a916683d
  dergoegge:
    Code review ACK f6a916683d
  achow101:
    ACK f6a916683d

Tree-SHA512: 0662000bd68db146f55981de4adc2e2b07cbfda222b1176569d61c22055e5556752ffd648426f69687ed1cc203105515e7304c12b915d6270df8e41a4a0e1eaa
2022-08-15 15:43:41 -04:00
fanquake
dc9d662683
Merge bitcoin/bitcoin#25235: GetExternalSigner(): fail if multiple signers are found
292b1a3e9c GetExternalSigner(): fail if multiple signers are found (amadeuszpawlik)

Pull request description:

  If there are multiple external signers, `GetExternalSigner()` will
  just pick the first one in the list. If the user has two or more
  hardware wallets connected at the same time, he might not notice this.

  This PR adds a check and fails with suitable message, forcing the user to disconnect all but one external signer, so that there is no ambiguity as to which external signer was used.

ACKs for top commit:
  Sjors:
    tACK 292b1a3e9c
  achow101:
    ACK 292b1a3e9c

Tree-SHA512: e2a41d3eecc607d4f94e708614bed0f3545f7abba85f300c5a5f0d3d17d72c815259734accc5ca370953eacd290f27894ba2c18016f5e9584cd50fa1ec2fbb0b
2022-08-13 16:08:19 +01:00
Suhas Daftuar
f6a916683d Add functional test for block announcements during initial headers sync 2022-08-12 17:13:00 -04:00
MacroFake
29c195cf6a
Merge bitcoin/bitcoin#25792: test: add tests for datacarrier and datacarriersize options
8b3d2bbd0d test: add tests for `datacarrier` and `datacarriersize` options (w0xlt)

Pull request description:

  As suggested in https://github.com/bitcoin/bitcoin/issues/25787, this PR adds tests for `datacarrier` and `datacarriersize` initialization options.

  Close https://github.com/bitcoin/bitcoin/issues/25787.

ACKs for top commit:
  theStack:
    re-ACK 8b3d2bbd0d
  stickies-v:
    re-ACK 8b3d2bbd0d

Tree-SHA512: 962638ac9659f9d641bc5d1eff0571a08085dc7d4981b534b7ede03e4c702abd7048d543c199a588e2f94567b6d2393280e686629b19d7f4b24d365662be5e40
2022-08-11 18:04:30 +02:00
w0xlt
8b3d2bbd0d
test: add tests for datacarrier and datacarriersize options
Co-authored-by: Sebastian Falbesoner <sebastian.falbesoner@gmail.com>
2022-08-11 12:05:09 -03:00
fanquake
0094ff3947
Merge bitcoin/bitcoin#25812: psbt: Avoid unsigned int overflow in PSBT_IN_TAP_BIP32_DERIVATION
70a55c059b psbt: Avoid unsigned int overflow in PSBT_IN_TAP_BIP32_DERIVATION (Andrew Chow)

Pull request description:

  Fixes #25749

ACKs for top commit:
  instagibbs:
    ACK 70a55c059b
  darosior:
    re-utACK 70a55c059b
  jonatack:
    Review ACK 70a55c059b, this should avoid the issue reported in https://github.com/bitcoin/bitcoin/issues/25749

Tree-SHA512: 6bb58e1cda9a5baa50fcd24f818b5b27ed94f0d33da3f71f6e457618176611bf2a84e1864e9a48d9303c301252bc4c1dee8b19a67dd713e849fb9442851ca341
2022-08-11 10:12:20 +01:00
MacroFake
251c535800
Merge bitcoin/bitcoin#25810: scripted-diff: test: rename MAX_{ANCESTORS,DESCENDANTS} to DEFAULT_{ANCESTOR,DESCENDANT}_LIMIT
b4a5ab96b4 test: refactor: deduplicate `DEFAULT_{ANCESTOR,DESCENDANT}_LIMIT` constants (Sebastian Falbesoner)
0fda1c7df6 scripted-diff: test: rename `MAX_{ANCESTORS,DESCENDANTS}` to `DEFAULT_{ANCESTOR,DESCENDANT}_LIMIT` (Sebastian Falbesoner)

Pull request description:

  This PR renames the default in-mempool max ancestors/descendants constants `MAX_ANCESTORS`/`MAX_DESCENDANTS` in the functional tests to match the ones in the codebase:
  c012875b9d/src/policy/policy.h (L58-L59)
  c012875b9d/src/policy/policy.h (L62-L63)
  The custom limit constants `MAX_ANCESTORS_CUSTOM`/`MAX_DESCENDANTS_CUSTOM` are also renamed accordingly to better fit to this naming style. In the second commit, the default constants are deduplicated by moving them into the `messages.py` module. (Not sure if this module is really appropriate, as it doesn't have a connection to messages. If someone has a good suggestion, would be glad to hear it.)

ACKs for top commit:
  w0xlt:
    ACK b4a5ab96b4
  glozow:
    utACK b4a5ab96b4
  fanquake:
    ACK b4a5ab96b4

Tree-SHA512: a15c8256170afce3e383fceddcb562f588a02be97ce4202c84a2ebf22d73ab843f5e5a7d7c98e9ea044d8bcb7a4aeae0081d0e84c53e8fc0edffbcca00460139
2022-08-10 19:23:35 +02:00
MacroFake
f89ce1fdb5
Merge bitcoin/bitcoin#25811: doc: test: suggest multi-line imports in functional test style guide
4edc689382 doc: test: suggest multi-line imports in functional test style guide (Sebastian Falbesoner)

Pull request description:

  As long as I remember contributing to functional tests (~2-3 years), it was always kind of an unwritten rule that multi-line imports are preferred over single-line imports in order to reduce the possibility of potential merge conflicts -- at least if more than one symbol from a module is imported. This PR adds this rule to the style guide and adapts the example test accordingly. (Inspired by https://github.com/bitcoin/bitcoin/pull/25792#discussion_r941180819).

ACKs for top commit:
  kouloumos:
    ACK 4edc689382
  1440000bytes:
    ACK 4edc689382
  w0xlt:
    ACK 4edc689382
  fanquake:
    ACK 4edc689382

Tree-SHA512: c7b6ff62f601f4e57cc8334d291662987d6737ebca8d81c063280409f4412302172f1404ec16afc9a13007bcdba55bdab66b9b80363287e287888929cb386584
2022-08-10 19:22:14 +02:00
Andrew Chow
70a55c059b psbt: Avoid unsigned int overflow in PSBT_IN_TAP_BIP32_DERIVATION 2022-08-10 11:58:17 -04:00
josibake
f47ff71761
test: only run test for descriptor wallets
since this test uses bech32m, we skip unless sqlite is used, which is the
same as checking if we are using descriptor wallets or not
2022-08-10 15:19:32 +02:00
MacroFake
aac200801b
Merge bitcoin/bitcoin#25794: test, tracing: don't rely on block_connected USDT event order in tests
0532aa7444 test: don't rely on usdt block_conn event order (0xb10c)

Pull request description:

  Relying on block_connected event order in the USDT interface tests turned out to be brittle.

  Closes https://github.com/bitcoin/bitcoin/issues/25793
  Closes https://github.com/bitcoin/bitcoin/issues/25764

Top commit has no ACKs.

Tree-SHA512: 40b5012ac80a8eac9d2f374cd39304488009c29adb474dc5e8c03b96c354be358298d2ddee8de480afecc187e1bf42ee119b7aa6216b086a8bf77b7e1310213c
2022-08-10 14:04:40 +02:00
MacroFake
ebf094ff3a
Merge bitcoin/bitcoin#25731: test: negative/unknown rpcserialversion should throw an init error
155344960b test: negative/unknown `rpcserialversion` should throw an init error (brunoerg)

Pull request description:

  This PR adds test coverage for the following init errors:
  41205bf442/src/init.cpp (L1025-L1030)

Top commit has no ACKs.

Tree-SHA512: 4456949e9a13908a5a59b13ed57bc3002b7ffd626e8dfb0346aa2600937ba1ef1b69cbae45cdb6bbc1c019dbcd64844e736a2ddd671a4704e53804355b6ea9f9
2022-08-10 13:51:44 +02:00
Andrew Chow
ac59112a6a
Merge bitcoin/bitcoin#23480: Add rawtr() descriptor for P2TR with specified (tweaked) output key
544b4332f0 Add wallet tests for spending rawtr() (Pieter Wuille)
e1e3081200 If P2TR tweaked key is available, sign with it (Pieter Wuille)
8d9670ccb7 Add rawtr() descriptor for P2TR with unknown tweak (Pieter Wuille)

Pull request description:

  It may be useful to be able to represent P2TR outputs in descriptors whose script tree and/or internal key aren't known. This PR does that, by adding a `rawtr(KEY)` descriptor, where the KEY represents the output key directly. If the private key corresponding to that output key is known, it also permits signing with it.

  I'm not convinced this is desirable, but presumably "tr(KEY)" sounds more intended for direct use than "rawtr(KEY)".

ACKs for top commit:
  achow101:
    ACK 544b4332f0
  sanket1729:
    code review ACK 544b4332f0
  w0xlt:
    reACK 544b4332f0

Tree-SHA512: 0de08de517468bc22ab0c00db471ce33144f5dc211ebc2974c6ea95709f44e830532ec5cdb0128c572513d352120bd651c4559516d4500b5b0a3d257c4b45aca
2022-08-09 16:36:00 -04:00
Sebastian Falbesoner
4edc689382 doc: test: suggest multi-line imports in functional test style guide 2022-08-09 18:04:20 +02:00
Sebastian Falbesoner
b4a5ab96b4 test: refactor: deduplicate DEFAULT_{ANCESTOR,DESCENDANT}_LIMIT constants 2022-08-09 15:22:38 +02:00
Sebastian Falbesoner
0fda1c7df6 scripted-diff: test: rename MAX_{ANCESTORS,DESCENDANTS} to DEFAULT_{ANCESTOR,DESCENDANT}_LIMIT
-BEGIN VERIFY SCRIPT-
ren() { sed -i "s:$1:$2:g" $(git grep -l "$1" ./test); }

ren MAX_ANCESTORS_CUSTOM    CUSTOM_ANCESTOR_LIMIT
ren MAX_DESCENDANTS_CUSTOM  CUSTOM_DESCENDANT_LIMIT
ren MAX_ANCESTORS           DEFAULT_ANCESTOR_LIMIT
ren MAX_DESCENDANTS         DEFAULT_DESCENDANT_LIMIT
-END VERIFY SCRIPT-
2022-08-09 14:59:47 +02:00
Andrew Chow
e7ca8afef6
Merge bitcoin/bitcoin#25782: test: check that verifymessage RPC fails for non-P2PKH addresses
68006c10ab test: check that `verifymessage` RPC fails for non-P2PKH addresses (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `verifymessage` RPC, for the case that a non-P2PKH (but otherwise valid) address is passed:
  e09ad284c7/src/util/message.cpp (L38-L40)
  e09ad284c7/src/rpc/signmessage.cpp (L48-L49)
  The passed addresses to trigger the error are of the types nested segwit (P2SH-P2WPKH) and native segwit (P2WPKH) and are created with a helper function `addresses_from_privkey` using descriptors and the `deriveaddresses` RPC. At some point in the future, if we have BIP322 support, all those will likely succeed and can then be moved from error-throwing to the succedding assert loop.

ACKs for top commit:
  achow101:
    ACK 68006c10ab
  w0xlt:
    ACK 68006c10ab

Tree-SHA512: fec4ed97460787c2ef3d04e3fce89c9365c87207c8358b59c41890f3738355c002e64f289ab4aef794ef4dfd5c867be8b67d736fb620489204f2c6bfb8d3363c
2022-08-08 19:07:14 -04:00
Hennadii Stepanov
fea75ad3ca
refactor: Drop boost/algorithm/string/replace.hpp dependency 2022-08-08 11:53:23 +01:00
0xb10c
0532aa7444
test: don't rely on usdt block_conn event order
Relying on block_connected event order in the USDT interface tests
turned out to be brittle.

Fixes https://github.com/bitcoin/bitcoin/issues/25793
Fixes https://github.com/bitcoin/bitcoin/issues/25764
2022-08-06 13:59:38 +02:00
Andrew Chow
35305c759a
Merge bitcoin/bitcoin#22751: rpc/wallet: add simulaterawtransaction RPC
db10cf8ae3 rpc/wallet: add simulaterawtransaction RPC (Karl-Johan Alm)
701a64f548 test: add support for Decimal to assert_approx (Karl-Johan Alm)

Pull request description:

  (note: this was originally titled "add analyzerawtransaction RPC")

  This command iterates over the inputs and outputs of the given transactions, and tallies up the balance change for the given wallet. This can be useful e.g. when verifying that a coin join like transaction doesn't contain unexpected inputs that the wallet will then sign for unintentionally.

  I originally proposed this to Elements (https://github.com/ElementsProject/elements/pull/1016) and it was suggested that I propose this upstream.

  There is an alternative #22776 to instead add this info to `getbalances` when providing an optional transaction as argument.

ACKs for top commit:
  jonatack:
    ACK db10cf8ae3
  achow101:
    re-ACK db10cf8ae3

Tree-SHA512: adf222ec7dcdc068d007ae6f465dbc35b692dc7bb2db337be25340ad0c2f9c64cfab4124df23400995c700f41c83c29a2c34812121782c26063b100c7969b89d
2022-08-05 15:19:03 -04:00
Sebastian Falbesoner
68006c10ab test: check that verifymessage RPC fails for non-P2PKH addresses 2022-08-05 11:59:56 +02:00
MacroFake
7d82f86341
Merge bitcoin/bitcoin#25650: script: default to necessary tags in test/get_previous_releases.py
21a9e94dbb ci: remove hardcoded tag list from ci scripts (josibake)
d530ba390e doc: update test/README.md (josibake)
614d4682ba script: default to necessary tags in get_previous_releases.py (josibake)

Pull request description:

  Almost every time I need to use this script, I forget the tag list is needed and that a specific set of tags is needed for the backwards compatibility tests to work. I end up wasting time reading through the script and googling to find the tag list before remembering it is in `test/README.md`

  I assume (hope) I'm not the only one this happens to, so I figured it would make more sense to have the script default to downloading/building the necessary tags. This has the added benefit of making the script the source of truth: the script already needs to be updated with the SHA256_SUM of the binary for every new tag that is added, so it makes sense to use `SHA256_SUMS` list as the necessary tag list. This means there is less risk of the README and the script drifting (i.e updating the readme with a new tag and forgetting to update the script, or updating the script and forgetting to update the README). Now all that needs to happen is to update the `SHA256_SUMS` list in the script and everything Just Works (TM)

ACKs for top commit:
  Sjors:
    re-tACK 21a9e94dbb

Tree-SHA512: 97b488227a89a6827584edd251820a7074fad75dfd7f26f1aa5f858e2521d2e02effd0f11e6dc4676e1155d3d5aba6ff94a4b58ffef80dc201376afd5927deb9
2022-08-05 10:51:06 +02:00
Karl-Johan Alm
db10cf8ae3
rpc/wallet: add simulaterawtransaction RPC
This command iterates over the inputs and outputs of the given transactions, and tallies up the balance change for the given wallet. This can be useful e.g. when verifying that a coin join like transaction doesn't contain unexpected inputs that the wallet will then sign for unintentionally.
2022-08-05 09:48:09 +09:00
glozow
1dc03dda05
[doc] remove non-signaling mentions of BIP125
Our RBF policy is different from the rules specified in BIP125. For
example, the BIP does not mention Rule 6, and our Rule 4 uses the
(configurable) incremental relay feerate (distinct from the
minimum relay feerate). Those interested in our policy should refer to
doc/policy/mempool-replacements.md instead. These rules may also
continue to diverge with package RBF and other RBF improvements. Keep
references to the BIP125 signaling wrt sequence numbers, since that is
still correct and widely used. It is helpful to refer to this as "BIP125
signaling" since it is unambiguous and succint, especially if we have
multiple ways to signal replaceability in the future.

The rule numbers in doc/policy/mempool-replacements.md correspond
largely to those of BIP 125, so we can still refer to them like "Rule 5."
2022-08-04 16:56:33 +01:00
MacroFake
fa2537cf0a
test: Target exact weight in MiniWallet _bulk_tx
Also, replace broad -acceptnonstdtxn=1 with -datacarriersize=100000
2022-08-03 12:02:20 +02:00
MacroFake
9155f9b7af
Merge bitcoin/bitcoin#25379: test: use MiniWallet to simplify mempool_package_limits.py tests
f2f6068b69 test: MiniWallet: add `send_self_transfer_chain` to create chain of txns (Andreas Kouloumos)
1d6b438ef0 test: use MiniWallet to simplify mempool_package_limits.py tests (Andreas Kouloumos)

Pull request description:

  While `wallet.py` includes the MiniWallet class and some helper methods, it also includes some methods that have been moved there without having any direct relation with the MiniWallet class. Specifically `make_chain`, `create_child_with_parents` and `create_raw_chain` methods that were extracted from `rpc_packages.py` at f8253d69d6 in order to be used on both `mempool_package_limits.py` and `rpc_packages.py`.

  Since that change, due to the introduction of additional methods in MiniWallet, the functionality of those methods can now be replicated with the existing MiniWallet methods and simultaneously simplify those tests by using the MiniWallet.

  This PR's goals are

  -  to simplify the `mempool_package_limits.py` functional tests with usage of the MiniWallet.
  -  to make progress towards the removal of the `make_chain`, `create_child_with_parents` and `create_raw_chain` methods of `wallet.py`.

  For the purpose of the aforementioned goals, a helper method `MiniWallet.send_self_transfer_chain` is introduced and method `bulk_transaction` has been integrated in `create_self_transfer*` methods using an optional `target_weight` option.

ACKs for top commit:
  MarcoFalke:
    ACK f2f6068b69 👜

Tree-SHA512: 3ddfa0046168cbf7904ec6b1ca233b3fdd4f30db6aefae108b6d7fb69f34ef6fb2cf4fa7cef9473ce1434a0cc8149d236441a685352fef35359a2b7ba0d951eb
2022-08-03 11:12:05 +02:00
MacroFake
fa148602e6
Remove ::fRequireStandard global 2022-08-02 15:23:24 +02:00
Karl-Johan Alm
701a64f548
test: add support for Decimal to assert_approx 2022-08-02 10:11:12 +09:00
Andreas Kouloumos
f2f6068b69 test: MiniWallet: add send_self_transfer_chain to create chain of txns
With this new method, a chain of transactions can be created. This
method is introduced to further simplify the mempool_package_limits.py
tests.
2022-08-01 19:11:36 +03:00
Andreas Kouloumos
1d6b438ef0 test: use MiniWallet to simplify mempool_package_limits.py tests
Moved `bulk_transaction` into MiniWallet class as `_bulk_tx` private
helper method to be used when the newly added `target_weight` option is
passed to `create_self_transfer*`
2022-08-01 19:11:35 +03:00
brunoerg
155344960b test: negative/unknown rpcserialversion should throw an init error 2022-08-01 10:55:05 -03:00
MacroFake
2bca32b7c3
Merge bitcoin/bitcoin#24799: Add test case mimicking issue 24765
395767e9f1 Add test case mimicking issue 24765 (Pieter Wuille)

Pull request description:

  This adds a functional test for the concern brought up in #24765. It turned out to be a non-issue, but since I wrote it anyway, it can't hurt to add it.

Top commit has no ACKs.

Tree-SHA512: fc8d57129d8c68f6d9a41b94b5ff676c87b31f53bc958195d4fe312530ec3e038ebd0bc5e8b9d56be77b7b63fd94574685901901404a4ab8726a5e09d89e86c8
2022-08-01 11:58:57 +02:00
MacroFake
eeb5a94e27
Merge bitcoin/bitcoin#25528: ci: run USDT interface tests in the CI
cc7335edc8 ci: run USDT interface test in a VM (0xb10c)
dba6f82342 test: adopt USDT utxocache interface tests (0xb10c)
220a5a2841 test: hook into PID in tracing tests (0xb10c)

Pull request description:

  Changes a CI task that runs test the previously not run `test/functional/interface_usdt_*.py` functional tests (added in https://github.com/bitcoin/bitcoin/pull/24358).

  This task is run as CirussCI `compute_engine_instance` VM as hooking into the tracepoints is not possible in CirrusCI docker containers (https://github.com/bitcoin/bitcoin/issues/23296#issuecomment-1024920845). We use an unoffical PPA and untrusted  `bpfcc-tools` package in the CI as the Ubuntu jammy and Debian bullseye packages are outdated. We hope use an official package when new Ubuntu/Debian releases are available for the use with Google Compute Engine.

  We make sure to hook into `bitcoind` binaries in USDT interface tests via their PID, instead of their path. This makes sure multiple functional tests running in parallel don't interfere with each other.

  The utxocache USDT interface tests is adopted to a change of the functional test framework that wasn't detected as the tests weren't run in the CI. As the tracepoints expose internals, it can happen that we need to adopt the interface test when internals change. This is a bit awkward, and if it happens to frequently, we should consider generalizing the tests a bit more. For now it's fine, I think.

  See the individual commit messages for more details on the changes.

  Fixes https://github.com/bitcoin/bitcoin/issues/24782
  Fixes https://github.com/bitcoin/bitcoin/issues/23296

  I'd like to hear from reviewers:
  - Are we OK with using the [`hadret/bpfcc`](https://launchpad.net/~hadret/+archive/ubuntu/bpfcc) PPA for now? There is a clear plan when to drop it and as is currently, it could only impact the newly added VM task.
  - ~~Adding a new task increases CI runtime and costs. Should an existing `container` CI task be ported to a VM and reused instead?~~ Yes, see https://github.com/bitcoin/bitcoin/pull/25528#issuecomment-1179509525

ACKs for top commit:
  MarcoFalke:
    cr ACK cc7335edc8

Tree-SHA512: b7fddccc0a77d82371229d048abe0bf2c4ccaa45906497ef3040cf99e7f05561890aef4c253c40e4afc96bb838c9787fae81c8454c6fd9db583276e005a4ccb3
2022-08-01 11:27:29 +02:00
MacroFake
c5ba1d92b6
Merge bitcoin/bitcoin#25610: wallet, rpc: Opt in to RBF by default
ab3c06db1a doc: Release notes for default RBF (Andrew Chow)
61d9149e78 rpc: Default rbf enabled (Andrew Chow)
e3c33637ba wallet: Enable -walletrbf by default (Andrew Chow)

Pull request description:

  The GUI currently opts in to RBF by default, but RPCs do not, and `-walletrbf` is default disabled. This PR makes the default in those two places to also opt in.

  The last time this was proposed (#9527), the primary objections were the novelty at the time, the inability to bump transactions, and the gui not having the option to disable rbf. In the 5 years since, RBF usage has steadily grown, with ~27% of txs opting in. The GUI has the option to enable/disable RBF, and is also defaulted to having it enabled. And we have the ability to bump RBF'd transactions in both the RPC and the GUI. So I think it makes sense to finally change the default to always opt in to RBF.

ACKs for top commit:
  darosior:
    reACK ab3c06db1a
  aureleoules:
    ACK ab3c06db1a.
  glozow:
    utACK ab3c06db1a

Tree-SHA512: 81b012c5033e270f86a87a6a196ccc549eb54b158eebf88e917cc6621d40d7bdcd1566b602688907dd5d364b95a557b29f97dce869cea512e339588262c027b6
2022-08-01 10:53:11 +02:00
Andrew Chow
1abbae65eb
Merge bitcoin/bitcoin#24584: wallet: avoid mixing different OutputTypes during coin selection
71d1d13627 test: add unit test for AvailableCoins (josibake)
da03cb41a4 test: functional test for new coin selection logic (josibake)
438e04845b wallet: run coin selection by `OutputType` (josibake)
77b0707206 refactor: use CoinsResult struct in SelectCoins (josibake)
2e67291ca3 refactor: store by OutputType in CoinsResult (josibake)

Pull request description:

  # Concept

  Following https://github.com/bitcoin/bitcoin/pull/23789, Bitcoin Core wallet will now generate a change address that matches the payment address type. This improves privacy by not revealing which of the outputs is the change at the time of the transaction in scenarios where the input address types differ from the payment address type. However, information about the change can be leaked in a later transaction. This proposal attempts to address that concern.

  ## Leaking information in a later transaction

  Consider the following scenario:

  ![mix input types(1)](https://user-images.githubusercontent.com/7444140/158597086-788339b0-c698-4b60-bd45-9ede4cd3a483.png)

  1. Alice has a wallet with bech32 type UTXOs and pays Bob, who gives her a P2SH address
  2. Alice's wallet generates a P2SH change output, preserving her privacy in `txid: a`
  3. Alice then pays Carol, who gives her a bech32 address
  4. Alice's wallet combines the P2SH UTXO with a bech32 UTXO and `txid: b` has two bech32 outputs

  From a chain analysis perspective, it is reasonable to infer that the P2SH input in `txid: b` was the change from `txid: a`. To avoid leaking information in this scenario, Alice's wallet should avoid picking the P2SH output and instead fund the transaction with only bech32 Outputs. If the payment to Carol can be funded with just the P2SH output, it should be preferred over the bech32 outputs as this will convert the P2SH UTXO to bech32 UTXOs via the payment and change outputs of the new transaction.

  **TLDR;** Avoid mixing output types, spend non-default `OutputTypes` when it is economical to do so.

  # Approach

  `AvailableCoins` now populates a struct, which makes it easier to access coins by `OutputType`. Coin selection tries to find a funding solution by each output type and chooses the most economical by waste metric. If a solution can't be found without mixing, coin selection runs over the entire wallet, allowing mixing, which is the same as the current behavior.

  I've also added a functional test (`test/functional/wallet_avoid_mixing_output_types.py`) and unit test (`src/wallet/test/availablecoins_tests.cpp`.

ACKs for top commit:
  achow101:
    re-ACK 71d1d13627
  aureleoules:
    ACK 71d1d13627.
  Xekyo:
    reACK 71d1d13627 via `git range-diff master 6530d19 71d1d13`
  LarryRuane:
    ACK 71d1d13627

Tree-SHA512: 2e0716efdae5adf5479446fabc731ae81d595131d3b8bade98b64ba323d0e0c6d964a67f8c14c89c428998bda47993fa924f3cfca1529e2bd49eaa4e31b7e426
2022-07-28 18:16:51 -04:00
Andrew Chow
317ef0368b
Merge bitcoin/bitcoin#25670: test: check that combining PSBTs with different txs fails
4e616d20c9 test: check that combining PSBTs with different txs fails (Sebastian Falbesoner)
2a428c7989 test: support passing PSBTMaps directly to PSBT ctor (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `combinepsbt` RPC, in the case of combining two PSBTs with different transactions:
  b8067cd435/src/psbt.cpp (L24-L27)
  The calling function `CombinePSBTs` checks for the false return value and then returns the transaction error string `PSBT_MISMATCH`:
  b8067cd435/src/psbt.cpp (L433-L435)
  b8067cd435/src/util/error.cpp (L30-L31)

ACKs for top commit:
  instagibbs:
    reACK 4e616d20c9
  achow101:
    ACK 4e616d20c9

Tree-SHA512: 45b2b224b13b44ad69ae62e4bc20f74cab32770cf8127b026ec47a7520f7253148fdbf1fad612afece59e45a6738bef9a351ae87ea98dc83d095cc78f6db0318
2022-07-28 17:34:28 -04:00
Aurèle Oulès
7ab43eb811
test: remove unused if statements 2022-07-25 09:59:05 +02:00
Sebastian Falbesoner
4e616d20c9 test: check that combining PSBTs with different txs fails 2022-07-23 09:08:54 +02:00
Sebastian Falbesoner
2a428c7989 test: support passing PSBTMaps directly to PSBT ctor
This will allow to create simple PSBTs as short one-liners, without the
need to have three individual assignments (globals, inputs, outputs).
2022-07-23 08:48:08 +02:00
Andrew Chow
d67f89bd95
Merge bitcoin/bitcoin#25625: test: add test for decoding PSBT with per-input preimage types
71a751f6c3 test: add test for decoding PSBT with per-input preimage types (Sebastian Falbesoner)
faf43378e2 refactor: move helper `random_bytes` to util library (Sebastian Falbesoner)
fdc1ca3896 test: add constants for PSBT key types (BIP 174) (Sebastian Falbesoner)
1b035c03f9 refactor: move PSBT(Map) helpers from signet miner to test framework (Sebastian Falbesoner)
7c0dfec2dd refactor: move `from_binary` helper from signet miner to test framework (Sebastian Falbesoner)
597a4b35f6 scripted-diff: rename `FromBinary` helper to `from_binary` (signet miner) (Sebastian Falbesoner)

Pull request description:

  This PR adds missing test coverage for the `decodepsbt` RPC in the case that a PSBT with on of the per-input preimage types (`PSBT_IN_RIPEMD160`, `PSBT_IN_SHA256`, `PSBT_IN_HASH160`, `PSBT_IN_HASH256`; see [BIP 174](https://github.com/bitcoin/bips/blob/master/bip-0174.mediawiki#Specification)) is passed. As preparation, the first four commits move the already existing helpers for (de)serialization of PSBTs and PSBTMaps from the signet miner to the test framework (in a new module `psbt.py`), which should be quite useful for further tests to easily create PSBTs.

ACKs for top commit:
  achow101:
    ACK 71a751f6c3

Tree-SHA512: 04f2671612d94029da2ac8dc15ff93c4c8fcb73fe0b8cf5970509208564df1f5e32319b53ae998dd6e544d37637a9b75609f27a3685da51f603f6ed0555635fb
2022-07-20 16:46:39 -04:00
josibake
d530ba390e
doc: update test/README.md
take the hardcoded list out of the readme. this way, we only need to
update the script as new tags are added
2022-07-20 15:52:03 +02:00