Commit graph

1301 commits

Author SHA1 Message Date
Samuel Dobson
b1a672d158
Merge bitcoin/bitcoin#22337: wallet: Use bilingual_str for errors
92993aa5cf Change SignTransaction's input_errors to use bilingual_str (Andrew Chow)
171366e89b Use bilingual_str for address fetching functions (Andrew Chow)
9571c69b51 Add bilingual_str::clear() (Andrew Chow)

Pull request description:

  In a couple of places in the wallet, errors are `std::string`. In order for these errors to be translated, change them to use `bilingual_str`.

ACKs for top commit:
  hebasto:
    re-ACK 92993aa5cf, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/22337#pullrequestreview-694542729) review, verified with
  klementtan:
    Code review ACK 92993aa5cf
  meshcollider:
    Code review ACK 92993aa5cf

Tree-SHA512: 5400e419dd87db8c49b67ed0964de2d44b58010a566ca246f2f0760ed9ef6a9b6f6df7a6adcb211b315b74c727bfe8c7d07eb5690b5922fa5828ceef4c83461f
2021-08-09 14:45:12 +12:00
Samuel Dobson
a162edfdd1
Merge bitcoin/bitcoin#22359: wallet: Do not set fInMempool in transactionAddedToMempool when tx is not in the mempool
fa6fd3dd6a wallet: Properly set fInMempool in mempool notifications (MarcoFalke)

Pull request description:

  A wallet method (like bumping the fee) might have set `fInMempool` to false because the transaction was removed from the mempool (See commit fa4e088cba).

  Avoid setting it back to true (incorrectly) in the validation interface background thread.

  Fixes #22357

ACKs for top commit:
  ryanofsky:
    Code review ACK fa6fd3dd6a. Only change since last review is extending workaround to `transactionRemovedFromMempool`. Since we know this workaround is imperfect and the goal of this PR is mainly to fix CI errors, I would probably be inclined to limit the workaround to as few places as possible where we have seen actual failures, instead of adding the workaround to as many places as possible, where there is some chance it might trigger new failures. But since this workaround is so straightforward and almost looks like a real fix, probably it doesn't matter.
  meshcollider:
    utACK fa6fd3dd6a

Tree-SHA512: d690136a577f1f532aa1fee80d3f6600ff7fc61286fbf564a53d7938d5ae52d33f0dbb0fef8b8c041a4970fb424f0b9f1ee7ce791e0ff8354e0000ecc9e22b84
2021-08-09 14:21:22 +12:00
fanquake
32fa49a184
make ParseOutputType return a std::optional<OutputType> 2021-08-04 19:20:32 +08:00
Andrew Chow
25d99e6511 Reorder dumpwallet so that cs_main functions go first
DEBUG_LOCKORDER expects cs_wallet, cs_main, and cs_KeyStore to be
acquired in that order. However dumpwallet would take these in the order
cs_wallet, cs_KeyStore, cs_main. So when configured with
`--enable-debug`, it is possible to hit the lock order assertion when
using dumpwallet.

To fix this, cs_wallet and cs_KeyStore are no longer locked at the same
time. Instead cs_wallet will be locked first. Then the functions which
lock cs_main will be run. Lastly cs_KeyStore will be locked afterwards.
This avoids the lock order issue.

Furthermore, since GetKeyBirthTimes (only used by dumpwallet) also uses
a function that locks cs_main, and itself also locks cs_KeyStore, the
same reordering is done here.
2021-07-19 12:25:11 -04:00
Andrew Chow
92993aa5cf Change SignTransaction's input_errors to use bilingual_str 2021-07-01 12:57:53 -04:00
Andrew Chow
171366e89b Use bilingual_str for address fetching functions
For GetNewDestination, GetNewChangeDestination, and
GetReservedDestination, use bilingual_str for
errors
2021-07-01 12:57:51 -04:00
fanquake
185acdb5e8
Merge bitcoin/bitcoin#22334: wallet: do not spam about non-existent spk managers
6084d2caed wallet: do not spam about non-existent spk managers (S3RK)

Pull request description:

  Avoid spam in logs during `loadwallet`, `listdescriptors` and probably other commands as well.

  **`loadwallet` Before:**
  ```
  2021-06-24T06:31:45Z init message: Loading wallet…
  2021-06-24T06:31:45Z [desc] Wallet File Version = 169900
  2021-06-24T06:31:45Z [desc] Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total. Unknown wallet records: 0
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Wallet completed loading in             197ms
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] setKeyPool.size() = 0
  2021-06-24T06:31:45Z [desc] mapWallet.size() = 0
  2021-06-24T06:31:45Z [desc] m_address_book.size() = 0
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] External scriptPubKey Manager for output type 2 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 0 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 1 does not exist
  2021-06-24T06:31:45Z [desc] Internal scriptPubKey Manager for output type 2 does not exist
  {
    "name": "desc",
    "warning": ""
  }
  ```

  **After:**
  ```
  2021-06-24T06:26:58Z init message: Loading wallet…
  2021-06-24T06:26:58Z [desc] Wallet File Version = 169900
  2021-06-24T06:26:58Z [desc] Keys: 0 plaintext, 0 encrypted, 0 w/ metadata, 0 total. Unknown wallet records: 0
  2021-06-24T06:26:58Z [desc] Wallet completed loading in             158ms
  2021-06-24T06:26:58Z [desc] setKeyPool.size() = 0
  2021-06-24T06:26:58Z [desc] mapWallet.size() = 0
  2021-06-24T06:26:58Z [desc] m_address_book.size() = 0
  {
    "name": "desc",
    "warning": ""
  }
  ```

ACKs for top commit:
  achow101:
    ACK 6084d2caed

Tree-SHA512: c7d7345c3182a575db088fd731b7f6e428c42e4f3f2e10d5adb50bf74a2defe88768e65ebb91a08590be48cf766a5697e36fafa73f68ffe45e76a60600f072e2
2021-07-01 19:11:20 +08:00
MarcoFalke
fa6fd3dd6a
wallet: Properly set fInMempool in mempool notifications 2021-07-01 10:45:55 +02:00
Andrew Chow
b945a31afa wallet: erase spkmans rather than setting to nullptr
In many places in ScriptPubKeyMan managing code, we assume that the
ScriptPubKeyMan being retrieved actually exists and is not a nullptr.
Thus removing a ScriptPubKeyMan requires erasing the object from the
map rather than setting it to a nullptr.
2021-07-01 01:22:38 -04:00
fanquake
5a95c5179c
Merge bitcoin/bitcoin#20191: wallet, refactor: make DescriptorScriptPubKeyMan agnostic of internal flag
181181019c refactor: remove m_internal from DescriptorSPKman (S3RK)

Pull request description:

  Rationale: improve consistency between `CWallet` and `DescriptorScriptPubKeyMan`; simplify `ScriptPubKeyMan` interface.

  Descriptor in itself is neither internal or external. It's responsibility of a wallet to assign and manage descriptors for a specific purpose. Duplicating information about internalness of a descriptor could lead to inconsistencies and unexpected behaviour (for example misreporting keypool size).

ACKs for top commit:
  instagibbs:
    reACK 181181019c
  achow101:
    reACK 181181019c

Tree-SHA512: d5613b7f6795b290bfa0fd8cb0536de1714d0cf72cba402266bd06d550758ebad690b54fc0a336a1c7414b5814aa4a37c90a6ae89926474a97d30956d7e034ff
2021-07-01 10:16:33 +08:00
fanquake
045bb06ebd
Merge bitcoin/bitcoin#19651: wallet: importdescriptors update existing
3efaf83c75 wallet: deactivate descriptor (S3RK)
6737d9655b test: wallet importdescriptors update existing (S3RK)
586f1d53d6 wallet: maintain SPK consistency on internal flag change (S3RK)
f1b7db1474 wallet: don't mute exceptions in importdescriptors (S3RK)
bf68ebc1cd wallet: allow to import same descriptor twice (S3RK)

Pull request description:

  Rationale: allow updating existing descriptors with `importdescriptors` command.

  Currently if you run same `importdescriptors` command twice with a descriptor containing private key you will get very confusing error — `Missing required fields`. What happens is that Wallet tries to write imported private key to the disk, but it exists already so we get `DB_KEYEXIST (-30995)` from BerkelyDB. Please note, that we set `DB_NOOVERWRITE` (I guess not to lose some keys accidentally). The exception is caught in `catch (...)` in rpcdump.cpp with a generic error.

  With this PR if a descriptor is already present than we will update its activeness, internalness, label, range and next_index.
  For the range only expansion is allowed (range start can only decrease, range end increase).

ACKs for top commit:
  achow101:
    re-ACK 3efaf83c75
  meshcollider:
    Code review ACK 3efaf83c75
  jonatack:
    Light ACK 3efaf83c75 per `git range-diff a000cb0 5d96704 3efaf83` and as a sanity check, re-debug-built on debian with gcc 10.2.1 and clang 11, ran wallet_importdescriptors.py

Tree-SHA512: 122c4b621d64ec8a3b625f3aed9f01a2b5cbaf2029ad0325b5ff38d67fff5cd35324335fabe2dd5169548b01b267c81be6ae0f5c834342f3d5f6eeed515c4843
2021-07-01 10:06:56 +08:00
Samuel Dobson
722776c0fd
Merge bitcoin/bitcoin#21329: descriptor wallet: Cache last hardened xpub and use in normalized descriptors
e6cf0ed92d wallet, rpc: listdescriptors does not need unlocked (Andrew Chow)
3280704886 Pass in DescriptorCache to ToNormalizedString (Andrew Chow)
7a26ff10c2 Change DescriptorImpl::ToStringHelper to use an enum (Andrew Chow)
75530c93a8 Remove priv option for ToNormalizedString (Andrew Chow)
74fede3b8b wallet: Upgrade existing descriptor caches (Andrew Chow)
432ba9e543 wallet: Store last hardened xpub cache (Andrew Chow)
d87b544b83 descriptors: Cache last hardened xpub (Andrew Chow)
cacc391098 Move DescriptorCache writing to WalletBatch (Andrew Chow)
0b4c8ef75c Refactor Cache merging and writing (Andrew Chow)
976b53b085 Revert "Cache parent xpub inside of BIP32PubkeyProvider" (Andrew Chow)

Pull request description:

  Currently fetching a normalized descriptor requires the wallet to be unlocked as it needs the private keys to derive the last hardened xpub. This is not very user friendly as normalized descriptors shouldn't require and don't involve the private keys except for derivation. We solve this problem by caching the last hardened xpub (which has to be derived at some point when generating the address pool).

  However the last hardened xpub was not already being cached. We only cached the immediate parent xpub and derived child keys. For example, with a descriptor derivation path of `/84'/0'/0'/0/*`, the parent xpub that is cached is `m/84'/0'/0'/0`, and the child keys of `m/84'/0'/0'/0/i` (note that child keys would not be cached in this case). This parent xpub is not suitable for the normalized descriptor form as we want the key at `m/84'/0'/0'`. So this PR adds another field to `DescriptorCache` to cache the last hardened xpub so that we can use them for normalized descriptors.

  Since `DescriptorCache` is changing, existing descriptor wallets need to be upgraded to use this new cache. The upgrade will occur in the background either at loading time (if the wallet is not encrypted) or at unlocking time in the same manner that `UpgradeKeyMetadata` operates. It will use a new wallet flag `WALLET_FLAG_LAST_HARDENED_XPUB_CACHED` to indicate whether the descriptor wallet has the last hardened xpub cache.

  Lastly `listdescriptors` will not require the wallet to be locked and `getaddressinfo`'s `parent_desc` will always be output (assuming the upgrade has occurred).

ACKs for top commit:
  fjahr:
    tACK e6cf0ed92d
  S3RK:
    reACK e6cf0ed
  jonatack:
    Semi ACK e6cf0ed92d reviewed, debug-built and ran unit tests and some of the descriptor functional tests at each commit. I'm not very familiar with this code and it could be clearer to the uninitiated IMHO, so I'm not confident enough to give a full ACK. Various minor suggestions follow, most of them for readability, feel free to pick and choose.
  meshcollider:
    Code review + functional test run ACK e6cf0ed92d

Tree-SHA512: ac27aade8644525cd65bfcaf27ff32afb974085b1451faf4ff68c6671a690bd6a41d4f39a33cbf461ae0fbe85995c0a4c08dbd36171da1c1d2a1d00053ad298d
2021-07-01 09:58:40 +12:00
S3RK
181181019c refactor: remove m_internal from DescriptorSPKman
Descriptor in itself is neither internal or external.
It's responsibility of a wallet to assign and manage descriptors
for a specific purpose. Duplicating such information could lead to
inconsistencies and unexpected behaviour.
2021-06-30 08:37:50 +02:00
S3RK
6084d2caed wallet: do not spam about non-existent spk managers 2021-06-29 08:16:39 +02:00
S3RK
3efaf83c75 wallet: deactivate descriptor 2021-06-28 21:44:50 +02:00
S3RK
586f1d53d6 wallet: maintain SPK consistency on internal flag change 2021-06-28 21:44:50 +02:00
S3RK
bf68ebc1cd wallet: allow to import same descriptor twice 2021-06-28 21:37:37 +02:00
MarcoFalke
8888cf45f5
Remove unused wallet pointer from NotifyAddressBookChanged 2021-06-28 10:39:49 +02:00
MarcoFalke
faf3640303
Remove unused wallet pointer from NotifyTransactionChanged signal 2021-06-28 10:38:32 +02:00
Andrew Chow
74fede3b8b wallet: Upgrade existing descriptor caches
Add functions to upgrade existing descriptor caches to support the use
of last hardened xpub caching.
2021-06-24 13:49:22 -04:00
Andrew Chow
754f134a50 wallet: Add error message to GetReservedDestination
Adds an error output parameter to all GetReservedDestination functions
so that callers can get the actual reason that a change address could
not be fetched. This more closely matches GetNewDestination. This allows
for more granular error messages, such as one that indicates that
bech32m addresses cannot be generated yet.
2021-06-22 21:57:04 -04:00
Andrew Chow
699dfcd8ad Opportunistically use bech32m change addresses if available
If a transaction as a segwit output, use a bech32m change address if
they are available. If not, fallback to bech32. If bech32 change
addresses are unavailable, fallback to the default address type.
2021-06-22 21:53:11 -04:00
Andrew Chow
0262536c34 Add OutputType::BECH32M
Bech32m addresses need their own OutputType

We are not ready to create DescriptorScriptPubKeyMans which produce
bech32m addresses. So don't allow generating them.
2021-06-22 21:53:11 -04:00
Andrew Chow
177c15d2f7 Limit LegacyScriptPubKeyMan address types
Make sure that LegacyScriptPubKeyMan can only be used for legacy,
p2sh-segwit, and bech32 address types.
2021-06-22 21:53:11 -04:00
Samuel Dobson
5c2e2afe99
Merge bitcoin/bitcoin#21365: Basic Taproot signing support for descriptor wallets
458a345b05 Add support for SIGHASH_DEFAULT in RPCs, and make it default (Pieter Wuille)
c0f0c8eccb tests: check spending of P2TR (Pieter Wuille)
a2380127e9 Basic Taproot signing logic in script/sign.cpp (Pieter Wuille)
49487bc3b6 Make GetInputUTXO safer: verify non-witness UTXO match (Pieter Wuille)
fd3f6890f3 Construct and use PrecomputedTransactionData in PSBT signing (Pieter Wuille)
5cb6502ac5 Construct and use PrecomputedTransactionData in SignTransaction (Pieter Wuille)
5d2e22437b Don't nuke witness data when signing fails (Pieter Wuille)
ce9353164b Permit full precomputation in PrecomputedTransactionData (Pieter Wuille)
e841fb503d Add precomputed txdata support to MutableTransactionSignatureCreator (Pieter Wuille)
a91d532338 Add CKey::SignSchnorr function for BIP 340/341 signing (Pieter Wuille)
e77a2839b5 Use HandleMissingData also in CheckSchnorrSignature (Pieter Wuille)
dbb0ce9fbf Add TaprootSpendData data structure, equivalent to script map for P2[W]SH (Pieter Wuille)

Pull request description:

  Builds on top of #22051, adding signing support after derivation support.

  Nothing is changed in descriptor features. Signing works for key path and script path spending, through the normal sending functions, and PSBT-based RPCs. However, PSBT usability is rather low as no extensions have been defined to convey Taproot-specific information, so all script information must be known to the signing wallet.

ACKs for top commit:
  achow101:
    re-ACK 458a345b05
  fjahr:
    Code review ACK 458a345b05
  Sjors:
    ACK 458a345b05

Tree-SHA512: 30ed212cf7754763a4a81624ebc084c51727b8322711ac0b390369213c1a891d367ed8b123882ac08c99595320c11ec57ee42304ff22a69afdc3d1a0d55cc711
2021-06-18 09:12:44 +12:00
Sjors Provoost
4455145e26
refactor: reduce #ifdef ENABLE_EXTERNAL_SIGNER usage
In particular this make the node interface independent on whether external signer support is compiled.
2021-06-16 10:48:58 +02:00
Sjors Provoost
7d9453041b
refactor: clean up external_signer.h includes
Co-Authored-By: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
2021-06-16 10:48:38 +02:00
Pieter Wuille
458a345b05 Add support for SIGHASH_DEFAULT in RPCs, and make it default
For non-Taproot signatures, this is interpreted as SIGHASH_ALL.
2021-06-12 12:38:17 -07:00
Pieter Wuille
fd3f6890f3 Construct and use PrecomputedTransactionData in PSBT signing 2021-06-12 12:25:28 -07:00
W. J. van der Laan
907d636e5e
Merge bitcoin/bitcoin#21353: interfaces: Stop exposing wallet destdata to gui
f5ba424cd4 wallet: Add IsAddressUsed / SetAddressUsed methods (Russell Yanofsky)
62252c95e5 interfaces: Stop exposing wallet destdata to gui (Russell Yanofsky)
985430d9b2 test: Add gui test for wallet receive requests (Russell Yanofsky)

Pull request description:

  Stop giving GUI access to destdata rows in database. Replace with narrow API just for saving and reading receive request information.

  This simplifies code and should prevent the GUI from interfering with other destdata like address-used status. It also adds some more GUI test coverage.

  There are no changes in behavior.

ACKs for top commit:
  jarolrod:
    tACK f5ba424cd4
  laanwj:
    Code review ACK f5ba424cd4

Tree-SHA512: 5423df4786e537a59013cb5bfb9e1bc29a7ca4b8835360c00cc2165a59f925fdc355907a4ceb8bca0285bb4946ba235bffa7645537a951ad03fd3b4cee17b6b0
2021-06-03 15:57:30 +02:00
Samuel Dobson
55a156fca0
Merge bitcoin/bitcoin#21207: MOVEONLY: CWallet transaction code out of wallet.cpp/.h
c7bd5842e4 MOVEONLY: CWallet transaction code out of wallet.cpp/.h (Russell Yanofsky)

Pull request description:

  This commit just moves function without making any changes. It can be reviewed with `git log -p -n1 --color-moved=dimmed_zebra`

  Motivation for this change is to make `wallet.cpp/h` less monolithic and start to make wallet transaction state tracking comprehensible so bugs in https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking can be fixed safely without introducing new problems.

  This moves wallet classes and methods that deal with transactions out of `wallet.cpp/.h` into better organized files:

  - `transaction.cpp/.h` - CWalletTx and CMerkleTx class definitions
  - `receive.cpp/.h` - functions checking received transactions and computing balances
  - `spend.cpp/.h` - functions creating transactions and finding spendable coins

  After #20773, when loading is separated from syncing it will also be possible to move more `wallet.cpp/.h` functions to:

  - `sync.cpp/.h` - functions handling chain notifications and rescanning

  This commit arranges `receive.cpp` and `spend.cpp` functions in dependency order so it's possible to skim `receive.cpp` and get an idea of how computing balances works, and skim `spend.cpp` and get an idea of how transactions are created, without having to jump all over `wallet.cpp` where functions are not in order and there is a lot of unrelated code.

  Followup commit "refactor: Detach wallet transaction methods" in https://github.com/bitcoin/bitcoin/pull/21206 follows up this PR and tweaks function names and arguments to reflect new locations. The two commits are split into separate PRs because this commit is more work to maintain and less work to review, while the other commit is less work to maintain and more work to review, so hopefully this commit can be merged earlier.

ACKs for top commit:
  Sjors:
    re-utACK c7bd5842e4
  fjahr:
    utACK c7bd5842e4
  promag:
    Code review ACK c7bd5842e4, verified move only claim.
  meshcollider:
    Dimmed-zebra-check and functional test run ACK c7bd5842e4

Tree-SHA512: 4981de6911cb1196774db375494355cc9af59b52456129c002d264a77cd9ed6175f8ecbb6b2f492a59a4d5a0def21a39d96fa79c9f4d99be0992985f553be32f
2021-05-30 22:00:40 +12:00
Hennadii Stepanov
bfb53ddda9
scripted-diff: Fix ellipsis after pr20773
-BEGIN VERIFY SCRIPT-
sed -i 's|_("Loading wallet...")|_("Loading wallet…")|' src/wallet/*.cpp
-END VERIFY SCRIPT-
2021-05-27 13:46:36 +03:00
Russell Yanofsky
c7bd5842e4 MOVEONLY: CWallet transaction code out of wallet.cpp/.h
This commit just moves functions without making any changes. It can be
reviewed with `git log -p -n1 --color-moved=dimmed_zebra`

Motivation for this change is to make wallet.cpp/h less monolithic and
start to make wallet transaction state tracking comprehensible so bugs
in
https://github.com/bitcoin-core/bitcoin-devwiki/wiki/Wallet-Transaction-Conflict-Tracking
can be fixed safely without introducing new problems.

This commit moves wallet classes and methods that deal with transactions
out of wallet.cpp/.h into better organized files:

- transaction.cpp/.h - CWalletTx and CMerkleTx class definitions
- receive.cpp/.h - functions checking received transactions and computing balances
- spend.cpp/.h - functions creating transactions and finding spendable coins

After #20773, when loading is separated from syncing it will also be
possible to move more wallet.cpp/.h functions to:

- sync.cpp/.h - functions handling chain notifications and rescanning

This commit arranges receive.cpp and spend.cpp functions in dependency
order so it's possible to skim receive.cpp and get an idea of how
computing balances works, and skim spend.cpp and get an idea of how
transactions are created, without having to jump all over wallet.cpp
where functions are not in order and there is a lot of unrelated code.

Followup commit "refactor: Detach wallet transaction methods" in
https://github.com/bitcoin/bitcoin/pull/21206 follows up this PR and
tweaks function names and arguments to reflect new locations. The two
commits are split into separate PRs because this commit is more work to
maintain and less work to review, while the other commit is less work to
maintain and more work to review, so hopefully this commit can be merged
earlier.
2021-05-26 06:32:51 -05:00
fanquake
ecddd12482
Merge bitcoin/bitcoin#18418: wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100
e6fe1c37d0 rpc: Improve avoidpartialspends and avoid_reuse documentation (Fabian Jahr)
8f073076b1 wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100 (Fabian Jahr)

Pull request description:

  Follow-up to #17824.

  This increases OUTPUT_GROUP_MAX_ENTRIES to 100 which means that OutputGroups will now be up to 100 outputs large, up from previously 10. The main motivation for this change is that during the PR review club on #17824 [several participants signaled](https://bitcoincore.reviews/17824.html#l-339) that 100 might be a better value here.

  I think fees should be manageable for users but more importantly, users should know what they can expect when using the wallet with this configuration, so I also tried to clarify the documentation on `-avoidpartialspends` and `avoid_reuse` a bit. If there are other additional ways how or docs where users can be made aware of the potential consequences of using these parameters, please let me know. Another small upside is that [there seem to be a high number of batching transactions with 100 and 200 inputs](https://miro.medium.com/max/3628/1*sZ5eaBSbsJsHx-J9iztq2g.png)([source](https://medium.com/@hasufly/an-analysis-of-batching-in-bitcoin-9bdf81a394e0)) giving these transactions a bit of a larger anonymity set, although that is probably a very weak argument.

ACKs for top commit:
  jnewbery:
    ACK e6fe1c37d0
  Xekyo:
    retACK e6fe1c37d0
  rajarshimaitra:
    tACK `e6fe1c3`
  achow101:
    ACK e6fe1c37d0
  glozow:
    code review ACK e6fe1c37d0

Tree-SHA512: 79685c58bafa64ed8303b0ecd616fce50fc9a2b758aa79833e4ad9f15760e09ab60c007bc16ab4cbc4222e644cfd154f1fa494b0f3a5d86faede7af33a6f2826
2021-05-26 19:32:51 +08:00
Gregory Sanders
881a3e2e17 Replace size/weight estimate tuple with struct for named fields 2021-05-26 07:33:09 +08:00
Samuel Dobson
6b254814c0
Merge bitcoin/bitcoin#17331: Use effective values throughout coin selection
51a3ac242c Have OutputGroup determine the value to use (Andrew Chow)
6d6d278475 Change SelectCoins_test to actually test SelectCoins (Andrew Chow)
9d3bd74ab4 Remove CreateTransaction while loop and some related variables (Andrew Chow)
6f0d5189af Remove use_bnb and bnb_used (Andrew Chow)
de26eb0e1f Do both BnB and Knapsack coin selection in SelectCoinsMinConf (Andrew Chow)
01dc8ebda5 Have KnapsackSolver actually use effective values (Andrew Chow)
bf26e018de Roll static tx fees into nValueToSelect instead of having it be separate (Andrew Chow)
cc3f14b27c Move output reductions for fee to after coin selection (Andrew Chow)
d97d25d950 Make cost_of_change part of CoinSelectionParams (Andrew Chow)
af5867c896 Move some calculations to common code in SelectCoinsMinConf (Andrew Chow)
1bf4a62cb6 scripted-diff: rename some variables (Andrew Chow)

Pull request description:

  Changes `KnapsackSolver` to use effective values instead of just the nominal txout value. Since fees are taken into account during the selection itself, we finally get rid of the `CreateTransaction` loop as well as a few other things that only were only necessary because of that loop.

  This should not change coin selection behavior at all (except maybe remove weird edge cases that were caused by the loop). In order to keep behavior the same, `KnapsackSolver` will select outputs with a negative effective value (as it did before).

ACKs for top commit:
  ryanofsky:
    Code review ACK 51a3ac242c. Looks good to go!
  instagibbs:
    review ACK 51a3ac242c
  meshcollider:
    re-light-utACK 51a3ac242c

Tree-SHA512: 372c27e00edcd5dbf85177421ba88f20bfdaf1791b6e3dc022c44876ecc379403e2375ed69e71c512c49e6af87641001ff385c4b25ab93684b3a08a53bf3824e
2021-05-26 01:35:43 +12:00
Andrew Chow
51a3ac242c Have OutputGroup determine the value to use
Instead of hijacking the effective_feerate to use the correct value
during coin selection, have OutputGroup be aware of whether we are
subtracting the fee from the outputs and provide the correct value to
use for selection.

To do this, OutputGroup now takes CoinSelectionParams and has a new
function GetSelectionAmount().
2021-05-19 15:35:11 -04:00
Andrew Chow
9d3bd74ab4 Remove CreateTransaction while loop and some related variables
Remove the CreateTransaction while loop. Removes variables that were
only needed because of that loop. Also renames a few variables and
moves their declarations to where they are used.

Some subtractFeeFromOutputs handling is moved to after coin selection
in order to reduce their amounts once the fee is known.

If subtracting the fee reduces the change to dust, we will also now
remove the change output
2021-05-19 14:58:03 -04:00
Andrew Chow
6f0d5189af Remove use_bnb and bnb_used
These booleans are no longer needed
2021-05-19 14:37:17 -04:00
Andrew Chow
de26eb0e1f Do both BnB and Knapsack coin selection in SelectCoinsMinConf
Instead of switching which algorithm to use based on use_bnb, just run
both in SelectCoinsMinConf. If BnB fails, do Knapsack.
2021-05-19 14:31:28 -04:00
Andrew Chow
01dc8ebda5 Have KnapsackSolver actually use effective values
Although the CreateTransaction loop currently remains, it should be
largely unused. KnapsackSolver will now account for transaction fees
when doing its selection.

In the previous commit, SelectCoinsMinConf was refactored to have some
calculations become shared for KnapsackSolver and SelectCoinsBnB. In
this commit, KnapsackSolver will now use the not_input_fees and
effective_feerate so that it include the fee for non-input things
(excluding a change output) so that the algorithm will select enough to
cover those fees. This is necessary for selecting on effective values.

Additionally, the OutputGroups
created for KnapsackSolver will actually have their effective values
calculated and set, and KnapsackSolver will do its selection on those
effective values.

Lastly, SelectCoins is modified to use the same value for preselected
inputs for BnB and KnapsackSolver. While it will still use the real
value when subtracting the fee from outputs, this behavior will be
the same regardless of the algo used for selecting additional inputs.
2021-05-19 14:25:06 -04:00
Andrew Chow
bf26e018de Roll static tx fees into nValueToSelect instead of having it be separate
The fees for transaction overhead and recipient outputs are now included
in nTargetValue instead of being a separate parameter. For the coin
selection algorithms, it doesn't matter that these are separate as in
either case, the algorithm needs to select enough to cover these fees.

Note that setting nValueToSelect is changed as it now includes
not_input_fees. Without the change to how nValueToSelect is increased
for KnapsackSolver, this would result in overpaying fees. The change to
increase by the difference between nFeeRet and not_input_fees allows
this to have the same behavior as previously.

Additionally, because we assume that KnapsackSolver will always find a
solution that requires change (we assume that BnB always finds a
non-change solution), we also include the fee for the change output in
KnapsackSolver's target. As part of this, we also use the changeless
nFeeRet when iterating for KnapsackSolver. This is because we include
the change fee when doing KnapsackSolver, so nFeeRet on further
iterations won't include the change fee.
2021-05-19 13:33:31 -04:00
Andrew Chow
cc3f14b27c Move output reductions for fee to after coin selection
Simplifies CreateTransactionInternal without changing behavior. Removes
the pick_new_inputs variable by moving the subtract fee from amount
implementation to later in the loop to where it is possible to calculate
the fee for the transaction. This allows the fee to be subtracted from
the outputs within a single iteration, instead of calculating the fee in
the first iteration, and subtracting the fee in the second.

This also removes another scenario where a second iteration of the loop
finds a smaller input set (and thus smaller fees than the first
iteration) with no change and so a third iteration of the loop is done in order to make
a change output that contains the excess fees.

To handle these cases, we always create a change output which contains
the difference between selected input values and the recipient amounts.
Once the transaction fee is calculated, the change output is reduced (in
the normal case) or the recipient amounts are reduced (in the subtract
fee from amount case). All of this is done in a single iteration of the
loop.
2021-05-19 13:22:27 -04:00
Ivan Metlushko
489ebb7b34 wallet: make chain optional for CWallet::Create 2021-05-19 08:50:20 +02:00
Ivan Metlushko
d73ae93964 CWallet::Create move chain init message up into calling code 2021-05-19 08:50:20 +02:00
Russell Yanofsky
44c430ffac refactor: Add CWallet:::AttachChain method
This commit does not change behavior, it just moves code from
CWallet::CreateWalletFromFile to CWallet:::AttachChain so it can be updated in
the next commit.

This commit is most easily reviewed with
"git diff -w --color-moved=dimmed_zebra" or by diffing CWallet:::AttachChain
against the previous code with an external diff tool.
2021-05-19 08:50:20 +02:00
Ivan Metlushko
e2a47ce085 refactor: move first run detection to client code 2021-05-19 08:50:16 +02:00
Fabian Jahr
8f073076b1
wallet: Increase OUTPUT_GROUP_MAX_ENTRIES to 100 2021-05-18 02:09:18 +02:00
Andrew Chow
d97d25d950 Make cost_of_change part of CoinSelectionParams 2021-05-13 16:40:56 -04:00
Andrew Chow
af5867c896 Move some calculations to common code in SelectCoinsMinConf
To prepare for KnapsackSolver to use effective values, these
calculations are moved out of the BnB if block to allow for them to be
shared with KnapsackSolver in the future.
2021-05-13 16:40:56 -04:00