phantom/bitcoin
1
0
Fork 0
mirror of https://github.com/bitcoin/bitcoin.git synced 2025-01-25 10:43:19 -03:00
Code Issues Projects Releases Packages Wiki Activity Actions 1

Merge bitcoin/bitcoin#19602: wallet: Migrate legacy wallets to descriptor wallets

Browse source 
53e7ed075c doc: Release notes and other docs for migration (Andrew Chow)
9c44bfe244 Test migratewallet (Andrew Chow)
0b26e7cdf2 descriptors: addr() and raw() should return false for ToPrivateString (Andrew Chow)
31764c3f87 Add migratewallet RPC (Andrew Chow)
0bf7b38bff Implement MigrateLegacyToDescriptor (Andrew Chow)
e7b16f925a Implement MigrateToSQLite (Andrew Chow)
5b62f095e7 wallet: Refactor SetupDescSPKMs to take CExtKey (Andrew Chow)
22401f17e0 Implement LegacyScriptPubKeyMan::DeleteRecords (Andrew Chow)
35f428fae6 Implement LegacyScriptPubKeyMan::MigrateToDescriptor (Andrew Chow)
ea1ab390e4 scriptpubkeyman: Implement GetScriptPubKeys in Legacy (Andrew Chow)
e664af2976 Apply label to all scriptPubKeys of imported combo() (Andrew Chow)

Pull request description:

  This PR adds a new `migratewallet` RPC which migrates a legacy wallet to a descriptor wallet. Migrated wallets will need a new backup. If a wallet has watchonly stuff in it, a new watchonly descriptor wallet will be created containing those watchonly things. The related transactions, labels, and descriptors for those watchonly things will be removed from the original wallet. Migrated wallets will not have any of the legacy things be available for fetching from `getnewaddress` or `getrawchangeaddress`. Wallets that have private keys enabled will have newly generated descriptors. Wallets with private keys disabled will not have any active `ScriptPubKeyMan`s.

  For the basic HD wallet case of just generated keys, in addition to the standard descriptor wallet descriptors using the master key derived from the pre-existing hd seed, the migration will also create 3 descriptors for each HD chain in: a ranged combo external, a ranged combo internal, and a single key combo for the seed (the seed is a valid key that we can receive coins at!). The migrated wallet will then have newly generated descriptors as the active `ScriptPubKeyMan`s. This is equivalent to creating a new descriptor wallet and importing the 3 descriptors for each HD chain. For wallets containing non-HD keys, each key will have its own combo descriptor.

  There are also tests.

ACKs for top commit:
  Sjors:
    tACK 53e7ed075c
  w0xlt:
    reACK 53e7ed075c

Tree-SHA512: c0c003694ca2e17064922d08e8464278d314e970efb7df874b4fe04ec5d124c7206409ca701c65c099d17779ab2136ae63f1da2a9dba39b45f6d62cf93b5c60a
This commit is contained in:
Andrew Chow 2022-09-01 15:33:34 -04:00
commit 7921026a24
No known key found for this signature in database
GPG key ID: 17565732E08E5E41
15 changed files with 1458 additions and 34 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

27
doc/managing-wallets.md
View file

@ -120,4 +120,29 @@ After that, `getwalletinfo` can be used to check if the wallet has been fully re
$ bitcoin-cli -rpcwallet="restored-wallet" getwalletinfo
```
The restored wallet can also be loaded in the GUI via `File` ->`Open wallet`.
The restored wallet can also be loaded in the GUI via `File` ->`Open wallet`.
## Migrating Legacy Wallets to Descriptor Wallets
Legacy wallets (traditional non-descriptor wallets) can be migrated to become Descriptor wallets
through the use of the `migratewallet` RPC. Migrated wallets will have all of their addresses and private keys added to
a newly created Descriptor wallet that has the same name as the original wallet. Because Descriptor
wallets do not support having private keys and watch-only scripts, there may be up to two
additional wallets created after migration. In addition to a descriptor wallet of the same name,
there may also be a wallet named `<name>_watchonly` and `<name>_solvables`. `<name>_watchonly`
contains all of the watchonly scripts. `<name>_solvables` contains any scripts which the wallet
knows but is not watching the corresponding P2(W)SH scripts.
Migrated wallets will also generate new addresses differently. While the same BIP 32 seed will be
used, the BIP 44, 49, 84, and 86 standard derivation paths will be used. After migrating, a new
backup of the wallet(s) will need to be created.
Given that there is an extremely large number of possible configurations for the scripts that
Legacy wallets can know about, be watching for, and be able to sign for, `migratewallet` only
makes a best effort attempt to capture all of these things into Descriptor wallets. There may be
unforeseen configurations which result in some scripts being excluded. If a migration fails
unexpectedly or otherwise misses any scripts, please create an issue on GitHub. A backup of the
original wallet can be found in the wallet directory with the name `<name>-<timestamp>.legacy.bak`.
The backup can be restored using the `restorewallet` command as discussed in the
[Restoring the Wallet From a Backup](#16-restoring-the-wallet-from-a-backup) section

9
doc/release-notes-19602.md Normal file
View file

@ -0,0 +1,9 @@
Wallet
======
Migrating Legacy Wallets to Descriptor Wallets
---------------------------------------------
An experimental RPC `migratewallet` has been added to migrate Legacy (non-descriptor) wallets to
Descriptor wallets. More information about the migration process is available in the
[documentation](https://github.com/bitcoin/bitcoin/blob/master/doc/managing-wallets.md#migrating-legacy-wallets-to-descriptor-wallets).

4
src/script/descriptor.cpp
View file

@ -612,7 +612,7 @@ public:
return AddChecksum(ret);
}
bool ToPrivateString(const SigningProvider& arg, std::string& out) const final
bool ToPrivateString(const SigningProvider& arg, std::string& out) const override
{
bool ret = ToStringHelper(&arg, out, StringType::PRIVATE);
out = AddChecksum(out);
@ -698,6 +698,7 @@ public:
return OutputTypeFromDestination(m_destination);
}
bool IsSingleType() const final { return true; }
bool ToPrivateString(const SigningProvider& arg, std::string& out) const final { return false; }
};
/** A parsed raw(H) descriptor. */
@ -718,6 +719,7 @@ public:
return OutputTypeFromDestination(dest);
}
bool IsSingleType() const final { return true; }
bool ToPrivateString(const SigningProvider& arg, std::string& out) const final { return false; }
};
/** A parsed pk(P) descriptor. */

54
src/wallet/rpc/wallet.cpp
View file

@ -701,6 +701,59 @@ RPCHelpMan simulaterawtransaction()
};
}
static RPCHelpMan migratewallet()
{
return RPCHelpMan{"migratewallet",
"EXPERIMENTAL warning: This call may not work as expected and may be changed in future releases\n"
"\nMigrate the wallet to a descriptor wallet.\n"
"A new wallet backup will need to be made.\n"
"\nThe migration process will create a backup of the wallet before migrating. This backup\n"
"file will be named <wallet name>-<timestamp>.legacy.bak and can be found in the directory\n"
"for this wallet. In the event of an incorrect migration, the backup can be restored using restorewallet." +
HELP_REQUIRING_PASSPHRASE,
{},
RPCResult{
RPCResult::Type::OBJ, "", "",
{
{RPCResult::Type::STR, "wallet_name", "The name of the primary migrated wallet"},
{RPCResult::Type::STR, "watchonly_name", /*optional=*/true, "The name of the migrated wallet containing the watchonly scripts"},
{RPCResult::Type::STR, "solvables_name", /*optional=*/true, "The name of the migrated wallet containing solvable but not watched scripts"},
{RPCResult::Type::STR, "backup_path", "The location of the backup of the original wallet"},
}
},
RPCExamples{
HelpExampleCli("migratewallet", "")
+ HelpExampleRpc("migratewallet", "")
},
[&](const RPCHelpMan& self, const JSONRPCRequest& request) -> UniValue
{
std::shared_ptr<CWallet> wallet = GetWalletForJSONRPCRequest(request);
if (!wallet) return NullUniValue;
EnsureWalletIsUnlocked(*wallet);
WalletContext& context = EnsureWalletContext(request.context);
util::Result<MigrationResult> res = MigrateLegacyToDescriptor(std::move(wallet), context);
if (!res) {
throw JSONRPCError(RPC_WALLET_ERROR, util::ErrorString(res).original);
}
UniValue r{UniValue::VOBJ};
r.pushKV("wallet_name", res->wallet_name);
if (res->watchonly_wallet) {
r.pushKV("watchonly_name", res->watchonly_wallet->GetName());
}
if (res->solvables_wallet) {
r.pushKV("solvables_name", res->solvables_wallet->GetName());
}
r.pushKV("backup_path", res->backup_path.u8string());
return r;
},
};
}
// addresses
RPCHelpMan getaddressinfo();
RPCHelpMan getnewaddress();
@ -820,6 +873,7 @@ Span<const CRPCCommand> GetWalletRPCCommands()
{"wallet", &listwallets},
{"wallet", &loadwallet},
{"wallet", &lockunspent},
{"wallet", &migratewallet},
{"wallet", &newkeypool},
{"wallet", &removeprunedfunds},
{"wallet", &rescanblockchain},

323
src/wallet/scriptpubkeyman.cpp
View file

@ -999,9 +999,10 @@ bool LegacyScriptPubKeyMan::GetKeyOrigin(const CKeyID& keyID, KeyOriginInfo& inf
{
LOCK(cs_KeyStore);
auto it = mapKeyMetadata.find(keyID);
if (it != mapKeyMetadata.end()) {
meta = it->second;
if (it == mapKeyMetadata.end()) {
return false;
}
meta = it->second;
}
if (meta.has_key_origin) {
std::copy(meta.key_origin.fingerprint, meta.key_origin.fingerprint + 4, info.fingerprint);
@ -1658,6 +1659,318 @@ std::set<CKeyID> LegacyScriptPubKeyMan::GetKeys() const
return set_address;
}
const std::unordered_set<CScript, SaltedSipHasher> LegacyScriptPubKeyMan::GetScriptPubKeys() const
{
LOCK(cs_KeyStore);
std::unordered_set<CScript, SaltedSipHasher> spks;
// All keys have at least P2PK and P2PKH
for (const auto& key_pair : mapKeys) {
const CPubKey& pub = key_pair.second.GetPubKey();
spks.insert(GetScriptForRawPubKey(pub));
spks.insert(GetScriptForDestination(PKHash(pub)));
}
for (const auto& key_pair : mapCryptedKeys) {
const CPubKey& pub = key_pair.second.first;
spks.insert(GetScriptForRawPubKey(pub));
spks.insert(GetScriptForDestination(PKHash(pub)));
}
// For every script in mapScript, only the ISMINE_SPENDABLE ones are being tracked.
// The watchonly ones will be in setWatchOnly which we deal with later
// For all keys, if they have segwit scripts, those scripts will end up in mapScripts
for (const auto& script_pair : mapScripts) {
const CScript& script = script_pair.second;
if (IsMine(script) == ISMINE_SPENDABLE) {
// Add ScriptHash for scripts that are not already P2SH
if (!script.IsPayToScriptHash()) {
spks.insert(GetScriptForDestination(ScriptHash(script)));
}
// For segwit scripts, we only consider them spendable if we have the segwit spk
int wit_ver = -1;
std::vector<unsigned char> witprog;
if (script.IsWitnessProgram(wit_ver, witprog) && wit_ver == 0) {
spks.insert(script);
}
} else {
// Multisigs are special. They don't show up as ISMINE_SPENDABLE unless they are in a P2SH
// So check the P2SH of a multisig to see if we should insert it
std::vector<std::vector<unsigned char>> sols;
TxoutType type = Solver(script, sols);
if (type == TxoutType::MULTISIG) {
CScript ms_spk = GetScriptForDestination(ScriptHash(script));
if (IsMine(ms_spk) != ISMINE_NO) {
spks.insert(ms_spk);
}
}
}
}
// All watchonly scripts are raw
spks.insert(setWatchOnly.begin(), setWatchOnly.end());
return spks;
}
std::optional<MigrationData> LegacyScriptPubKeyMan::MigrateToDescriptor()
{
LOCK(cs_KeyStore);
if (m_storage.IsLocked()) {
return std::nullopt;
}
MigrationData out;
std::unordered_set<CScript, SaltedSipHasher> spks{GetScriptPubKeys()};
// Get all key ids
std::set<CKeyID> keyids;
for (const auto& key_pair : mapKeys) {
keyids.insert(key_pair.first);
}
for (const auto& key_pair : mapCryptedKeys) {
keyids.insert(key_pair.first);
}
// Get key metadata and figure out which keys don't have a seed
// Note that we do not ignore the seeds themselves because they are considered IsMine!
for (auto keyid_it = keyids.begin(); keyid_it != keyids.end();) {
const CKeyID& keyid = *keyid_it;
const auto& it = mapKeyMetadata.find(keyid);
if (it != mapKeyMetadata.end()) {
const CKeyMetadata& meta = it->second;
if (meta.hdKeypath == "s" || meta.hdKeypath == "m") {
keyid_it++;
continue;
}
if (m_hd_chain.seed_id == meta.hd_seed_id || m_inactive_hd_chains.count(meta.hd_seed_id) > 0) {
keyid_it = keyids.erase(keyid_it);
continue;
}
}
keyid_it++;
}
// keyids is now all non-HD keys. Each key will have its own combo descriptor
for (const CKeyID& keyid : keyids) {
CKey key;
if (!GetKey(keyid, key)) {
assert(false);
}
// Get birthdate from key meta
uint64_t creation_time = 0;
const auto& it = mapKeyMetadata.find(keyid);
if (it != mapKeyMetadata.end()) {
creation_time = it->second.nCreateTime;
}
// Get the key origin
// Maybe this doesn't matter because floating keys here shouldn't have origins
KeyOriginInfo info;
bool has_info = GetKeyOrigin(keyid, info);
std::string origin_str = has_info ? "[" + HexStr(info.fingerprint) + FormatHDKeypath(info.path) + "]" : "";
// Construct the combo descriptor
std::string desc_str = "combo(" + origin_str + HexStr(key.GetPubKey()) + ")";
FlatSigningProvider keys;
std::string error;
std::unique_ptr<Descriptor> desc = Parse(desc_str, keys, error, false);
WalletDescriptor w_desc(std::move(desc), creation_time, 0, 0, 0);
// Make the DescriptorScriptPubKeyMan and get the scriptPubKeys
auto desc_spk_man = std::unique_ptr<DescriptorScriptPubKeyMan>(new DescriptorScriptPubKeyMan(m_storage, w_desc));
desc_spk_man->AddDescriptorKey(key, key.GetPubKey());
desc_spk_man->TopUp();
auto desc_spks = desc_spk_man->GetScriptPubKeys();
// Remove the scriptPubKeys from our current set
for (const CScript& spk : desc_spks) {
size_t erased = spks.erase(spk);
assert(erased == 1);
assert(IsMine(spk) == ISMINE_SPENDABLE);
}
out.desc_spkms.push_back(std::move(desc_spk_man));
}
// Handle HD keys by using the CHDChains
std::vector<CHDChain> chains;
chains.push_back(m_hd_chain);
for (const auto& chain_pair : m_inactive_hd_chains) {
chains.push_back(chain_pair.second);
}
for (const CHDChain& chain : chains) {
for (int i = 0; i < 2; ++i) {
// Skip if doing internal chain and split chain is not supported
if (chain.seed_id.IsNull() || (i == 1 && !m_storage.CanSupportFeature(FEATURE_HD_SPLIT))) {
continue;
}
// Get the master xprv
CKey seed_key;
if (!GetKey(chain.seed_id, seed_key)) {
assert(false);
}
CExtKey master_key;
master_key.SetSeed(seed_key);
// Make the combo descriptor
std::string xpub = EncodeExtPubKey(master_key.Neuter());
std::string desc_str = "combo(" + xpub + "/0'/" + ToString(i) + "'/*')";
FlatSigningProvider keys;
std::string error;
std::unique_ptr<Descriptor> desc = Parse(desc_str, keys, error, false);
uint32_t chain_counter = std::max((i == 1 ? chain.nInternalChainCounter : chain.nExternalChainCounter), (uint32_t)0);
WalletDescriptor w_desc(std::move(desc), 0, 0, chain_counter, 0);
// Make the DescriptorScriptPubKeyMan and get the scriptPubKeys
auto desc_spk_man = std::unique_ptr<DescriptorScriptPubKeyMan>(new DescriptorScriptPubKeyMan(m_storage, w_desc));
desc_spk_man->AddDescriptorKey(master_key.key, master_key.key.GetPubKey());
desc_spk_man->TopUp();
auto desc_spks = desc_spk_man->GetScriptPubKeys();
// Remove the scriptPubKeys from our current set
for (const CScript& spk : desc_spks) {
size_t erased = spks.erase(spk);
assert(erased == 1);
assert(IsMine(spk) == ISMINE_SPENDABLE);
}
out.desc_spkms.push_back(std::move(desc_spk_man));
}
}
// Add the current master seed to the migration data
if (!m_hd_chain.seed_id.IsNull()) {
CKey seed_key;
if (!GetKey(m_hd_chain.seed_id, seed_key)) {
assert(false);
}
out.master_key.SetSeed(seed_key);
}
// Handle the rest of the scriptPubKeys which must be imports and may not have all info
for (auto it = spks.begin(); it != spks.end();) {
const CScript& spk = *it;
// Get birthdate from script meta
uint64_t creation_time = 0;
const auto& mit = m_script_metadata.find(CScriptID(spk));
if (mit != m_script_metadata.end()) {
creation_time = mit->second.nCreateTime;
}
// InferDescriptor as that will get us all the solving info if it is there
std::unique_ptr<Descriptor> desc = InferDescriptor(spk, *GetSolvingProvider(spk));
// Get the private keys for this descriptor
std::vector<CScript> scripts;
FlatSigningProvider keys;
if (!desc->Expand(0, DUMMY_SIGNING_PROVIDER, scripts, keys)) {
assert(false);
}
std::set<CKeyID> privkeyids;
for (const auto& key_orig_pair : keys.origins) {
privkeyids.insert(key_orig_pair.first);
}
std::vector<CScript> desc_spks;
// Make the descriptor string with private keys
std::string desc_str;
bool watchonly = !desc->ToPrivateString(*this, desc_str);
if (watchonly && !m_storage.IsWalletFlagSet(WALLET_FLAG_DISABLE_PRIVATE_KEYS)) {
out.watch_descs.push_back({desc->ToString(), creation_time});
// Get the scriptPubKeys without writing this to the wallet
FlatSigningProvider provider;
desc->Expand(0, provider, desc_spks, provider);
} else {
// Make the DescriptorScriptPubKeyMan and get the scriptPubKeys
WalletDescriptor w_desc(std::move(desc), creation_time, 0, 0, 0);
auto desc_spk_man = std::unique_ptr<DescriptorScriptPubKeyMan>(new DescriptorScriptPubKeyMan(m_storage, w_desc));
for (const auto& keyid : privkeyids) {
CKey key;
if (!GetKey(keyid, key)) {
continue;
}
desc_spk_man->AddDescriptorKey(key, key.GetPubKey());
}
desc_spk_man->TopUp();
auto desc_spks_set = desc_spk_man->GetScriptPubKeys();
desc_spks.insert(desc_spks.end(), desc_spks_set.begin(), desc_spks_set.end());
out.desc_spkms.push_back(std::move(desc_spk_man));
}
// Remove the scriptPubKeys from our current set
for (const CScript& desc_spk : desc_spks) {
auto del_it = spks.find(desc_spk);
assert(del_it != spks.end());
assert(IsMine(desc_spk) != ISMINE_NO);
it = spks.erase(del_it);
}
}
// Multisigs are special. They don't show up as ISMINE_SPENDABLE unless they are in a P2SH
// So we have to check if any of our scripts are a multisig and if so, add the P2SH
for (const auto& script_pair : mapScripts) {
const CScript script = script_pair.second;
// Get birthdate from script meta
uint64_t creation_time = 0;
const auto& it = m_script_metadata.find(CScriptID(script));
if (it != m_script_metadata.end()) {
creation_time = it->second.nCreateTime;
}
std::vector<std::vector<unsigned char>> sols;
TxoutType type = Solver(script, sols);
if (type == TxoutType::MULTISIG) {
CScript sh_spk = GetScriptForDestination(ScriptHash(script));
CTxDestination witdest = WitnessV0ScriptHash(script);
CScript witprog = GetScriptForDestination(witdest);
CScript sh_wsh_spk = GetScriptForDestination(ScriptHash(witprog));
// We only want the multisigs that we have not already seen, i.e. they are not watchonly and not spendable
// For P2SH, a multisig is not ISMINE_NO when:
// * All keys are in the wallet
// * The multisig itself is watch only
// * The P2SH is watch only
// For P2SH-P2WSH, if the script is in the wallet, then it will have the same conditions as P2SH.
// For P2WSH, a multisig is not ISMINE_NO when, other than the P2SH conditions:
// * The P2WSH script is in the wallet and it is being watched
std::vector<std::vector<unsigned char>> keys(sols.begin() + 1, sols.begin() + sols.size() - 1);
if (HaveWatchOnly(sh_spk) || HaveWatchOnly(script) || HaveKeys(keys, *this) || (HaveCScript(CScriptID(witprog)) && HaveWatchOnly(witprog))) {
// The above emulates IsMine for these 3 scriptPubKeys, so double check that by running IsMine
assert(IsMine(sh_spk) != ISMINE_NO || IsMine(witprog) != ISMINE_NO || IsMine(sh_wsh_spk) != ISMINE_NO);
continue;
}
assert(IsMine(sh_spk) == ISMINE_NO && IsMine(witprog) == ISMINE_NO && IsMine(sh_wsh_spk) == ISMINE_NO);
std::unique_ptr<Descriptor> sh_desc = InferDescriptor(sh_spk, *GetSolvingProvider(sh_spk));
out.solvable_descs.push_back({sh_desc->ToString(), creation_time});
const auto desc = InferDescriptor(witprog, *this);
if (desc->IsSolvable()) {
std::unique_ptr<Descriptor> wsh_desc = InferDescriptor(witprog, *GetSolvingProvider(witprog));
out.solvable_descs.push_back({wsh_desc->ToString(), creation_time});
std::unique_ptr<Descriptor> sh_wsh_desc = InferDescriptor(sh_wsh_spk, *GetSolvingProvider(sh_wsh_spk));
out.solvable_descs.push_back({sh_wsh_desc->ToString(), creation_time});
}
}
}
// Make sure that we have accounted for all scriptPubKeys
assert(spks.size() == 0);
return out;
}
bool LegacyScriptPubKeyMan::DeleteRecords()
{
LOCK(cs_KeyStore);
WalletBatch batch(m_storage.GetDatabase());
return batch.EraseRecords(DBKeys::LEGACY_TYPES);
}
util::Result<CTxDestination> DescriptorScriptPubKeyMan::GetNewDestination(const OutputType type)
{
// Returns true if this descriptor supports getting new addresses. Conditions where we may be unable to fetch them (e.g. locked) are caught later
@ -2327,14 +2640,14 @@ const WalletDescriptor DescriptorScriptPubKeyMan::GetWalletDescriptor() const
return m_wallet_descriptor;
}
const std::vector<CScript> DescriptorScriptPubKeyMan::GetScriptPubKeys() const
const std::unordered_set<CScript, SaltedSipHasher> DescriptorScriptPubKeyMan::GetScriptPubKeys() const
{
LOCK(cs_desc_man);
std::vector<CScript> script_pub_keys;
std::unordered_set<CScript, SaltedSipHasher> script_pub_keys;
script_pub_keys.reserve(m_map_script_pub_keys.size());
for (auto const& script_pub_key: m_map_script_pub_keys) {
script_pub_keys.push_back(script_pub_key.first);
script_pub_keys.insert(script_pub_key.first);
}
return script_pub_keys;
}

14
src/wallet/scriptpubkeyman.h
View file

@ -242,6 +242,9 @@ public:
virtual uint256 GetID() const { return uint256(); }
/** Returns a set of all the scriptPubKeys that this ScriptPubKeyMan watches */
virtual const std::unordered_set<CScript, SaltedSipHasher> GetScriptPubKeys() const { return {}; };
/** Prepends the wallet name in logging output to ease debugging in multi-wallet use cases */
template<typename... Params>
void WalletLogPrintf(std::string fmt, Params... parameters) const {
@ -262,6 +265,8 @@ static const std::unordered_set<OutputType> LEGACY_OUTPUT_TYPES {
OutputType::BECH32,
};
class DescriptorScriptPubKeyMan;
class LegacyScriptPubKeyMan : public ScriptPubKeyMan, public FillableSigningProvider
{
private:
@ -507,6 +512,13 @@ public:
const std::map<CKeyID, int64_t>& GetAllReserveKeys() const { return m_pool_key_to_index; }
std::set<CKeyID> GetKeys() const override;
const std::unordered_set<CScript, SaltedSipHasher> GetScriptPubKeys() const override;
/** Get the DescriptorScriptPubKeyMans (with private keys) that have the same scriptPubKeys as this LegacyScriptPubKeyMan.
* Does not modify this ScriptPubKeyMan. */
std::optional<MigrationData> MigrateToDescriptor();
/** Delete all the records ofthis LegacyScriptPubKeyMan from disk*/
bool DeleteRecords();
};
/** Wraps a LegacyScriptPubKeyMan so that it can be returned in a new unique_ptr. Does not provide privkeys */
@ -630,7 +642,7 @@ public:
void WriteDescriptor();
const WalletDescriptor GetWalletDescriptor() const EXCLUSIVE_LOCKS_REQUIRED(cs_desc_man);
const std::vector<CScript> GetScriptPubKeys() const;
const std::unordered_set<CScript, SaltedSipHasher> GetScriptPubKeys() const override;
bool GetDescriptorString(std::string& out, const bool priv) const;

37
src/wallet/test/ismine_tests.cpp
View file

@ -43,11 +43,13 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore does not have key
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has key
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2PK uncompressed
@ -60,11 +62,13 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore does not have key
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has key
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(uncompressedKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2PKH compressed
@ -77,11 +81,13 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore does not have key
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has key
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2PKH uncompressed
@ -94,11 +100,13 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore does not have key
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has key
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(uncompressedKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2SH
@ -113,16 +121,19 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore does not have redeemScript or key
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has redeemScript but no key
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(redeemScript));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has redeemScript and key
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// (P2PKH inside) P2SH inside P2SH (invalid)
@ -141,6 +152,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// (P2PKH inside) P2SH inside P2WSH (invalid)
@ -159,6 +171,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// P2WPKH inside P2WSH (invalid)
@ -175,6 +188,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// (P2PKH inside) P2WSH inside P2WSH (invalid)
@ -193,6 +207,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// P2WPKH compressed
@ -208,6 +223,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(scriptPubKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2WPKH uncompressed
@ -222,11 +238,13 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore has key, but no P2SH redeemScript
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has key and P2SH redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(scriptPubKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// scriptPubKey multisig
@ -240,24 +258,28 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore does not have any keys
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has 1/2 keys
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(uncompressedKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has 2/2 keys
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[1]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has 2/2 keys and the script
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(scriptPubKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// P2SH multisig
@ -274,11 +296,13 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore has no redeemScript
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(redeemScript));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2WSH multisig with compressed keys
@ -295,16 +319,19 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore has keys, but no witnessScript or P2SH redeemScript
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has keys and witnessScript, but no P2SH redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(witnessScript));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has keys, witnessScript, P2SH redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(scriptPubKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// P2WSH multisig with uncompressed key
@ -321,16 +348,19 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore has keys, but no witnessScript or P2SH redeemScript
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has keys and witnessScript, but no P2SH redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(witnessScript));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has keys, witnessScript, P2SH redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(scriptPubKey));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// P2WSH multisig wrapped in P2SH
@ -346,18 +376,21 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
// Keystore has no witnessScript, P2SH redeemScript, or keys
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has witnessScript and P2SH redeemScript, but no keys
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(redeemScript));
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddCScript(witnessScript));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
// Keystore has keys, witnessScript, P2SH redeemScript
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[0]));
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->AddKey(keys[1]));
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_SPENDABLE);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 1);
}
// OP_RETURN
@ -372,6 +405,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// witness unspendable
@ -386,6 +420,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// witness unknown
@ -400,6 +435,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
// Nonstandard
@ -414,6 +450,7 @@ BOOST_AUTO_TEST_CASE(ismine_standard)
result = keystore.GetLegacyScriptPubKeyMan()->IsMine(scriptPubKey);
BOOST_CHECK_EQUAL(result, ISMINE_NO);
BOOST_CHECK(keystore.GetLegacyScriptPubKeyMan()->GetScriptPubKeys().count(scriptPubKey) == 0);
}
}

519
src/wallet/wallet.cpp
View file

@ -3434,6 +3434,29 @@ void CWallet::LoadDescriptorScriptPubKeyMan(uint256 id, WalletDescriptor& desc)
}
}
void CWallet::SetupDescriptorScriptPubKeyMans(const CExtKey& master_key)
{
AssertLockHeld(cs_wallet);
for (bool internal : {false, true}) {
for (OutputType t : OUTPUT_TYPES) {
auto spk_manager = std::unique_ptr<DescriptorScriptPubKeyMan>(new DescriptorScriptPubKeyMan(*this));
if (IsCrypted()) {
if (IsLocked()) {
throw std::runtime_error(std::string(__func__) + ": Wallet is locked, cannot setup new descriptors");
}
if (!spk_manager->CheckDecryptionKey(vMasterKey) && !spk_manager->Encrypt(vMasterKey, nullptr)) {
throw std::runtime_error(std::string(__func__) + ": Could not encrypt new descriptors");
}
}
spk_manager->SetupDescriptorGeneration(master_key, t, internal);
uint256 id = spk_manager->GetID();
m_spk_managers[id] = std::move(spk_manager);
AddActiveScriptPubKeyMan(id, t, internal);
}
}
}
void CWallet::SetupDescriptorScriptPubKeyMans()
{
AssertLockHeld(cs_wallet);
@ -3449,23 +3472,7 @@ void CWallet::SetupDescriptorScriptPubKeyMans()
CExtKey master_key;
master_key.SetSeed(seed_key);
for (bool internal : {false, true}) {
for (OutputType t : OUTPUT_TYPES) {
auto spk_manager = std::unique_ptr<DescriptorScriptPubKeyMan>(new DescriptorScriptPubKeyMan(*this));
if (IsCrypted()) {
if (IsLocked()) {
throw std::runtime_error(std::string(__func__) + ": Wallet is locked, cannot setup new descriptors");
}
if (!spk_manager->CheckDecryptionKey(vMasterKey) && !spk_manager->Encrypt(vMasterKey, nullptr)) {
throw std::runtime_error(std::string(__func__) + ": Could not encrypt new descriptors");
}
}
spk_manager->SetupDescriptorGeneration(master_key, t, internal);
uint256 id = spk_manager->GetID();
m_spk_managers[id] = std::move(spk_manager);
AddActiveScriptPubKeyMan(id, t, internal);
}
}
SetupDescriptorScriptPubKeyMans(master_key);
} else {
ExternalSigner signer = ExternalSignerScriptPubKeyMan::GetExternalSigner();
@ -3633,9 +3640,13 @@ ScriptPubKeyMan* CWallet::AddWalletDescriptor(WalletDescriptor& desc, const Flat
return nullptr;
}
CTxDestination dest;
if (!internal && ExtractDestination(script_pub_keys.at(0), dest)) {
SetAddressBook(dest, label, "receive");
if (!internal) {
for (const auto& script : script_pub_keys) {
CTxDestination dest;
if (ExtractDestination(script, dest)) {
SetAddressBook(dest, label, "receive");
}
}
}
}
@ -3644,4 +3655,472 @@ ScriptPubKeyMan* CWallet::AddWalletDescriptor(WalletDescriptor& desc, const Flat
return spk_man;
}
bool CWallet::MigrateToSQLite(bilingual_str& error)
{
AssertLockHeld(cs_wallet);
WalletLogPrintf("Migrating wallet storage database from BerkeleyDB to SQLite.\n");
if (m_database->Format() == "sqlite") {
error = _("Error: This wallet already uses SQLite");
return false;
}
// Get all of the records for DB type migration
std::unique_ptr<DatabaseBatch> batch = m_database->MakeBatch();
std::vector<std::pair<SerializeData, SerializeData>> records;
if (!batch->StartCursor()) {
error = _("Error: Unable to begin reading all records in the database");
return false;
}
bool complete = false;
while (true) {
CDataStream ss_key(SER_DISK, CLIENT_VERSION);
CDataStream ss_value(SER_DISK, CLIENT_VERSION);
bool ret = batch->ReadAtCursor(ss_key, ss_value, complete);
if (!ret) {
break;
}
SerializeData key(ss_key.begin(), ss_key.end());
SerializeData value(ss_value.begin(), ss_value.end());
records.emplace_back(key, value);
}
batch->CloseCursor();
batch.reset();
if (!complete) {
error = _("Error: Unable to read all records in the database");
return false;
}
// Close this database and delete the file
fs::path db_path = fs::PathFromString(m_database->Filename());
fs::path db_dir = db_path.parent_path();
m_database->Close();
fs::remove(db_path);
// Make new DB
DatabaseOptions opts;
opts.require_create = true;
opts.require_format = DatabaseFormat::SQLITE;
DatabaseStatus db_status;
std::unique_ptr<WalletDatabase> new_db = MakeDatabase(db_dir, opts, db_status, error);
assert(new_db); // This is to prevent doing anything further with this wallet. The original file was deleted, but a backup exists.
m_database.reset();
m_database = std::move(new_db);
// Write existing records into the new DB
batch = m_database->MakeBatch();
bool began = batch->TxnBegin();
assert(began); // This is a critical error, the new db could not be written to. The original db exists as a backup, but we should not continue execution.
for (const auto& [key, value] : records) {
CDataStream ss_key(key, SER_DISK, CLIENT_VERSION);
CDataStream ss_value(value, SER_DISK, CLIENT_VERSION);
if (!batch->Write(ss_key, ss_value)) {
batch->TxnAbort();
m_database->Close();
fs::remove(m_database->Filename());
assert(false); // This is a critical error, the new db could not be written to. The original db exists as a backup, but we should not continue execution.
}
}
bool committed = batch->TxnCommit();
assert(committed); // This is a critical error, the new db could not be written to. The original db exists as a backup, but we should not continue execution.
return true;
}
std::optional<MigrationData> CWallet::GetDescriptorsForLegacy(bilingual_str& error) const
{
AssertLockHeld(cs_wallet);
LegacyScriptPubKeyMan* legacy_spkm = GetLegacyScriptPubKeyMan();
if (!legacy_spkm) {
error = _("Error: This wallet is already a descriptor wallet");
return std::nullopt;
}
std::optional<MigrationData> res = legacy_spkm->MigrateToDescriptor();
if (res == std::nullopt) {
error = _("Error: Unable to produce descriptors for this legacy wallet. Make sure the wallet is unlocked first");
return std::nullopt;
}
return res;
}
bool CWallet::ApplyMigrationData(MigrationData& data, bilingual_str& error)
{
AssertLockHeld(cs_wallet);
LegacyScriptPubKeyMan* legacy_spkm = GetLegacyScriptPubKeyMan();
if (!legacy_spkm) {
error = _("Error: This wallet is already a descriptor wallet");
return false;
}
for (auto& desc_spkm : data.desc_spkms) {
if (m_spk_managers.count(desc_spkm->GetID()) > 0) {
error = _("Error: Duplicate descriptors created during migration. Your wallet may be corrupted.");
return false;
}
m_spk_managers[desc_spkm->GetID()] = std::move(desc_spkm);
}
// Remove the LegacyScriptPubKeyMan from disk
if (!legacy_spkm->DeleteRecords()) {
return false;
}
// Remove the LegacyScriptPubKeyMan from memory
m_spk_managers.erase(legacy_spkm->GetID());
m_external_spk_managers.clear();
m_internal_spk_managers.clear();
// Setup new descriptors
SetWalletFlag(WALLET_FLAG_DESCRIPTORS);
if (!IsWalletFlagSet(WALLET_FLAG_DISABLE_PRIVATE_KEYS)) {
// Use the existing master key if we have it
if (data.master_key.key.IsValid()) {
SetupDescriptorScriptPubKeyMans(data.master_key);
} else {
// Setup with a new seed if we don't.
SetupDescriptorScriptPubKeyMans();
}
}
// Check if the transactions in the wallet are still ours. Either they belong here, or they belong in the watchonly wallet.
// We need to go through these in the tx insertion order so that lookups to spends works.
std::vector<uint256> txids_to_delete;
for (const auto& [_pos, wtx] : wtxOrdered) {
if (!IsMine(*wtx->tx) && !IsFromMe(*wtx->tx)) {
// Check it is the watchonly wallet's
// solvable_wallet doesn't need to be checked because transactions for those scripts weren't being watched for
if (data.watchonly_wallet) {
LOCK(data.watchonly_wallet->cs_wallet);
if (data.watchonly_wallet->IsMine(*wtx->tx) || data.watchonly_wallet->IsFromMe(*wtx->tx)) {
// Add to watchonly wallet
if (!data.watchonly_wallet->AddToWallet(wtx->tx, wtx->m_state)) {
error = _("Error: Could not add watchonly tx to watchonly wallet");
return false;
}
// Mark as to remove from this wallet
txids_to_delete.push_back(wtx->GetHash());
continue;
}
}
// Both not ours and not in the watchonly wallet
error = strprintf(_("Error: Transaction %s in wallet cannot be identified to belong to migrated wallets"), wtx->GetHash().GetHex());
return false;
}
}
// Do the removes
if (txids_to_delete.size() > 0) {
std::vector<uint256> deleted_txids;
if (ZapSelectTx(txids_to_delete, deleted_txids) != DBErrors::LOAD_OK) {
error = _("Error: Could not delete watchonly transactions");
return false;
}
if (deleted_txids != txids_to_delete) {
error = _("Error: Not all watchonly txs could be deleted");
return false;
}
// Tell the GUI of each tx
for (const uint256& txid : deleted_txids) {
NotifyTransactionChanged(txid, CT_UPDATED);
}
}
// Check the address book data in the same way we did for transactions
std::vector<CTxDestination> dests_to_delete;
for (const auto& addr_pair : m_address_book) {
// Labels applied to receiving addresses should go based on IsMine
if (addr_pair.second.purpose == "receive") {
if (!IsMine(addr_pair.first)) {
// Check the address book data is the watchonly wallet's
if (data.watchonly_wallet) {
LOCK(data.watchonly_wallet->cs_wallet);
if (data.watchonly_wallet->IsMine(addr_pair.first)) {
// Add to the watchonly. Preserve the labels, purpose, and change-ness
std::string label = addr_pair.second.GetLabel();
std::string purpose = addr_pair.second.purpose;
if (!purpose.empty()) {
data.watchonly_wallet->m_address_book[addr_pair.first].purpose = purpose;
}
if (!addr_pair.second.IsChange()) {
data.watchonly_wallet->m_address_book[addr_pair.first].SetLabel(label);
}
dests_to_delete.push_back(addr_pair.first);
continue;
}
}
if (data.solvable_wallet) {
LOCK(data.solvable_wallet->cs_wallet);
if (data.solvable_wallet->IsMine(addr_pair.first)) {
// Add to the solvable. Preserve the labels, purpose, and change-ness
std::string label = addr_pair.second.GetLabel();
std::string purpose = addr_pair.second.purpose;
if (!purpose.empty()) {
data.solvable_wallet->m_address_book[addr_pair.first].purpose = purpose;
}
if (!addr_pair.second.IsChange()) {
data.solvable_wallet->m_address_book[addr_pair.first].SetLabel(label);
}
dests_to_delete.push_back(addr_pair.first);
continue;
}
}
// Not ours, not in watchonly wallet, and not in solvable
error = _("Error: Address book data in wallet cannot be identified to belong to migrated wallets");
return false;
}
} else {
// Labels for everything else (send) should be cloned to all
if (data.watchonly_wallet) {
LOCK(data.watchonly_wallet->cs_wallet);
// Add to the watchonly. Preserve the labels, purpose, and change-ness
std::string label = addr_pair.second.GetLabel();
std::string purpose = addr_pair.second.purpose;
if (!purpose.empty()) {
data.watchonly_wallet->m_address_book[addr_pair.first].purpose = purpose;
}
if (!addr_pair.second.IsChange()) {
data.watchonly_wallet->m_address_book[addr_pair.first].SetLabel(label);
}
continue;
}
if (data.solvable_wallet) {
LOCK(data.solvable_wallet->cs_wallet);
// Add to the solvable. Preserve the labels, purpose, and change-ness
std::string label = addr_pair.second.GetLabel();
std::string purpose = addr_pair.second.purpose;
if (!purpose.empty()) {
data.solvable_wallet->m_address_book[addr_pair.first].purpose = purpose;
}
if (!addr_pair.second.IsChange()) {
data.solvable_wallet->m_address_book[addr_pair.first].SetLabel(label);
}
continue;
}
}
}
// Remove the things to delete
if (dests_to_delete.size() > 0) {
for (const auto& dest : dests_to_delete) {
if (!DelAddressBook(dest)) {
error = _("Error: Unable to remove watchonly address book data");
return false;
}
}
}
// Connect the SPKM signals
ConnectScriptPubKeyManNotifiers();
NotifyCanGetAddressesChanged();
WalletLogPrintf("Wallet migration complete.\n");
return true;
}
bool DoMigration(CWallet& wallet, WalletContext& context, bilingual_str& error, MigrationResult& res) EXCLUSIVE_LOCKS_REQUIRED(wallet.cs_wallet)
{
AssertLockHeld(wallet.cs_wallet);
// Get all of the descriptors from the legacy wallet
std::optional<MigrationData> data = wallet.GetDescriptorsForLegacy(error);
if (data == std::nullopt) return false;
// Create the watchonly and solvable wallets if necessary
if (data->watch_descs.size() > 0 || data->solvable_descs.size() > 0) {
DatabaseOptions options;
options.require_existing = false;
options.require_create = true;
// Make the wallets
options.create_flags = WALLET_FLAG_DISABLE_PRIVATE_KEYS | WALLET_FLAG_BLANK_WALLET | WALLET_FLAG_DESCRIPTORS;
if (wallet.IsWalletFlagSet(WALLET_FLAG_AVOID_REUSE)) {
options.create_flags |= WALLET_FLAG_AVOID_REUSE;
}
if (wallet.IsWalletFlagSet(WALLET_FLAG_KEY_ORIGIN_METADATA)) {
options.create_flags |= WALLET_FLAG_KEY_ORIGIN_METADATA;
}
if (data->watch_descs.size() > 0) {
wallet.WalletLogPrintf("Making a new watchonly wallet containing the watched scripts\n");
DatabaseStatus status;
std::vector<bilingual_str> warnings;
std::string wallet_name = wallet.GetName() + "_watchonly";
data->watchonly_wallet = CreateWallet(context, wallet_name, std::nullopt, options, status, error, warnings);
if (status != DatabaseStatus::SUCCESS) {
error = _("Error: Failed to create new watchonly wallet");
return false;
}
res.watchonly_wallet = data->watchonly_wallet;
LOCK(data->watchonly_wallet->cs_wallet);
// Parse the descriptors and add them to the new wallet
for (const auto& [desc_str, creation_time] : data->watch_descs) {
// Parse the descriptor
FlatSigningProvider keys;
std::string parse_err;
std::unique_ptr<Descriptor> desc = Parse(desc_str, keys, parse_err, /* require_checksum */ true);
assert(desc); // It shouldn't be possible to have the LegacyScriptPubKeyMan make an invalid descriptor
assert(!desc->IsRange()); // It shouldn't be possible to have LegacyScriptPubKeyMan make a ranged watchonly descriptor
// Add to the wallet
WalletDescriptor w_desc(std::move(desc), creation_time, 0, 0, 0);
data->watchonly_wallet->AddWalletDescriptor(w_desc, keys, "", false);
}
// Add the wallet to settings
UpdateWalletSetting(*context.chain, wallet_name, /*load_on_startup=*/true, warnings);
}
if (data->solvable_descs.size() > 0) {
wallet.WalletLogPrintf("Making a new watchonly wallet containing the unwatched solvable scripts\n");
DatabaseStatus status;
std::vector<bilingual_str> warnings;
std::string wallet_name = wallet.GetName() + "_solvables";
data->solvable_wallet = CreateWallet(context, wallet_name, std::nullopt, options, status, error, warnings);
if (status != DatabaseStatus::SUCCESS) {
error = _("Error: Failed to create new watchonly wallet");
return false;
}
res.solvables_wallet = data->solvable_wallet;
LOCK(data->solvable_wallet->cs_wallet);
// Parse the descriptors and add them to the new wallet
for (const auto& [desc_str, creation_time] : data->solvable_descs) {
// Parse the descriptor
FlatSigningProvider keys;
std::string parse_err;
std::unique_ptr<Descriptor> desc = Parse(desc_str, keys, parse_err, /* require_checksum */ true);
assert(desc); // It shouldn't be possible to have the LegacyScriptPubKeyMan make an invalid descriptor
assert(!desc->IsRange()); // It shouldn't be possible to have LegacyScriptPubKeyMan make a ranged watchonly descriptor
// Add to the wallet
WalletDescriptor w_desc(std::move(desc), creation_time, 0, 0, 0);
data->solvable_wallet->AddWalletDescriptor(w_desc, keys, "", false);
}
// Add the wallet to settings
UpdateWalletSetting(*context.chain, wallet_name, /*load_on_startup=*/true, warnings);
}
}
// Add the descriptors to wallet, remove LegacyScriptPubKeyMan, and cleanup txs and address book data
if (!wallet.ApplyMigrationData(*data, error)) {
return false;
}
return true;
}
util::Result<MigrationResult> MigrateLegacyToDescriptor(std::shared_ptr<CWallet>&& wallet, WalletContext& context)
{
MigrationResult res;
bilingual_str error;
std::vector<bilingual_str> warnings;
// Make a backup of the DB
std::string wallet_name = wallet->GetName();
fs::path this_wallet_dir = fs::absolute(fs::PathFromString(wallet->GetDatabase().Filename())).parent_path();
fs::path backup_filename = fs::PathFromString(strprintf("%s-%d.legacy.bak", wallet_name, GetTime()));
fs::path backup_path = this_wallet_dir / backup_filename;
if (!wallet->BackupWallet(fs::PathToString(backup_path))) {
return util::Error{_("Error: Unable to make a backup of your wallet")};
}
res.backup_path = backup_path;
// Unload the wallet so that nothing else tries to use it while we're changing it
if (!RemoveWallet(context, wallet, /*load_on_start=*/std::nullopt, warnings)) {
return util::Error{_("Unable to unload the wallet before migrating")};
}
UnloadWallet(std::move(wallet));
// Load the wallet but only in the context of this function.
// No signals should be connected nor should anything else be aware of this wallet
WalletContext empty_context;
empty_context.args = context.args;
DatabaseOptions options;
options.require_existing = true;
DatabaseStatus status;
std::unique_ptr<WalletDatabase> database = MakeWalletDatabase(wallet_name, options, status, error);
if (!database) {
return util::Error{Untranslated("Wallet file verification failed.") + Untranslated(" ") + error};
}
std::shared_ptr<CWallet> local_wallet = CWallet::Create(empty_context, wallet_name, std::move(database), options.create_flags, error, warnings);
if (!local_wallet) {
return util::Error{Untranslated("Wallet loading failed.") + Untranslated(" ") + error};
}
bool success = false;
{
LOCK(local_wallet->cs_wallet);
// First change to using SQLite
if (!local_wallet->MigrateToSQLite(error)) return util::Error{error};
// Do the migration, and cleanup if it fails
success = DoMigration(*local_wallet, context, error, res);
}
if (success) {
// Migration successful, unload the wallet locally, then reload it.
assert(local_wallet.use_count() == 1);
local_wallet.reset();
LoadWallet(context, wallet_name, /*load_on_start=*/std::nullopt, options, status, error, warnings);
res.wallet_name = wallet_name;
} else {
// Migration failed, cleanup
// Copy the backup to the actual wallet dir
fs::path temp_backup_location = fsbridge::AbsPathJoin(GetWalletDir(), backup_filename);
fs::copy_file(backup_path, temp_backup_location, fs::copy_options::none);
// Remember this wallet's walletdir to remove after unloading
std::vector<fs::path> wallet_dirs;
wallet_dirs.push_back(fs::PathFromString(local_wallet->GetDatabase().Filename()).parent_path());
// Unload the wallet locally
assert(local_wallet.use_count() == 1);
local_wallet.reset();
// Make list of wallets to cleanup
std::vector<std::shared_ptr<CWallet>> created_wallets;
created_wallets.push_back(std::move(res.watchonly_wallet));
created_wallets.push_back(std::move(res.solvables_wallet));
// Get the directories to remove after unloading
for (std::shared_ptr<CWallet>& w : created_wallets) {
wallet_dirs.push_back(fs::PathFromString(w->GetDatabase().Filename()).parent_path());
}
// Unload the wallets
for (std::shared_ptr<CWallet>& w : created_wallets) {
if (!RemoveWallet(context, w, /*load_on_start=*/false)) {
error += _("\nUnable to cleanup failed migration");
return util::Error{error};
}
UnloadWallet(std::move(w));
}
// Delete the wallet directories
for (fs::path& dir : wallet_dirs) {
fs::remove_all(dir);
}
// Restore the backup
DatabaseStatus status;
std::vector<bilingual_str> warnings;
if (!RestoreWallet(context, temp_backup_location, wallet_name, /*load_on_start=*/std::nullopt, status, error, warnings)) {
error += _("\nUnable to restore backup of wallet.");
return util::Error{error};
}
// Move the backup to the wallet dir
fs::copy_file(temp_backup_location, backup_path, fs::copy_options::none);
fs::remove(temp_backup_location);
return util::Error{error};
}
return res;
}
} // namespace wallet

27
src/wallet/wallet.h
View file

@ -316,7 +316,7 @@ private:
std::string m_name;
/** Internal database handle. */
std::unique_ptr<WalletDatabase> const m_database;
std::unique_ptr<WalletDatabase> m_database;
/**
* The following is used to keep track of how far behind the wallet is
@ -907,6 +907,7 @@ public:
void DeactivateScriptPubKeyMan(uint256 id, OutputType type, bool internal);
//! Create new DescriptorScriptPubKeyMans and add them to the wallet
void SetupDescriptorScriptPubKeyMans(const CExtKey& master_key) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
void SetupDescriptorScriptPubKeyMans() EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
//! Return the DescriptorScriptPubKeyMan for a WalletDescriptor if it is already in the wallet
@ -919,6 +920,20 @@ public:
//! Add a descriptor to the wallet, return a ScriptPubKeyMan & associated output type
ScriptPubKeyMan* AddWalletDescriptor(WalletDescriptor& desc, const FlatSigningProvider& signing_provider, const std::string& label, bool internal) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
/** Move all records from the BDB database to a new SQLite database for storage.
* The original BDB file will be deleted and replaced with a new SQLite file.
* A backup is not created.
* May crash if something unexpected happens in the filesystem.
*/
bool MigrateToSQLite(bilingual_str& error) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
//! Get all of the descriptors from a legacy wallet
std::optional<MigrationData> GetDescriptorsForLegacy(bilingual_str& error) const EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
//! Adds the ScriptPubKeyMans given in MigrationData to this wallet, removes LegacyScriptPubKeyMan,
//! and where needed, moves tx and address book entries to watchonly_wallet or solvable_wallet
bool ApplyMigrationData(MigrationData& data, bilingual_str& error) EXCLUSIVE_LOCKS_REQUIRED(cs_wallet);
};
/**
@ -977,6 +992,16 @@ bool RemoveWalletSetting(interfaces::Chain& chain, const std::string& wallet_nam
bool DummySignInput(const SigningProvider& provider, CTxIn &tx_in, const CTxOut &txout, const CCoinControl* coin_control = nullptr);
bool FillInputToWeight(CTxIn& txin, int64_t target_weight);
struct MigrationResult {
std::string wallet_name;
std::shared_ptr<CWallet> watchonly_wallet;
std::shared_ptr<CWallet> solvables_wallet;
fs::path backup_path;
};
//! Do all steps to migrate a legacy wallet to a descriptor wallet
util::Result<MigrationResult> MigrateLegacyToDescriptor(std::shared_ptr<CWallet>&& wallet, WalletContext& context);
} // namespace wallet
#endif // BITCOIN_WALLET_WALLET_H

40
src/wallet/walletdb.cpp
View file

@ -59,6 +59,7 @@ const std::string WALLETDESCRIPTORCKEY{"walletdescriptorckey"};
const std::string WALLETDESCRIPTORKEY{"walletdescriptorkey"};
const std::string WATCHMETA{"watchmeta"};
const std::string WATCHS{"watchs"};
const std::unordered_set<std::string> LEGACY_TYPES{CRYPTED_KEY, CSCRIPT, DEFAULTKEY, HDCHAIN, KEYMETA, KEY, OLD_KEY, POOL, WATCHMETA, WATCHS};
} // namespace DBKeys
//
@ -1083,6 +1084,45 @@ bool WalletBatch::WriteWalletFlags(const uint64_t flags)
return WriteIC(DBKeys::FLAGS, flags);
}
bool WalletBatch::EraseRecords(const std::unordered_set<std::string>& types)
{
// Get cursor
if (!m_batch->StartCursor())
{
return false;
}
// Iterate the DB and look for any records that have the type prefixes
while (true)
{
// Read next record
CDataStream key(SER_DISK, CLIENT_VERSION);
CDataStream value(SER_DISK, CLIENT_VERSION);
bool complete;
bool ret = m_batch->ReadAtCursor(key, value, complete);
if (complete) {
break;
}
else if (!ret)
{
m_batch->CloseCursor();
return false;
}
// Make a copy of key to avoid data being deleted by the following read of the type
Span<const unsigned char> key_data = MakeUCharSpan(key);
std::string type;
key >> type;
if (types.count(type) > 0) {
m_batch->Erase(key_data);
}
}
m_batch->CloseCursor();
return true;
}
bool WalletBatch::TxnBegin()
{
return m_batch->TxnBegin();

6
src/wallet/walletdb.h
View file

@ -84,6 +84,9 @@ extern const std::string WALLETDESCRIPTORCKEY;
extern const std::string WALLETDESCRIPTORKEY;
extern const std::string WATCHMETA;
extern const std::string WATCHS;
// Keys in this set pertain only to the legacy wallet (LegacyScriptPubKeyMan) and are removed during migration from legacy to descriptors.
extern const std::unordered_set<std::string> LEGACY_TYPES;
} // namespace DBKeys
/* simple HD chain data model */
@ -276,6 +279,9 @@ public:
//! write the hdchain model (external chain child index counter)
bool WriteHDChain(const CHDChain& chain);
//! Delete records of the given types
bool EraseRecords(const std::unordered_set<std::string>& types);
bool WriteWalletFlags(const uint64_t flags);
//! Begin a new transaction
bool TxnBegin();

14
src/wallet/walletutil.h
View file

@ -104,6 +104,20 @@ public:
WalletDescriptor() {}
WalletDescriptor(std::shared_ptr<Descriptor> descriptor, uint64_t creation_time, int32_t range_start, int32_t range_end, int32_t next_index) : descriptor(descriptor), creation_time(creation_time), range_start(range_start), range_end(range_end), next_index(next_index) {}
};
class CWallet;
class DescriptorScriptPubKeyMan;
/** struct containing information needed for migrating legacy wallets to descriptor wallets */
struct MigrationData
{
CExtKey master_key;
std::vector<std::pair<std::string, int64_t>> watch_descs;
std::vector<std::pair<std::string, int64_t>> solvable_descs;
std::vector<std::unique_ptr<DescriptorScriptPubKeyMan>> desc_spkms;
std::shared_ptr<CWallet> watchonly_wallet{nullptr};
std::shared_ptr<CWallet> solvable_wallet{nullptr};
};
} // namespace wallet
#endif // BITCOIN_WALLET_WALLETUTIL_H

1
test/functional/test_runner.py
View file

@ -341,6 +341,7 @@ BASE_SCRIPTS = [
'feature_dirsymlinks.py',
'feature_help.py',
'feature_shutdown.py',
'wallet_migration.py',
'p2p_ibd_txrelay.py',
# Don't append tests at the end to avoid merge conflicts
# Put them in a random line within the section that fits their approximate run-time

10
test/functional/tool_wallet.py
View file

@ -68,7 +68,7 @@ class ToolWalletTest(BitcoinTestFramework):
result = 'unchanged' if new == old else 'increased!'
self.log.debug('Wallet file timestamp {}'.format(result))
def get_expected_info_output(self, name="", transactions=0, keypool=2, address=0):
def get_expected_info_output(self, name="", transactions=0, keypool=2, address=0, imported_privs=0):
wallet_name = self.default_wallet_name if name == "" else name
if self.options.descriptors:
output_types = 4 # p2pkh, p2sh, segwit, bech32m
@ -83,7 +83,7 @@ class ToolWalletTest(BitcoinTestFramework):
Keypool Size: %d
Transactions: %d
Address Book: %d
''' % (wallet_name, keypool * output_types, transactions, address))
''' % (wallet_name, keypool * output_types, transactions, imported_privs * 3 + address))
else:
output_types = 3 # p2pkh, p2sh, segwit. Legacy wallets do not support bech32m.
return textwrap.dedent('''\
@ -97,7 +97,7 @@ class ToolWalletTest(BitcoinTestFramework):
Keypool Size: %d
Transactions: %d
Address Book: %d
''' % (wallet_name, keypool, transactions, address * output_types))
''' % (wallet_name, keypool, transactions, (address + imported_privs) * output_types))
def read_dump(self, filename):
dump = OrderedDict()
@ -219,7 +219,7 @@ class ToolWalletTest(BitcoinTestFramework):
# shasum_before = self.wallet_shasum()
timestamp_before = self.wallet_timestamp()
self.log.debug('Wallet file timestamp before calling info: {}'.format(timestamp_before))
out = self.get_expected_info_output(address=1)
out = self.get_expected_info_output(imported_privs=1)
self.assert_tool_output(out, '-wallet=' + self.default_wallet_name, 'info')
timestamp_after = self.wallet_timestamp()
self.log.debug('Wallet file timestamp after calling info: {}'.format(timestamp_after))
@ -250,7 +250,7 @@ class ToolWalletTest(BitcoinTestFramework):
shasum_before = self.wallet_shasum()
timestamp_before = self.wallet_timestamp()
self.log.debug('Wallet file timestamp before calling info: {}'.format(timestamp_before))
out = self.get_expected_info_output(transactions=1, address=1)
out = self.get_expected_info_output(transactions=1, imported_privs=1)
self.assert_tool_output(out, '-wallet=' + self.default_wallet_name, 'info')
shasum_after = self.wallet_shasum()
timestamp_after = self.wallet_timestamp()

407
test/functional/wallet_migration.py Executable file
View file

@ -0,0 +1,407 @@
#!/usr/bin/env python3
# Copyright (c) 2020 The Bitcoin Core developers
# Distributed under the MIT software license, see the accompanying
# file COPYING or http://www.opensource.org/licenses/mit-license.php.
"""Test Migrating a wallet from legacy to descriptor."""
import os
import random
from test_framework.descriptors import descsum_create
from test_framework.test_framework import BitcoinTestFramework
from test_framework.util import (
assert_equal,
assert_raises_rpc_error,
find_vout_for_address,
)
from test_framework.wallet_util import (
get_generate_key,
)
class WalletMigrationTest(BitcoinTestFramework):
def set_test_params(self):
self.setup_clean_chain = True
self.num_nodes = 1
self.extra_args = [[]]
self.supports_cli = False
def skip_test_if_missing_module(self):
self.skip_if_no_wallet()
self.skip_if_no_sqlite()
self.skip_if_no_bdb()
def assert_is_sqlite(self, wallet_name):
wallet_file_path = os.path.join(self.nodes[0].datadir, "regtest/wallets", wallet_name, self.wallet_data_filename)
with open(wallet_file_path, 'rb') as f:
file_magic = f.read(16)
assert_equal(file_magic, b'SQLite format 3\x00')
assert_equal(self.nodes[0].get_wallet_rpc(wallet_name).getwalletinfo()["format"], "sqlite")
def create_legacy_wallet(self, wallet_name):
self.nodes[0].createwallet(wallet_name=wallet_name)
wallet = self.nodes[0].get_wallet_rpc(wallet_name)
assert_equal(wallet.getwalletinfo()["descriptors"], False)
assert_equal(wallet.getwalletinfo()["format"], "bdb")
return wallet
def assert_addr_info_equal(self, addr_info, addr_info_old):
assert_equal(addr_info["address"], addr_info_old["address"])
assert_equal(addr_info["scriptPubKey"], addr_info_old["scriptPubKey"])
assert_equal(addr_info["ismine"], addr_info_old["ismine"])
assert_equal(addr_info["hdkeypath"], addr_info_old["hdkeypath"])
assert_equal(addr_info["solvable"], addr_info_old["solvable"])
assert_equal(addr_info["ischange"], addr_info_old["ischange"])
assert_equal(addr_info["hdmasterfingerprint"], addr_info_old["hdmasterfingerprint"])
def assert_list_txs_equal(self, received_list_txs, expected_list_txs):
for d in received_list_txs:
if "parent_descs" in d:
del d["parent_descs"]
for d in expected_list_txs:
if "parent_descs" in d:
del d["parent_descs"]
assert_equal(received_list_txs, expected_list_txs)
def test_basic(self):
default = self.nodes[0].get_wallet_rpc(self.default_wallet_name)
self.log.info("Test migration of a basic keys only wallet without balance")
basic0 = self.create_legacy_wallet("basic0")
addr = basic0.getnewaddress()
change = basic0.getrawchangeaddress()
old_addr_info = basic0.getaddressinfo(addr)
old_change_addr_info = basic0.getaddressinfo(change)
assert_equal(old_addr_info["ismine"], True)
assert_equal(old_addr_info["hdkeypath"], "m/0'/0'/0'")
assert_equal(old_change_addr_info["ismine"], True)
assert_equal(old_change_addr_info["hdkeypath"], "m/0'/1'/0'")
# Note: migration could take a while.
basic0.migratewallet()
# Verify created descriptors
assert_equal(basic0.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("basic0")
# The wallet should create the following descriptors:
# * BIP32 descriptors in the form of "0'/0'/*" and "0'/1'/*" (2 descriptors)
# * BIP44 descriptors in the form of "44'/1'/0'/0/*" and "44'/1'/0'/1/*" (2 descriptors)
# * BIP49 descriptors, P2SH(P2WPKH), in the form of "86'/1'/0'/0/*" and "86'/1'/0'/1/*" (2 descriptors)
# * BIP84 descriptors, P2WPKH, in the form of "84'/1'/0'/1/*" and "84'/1'/0'/1/*" (2 descriptors)
# * BIP86 descriptors, P2TR, in the form of "86'/1'/0'/0/*" and "86'/1'/0'/1/*" (2 descriptors)
# * A combo(PK) descriptor for the wallet master key.
# So, should have a total of 11 descriptors on it.
assert_equal(len(basic0.listdescriptors()["descriptors"]), 11)
# Compare addresses info
addr_info = basic0.getaddressinfo(addr)
change_addr_info = basic0.getaddressinfo(change)
self.assert_addr_info_equal(addr_info, old_addr_info)
self.assert_addr_info_equal(change_addr_info, old_change_addr_info)
addr_info = basic0.getaddressinfo(basic0.getnewaddress("", "bech32"))
assert_equal(addr_info["hdkeypath"], "m/84'/1'/0'/0/0")
self.log.info("Test migration of a basic keys only wallet with a balance")
basic1 = self.create_legacy_wallet("basic1")
for _ in range(0, 10):
default.sendtoaddress(basic1.getnewaddress(), 1)
self.generate(self.nodes[0], 1)
for _ in range(0, 5):
basic1.sendtoaddress(default.getnewaddress(), 0.5)
self.generate(self.nodes[0], 1)
bal = basic1.getbalance()
txs = basic1.listtransactions()
basic1.migratewallet()
assert_equal(basic1.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("basic1")
assert_equal(basic1.getbalance(), bal)
self.assert_list_txs_equal(basic1.listtransactions(), txs)
# restart node and verify that everything is still there
self.restart_node(0)
default = self.nodes[0].get_wallet_rpc(self.default_wallet_name)
self.nodes[0].loadwallet("basic1")
basic1 = self.nodes[0].get_wallet_rpc("basic1")
assert_equal(basic1.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("basic1")
assert_equal(basic1.getbalance(), bal)
self.assert_list_txs_equal(basic1.listtransactions(), txs)
self.log.info("Test migration of a wallet with balance received on the seed")
basic2 = self.create_legacy_wallet("basic2")
basic2_seed = get_generate_key()
basic2.sethdseed(True, basic2_seed.privkey)
assert_equal(basic2.getbalance(), 0)
# Receive coins on different output types for the same seed
basic2_balance = 0
for addr in [basic2_seed.p2pkh_addr, basic2_seed.p2wpkh_addr, basic2_seed.p2sh_p2wpkh_addr]:
send_value = random.randint(1, 4)
default.sendtoaddress(addr, send_value)
basic2_balance += send_value
self.generate(self.nodes[0], 1)
assert_equal(basic2.getbalance(), basic2_balance)
basic2_txs = basic2.listtransactions()
# Now migrate and test that we still see have the same balance/transactions
basic2.migratewallet()
assert_equal(basic2.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("basic2")
assert_equal(basic2.getbalance(), basic2_balance)
self.assert_list_txs_equal(basic2.listtransactions(), basic2_txs)
def test_multisig(self):
default = self.nodes[0].get_wallet_rpc(self.default_wallet_name)
# Contrived case where all the multisig keys are in a single wallet
self.log.info("Test migration of a wallet with all keys for a multisig")
multisig0 = self.create_legacy_wallet("multisig0")
addr1 = multisig0.getnewaddress()
addr2 = multisig0.getnewaddress()
addr3 = multisig0.getnewaddress()
ms_info = multisig0.addmultisigaddress(2, [addr1, addr2, addr3])
multisig0.migratewallet()
assert_equal(multisig0.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("multisig0")
ms_addr_info = multisig0.getaddressinfo(ms_info["address"])
assert_equal(ms_addr_info["ismine"], True)
assert_equal(ms_addr_info["desc"], ms_info["descriptor"])
assert_equal("multisig0_watchonly" in self.nodes[0].listwallets(), False)
assert_equal("multisig0_solvables" in self.nodes[0].listwallets(), False)
pub1 = multisig0.getaddressinfo(addr1)["pubkey"]
pub2 = multisig0.getaddressinfo(addr2)["pubkey"]
# Some keys in multisig do not belong to this wallet
self.log.info("Test migration of a wallet that has some keys in a multisig")
self.nodes[0].createwallet(wallet_name="multisig1")
multisig1 = self.nodes[0].get_wallet_rpc("multisig1")
ms_info = multisig1.addmultisigaddress(2, [multisig1.getnewaddress(), pub1, pub2])
ms_info2 = multisig1.addmultisigaddress(2, [multisig1.getnewaddress(), pub1, pub2])
assert_equal(multisig1.getwalletinfo()["descriptors"], False)
addr1 = ms_info["address"]
addr2 = ms_info2["address"]
txid = default.sendtoaddress(addr1, 10)
multisig1.importaddress(addr1)
assert_equal(multisig1.getaddressinfo(addr1)["ismine"], False)
assert_equal(multisig1.getaddressinfo(addr1)["iswatchonly"], True)
assert_equal(multisig1.getaddressinfo(addr1)["solvable"], True)
self.generate(self.nodes[0], 1)
multisig1.gettransaction(txid)
assert_equal(multisig1.getbalances()["watchonly"]["trusted"], 10)
assert_equal(multisig1.getaddressinfo(addr2)["ismine"], False)
assert_equal(multisig1.getaddressinfo(addr2)["iswatchonly"], False)
assert_equal(multisig1.getaddressinfo(addr2)["solvable"], True)
# Migrating multisig1 should see the multisig is no longer part of multisig1
# A new wallet multisig1_watchonly is created which has the multisig address
# Transaction to multisig is in multisig1_watchonly and not multisig1
multisig1.migratewallet()
assert_equal(multisig1.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("multisig1")
assert_equal(multisig1.getaddressinfo(addr1)["ismine"], False)
assert_equal(multisig1.getaddressinfo(addr1)["iswatchonly"], False)
assert_equal(multisig1.getaddressinfo(addr1)["solvable"], False)
assert_raises_rpc_error(-5, "Invalid or non-wallet transaction id", multisig1.gettransaction, txid)
assert_equal(multisig1.getbalance(), 0)
assert_equal(multisig1.listtransactions(), [])
assert_equal("multisig1_watchonly" in self.nodes[0].listwallets(), True)
ms1_watchonly = self.nodes[0].get_wallet_rpc("multisig1_watchonly")
ms1_wallet_info = ms1_watchonly.getwalletinfo()
assert_equal(ms1_wallet_info['descriptors'], True)
assert_equal(ms1_wallet_info['private_keys_enabled'], False)
self.assert_is_sqlite("multisig1_watchonly")
assert_equal(ms1_watchonly.getaddressinfo(addr1)["ismine"], True)
assert_equal(ms1_watchonly.getaddressinfo(addr1)["solvable"], True)
# Because addr2 was not being watched, it isn't in multisig1_watchonly but rather multisig1_solvables
assert_equal(ms1_watchonly.getaddressinfo(addr2)["ismine"], False)
assert_equal(ms1_watchonly.getaddressinfo(addr2)["solvable"], False)
ms1_watchonly.gettransaction(txid)
assert_equal(ms1_watchonly.getbalance(), 10)
# Migrating multisig1 should see the second multisig is no longer part of multisig1
# A new wallet multisig1_solvables is created which has the second address
# This should have no transactions
assert_equal("multisig1_solvables" in self.nodes[0].listwallets(), True)
ms1_solvable = self.nodes[0].get_wallet_rpc("multisig1_solvables")
ms1_wallet_info = ms1_solvable.getwalletinfo()
assert_equal(ms1_wallet_info['descriptors'], True)
assert_equal(ms1_wallet_info['private_keys_enabled'], False)
self.assert_is_sqlite("multisig1_solvables")
assert_equal(ms1_solvable.getaddressinfo(addr1)["ismine"], False)
assert_equal(ms1_solvable.getaddressinfo(addr1)["solvable"], False)
assert_equal(ms1_solvable.getaddressinfo(addr2)["ismine"], True)
assert_equal(ms1_solvable.getaddressinfo(addr2)["solvable"], True)
assert_equal(ms1_solvable.getbalance(), 0)
assert_equal(ms1_solvable.listtransactions(), [])
def test_other_watchonly(self):
default = self.nodes[0].get_wallet_rpc(self.default_wallet_name)
# Wallet with an imported address. Should be the same thing as the multisig test
self.log.info("Test migration of a wallet with watchonly imports")
self.nodes[0].createwallet(wallet_name="imports0")
imports0 = self.nodes[0].get_wallet_rpc("imports0")
assert_equal(imports0.getwalletinfo()["descriptors"], False)
# Exteranl address label
imports0.setlabel(default.getnewaddress(), "external")
# Normal non-watchonly tx
received_addr = imports0.getnewaddress()
imports0.setlabel(received_addr, "Receiving")
received_txid = default.sendtoaddress(received_addr, 10)
# Watchonly tx
import_addr = default.getnewaddress()
imports0.importaddress(import_addr)
imports0.setlabel(import_addr, "imported")
received_watchonly_txid = default.sendtoaddress(import_addr, 10)
# Received watchonly tx that is then spent
import_sent_addr = default.getnewaddress()
imports0.importaddress(import_sent_addr)
received_sent_watchonly_txid = default.sendtoaddress(import_sent_addr, 10)
received_sent_watchonly_vout = find_vout_for_address(self.nodes[0], received_sent_watchonly_txid, import_sent_addr)
send = default.sendall(recipients=[default.getnewaddress()], options={"inputs": [{"txid": received_sent_watchonly_txid, "vout": received_sent_watchonly_vout}]})
sent_watchonly_txid = send["txid"]
self.generate(self.nodes[0], 1)
balances = imports0.getbalances()
spendable_bal = balances["mine"]["trusted"]
watchonly_bal = balances["watchonly"]["trusted"]
assert_equal(len(imports0.listtransactions(include_watchonly=True)), 4)
# Migrate
imports0.migratewallet()
assert_equal(imports0.getwalletinfo()["descriptors"], True)
self.assert_is_sqlite("imports0")
assert_raises_rpc_error(-5, "Invalid or non-wallet transaction id", imports0.gettransaction, received_watchonly_txid)
assert_raises_rpc_error(-5, "Invalid or non-wallet transaction id", imports0.gettransaction, received_sent_watchonly_txid)
assert_raises_rpc_error(-5, "Invalid or non-wallet transaction id", imports0.gettransaction, sent_watchonly_txid)
assert_equal(len(imports0.listtransactions(include_watchonly=True)), 1)
imports0.gettransaction(received_txid)
assert_equal(imports0.getbalance(), spendable_bal)
assert_equal("imports0_watchonly" in self.nodes[0].listwallets(), True)
watchonly = self.nodes[0].get_wallet_rpc("imports0_watchonly")
watchonly_info = watchonly.getwalletinfo()
assert_equal(watchonly_info["descriptors"], True)
self.assert_is_sqlite("imports0_watchonly")
assert_equal(watchonly_info["private_keys_enabled"], False)
watchonly.gettransaction(received_watchonly_txid)
watchonly.gettransaction(received_sent_watchonly_txid)
watchonly.gettransaction(sent_watchonly_txid)
assert_equal(watchonly.getbalance(), watchonly_bal)
assert_raises_rpc_error(-5, "Invalid or non-wallet transaction id", watchonly.gettransaction, received_txid)
assert_equal(len(watchonly.listtransactions(include_watchonly=True)), 3)
def test_no_privkeys(self):
default = self.nodes[0].get_wallet_rpc(self.default_wallet_name)
# Migrating an actual watchonly wallet should not create a new watchonly wallet
self.log.info("Test migration of a pure watchonly wallet")
self.nodes[0].createwallet(wallet_name="watchonly0", disable_private_keys=True)
watchonly0 = self.nodes[0].get_wallet_rpc("watchonly0")
info = watchonly0.getwalletinfo()
assert_equal(info["descriptors"], False)
assert_equal(info["private_keys_enabled"], False)
addr = default.getnewaddress()
desc = default.getaddressinfo(addr)["desc"]
res = watchonly0.importmulti([
{
"desc": desc,
"watchonly": True,
"timestamp": "now",
}])
assert_equal(res[0]['success'], True)
default.sendtoaddress(addr, 10)
self.generate(self.nodes[0], 1)
watchonly0.migratewallet()
assert_equal("watchonly0_watchonly" in self.nodes[0].listwallets(), False)
info = watchonly0.getwalletinfo()
assert_equal(info["descriptors"], True)
assert_equal(info["private_keys_enabled"], False)
self.assert_is_sqlite("watchonly0")
# Migrating a wallet with pubkeys added to the keypool
self.log.info("Test migration of a pure watchonly wallet with pubkeys in keypool")
self.nodes[0].createwallet(wallet_name="watchonly1", disable_private_keys=True)
watchonly1 = self.nodes[0].get_wallet_rpc("watchonly1")
info = watchonly1.getwalletinfo()
assert_equal(info["descriptors"], False)
assert_equal(info["private_keys_enabled"], False)
addr1 = default.getnewaddress(address_type="bech32")
addr2 = default.getnewaddress(address_type="bech32")
desc1 = default.getaddressinfo(addr1)["desc"]
desc2 = default.getaddressinfo(addr2)["desc"]
res = watchonly1.importmulti([
{
"desc": desc1,
"keypool": True,
"timestamp": "now",
},
{
"desc": desc2,
"keypool": True,
"timestamp": "now",
}
])
assert_equal(res[0]["success"], True)
assert_equal(res[1]["success"], True)
# Before migrating, we can fetch addr1 from the keypool
assert_equal(watchonly1.getnewaddress(address_type="bech32"), addr1)
watchonly1.migratewallet()
info = watchonly1.getwalletinfo()
assert_equal(info["descriptors"], True)
assert_equal(info["private_keys_enabled"], False)
self.assert_is_sqlite("watchonly1")
# After migrating, the "keypool" is empty
assert_raises_rpc_error(-4, "Error: This wallet has no available keys", watchonly1.getnewaddress)
def test_pk_coinbases(self):
self.log.info("Test migration of a wallet using old pk() coinbases")
wallet = self.create_legacy_wallet("pkcb")
addr = wallet.getnewaddress()
addr_info = wallet.getaddressinfo(addr)
desc = descsum_create("pk(" + addr_info["pubkey"] + ")")
self.nodes[0].generatetodescriptor(1, desc, invalid_call=False)
bals = wallet.getbalances()
wallet.migratewallet()
assert_equal(bals, wallet.getbalances())
def run_test(self):
self.generate(self.nodes[0], 101)
# TODO: Test the actual records in the wallet for these tests too. The behavior may be correct, but the data written may not be what we actually want
self.test_basic()
self.test_multisig()
self.test_other_watchonly()
self.test_no_privkeys()
self.test_pk_coinbases()
if __name__ == '__main__':
WalletMigrationTest().main()