Commit graph

22539 commits

Author SHA1 Message Date
Andrew Chow
7cecf10ac3 Replace LegacyScriptPubKeyMan::IsCrypted with LegacyScriptPubKeyMan::HasEncryptionKeys 2019-12-06 15:05:48 -05:00
Andrew Chow
bf6417142f Remove SetCrypted() and fUseCrypto; Change IsCrypted()'s implementation
Removes SetCrypted() and fUseCrypto as we don't need them anymore.
SetCrypted calls in LegacyScriptPubKeyMan are replaced with mapKeys.empty()

IsCrypted() is changed to just call HasEncryptionKeys()
2019-12-06 15:05:48 -05:00
Andrew Chow
77a777118e Rename EncryptKeys to Encrypt and pass in the encrypted batch to use 2019-12-06 15:05:48 -05:00
Andrew Chow
35f962fcf0 Clear mapKeys before encrypting
Does not change behavior. Needed to make AddCryptedKeyInner() work
with SetCrypted() being gone.
2019-12-06 15:05:48 -05:00
Andrew Chow
14b5efd66f Move fDecryptionThoroughlyChecked from CWallet to LegacyScriptPubKeyMan 2019-12-06 15:05:48 -05:00
Andrew Chow
97c0374a46 Move Unlock implementation to LegacyScriptPubKeyMan
CWallet::Unlock is changed to call ScriptPubKeyMan::CheckDecryptionKey
and the original implementation of Unlock is renamed to CheckDecryptionKey.
2019-12-06 15:05:47 -05:00
Andrew Chow
e576b135d6 Replace LegacyScriptPubKeyMan::vMasterKey with GetDecryptionKey() 2019-12-06 15:05:08 -05:00
Andrew Chow
fd9d6eebc1 Add GetEncryptionKey() and HasEncryptionKeys() to WalletStorage
Adds functions in WalletStorage that allow ScriptPubKeyMans to check
and get encryption keys from the wallet.
2019-12-06 15:05:08 -05:00
Hennadii Stepanov
711e0449cf
ci: Remove trusty build
Co-authored-by: Emil Engler <me@emilengler.com>
2019-12-06 21:54:37 +02:00
Hennadii Stepanov
7f3ae22468
ci: Add CentOS 7 build 2019-12-06 21:44:35 +02:00
fanquake
4ee8a58ce7
Merge #17373: wallet: Various fixes and cleanup to keypool handling in LegacyScriptPubKeyMan and CWallet
886f1731be Key pool: Fix omitted pre-split count in GetKeyPoolSize (Andrew Chow)
386a994b85 Key pool: Change ReturnDestination interface to take address instead of key (Andrew Chow)
ba41aa4969 Key pool: Move LearnRelated and GetDestination calls (Andrew Chow)
65833a7407 Add OutputType and CPubKey parameters to KeepDestination (Andrew Chow)
9fcf8ce7ae Rename Keep/ReturnKey to Keep/ReturnDestination and remove the wrapper (Andrew Chow)
596f6460f9 Key pool: Move CanGetAddresses call (Andrew Chow)

Pull request description:

  * The `pwallet->CanGetAddresses()` call in `ReserveDestination::GetReservedDestination` to `LegacyScriptPubKeyMan::GetReservedDestination` so that the sanity check results in a failure when a `ScriptPubKeyMan` individually cannot get a destination, not when any of the `ScriptPubKeyMan`s can't.
  * `ScriptPubKeyMan::GetReservedDestination` is changed to return the destination so that future `ScriptPubKeyMan`s can return destinations constructed in other ways. This is implemented for `LegacyScriptPubKeyMan` by moving key-to-destination code from `CWallet` to `LegacyScriptPubKeyMan`
  * In order for `ScriptPubKeyMan` to be generic and work with future `ScriptPubKeyMan`s, `ScriptPubKeyMan::ReturnDestination` is changed to take a `CTxDestination` instead of a `CPubKey`. Since `LegacyScriptPubKeyMan` still deals with keys internally, a new map `m_reserved_key_to_index` is added in order to track the keypool indexes that have been reserved.
  * A bug is fixed in how the total keypool size is calculated as it was omitting `set_pre_split_keypool` which is a bug.

  Split from #17261

ACKs for top commit:
  ryanofsky:
    Code review ACK 886f1731be. Only change is moving earlier fix to a better commit (same end result).
  promag:
    Code review ACK 886f1731be.
  instagibbs:
    code review re-ACK 886f1731be
  Sjors:
    Code review re-ACK 886f1731be

Tree-SHA512: f4be290759f63fdc920d5c02bd0d09acc4b06a5f053787d4afcd3c921b2e35d2bd97617fadae015da853dc189f559fb8d2c6e58d53e4cabfac9af151cd97ad19
2019-12-06 13:37:30 -05:00
MarcoFalke
da1af855f9
Merge #17275: pubkey: Assert CPubKey's ECCVerifyHandle precondition
d8daa8f371 pubkey: Assert CPubKey's ECCVerifyHandle precondition (practicalswift)

Pull request description:

  Assert `CPubKey`'s `ECCVerifyHandle` precondition.

  This makes it more clear for fuzzing harness writers and others that `ECCVerifyHandle` is expected to be held when interacting with `CPubKey`.

  Related PR #17274.

ACKs for top commit:
  sipa:
    ACK d8daa8f371

Tree-SHA512: 9e74086599799dc9b5c3fb8357445b662e5bf896d826af63d6d6b6ddb616612966f3bb5de3bd3ae0e692c47de85672f64b8ab6d3a1c45899dc25ba46990b5ec7
2019-12-06 13:35:37 -05:00
practicalswift
6338c02034 tests: Fix fuzzing harness for descriptor parsing (descriptor_parse) 2019-12-06 18:25:51 +00:00
nijynot
a5a705b46d lcov: filter depends from coverage report 2019-12-06 19:14:15 +01:00
practicalswift
5ac804a9eb tests: Use a default of supports_cli=True (instead of supports_cli=False) 2019-12-06 15:24:06 +00:00
practicalswift
993e38a4e2 tests: Mark functional tests not supporting bitcoin-cli (--usecli) as such 2019-12-06 14:40:28 +00:00
practicalswift
709afb2a7d tests: Test serialisation as part of deserialisation fuzzing. Test round-trip equality where possible. Avoid code repetition. 2019-12-06 09:15:56 +00:00
practicalswift
597d10ceb9 tests: Add fuzzing harness for various functions consuming only integrals 2019-12-06 09:14:17 +00:00
practicalswift
575383b3e1 tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-06 09:14:03 +00:00
practicalswift
d5766f223f tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus 2019-12-06 09:11:02 +00:00
practicalswift
e75ecb91c7 tests: Add fuzzing harness for various CTxOut related functions 2019-12-06 09:10:44 +00:00
practicalswift
ce935292c0 tests: Add fuzzing harness for various CTxIn related functions 2019-12-06 09:10:31 +00:00
Wladimir J. van der Laan
978b25528c util: Update tinyformat to upstream
Last update was in 2017.
Updates tinyformat to upstream commit 705e3f4e1de922069bf715746d35bd2364b1f98f.
Re-apply bitcoin core specific changes.

No changes that affect our use, as far as I can see, but this gets rid
of the gcc `-Wimplicit-fallthrough` warnings, at least.
2019-12-06 10:02:08 +01:00
Wladimir J. van der Laan
cb11324a63
Merge #17051: tests: Add deserialization fuzzing harnesses
897849d8c2 tests: Add deserialization fuzzing harnesses (practicalswift)
16f0a186dc tests: Add corpora suppression (FUZZERS_MISSING_CORPORA) for fuzzers missing in https://github.com/bitcoin-core/qa-assets/tree/master/fuzz_seed_corpus (practicalswift)

Pull request description:

  Add deserialization fuzzing harnesses.

  **Testing this PR**

  Run:

  ```
  $ CC=clang CXX=clang++ ./configure --enable-fuzz --with-sanitizers=address,fuzzer,undefined
  $ make
  $ contrib/devtools/test_fuzzing_harnesses.sh 'addr_info|block_file_info|block_filter|block_header|ext_key|ext_pub_key|fee_rate|flat_file|key_origin|merkle_block|mutable_transaction|out_point|partial_merkle_tree|partially_signed_transaction|prefilled_transaction|psbt_input|psbt_output|pub_key|script_deserialize|sub_net|tx_in' 10
  ```

  `test_fuzzing_harnesses.sh` can be found in PR #17000.

ACKs for top commit:
  laanwj:
    thanks, ACK 897849d8c2

Tree-SHA512: 5a270a3002cc23b725f7b35476a43777b2b00b4d089cc006372e2fcc7afa430afaa3c1430f778ae08fc53dd85a13e7bd2fab0449c319f676423226e189a417f6
2019-12-06 09:45:26 +01:00
Wladimir J. van der Laan
c7c9c44278
Merge #17663: build: pass -dead_strip_dylibs to ld on macOS
bd44711e1b build: pass -dead_strip_dylibs to ld on macOS (fanquake)

Pull request description:

  This strips some unused dylibs from bitcoin-qt.

  ```diff
  otool -L src/qt/bitcoin-qt
    /usr/lib/libSystem.B.dylib
  - /System/Library/Frameworks/DiskArbitration.framework/Versions/A/DiskArbitration
    /System/Library/Frameworks/IOKit.framework/Versions/A/IOKit
    /System/Library/Frameworks/Foundation.framework/Versions/C/Foundation
    /System/Library/Frameworks/CoreServices.framework/Versions/A/CoreServices
    /System/Library/Frameworks/AppKit.framework/Versions/C/AppKit
    /System/Library/Frameworks/ApplicationServices.framework/Versions/A/ApplicationServices
    /System/Library/Frameworks/CoreFoundation.framework/Versions/A/CoreFoundation
  -/System/Library/Frameworks/Security.framework/Versions/A/Security
    /System/Library/Frameworks/SystemConfiguration.framework/Versions/A/SystemConfiguration
    /System/Library/Frameworks/CoreGraphics.framework/Versions/A/CoreGraphics
    /System/Library/Frameworks/OpenGL.framework/Versions/A/OpenGL
  -/System/Library/Frameworks/AGL.framework/Versions/A/AGL
    /System/Library/Frameworks/Carbon.framework/Versions/A/Carbon
    /usr/lib/libc++.1.dylib
    /System/Library/Frameworks/CFNetwork.framework/Versions/A/CFNetwork
    /System/Library/Frameworks/CoreText.framework/Versions/A/CoreText
    /System/Library/Frameworks/ImageIO.framework/Versions/A/ImageIO
    /usr/lib/libobjc.A.dylib
  ```

  `AGL` - ObjC wrapper for OpenGL.
  `DiskArbitration` - mount/unmount notifications and events.
  `Security` - low level security operations, authentication services.

  From `man ld`:
  ```
  Remove dylibs that are unreachable by the entry point or exported symbols.
  That is, suppresses the generation of load command commands for dylibs
  which supplied no symbols during the link. This option should not be
  used when linking against a dylib which is required at runtime for
  some indirect reason such as the dylib has an important initializer.
  ```

ACKs for top commit:
  theuni:
    ACK bd44711e1b.

Tree-SHA512: 9592ce2966d28cb6c58e01efd401f56a4baa5dc5be5313f4fe8454632b578608be65a23c8602772049cd4655a9cb020fdd40d6622a244c301920d8c3db43f99a
2019-12-06 09:11:14 +01:00
Pieter Wuille
5909bcd3bf Add bounds checks in key_io before DecodeBase58Check 2019-12-05 16:31:09 -08:00
willyk
c966ff14c7
gitian: fixed SC2001 regex
the `-` is not a special symbol and should not have `%` in front of it.
2019-12-05 13:00:34 -08:00
MarcoFalke
cf43f3f0a8
Merge #17674: tests: Add initialization order fiasco detection in Travis
1f9d5af4f1 tests: Add initialization order fiasco detection in Travis (practicalswift)

Pull request description:

  Add initialization order fiasco detection in Travis :)

  Context: https://github.com/bitcoin/bitcoin/pull/17670#issuecomment-562035813

  This would have caught the `events_hasher` initialization order issue introduced in #17573  and fixed in #17670.

  Output in case of an initialization order fiasco:

  ```
  ==7934==ERROR: AddressSanitizer: initialization-order-fiasco on address 0x557098d79200 at pc 0x55709796b9a3 bp 0x7ffde524dc30 sp 0x7ffde524dc28
  READ of size 8 at 0x557098d79200 thread T0
      #0 0x55709796b9a2 in CSHA256::Finalize(unsigned char*) src/crypto/sha256.cpp:667:25
      #1 0x5570978150e9 in SeedEvents(CSHA512&) src/random.cpp:462:19
      #2 0x5570978145e1 in SeedSlow(CSHA512&) src/random.cpp:482:5
      #3 0x5570978149a3 in SeedStartup(CSHA512&, (anonymous namespace)::RNGState&) src/random.cpp:527:5
      #4 0x55709781102d in ProcRand(unsigned char*, int, RNGLevel) src/random.cpp:571:9
      #5 0x557097810d19 in GetRandBytes(unsigned char*, int) src/random.cpp:576:59
      #6 0x557096c2f9d5 in (anonymous namespace)::CSignatureCache::CSignatureCache() src/script/sigcache.cpp:34:9
      #7 0x557096511977 in __cxx_global_var_init.7 src/script/sigcache.cpp:67:24
      #8 0x5570965119f8 in _GLOBAL__sub_I_sigcache.cpp src/script/sigcache.cpp
      #9 0x557097bba4ac in __libc_csu_init (src/bitcoind+0x18554ac)
      #10 0x7f214b1c2b27 in __libc_start_main /build/glibc-OTsEL5/glibc-2.27/csu/../csu/libc-start.c:266
      #11 0x5570965347d9 in _start (src/bitcoind+0x1cf7d9)

  0x557098d79200 is located 96 bytes inside of global variable 'events_hasher' defined in 'random.cpp:456:16' (0x557098d791a0) of size 104
    registered at:
      #0 0x557096545dfd in __asan_register_globals compiler-rt/lib/asan/asan_globals.cpp:360:3
      #1 0x557097817f8b in asan.module_ctor (src/bitcoind+0x14b2f8b)

  SUMMARY: AddressSanitizer: initialization-order-fiasco src/crypto/sha256.cpp:667:25 in CSHA256::Finalize(unsigned char*)
  ```

ACKs for top commit:
  promag:
    Tested ACK 1f9d5af4f1, got
  MarcoFalke:
    ACK 1f9d5af4f1 👔

Tree-SHA512: f24ac0a313df7549193bd7f4fcfdf9b72bdfc6a6ee31d0b08e6d0752e5108fbd532106b6c86377ae0641258c9adb4921872e5d9a0154c0284e03315e0777102c
2019-12-05 15:56:14 -05:00
MarcoFalke
5d0b7f9e3d
Merge #17670: Move events_hasher into RNGState()
8bda0960f9 Move events_hasher into RNGState() (Pieter Wuille)

Pull request description:

  This moves `events_hasher` and `events_mutex` into `RNGState()` in random.cpp. This guarantees (through the existing `GetRNGState()` function) that the mutex is always created before any events are added, even when that happens inside global initializers.

  Fixes the issue reported here: https://github.com/bitcoin/bitcoin/pull/17573#issuecomment-561828251, and includes the annotation from #17666).

ACKs for top commit:
  MarcoFalke:
    re-ACK 8bda0960f9 🥈
  sipsorcery:
    re-ACK 8bda0960f9.

Tree-SHA512: 78702d668764df19e9d61d87d82eca71cceca87d5351b740e13e732a1c18a3d53d7fbaaf63245266da597370bfebec9fa6a4749c15ec5a78dcfe6122c33553ed
2019-12-05 15:55:27 -05:00
MarcoFalke
910a4301b1
Merge #17522: test: Wait until mempool is loaded in wallet_abandonconflict
dddd09eb33 test: Wait until mempool is loaded in wallet_abandonconflict (MarcoFalke)

Pull request description:

  This might or might not fix intermittent issues such as https://ci.appveyor.com/project/DrahtBot/bitcoin/builds/28724018#L4091

  I believe the mempool was not loaded fully after the restart, in which case it was not dumped either on the next restart. Thus, the previous mempool was attempted to be loaded a second time, which succeeded and contained the txs.

ACKs for top commit:
  laanwj:
    ACK dddd09eb33

Tree-SHA512: ab7061f946b5e5388f825dddceadb125f5197b24af3a7fcf1e700235d106a323419a56bfb4d84a2e27442e0de63e540c623b704343d83a98deaab3c02fcbdcbe
2019-12-05 15:52:17 -05:00
MarcoFalke
fa660d65d7
node: Use mempool from node context instead of global 2019-12-05 14:22:05 -05:00
MarcoFalke
facbaf092f
rpc: Use mempool from node context instead of global
Currently they are identical, but in the future we might want to turn
the mempool into a unique_ptr. Replacing the global with the mempool
pointer from the node context simplifies this step.
2019-12-05 13:45:49 -05:00
Pieter Wuille
8bda0960f9 Move events_hasher into RNGState() 2019-12-05 09:49:11 -08:00
MarcoFalke
11113247c3
depends: Support for S390X targets 2019-12-05 12:12:46 -05:00
Luke Dashjr
989fd539d5 depends: Support for 64-bit POWER targets 2019-12-05 12:06:28 -05:00
Andrew Chow
6e77a7b65c keypool: Add comment about TopUp and when to use it 2019-12-05 11:51:52 -05:00
Wladimir J. van der Laan
6fff333c9f
Merge #17507: random: mark RandAddPeriodic and SeedPeriodic as noexcept
55b2cb199c random: mark RandAddPeriodic and SeedPeriodic as noexcept (fanquake)
461e547877 doc: correct random.h docs after #17270 (fanquake)

Pull request description:

  The usage of `MilliSleep()` in SeedPeriodic (previously SeedSleep) was
  [removed](d61f2bb076) in #17270, meaning it, and its users can now be marked `noexcept`.

  This also corrects the docs in random.h for some of the changes in #17270.

ACKs for top commit:
  practicalswift:
    ACK 55b2cb199c
  laanwj:
    ACK 55b2cb199c
  sipa:
    ACK 55b2cb199c

Tree-SHA512: 672d369796e7c4f9b4d98dc545e5454999fa1bef373871994a26041d6163c58909e2255e4f820d3ef011679aa3392754eb57477306a89f5fd3d57e2bd7f0811a
2019-12-05 15:14:42 +01:00
fanquake
2359a4790d
depends: don't use OpenGL in Qt on macOS 2019-12-05 09:07:03 -05:00
fanquake
ba0cad2702
build: pass -dead_strip_dylibs to ld on macOS
This strips some unused dylibs from bitcoin-qt.

From man ld:
Remove dylibs that are unreachable by the entry point or exported symbols. 
That is, suppresses the generation of load command commands for dylibs 
which supplied no symbols during the link. This option should not be 
used when linking against a dylib which is required at runtime for 
some indirect reason such as the dylib has an important initializer.
2019-12-05 08:41:15 -05:00
practicalswift
1f9d5af4f1 tests: Add initialization order fiasco detection in Travis 2019-12-05 10:54:18 +00:00
willyk
b11d35b5e2 Fixed wget call in gitian-build.py 2019-12-05 02:21:09 -08:00
fanquake
5aee0e2163
Merge #17650: util: remove unwanted fields from bitcoin-cli -getinfo
01c8701559 util: remove unwanted fields from bitcoin-cli -getinfo (malevolent)

Pull request description:

  Removed the following fields from -getinfo: protocolversion, walletversion and keypoololdest. This change closes #17314 .

ACKs for top commit:
  laanwj:
    ACK 01c8701559
  achow101:
    ACK 01c8701559
  practicalswift:
    ACK 01c8701559 -- diff looks correct

Tree-SHA512: c98f2e8a3fee04d46766f70cb88f4e49e892a4424eff8940a7d48e9e808597b702427225788f984f5c3641591fd8d86acee56774afde1d57a4259c31d971ea08
2019-12-04 14:59:12 -05:00
MarcoFalke
fee01bb053
Merge #17517: ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le
fa40e48c50 ci: Remove unparseable lines from supp file for old xenial clang tsan (MarcoFalke)
fa1bfc476c ci: ubsan report_error_type=1 and add suppressions (MarcoFalke)
fa69cef13e test: Print stderr when subprocess fails (MarcoFalke)
2222c30586 test: Use char instead of unsigned char (MarcoFalke)
faa8023ce9 ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le (MarcoFalke)

Pull request description:

  Use clang-8 instead of default clang (which is clang-6 on Bionic) to avoid spurious segfaults when running the ci system on ppc64le

ACKs for top commit:
  practicalswift:
    ACK fa40e48c50 assuming Travis is happy -- diff looks correct :)

Tree-SHA512: f4f26232d3a0ef38da245869340f723d279a3db9823befbc735fb5a00024dae041c7306d7ae55d2488e6f86aa96cdea155b007aefb561fba505141e8dbc717dc
2019-12-04 13:23:16 -05:00
MarcoFalke
fa40e48c50
ci: Remove unparseable lines from supp file for old xenial clang tsan 2019-12-04 12:16:52 -05:00
MarcoFalke
fa1bfc476c
ci: ubsan report_error_type=1 and add suppressions 2019-12-04 12:16:38 -05:00
MarcoFalke
fa69cef13e
test: Print stderr when subprocess fails 2019-12-04 11:25:32 -05:00
MarcoFalke
2222c30586
test: Use char instead of unsigned char 2019-12-04 09:32:19 -05:00
MarcoFalke
faa8023ce9
ci: Bump to clang-8 for asan build to avoid segfaults on ppc64le 2019-12-04 09:32:15 -05:00
Wladimir J. van der Laan
41919631d5
Merge #17573: Seed RNG with precision timestamps on receipt of net messages.
02d8c56a18 Seed RNG with precision timestamps on receipt of net messages. (Matt Corallo)

Pull request description:

  See title. Exposes a generic dead-simple "SeedEvent" interface, but currently just used for net messages.

ACKs for top commit:
  sipa:
    utACK 02d8c56a18
  laanwj:
    ACK 02d8c56a18
  meshcollider:
    utACK 02d8c56a18

Tree-SHA512: 28eb39a201ee2b13393c5c64dbf7c1913f3482f095969ef5141bfe549ce77dd63bb5f14738f6eedb296c686ea36014aa157b9c5e8059710a318590f30e9caa14
2019-12-04 13:12:44 +01:00
Wladimir J. van der Laan
dcbe024f5e
Merge #17648: doc: rename wallet-tool references to bitcoin-wallet
e7ad4a2f8c doc: rename wallet-tool references to bitcoin-wallet (Wilson Ccasihue S)

Pull request description:

  Fix. text reference to executable bitcoin-wallet instead of wallet-tool, there is not a wallet-tool at bin/ folder.

ACKs for top commit:
  fanquake:
    ACK e7ad4a2f8c - thanks for following up.

Tree-SHA512: aed41b08947728a4ff3a97a62858ee7c86e2e5d57dcbbd0aab492dae3d8a548bb60541924e68cf3a0aa3d53d7db0012b489462b466919cd83f05b2aa88b7fff7
2019-12-04 11:00:40 +01:00