Commit graph

2903 commits

Author SHA1 Message Date
MarcoFalke
ebd8d66454
Merge #19203: net: Add regression fuzz harness for CVE-2017-18350. Add FuzzedSocket.
366e3e1f89 fuzz: Add FUZZED_SOCKET_FAKE_LATENCY mode to FuzzedSock to allow for fuzzing timeout logic (practicalswift)
b22d4c1607 fuzz: Add fuzzing harness for Socks5(...) (practicalswift)

Pull request description:

  Add [regression fuzz harness](https://twitter.com/kayseesee/status/1205287895923212289) for CVE-2017-18350. This fuzzing harness would have found CVE-2017-18350 within a minute of fuzzing :)

  See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).

  Happy fuzzing :)

ACKs for top commit:
  vasild:
    ACK 366e3e1f89

Tree-SHA512: 5d8e1863b635efd10ccb11678b71472ba1523c3ef16affa7f9cd638635c1a9c307e28f432d5b87eb0c9cd1c3c1aeafbb24fa7ae86fe4e5090fda2e20d542b6ca
2021-03-03 14:41:05 +01:00
Anthony Towns
0c471a5f30
tests: check never active versionbits 2021-03-03 14:14:40 +01:00
Anthony Towns
3ba9283a47
tests: more helpful errors for failing versionbits tests
Co-authored-by: Sjors Provoost <sjors@sprovoost.nl>
2021-03-03 14:12:16 +01:00
MarcoFalke
fa576b4532
Move MakeNoLogFileContext to common libtest_util, and use it in bench
Can be reviewed with --color-moved=dimmed-zebra
2021-03-03 09:17:37 +01:00
practicalswift
366e3e1f89 fuzz: Add FUZZED_SOCKET_FAKE_LATENCY mode to FuzzedSock to allow for fuzzing timeout logic 2021-03-02 21:44:51 +00:00
practicalswift
b22d4c1607 fuzz: Add fuzzing harness for Socks5(...) 2021-03-02 21:43:42 +00:00
practicalswift
1f05dbd06d util: Avoid invalid integer negation in ValueFromAmount: make ValueFromAmount(const CAmount& n) well-defined also when n is std::numeric_limits<CAmount>::min() 2021-03-02 16:05:28 +00:00
practicalswift
7cc75c9ba3 util: Avoid invalid integer negation in FormatMoney: make FormatMoney(const CAmount& n) well-defined also when n is std::numeric_limits<CAmount>::min() 2021-03-02 16:05:28 +00:00
practicalswift
10d4477dae tests: Add fuzzing harness for TorController 2021-03-02 12:21:32 +00:00
Vasil Dimov
cff65c4a27
net: extend CNetAddr::SetSpecial() to support I2P
Recognize also I2P addresses in the form `base32hashofpublickey.b32.i2p`
from `CNetAddr::SetSpecial()`.

This makes `Lookup()` support them, which in turn makes it possible to
manually connect to an I2P node by using
`-proxy=i2p_socks5_proxy:port -addnode=i2p_address.b32.i2p:port`

Co-authored-by: Lucas Ontivero <lucasontivero@gmail.com>
2021-03-01 13:22:11 +01:00
Anthony Towns
f8c0688b94 scripted-diff: Update txorphanage naming convention
-BEGIN VERIFY SCRIPT-
sed -i 's/mapOrphanTransactionsByPrev/m_outpoint_to_orphan_it/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/mapOrphanTransactions/m_orphans/g' src/txorphanage.h src/txorphanage.cpp src/net_processing.cpp src/test/denialofservice_tests.cpp
sed -i 's/g_orphan_list/m_orphan_list/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/g_orphans_by_wtxid/m_wtxid_to_orphan_it/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/nMaxOrphans/max_orphans/g' src/txorphanage.h src/txorphanage.cpp
sed -i 's/COrphanTx/OrphanTx/g' src/txorphanage.h src/txorphanage.cpp src/test/denialofservice_tests.cpp
-END VERIFY SCRIPT-
2021-02-27 01:08:09 +10:00
Anthony Towns
6bd4963c06 txorphanage: Move functions and data into class
Collects all the orphan handling globals into a single member var in
net_processing, and ensures access is encapuslated into the interface
functions. Also adds doxygen comments for methods.
2021-02-27 01:07:55 +10:00
Anthony Towns
26d1a6ccd5 denialofservices_tests: check txorphanage's AddTx
Rather than checking net_processing's internal implementation of
AddOrphanTx, test txorphanage's exported AddTx interface. Note that
this means AddToCompactExtraTransactions is no longer tested here.
2021-02-26 23:55:10 +10:00
Anthony Towns
9d5313df7e move-only: Add txorphanage module
This module captures orphan tracking code for tx relay.

Can be reviewed with --color-moved=dimmed-zebra
2021-02-26 23:55:03 +10:00
MarcoFalke
cac10e66d2
Merge #21264: fuzz: Two scripted diff renames
fae216a73d scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext (MarcoFalke)
fa4fbec03e scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTION (MarcoFalke)

Pull request description:

  Split out two renames from #21003:

  * `PROVIDE_FUZZ_MAIN_FUNCTION`. *Reason*: This in only used by fuzzing, so the name should indicate that.
  * `MakeNoLogFileContext`. *Reason*: Better reflects what the helper does. Also, prepares it to be used in non-fuzz tests in the future.

ACKs for top commit:
  practicalswift:
    cr ACK fae216a73d: scripted-diff looks correct

Tree-SHA512: e5d347746f5da72b0c86fd4f07ac2e4b3016e88e8c97a830c73bd79d0af6d0245fe7712487fc20344d6cc25958941716c1678124a123930407e3a437265b71df
2021-02-25 14:42:48 +01:00
Hennadii Stepanov
df8f2a11dc
test: Replace accidentally placed bit-OR with logical-OR 2021-02-24 12:57:03 +02:00
Wladimir J. van der Laan
a9335e4f12
Merge #16546: External signer support - Wallet Box edition
f75e0c1edd doc: add external-signer.md (Sjors Provoost)
d4b0107d68 rpc: send: support external signer (Sjors Provoost)
245b4457cf rpc: signerdisplayaddress (Sjors Provoost)
7ebc7c0215 wallet: ExternalSigner: add GetDescriptors method (Sjors Provoost)
fc5da520f5 wallet: add GetExternalSigner() (Sjors Provoost)
259f52cc33 test: external_signer wallet flag is immutable (Sjors Provoost)
2655197e1c rpc: add external_signer option to createwallet (Sjors Provoost)
2700f09c41 rpc: signer: add enumeratesigners to list external signers (Sjors Provoost)
07b7c940a7 rpc: add external signer RPC files (Sjors Provoost)
8ce7767071 wallet: add ExternalSignerScriptPubKeyMan (Sjors Provoost)
157ea7c614 wallet: add external_signer flag (Sjors Provoost)
f3e6ce78fb test: add external signer test (Sjors Provoost)
8cf543f96d wallet: add -signer argument for external signer command (Sjors Provoost)
f7eb7ecc67 test: framework: add skip_if_no_external_signer (Sjors Provoost)
87a97941f6 configure: add --enable-external-signer (Sjors Provoost)

Pull request description:

  Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).

  This PR lets `bitcoind` call an arbitrary command `-signer=<cmd>`, e.g. a hardware wallet driver,  where it can fetch public keys, ask to display an address, and sign a transaction (using PSBT under the hood).

  It's design to work with https://github.com/bitcoin-core/HWI, which supports multiple hardware wallets. Any command with the same arguments and return values will work. It simplifies the manual procedure described [here](https://github.com/bitcoin-core/HWI/blob/master/docs/bitcoin-core-usage.md).

  Usage is documented in [doc/external-signer.md](
  https://github.com/Sjors/bitcoin/blob/2019/08/hww-box2/doc/external-signer.md), which also describes what protocol a different signer binary should conform to.

  Use `--enable-external-signer` to opt in, requires Boost::Process:

  ```
  Options used to compile and link:
    with wallet     = yes
    with gui / qt   = no
    external signer = yes
  ```

  It adds the following RPC methods:
  * `enumeratesigners`: asks <cmd> for a list of signers (e.g. devices) and their master key fingerprint
  * `signerdisplayaddress <address>`:  asks <cmd> to display an address

  It enhances the following RPC methods:
  * `createwallet`: takes an additional `external_signer` argument and fetches keys from device
  * `send`: automatically sends transaction to device and waits

  Usage TL&DR:
  * clone HWI repo somewhere and launch `bitcoind -signer=../HWI/hwi.py`
  * check if you can see your hardware device: `bitcoin-cli enumeratesigners`
  * create wallet and auto import keys `bitcoin-cli createwallet "hww" true true "" true true true`
  * display address on device: `bitcoin-cli signerdisplayaddress ...`
  * to spend, use `send` RPC and approve transaction on device

  Prerequisites:
  - [x] #21127 load wallet flags before everything else
  - [x] #21182 remove mostly pointless BOOST_PROCESS macro

  Potentially useful followups:
  - GUI support: bitcoin-core/gui#4
  - bumpfee support
  - (automatically) verify (a subset of) keys on the device after import, through message signing

ACKs for top commit:
  laanwj:
    re-ACK f75e0c1edd

Tree-SHA512: 7db8afd54762295c1424c3f01d8c587ec256a72f34bd5256e04b21832dabd5dc212be8ab975ae3b67de75259fd569a561491945750492f417111dc7b6641e77f
2021-02-23 17:56:43 +01:00
Wladimir J. van der Laan
c263c3d7d2
Merge #19698: test: apply strict verification flags for transaction tests and assert backwards compatibility
5786a818e1 Verify that all validation flags are backward compatible (gzhao408)
b10ce9aa48 [test] check verification flags are minimal/maximal (gzhao408)
a260c22cad [test] Check for invalid flag combinations (gzhao408)
a7098a2a8d [refactor] use CheckTxScripts, TrimFlags, FillFlags (gzhao408)
7a77727b2f Apply minimal validation flags to tx_invalid tests (gzhao408)
9532591bed [test] add BADTX setting for invalid txns that fail CheckTransaction (gzhao408)
4c06ebf128 [test] fix two witness tests in invalid tests with empty vout (gzhao408)
158a0b268c Apply maximal validation flags to tx_valid tests (gzhao408)
0a76a39b63 [test] fix CSV test missing OP_ADD (gzhao408)
19db590d04 [test] remove unnecessary OP_1s from CSV and CLTV tests (gzhao408)

Pull request description:

  This uses the first 4 commits of #15045, rebased and added some comments. The diff is quite large already and I want to make it easy to review, so I'm splitting it into 2 PRs (transaction and script). Script one is WIP, I'll link it when I open it.

  Interpretation of scripts is dependent on the script verification flags passed in.
  In tests, we should always apply **maximal** verification flags when checking that a transaction is **valid**; any additional flags should invalidate the transaction. A transaction should not be valid because we forgot to include a flag, and we should apply all flags by default.
  We should apply **minimal** verification flags when asserting that a transaction is **invalid**; if verification flags are applied, removing any one of them should mean the transaction is valid.
  New verify flags must be backwards compatible; tests should check backwards compatibility and apply the new flags by default. All `tx_invalid` tests should continue to be invalid with the exact same verify flags. All `tx_valid` tests that don't pass with new flags should _explicitly_ indicate that the flags need to be excluded, and fail otherwise.

  1. Flip the meaning of `verifyFlags` in tx_valid.json to mean _excluded_ verification flags instead of included flags. Edit the test data accordingly.
  2. Trim unneeded flags from tx_invalid.json.
  3. Add check to verify that tx_valid tests have maximal flags and tx_invalid tests have minimal flags.
  4. Add checks to verify that flags are soft forks (#10699) i.e. adding any flag should only decrease the number of acceptable scripts. Test by adding/removing random flags.

ACKs for top commit:
  achow101:
    ACK 5786a818e1
  laanwj:
    ACK 5786a818e1

Tree-SHA512: 19195d8cf3299e62f47dd3443ae4a95430c5c9d497993a18ab80de9e24b1869787af972774993bf05717784879bc4592fdabaae0fddebd437963d8f3c96d9a73
2021-02-23 11:15:24 +01:00
Carl Dong
4927c9e699 validation: Remove global ::LoadGenesisBlock 2021-02-22 11:48:39 -05:00
MarcoFalke
fae216a73d
scripted-diff: Rename MakeFuzzingContext to MakeNoLogFileContext
-BEGIN VERIFY SCRIPT-
 # Rename
 sed -i -e 's/MakeFuzzingContext/MakeNoLogFileContext/g' $(git grep -l MakeFuzzingContext)
 # Bump the copyright of touched files in this scripted diff to avoid touching them again later
 ./contrib/devtools/copyright_header.py update ./src/test/fuzz/
-END VERIFY SCRIPT-
2021-02-22 10:27:22 +01:00
MarcoFalke
fa4fbec03e
scripted-diff: Rename PROVIDE_MAIN_FUNCTION -> PROVIDE_FUZZ_MAIN_FUNCTION
-BEGIN VERIFY SCRIPT-
sed -i -e 's/PROVIDE_MAIN_FUNCTION/PROVIDE_FUZZ_MAIN_FUNCTION/g' $(git grep -l PROVIDE_MAIN_FUNCTION)
-END VERIFY SCRIPT-
2021-02-22 10:27:17 +01:00
MarcoFalke
34d7030063
Merge #21202: [validation] Two small clang lock annotation improvements
25c57d6409 [doc] Add a note about where lock annotations should go. (Amiti Uttarwar)
ad5f01b960 [validation] Move the lock annotation from function definition to declaration (Amiti Uttarwar)

Pull request description:

  Based on reviewing #21188

  the first commit switches the lock annotations on `CheckInputScripts` to be on the function declaration instead of on the function definition. this ensures that all call sites are checked, not just ones that come after the definition.

  the second commit adds a note to the developer-notes section to clarify where the annotations should be applied.

ACKs for top commit:
  MarcoFalke:
    ACK 25c57d6409 🥘
  promag:
    Code review ACK 25c57d6409.

Tree-SHA512: 61b6ef856bf6c6016d535fbdd19daf57b9e59fe54a1f30d47282a071b9b9d60b2466b044ee57929e0320cb1bdef52e7a1687cacaa27031bbc43d058ffffe22ba
2021-02-22 09:47:15 +01:00
Sjors Provoost
87a97941f6
configure: add --enable-external-signer
This option replaces --with-boost-process

This prepares external signer support to be disabled by default.
It adds a configure option to enable this feature and to check
if Boost::Process is present.

This also exposes ENABLE_EXTERNAL_SIGNER to the test suite via test/config.ini
2021-02-21 16:27:10 +01:00
MarcoFalke
828bb776d2
Merge #20750: [Bundle 2/n] Prune g_chainman usage in mempool-related validation functions
e8ae1db864 style-only: Make AcceptToMemoryPool signature readable (Carl Dong)
8f5c100064 style-only: Make CheckSequenceLock signature readable (Carl Dong)
8c824819c8 validation: Use *this in CChainState::LoadMempool (Carl Dong)
0a9a24d8c7 validation: Pass in chainstate to UpdateMempoolForReorg (Carl Dong)
7142018812 validation: Pass in chainstate to CTxMemPool::removeForReorg (Carl Dong)
71734c65dc validation: Pass in chain to ::TestLockPointValidity (Carl Dong)
120aaba9ac tree-wide: Fix erroneous AcceptToMemoryPool replacements (Carl Dong)
417dafc1ee validation: Remove old AcceptToMemoryPool w/o chainstate param (Carl Dong)
3704433c4f scripted-diff: Invoke ::AcceptToMemoryPool with chainstate (Carl Dong)
229bc37b5f validation: Pass in chainstate to ::AcceptToMemoryPool (Carl Dong)
d0da7ea57a validation: Pass in chainstate to ::LoadMempool (Carl Dong)
3a205c43dc validation: Pass in chainstate to AcceptToMemoryPoolWithTime (Carl Dong)
d8a816329c validation: Add chainstate member to MemPoolAccept (Carl Dong)
4c15942b79 validation: Pass in chainstate to ::CheckSequenceLocks (Carl Dong)
577b774d0c validation: Remove old CheckFinalTx w/o chain tip param (Carl Dong)
7031cf89db scripted-diff: Invoke ::CheckFinalTx with chain tip (Carl Dong)
d015eaa550 validation: Pass in chain tip to ::CheckFinalTx (Carl Dong)
252b489c9f validation: Pass in coins tip to CheckInputsFromMempoolAndCache (Carl Dong)
73a6d2b7be validation: Pass in chainstate to IsCurrentForFeeEstimation (Carl Dong)
d1f932b0b0 validation: Pass in coins cache to ::LimitMempoolSize (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  	1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  	2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  	3. Remove `old_function`

ACKs for top commit:
  glozow:
    reACK e8ae1db864 via `git range-diff 15f0042...e8ae1db`, only change is fixing ATMP call from conflict
  MarcoFalke:
    ACK e8ae1db864 📣

Tree-SHA512: 6af50f04940a69c5c3d3796a24f32f963fa02503cdc1155cc11fff832a99172b407cd163a19793080a5af98580f051b48195b62ec4a797ba2763b4883174153d
2021-02-20 09:21:45 +01:00
MarcoFalke
f1c339d452
Merge #21211: test: Move P2WSH_OP_TRUE to shared test library
22220ef6d5 test: Move P2WSH_OP_TRUE to shared test library (MarcoFalke)

Pull request description:

  Otherwise it can't be used in other tests (unit, fuzz, bench, ...)

ACKs for top commit:
  darosior:
    ACK 22220ef6d5

Tree-SHA512: 1b636e751281291f7c21ac51c3d014f6a565144c9482974391c516228e756442b077655eda970eb8bdb12974b97855a909b2b60d518026a8d5f41aa15ec7cbc8
2021-02-19 13:14:25 +01:00
MarcoFalke
09eb46c943
Merge #21187: Net processing: Only call PushAddress() from net_processing
3e68efa615 [net] Move checks from GetLocalAddrForPeer to caller (John Newbery)
d21d2b264c [net] Change AdvertiseLocal to GetLocalAddrForPeer (John Newbery)

Pull request description:

  This is the first part of #21186. It slightly disentangles addr handling in net/net_processing by making it explicit that net_processing is responsible for pushing addr records into `vAddrToSend`.

ACKs for top commit:
  MarcoFalke:
    re-ACK 3e68efa615 🍅

Tree-SHA512: 9af50c41f5a977e2e277f24a589db38e2980b353401def5e74b108ac5f493d9b5d6b1b8bf15323a4d66321495f04bc271450fcef7aa7d1c095f051a4f8e9b15f
2021-02-19 12:58:41 +01:00
MarcoFalke
6a680a6236
Merge #21226: build: Fix fuzz binary compilation under windows
56ace907b9 Fix fuzz binary compilation under windows (Dan Benjamin)

Pull request description:

  Small change to allow the fuzz binary to compile under windows. Also removed --disable-fuzz-binary from the windows CI test. This fixes #21212.

ACKs for top commit:
  MarcoFalke:
    review ACK 56ace907b9 the best bugfixes are the ones removing code

Tree-SHA512: 6088fd955a5e511b5ca1b3eaa8469a889eb6d994c2827acac7695dac6e4e320a344b45f4015a2f279b16df0d4b23ec4df13304ae6315395ad2fe8c5b526cada4
2021-02-19 11:30:41 +01:00
Dan Benjamin
56ace907b9 Fix fuzz binary compilation under windows 2021-02-18 17:35:24 -05:00
Carl Dong
3704433c4f scripted-diff: Invoke ::AcceptToMemoryPool with chainstate
-BEGIN VERIFY SCRIPT-
find_regex='\bAcceptToMemoryPool\(' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E 's@'"$find_regex"'@\0::ChainstateActive(), @g'
-END VERIFY SCRIPT-
2021-02-18 14:49:06 -05:00
Carl Dong
4c15942b79 validation: Pass in chainstate to ::CheckSequenceLocks 2021-02-18 14:43:28 -05:00
Carl Dong
7031cf89db scripted-diff: Invoke ::CheckFinalTx with chain tip
-BEGIN VERIFY SCRIPT-
find_regex='\bCheckFinalTx\(' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E 's@'"$find_regex"'@\0::ChainActive().Tip(), @g'
-END VERIFY SCRIPT-
2021-02-18 14:43:28 -05:00
Jonas Schnelli
bedb8d88bc Avoid comparision of integers with different signs 2021-02-18 20:34:10 +01:00
Wladimir J. van der Laan
b805dbb0b9
Merge #19809: log: Prefix log messages with function name and source code location if -logsourcelocations is set
b4511e2e2e log: Prefix log messages with function name if -logsourcelocations is set (practicalswift)

Pull request description:

  Prefix log messages with function name if `-logfunctionnames` is set.

  Yes, exactly like `-logthreadnames` but for function names instead of thread names :)

  This is a small developer ergonomics improvement: I've found this to be a cheap/simple way to correlate log output and originating function.

  For me it beats the ordinary cycle of 1.) try to figure out a regexp matching the static part of the dynamic log message, 2.) `git grep -E 'Using .* MiB out of .* requested for signature cache'`, 3.) `mcedit filename.cpp` (`openemacs filename.cpp` works too!) and 4.) search for log message and scroll up to find the function name :)

  Without any logging parameters:

  ```
  $ src/bitcoind -regtest
  2020-08-25T03:29:04Z Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z block tree size = 1
  2020-08-25T03:29:04Z nBestHeight = 0
  2020-08-25T03:29:04Z Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z 0 addresses found from DNS seeds
  ```

  With `-logthreadnames` and `-logfunctionnames`:

  ```
  $ src/bitcoind -regtest -logthreadnames -logfunctionnames
  2020-08-25T03:29:04Z [init] [ReportHardwareRand] Using RdRand as an additional entropy source
  2020-08-25T03:29:04Z [init] [InitSignatureCache] Using 16 MiB out of 32/2 requested for signature cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [InitScriptExecutionCache] Using 16 MiB out of 32/2 requested for script execution cache, able to store 524288 elements
  2020-08-25T03:29:04Z [init] [LoadChainTip] Loaded best chain: hashBestChain=0fff88f13cb7b2c71f2a335e3a4fc328bf5beb436012afca590b1a11466e22ff height=0 date=2011-02-02T23:16:42Z progress=1.000000
  2020-08-25T03:29:04Z [init] [AppInitMain] block tree size = 1
  2020-08-25T03:29:04Z [init] [AppInitMain] nBestHeight = 0
  2020-08-25T03:29:04Z [loadblk] [LoadMempool] Imported mempool transactions from disk: 0 succeeded, 0 failed, 0 expired, 0 already there, 0 waiting for initial broadcast
  2020-08-25T03:29:04Z [dnsseed] [ThreadDNSAddressSeed] 0 addresses found from DNS seeds
  ```

ACKs for top commit:
  laanwj:
    Code review ACK b4511e2e2e
  MarcoFalke:
    review ACK b4511e2e2e 🌃

Tree-SHA512: d100f5364630c323f31d275259864c597f7725e462d5f4bdedcc7033ea616d7fc0d16ef1b2af557e692f4deea73c6773ccfc681589e7bf6ba970b9ec169040c7
2021-02-18 14:37:51 +01:00
John Newbery
d21d2b264c [net] Change AdvertiseLocal to GetLocalAddrForPeer
Gossiping addresses to peers is the responsibility of net processing.
Change AdvertiseLocal() in net to just return an (optional) address
for net processing to advertise. Update function name to reflect
new responsibility.
2021-02-18 09:28:06 +00:00
Samuel Dobson
db656db2ed
Merge #19136: wallet: add parent_desc to getaddressinfo
de6b389d5d tests: Test getaddressinfo parent_desc (Andrew Chow)
e4ac869a0a rpc: Add parent descriptor to getaddressinfo output (Andrew Chow)
bbe4a36152 wallet: Add GetDescriptorString to DescriptorScriptPubKeyMan (Andrew Chow)
9be1437c49 descriptors: Add ToNormalizedString and tests (Andrew Chow)

Pull request description:

  Adds `parent_desc` field to the `getaddressinfo` RPC to export a public descriptor. Using the given address, `getaddressinfo` will look up which `DescriptorScriptPubKeyMan` can be used to produce that address. It will then return the descriptor for that `DescriptorScriptPubKeyMan` in the `parent_desc` field. The descriptor will be in a normalized form where the xpub at the last hardened step is derived so that the descriptor can be imported to other wallets. Tests are added to check that the correct descriptor is being returned for the wallet's addresses and that these descriptors can be imported and used in other wallets.

  As part of this PR, a `ToNormalizedString` function is added to the descriptor classes. This really only has an effect on `BIP32PubkeyProvider`s that have hardened derivation steps. Tests are added to check that normalized descriptors are returned.

ACKs for top commit:
  Sjors:
    utACK de6b389d5d
  S3RK:
    Tested ACK de6b389
  jonatack:
    Tested ACK de6b389d5d modulo a few minor comments
  fjahr:
    Code review ACK de6b389d5d
  meshcollider:
    Tested ACK de6b389d5d

Tree-SHA512: a633e4a39f2abbd95afd7488484cfa66fdd2651dac59fe59f2b80a0940a2a4a13acf889c534a6948903d701484a2ba1218e3081feafe0b9a720dccfa9e43ca2b
2021-02-18 21:51:16 +13:00
MarcoFalke
cd66d8b1d8
Merge #20429: refactor: replace (sizeof(a)/sizeof(a[0])) with C++17 std::size
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17) (Sebastian Falbesoner)
365539c846 refactor: init vectors via std::{begin,end} to avoid pointer arithmetic (Sebastian Falbesoner)
63d4ee1968 refactor: iterate arrays via C++11 range-based for loops if idx is not needed (Sebastian Falbesoner)

Pull request description:

  This refactoring PR picks up the idea of #19626 and replaces all occurences of `sizeof(x)/sizeof(x[0])` (or `sizeof(x)/sizeof(*x)`, respectively) with the now-available C++17 [`std::size`](https://en.cppreference.com/w/cpp/iterator/size)  (as [suggested by sipa](https://github.com/bitcoin/bitcoin/pull/19626#issuecomment-666487228)), making the macro `ARRAYLEN` obsolete.

  As preparation for this, two other changes are done to eliminate `sizeof(x)/sizeof(x[0])` usage:
  * all places where arrays are iterated via an index are changed to use C++11 range-based for loops If the index' only purpose is to access the array element (as [suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/19626#discussion_r463404541)).
  * `std::vector` initializations are done via `std::begin` and `std::end` rather than using pointer arithmetic to calculate the end (also [suggested by MarcoFalke](https://github.com/bitcoin/bitcoin/pull/20429#discussion_r567418821)).

ACKs for top commit:
  practicalswift:
    cr ACK e829c9afbf: patch looks correct
  fanquake:
    ACK e829c9afbf
  MarcoFalke:
    review ACK e829c9afbf 🌩

Tree-SHA512: b01d32c04b9e04d562b7717cae00a651ec9a718645047a90761be6959e0cc2adbd67494e058fe894641076711bb09c3b47a047d0275c736f0b2218e1ce0d193d
2021-02-18 07:53:37 +01:00
Amiti Uttarwar
25c57d6409 [doc] Add a note about where lock annotations should go. 2021-02-17 15:58:23 -08:00
Amiti Uttarwar
ad5f01b960 [validation] Move the lock annotation from function definition to declaration
When the annotation is on the definition, it does not check call sites between
the declaration and the definition.
2021-02-17 15:45:11 -08:00
Wladimir J. van der Laan
372dd8da24
Merge #21110: util: remove Boost posix_time usage from GetTime*
9266f7497f util: Use std::chrono for time getters (MarcoFalke)
3c2e16be22 time: add runtime sanity check (Cory Fields)

Pull request description:

  I have a followup that should remove the last of our `boost:posix_time` usage in `ParseISO8601DateTime`, but that will likely need more cross-platform testing/discussion, so have just split them up as this change is straight forward.

ACKs for top commit:
  practicalswift:
    Tested ACK 9266f7497f
  laanwj:
    Code review ACK 9266f7497f

Tree-SHA512: 5471a60e65e9fa8ef48320743ef637f1d162724e717e0f5509118e1e5732fc0844656a9c09d3d1300eb657dcc7a1e1e67305d8c9ef959c63be67393607dd4ceb
2021-02-17 20:38:08 +01:00
MarcoFalke
22220ef6d5
test: Move P2WSH_OP_TRUE to shared test library 2021-02-17 11:36:30 +01:00
MarcoFalke
569b5ba1dc
Merge #21121: [test] Small unit test improvements, including helper to make mempool transaction
1363b6c27d [doc / util] Use comments to clarify time unit for int64_t type. (Amiti Uttarwar)
47a7a1687d [util] Introduce a SetMockTime that takes chrono time (Amiti Uttarwar)
df6a5fc1df [util] Change GetMockTime to return chrono type instead of int (Amiti Uttarwar)
a2d908e1da [test] Throw error instead of segfaulting in failure scenario (Amiti Uttarwar)
9a3bbe8fc5 [test] Introduce a unit test helper to create a valid mempool transaction. (Amiti Uttarwar)

Pull request description:

  Some miscellaneous improvements that came up when working on #21061
  - The first commit is a helper to make valid mempool transactions & submit via ATMP. Introducing in this PR, using in #21061.
  - The second commit is a small improvement in `miner_tests.cpp` that uses `BOOST_REQUIRE_EQUAL` to properly terminate the program instead of segfaulting in the failure scenario where the blocks do not include the expected number of transactions.
  - The third commit changes the function signature of `GetMockTime()` to return a chrono type.
  - The fourth & fifth commit overload `SetMockTime` to also accept chrono type, and adds documentation to indicate that the `int64_t` function signature is deprecated.

ACKs for top commit:
  vasild:
    ACK 1363b6c27d

Tree-SHA512: c72574d73668ea04ee4c33858f8de68b368780f445e05afb569aaf8564093f8112259b3afe93cf6dc2ee12a1ab5af1130ac73c16416132c1ba2851c054a67d78
2021-02-17 10:40:09 +01:00
Cory Fields
3c2e16be22
time: add runtime sanity check
std::chrono::system_clock.time_since_epoch and time_t(0) are not guaranteed
to use the Unix epoch timestamp, but in practice they almost certainly will.
Any differing behavior will be assumed to be an error, unless certain
platforms prove to consistently deviate, at which point we'll cope with it
by adding offsets.

Do a quick runtime check to verify that
time_t(0) == std::chrono::system_clock's epoch time == unix epoch.

Co-authored-by: Anthony Towns <aj@erisian.com.au>
2021-02-17 12:26:04 +08:00
fanquake
7c8e605bf4
Merge #21159: test: fix sign comparison warning in socket tests
9cc8e30125 test: fix sign comparison warning in socket tests (fanquake)

Pull request description:

  This fixes:
  ```bash
  In file included from test/sock_tests.cpp:10:
  In file included from /usr/local/include/boost/test/unit_test.hpp:18:
  In file included from /usr/local/include/boost/test/test_tools.hpp:46:
  /usr/local/include/boost/test/tools/old/impl.hpp:107:17: warning: comparison of integers of different signs: 'const long' and 'const unsigned long' [-Wsign-compare]
      return left == right;
             ~~~~ ^  ~~~~~
  ```

  which was introduced in #20788.

ACKs for top commit:
  practicalswift:
    cr ACK 9cc8e30125
  vasild:
    ACK 9cc8e30125

Tree-SHA512: 7069a4fde5cec01be03f8477fe396e53658f170efbf1d9ef3339d553bb90a2be9f4acd6b348127b14cd2f91426e0cd1fc35d2d3c9f201cf748c0cf50f47e46a5
2021-02-17 08:26:42 +08:00
Amiti Uttarwar
a2d908e1da [test] Throw error instead of segfaulting in failure scenario
If the miner code is faulty and does not include any transactions in a block,
the code segfaults when it tries to access block transactions. Instead, add a
check that safely aborts the process.
2021-02-16 12:23:00 -08:00
Amiti Uttarwar
9a3bbe8fc5 [test] Introduce a unit test helper to create a valid mempool transaction. 2021-02-16 12:23:00 -08:00
Wladimir J. van der Laan
92fee79dab
Merge #19806: validation: UTXO snapshot activation
1afc0e4aa1 doc: remove potentially confusing ChainstateManager comment (James O'Beirne)
769a1ef9fd test: Add tests with maleated snapshot data (Fabian Jahr)
4d8de04f32 tests: add snapshot activation test (James O'Beirne)
31d225274f tests: add deterministic chain generation unittest fixture (James O'Beirne)
6606a4f8c6 move-onlyish: break out CreateUTXOSnapshot from dumptxoutset (James O'Beirne)
ad949ba449 txdb: don't reset during in-memory cache resize (James O'Beirne)
f6e2da5fb7 simplify ChainstateManager::SnapshotBlockhash() return semantics (James O'Beirne)
7a6c46b37e chainparams: add allowed assumeutxo values (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This change proposes logic for activating UTXO snapshots, which is unused at the moment aside from an included unittest. There are a few moveonyish/refactoring commits to allow for halfway decent unittests.

  Basic structure is included for specifying and checking the assumeutxo hash values used to validate activated snapshots. Initially I had specified a few height/hash pairs for mainnet in this change, but because of the security-critical nature of those parameters, I figured it was better to leave their inclusion to a future PR that includes only that change - my intent being that reviewers will be more likely to verify those parameters firsthand in a dedicated PR.

  Aside from that and the snapshot activation logic, there are a few related changes:

  - ~~allow caching the `nChainTx` value in the CCoinsViewDB; this is set during snapshot activation. Because we don't necessarily have access to the full chain at the time of snapshot load, this value is communicated through the snapshot metadata and must be cached within the chainstate to survive restarts.~~
  - break out `CreateUTXOSnapshot()` from dumptxoutset. This is essentially a move-only change to allow the reuse of snapshot creation logic from within unittests.
  - ...and a few other misc. changes that are solely related to unittests.

  The move-onlyish commit is most easily reviewed with `--color-moved=zebra`.

ACKs for top commit:
  fjahr:
    Code review ACK 1afc0e4aa1
  laanwj:
    Code review ACK 1afc0e4aa1

Tree-SHA512: a4e4f0698f00a53ec298b5e8b7ef1c9fdf0185f95139d1b1f63cfdf6cbbd6d17b8c6e51bbf1de2e5f1a946bf49f8466232698ef55acce5a012c80b067da366ea
2021-02-16 19:23:06 +01:00
fanquake
9bbf08bf98
Merge #20721: Net: Move ping data to net_processing
a5e15ae45c scripted-diff: rename ping members (John Newbery)
45dcf22661 [net processing] Move ping data fields to net processing (John Newbery)
dd2646d12c [net processing] Move ping timeout logic to net processing (John Newbery)
0b43b81f69 [net processing] Move send ping message logic into function (John Newbery)
1a07600b4b [net] Add RunInactivityChecks() (John Newbery)
f8b3058992 [net processing] Add Peer& arg to MaybeDiscourageAndDisconnect() (John Newbery)

Pull request description:

  This continues the work of moving application layer data into net_processing, by moving all ping data into the new Peer object added in #19607.

  For motivation, see #19398.

ACKs for top commit:
  glozow:
    reACK a5e15ae45c
  MarcoFalke:
    review ACK a5e15ae45c 🥉
  amitiuttarwar:
    ACK a5e15ae45c

Tree-SHA512: fb84241613d6a6e1f2832fa5378030b5877a02e8308188f57ab545a6eaf2ab731a93abb7dcd3a7f7285bb66700f938096378a8e90cd6a3e6f3309f81d85a344e
2021-02-16 18:48:30 +08:00
MarcoFalke
b55dc3ad84
Merge #21185: fuzz: Remove expensive and redundant muhash from crypto fuzz target
ffff84a9cb fuzz: Remove expensive and redundant muhash from crypto fuzz target (MarcoFalke)

Pull request description:

  Remove because it is redundant with `src/test/fuzz/muhash.cpp` and incredibly expensive

ACKs for top commit:
  practicalswift:
    Tested ACK ffff84a9cb

Tree-SHA512: c91ea2406db857127c789b9cdeb714a719d88b54132e9cef74fffd229532d874b6c043353793ec687504b5784afc74995f8982243d41f976b63d57454a5ed339
2021-02-16 07:54:28 +01:00
John Newbery
a5e15ae45c scripted-diff: rename ping members
-BEGIN VERIFY SCRIPT-
sed -i 's/fPingQueued/m_ping_queued/g' src/net_processing.cpp
sed -i 's/nMinPingUsecTime/m_min_ping_time/g' src/net.* src/net_processing.cpp src/test/net_tests.cpp
sed -i 's/nPingNonceSent/m_ping_nonce_sent/g' src/net_processing.cpp
sed -i 's/nPingUsecTime/m_last_ping_time/g' src/net.*
-END VERIFY SCRIPT-
2021-02-15 16:15:51 +00:00
MarcoFalke
ffff84a9cb
fuzz: Remove expensive and redundant muhash from crypto fuzz target 2021-02-15 14:39:08 +01:00
Jon Atack
2ee4a7a9ec
net: remove CNode::m_inbound_onion defaults for explicitness
and to allow the compiler to warn if uninitialized in the ctor
or omitted in the caller.
2021-02-12 22:32:08 +01:00
Jon Atack
24bda56c29
net: make CNode::m_inbound_onion public, drop getter, update tests 2021-02-12 22:23:15 +01:00
Fabian Jahr
769a1ef9fd
test: Add tests with maleated snapshot data 2021-02-12 07:53:40 -06:00
James O'Beirne
4d8de04f32
tests: add snapshot activation test 2021-02-12 07:53:37 -06:00
James O'Beirne
31d225274f
tests: add deterministic chain generation unittest fixture 2021-02-12 07:53:36 -06:00
James O'Beirne
f6e2da5fb7
simplify ChainstateManager::SnapshotBlockhash() return semantics
Don't return null snapshotblockhash values to avoid caller complexity/confusion.
2021-02-12 07:53:29 -06:00
James O'Beirne
7a6c46b37e
chainparams: add allowed assumeutxo values
Values for mainnet and testnet will be specified in a follow-up PR that can be
scrutinized accordingly. This structure is required for use in snapshot activation
logic.
2021-02-12 07:53:22 -06:00
Wladimir J. van der Laan
9996b1806a
Merge #21064: refactor: use std::shared_mutex & remove Boost Thread
060a2a64d4 ci: remove boost thread installation (fanquake)
06e1d7d81d build: don't build or use Boost Thread (fanquake)
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)

Pull request description:

  This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).

  Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.

  Two other points:

  [Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
  > It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.

  Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
  * The version of Boost.
  * The platform you're building for.
  * Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
  * Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.

  A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.

  With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.

  Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
  Also related to #21022 - pthread sanity checking.

ACKs for top commit:
  laanwj:
    Code review ACK 060a2a64d4
  vasild:
    ACK 060a2a64d4

Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
2021-02-12 11:39:36 +01:00
fanquake
9cc8e30125
test: fix sign comparison warning in socket tests
This fixes:
```bash
In file included from test/sock_tests.cpp:10:
In file included from /usr/local/include/boost/test/unit_test.hpp:18:
In file included from /usr/local/include/boost/test/test_tools.hpp:46:
/usr/local/include/boost/test/tools/old/impl.hpp:107:17: warning: comparison of integers of different signs: 'const long' and 'const unsigned long' [-Wsign-compare]
    return left == right;
           ~~~~ ^  ~~~~~
```

which was introduced in #20788.
2021-02-12 10:12:00 +08:00
MarcoFalke
0ad4656790
Merge #20370: fuzz: version handshake
fabce459bb fuzz: version handshake (MarcoFalke)

Pull request description:

  Not fuzzing the version handshake will limit fuzz coverage

ACKs for top commit:
  practicalswift:
    cr ACK fabce459bb: patch looks very much correct

Tree-SHA512: 4091d27d39edee781d033e471b352084bb54df250d0890e4821a325926a44dff9b26a2614d67dd0529f73bd366b075d7a0a1a570c2837de286a1b93a59a8fb91
2021-02-11 17:25:33 +01:00
MarcoFalke
8e1913ae02
Merge #21062: refactor: return MempoolAcceptResult from ATMP
53e716ea11 [refactor] improve style for touched code (gzhao408)
174cb5330a [refactor] const ATMPArgs and non-const Workspace (gzhao408)
f82baf0762 [refactor] return MempoolAcceptResult (gzhao408)
9db10a5506 [refactor] clean up logic in testmempoolaccept (gzhao408)

Pull request description:

  This is the first 4 commits of #20833, and does refactoring only. It should be relatively simple to review, and offers a few nice things:
  - It makes accessing values that don't make sense (e.g. fee) when the tx is invalid an error.
  - Returning `MempoolAcceptResult` from ATMP makes the interface cleaner. The caller can get a const instead of passing in a mutable "out" param.
  - We don't have to be iterating through a bunch of lists for package validation, we can just return a `std::vector<MempoolAcceptResult>`.
  - We don't have to refactor all ATMP call sites again if/when we want to return more stuff from it.

ACKs for top commit:
  MarcoFalke:
    ACK 53e716ea11 💿
  jnewbery:
    Code review ACK 53e716ea11
  ariard:
    Code Review ACK 53e716e, I did tweak a bit the touched paths to see if we had good test coverage. Didn't find holes.

Tree-SHA512: fa6ec324a08ad9e6e55948615cda324cba176255708bf0a0a0f37cedb7a75311aa334ac6f223be7d8df3c7379502b1081102b9589f9a9afa1713ad3d9ab3c24f
2021-02-11 14:45:41 +01:00
Wladimir J. van der Laan
a1be08405d
Merge #20788: net: add RAII socket and use it instead of bare SOCKET
615ba0eb96 test: add Sock unit tests (Vasil Dimov)
7bd21ce1ef style: rename hSocket to sock (Vasil Dimov)
04ae846904 net: use Sock in InterruptibleRecv() and Socks5() (Vasil Dimov)
ba9d73268f net: add RAII socket and use it instead of bare SOCKET (Vasil Dimov)
dec9b5e850 net: move CloseSocket() from netbase to util/sock (Vasil Dimov)
aa17a44551 net: move MillisToTimeval() from netbase to util/time (Vasil Dimov)

Pull request description:

  Introduce a class to manage the lifetime of a socket - when the object
  that contains the socket goes out of scope, the underlying socket will
  be closed.

  In addition, the new `Sock` class has a `Send()`, `Recv()` and `Wait()`
  methods that can be overridden by unit tests to mock the socket
  operations.

  The `Wait()` method also hides the
  `#ifdef USE_POLL poll() #else select() #endif` technique from higher
  level code.

ACKs for top commit:
  laanwj:
    Re-ACK 615ba0eb96
  jonatack:
    re-ACK 615ba0eb96

Tree-SHA512: 3003e6bc0259295ca0265ccdeb1522ee25b4abe66d32e6ceaa51b55e0a999df7ddee765f86ce558a788c1953ee2009bfa149b09d494593f7d799c0d7d930bee8
2021-02-11 14:07:33 +01:00
MarcoFalke
fabce459bb
fuzz: version handshake 2021-02-11 12:45:57 +01:00
MarcoFalke
e498aeffbe
Merge #20211: Use -Wswitch for TxoutType where possible
fa650ca7f1 Use -Wswitch for TxoutType where possible (MarcoFalke)
fa59e0b5bd test: Add missing script_standard_Solver_success cases (MarcoFalke)

Pull request description:

  This removes unused `default:` cases for all `switch` statements on `TxoutType` and adds the cases (`MULTISIG`, `NULL_DATA`, `NONSTANDARD`) to `ExtractDestination` for clarity.

  Also, the compiler is now able to use `-Wswitch`.

ACKs for top commit:
  practicalswift:
    cr ACK fa650ca7f1: patch looks correct and `assert(false);` is better than UB :)
  hebasto:
    ACK fa650ca7f1, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 282458b6523bd8923a0c0f5c423d1db2dce2a2d1b1d1dae455415c6fc995bb41ce82c1f9b0a1c0dcc6d874d171e04c30eca585f147582f52c7048c140358630a
2021-02-11 11:48:12 +01:00
Vasil Dimov
615ba0eb96
test: add Sock unit tests 2021-02-11 10:44:53 +01:00
MarcoFalke
fa4bc897fc
fuzz: Fail if message type is not fuzzed 2021-02-11 09:40:41 +01:00
MarcoFalke
faefed8cd5
fuzz: Count message type fuzzers before main() 2021-02-11 09:39:32 +01:00
MarcoFalke
b69eab9025
Merge #20663: fuzz: Hide script_assets_test_minimizer
fac726b1b8 doc: Fixup docs in fuzz/script_assets_test_minimizer.cpp (MarcoFalke)
fafca47adc fuzz: Hide script_assets_test_minimizer (MarcoFalke)

Pull request description:

  This is not an actual fuzz target. It is a hack to exploit the built-in capability of fuzz engines to measure coverage.

ACKs for top commit:
  practicalswift:
    cr ACK fac726b1b8: patch looks correct and touches only `src/test/fuzz/`

Tree-SHA512: 0652dd8d9e95746b0906be4044467435d8204a34a30366ae9bdb75b9cb0788d429db7cedf2760fd543565d9d4f7ee206873ed10a29dd715a792a26337f65b53c
2021-02-11 07:54:22 +01:00
Wladimir J. van der Laan
d202054675
Merge #21052: refactor: Replace fs::unique_path with GetUniquePath(path) calls
1bca2aa694 Introduce GetUniquePath(base) helper method to replace boost::filesystem::unique_path() which is not available in std::filesystem. (Kiminuo)

Pull request description:

  This PR makes it easier in #20744 to remove our dependency on the `boost::filesystem::unique_path()` function which does not have a direct equivalent in C++17.

  This PR attempts to re-implement `boost::filesystem::unique_path()` as `GetUniquePath(path)` but the implementations are not meant to be the same.

  Note:

  * Boost 1.75.0 implementation of `unique_path`: 9cab675b71/src/unique_path.cpp (L235)

  * In the previous implementation, I attempted to add:
      ```cpp
      fs::path GetUniquePath(const fs::path& base)
      {
          FastRandomContext rnd;
          fs::path tmpFile = base / HexStr(rnd.randbytes(8));
          return tmpFile;
      }
      ```

      to `fs.cpp` but this leads to a circular dependency: "fs -> random -> logging -> fs". That is why the modified implementation adds a new file.

ACKs for top commit:
  laanwj:
    Code review ACK 1bca2aa694
  ryanofsky:
    Code review ACK 1bca2aa694. It's a simple change and extra test coverage is nice

Tree-SHA512: f324bdf0e254160c616b5033c3ece33d87db23eb0135acee99346ade7b5cf0d30f3ceefe359a25a8e9b53ba8e4419f459c2bdd369e10fc0152ce95031d1f221c
2021-02-09 22:22:13 +01:00
gzhao408
f82baf0762 [refactor] return MempoolAcceptResult
This creates a cleaner interface with ATMP, allows us to make results const,
and makes accessing values that don't make sense (e.g. fee when tx is
invalid) an error.
2021-02-09 07:01:52 -08:00
Hennadii Stepanov
723eb4326b
test: Fix Windows cross build 2021-02-08 15:17:48 +02:00
MarcoFalke
fac726b1b8
doc: Fixup docs in fuzz/script_assets_test_minimizer.cpp 2021-02-08 10:12:22 +01:00
MarcoFalke
fafca47adc
fuzz: Hide script_assets_test_minimizer
Can be reviewed with --ignore-all-space
2021-02-08 10:11:59 +01:00
Wladimir J. van der Laan
173cf31299
Merge #20839: fuzz: Avoid extraneous copy of input data, using Span<>
faf7d7418c fuzz: Avoid extraneous copy of input data, using Span<> (MarcoFalke)

Pull request description:

  Seeing speedup here in the fuzz framework part (non-fuzz-target part). Speedup is only visible for input data larger than 100kB.

ACKs for top commit:
  practicalswift:
    cr ACK faf7d7418c: patch looks correct :)
  laanwj:
    Code review ACK faf7d7418c

Tree-SHA512: 41af7118846e0dfee237a6d5269a6c7cfbc775d7bd1cc2a85814cb60f6c2b37fe7fd35f1a788d4f08e6e0202c48b71054b67d2931160c445c79fc59e5347dadf
2021-02-05 14:57:08 +01:00
Kiminuo
1bca2aa694 Introduce GetUniquePath(base) helper method to replace boost::filesystem::unique_path() which is not available in std::filesystem. 2021-02-04 11:38:09 +01:00
MarcoFalke
4e946ebcf1
Merge #20715: util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet
fa61b9d1a6 util: Add ArgsManager::GetCommand() and use it in bitcoin-wallet (MarcoFalke)
7777105a24 refactor: Move all command dependend checks to ExecuteWalletToolFunc (MarcoFalke)
fa06bce4ac test: Add tests (MarcoFalke)
fac05ccdad wallet: [refactor] Pass ArgsManager to WalletAppInit (MarcoFalke)

Pull request description:

  This not only moves the parsing responsibility out from the wallet tool, but it also makes it easier to implement bitcoin-util #19937

  Fixes: #20902

ACKs for top commit:
  ajtowns:
    ACK fa61b9d1a6
  fjahr:
    Code review ACK fa61b9d1a6

Tree-SHA512: 79622b806e8bf9dcd0dc24a8a6687345710df57720992e83a41cd8d6762a6dc112044ebc58fcf6e8fbf45de29a79b04873c5b8c2494a1eaaf902a2884703e47b
2021-02-04 09:12:05 +01:00
MarcoFalke
faf7d7418c
fuzz: Avoid extraneous copy of input data, using Span<> 2021-02-03 19:30:14 +01:00
gzhao408
5786a818e1 Verify that all validation flags are backward compatible
See #10699, i.e. adding a flag should always reduce the
number of acceptable scripts.

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 09:17:58 -08:00
gzhao408
b10ce9aa48 [test] check verification flags are minimal/maximal
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 09:17:58 -08:00
gzhao408
a260c22cad [test] Check for invalid flag combinations 2021-02-02 09:17:58 -08:00
gzhao408
a7098a2a8d [refactor] use CheckTxScripts, TrimFlags, FillFlags
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
7a77727b2f Apply minimal validation flags to tx_invalid tests
- Reduce the number of validation flags used, to a minimally required set to fail a test

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
9532591bed [test] add BADTX setting for invalid txns that fail CheckTransaction
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
4c06ebf128 [test] fix two witness tests in invalid tests with empty vout
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
158a0b268c Apply maximal validation flags to tx_valid tests
- Apply all validation flags by default
- Invert the meaning of verifyFlags as flags being excluded

Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
0a76a39b63 [test] fix CSV test missing OP_ADD
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
gzhao408
19db590d04 [test] remove unnecessary OP_1s from CSV and CLTV tests
Co-authored-by: Johnson Lau <jl2012@xbt.hk>
2021-02-02 08:58:55 -08:00
fanquake
8e55981ef8
refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests
Co-authored-by: MarcoFalke falke.marco@gmail.com
Co-authored-by: sinetek pitwuu@gmail.com
2021-02-02 12:35:40 +08:00
Wladimir J. van der Laan
2c0fc856a6
Merge #20464: refactor: Treat CDataStream bytes as uint8_t
fa29272459 Remove redundant MakeUCharSpan wrappers (MarcoFalke)
faf4aa2f47 Remove CDataStream::Init in favor of C++11 member initialization (MarcoFalke)
fada14b948 Treat CDataStream bytes as uint8_t (MarcoFalke)
fa8bdb048e refactor: Drop CDataStream constructors in favor of one taking a Span of bytes (MarcoFalke)
faa96f841f Remove unused CDataStream methods (MarcoFalke)

Pull request description:

  Using `uint8_t` for raw bytes has a style benefit:
  * The signedness is clear from reading the code, as it does not depend on the architecture

  Other clean-ups in this pull include:
  * Remove unused methods
  * Constructor is simplified with `Span`
  * Remove `Init()` member in favor of C++11 member initialization

ACKs for top commit:
  laanwj:
    code review ACK fa29272459
  theStack:
    ACK fa29272459 🍾

Tree-SHA512: 931ee28bd99843d7e894b48e90e1187ffb0278677c267044b3c0c255069d9bbd9298ab2e539b1002a30b543d240450eaec718ef4ee95a7fd4be0a295e926343f
2021-02-01 15:17:28 +01:00
Wladimir J. van der Laan
d0d256536c
Merge #21016: refactor: remove boost::thread_group usage
dc8be12510 refactor: remove boost::thread_group usage (fanquake)

Pull request description:

  Post #18710, there isn't much left using `boost::thread_group`, so should just be able to replace it with the standard library. This also removes the last use of `boost::thread_interrupted`.

  After this change, last piece of Boost Thread we'd be using is `boost::shared_mutex`. See the commentary [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) as to why it may be non-trivial to swap that for `std::shared_mutex` in the near future.

  Closes #17307

ACKs for top commit:
  laanwj:
    Code review re-ACK dc8be12510
  MarcoFalke:
    review ACK dc8be12510 🔁
  jonatack:
    Non-expert code review ACK dc8be12510, also checked range-diff since last review and that local debug build is clean with gcc 10.2.1-6 on Debian

Tree-SHA512: 5510e2d760cce824234207dc86b1551ca8f21cbf3a2ce753c0254a0d03ffd83c94e449aec202fb7bd76e6fc64df783a6b70a736b0add9ece3734bb9c8ce8fc2f
2021-02-01 13:27:28 +01:00
Wladimir J. van der Laan
44f4bcd302
Merge #20749: [Bundle 1/n] Prune g_chainman usage related to ::LookupBlockIndex
67c9a83df1 style-only: Remove redundant sentence in ActivateBestChain comment (Carl Dong)
b8e95658d5 style-only: Make TestBlockValidity signature readable (Carl Dong)
0cdad75390 validation: Use accessible chainstate in ChainstateManager::ProcessNewBlock (Carl Dong)
ea4fed9021 validation: Use existing chainstate in ChainstateManager::ProcessNewBlockHeaders (Carl Dong)
e0dc305727 validation: Move LoadExternalBlockFile to CChainState (Carl Dong)
5f8cd7b3a5 validation: Remove global ::ActivateBestChain (Carl Dong)
2a696472a1 validation: Pass in chainstate to ::NotifyHeaderTip (Carl Dong)
9c300cc8b3 validation: Pass in chainstate to TestBlockValidity (Carl Dong)
0e17c833cd validation: Make CChainState.m_blockman public (Carl Dong)
d363d06bf7 validation: Pass in blockman to ContextualCheckBlockHeader (Carl Dong)
f11d11600d validation: Move GetLastCheckpoint to BlockManager (Carl Dong)
e4b95eefbc validation: Move GetSpendHeight to BlockManager (Carl Dong)
b026e318c3 validation: Move FindForkInGlobalIndex to BlockManager (Carl Dong)
3664a150ac validation: Remove global LookupBlockIndex (Carl Dong)
eae54e6e60 scripted-diff: Use BlockManager::LookupBlockIndex (Carl Dong)
15d20f40e1 validation: Move LookupBlockIndex to BlockManager (Carl Dong)
f92dc6557a validation: Guard the active_chainstate with cs_main (Carl Dong)

Pull request description:

  Overall PR: #20158 (tree-wide: De-globalize ChainstateManager)

  Note to reviewers:
  1. This bundle may _apparently_ introduce usage of `g_chainman` or `::Chain(state|)Active()` globals, but these are resolved later on in the overall PR. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  2. There may be seemingly obvious local references to `ChainstateManager` or other validation objects which are not being used in callers of the current function in question, this is done intentionally to **_keep each commit centered around one function/method_** to ease review and to make the overall change systematic. We don't assume anything about our callers. Rest assured that once we are considering that particular caller in later commits, we will use the obvious local references. [Commits of overall PR](https://github.com/bitcoin/bitcoin/pull/20158/commits)
  3. When changing a function/method that has many callers (e.g. `LookupBlockIndex` with 55 callers), it is sometimes easier (and less error-prone) to use a scripted-diff. When doing so, there will be 3 commits in sequence so that every commit compiles like so:
  1. Add `new_function`, make `old_function` a wrapper of `new_function`, divert all calls to `old_function` to `new_function` **in the local module only**
  2. Scripted-diff to divert all calls to `old_function` to `new_function` **in the rest of the codebase**
  3. Remove `old_function`

ACKs for top commit:
  jnewbery:
    utACK 67c9a83df1
  laanwj:
    re-ACK 67c9a83df1
  ryanofsky:
    Code review ACK 67c9a83df1. Changes since last review:

Tree-SHA512: 8744aba2dd57a40cd2fedca809b0fe24d771bc60da1bffde89601999384aa0df428057a86644a3f72fbeedbc8b04db6c4fd264ea0db2e73c279e5acc6d056cbf
2021-02-01 13:09:46 +01:00
Sebastian Falbesoner
e829c9afbf refactor: replace sizeof(a)/sizeof(a[0]) by std::size (C++17)
Removes the macro ARRAYLEN and also substitutes all other uses of the same
"sizeof(a)/sizeof(a[0])" pattern by std::size, available since C++17.
2021-01-31 17:35:16 +01:00
Sebastian Falbesoner
63d4ee1968 refactor: iterate arrays via C++11 range-based for loops if idx is not needed 2021-01-31 17:07:46 +01:00
practicalswift
dee2d6fbf9 fuzz: Avoid designated initialization (C++20) in fuzz tests 2021-01-30 08:22:32 +00:00
Russell Yanofsky
6158a6d397 refactor: Replace JSONRPCRequest fHelp field with mode field
No change in behavior
2021-01-29 18:09:46 -05:00
fanquake
dc8be12510
refactor: remove boost::thread_group usage 2021-01-29 15:39:44 +08:00
Carl Dong
e0dc305727 validation: Move LoadExternalBlockFile to CChainState
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

LoadExternalBlockFile mainly acts on CChainState.
2021-01-28 14:15:26 -05:00
Carl Dong
5f8cd7b3a5 validation: Remove global ::ActivateBestChain
Instead use CChainState::ActivateBestChain, which is what the global one
calls anyway.
2021-01-28 14:15:26 -05:00
Carl Dong
eae54e6e60 scripted-diff: Use BlockManager::LookupBlockIndex
[META] In a previous commit, we moved ::LookupBlockIndex to become a
       member function of BlockManager. This commit is split out from
       that one since it can be expressed nicely as a scripted-diff.

-BEGIN VERIFY SCRIPT-
find_regex='LookupBlockIndex' \
    && git grep -l -E "$find_regex" -- src \
        | grep -v '^src/validation\.\(cpp\|h\)$' \
        | xargs sed -i -E "s@${find_regex}@g_chainman.m_blockman.LookupBlockIndex@g"
-END VERIFY SCRIPT-
2021-01-28 14:15:26 -05:00
MarcoFalke
4d5eaf7a90
Merge #20995: fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION
fad3d7625a fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION (MarcoFalke)
fa99e33aeb fuzz: move-only FillNode implementation to cpp file (MarcoFalke)

Pull request description:

  This fixes a fuzz bug introduced in #20881. Previously the nodes in the fuzz tests had their version initialized to a constant (`PROTOCOL_VERSION`). After #20881, the nodes have their version initialized to an arbitrary signed integer. This is problematic for several reasons:

  * Both `nVersion` and `m_greatest_common_version` may be initialized to `0`. If a `version` message is processed, this leads to a crash, because `m_greatest_common_version` must be `INIT_PROTO_VERSION` while the `version` message is processed. See #20138
  * The "valid" range for `nVersion` is `[MIN_PEER_PROTO_VERSION, std::numeric_limits<int32_t>::max()]` (see check in net_processing)
  * The "valid" range for `m_greatest_common_version` is `std::min(nVersion, PROTOCOL_VERSION)` (see net_processing)

  Fix all issues by initializing `nVersion` and `m_greatest_common_version` to their valid ranges.

  -----

  The crashers, if someone wants to try this at home:

  ```
  ( echo 'dmVyc2lvbgAWFhYWFhYWFhYWFhYWFhYWFhYWFhZp/29uAPX//xYWFhYWFhYWFhYWFhYWFhYWFhYW
  FhYWFhYWaW9uAOr1//8WFhYWFha0ZXJzaW9uAPX//wAAAAAAABAAAAAAAAAAAAC0ZXJzaW9uAPX/
  /wBPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08AAAAAABAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
  AAAAAAAAAAAAAAB2ZXJzaW9uAACDJIO9vXYKAAAAAAAAAAAAAAAAAAAAAAB2ZfS1qmu1qhUVFWs=' | base64 --decode > /tmp/a ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/a
  ```
  ```
  ( echo 'dmVyc2lvbgD//wAhTmiqN///NDcAAACENDL/iv//8DYAAHL///////79/RtcAJqamhqa/QEAAAD/
  ///+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZyq8SaGVhZGVycwAAAAD/NDcAAACENDL/iv//
  8DYAAHL///////79/RtcAJqamhqa/QEAAAD////+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZ
  yq8SaGVhZGVycwAAAADPAQAAACAGIm5GERoLWS1wb3J061u/KMNPOkwFXqZ///b5IgIAAD+5ubkb
  XD5hZGRyAJqamhqasP0BAAAAAAAAAP0BAAAAIf39/R0dHQAAAAAAMgAA///7//+gXqZ///b5IgIA
  AD+5ubm5ubm5AAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAFgAAAAAAAAAAAAlBmv39/f1/f39B
  f39hZGRyAG5vAACaLgAdGzY2zwEAAAAgBiJuRhEaC1ktcG9ydOtbvyjDTzpMBV6mf//2+SICAAA/
  ubm5G1w+YWRkcgCampoamrD9AQAAAAAAAAD9AQAAACH9/f0dHR0AAAAAADIAAP//+///oF6mf//2
  +SICAAA/ubm5ubm5uQAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAABYAAAAAAAAAAAAJQZr9/f39
  f39/QX9/YWRkcgBubwAAmi4AHRs2NjY2NjY2NjYCAgI2NgIA/f39/f39Nv39/TUmABxc' | base64 --decode > /tmp/b ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/b
  ```

ACKs for top commit:
  practicalswift:
    cr ACK fad3d7625a

Tree-SHA512: ea64ee99b94d8e619e3949d2d21252c1236412c0e40f44f2b73595ca70cd2da0bdab005fb1a54f65fb291e7b07fdd33577ce4a3a078ca933246b511ebcb0e52a
2021-01-28 15:13:25 +01:00