5a9ee0869b tests: Add fuzzing harness for node eviction logic (practicalswift)
Pull request description:
Add fuzzing harness for node eviction logic.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
cr ACK 5a9ee0869b
Tree-SHA512: c2401d22134867e23dab1ba94ae7ef36fdf52aa0588fdc4705d9cb765ddf979fd775fdf153ce2359f1bc1787cf60bf0ebcd47c7aa29c672e6a253fa58cac292d
c119ba3c9b [doc] clarify getdata limit after #14897 (Michael Polzer)
Pull request description:
GETDATA is limited to blocks and transactions now and can't be used for other non-block data
ACKs for top commit:
laanwj:
ACK c119ba3c9b
theStack:
ACK c119ba3c9b
benthecarman:
ACK c119ba3c9b
Tree-SHA512: d6e9c109bcce4ef004ec83a9ec591163279476524dec97ed5f5c34e322dca35af66a168f0878ff972bbcec79d81623903f3619fedf8f88cdced3f3f66a779173
fa511042b0 doc: [test] Remove outdated comment in fuzz runner (MarcoFalke)
Pull request description:
All folders are soft-created with `os.makedirs`
ACKs for top commit:
RiccardoMasutti:
ACK fa51104
Tree-SHA512: 4051688946a205a981bbb005300fe3263495ead26591042b38ae44f4297c7689a613b560052fb5405a62054734d2599cfb0554a37c7b7369fb3a3636743d04a8
fa26303286 fuzz: Check that NULL_DATA is unspendable (MarcoFalke)
Pull request description:
* Every script of type NULL_DATA must be unspendable
* The only know types of unspendable scripts are NULL_DATA and certain NONSTANDARD scripts
ACKs for top commit:
sipa:
utACK fa26303286
Tree-SHA512: 8297fbacf32b4868b12accc1c052d352d02d96540a1fc883de9d04a3df8734116deecc33046495c9a3af6d79fec7f8d63afbfa5e401a2ca8d7c70f0f13735c0d
87fe104537 depends: Use more legible qmake commands in qt package (Hennadii Stepanov)
bf35a8da6e depends: Do not set build_subdir for qt package (Hennadii Stepanov)
Pull request description:
Rather than using `cd` to jump all over the place, perform all `(q)make` commands from the top level directory.
Looking at bash like `cd ../../../..` gives me a headache.
Credits to **fanquake**.
This PR is an alternative to #20504 that works without any additional [non-trivial hack](https://github.com/bitcoin/bitcoin/pull/20504#issuecomment-734730336).
ACKs for top commit:
promag:
Tested ACK 87fe104537.
fanquake:
ACK 87fe104537
Tree-SHA512: 1d2a13b5358fc7406c5363ddd62fd363dbc0ec5ace68946e4d3e6e8620419afaa64ef2837488aaed226174e01e8897495085540f7126b80f8b2372d21b5b29f9
66d012ad7f test: RPC: getblock fee calculations (Elliott Jin)
bf7d6e31b1 RPC: getblock: tx fee calculation for verbosity 2 via Undo data (Elliott Jin)
Pull request description:
This change is progress towards #18771 . It adapts the fee calculation part of #16083 and addresses some feedback. The additional "verbosity level 3" features are planned for a future PR.
**Original PR description:**
> Using block undo data (like in #14802) we can now show fee information for each transaction in a block without the need for additional -txindex and/or a ton of costly lookups. For a start we'll add transaction fee information to getblock verbosity level 2. This comes at a negligible speed penalty (<1%).
ACKs for top commit:
luke-jr:
tACK 66d012ad7f
fjahr:
tACK 66d012ad7f
MarcoFalke:
review ACK 66d012ad7f 🗜
Tree-SHA512: be1fe4b866946a8dc36427f7dc72a20e10860e320a28fa49bc85bd2a93a0d699768179be29fa52e18b2ed8505d3ec272e586753ef2239b4230e0aefd233acaa2
812baaa1f8 Switch to BIP341's suggested scheme for outputs without script (Pieter Wuille)
Pull request description:
BIP341 suggests using Hash<sub>TapTweak</sub>(pubkey) to derive the tweak in case of key-only outputs. The functional test framework currently uses Hash<sub>TapTweak</sub>(pubkey || 0x00...00) instead. Change this.
There is no technical reason to prefer one over the other, but in case someone looks at it for inspiration, it's better to be consistent with the BIP.
ACKs for top commit:
laanwj:
ACK 812baaa1f8
instagibbs:
ACK 812baaa1f8
Tree-SHA512: 02576c38776ec786255f49d7edecdb1ed8a9dcf0f547d58c23099588b4c3296edf279b103a6eb80e0f07d3c5ee9743f67d152f5244fd63adc6613b004f6969ed
176325a5a4 [net processing] Remove dropmessagestest (John Newbery)
Pull request description:
-dropmessagestest is a command line option that causes 1 in n received
messages to be dropped. The Bitcoin P2P protocol is stateful and in
general cannot handle messages being dropped. Dropped
version/verack/ping/pong messages will cause the connection to time out
and be torn down. Other dropped messages may also cause the peer to
believe that the peer has stalled and tear down the connection.
It seems difficult to uncover any actual issues with -dropmessagestest,
and any coverage that could be generated would probably be easier to
trigger with fuzz testing.
ACKs for top commit:
MarcoFalke:
cr ACK 176325a5a4
practicalswift:
cr ACK 176325a5a4
dhruv:
cr ACK 176325a
amitiuttarwar:
ACK 176325a5a4
Tree-SHA512: bd582e5e8c9eb272a5d8ec01ff07c36c0033fbb84c30d1c72c87a7a6c7290021dcaf7bf549179a8b95aeb4f7243158d5593bc7fcf1ec16213782e470fe36bb89
fa957f8dc9 test: Add race:SendZmqMessage tsan suppression (MarcoFalke)
Pull request description:
Add suppression for `race:SendZmqMessage`, which isn't covered by the existing `zmq::*` suppression
Fixes#20618
ACKs for top commit:
hebasto:
re-ACK fa957f8dc9, as my previous comment is not directly related to this pull changes.
Tree-SHA512: 8642a8b79bbfa4bee89042b66e528f27fd78c5e84a33023df440662e9114e31445fd7b04940f44b11fa4ab7438d346385a21816289c818cce9958a9b16730452
3e1571285f Update TSan suppressions (Hennadii Stepanov)
Pull request description:
It seems possible now to drop some TSan suppressions.
Top commit has no ACKs.
Tree-SHA512: 94518fd2f3a7168b2989424de0696e42c8f509b833aafbc7e75f4c1180a0b8d9a47f43c50d06b03b26a924643afe86274b2062c9d456c17a68576d19566ed66f
-dropmessagestest is a command line option that causes 1 in n received
messages to be dropped. The Bitcoin P2P protocol is stateful and in
general cannot handle messages being dropped. Dropped
version/verack/ping/pong messages will cause the connection to time out
and be torn down. Other dropped messages may also cause the peer to
believe that the peer has stalled and tear down the connection.
It seems difficult to uncover any actual issues with -dropmessagestest,
and any coverage that could be generated would probably be easier to
trigger with fuzz testing.
d71e29e3e8 qa: Correct epoll_ctl data race suppression (Hennadii Stepanov)
Pull request description:
Fixup of #20218. Comments must start from the beginning of the line.
ACKs for top commit:
MarcoFalke:
review ACK d71e29e3e8
Tree-SHA512: 4d8663ab505c347bcb62c2f118656e3343d5179825be0d1b86761ffdfdae1e7462002bf226a54dfc94be5885ce7f2633abaf70421ea35bf06eddad8e99fb9683
3002b4af2b [net processing] Guard m_continuation_block with m_block_inv_mutex (John Newbery)
184557e8e0 [net processing] Move hashContinue to net processing (John Newbery)
c853ef002e scripted-diff: rename vBlockHashesToAnnounce and vInventoryBlockToSend (John Newbery)
53b7ac1b7d [net processing] Move block inventory data to Peer (John Newbery)
78040f9168 [net processing] Rename nStartingHeight to m_starting_height (John Newbery)
77a2c2f8f9 [net processing] Move nStartingHeight to Peer (John Newbery)
717a374e74 [net processing] Improve documentation for Peer destruction/locking (John Newbery)
Pull request description:
This continues the work of moving application layer data into net_processing, by moving all block inventory state into the new Peer object added in #19607.
For motivation, see #19398.
ACKs for top commit:
jonatack:
re-ACK 3002b4af2b per `git diff 9aad3e4 3002b4a`
Sjors:
Code review re-ACK 3002b4af2b
MarcoFalke:
re-ACK 3002b4af2b🌓
Tree-SHA512: eb2b474b73b025791ee3e6e41809926b332b48468763219f31638ca390f427632f05902dfc6a2c6bdc1ce47b215782f67874ddbf05b97d77d5897b7e2abfe4d9
e3d2ba7c70 fuzz: Update FuzzedDataProvider.h from upstream (LLVM) (practicalswift)
Pull request description:
Update `FuzzedDataProvider.h` from upstream (LLVM).
Upstream revision: 6d0488f75b/compiler-rt/include/fuzzer/FuzzedDataProvider.h
Changes since last update:
* [[compiler-rt] FuzzedDataProvider: add ConsumeData and method.](20a604d3f5)
* [[compiler-rt] Fix a typo in a comment in FuzzedDataProvider.h.](5517d3b80b)
* [[compiler-rt] Add ConsumeRandomLengthString() version without arguments.](2136d17d8d)
* [[compiler-rt] Refactor FuzzedDataProvider for better readability.](1262db1b6a)
* [[compiler-rt] FuzzedDataProvider: make linter happy.](1e65209e04)
* [[compiler-rt] Mark FDP non-template methods inline to avoid ODR violations.](6d0488f75b)
ACKs for top commit:
MarcoFalke:
ACK e3d2ba7c70🌛
Tree-SHA512: 62cb27906f08fd07983f4a8fbbd381c12ed185617a58f1ebc8564c87e638086f952417f4f6481fbd91b9a313aff00e944215393734566c219c074512991f8057
Upstream revision: 6d0488f75b/compiler-rt/include/fuzzer/FuzzedDataProvider.h
Changes:
* [compiler-rt] FuzzedDataProvider: add ConsumeData and method.
* [compiler-rt] Fix a typo in a comment in FuzzedDataProvider.h.
* [compiler-rt] Add ConsumeRandomLengthString() version without arguments.
* [compiler-rt] Refactor FuzzedDataProvider for better readability.
* [compiler-rt] FuzzedDataProvider: make linter happy.
* [compiler-rt] Mark FDP non-template methods inline to avoid ODR violations.
1fbb8b6fa3 script: Remove outdated extract-osx-sdk.sh (Hennadii Stepanov)
Pull request description:
The removed script was used to prepare `MacOSX10.11.sdk` which we do not use since 0.20 (7e2104433c from #16392).
See https://github.com/bitcoin-core/docs/pull/68.
ACKs for top commit:
laanwj:
ACK 1fbb8b6fa3
Tree-SHA512: ab5128ec2c505a43b1a12e06abf38d8fa8f62a03efe760de2c842550e82f68ca512dee1b6c220a4f8af2465a9c35a429ca91cb179cc4ce016fcd87c61fbc8b07
fab46b34f4 test: Fix restart node race (MarcoFalke)
Pull request description:
It is not allowed to start a node before it has been fully stopped. Otherwise it could lead to intermittent issues due to access issues (e.g. cookie file https://cirrus-ci.com/task/6409665024098304?command=ci#L4793)
Fix that by waiting for the node to fully stop.
ACKs for top commit:
laanwj:
code review ACK fab46b34f4
Tree-SHA512: 7605cac0573a7b04f05ff110d0131e8940d87f7baf6d698505ed16b363d4d15b1e552c5ffd1a187c8fe5639f7e265c3122734c85283275746e46bd789614fd21
b23349b880 rpc: Add missing description of vout in getrawtransaction help text (Ben Carman)
Pull request description:
In `getrawtransaction` the vout did not have a description. I gave it the same description as the one used in `decoderawtransaction`.
ACKs for top commit:
MarcoFalke:
ACK b23349b880🏯
Tree-SHA512: 3833b97c82a46dfeb7ac825d4b2514b4b05ce54ac41f2144a8e2f2093b3411fe1d090c1e5b0c3d09200a2ea164c8d17ece12cdb43bbaeaeccc51a9da6dd7b7a3
fada8b019a test: Add missing assignment in mempool_resurrect.py (MarcoFalke)
Pull request description:
Top commit has no ACKs.
Tree-SHA512: f438d85cd14a91eabfc380d9ee120cc7a7f9103cf0cd1cf565f675f386f82d966901c0ad3f60b8c462642fbf0a3791dbbd774f9b07668d22b58eb575c8d702c1
11a32722f0 test: run mempool_resurrect.py even with wallet disabled (Michael Dietz)
Pull request description:
Another functional test rewritten as proposed in #20078
**Request for help:**
`node.gettransaction(txid)` fails for transactions sent with `wallet.send_self_transfer`. Even though the `txid`s look correct, are added to the mempool correctly, and removed from the mempool when a block is mined - all as expected.
However, `node.gettransaction(txid)` throws the error:
```sh
Traceback (most recent call last):
File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/test_framework.py", line 126, in main
self.run_test()
File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in run_test
assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
File "/Users/michaeldietz/Documents/bitcoin/test/functional/mempool_resurrect.py", line 43, in <lambda>
assert_equal(len(list(filter(lambda txid: node.gettransaction(txid)["confirmations"] > 0, spends_ids))), len(spends_ids))
File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/coverage.py", line 47, in __call__
return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
File "/Users/michaeldietz/Documents/bitcoin/test/functional/test_framework/authproxy.py", line 146, in __call__
raise JSONRPCException(response['error'], status)
test_framework.authproxy.JSONRPCException: Invalid or non-wallet transaction id (-5)
```
Anyone know what's going wrong / can point me in the right direction if I'm making a mistake, or `MiniWallet` needs to be improved for this to work correctly?
ACKs for top commit:
MarcoFalke:
ACK 11a32722f0
Tree-SHA512: 13d83a13ec23920db716e99b68670e61329d1cc73b12063d85bc1679ee6425a9951da4d2e392ca1f27760be7be049ccdc6f504e192ed5cd24ed0ba003b66fab3
fa4435e22f Replace boost::optional with std::optional (MarcoFalke)
fa7e803f3e Remove unused MakeOptional (MarcoFalke)
fadd4029dc psbt: Assert that tx has a value in UpdatePSBTOutput (MarcoFalke)
Pull request description:
Now that we can use std::optional from the vanilla standard library, drop the third-party boost dependency
ACKs for top commit:
practicalswift:
cr ACK fa4435e22f: patch looks correct!
laanwj:
code review ACK fa4435e22f
hebasto:
ACK fa4435e22f, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 50c5a1a130cac65e043e0177ba5b009fc2ba09343af4e23322ff2eb32184a55f8f2dea66e7a1b9d9acf56bc164eef4e47448750549a07f3b661199ac9bf9afef
3c2478c385 ci: Print COMMIT_RANGE to the log as it was in Travis CI (Hennadii Stepanov)
c123892c2e ci: Drop Travis-specific workaround for shellcheck (Hennadii Stepanov)
10af252d97 ci: Drop Travis-specific way to set COMMIT_RANGE variable (Hennadii Stepanov)
93504da3a9 ci: Fix COMMIT_RANGE variable value for PRs (Hennadii Stepanov)
Pull request description:
This PR:
- is a #20658 and #20682 followup
- set the `COMMIT_RANGE` variable correctly for PRs
- cleans up Travis-specific code
- prints COMMIT_RANGE value to the log for convenience as it was in Travis CI
ACKs for top commit:
MarcoFalke:
ACK 3c2478c385
Tree-SHA512: beb933352b10fd5eb3e66373ddb62439e4f3a03b50fb037ee89fa92c0706cec41d05f2d307f15bb18d1e634e6464f4e123b7e2f88703c8edfd145d8d6eff0b1a
d8b9cec25b inline non-member functions with body in fuzzing headers (Patrick Strateman)
Pull request description:
Resolves the issue noted [here](https://github.com/bitcoin/bitcoin/pull/20560#issuecomment-748339644)
ACKs for top commit:
MarcoFalke:
ACK d8b9cec25b
Tree-SHA512: fb34707e2d2c5b664d4160e0e4b56e3df9fb2c9045da6ddea7139e0b4982262c4e085812a8543a6221febc9cd0815423b8287fec66baae3236e5f3339cc9df8c
