543bf745d3 gitian-linux: Extend noexec-stack workaround to powerpc (Wladimir J. van der Laan)
00f67c8aa1 gitian-linux: Build binaries for 64-bit POWER (Luke Dashjr)
63fc2b1782 gitian: Properly quote arguments in wrappers (Luke Dashjr)
798bc0b29a Support glibc-back-compat on 64-bit POWER (Luke Dashjr)
Pull request description:
Rebase of #14066 by luke-jr.
Let's try to get PowerPC support in in the beginning of the 22.0 cycle so that it gets some testing, and is not a last-minute decision this time, like for last … 2 or 3 major versions.
The symbol/security tooling-related changes have been dropped since they were part of #20434.
Top commit has no ACKs.
Tree-SHA512: df0f8cd320c90f359f8b512c5cb8b59bb277516b57a05482cc8923c656106513b7428e315aaa8ab53e0bd6f80556b07d3639c47f6d9913bcfbfe388b39ef47c4
[META] This commit should be followed up by removing the comments and
assertions meant only to show that the change is correct.
LoadExternalBlockFile mainly acts on CChainState.
[META] This commit should be followed up by removing the comments and
assertions meant only to show that the change is correct.
GetLastCheckPoint mainly acts on BlockManager.
[META] This commit should be followed up by removing the comments and
assertions meant only to show that the change is correct.
GetSpendHeight only acts on BlockManager.
[META] This commit should be followed up by removing the comments and
assertions meant only to show that the change is correct.
FindForkInGlobalIndex only acts on BlockManager.
Note to reviewers: Since FindForkInGlobalIndex is always called with
::ChainActive() as its first parameter, it is possible to move
FindForkInGlobalIndex to CChainState and remove this const CChain&
parameter to instead use m_chain. However, it seems like the original
intention was for FindForkInGlobalIndex to work with _any_ chain, not
just the current active chain. Let me know if this should be changed.
[META] In a previous commit, we moved ::LookupBlockIndex to become a
member function of BlockManager. This commit is split out from
that one since it can be expressed nicely as a scripted-diff.
-BEGIN VERIFY SCRIPT-
find_regex='LookupBlockIndex' \
&& git grep -l -E "$find_regex" -- src \
| grep -v '^src/validation\.\(cpp\|h\)$' \
| xargs sed -i -E "s@${find_regex}@g_chainman.m_blockman.LookupBlockIndex@g"
-END VERIFY SCRIPT-
[META] This commit should be followed up by a scripted-diff commit which
fixes calls to LookupBlockIndex tree-wide.
[META] This commit should be followed up by removing the comments and
assertions meant only to show that the change is correct.
LookupBlockIndex only acts on BlockManager.
The current readme is a little bit outdated and contains some grammatical mistakes. This commit updates the doc so that:
- It is easier to follow and is more informative
- Fixes grammatical mistakes
- Modernizes the Qt Creater setup instructions
- Adds UNIX instructions for Qt Creator setup
fa04f9b4dd rpc: Remove duplicate name and argNames from CRPCCommand (MarcoFalke)
fa92912b4b rpc: Use RPCHelpMan for check-rpc-mappings linter (MarcoFalke)
faf835680b rpc: [refactor] Use concise C++11 code in CRPCConvertTable constructor (MarcoFalke)
Pull request description:
Currently, the RPC argument names are specified twice to simplify consistency linting. To avoid having to specify the argnames twice when adding new arguments, remove the linter and add an equivalent test based on RPCHelpMan.
ACKs for top commit:
laanwj:
ACK fa04f9b4dd
Tree-SHA512: 3f5f32f5a09b22d879f24aa67031639d2612cff481d6aebc6cfe6fd757cafb3e7bf72120b30466f59292a260747b71e57322c189d5478b668519b9f32fcde31a
fad3d7625a fuzz: Avoid initializing version to less than MIN_PEER_PROTO_VERSION (MarcoFalke)
fa99e33aeb fuzz: move-only FillNode implementation to cpp file (MarcoFalke)
Pull request description:
This fixes a fuzz bug introduced in #20881. Previously the nodes in the fuzz tests had their version initialized to a constant (`PROTOCOL_VERSION`). After #20881, the nodes have their version initialized to an arbitrary signed integer. This is problematic for several reasons:
* Both `nVersion` and `m_greatest_common_version` may be initialized to `0`. If a `version` message is processed, this leads to a crash, because `m_greatest_common_version` must be `INIT_PROTO_VERSION` while the `version` message is processed. See #20138
* The "valid" range for `nVersion` is `[MIN_PEER_PROTO_VERSION, std::numeric_limits<int32_t>::max()]` (see check in net_processing)
* The "valid" range for `m_greatest_common_version` is `std::min(nVersion, PROTOCOL_VERSION)` (see net_processing)
Fix all issues by initializing `nVersion` and `m_greatest_common_version` to their valid ranges.
-----
The crashers, if someone wants to try this at home:
```
( echo 'dmVyc2lvbgAWFhYWFhYWFhYWFhYWFhYWFhYWFhZp/29uAPX//xYWFhYWFhYWFhYWFhYWFhYWFhYW
FhYWFhYWaW9uAOr1//8WFhYWFha0ZXJzaW9uAPX//wAAAAAAABAAAAAAAAAAAAC0ZXJzaW9uAPX/
/wBPT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT09PT08AAAAAABAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAAAAAAAAAAAAAAACgAAAAAAAAAAgAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
AAAAAAAAAAAAAAB2ZXJzaW9uAACDJIO9vXYKAAAAAAAAAAAAAAAAAAAAAAB2ZfS1qmu1qhUVFWs=' | base64 --decode > /tmp/a ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/a
```
```
( echo 'dmVyc2lvbgD//wAhTmiqN///NDcAAACENDL/iv//8DYAAHL///////79/RtcAJqamhqa/QEAAAD/
///+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZyq8SaGVhZGVycwAAAAD/NDcAAACENDL/iv//
8DYAAHL///////79/RtcAJqamhqa/QEAAAD////+/f1oZWFkZXJzAAAAAM8BAAAAIAYibkYRGgtZ
yq8SaGVhZGVycwAAAADPAQAAACAGIm5GERoLWS1wb3J061u/KMNPOkwFXqZ///b5IgIAAD+5ubkb
XD5hZGRyAJqamhqasP0BAAAAAAAAAP0BAAAAIf39/R0dHQAAAAAAMgAA///7//+gXqZ///b5IgIA
AD+5ubm5ubm5AAAAAAAAAAAAAAAAAAAAAAAAAACAAAAAAAAAFgAAAAAAAAAAAAlBmv39/f1/f39B
f39hZGRyAG5vAACaLgAdGzY2zwEAAAAgBiJuRhEaC1ktcG9ydOtbvyjDTzpMBV6mf//2+SICAAA/
ubm5G1w+YWRkcgCampoamrD9AQAAAAAAAAD9AQAAACH9/f0dHR0AAAAAADIAAP//+///oF6mf//2
+SICAAA/ubm5ubm5uQAAAAAAAAAAAAAAAAAAAAAAAAAAgAAAAAAAABYAAAAAAAAAAAAJQZr9/f39
f39/QX9/YWRkcgBubwAAmi4AHRs2NjY2NjY2NjYCAgI2NgIA/f39/f39Nv39/TUmABxc' | base64 --decode > /tmp/b ) && FUZZ=process_message_version ./src/test/fuzz/fuzz /tmp/b
```
ACKs for top commit:
practicalswift:
cr ACK fad3d7625a
Tree-SHA512: ea64ee99b94d8e619e3949d2d21252c1236412c0e40f44f2b73595ca70cd2da0bdab005fb1a54f65fb291e7b07fdd33577ce4a3a078ca933246b511ebcb0e52a
957895c715 util: Log static plugins meta data and style (Hennadii Stepanov)
Pull request description:
This PR is a follow-up of https://github.com/bitcoin/bitcoin/pull/17826, and adds additional info about the imported static plugins and the used style to the `debug.log` I found useful for testing (e.g., with `QT_QPA_PLATFORM`, `QT_QPA_PLATFORMTHEME`, `QT_STYLE_OVERRIDE` variables) and debugging issues (e.g., https://github.com/bitcoin/bitcoin/pull/19716#issuecomment-674052881).
The excerpt from the log:
```
2020-11-15T18:41:45Z [main] Bitcoin Core version v0.20.99.0-f0b933f78 (release build)
2020-11-15T18:41:45Z [main] Qt 5.9.8 (static), plugin=xcb (static)
2020-11-15T18:41:45Z [main] Static plugins:
2020-11-15T18:41:45Z [main] QXcbIntegrationPlugin, version 329992
2020-11-15T18:41:45Z [main] Style: fusion / QFusionStyle
...
```
ACKs for top commit:
jarolrod:
ACK 957895c715, Tested on macOS 11.1
jonasschnelli:
utACK 957895c715
Tree-SHA512: 0e46db7560f380fbda8ce5e53faa5d419a456e90ca595ce46be8e3030c99d3a113586edad1988a97e9bf0279e944f975968ed1156817bc16723ed31c64850239
4e1154dfd1 qt: Use "fusion" style on macOS Big Sur with old Qt (Hennadii Stepanov)
Pull request description:
The "macintosh" style is broken on macOS Big Sur:
- https://github.com/bitcoin/bitcoin/issues/20555#issuecomment-756264648
- #136
ACKs for top commit:
MarcoFalke:
review ACK 4e1154dfd1 can't test
jarolrod:
ACK 4e1154dfd1
jonasschnelli:
Tested ACK 4e1154dfd1
Tree-SHA512: c2e0f7be220c8b34b182c73e362f41d0e8c8c002e766fcb5491c62f3cfb9f70eabbd32b29baefa152135efc5f83b15534c1c2459e500a586b0f64c5aa8acf614
647b81b709 wallet, rpc: add listdescriptors command (Ivan Metlushko)
Pull request description:
Looking for concept ACKs
**Rationale**: allow users to inspect the contents of their newly created descriptor wallets.
Currently the command only returns xpubs which is not very useful in itself, but there are multiples ways to extend it:
* add an option to export xprv
* with #19136 it'll be possible to return normalised descriptors suitable for a watch-only purposes
The output is compatible with `importdescriptors` command so it could be easily used for backup/recover purposes.
**Output example:**
```json
[
{
"desc": "wpkh(tpubD6NzVbkrYhZ4WW6E2ZETFyNfq2hfF23SKxqSGFvUpPAY58jmmuBybwqwFihAyQPk9KnwTt5516NDZRJ7k5QPeKjy7wuVd5WvXNxwwAs5tUD/*)#nhavpr5h",
"timestamp": 1296688602,
"active": false,
"range": [
0,
999
],
"next": 0
}
]
```
ACKs for top commit:
jonatack:
re-ACK 647b81b709 rebased to master, debug builds cleanly, reviewed diff since last review, tested with a descriptor wallet (and with a legacy wallet)
achow101:
re-ACK 647b81b
Tree-SHA512: 51a3620bb17c836c52cecb066d4fa9d5ff418af56809046eaee0528c4dc240a4e90fff5711ba96e399c6664e00b9ee8194e33852b1b9e75af18061296e19a8a7
a6739cc868 rpc: Add specific error code for "wallet already loaded" (Wladimir J. van der Laan)
Pull request description:
Add a separate RPC error code for "wallet already loaded" to avoid having to match on message to detect this.
Requested by shesek for rust-bitcoinrpc.
If concept ACKed needs:
- [ ] Release note
- [x] A functional test (updated the existing test to make it pass, I think this is enough)
ACKs for top commit:
jonasschnelli:
Code Review ACK a6739cc868
promag:
Code review ACK a6739cc868.
Tree-SHA512: 9091872e6ea148aec733705d6af330f72a02f23b936b892ac28f9023da7430af6332418048adbee6014305b812316391812039e9180f7f3362d11f206c13b7d0
2a39ccf133 Add include for std::bind. (sinetek)
Pull request description:
Hi, this patch adds in <functional> because the GUI code makes use of std::bind.
That's all.
ACKs for top commit:
jonasschnelli:
utACK 2a39ccf133
Tree-SHA512: fb5ac07d9cd5d006182b52857b289a9926362a2f1bfa4f7f1c78a088670e2ccf39ca28214781df82e8de3909fa3e69685fe1124a7e3ead758575839f5f2277a9
a2a3f4cd8d qt: drop workaround for QTBUG-42503 which was fixed in Qt 5.5.0 (Pavol Rusnak)
Pull request description:
Fixes https://github.com/bitcoin-core/gui/issues/101
ACKs for top commit:
jonasschnelli:
Tested ACK a2a3f4cd8d -
Tree-SHA512: 9ccbc20ff7991ec70cac7d4e4413f9d80a1de453e217d03927a5f167e87eae7f369f0ad437c40c8107e5384c5c5c758659ed0db923837a38f8d705f01f9cb798
232d1f92bb Add information to "Confirm fee bump" window (Prayank)
Pull request description:
+ Add information in bump fee confirmation box according to the documentation: https://bitcoincore.org/en/doc/0.20.0/rpc/wallet/bumpfee/
+ Workaround to fix issue: https://github.com/bitcoin/bitcoin/issues/20795 in which user isn't aware of new inputs, outputs added to replacement transaction before broadcasting. Initial transaction had used coin control features and custom change address. Until the issue is fixed by change in coin selection algorithm we can add this warning.
+ Waiting for comments from devs who are working on coin selection algorithm PRs or involved in related research. However got two comments from Luke Dashjr and Pieter Wuille:
_luke-jr: Reducing the change output also could be a privacy problem, since it identifies which output was change._
_sipa: Wallet doesn't know the original transaction was using coin control. So I think its expected that if you use automatic fee bumping, you'll get whatever the coin selection algorithm decides. As for why its not decreasing the change and instead adding another input, that may be a bug._ (IRC: #bitcoin-core-dev)
ACKs for top commit:
jonasschnelli:
Tested ACK - 232d1f92bb
Tree-SHA512: 2ff65db1ddb1d4a45f82670b6ca303a0bf48acf3d09defffc21f44ec81cb6182268959706f592f3442aae5db48f43b8ea86973d74ec2721be93d209ce0414953
f0f8b1a076 fuzz: Add UBSan suppressions needed for fuzz tests to not warn under -fsanitize=integer (practicalswift)
58232e3ffb fuzz: Avoid -fsanitize=integer warnings in fuzzing harnesses (practicalswift)
Pull request description:
Add UBSan suppressions needed for fuzz tests to not warn under `-fsanitize=integer`.
Avoid `-fsanitize=integer` warnings in fuzzing harnesses.
Suppressed warnings (excluding warnings from `src/crypto/` and `src/test/`):
```
addrman.cpp:306:24: runtime error: implicit conversion from type 'long' of value 5190149478 (64-bit, signed) to type 'uint32_t' (aka 'unsigned int') changed the value to 895182182 (32-bit, unsigned)
addrman.h:446:43: runtime error: implicit conversion from type 'int' of value -22 (32-bit, signed) to type 'const uint8_t' (aka 'const unsigned char') changed the value to 234 (8-bit, unsigned)
arith_uint256.cpp:32:35: runtime error: left shift of 1712128 by 24 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
arith_uint256.cpp:47:39: runtime error: left shift of 4294966784 by 31 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
chain.cpp:151:12: runtime error: implicit conversion from type 'int' of value -1 (32-bit, signed) to type 'unsigned long' changed the value to 18446744073709551615 (64-bit, unsigned)
coins.cpp:114:22: runtime error: unsigned integer overflow: 0 - 96 cannot be represented in type 'unsigned long'
compressor.cpp:162:33: runtime error: unsigned integer overflow: 15617702637291228364 * 10 cannot be represented in type 'unsigned long'
compressor.cpp:188:11: runtime error: unsigned integer overflow: 2265760372865400000 * 10 cannot be represented in type 'unsigned long'
hash.cpp:13:15: runtime error: left shift of 1692305888 by 15 places cannot be represented in type 'uint32_t' (aka 'unsigned int')
pubkey.h:152:23: runtime error: unsigned integer overflow: 0 - 1 cannot be represented in type 'unsigned int'
streams.h:570:31: runtime error: left shift of 350879 by 52 places cannot be represented in type 'uint64_t' (aka 'unsigned long')
util/bip32.cpp:57:36: runtime error: left shift of 3241096244 by 1 places cannot be represented in type 'unsigned int'
util/strencodings.cpp:562:38: runtime error: implicit conversion from type 'unsigned char' of value 255 (8-bit, unsigned) to type 'char' changed the value to -1 (8-bit, signed)
util/strencodings.h:164:24: runtime error: implicit conversion from type 'int' of value -74 (32-bit, signed) to type 'unsigned long' changed the value to 18446744073709551542 (64-bit, unsigned)
```
The warnings above happen here:
32b191fb66/src/addrman.cpp (L306)32b191fb66/src/addrman.h (L446)32b191fb66/src/arith_uint256.cpp (L32)32b191fb66/src/arith_uint256.cpp (L47)32b191fb66/src/chain.cpp (L151)32b191fb66/src/coins.cpp (L114)32b191fb66/src/compressor.cpp (L162)32b191fb66/src/compressor.cpp (L188)32b191fb66/src/hash.cpp (L13)32b191fb66/src/pubkey.h (L152)32b191fb66/src/streams.h (L570)32b191fb66/src/util/bip32.cpp (L57)32b191fb66/src/util/strencodings.cpp (L562)32b191fb66/src/util/strencodings.h (L164)
ACKs for top commit:
MarcoFalke:
review ACK f0f8b1a076🤚
Tree-SHA512: a8f04f7cc055d03653161de1d9d14d106a6280cea1e86a1243abcd57cf8e61dcf5f731d0ab0da5b390790e816022ff7a70759a641463bc7e3303076b8667009f
77114462f2 raise helpMessageDialog (randymcmillan)
Pull request description:
the raise() method brings the helpMessageDialog to the top if it is obscured by another window.
ACKs for top commit:
promag:
Code review ACK 77114462f2.
hebasto:
ACK 77114462f2, tested on:
Tree-SHA512: 0d5b107aa9a5ce3891e88ef69f64461c8b23d17476b798691119e84bfc78e16b2491c798adb5d6cc347af3b7f18729593d7924090c336114a3cf34fbee344bfb
Check if "Coin Control features" are enabled to display warning before broadcasting replacement transaction
Workaround to fix issue: bitcoin/bitcoin#20795
Co-authored-by: Jon Atack <jon@atack.com>
After the merge of #18710, the linter is warning:
```bash
A new Boost dependency in the form of "boost/thread/mutex.hpp" appears to have been introduced:
src/sync.cpp:#include <boost/thread/mutex.hpp>
src/test/sync_tests.cpp:#include <boost/thread/mutex.hpp>
^---- failure generated from test/lint/lint-includes.sh
```
the interim #19337 was merged, which introduced more `boost::mutex` usage.
Given we no longer use `boost::mutex`, just remove the double lock test
and remaining includes.
8f0b64fb51 Better error messages for invalid addresses (Bezdrighin)
Pull request description:
This PR addresses #20809.
We add more detailed error messages in case an invalid address is provided inside the 'validateaddress' and 'getaddressinfo' RPC calls. This also covers the case when a user provides an address from a wrong network.
We also add a functional test to test the new error messages.
ACKs for top commit:
kristapsk:
ACK 8f0b64fb51
meshcollider:
Code review ACK 8f0b64fb51
Tree-SHA512: ca0f806ab573e96b79e98d9f8c810b81fa99c638d9b5e4d99dc18c8bd2568e6a802ec305fdfb2983574a97a19a46fd53b77645f8078fb77e9deb24ad2a22cf93
bb6fcc75d1 refactor: Drop boost::thread stuff in CCheckQueue (Hennadii Stepanov)
6784ac471b bench: Use CCheckQueue local thread pool (Hennadii Stepanov)
dba30695fc test: Use CCheckQueue local thread pool (Hennadii Stepanov)
01511776ac Add local thread pool to CCheckQueue (Hennadii Stepanov)
0ef938685b refactor: Use member initializers in CCheckQueue (Hennadii Stepanov)
Pull request description:
This PR:
- gets rid of `boost::thread_group` in the `CCheckQueue` class
- allows thread safety annotation usage in the `CCheckQueue` class
- is alternative to #14464 (https://github.com/bitcoin/bitcoin/pull/18710#issuecomment-616618525, https://github.com/bitcoin/bitcoin/pull/18710#issuecomment-617291612)
Also, with this PR (I hope) it could be easier to resurrect a bunch of brilliant ideas from #9938.
Related: #17307
ACKs for top commit:
laanwj:
Code review ACK bb6fcc75d1
LarryRuane:
ACK bb6fcc75d1
jonatack:
Code review ACK bb6fcc75d1 and verified rebase to master builds cleanly with unit/functional tests green
Tree-SHA512: fddeb720d5a391b48bb4c6fa58ed34ccc3f57862fdb8e641745c021841c8340e35c5126338271446cbd98f40bd5484f27926aa6c3e76fa478ba1efafe72e73c1
Currently it was not possible to run just the BlockToJsonVerboes benchmarsk because it did not set up everything it needed, running `bench_bitcoin -filter=BlockToJsonVerbose` caused this assert to fail:
```
bench_bitcoin: chainparams.cpp:506: const CChainParams& Params(): Assertion `globalChainParams' failed.
```
Initializing TestingSetup fixes this.
This commit addresses #20809.
We add an additional 'error' property in the result of 'validateaddress' in case the address is not valid that gives a short description of why the address in invalid. We also change the error message returned by 'getaddressinfo' in case the address is invalid.
This commit adds the CaptureMessage function. This will later be called
when any message is sent or received. The capture directory is fixed,
in a new folder "message_capture" in the datadir. Peers will then have
their own subfolders, named with their IP address and port, replacing
colons with underscores to keep compatibility with Windows. Inside,
received and sent messages will be captured into two binary files,
msgs_recv.dat and msgs_sent.dat.
e.g.
message_capture/203.0.113.7_56072/msgs_recv.dat
message_capture/203.0.113.7_56072/msgs_sent.dat
The format has been designed as to result in a minimal performance
impact. A parsing script is added in a later commit.
bf100f8170 [net] Cleanup InactivityChecks() and add commenting about time (John Newbery)
06fa85cd50 [net] InactivityCheck() takes a CNode reference (John Newbery)
Pull request description:
This is a pure refactor and should not change any behavior. It clarifies and documents the InactivityCheck() function
This makes #20721 easier to review. In particular, this function uses a mixture of (unmockable) system time and mockable time. It's important to understand where those are being used when reviewing #20721.
#20721 doesn't require this change, so if others don't agree that it's useful and makes review easier, then I'm happy to close this and just do #20721 directly.
ACKs for top commit:
fanquake:
ACK bf100f8170
MarcoFalke:
review ACK bf100f8170💫
Tree-SHA512: 7b001de2a5fbe8a6dc37baeae930db5775290afb2e8a6aecdf13161f1e5b06ef813bc6291d8ee5cefcf1e430c955ea702833a8db84192eebe6e6acf0b9304cb2
962444295d zmq: deduplicate 'sequence' publisher message creation/sending (Sebastian Falbesoner)
Pull request description:
This small PR deduplicates common low-level creation and sending code for the 'sequence' zmq publisher message (methods `NotifyBlock{Connect,Disconnect}()`, `NotifyTransaction{Acceptance,Removal}()` in the class `CZMQPublishSequenceNotifier`) by introducing a helper function.
ACKs for top commit:
jonatack:
Code review re-ACK 962444295d per `git diff f231ffd 9624442`
instagibbs:
utACK 962444295d
Tree-SHA512: de0750d923f36d1a5751331e88eec8a1605cb88c93318830913210485e2bff712310484f18a0fb626df6ef32ce0b0cf57f4421ce62656e05fce7011a0c3c2d0e
f1694757dd guix: Fix typo (Carl Dong)
771c4b98a8 guix: README: Add darwin HOSTS entry (Carl Dong)
8dbf18cb1d guix: Check for macOS SDK before building anything (Carl Dong)
34b23f597e guix: Set ZERO_AR_DATE for darwin build determinism (Carl Dong)
f3835dc6a3 build: Make xorrisofs reproducible with -volume_date (Carl Dong)
c9eb4cf3a0 guix: Add support for darwin builds (Carl Dong)
37fe73a092 build: Add var printing target to src/Makefile.am (Carl Dong)
Pull request description:
This PR brings our Guix builds on par with Gitian in terms of supported architectures.
Reviewers: if you run a build, please submit:
```
find output/ -type f -print0 | env LC_ALL=C sort -z | xargs -r0 sha256sum
```
So that we can compare hashes and ensure reproducibility!
ACKs for top commit:
fanquake:
ACK f1694757dd - I think we can make some small usability improvements, but this is ok to merge now.
Tree-SHA512: 4af2b71654a9736467dcc681d10601c6eee37800d7847011a50585455b67b55d61742ca5604585f310a2fd75335b674e5e27dfb5169cb2f26e112aa4c411d8be
da9caa1ced Replace fs::absolute calls with AbsPathJoin calls (Kiminuo)
66576c4fd5 test: Clear forced -walletdir setting after wallet init_tests (Kiminuo)
Pull request description:
This adds better test coverage and will make it easier in #20744 to remove our dependency on the two-argument boost::filesystem::absolute() function which does not have a direct equivalent in C++17.
This PR doesn't change behavior aside from adding an assert and fixing a test bug.
ACKs for top commit:
jonatack:
Code review ACK da9caa1ced only doxygen improvements since my last review per `git diff d867d7a da9caa1`
MarcoFalke:
review ACK da9caa1ced📯
ryanofsky:
Code review ACK da9caa1ced. Just comment and test tweaks since previous review.
Tree-SHA512: c940ee60f3ba374d4927cf34cf12d27c4c735c94af591fbc0ca408c641b30f8f8fbcfe521d66bfbddf9877a1fc8cd99bd8a47ebcd2fa59789de6bd87a7b9cf4d
b396467053 locks: Annotate CTxMemPool::check to require cs_main (Carl Dong)
Pull request description:
```
Currently, CTxMemPool::check locks CTxMemPool's own cs member, then
calls GetSpendHeight which locks cs_main. This can potentially cause an
undesirable lock invesion since CTxMemPool's cs is supposed to be locked
after cs_main.
This does not cause us any problems right now because all callers of
CTxMemPool already lock cs_main before calling CTxMemPool::check, which
means that the LOCK(cs_main) in GetSpendHeight becomes benign.
However, it is currently possible for new code to be added which calls
CTxMemPool::check without locking cs_main (which would be dangerous).
Therefore we should make it explicit that cs_main needs to be held
before calling CTxMemPool::check.
NOTE: After all review-only assertions are removed in "#20158 |
tree-wide: De-globalize ChainstateManager", and assuming that we
keep the changes in "validation: Pass in spendheight to
CTxMemPool::check", we can re-evaluate to see if this annotation
is still necessary.
```
-----
Previous discussions:
1. https://github.com/bitcoin/bitcoin/pull/20158#discussion_r520639845
2. https://github.com/bitcoin/bitcoin/pull/20158#pullrequestreview-557117202
3. https://github.com/bitcoin/bitcoin/pull/20749#discussion_r559425521
ACKs for top commit:
jnewbery:
Code review ACK b396467053
MarcoFalke:
ACK b396467053
jonatack:
ACK b396467053 review and debug built, verified that `cs_main` is held by callers of `CTxMemPool::check()` in `PeerManagerImpl::ProcessOrphanTx()`, `PeerManagerImpl::ProcessMessage()`, and `CChainState::ActivateBestChainStep()`
Tree-SHA512: 4635cddb4aa1af9532bb657b2f9c4deec4568d16ba28c574eae91bb77368cd40e23c3c720a9de11cec78e7ad678a44a5e25af67f13214b86b56e777e0c35a026
abb6fa7285 fuzz: Initialize a full TestingSetup where appropriate (Carl Dong)
713314abfa fuzz: Consolidate fuzzing TestingSetup initialization (Carl Dong)
Pull request description:
```
Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:
1. Calling InitializeFuzzingContext, which implicitly constructs a static
const BasicTestingSetup
2. Directly constructing a static const BasicTestingSetup in the initialize_*
function
3. Directly constructing a static TestingSetup and reproducing the
initialization arguments (I'm assuming because
InitializeFuzzingContext only initializes a BasicTestingSetup)
The new, relatively-simple MakeFuzzingContext function allows us to
consolidate these methods of initialization by being flexible enough to
be used in all situations. It:
1. Is templated so that we can choose to initialize any of
the *TestingSetup classes
2. Has sane defaults which are often used in fuzzers but are also
easily overridable
3. Returns a unique_ptr, explicitly transferring ownership to the caller
to deal with according to its situation
```
~~Question for fuzzing people: was it intentional that `src/test/fuzz/net.cpp` would directly instantiate the `BasicTestingSetup` and thus omit the `"-nodebuglogfile"` flag?~~ [Answered](https://github.com/bitcoin/bitcoin/pull/20946#issuecomment-761537108)
ACKs for top commit:
MarcoFalke:
ACK abb6fa7285
Tree-SHA512: 96a5ca6f4cd5ea0e9483b60165b31ae3e9003918c700a7f6ade48010f419f2a6312e10b816b3187f1d263798827571866e4c4ac0bbfb2e0c79dfad254cda68e7
Previously, the {Basic,}TestingSetup for fuzzers were set up in many ways:
1. Calling InitializeFuzzingContext, which implicitly constructs a static
const BasicTestingSetup
2. Directly constructing a static const BasicTestingSetup in the initialize_*
function
3. Directly constructing a static TestingSetup and reproducing the
initialization arguments (I'm assuming because
InitializeFuzzingContext only initializes a BasicTestingSetup)
The new, relatively-simple MakeFuzzingContext function allows us to
consolidate these methods of initialization by being flexible enough to
be used in all situations. It:
1. Is templated so that we can choose to initialize any of
the *TestingSetup classes
2. Has sane defaults which are often used in fuzzers but are also
easily overridable
3. Returns a unique_ptr, explicitly transferring ownership to the caller
to deal with according to its situation
Currently, CTxMemPool::check locks CTxMemPool's own cs member, then
calls GetSpendHeight which locks cs_main. This can potentially cause an
undesirable lock invesion since CTxMemPool's cs is supposed to be locked
after cs_main.
This does not cause us any problems right now because all callers of
CTxMemPool already lock cs_main before calling CTxMemPool::check, which
means that the LOCK(cs_main) in GetSpendHeight becomes benign.
However, it is currently possible for new code to be added which calls
CTxMemPool::check without locking cs_main (which would be dangerous).
Therefore we should make it explicit that cs_main needs to be held
before calling CTxMemPool::check.
NOTE: After all review-only assertions are removed in "#20158 |
tree-wide: De-globalize ChainstateManager", and assuming that we
keep the changes in "validation: Pass in spendheight to
CTxMemPool::check", we can re-evaluate to see if this annotation
is still necessary.
ad57fb756b wallet: Add BerkeleyDB version sanity check at init time (Wladimir J. van der Laan)
Pull request description:
Detect version conflicts between the run-time BerkeleyDB library and the one used during compilation.
This is very unsafe (can result in anything from crashes to corruption) so shut down when one is detected.
ACKs for top commit:
decryp2kanon:
utACK ad57fb7
achow101:
ACK ad57fb756b
theStack:
utACK ad57fb756b
meshcollider:
Code review ACK ad57fb756b
Tree-SHA512: 99cd7d836bffbdeb3d4e14053f7139cc85a6d42e631a3f9a3058a848042446b364faee127500f5acb374616e6a61ab2bedebfac1ba9bc993b4d6227114c2a6c2
ea0a7ec949 Remove deprecated bumpfee behavior (Andrew Chow)
Pull request description:
Removes the deprecation message, behavior, and test.
This was marked for removal in 22.0.
ACKs for top commit:
promag:
ACK ea0a7ec949, maybe add need release notes tag.
Tree-SHA512: d1626906849f6ee37213c32e5f8c1433ad8fb7beabcd88f5801b1964b322171a2341bdfbd9a3a5ab39b2fd9d9c6a05f73298583423a73cab1275653105c03e8e
Also clean up and better comment the function. InactivityChecks() uses a
mixture of (non-mockable) system time and mockable time. Make sure
that's well documented.
Despite being marked as const in CConnman before this commit, the
function did mutate the state of the passed in CNode, which is contained
in vNodes, which is a member of CConnman. To make the function truly
const in CConnman and all its data, instead make InactivityChecks() a
pure function, return whether the peer should be disconnected, and let
the calling function (SocketHandler()) update the CNode object. Also
make the CNode& argument const.
22eb7930a6 tracing: add tracing framework (William Casarin)
933ab8a720 build: detect sys/sdt.h for eBPF tracing (William Casarin)
Pull request description:
Instead of writing ad-hoc logging everywhere (eg: #19509), we can take advantage of linux user static defined traces, aka. USDTs ( not the stablecoin 😅 )
The linux kernel can hook into these tracepoints at runtime, but otherwise they have little to no performance impact. Traces can pass data which can be printed externally via tools such as bpftrace. For example, here's one that prints incoming and outgoing network messages:
# Examples
## Network Messages
```
#!/usr/bin/env bpftrace
BEGIN
{
printf("bitcoin net msgs\n");
@start = nsecs;
}
usdt:./src/bitcoind:net:push_message
{
$ip = str(arg0);
$peer_id = (int64)arg1;
$command = str(arg2);
$data_len = arg3;
$data = buf(arg3,arg4);
$t = (nsecs - @start) / 100000;
printf("%zu outbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);
@outbound[$command]++;
}
usdt:./src/bitcoind:net:process_message
{
$ip = str(arg0);
$peer_id = (int64)arg1;
$command = str(arg2);
$data_len = arg3;
$data = buf(arg3,arg4);
$t = (nsecs - @start) / 100000;
printf("%zu inbound %s %s %zu %d %r\n", $t, $command, $ip, $peer_id, $data_len, $data);
@inbound[$ip, $command]++;
}
```
$ sudo bpftrace netmsg.bt
output: https://jb55.com/s/b11312484b601fb3.txt
if you look at the bottom of the output you can see a histogram of all the messages grouped by message type and IP. nice!
## IBD Benchmarking
```
#!/usr/bin/env bpftrace
BEGIN
{
printf("IBD to 500,000 bench\n");
}
usdt:./src/bitcoind:CChainState:ConnectBlock
{
$height = (uint32)arg0;
if ($height == 1) {
printf("block 1 found, starting benchmark\n");
@start = nsecs;
}
if ($height >= 500000) {
@end = nsecs;
@duration = @end - @start;
exit();
}
}
END {
printf("duration %d ms\n", @duration / 1000000)
}
```
This one hooks into ConnectBlock and prints the IBD time to height 500,000 starting from the first call to ConnectBlock
Userspace static tracepoints give lots of flexibility without invasive logging code. It's also more flexible than ad-hoc logging code, allowing you to instrument many different aspects of the system without having to enable per-subsystem logging.
Other ideas: tracepoints for lock contention, threads, what else?
Let me know what ya'll think and if this is worth adding to bitcoin.
## TODO
- [ ] docs?
- [x] Integrate systemtap-std-dev/libsystemtap into build (provides the <sys/sdt.h> header)
- [x] ~dtrace macos support? (is this still a thing?)~ going to focus on linux for now
ACKs for top commit:
laanwj:
Tested ACK 22eb7930a6
0xB10C:
Tested ACK 22eb7930a6
Tree-SHA512: 69242242112b679c8a12a22b3bc50252c305894fb3055ae6e13d5f56221d858e58af1d698af55e23b69bdb7abedb5565ac6b45fa5144087b77a17acd04646a75
54ce4fac80 build: improve macro for testing -latomic requirement (fanquake)
2c010b9c56 add std::atomic include to bitcoin-util.cpp (fanquake)
Pull request description:
Since the merge of #19937, riscv builds have been failing, due to a link issue with [`std::atomic_exchange`](https://en.cppreference.com/w/cpp/atomic/atomic_exchange) in `bitcoin-util`:
```bash
CXXLD bitcoin-util
bitcoin_util-bitcoin-util.o: In function `grind_task':
/home/ubuntu/build/bitcoin/distsrc-riscv64-linux-gnu/src/bitcoin-util.cpp:98: undefined reference to `__atomic_exchange_1'
collect2: error: ld returned 1 exit status
```
We have a [macro](https://github.com/bitcoin/bitcoin/blob/master/build-aux/m4/l_atomic.m4) that tries to determine when `-latomic` is required, however it doesn't quite work well enough, as it's currently determining it isn't needed:
```bash
./autogen.sh
./configure --prefix=/home/ubuntu/bitcoin/depends/riscv64-linux-gnu
...
checking whether std::atomic can be used without link library... yes
```
This PR adds a call to `std::atomic_exchange` to the macro, which will get us properly linked against `-latomic` on riscv:
```bash
checking whether std::atomic can be used without link library... no
checking whether std::atomic needs -latomic... yes
```
Also adds an `<atomic>` include to `bitcoin-util.cpp`.
ACKs for top commit:
laanwj:
Tested ACK 54ce4fac80
Tree-SHA512: 963c875097ee96b131163ae8109bcf8fecf4451d20faa2f3d223f9938ea3d8d1ed5604e12ad82c2b4b1c605fd293a9b6b08fefc00dd3e68d09c49e95029c6f50
Only rebucket if the asmap checksum has changed, not if the file format
has changed but no asmap is provided.
Also, don't try to add an entry to another bucket if it already appears
in ADDRMAN_NEW_BUCKETS_PER_ADDRESS buckets.
Version implies that higher numbers take precendence. This is really a
checksum, to check whether the provided asmap is the same as the one
used when the peers.dat file was serialized.
Also update the comments to explain where/why this is used.
An addrman entry can appear in up to 8 new table buckets. We store this
entry->bucket indexing during shutdown so that on restart we can restore
the entries to their correct buckets.
Commit ec45646de9 broke the
deserialization code so that each entry could only be put in up to one
new bucket. Fix that.
This brings PushMessage and ProcessMessages further in line with the
style guide by fixing their if statements.
LogMessage is later called, inside an if statement, inside both of these
methods.
c061800bb1 build: fix RELOC_SECTION security check for bitcoin-util (fanquake)
Pull request description:
The binutils we use for gitian builds strips the reloc section from
Windows binaries, which breaks ASLR. As a temporary workaround, export
main(). This is the same workaround as #18702 (bitcoin-cli), and will
fix the currently failing security check:
```bash
+ make -j1 -C src check-security
make: Entering directory '/home/ubuntu/build/bitcoin/distsrc-x86_64-w64-mingw32/src'
Checking binary security...
bitcoin-util.exe: failed RELOC_SECTION
make: *** [check-security] Error 1
```
Relevant upstream issue:
https://sourceware.org/bugzilla/show_bug.cgi?id=19011
ACKs for top commit:
dongcarl:
ACK c061800bb1
laanwj:
ACK c061800bb1
Tree-SHA512: a1a4da0b2cddfc377190b9044a04f42a859ca79f11ce2c2ab4c3d066a2786c34d5446d75f8eec634f308d2d3349ebbd7c9f76dcaebeeb28e471c829851592367
Detect version conflicts between the run-time BerkeleyDB library and the one used during compilation.
This is very unsafe (can result in anything from crashes to corruption) so shut down when one is detected.
It is not documented that if you attempt to send a transaction
which already exists in a block, an RPC_TRANSACTION_ALREADY_IN_CHAIN
exception will be raised. This should be documented so that developers
are aware that this exception is raised.
This adds better test coverage and will make it easier in #20744 to remove our dependency on the two-argument boost::filesystem::absolute() function which does not have a direct equivalent in C++17.
Leaving this value set interfered with the CreateWallet test if it happened to execute later in the test ordering. Specifically it would cause CreateWallet test to write data to the current directory instead of temporary test directory.
fa0aa87071 rpc: Return wtxid from testmempoolaccept (MarcoFalke)
Pull request description:
It would be nice if `testmempoolaccept` returned the unique wtxid directly to avoid a costly `decoderawtransaction` roundtrip
ACKs for top commit:
mjdietzx:
utACK fa0aa87071
stackman27:
utACK [`fa0aa87`](fa0aa87071)
glozow:
cr ACK fa0aa87071
Tree-SHA512: 05dbaf46d93e47e9eedb725c2f57175e6d4e1722da0531fe4f80e74fc2518911da87634f25f61fa2bc8d87a3017e82fd0684b09a0a9706d71deed970035c2e7a
fa0a864b38 fuzz: Use mocktime in process_message* fuzz targets (MarcoFalke)
Pull request description:
Use mocktime to allow time to advance deterministically during execution of a fuzz input. This also allows to drop the call to `JumpOutOfIbd`.
ACKs for top commit:
practicalswift:
cr ACK fa0a864b38
Tree-SHA512: e92fc70ec6bd49760173cb202549f364304e22b3f7127b9a4da8447cf9341008e477ad42c2599c2fde167bbcbc0e2d139709b4ef6371788bc2c1c3b7f589e11d
The binutils we use for gitian builds strips the reloc section from
Windows binaries, which breaks ASLR. As a temporary workaround, export
main(). This is the same workaround as #18702 (bitcoin-cli), and will
fix the currently failing security check:
```bash
+ make -j1 -C src check-security
make: Entering directory '/home/ubuntu/build/bitcoin/distsrc-x86_64-w64-mingw32/src'
Checking binary security...
bitcoin-util.exe: failed RELOC_SECTION
make: *** [check-security] Error 1
```
Relevant upstream issue:
https://sourceware.org/bugzilla/show_bug.cgi?id=19011
2f463f57e3 [doc] for CheckInputsFromMempoolAndCache (gzhao408)
85cc6bed64 lock annotations for MemPoolAccept functions (gzhao408)
Pull request description:
This is a very small PR that adds some lock annotations to clarify that, now, the `pool.cs` lock is held throughout tx validation for mempool. The comments in `CheckInputsFromMempoolAndCache` were unclear/outdated so I updated those as well.
~This PR is a cleanup. It removes unnecessary code that doesn't do much.~
ACKs for top commit:
sdaftuar:
utACK 2f463f57e3
jnewbery:
utACK 2f463f57e3
MarcoFalke:
cr ACK 2f463f57e3
fanquake:
ACK 2f463f57e3
Tree-SHA512: 5863a546b00ef02eba8f208c2c04c32f64671f17c967a5e3c51332fc0f472e5e9addddae075d0b91c77caebe1be9331317646b0bec802e86d9550773fd9621a9
fa75d40ef8 fuzz: Introduce CallOneOf helper to replace switch-case (MarcoFalke)
Pull request description:
The current `switch (fuzzed_data_provider.ConsumeIntegralInRange<int>(0, nn)) { case 0: ... case 1: ... case nn: ...` has several problems:
* It makes it hard to review newly added targets, because it requires manual counting of cases
* It makes it hard to update a target, because updating all case labels is trivial, but tedious to review and causes merge conflicts
* ~~Updating the target raises the question whether the case labels should be preserved to not invalidate the existing fuzz inputs format. Fuzz input format might already change implicitly on every commit, so this isn't something worthwhile to pursue.~~ Edit: This pull doesn't fix this problem.
Fix all issues by adding a new `CallOneOf` helper
ACKs for top commit:
ajtowns:
ACK fa75d40ef8 - code review only
jnewbery:
utACK fa75d40ef8
Tree-SHA512: 2daa602b240b86c8e85a024e008f03a57ba60349377eed771f4d21a97a9dba9b66e93fff16ff1992018d4330be7a1a276944c3dfdf698748ce135626c380e563
8775691383 Bugfix: GUI: Restore SendConfirmationDialog button default to "Yes" (Luke Dashjr)
Pull request description:
The SendConfirmationDialog is used for bumping the fee, where "Send" doesn't really make sense
Originally https://github.com/bitcoin/bitcoin/pull/17463, but rewritten here much simpler based on other merged changes.
ACKs for top commit:
hebasto:
ACK 8775691383, tested on Linux Mint 20.1 (x86_64, Qt 5.12.8):
Tree-SHA512: 3953cc9c09613c9a629def8b4dc061b537f148ddcb378430645602e0be0f3a9f1cff083aa685b94b2e9372300d02ec97e0d9ea89db6e3c6feec86795090f0f77
fc726e0138 doc, rpc: add missing signet mentions in network name lists (Sebastian Falbesoner)
Pull request description:
This small PR adds a few missing mentions of signet w.r.t. chain enumerations:
- RPC `getblockchaininfo`: result description for `"chain"`
- RPC `getmininginfo`: result description for `"chain"`
- REST interface documentation:
- default ports listing for each chain
- `"chain"` description for `chaininfo` endpoint result
The instances were identified via `git grep -i "main.*test.*reg"`.
ACKs for top commit:
ajtowns:
ACK fc726e0138 -- quick code review only
benthecarman:
ACK fc726e0138
Tree-SHA512: 62cdc6ef74fa10db75cc04b9eaf7367183f726b3fee3d21fdf741b3816669dd21508735e89da389ddac980f49773ab229263748d1399553375fefe4526361846
This removes a conditional that checks if pindex is equal to nullptr.
This check is redundant because the branch where pindex is set returns at an earlier time. Additionaly, The independence of the earlier and later pindex is made clearer.
bc99ae77e4 scripted-diff: Fix typo in stub manual pages (Wladimir J. van der Laan)
b5e93f873a doc: Add manual page generation for bitcoin-util (Wladimir J. van der Laan)
Pull request description:
- Add `-version` option to `bitcoin-util`
- Add `bitcoin-util` call to `gen-manpages.sh`
- Add stub manual page `bitcoin-util.1`
- Add install of `bitcoin-util.1` to build system
ACKs for top commit:
fanquake:
ACK bc99ae77e4
Tree-SHA512: 948df66c62bbca1cf6da26845dfa63f8f5d036a3d5744add468dd1ce7f442c123d7b0db7011c2e8e3ee6539fd391c7ee2c21b706ec81b21b02821c9501cd077d
281fd1a4a0 Replace KeyIDHasher with SaltedSipHasher (Andrew Chow)
210b693db6 Add generic SaltedSipHasher (Andrew Chow)
95e61c1cf2 Move Hashers to util/hasher.{cpp/h} (Andrew Chow)
Pull request description:
There are existing `SaltedOutPointHasher` and `SaltedTxidHasher` classes used for `std::unordered_map` and `std::unordered_set` that could be useful in other places in the codebase. So we these to their own `saltedhash.{cpp/h}` file. An existing `KeyIDHasher` is moved there too. Additionally, `ScriptIDHasher`, `SaltedPubkeyHasher`, and `SaltedScriptHasher` are added so that they can be used in future work.
`KeyIDHasher` and `ScriptIDHasher` are not salted so that equality comparisons of maps and sets keyed by `CKeyID` and `CScriptID` will actually work.
Split from #19602 (and a few other PRs/branches I have).
ACKs for top commit:
laanwj:
Code review ACK 281fd1a4a0
jonatack:
ACK 281fd1a4a0, code review, debug build and ran bitcoind after rebasing to master @ dff0f6f753
fjahr:
utACK 281fd1a4a0
Tree-SHA512: bb03b231ccf3c9ecefc997b8da9c3770af4819f9be5b0a72997a103864e84046a2ac39b8eadf0dc9247bdccd53f86f433642e3a098882e6748341a9e7736271b
595a34dbea contrib/signet: Document miner script in README.md (Anthony Towns)
ff7dbdc08a contrib/signet: Add script for generating a signet chain (Anthony Towns)
13762bcc96 Add bitcoin-util command line utility (Anthony Towns)
95d5d5e625 rpc: allow getblocktemplate for test chains when unconnected or in IBD (Anthony Towns)
81c54dec20 rpc: update getblocktemplate with signet rule, include signet_challenge (Anthony Towns)
Pull request description:
Adds `contrib/signet/miner` for mining signet blocks.
Adds `bitcoin-util` cli utility, with the idea being it can provide bitcoin related functionality that does not rely on the ability to access a running node. Only subcommand currently is "grind" which takes a hex-encoded header and grinds its nonce until its nBits is satisfied.
Updates `getblocktemplate` to include `signet_challenge` field, and makes `getblocktemplate` require the signet rule when invoked on the signet change. Removes connectivity and IBD checks from `getblocktemplate` when applied to a test chain (regtest, testnet, signet).
ACKs for top commit:
laanwj:
code review ACK 595a34dbea
Tree-SHA512: 8d43297710fdc1edc58acd9b53e1bd1671e5724f7097b40ab73653715dc8becc70534c4496cbba9290f4dd6538a7a3d5830eb85f83391ea31a3bb5b9d3378cc3
3eb94ec81b sync: Use decltype(auto) return type for WITH_LOCK (Carl Dong)
Pull request description:
> Now that we're using C++17, we can use the decltype(auto) return type
> for functions and lambda expressions.
>
> As demonstrated in this commit, this can simplify cases where previously
> the compiler failed to deduce the correct return type.
>
> Just for reference, for the "assign to ref" cases fixed here, there are
> 3 possible solutions:
>
> - Return a pointer and immediately deref as used before this commit
> - Make sure the function/lambda returns declspec(auto) as used after
> this commit
> - Class& i = WITH_LOCK(..., return std::ref(...));
>
> -----
>
> References:
> 1. https://en.cppreference.com/w/cpp/language/function#Return_type_deduction
> 2. https://en.cppreference.com/w/cpp/language/template_argument_deduction#Other_contexts
> 3. https://en.cppreference.com/w/cpp/language/auto
> 4. https://en.cppreference.com/w/cpp/language/decltype
>
> Explanations:
> 1. https://stackoverflow.com/a/21369192
> 2. https://stackoverflow.com/a/21369170
Thanks to sipa and ryanofsky for helping me understand this
ACKs for top commit:
jnewbery:
utACK 3eb94ec81b
hebasto:
ACK 3eb94ec81b, I have reviewed the code and it looks OK, I agree it can be merged. I have verified possible warnings:
ryanofsky:
Code review ACK 3eb94ec81b
Tree-SHA512: 5f55c7722aeca8ea70e5c1a8db93e93ba0e356e8967e7f607ada38003df4b153d73c29bd2cea8d7ec1344720d37d857ea7dbfd2a88da1d92e0e9cbb9abd287df
b4dd2ef800 [test] Test the add_outbound_p2p_connection functionality (Amiti Uttarwar)
602e69e427 [test] P2PBlocksOnly - Test block-relay-only connections. (Amiti Uttarwar)
8bb6beacb1 [test/refactor] P2PBlocksOnly - Extract transaction violation test into helper. (Amiti Uttarwar)
99791e7560 [test/refactor] P2PBlocksOnly - simplify transaction creation using blocktool helper. (Amiti Uttarwar)
3997ab9154 [test] Add test framework support to create outbound connections. (Amiti Uttarwar)
5bc04e8837 [rpc/net] Introduce addconnection to test outbounds & blockrelay (Amiti Uttarwar)
Pull request description:
The existing functional test framework uses the `addnode` RPC to spin up manual connections between bitcoind nodes. This limits our ability to add integration tests for our networking code, which often executes different code paths for different connection types.
**This PR enables creating `outbound` & `block-relay-only` P2P connections in the functional tests.** This allows us to increase our p2p test coverage, since we can now verify expectations around these connection types.
This builds out the [prototype](https://github.com/bitcoin/bitcoin/issues/14210#issuecomment-527421978) proposed by ajtowns in #14210. 🙌🏽
An overview of this branch:
- introduces a new test-only RPC function `addconnection` which initiates opening an `outbound` or `block-relay-only` connection. (conceptually similar to `addnode` but for different connection types & restricted to regtest)
- adds `test_framework` support so a mininode can open an `outbound`/`block-relay-only` connection to a `P2PInterface`/`P2PConnection`.
- updates `p2p_blocksonly` tests to create a `block-relay-only` connection & verify expectations around transaction relay.
- introduces `p2p_add_connections` test that checks the behaviors of the newly introduced `add_outbound_p2p_connection` test framework function.
With these changes, there are many more behaviors that we can add integration tests for. The blocksonly updates is just one example.
Huge props to ajtowns for conceiving the approach & providing me feedback as I've built out this branch. Also thank you to jnewbery for lots of thoughtful input along the way.
ACKs for top commit:
troygiorshev:
reACK b4dd2ef800
jnewbery:
utACK b4dd2ef800
MarcoFalke:
Approach ACK b4dd2ef800🍢
Tree-SHA512: d1cba768c19c9c80e6a38b1c340cc86a90701b14772c4a0791c458f9097f6a4574b4a4acc7d13d6790c7b1f1f197e2c3d87996270f177402145f084ef8519a6b
aaaa987840 refactor: Use C++17 std::array deduction for ALL_FEE_ESTIMATE_HORIZONS (MarcoFalke)
fa39cdd072 refactor: Use C++17 std::array deduction for OUTPUT_TYPES (MarcoFalke)
Pull request description:
With the new C++17 array deduction rules, an array encompassing all values in an enum can be specified in the same header file that specifies the enum. This is useful to avoid having to repeatedly enumerate all enum values in the code. E.g. the RPC code, but also the fuzz code.
ACKs for top commit:
theStack:
cr ACK aaaa987840⚙️
fanquake:
ACK aaaa987840
Tree-SHA512: b71bd98f3ca07ddfec385735538ce89a4952e418b52dc990fb160187ccef1fc7ebc139d42988b6f7b48df24823af61f803b83d47fb7a3b82475f0c0b109bffb7
3642b2ed34 refactor, net: Increase CNode data member encapsulation (Hennadii Stepanov)
acebb79d3f refactor, move-only: Relocate CNode private members (Hennadii Stepanov)
Pull request description:
All protected `CNode` data members could be private.
ACKs for top commit:
jnewbery:
utACK 3642b2ed34
MarcoFalke:
review ACK 3642b2ed34 🏛
Tree-SHA512: 8435e3c43c3b7a3107d58cb809b8b5e1a1c0068677e249bdf0fc6ed24140ac4fc4efe2a280a1ee86df180d738c0c9e10772308690607954db6713000cf6e728d
39b43298d9 test: add test for banning of non-IP addresses (Vasil Dimov)
94d335da7f net: allow CSubNet of non-IP networks (Vasil Dimov)
Pull request description:
Allow creation of valid `CSubNet` objects of non-IP networks and only
match the single address they were created from (like /32 for IPv4 or
/128 for IPv6).
This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
and in `BanMan` which assume that creating a subnet from any address
using the `CSubNet(CNetAddr)` constructor would later match that address
only. Before this change a non-IP subnet would be invalid and would not
match any address.
ACKs for top commit:
jonatack:
Code review re-ACK 39b43298d9 per `git diff 5e95ce6 39b4329`; only change since last review is improvements to the functional test; verified the test fails on master @ 616eace0 where expected (`assert(self.is_banned(node, tor_addr))` fails and unban unfails)
laanwj:
code review ACK 39b43298d9
Tree-SHA512: 3239b26d0f2fa2d1388b4fdbc1d05ce4ac1980be699c6ec46049409baefcb2006b1e72b889871e2210e897f6725c48e873f68457eea7e6e4958ab4f959d20297
b3e9bcaac8 qt, refactor: Drop no longer used PeerTableModel::getNodeStats function (Hennadii Stepanov)
49c604077c qt: Use PeerTableModel::StatsRole (Hennadii Stepanov)
35007edf9c qt: Add PeerTableModel::StatsRole (Hennadii Stepanov)
Pull request description:
This PR allows to access to the `CNodeCombinedStats` instance directly from any view object.
The `PeerTableModel::getNodeStats` member function removed as a kind of layer violation.
No behavior changes.
Also other pulls (bugfixes) are based on this one: #18 and #164.
ACKs for top commit:
jonatack:
Tested re-ACK b3e9bcaac8 per `git range-diff ae8f797 4c05fe0 b3e9bca`
promag:
Code review ACK b3e9bcaac8.
jonasschnelli:
utACK b3e9bcaac8
Tree-SHA512: 6ba50d5dd2c0373655d491ce8b130c47d598da2db5ff4b00633f447404c7e70f8562ead53ddf166e851384d9632ff9146a770c99845c2cdd3ff7250677e4c130
faa8f68943 Replace boost::variant with std::variant (MarcoFalke)
Pull request description:
Now that we can use std::variant from the vanilla standard library, drop the third-party boost variant dependency
ACKs for top commit:
fjahr:
Code review ACK faa8f68943
fanquake:
ACK faa8f68943
Tree-SHA512: 6e3aecd33b00c2e31a763f999247944d5b2ce5e3018f1965c516c1000cd08ff6703a8d50fb0be64883153da2925ae72986b8a6b96586db74057bd05d6f4986e6
fad1f0fd33 net: Remove unused cs_feeFilter (MarcoFalke)
Pull request description:
A `RecursiveMutex` is overkill for setting or reading a plain integer. Even a `Mutex` is overkill, when a plain `std::atomic` can be used.
This removes 11 lines of code. Also, it is cutting down on the number of locks put on the stack at the same time, which complicates review looking out for potential lock contention.
ACKs for top commit:
jnewbery:
utACK fad1f0fd33
practicalswift:
cr ACK fad1f0fd33: patch looks correct
Tree-SHA512: 647f9b954fbf52e138d3e710937eb9131b390fef0deae03fd6a162d5a18b9f194010800bbddc8f89208d91be2802dff11c3884d04b3dd233865abd12aa3cde06
Allow creation of valid `CSubNet` objects of non-IP networks and only
match the single address they were created from (like /32 for IPv4 or
/128 for IPv6).
This fixes a deficiency in `CConnman::DisconnectNode(const CNetAddr& addr)`
and in `BanMan` which assume that creating a subnet from any address
using the `CSubNet(CNetAddr)` constructor would later match that address
only. Before this change a non-IP subnet would be invalid and would not
match any address.
06ba9b3008 rpc: move getpeerinfo connection_type help to correct place (Jon Atack)
c95fe6e38f gui: improve connection type tooltip (Hennadii Stepanov)
2c19ba2e1d gui: replace Direction with Connection Type in peer details (Jon Atack)
7e2beab2d2 gui: create GUIUtil::ConnectionTypeToQString utility function (Jon Atack)
Pull request description:
Closes#159.
ACKs for top commit:
hebasto:
re-ACK 06ba9b3008, the tooltip content is organized as unordered list.
jarolrod:
re-ACK 06ba9b3008
Tree-SHA512: 24f46494ceb42ed308e4a4f2a543dbc4f4e6409a6f738c145a9f16e175bf69d411cbc944a4fd969f1829d57644dfbc194182fa8d4e9e6bce82acbeca8c673748
fad327ca65 fuzz: net permission flags in net processing (MarcoFalke)
Pull request description:
to increase coverage
ACKs for top commit:
Crypt-iQ:
cr ACK fad327c
practicalswift:
ACK fad327ca65
Tree-SHA512: f8643d1774ff13524ab97ab228ad070489e080435e5742af26e6e325fd002e4c1fd78b9887e11622e79d6fe0c4daaddce5e033e6cd4b32e50fd68b434aab7333
faecb74562 Expose integral m_conn_type in CNodeStats, remove m_conn_type_string (Jon Atack)
Pull request description:
Currently, strings are stored for what are actually integral (strong) enum types. This is fine, because the strings are only used as-is for the debug log and RPC. However, it complicates using them in the GUI. User facing strings in the GUI should be translated and only string literals can be picked up for translation, not runtime `std::string`s.
Fix that by removing the `std::string` members and replace them by strong enum integral types.
ACKs for top commit:
jonatack:
Code review ACK faecb74562
theStack:
Code review ACK faecb74562🌲
Tree-SHA512: 24df2bd0645432060e393eb44b8abaf20fe296457d07a867b0e735c3e2e75af7b03fc6bfeca734ec33ab816a7c8e1f8591a5ec342f3afe3098a4e41f5c2cfebb
ef712298c3 util: Check for file being NULL in DirectoryCommit (Luke Dashjr)
4574904038 Fix possible data race when committing block files (Evan Klitzke)
220bb16cbe util: Introduce DirectoryCommit commit function to sync a directory (Evan Klitzke)
ce5cbaea63 util.h: Document FileCommit function (Evan Klitzke)
844d650eea util: Prefer Mac-specific F_FULLSYNC over fdatasync in FileCommit (Evan Klitzke)
f6cec0bcaf util: Refactor FileCommit from an #if sequence nested in #else, to a sequence of #elif (Evan Klitzke)
Pull request description:
Reviving #12696
ACKs for top commit:
laanwj:
Code review ACK ef712298c3
Tree-SHA512: 07d650990ef4c18d645dee3f9a199a940683ad17557d79d93979a76c4e710d8d70e6eae01d1a5991494a24a7654eb7db868be0c34a31e70b2509945d95bc9cce
a191e23b8e doc: Add release notes (Hennadii Stepanov)
ae749d12dd doc: Add libnatpmp stuff (Hennadii Stepanov)
e28f9be87a ci: Add libnatpmp-dev package to some builds (Hennadii Stepanov)
5a0185b6c9 gui: Add NAT-PMP network option (Hennadii Stepanov)
a39f7336a3 net: Add -natpmp command line option (Hennadii Stepanov)
28acffd9d5 net: Add NAT-PMP to port mapping loop (Hennadii Stepanov)
a8d9f275d0 net: Add libnatpmp support (Hennadii Stepanov)
58e8364dcd gui: Apply port mapping changes on dialog exit (Hennadii Stepanov)
cf151cc68c scripted-diff: Rename UPnP stuff (Hennadii Stepanov)
4e91b1e24d net: Add flags for port mapping protocols (Hennadii Stepanov)
8b50d1b5bb net: Keep trying to use UPnP when -upnp=1 (Hennadii Stepanov)
28e2961fd6 refactor: Replace magic number with named constant (Hennadii Stepanov)
02ccf69dd6 refactor: Move port mapping code to its own module (Hennadii Stepanov)
Pull request description:
Close#11902
This PR is an alternative to:
- #12288
- #15717
To compile with NAT-PMP support on Ubuntu [`libnatpmp-dev`](https://packages.ubuntu.com/source/bionic/libnatpmp) should be available.
Log excerpt:
```
2020-02-05T20:12:28Z [mapport] NAT-PMP: public address = 95.164.65.194
2020-02-05T20:12:28Z [mapport] AddLocal(95.164.65.194:18333,3)
2020-02-05T20:12:28Z [mapport] NAT-PMP: port mapping successful.
```
See: [`libnatpmp`](https://miniupnp.tuxfamily.org/libnatpmp.html)
---
Some follow-ups are out of this PR's scope:
- mention NAT-PMP library in the version message
- ~integrate NAT-PMP into the GUI~ (already [added](https://github.com/bitcoin/bitcoin/pull/18077#issuecomment-589405068))
ACKs for top commit:
laanwj:
Tested and code review ACK a191e23b8e
Tree-SHA512: 10e19267c21bf30f20ff1abfc882d526049f0e790b95e12f109dc2bed7c0aef45de03eaf967f4e667e7509be04f1873a5c508087393d947205f3aab2ad6d7cf1
Add a new RPC endpoint to enable opening outbound connections from
the tests. The functional test framework currently uses the addnode RPC, which
has different behavior than general outbound peers. These changes enable
creating both full-relay and block-relay-only connections. The new RPC
endpoint calls through to a newly introduced AddConnection method on
CConnman that ensures we stay within the allocated max.
9815332d51 test: Change MuHash Python implementation to match cpp version again (Fabian Jahr)
01297fb3ca fuzz: Add MuHash consistency fuzz test (Fabian Jahr)
b111410914 test: Add MuHash3072 fuzz test (Fabian Jahr)
c122527385 bench: Add Muhash benchmarks (Fabian Jahr)
7b1242229d test: Add MuHash3072 unit tests (Fabian Jahr)
adc708c98d crypto: Add MuHash3072 implementation (Fabian Jahr)
0b4d290bf5 crypto: Add Num3072 implementation (Fabian Jahr)
589f958662 build: Check for 128 bit integer support (Fabian Jahr)
Pull request description:
This is the first split of #18000 which implements the Muhash algorithm and uses it to calculate the UTXO set hash in `gettxoutsetinfo`.
ACKs for top commit:
laanwj:
Code review ACK 9815332d51
Tree-SHA512: 4bc090738f0e3d80b74bdd8122e24a8ce80121120fd37c7e4335a73e7ba4fcd7643f2a2d559e2eebf54b8e3a3bd5f12cfb27ba61ded135fda210a07a233eae45
eeee43bc48 fuzz: Use ConsumeWeakEnum for ServiceFlags (MarcoFalke)
fa9949b914 fuzz: Add ConsumeWeakEnum helper, Extract ALL_NET_PERMISSION_FLAGS (MarcoFalke)
faaef9434c fuzz: [refactor] Extract ALL_CONNECTION_TYPES constant (MarcoFalke)
fa42da2d54 fuzz: Use ConsumeNode in process_message target (MarcoFalke)
fa121f058f fuzz: Use ConsumeNode in process_messages target (MarcoFalke)
Pull request description:
The fuzz tests have several problems:
* The array passed to the fuzz engine to pick `net_permission_flags` is outdated
* The process_message* targets has the service flags as well as connection type hardcoded, limiting potential coverage
* The service flags deserialization from the fuzz engine doesn't allow for easy "exact matches". The fuzz engine has to explore a 64-bit space to hit an "exact match" (only one bit set)
Fix all issues in the commits in this pull
ACKs for top commit:
mzumsande:
ACK eeee43bc48 after rebase.
Tree-SHA512: 1ad9520c7e708b7f4994ae8f77886ffca33d7c542756e2a3e07dbbbe59e360f9fcaccf2e2fb57d9bc731d4aeb4938fb1c5c546e9d2744b007af5626f5cb377fe
195fcb53a0 qt: Follow Qt docs when implementing rowCount and columnCount (Hennadii Stepanov)
Pull request description:
[`QAbstractItemModel::rowCount`](https://doc.qt.io/qt-5/qabstractitemmodel.html#rowCount):
> **Note:** When implementing a table based model, `rowCount()` should return 0 when the parent is valid.
[`QAbstractItemModel::columnCount`](https://doc.qt.io/qt-5/qabstractitemmodel.html#columnCount):
> **Note:** When implementing a table based model, `columnCount()` should return 0 when the parent is valid.
ACKs for top commit:
jarolrod:
Tested ACK 195fcb53a0. Compiled and ran on macOS (Big Sur 11.1 and Catalina 10.15.7), Arch Linux, and FreeBSD. visually verified no weird effects with the `Address`, `Ban`, `Peer`, and `Transaction` tables. As already stated, the code change brings us inline with what the QT Docs recommend.
Tree-SHA512: 179a3430e68e77b22cdf642964cd96c023a2286ee256bbeb25b43df3d2eef6f59978c8d92173c6be5071d127fdcd6aa338142f6eaf003ff08e4abd65172d20ca
fa210689e2 net: Move SocketSendData lock annotation to header (MarcoFalke)
fa0a71781a net: Move CConnman/NetEventsInterface after CNode in header file (MarcoFalke)
Pull request description:
Lock annotations must be in the header, otherwise the will have limited or no effect
ACKs for top commit:
jnewbery:
utACK fa210689e2
Tree-SHA512: c25aac793662227f2bb163e2f98cd38e89b43a03bad925d192d6166dee76a456110eaf55be5e4b4f2ddcf20bde7e3fc82a7a4338670a1f2d027d01b4a8908303
90f9fc274b qt: Save QSplitter state in QSettings (Hennadii Stepanov)
Pull request description:
This PR adds the ability to save the `QSplitter` widget state in `QSettings` during shutdown, and restore it on startup.
A user no longer needs to adjust the splitter every time :)
ACKs for top commit:
jonasschnelli:
utACK 90f9fc274b
jonatack:
ACK 90f9fc274b this sets the "PeersTabSplitterSizes" value in the RPCConsole dtor and restores it in the RPCConsole ctor; tested in Debian with various split settings, tab open/close sequences, and shutdown methods, and the Peers window split state was faithfully maintained.
Tree-SHA512: efbd6a4cee512982944955d36775e75a8a217b1dc49e62d42c6e402d2710dd44324b2c3c1edeb5fe38d9229e0e4a39734d1f4e63405ade8694762e1bbf72020b
31b136e580 Don't declare de facto const reference variables as non-const (practicalswift)
1c65c075ee Don't declare de facto const member functions as non-const (practicalswift)
Pull request description:
_Meta: This is the second and final part of the `const` refactoring series (part one: #20581). **I promise: no more refactoring PRs from me in a while! :)** I'll now go back to focusing on fuzzing/hardening!_
Changes in this PR:
* Don't declare de facto const member functions as non-const
* Don't declare de facto const reference variables as non-const
Awards for finding candidates for the above changes go to:
* `clang-tidy`'s [`readability-make-member-function-const`](https://clang.llvm.org/extra/clang-tidy/checks/readability-make-member-function-const.html) check ([list of `clang-tidy` checks](https://clang.llvm.org/extra/clang-tidy/checks/list.html))
* `cppcheck`'s `constVariable` check ([list of `cppcheck` checks](https://sourceforge.net/p/cppcheck/wiki/ListOfChecks/))
See #18920 for instructions on how to analyse Bitcoin Core using Clang Static Analysis, `clang-tidy` and `cppcheck`.
ACKs for top commit:
ajtowns:
ACK 31b136e580
jonatack:
ACK 31b136e580
theStack:
ACK 31b136e580❄️
Tree-SHA512: f58f8f00744219426874379e9f3e9331132b9b48e954d24f3a85cbb858fdcc98009ed42ef7e7b4619ae8af9fc240a6d8bfc1c438db2e97b0ecd722a80dcfeffe
6f2c4fd077 netinfo: add user help documentation (Jon Atack)
Pull request description:
This is the help doc commit of #20764 without the rest of the PR or anything new since the 0.21.0 branch-off in order to target giving users a -netinfo help doc for 0.21.
- to test the new help
```
$ ./src/bitcoin-cli -netinfo help
```
- to see the updated short help
```
$ ./src/bitcoin-cli -help | grep -A4 netinfo
```
<details><summary><code>-netinfo</code> help doc</summary><p>
```
$ ./src/bitcoin-cli -netinfo help
-netinfo level "help"
Returns a network peer connections dashboard with information from the remote server.
Under the hood, -netinfo fetches the data by calling getpeerinfo and getnetworkinfo.
An optional integer argument from 0 to 4 can be passed for different peers listings.
Pass "help" to see this detailed help documentation.
If more than one argument is passed, only the first one is read and parsed.
Suggestion: use with the Linux watch(1) command for a live dashboard; see example below.
Arguments:
1. level (integer 0-4, optional) Specify the info level of the peers dashboard (default 0):
0 - Connection counts and local addresses
1 - Like 0 but with a peers listing (without address or version columns)
2 - Like 1 but with an address column
3 - Like 1 but with a version column
4 - Like 1 but with both address and version columns
2. help (string "help", optional) Print this help documentation instead of the dashboard.
Result:
* The peers listing in levels 1-4 displays all of the peers sorted by direction and minimum ping time:
Column Description
------ -----------
<-> Direction
"in" - inbound connections are those initiated by the peer
"out" - outbound connections are those initiated by us
type Type of peer connection
"full" - full relay, the default
"block" - block relay; like full relay but does not relay transactions or addresses
net Network the peer connected through ("ipv4", "ipv6", "onion", "i2p", or "cjdns")
mping Minimum observed ping time, in milliseconds (ms)
ping Last observed ping time, in milliseconds (ms)
send Time since last message sent to the peer, in seconds
recv Time since last message received from the peer, in seconds
txn Time since last novel transaction received from the peer and accepted into our mempool, in minutes
blk Time since last novel block passing initial validity checks received from the peer, in minutes
age Duration of connection to the peer, in minutes
asmap Mapped AS (Autonomous System) number in the BGP route to the peer, used for diversifying
peer selection (only displayed if the -asmap config option is set)
id Peer index, in increasing order of peer connections since node startup
address IP address and port of the peer
version Peer version and subversion concatenated, e.g. "70016/Satoshi:21.0.0/"
* The connection counts table displays the number of peers by direction, network, and the totals
for each, as well as a column for block relay peers.
* The local addresses table lists each local address broadcast by the node, the port, and the score.
Examples:
Connection counts and local addresses only
> bitcoin-cli -netinfo
Compact peers listing
> bitcoin-cli -netinfo 1
Full dashboard
> bitcoin-cli -netinfo 4
Full live dashboard, adjust --interval or --no-title as needed (Linux)
> watch --interval 1 --no-title ./src/bitcoin-cli -netinfo 4
See this help
> bitcoin-cli -netinfo help
```
</p></details>
ACKs for top commit:
laanwj:
ACK 6f2c4fd077
Tree-SHA512: dd49b1ce65546dacfb8ba9f9d57de0eae55560fd05533cf26c0b5d6ec65bf1de789c3287e90a0e2f47707532fab2fe62919a4192a7ffd58ac8eec18293e9aaeb
378aedc452 [net] Add cs_vSend lock annotations (John Newbery)
673254515a [net] Move RecordBytesSent() call out of cs_vSend lock (John Newbery)
Pull request description:
RecordBytesSent() does not require cs_vSend to be locked, so reduce the scope of cs_vSend.
Also correctly annotate the CNode data members that are guarded by cs_vSend.
This is a simpler alternative to #19673.
ACKs for top commit:
jnewbery:
ok, reverting to commit 378aedc which has two ACKs already. Any style issues can be fixed up in future PRs.
troygiorshev:
ACK 378aedc452
theStack:
re-ACK 378aedc452
MarcoFalke:
review ACK 378aedc452🔌
Tree-SHA512: e9cd6c472b7e1479120c1bf2d1c640cf6d18c7d589a5f9b7dfc4875e5790adaab403a7a1b945a47e79e7249a614b8583270e4549f89b22e8a9edb2e4818b0d07
This change (1) prevents overlapping date and amount strings,
and (2) guaranties that "eye" sign at the end of the watch-only
address/label is always visible.
0e51a35512 refactor: Use Mutex type for some mutexes in CNode class (Hennadii Stepanov)
Pull request description:
No need the `RecursiveMutex` type for the `CNode::cs_vSend`, `CNode::cs_hSocket` and `CNode::cs_vRecv`.
Related to #19303.
ACKs for top commit:
jnewbery:
utACK 0e51a35512
MarcoFalke:
review ACK 0e51a35512🔊
Tree-SHA512: 678ee5e3c15ad21a41cb86ec7179741bd505a138638fdc07f41d6d677c38fbf2208219bfc0509e3675e721fc8d8816e858070db7b87c5d72ad93aae81f7e1636
819d03b932 refactor: took out unused member functions (Zero)
ed69213c2b build: enable unused member function diagnostic (Zero)
Pull request description:
This PR enables the `-Wunused-member-function` compiler diagnostic, as discussed in #19702.
> **Notice**: The `unused-member-function` diagnostic is only available on clang. Therefore, clang should be used to test this PR.
- [x] Include the `-Wunused-member-function`diagnostic in `./configure.ac`. (ed69213c2b)
- [x] Resolve the reported warnings. (819d03b932)
Currently, enabling this flag no longer reports the following warnings:
> **Note**: output from `make 2>&1 | grep "warning: unused member function" | sort | uniq -c`
```
1 index/blockfilterindex.cpp:54:5: warning: unused member function 'DBHeightKey' [-Wunused-member-function]
2 script/bitcoinconsensus.cpp:50:9: warning: unused member function 'GetType' [-Wunused-member-function]
1 test/util_tests.cpp:1975:14: warning: unused member function 'operator=' [-Wunused-member-function]
```
All tests have passed locally (from `make check` & `src/test/test_bitcoin`).
This PR closes#19702.
ACKs for top commit:
practicalswift:
ACK 819d03b932 - patch still looks correct :)
MarcoFalke:
ACK 819d03b932
pox:
Tested ACK 819d03b932 with clang after `make clean`. No unused member function warnings.
theStack:
tested ACK 819d03b932
Tree-SHA512: 5fdfbbb02b3dc618a90a874a5caa5e01e596fc1d14a209e75a6981f01b253f9bca0cfac8fdd758dd7151986609fb76571c3745124a29cfd4f8cbb8d82a07272e
1112035d32 doc: fix various typos (Ikko Ashimine)
e8640849c7 doc: Use https URLs where possible (Sawyer Billings)
Pull request description:
Consolidates / fixes the changes from #20762, #20836, #20810. There is no output when `test/lint/lint-all.sh` is run.
Closes#20807.
ACKs for top commit:
MarcoFalke:
ACK 1112035d32
Tree-SHA512: 22ca824688758281a74e5ebc6a84a358142351434e34c88c6b36045d2d241ab95fd0958565fd2060f98317e62e683323b5320cc7ec13592bf340e6922294ed78
In 0.21, we added unbroadcast txids to mempool.dat. Commit 9c8a55d
added a try-block to prevent throwing a "failed to deserialize mempool data"
error when a user upgrades from 0.21 to 0.22. This exception handling is no
longer useful, so now we can remove it.
fa749fbea3 rpc: Replace boost::variant with std::variant for RPCArg.m_fallback (MarcoFalke)
Pull request description:
Now that we can use std::variant from the vanilla standard library, drop the third-party boost variant dependency.
Patch is split out from #20480. A step-by-step replacement is possible because we don't have our own `Variant` wrapper and the source code specifies `boost::variant` explicitly.
I think a step-by-step replacement should be preferred, because it simplifies review.
ACKs for top commit:
fjahr:
re-ACK fa749fbea3
Sjors:
re-ACK fa749fbea3
Tree-SHA512: 5e3c12b7d535f73065b4afa8df0a488f78fb25d2234f5ecbf740e624db03a34c35fea100eb7d37e84741721310e6450b7fb4296a2207a7ed1fa24485b3650981
fad140e311 test: Set correct nValue for multi-op-return policy check (MarcoFalke)
Pull request description:
`CTxOut::nValue` is default-initialized to `-1`. The dust-threshold for `OP_RETURN` outputs is `0`. Thus, the policy failure would be `dust` instead of `multi-op-return`. The test only passes because the dust check is currently not run.
Avoid that confusion by setting the value to `0`, to ensure the dust check passes.
ACKs for top commit:
theStack:
ACK fad140e311
Tree-SHA512: f0c7a68eb2c573d6595b2b129fa8fa2a34fa35c17691f448bf1c54ccf66059c37562e7480cde7b51c4de677038d7717873da4257147a5f60acc8bbcd25fb7e3f
efaf80e9bb fuzz: check that certain script TxoutType are nonstandard (Michael Dietz)
Pull request description:
- Every transaction of type NONSTANDARD must not be a standard script
- The only know types of nonstandard scripts are NONSTANDARD and certain NULL_DATA and MULTISIG scripts
When reviewing https://github.com/bitcoin/bitcoin/pull/20761 I figured this is very similar and might also be good to have
ACKs for top commit:
MarcoFalke:
ACK efaf80e9bb
Tree-SHA512: 6f563ee3104ea9d2633aad95f1d003474bea759d0f22636c37aa91b5536a6ff0800c42447285ca8ed12f1b3699bf781dae1e5e0a3362da578749cd3164a06ea4
86c495223f net: add CNode::IsInboundOnion() public getter and unit tests (Jon Atack)
6609eb8cb5 net: assert CNode::m_inbound_onion is inbound in ctor (Jon Atack)
993d1ecd19 test, fuzz: fix constructing CNode with invalid inbound_onion (Jon Atack)
Pull request description:
The goal of this PR is to be able to depend on `m_inbound_onion` in AttemptToEvictConnection in #20197:
- asserts `CNode::m_inbound_onion` is inbound in the CNode ctor to have a validity check at the class boundary
- fixes a unit test and a fuzz utility that were passing invalid inbound onion values to the CNode ctor
- drops an unneeded check in `CNode::ConnectedThroughNetwork()` for its inbound status
- adds a public getter `IsInboundOnion()` that also allows unit testing it
- adds unit test coverage
ACKs for top commit:
sipa:
utACK 86c495223f
LarryRuane:
ACK 86c495223f
vasild:
ACK 86c495223f
MarcoFalke:
review ACK 86c495223f🐍
Tree-SHA512: 21109105bc4e5e03076fadd489204be00eac710c9de0127708ca2d0a10a048ff81f640f589a7429967ac3eb51d35fe24bb2b12e53e7aa3efbc47aaff6396d204
fa0074e2d8 scripted-diff: Bump copyright headers (MarcoFalke)
Pull request description:
Needs to be done because no one has removed the years yet
ACKs for top commit:
practicalswift:
ACK fa0074e2d8
Tree-SHA512: 210e92acd7d400b556cf8259c3ec9967797420cfd19f0c2a4fa54cb2b3d32ad9ae27e771269201e7d554c0f4cd73a8b1c1a42c9f65d8685ca4d52e5134b071a3
4ddbcd0d9a fuzz: Add coverage for CDataStream consumer (practicalswift)
546a0764f3 fuzz: Fill various small fuzzing gaps (practicalswift)
Pull request description:
Fill various small fuzzing gaps.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK 4ddbcd0d9a
Tree-SHA512: d20f2cc0172f39948673846d088121782f39b4556df8b38fa14859cfa062c1519d18ee9601d4503ef1ba9613976cc5349c1fc0f0b9601a3d68127ffce1b1854e
454a4088a8 [doc] Add release notes for removed getpeerinfo fields. (Amiti Uttarwar)
b1a936d4ae [rpc] Remove deprecated "whitelisted" field from getpeerinfo (Amiti Uttarwar)
094c3beaa4 [rpc] Remove deprecated "banscore" field from getpeerinfo (Amiti Uttarwar)
537053336f [rpc] Remove deprecated "addnode" field from getpeerinfo (Amiti Uttarwar)
Pull request description:
This PR removes support for 3 fields on the `getpeerinfo` RPC that were deprecated in v0.21- `addnode`, `banscore` & `whitelisted`.
ACKs for top commit:
sipa:
utACK 454a4088a8
jnewbery:
ACK 454a4088a8.
Tree-SHA512: ccc0e90c0763eeb8529cf0c46162dbaca3f7773981b3b52d9925166ea7421aed086795d56b320e16c9340f68862388785f52a9b78314865070917b33180d7cd6
faccf8b1e1 refactor: Enable -Wswitch for FeeEstimateHorizon (MarcoFalke)
Pull request description:
This enables the `-Wswitch` compiler warning for `FeeEstimateHorizon` by removing the `default` case in `switch` statements.
ACKs for top commit:
practicalswift:
cr ACK faccf8b1e1
jonatack:
ACK faccf8b1e1
hebasto:
ACK faccf8b1e1, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 63a8dff6e8dead149ec2fa8319e7ff41022c9534d423d3086fd8f22be073dc4915f74c7fe9139ee681a8204730cf58c80ef40c93fb33032d586e68b4f78f557d
667d203687 [doc] Add permissions to the getpeerinfo help. (Amiti Uttarwar)
Pull request description:
This field was previously being returned, but missing from the RPCHelpMan. This PR uses the existing `NET_PERMISSIONS_DOC` to inform RPC users about this field.
```
"permissions" : [ (json array) Any special permissions that have been granted to this peer
"str", (string) bloomfilter (allow requesting BIP37 filtered blocks and transactions),
noban (do not ban for misbehavior; implies download),
forcerelay (relay transactions that are already in the mempool; implies relay),
relay (relay even in -blocksonly mode, and unlimited transaction announcements),
mempool (allow requesting BIP35 mempool contents),
download (allow getheaders during IBD, no disconnect after maxuploadtarget limit),
addr (responses to GETADDR avoid hitting the cache and contain random records with the most up-to-date info).
...
],
```
ACKs for top commit:
Sjors:
tACK 667d203687
Tree-SHA512: 973631b41d35d6333e3cb06b35277de869110f6ad6498c7e74f00c75202e8de1788a48755c21ac964903e5e6050a5e769a63866211aec9004cd665a727a54a3c
5a9ee0869b tests: Add fuzzing harness for node eviction logic (practicalswift)
Pull request description:
Add fuzzing harness for node eviction logic.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
cr ACK 5a9ee0869b
Tree-SHA512: c2401d22134867e23dab1ba94ae7ef36fdf52aa0588fdc4705d9cb765ddf979fd775fdf153ce2359f1bc1787cf60bf0ebcd47c7aa29c672e6a253fa58cac292d
c119ba3c9b [doc] clarify getdata limit after #14897 (Michael Polzer)
Pull request description:
GETDATA is limited to blocks and transactions now and can't be used for other non-block data
ACKs for top commit:
laanwj:
ACK c119ba3c9b
theStack:
ACK c119ba3c9b
benthecarman:
ACK c119ba3c9b
Tree-SHA512: d6e9c109bcce4ef004ec83a9ec591163279476524dec97ed5f5c34e322dca35af66a168f0878ff972bbcec79d81623903f3619fedf8f88cdced3f3f66a779173
fa26303286 fuzz: Check that NULL_DATA is unspendable (MarcoFalke)
Pull request description:
* Every script of type NULL_DATA must be unspendable
* The only know types of unspendable scripts are NULL_DATA and certain NONSTANDARD scripts
ACKs for top commit:
sipa:
utACK fa26303286
Tree-SHA512: 8297fbacf32b4868b12accc1c052d352d02d96540a1fc883de9d04a3df8734116deecc33046495c9a3af6d79fec7f8d63afbfa5e401a2ca8d7c70f0f13735c0d
66d012ad7f test: RPC: getblock fee calculations (Elliott Jin)
bf7d6e31b1 RPC: getblock: tx fee calculation for verbosity 2 via Undo data (Elliott Jin)
Pull request description:
This change is progress towards #18771 . It adapts the fee calculation part of #16083 and addresses some feedback. The additional "verbosity level 3" features are planned for a future PR.
**Original PR description:**
> Using block undo data (like in #14802) we can now show fee information for each transaction in a block without the need for additional -txindex and/or a ton of costly lookups. For a start we'll add transaction fee information to getblock verbosity level 2. This comes at a negligible speed penalty (<1%).
ACKs for top commit:
luke-jr:
tACK 66d012ad7f
fjahr:
tACK 66d012ad7f
MarcoFalke:
review ACK 66d012ad7f 🗜
Tree-SHA512: be1fe4b866946a8dc36427f7dc72a20e10860e320a28fa49bc85bd2a93a0d699768179be29fa52e18b2ed8505d3ec272e586753ef2239b4230e0aefd233acaa2
-dropmessagestest is a command line option that causes 1 in n received
messages to be dropped. The Bitcoin P2P protocol is stateful and in
general cannot handle messages being dropped. Dropped
version/verack/ping/pong messages will cause the connection to time out
and be torn down. Other dropped messages may also cause the peer to
believe that the peer has stalled and tear down the connection.
It seems difficult to uncover any actual issues with -dropmessagestest,
and any coverage that could be generated would probably be easier to
trigger with fuzz testing.
3002b4af2b [net processing] Guard m_continuation_block with m_block_inv_mutex (John Newbery)
184557e8e0 [net processing] Move hashContinue to net processing (John Newbery)
c853ef002e scripted-diff: rename vBlockHashesToAnnounce and vInventoryBlockToSend (John Newbery)
53b7ac1b7d [net processing] Move block inventory data to Peer (John Newbery)
78040f9168 [net processing] Rename nStartingHeight to m_starting_height (John Newbery)
77a2c2f8f9 [net processing] Move nStartingHeight to Peer (John Newbery)
717a374e74 [net processing] Improve documentation for Peer destruction/locking (John Newbery)
Pull request description:
This continues the work of moving application layer data into net_processing, by moving all block inventory state into the new Peer object added in #19607.
For motivation, see #19398.
ACKs for top commit:
jonatack:
re-ACK 3002b4af2b per `git diff 9aad3e4 3002b4a`
Sjors:
Code review re-ACK 3002b4af2b
MarcoFalke:
re-ACK 3002b4af2b🌓
Tree-SHA512: eb2b474b73b025791ee3e6e41809926b332b48468763219f31638ca390f427632f05902dfc6a2c6bdc1ce47b215782f67874ddbf05b97d77d5897b7e2abfe4d9
Upstream revision: 6d0488f75b/compiler-rt/include/fuzzer/FuzzedDataProvider.h
Changes:
* [compiler-rt] FuzzedDataProvider: add ConsumeData and method.
* [compiler-rt] Fix a typo in a comment in FuzzedDataProvider.h.
* [compiler-rt] Add ConsumeRandomLengthString() version without arguments.
* [compiler-rt] Refactor FuzzedDataProvider for better readability.
* [compiler-rt] FuzzedDataProvider: make linter happy.
* [compiler-rt] Mark FDP non-template methods inline to avoid ODR violations.
b23349b880 rpc: Add missing description of vout in getrawtransaction help text (Ben Carman)
Pull request description:
In `getrawtransaction` the vout did not have a description. I gave it the same description as the one used in `decoderawtransaction`.
ACKs for top commit:
MarcoFalke:
ACK b23349b880🏯
Tree-SHA512: 3833b97c82a46dfeb7ac825d4b2514b4b05ce54ac41f2144a8e2f2093b3411fe1d090c1e5b0c3d09200a2ea164c8d17ece12cdb43bbaeaeccc51a9da6dd7b7a3
fa4435e22f Replace boost::optional with std::optional (MarcoFalke)
fa7e803f3e Remove unused MakeOptional (MarcoFalke)
fadd4029dc psbt: Assert that tx has a value in UpdatePSBTOutput (MarcoFalke)
Pull request description:
Now that we can use std::optional from the vanilla standard library, drop the third-party boost dependency
ACKs for top commit:
practicalswift:
cr ACK fa4435e22f: patch looks correct!
laanwj:
code review ACK fa4435e22f
hebasto:
ACK fa4435e22f, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 50c5a1a130cac65e043e0177ba5b009fc2ba09343af4e23322ff2eb32184a55f8f2dea66e7a1b9d9acf56bc164eef4e47448750549a07f3b661199ac9bf9afef
This was previously done implicitly in boost::optional by BOOST_ASSERT.
Also, it was checked at runtime by valgrind if for some reason the
assert was disabled.
std::optional dereference won't assert, so add the Assert here
explicitly.
The explicit Assert also helps to document the code better.
a0a771843f contrib: Changes to checks for PowerPC64 (Luke Dashjr)
634f6ec4eb contrib: Parse ELF directly for symbol and security checks (Wladimir J. van der Laan)
Pull request description:
Instead of the ever-messier text parsing of the output of the readelf tool (which is clearly meant for human consumption not to be machine parseable), parse the ELF binaries directly.
Add a small dependency-less ELF parser specific to the checks.
This is slightly more secure, too, because it removes potential ambiguity due to misparsing and changes in the output format of `elfread`. It also allows for stricter and more specific ELF format checks in the future.
This removes the build-time dependency for `readelf`.
It passes the test-security-check for me locally, ~~though I haven't checked on all platforms~~. I've checked that this works on the cross-compile output for all ELF platforms supported by Bitcoin Core at the moment, as well as PPC64 LE and BE.
Top commit has no ACKs.
Tree-SHA512: 7f9241fec83ee512642fecf5afd90546964561efd8c8c0f99826dcf6660604a4db2b7255e1afb1e9bb0211fd06f5dbad18a6175dfc03e39761a40025118e7bfc
6d1e85f475 Clean up logging of outbound connection type (Suhas Daftuar)
Pull request description:
We have a function that converts `ConnectionType` enums to strings, so use it.
Suggested by ajtowns in https://github.com/bitcoin/bitcoin/pull/19858#discussion_r540791588
ACKs for top commit:
amitiuttarwar:
ACK 6d1e85f475
naumenkogs:
ACK 6d1e85f
Tree-SHA512: f5084d8b5257380696d9fde86a8873e190cd4553feb07fa49df39bbd9510bf5832d190a3bca1571c48370d16a17c7a34900857b21b27bec0777bfa710211d7bb
e71b656f31 qt: Align layout of checkboxes (Hennadii Stepanov)
Pull request description:
From https://github.com/bitcoin/bitcoin/issues/20555#issuecomment-747742627:
> 0.21.0rc3 compiled from source, Ubuntu 20.04 + i3: tested mainnet, testnet, and signet. created a descriptor wallet in qt, tested torv3 with addnode, and checked if anchors.dat is created on shutdown and removed on startup.
>
> I've noticed the "Advanced options" label being cut-off in the wallet creation dialog.
This PR should fix this issue.
ACKs for top commit:
MarcoFalke:
review ACK e71b656f31
Tree-SHA512: dcc997a440a96fc936296323afaf9f4040ac06cf5ad32724a88c2e2e16c726c35b1b2bacbb3e081839d8a69b5a6086b2570d6b2e98976f29b5427db2d434eee9
fae32f295c wallet: Add missing check for -descriptors wallet tool option (MarcoFalke)
faf8f61368 test: Add missing check for is_sqlite_compiled (MarcoFalke)
fa7dde1c41 wallet: Pass ArgsManager into ExecuteWalletToolFunc instead of using global (MarcoFalke)
Pull request description:
Also, fix a test failure when compiled without sqlite
ACKs for top commit:
ryanofsky:
Code review ACK fae32f295c. Thanks for implementing the -descriptors check and dealing with the test failure!
jonatack:
Code review utACK fae32f295c
Tree-SHA512: 3d7710694085822739a8316e4abc6db270799ca6ff6b0f9e5563ae240da65ae6a9cab7ba2647feae6ba540dac40b55b38ed41c8f6ed0bf02a3d1536284448927
23cac24dd3 tests: Test bitcoin-wallet dump and createfromdump (Andrew Chow)
a88c320041 wallettool: Add createfromdump command (Andrew Chow)
e1e7a90d5f wallettool: Add dump command (Andrew Chow)
Pull request description:
Adds two commands to the `bitcoin-wallet` tool: `dump` and `createfromdump`. These commands will be useful for a wallet storage migration in the future. It is also generally useful to have a storage agnostic dump like this. These commands are similar to BDB's `db_dump` and `db_load` tools. This can also be useful for manual construction of a wallet file for tests.
`dump` outputs every key-value pair from the wallet as comma separated hex. Each key-value pair is on its own line with the key and value in hex separated by a comma. This is output to the file specified by the new `-dumpfile` option.
`createfromdump` takes a file produced by `dump` and creates a new wallet file with exactly the records specified in that file.
A new option `-dumpfile` is added to the wallet tool. When used with `dump`, the records will be written to the specified file. When used with `createfromdump`, the file is read and the key-value pairs constructed from it. `createfromdump` requires `-dumpfile`.
A simple round-trip test is added to the `tool_wallet.py`.
This PR is based on #19334,
ACKs for top commit:
Sjors:
re-utACK 23cac24
MarcoFalke:
re review ACK 23cac24dd3 only change is rebase and removing useless shared_ptr wrapper 🎼
ryanofsky:
Code review ACK 23cac24dd3. Only changes since last review rebase and changing a pointer to a reference
Tree-SHA512: 2d63cf62baca3d16495aa698dc02f7d889c81b41015e9c92c23c275bb4a690fc176d351c3fd7f310bd6b17f5a936cc9be694cbecd702af741b96c0f530e72fa2
cc3044ccdb fix misleading comment about call to non-existing function (pox)
Pull request description:
The comment seems to be describing the subsequent call to `SyncTransaction` but refers to it as `SyncNotifications`, which is not any function currently in the codebase.
It's best to just remove the "what" aspect of the comment and focus on the "why", which also reduces the risk of similar documentation errors in the future, in case the function ever gets renamed, for example.
ACKs for top commit:
laanwj:
ACK cc3044ccdb
Tree-SHA512: 882ff17836ef585a603dc504f3dd21f56f682e49b28a0998f23fd16025826fbb083b7978db3ee70d0e0ff2c86fd6c3fd99a2361e5d45c765fdc5822c5f14c0a7
010eed3ce0 doc: warn that incoming conns are unlikely when not using default ports (Adam Jonas)
Pull request description:
Closes#5150.
This was mostly copied from #5285 by sulks, who has since quit GitHub.
The issue has remained open for 6 years, but the extra explanation still seems useful.
ACKs for top commit:
laanwj:
re-ACK 010eed3ce0
Tree-SHA512: d240fb06bba41ad8898ced59356c10adefc09f3abb33e277f8e2c5980b40678f2d237f286b476451bb29d2b94032a7dee2ada3b2efe004ed1c2509e70b48e40f
0c41c10830 doc: Remove shouty enums in net_processing comments (Suhas Daftuar)
Pull request description:
This uses the `CNode::ConnectionTypeAsString()` strings in place of the all-caps enums in a couple of comments in `net_processing`, as suggested by ajtowns in https://github.com/bitcoin/bitcoin/pull/19858#discussion_r540821050.
ACKs for top commit:
practicalswift:
ACK 0c41c10830
jnewbery:
ACK 0c41c10830
laanwj:
ACK 0c41c10830
Tree-SHA512: c8ab905e151ebb144c3f878277dc59d77591e4b39632658407b69b80b80d65825d5a391b01e2aea6af2fdf174c143dfe7d2f3eba84a020a58d7926458fdcd0a5
198fff88f3 GUI: Define MAX_DIGITS_BTC for magic number in BitcoinUnits::format (Luke Dashjr)
Pull request description:
A magic number snuck in with https://github.com/bitcoin/bitcoin/pull/16432
ACKs for top commit:
hebasto:
ACK 198fff88f3, I have reviewed the code and it looks OK, I agree it can be merged.
kristapsk:
utACK 198fff88f3
Tree-SHA512: 78dc23c2ae61bac41e5e34eebf57274599cb2ebb0b18d46e8a3228d42b256a1bc9bb17091c748f0f692ef1c4c241cfbd3e30a12bcd12222a234c1a9547ebe786
5021810650 Make CanFlushToDisk a const member function (practicalswift)
281cf99554 Do not run functions with necessary side-effects in assert() (practicalswift)
Pull request description:
Do not run functions with necessary side-effects in `assert()`.
ACKs for top commit:
laanwj:
Code review ACK 5021810650
sipa:
utACK 5021810650
theStack:
Code Review ACK 5021810650🟢
Tree-SHA512: 38b7faccc2f16a499f9b7b1b962b49eb58580b2a2bbf63ea49dcc418a5ecc8f21a0972fa953f66db9509c7239af67cfa2f9266423fd220963d091034d7332b96
ea36a453e3 [net] Make p2p recv buffer timeout 20 minutes for all peers (John Newbery)
Pull request description:
The timeout interval for the send and recv buffers was changed from 90
minutes to 20 minutes in commit f1920e86 in 2013, except for peers that
did not support the pong message (where the recv buffer timeout remained
at 90 minutes). A few observations:
- for peers that support BIP 31 (pong messages), this recv buffer
timeout is almost redundant with the ping timeout. We send a ping
message every two minutes, and set a timeout of twenty minutes to
receive the pong response. If the recv buffer was really timing out,
then the pong response would also time out.
- BIP 31 is supported by all nodes of p2p version 60000 and higher, and
has been in widespread use since 2013. I'd be very surprised if there
are many nodes on the network that don't support pong messages.
- The recv buffer timeout is not specified in any p2p BIP. We're free to
set it at any value we want.
- A peer that doesn't support BIP 31 and hasn't sent any message to us
at all in 90 minutes is unlikely to be useful for us, and is more likely
to be evicted AttemptToEvictConnection() since it'll have the worst
possible ping time and isn't providing blocks/transactions.
Therefore, we remove this check, and set the recv buffer timeout to 20
minutes for all peers. This removes the final p2p version dependent
logic from the net layer, so all p2p version data can move into the
net_processing layer.
Alternative approaches:
- Set the recv buffer timeout to 90 minutes for all peers. This almost
wouldn't be a behaviour change at all (pre-BIP 31 peers would still
have the same recv buffer timeout, and we can't ever reach a recv buffer
timeout higher than 21 minutes for post-BIP31 peers, because the pong
timeout would be hit first).
- Stop supporting peers that don't support BIP 31. BIP 31 has been in
use since 2012, and implementing it is trivial.
ACKs for top commit:
MarcoFalke:
review ACK ea36a453e3
promag:
Code review ACK ea36a453e3.
practicalswift:
cr ACK ea36a453e3: patch looks correct
ajtowns:
ACK ea36a453e3
sipa:
utACK ea36a453e3
jonatack:
Code review ACK ea36a453e3
Tree-SHA512: df290bb32d2b5d9e59a0125bb215baa92787f9d01542a7437245f1c478c7f9b9831e5f170d3cd0db2811e1b11b857b3e8b2e03376476b8302148e480d81aab19
89895773b7 Fix length of R check in test/key_tests.cpp:key_signature_tests (Dmitry Petukhov)
Pull request description:
The code before the fix only checked the length of R value of the last
signature in the loop, and only for equality (but the length can be
less than 32)
The fixed code checks that length of the R value is less than or equal
to 32 on each iteration of the loop
ACKs for top commit:
laanwj:
ACK 89895773b7
Tree-SHA512: 73eb2bb4a6c1c5fc11dd16851b28b43037ac06ef8cfc3b1f957429a1fca295c9422d67ec6c73c0e4bb4919f92e22dc5d733e84840b0d01ad1a529aa20d906ebf
37fe80e626 Only consider addrv2 peers for relay of non-addrv1 addresses (Pieter Wuille)
83f8821a6f refactor: add IsAddrCompatible() to CNode (Pieter Wuille)
Pull request description:
When selecting peers to relay an address to, only pick addrv2-capable ones if the address cannot be represented in addr(v1).
Without this I expect that propagation of torv3 addresses over the cleartext network will be very hard for a while.
ACKs for top commit:
jonatack:
ACK 37fe80e626
vasild:
ACK 37fe80e626
Tree-SHA512: 18a854ea43ad473cf89b9c5193b524109d7af75c26f7aa7e26cd72ad0db52f19c8001d566c607a7e6772bc314f770f09b6c3e07282d110c5daea193edc592cd2
173cc9b7be test: walettool create descriptors (Ivan Metlushko)
345e88eecf wallettool: add param to create descriptors wallet (Ivan Metlushko)
6d3af3ab62 wallettool: pass in DatabaseOptions into MakeWallet (Ivan Metlushko)
Pull request description:
Rationale: expose and promote descriptor wallets in more places; make cli tool more consistent with `createwallet` rpc.
Add `-descriptors` parameter which is off by default. When specified it will create a new descriptors wallet with sqlite backend, which is consistent with `createwallet` rpc.
This PR is based on a suggestion from **ryanofsky** https://github.com/bitcoin/bitcoin/pull/19137#discussion_r516779603
Example:
```
$ ./src/bitcoin-wallet -wallet=fewty -descriptors create
Topping up keypool...
Wallet info
===========
Name: fewty
Format: sqlite
Descriptors: yes
Encrypted: no
HD (hd seed available): yes
Keypool Size: 6000
Transactions: 0
Address Book: 0
```
```
$ ./src/bitcoin-wallet -wallet=fewty create
Topping up keypool...
Wallet info
===========
Name: fewty
Format: bdb
Descriptors: no
Encrypted: no
HD (hd seed available): yes
Keypool Size: 2000
Transactions: 0
Address Book: 0
```
ACKs for top commit:
achow101:
ACK 173cc9b7be
ryanofsky:
Code review ACK 173cc9b7be. This seems pretty nicely implemented now, with opportunities to clean up more and dedup later
MarcoFalke:
Concept ACK 173cc9b7be🌠
Tree-SHA512: cc32ba336ff709de2707ee15f495b4617908e8700ede8401a58e894f44cda485c544d644023c9a6604d88a62db9d92152383ee2e8abf691688c25cf6e222c622
cd03513dc2 init: Signal-safe instant shutdown (Wladimir J. van der Laan)
Pull request description:
Replace the 200ms polling loop with a faster and more efficient waiting operation. This should speed up short RPC tests.
This change has been tried a few times before, but abandoned every time because solutions used a condition variable which is not safe for use in signals, as they need to be reentrant.
On UNIX-ish OSes, use a safe way: a pipe. When shutdown is requested write a dummy byte to the pipe. Waiting for shutdown is a matter of a blocking read from the pipe.
On Windows, there are no signals so using a condition variable is safe.
This only affects bitcoind. The GUI is unaffected by this change, and keeps polling as before in `BitcoinGUI::detectShutdown()`. It might be possible to listen to a pipe there, too, but I'm not sure, and it's complicated by the GUI-node abstraction.
ACKs for top commit:
jonatack:
ACK cd03513dc2 tested on Debian 5.9.11-1 (2020-11-27) x86_64 GNU/Linux
Tree-SHA512: ed2f532f69fec4855c17bf7b8f3d0eb96e78ee2a3c13d374dd2c6add06e3ad6a190da8ed8f9d7a76532cf998222d67f57e35b206aec29675e96437448ae7e13c
fade6195b1 Move TX_MAX_STANDARD_VERSION to policy (MarcoFalke)
Pull request description:
`primitives` should only be used for the raw datastructures (parsing and format). It is not the right place to document relay policy.
ACKs for top commit:
laanwj:
Code review ACK fade6195b1
lontivero:
Concept ACK fade6195b1
Tree-SHA512: f809c4aecd14d7e9feaa7b50b9c0697232991eef36190cd960bcfb0ad6e20c71a4f6aab48c7747cf8a681eb14feda60c55b09a37f128673d519567224f29cd97
3e6657a14d Move signet onion seed from v2 to v3 (Sjors Provoost)
Pull request description:
Since v0.21 hidden services use the longer v3 address format.
It may make sense to backport this to the v0.21 branch, although onion nodes can always use the non-onion seeds.
ACKs for top commit:
MarcoFalke:
Concept ACK 3e6657a14d
laanwj:
ACK 3e6657a14d
Saibato:
tACK 3e6657a14d👍
Tree-SHA512: 9bb4d82345ab25d6ea971f8f106c30778ade2ba11a292e8d7449ea39581b91e51c4b34851b5c06f5dfbe07e7b04e5dc92c48c98c47a1c7ef3d5f350c3a3ad4f7
fa13e1b0c5 build: Add option --enable-danger-fuzz-link-all (MarcoFalke)
44444ba759 fuzz: Link all targets once (MarcoFalke)
Pull request description:
Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons:
* It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times
* It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files)
* It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons
* The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner
* It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file
* It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets
Fixes#20088
ACKs for top commit:
practicalswift:
Tested ACK fa13e1b0c5
sipa:
ACK fa13e1b0c5. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all
Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
fa86217e97 doc: Move add relay comment in net to correct place (MarcoFalke)
Pull request description:
The comment was previously attached to `m_addr_known`, but now it is attached to `id`, which is wrong.
Fix that by moving the comment to `RelayAddrsWithConn`.
ACKs for top commit:
practicalswift:
cr ACK fa86217e97: patch looks correct
jnewbery:
ACK fa86217e97
theStack:
Code review ACK fa86217e97🌳
Tree-SHA512: ec3d5f1996aded38947d2a5fd0bb63539e88f83964cd3254984002edfd51abb4dde813c7c81619a8a3a5c55b7e9ae83c8c5be8ad6c84b4593ed3bbf463fe8979
f7264fff0a Check if Cjdns address is valid (Lucas Ontivero)
Pull request description:
CJDNS addresses start with 0xFC and for that reason if a netaddr was unserialized with network type cjdns but its address prefix is not 0xFC then that netaddr should be considered invalid.
ACKs for top commit:
jonatack:
ACK f7264fff0a
practicalswift:
cr ACK f7264fff0a: patch looks correct
theStack:
ACK f7264fff0a✔️
Tree-SHA512: 5300df2ffbbd69c40271b6d8df96cca98eb3e1ee76aba62c9c76025d083788ab1f1332775890c63b06e02ca593863a867cd53956bce5962383e8450487898669
Replace the 200ms polling loop with a faster and more efficient waiting
operation.
This was tried a few times before, but given up every time because
solutions use a condition variable which is not safe for use in signals
as they need to be reentrant.
On UNIX-ish OSes, use a safe way: a pipe. When shutdown is requested
write a dummy byte to the pipe. Waiting for shutdown is a matter of a
blocking read from the pipe.
On Windows, there are no signals so using a condition variable is safe.
8c09c0c1d1 fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime() (practicalswift)
Pull request description:
Avoid time-based "non-determinism" in fuzzing harnesses by using mocked `GetTime()`.
Prior to this commit the fuzzing harnesses `banman`, `connman`, `net` and `rbf` had time-based "non-determinism". `addrman` is fixed in #20425. `process_message` and `process_messages` are left to fix: simply using mock time is not enough for them due to interaction with `IsInitialBlockDownload()`.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK 8c09c0c1d1
practicalswift:
> review ACK [8c09c0c](8c09c0c1d1)
Tree-SHA512: 32dfbead3dfd18cf4ff56dc2ea341aa977441b4e19a54879cf54fa5820c7e2b14b92c7e238d32fd785654f3b28cc82826ae66c03e94c292633c63c41196ba9a8
0475c8ba4d net: use std::chrono throughout maxOutbound logic (fanquake)
f805933e70 init: set nMaxOutboundLimit connection option directly (fanquake)
173d0d35f1 net: remove nMaxOutboundTimeframe from connection options (fanquake)
b117eb1486 net: remove SetMaxOutboundTimeframe (fanquake)
2f3f1aec1f net: remove SetMaxOutboundTarget (fanquake)
Pull request description:
Switch to using `std::chrono` types for the max outbound related logic.
Removes some unnecessary code from init.
ACKs for top commit:
jnewbery:
utACK 0475c8ba4d
MarcoFalke:
review ACK 0475c8ba4d🎭
Tree-SHA512: 5a6d5b61e0d4c08a235cfc0257dae65d09a5df019d8d230b1a58a3e2483ddf4a31efdefc885c4a02e4715e4180b0ed92ebc0a1c08b2bf476a391945114593514
03edb52eee qt: Remove redundant BitcoinGUI::setTrayIconVisible (Hennadii Stepanov)
17174f8328 gui: Replace "Hide tray icon" option with positive "Show tray icon" one (Hennadii Stepanov)
Pull request description:
This change makes easier both (1) using this option, and (2) reasoning about the code.
ACKs for top commit:
jonasschnelli:
utACK 03edb52eee
Tree-SHA512: 38e317492210d4fb13302dea383bd1f4f0ae1219d7ff2fdcb78607f15ac61a51969acaadb59b72c3f075b6356ef54368eb46fb49e6e1bd42db6d5804b97e232b
BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
which stage the `sendheaders` or `sendcmpct` messages should be sent.
Therefore we should tolerate them being sent before the version-verack
handshake is complete.
The timeout interval for the send and recv buffers was changed from 90
minutes to 20 minutes in commit f1920e86 in 2013, except for peers that
did not support the pong message (where the recv buffer timeout remained
at 90 minutes). A few observations:
- for peers that support BIP 31 (pong messages), this recv buffer
timeout is almost redundant with the ping timeout. We send a ping
message every two minutes, and set a timeout of twenty minutes to
receive the pong response. If the recv buffer was really timing out,
then the pong response would also time out.
- BIP 31 is supported by all nodes of p2p version 60000 and higher, and
has been in widespread use since 2013. I'd be very surprised if there
are many nodes on the network that don't support pong messages.
- The recv buffer timeout is not specified in any p2p BIP. We're free to
set it at any value we want.
- A peer that doesn't support BIP 31 and hasn't sent any message to us
at all in 90 minutes is unlikely to be useful for us, and is more likely
to be evicted AttemptToEvictConnection() since it'll have the worst
possible ping time and isn't providing blocks/transactions.
Therefore, we remove this check, and sent the recv buffer timeout to 20
minutes for all peers. This removes the final p2p version dependent
logic from the net layer, so all p2p version data can move into the
net_processing layer.
Alternative approaches:
- Set the recv buffer timeout to 90 minutes for all peers. This almost
wouldn't be a behaviour change at all (pre-BIP 31 peers would still
have the same recv buffer timeout, and we can't ever reach a recv buffer
timeout higher than 21 minutes for post-BIP31 peers, because the pong
timeout would be hit first).
- Stop supporting peers that don't support BIP 31. BIP 31 has been in
use since 2012, and implementing it is trivial.
7fabe0f359 net: don't relay to the address' originator (Vasil Dimov)
Pull request description:
For each address to be relayed we "randomly" pick 2 nodes to send the
address to (in `RelayAddress()`). However we do not take into
consideration that it does not make sense to relay the address back to
its originator (`CNode::PushAddress()` will do nothing in that case).
This means that if the originator is among the "randomly" picked nodes,
then we will relay to one node less than intended.
Fix this by skipping the originating node when choosing candidates to
relay to.
ACKs for top commit:
sdaftuar:
ACK 7fabe0f359 (this time I looked at the test, and verified the test breaks in expected ways if I break the code).
jnewbery:
utACK 7fabe0f359 (only net_processing changes. I haven't reviewed the test changes)
jonatack:
re-ACK 7fabe0f359 per `git range-diff b76abae fd897f8 7fabe0f`, change since last review is rebase and more readable Doxygen documentation
Tree-SHA512: c6a9d11c7afc97ab4e8960513f6416648d4a8c0c64b713c145a7482a7b9e54946f81386a3351e3ec0011e5594ba5ccff4d10c6f656bb80680d9f0d0a63366165
f6360088de [net processing] Clarify UpdatedBlockTip() (John Newbery)
94d2cc35be [net processing] Remove unnecesary nNewHeight variable in UpdatedBlockTip() (John Newbery)
8b57013473 [net processing] Remove nStartingHeight check from block relay (John Newbery)
Pull request description:
nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
0.2.9, P2P version 209) with the comment "better prevention of inventory
relaying during initial download". At that time, there was no function
to determine whether the node was still in Initial Block Download, so to
prevent syncing nodes from relaying old blocks to their peers, a check
was added to never relay a block to a peer where the height was lower
than 2000 less than the peer's best block. That check was updated
several times in later commits to ensure that we weren't relaying blocks
before the latest checkpoint if the peer didn't provide a
startingheight. The checkpoint comparison was changed to compare with an
estimate of the highest block in commit eae82d8e.
In commit 202e0194, all block relay was gated on being out of Initial
Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
removed since "we already checked IsIBD()".
We can remove the check against nStartingHeight entirely. If the node is
out of Initial Block Download, then its tip height must have been within
24 hours of current time, so should not be more than ~144 blocks behind
the most work tip.
This simplifies moving block inventory state into the `Peer` object (#19829).
ACKs for top commit:
Sjors:
utACK f636008
jonatack:
ACK f6360088de
MarcoFalke:
ACK f6360088de💽
ariard:
Code Review ACK f636008
Tree-SHA512: 4959cf35f1dcde46f34bffec1375729a157e1b2a1fd8a8ca33da9771c3c89a6c43e7050cdeeab8d90bb507b0795703db8c8bc304a1a5065ef00aae7a6992ca4f
4b7b58b3fe Update net_processing WTXID documentation per BIP339 (Jon Atack)
Pull request description:
BIP339 currently states:
*The wtxidrelay message MUST be sent in response to a version message from a peer whose protocol version is >= 70016 and prior to sending a verack. A wtxidrelay message received after a verack message MUST be ignored or treated as invalid.*
ACKs for top commit:
MarcoFalke:
ACK 4b7b58b3fe
practicalswift:
ACK 4b7b58b3fe
RiccardoMasutti:
ACK 4b7b58b
Tree-SHA512: 58ca6b197618cc73c70aa5de0a2d9d89a68b4cad9d5a708278ef17a9d6854d4362bcc384b6d29696642924977204a8fc120b31e91e2d97b6072b7b0d41c9f2dc
a33442fdc7 Remove m_is_manual_connection from CNodeState (Antoine Riard)
Pull request description:
Currently, this member is only used to exclude MANUAL peers from discouragement
in MaybePunishNodeForBlock(). Manual connections are already protected in
MaybeDiscourageAndDisconnect(), independently from their network
processing behaviors.
ACKs for top commit:
MarcoFalke:
cr ACK a33442fdc7
promag:
Code review ACK a33442fdc7.
jnewbery:
utACK a33442fdc7
amitiuttarwar:
code review ACK a33442fdc7
Tree-SHA512: cfe3f3dfa131373e3299002d34ae9e22ca6e1a966831bab32fcf06ff1d08f06095b4ab020cc4d267f3ec05ae23fbdc22373382ab828b999c0db11b8c842a4f0c
It's not actually possible to change this value, so remove the
indirection of it being a conn option.
DEFAULT_MAX_UPLOAD_TIMEFRAME is a compile time constant.
fac39c1983 wallet: document that tx in CreateTransaction is purely an out-param (MarcoFalke)
faac31521b Remove unused and confusing CTransaction constructor (MarcoFalke)
Pull request description:
The constructor is confusing and dangerous (as explained in the TODO), fix that by removing it.
ACKs for top commit:
laanwj:
Code review ACK fac39c1983
promag:
Code review ACK fac39c1983.
theStack:
Code review ACK fac39c1983
Tree-SHA512: e0c8cffce8d8ee0166b8e1cbfe85ed0657611e26e2af0d69fde70eceaa5d75cbde3eb489af0428fe4fc431360b4c791fb1cc21b8dee7d4c7a4f17df00836229d
faaad1bbac p2p: Ignore version msgs after initial version msg (MarcoFalke)
fad68afcff p2p: Ignore non-version msgs before version msg (MarcoFalke)
Pull request description:
Handshake misbehaviour doesn't cost us more than any other unknown message, so it seems odd to treat it differently
ACKs for top commit:
jnewbery:
utACK faaad1bbac
practicalswift:
ACK faaad1bbac: patch looks correct
Tree-SHA512: 9f30c3b5c1f6604fd02cff878f10999956152419a3dd9825f8267cbdeff7d06787418b41c7fde8a00a5e557fe89204546e05d5689042dbf7b07fbb7eb95cddff
f3d870fc22 wallet: List all wallets in non-SQLite or non-BDB builds (Russell Yanofsky)
d70dc89e78 refactor: Consolidate redundant wallet database path and exists functions (Russell Yanofsky)
6a7a63644c refactor: Drop call to GetWalletEnv in wallet salvage code (Russell Yanofsky)
6ee9cbdd18 refactor: Replace ListWalletDir() function with ListDatabases() (Russell Yanofsky)
5aaeb6cf87 MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir (Russell Yanofsky)
Pull request description:
This PR does not change behavior when bitcoin is built normally with both the SQLite and BDB libraries. It just makes non-SQLite and non-BDB builds more similar to the normal build. Specifically:
- It makes wallet directory lists always include all wallets so wallets don't appear missing depending on the build.
- It now triggers specific "Build does not support SQLite database format" and "Build does not support Berkeley DB database format" errors if a wallet can't be loaded instead of the more ambiguous and scary "Data is not in recognized format" error.
Both changes are implemented in the last commit. The previous commits are just refactoring cleanups that make the last commit possible and consolidate and reduce code.
ACKs for top commit:
achow101:
ACK f3d870fc22
promag:
Tested ACK f3d870fc22. Tested a --without-sqlite build with sqlite wallets.
Tree-SHA512: 029ad21559dbc338b5f351d05113c51bc25bce830f4f4e18bcd82287bc528275347a60249da65b91d252632aeb70b25d057bd59c704bfcaafb9f790bc5b59762
nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
0.2.9, P2P version 209) with the comment "better prevention of inventory
relaying during initial download". At that time, there was no function
to determine whether the node was still in Initial Block Download, so to
prevent syncing nodes from relaying old blocks to their peers, a check
was added to never relay a block to a peer where the height was lower
than 2000 less than the peer's best block. That check was updated
several times in later commits to ensure that we weren't relaying blocks
before the latest checkpoint if the peer didn't provide a
startingheight. The checkpoint comparison was changed to compare with an
estimate of the highest block in commit eae82d8e.
In commit 202e0194, all block relay was gated on being out of Initial
Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
removed since "we already checked IsIBD()".
We can remove the check against nStartingHeight entirely. If the node is
out of Initial Block Download, then its tip height must have been within
24 hours of current time, so should not be more than ~144 blocks behind
the most work tip.
b3a515c0be Clarify comments around outbound peer eviction (Suhas Daftuar)
daffaf03fb Periodically make block-relay connections and sync headers (Suhas Daftuar)
3cc8a7a0f5 Use conn_type to identify block-relay peers, rather than m_tx_relay == nullptr (Suhas Daftuar)
91d61952a8 Simplify and clarify extra outbound peer counting (Suhas Daftuar)
Pull request description:
To make eclipse attacks more difficult, regularly initiate outbound connections
and stay connected long enough to sync headers and potentially learn of new
blocks. If we learn a new block, rotate out an existing block-relay peer in
favor of the new peer.
This augments the existing outbound peer rotation that exists -- currently we
make new full-relay connections when our tip is stale, which we disconnect
after waiting a small time to see if we learn a new block. As block-relay
connections use minimal bandwidth, we can make these connections regularly and
not just when our tip is stale.
Like feeler connections, these connections are not aggressive; whenever our
timer fires (once every 5 minutes on average), we'll try to initiate a new
block-relay connection as described, but if we fail to connect we just wait for
our timer to fire again before repeating with a new peer.
ACKs for top commit:
ariard:
Code Review ACK b3a515c, only change since last time is dropping a useless `cs_main` taking. I manually tested a previous version of the PR, and not substantial change has been introduced since then which would alter behavior IMO.
jonatack:
Tested ACK b3a515c0be over several weeks, though this change and behavior could benefit from test coverage and other follow-ups (refactoring, etc.) described in the review feedback. I did not verify the behavior of `m_start_extra_block_relay_peers` only being enabled after initial chain sync. Since my last review, one unneeded `cs_main` lock was removed.
Tree-SHA512: 75fc6f8e8003e88e93f86b845caf2d30b8b9c0dbb0a6b8aabe4e24ea4f6327351f736a068a3b2720a8a581b789942a3a47f921e2afdb47e88bc50d078aa37b6f
Currently, this member is only used to exclude MANUAL peers from discouragement
in MaybePunishNodeForBlock(). Manual connections are already protected in
MaybeDiscourageAndDisconnect(), independently from their network
processing behaviors.
To make eclipse attacks more difficult, regularly initiate outbound connections
and stay connected long enough to sync headers and potentially learn of new
blocks. If we learn a new block, rotate out an existing block-relay peer in
favor of the new peer.
This augments the existing outbound peer rotation that exists -- currently we
make new full-relay connections when our tip is stale, which we disconnect
after waiting a small time to see if we learn a new block. As block-relay
connections use minimal bandwidth, we can make these connections regularly and
not just when our tip is stale.
Like feeler connections, these connections are not aggressive; whenever our
timer fires (once every 5 minutes on average), we'll try to initiate a new
block-relay connection as described, but if we fail to connect we just wait for
our timer to fire again before repeating with a new peer.
For each address to be relayed we "randomly" pick 2 nodes to send the
address to (in `RelayAddress()`). However we do not take into
consideration that it does not make sense to relay the address back to
its originator (`CNode::PushAddress()` will do nothing in that case).
This means that if the originator is among the "randomly" picked nodes,
then we will relay to one node less than intended.
Fix this by skipping the originating node when choosing candidates to
relay to.
fa0d8359b3 log: Clarify that failure to read fee_estimates.dat is non-fatal (MarcoFalke)
faefa5db5f log: Clarify that failure to write fee_estimates.dat is non-fatal (MarcoFalke)
Pull request description:
two minor logging fixups
ACKs for top commit:
practicalswift:
ACK fa0d8359b3: patch looks correct
laanwj:
Code review ACK fa0d8359b3
Tree-SHA512: d1e7e595d3b4a5e497ee7ab70f3be5783dafec2726ef8e012db836c15e8e622022859a4472d6b516fe19d327737b25fdfb509cd9aeb022ca847b13c54e55800a
0f949cde3d Add regression test for incorrect decoding (Pieter Wuille)
39c42c4420 Improve heuristic hex transaction decoding (Pieter Wuille)
Pull request description:
The current hex tx decoding logic will refuse to decode valid extended-encoded transactions if the result fails the heuristic sanity check, even when the legacy-encoding fails. Fix this.
Fixes#20579
ACKs for top commit:
achow101:
Code review ACK 0f949cde3d
jonatack:
Tested ACK 0f949cde3d
laanwj:
Code review ACK 0f949cde3d
Tree-SHA512: bd6dc80d824eb9a87026a623be910cac92173f8ce1c8b040c2246348c3cf0c6d64bcc40127b859e5e4da1efe88cf02a6945f7ebb91079799395145cb09d9c7a5
6fa72ceb80 test: add coverage for passing fee rate as a string (Jon Atack)
ce207d6b93 wallet, bugfix: allow send to take string fee rate values (Jon Atack)
Pull request description:
RPC send currently only accepts fee rates as numbers, which is a user-facing bug. It should accept fee rates as an amount, e.g. a string or a number, as documented in its help and like sendtoaddress, sendmany, fundrawtransaction, walletcreatefundedpsbt, and bumpfee. Provide a fix and regression test coverage.
ACKs for top commit:
MarcoFalke:
review ACK 6fa72ceb80
achow101:
Code review ACK 6fa72ceb80
promag:
Code review ACK 6fa72ceb80.
Tree-SHA512: 735f9269cb1b81953764b5283449c0b154bd62de034225be5bcedc515c84faf767fe8fe0741008679fe412922c847b00d116cb11aab775236b779c847ba87167
343dc4760f test: add test for high-bandwidth mode states in getpeerinfo (Sebastian Falbesoner)
dab6583307 doc: release note for new getpeerinfo fields "bip152_hb_{from,to}" (Sebastian Falbesoner)
a7ed00f8bb rpc: expose high-bandwidth mode states via getpeerinfo (Sebastian Falbesoner)
30bc8fab68 net: save high-bandwidth mode states in CNodeStats (Sebastian Falbesoner)
Pull request description:
Fixes#19676, "_For every peer expose through getpeerinfo RPC whether or not we selected them as HB peers, and whether or not they selected us as HB peers._" See [BIP152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki), in particular the [protocol flow diagram](https://github.com/bitcoin/bips/raw/master/bip-0152/protocol-flow.png). The newly introduced states are changed on the following places in the code:
* on reception of a `SENDCMPCT` message with valid version, the field `m_highbandwidth_from` is changed depending on the first integer parameter in the message (1=high bandwidth, 0=low bandwidth), i.e. it just mirrors the field `CNodeState.fPreferHeaderAndIDs`.
* after adding a `SENDCMPCT` message to the send queue, the field `m_highbandwidth_to` is changed depending on how the first integer parameter is set (same as above)
Note that after receiving `VERACK`, the node also sends `SENDCMPCT`, but that is only to announce the preferred version and never selects high-bandwidth mode, hence there is no need to change the state variables there, which are initialized to `false` anyways.
ACKs for top commit:
naumenkogs:
reACK 343dc4760f
jonatack:
re-ACK 343dc4760f per `git range-diff 7ea6499 4df1d12 343dc47`
Tree-SHA512: f4999e6a935266812c2259a9b5dc459710037d3c9e938006d282557cc225e56128f72965faffb207fc60c6531fab1206db976dd8729a69e8ca29d4835317b99f
34e33ab859 Remove g_relay_txes (John Newbery)
68334b3944 [net processing] Add m_ignores_incoming_txs to PeerManager and use internally (John Newbery)
4d510aa055 [init] Use MakeUnique<> to construct peerman (John Newbery)
f3f61d0eb9 [net processing] Add IgnoresIncomingTxs() function to PeerManager (John Newbery)
5805b8299f [net processing] Move PushNodeVersion into PeerManager (John Newbery)
Pull request description:
`g_relay_txes` is only required inside net_processing and is set only once at startup. Instead of having a global, move it to be a const member of PeerManager.
This requires moving `PushNodeVersion()` into `PeerManager`, which also allows us to remove the `connman` argument.
ACKs for top commit:
narula:
utACK 34e33ab859
MarcoFalke:
re-ACK 34e33ab85💐
Tree-SHA512: 33f75b522e5f34b243731932eb96cd6c8ce9db69b5186395e3718858bc715cec1711a663c6afc5880462812cbc15040930e2dc648b2acad6bc6502ad1397c5e3
Rewrite OutputGroups so that the logic is easier to follow and
understand.
There is a slight behavior change as OutputGroups will be grouped by
scriptPubKey rather than CTxDestination as before. This should have no
effect on users as all addresses are a CTxDestination. However by using
scriptPubKeys, we can correctly group outputs which fall into the
NoDestination case. But we also shouldn't have any NoDestination
outputs.
