fade6195b1 Move TX_MAX_STANDARD_VERSION to policy (MarcoFalke)
Pull request description:
`primitives` should only be used for the raw datastructures (parsing and format). It is not the right place to document relay policy.
ACKs for top commit:
laanwj:
Code review ACK fade6195b1
lontivero:
Concept ACK fade6195b1
Tree-SHA512: f809c4aecd14d7e9feaa7b50b9c0697232991eef36190cd960bcfb0ad6e20c71a4f6aab48c7747cf8a681eb14feda60c55b09a37f128673d519567224f29cd97
faf2c6e32e cirrus: Schedule one task with paid credits for faster CI feedback (MarcoFalke)
Pull request description:
During times of high activity in the repo, the scheduling of Cirrus CI tasks might put them a few hours in the future. This is fine when all the tasks eventually pass. Though for failing tasks, a failure should ideally be shown to the author and reviewer as soon as possible.
Compute credits can be used to schedule immediately: https://cirrus-ci.org/pricing/#compute-credits. Running all tasks with compute credits will probably be more expensive than our previous CI invoice. However, they are also more flexible.
As a start we could enable only a single task and revisit/re-evaluate the next steps in a month.
ACKs for top commit:
laanwj:
Concept ACK faf2c6e32e
fanquake:
ACK faf2c6e32e
practicalswift:
cr ACK faf2c6e32e: patch looks correct
Tree-SHA512: df599e1c4cc0394f7f03413ad29954ddc87b163b02640d8bfc0497a5dc8d237b8c963c1dd9d01ac83c4a044f575300763097dac2ea6d1a4a163a1cece342b743
3e6657a14d Move signet onion seed from v2 to v3 (Sjors Provoost)
Pull request description:
Since v0.21 hidden services use the longer v3 address format.
It may make sense to backport this to the v0.21 branch, although onion nodes can always use the non-onion seeds.
ACKs for top commit:
MarcoFalke:
Concept ACK 3e6657a14d
laanwj:
ACK 3e6657a14d
Saibato:
tACK 3e6657a14d👍
Tree-SHA512: 9bb4d82345ab25d6ea971f8f106c30778ade2ba11a292e8d7449ea39581b91e51c4b34851b5c06f5dfbe07e7b04e5dc92c48c98c47a1c7ef3d5f350c3a3ad4f7
fa13e1b0c5 build: Add option --enable-danger-fuzz-link-all (MarcoFalke)
44444ba759 fuzz: Link all targets once (MarcoFalke)
Pull request description:
Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons:
* It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times
* It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files)
* It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons
* The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner
* It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file
* It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets
Fixes#20088
ACKs for top commit:
practicalswift:
Tested ACK fa13e1b0c5
sipa:
ACK fa13e1b0c5. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all
Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
fa86217e97 doc: Move add relay comment in net to correct place (MarcoFalke)
Pull request description:
The comment was previously attached to `m_addr_known`, but now it is attached to `id`, which is wrong.
Fix that by moving the comment to `RelayAddrsWithConn`.
ACKs for top commit:
practicalswift:
cr ACK fa86217e97: patch looks correct
jnewbery:
ACK fa86217e97
theStack:
Code review ACK fa86217e97🌳
Tree-SHA512: ec3d5f1996aded38947d2a5fd0bb63539e88f83964cd3254984002edfd51abb4dde813c7c81619a8a3a5c55b7e9ae83c8c5be8ad6c84b4593ed3bbf463fe8979
f7264fff0a Check if Cjdns address is valid (Lucas Ontivero)
Pull request description:
CJDNS addresses start with 0xFC and for that reason if a netaddr was unserialized with network type cjdns but its address prefix is not 0xFC then that netaddr should be considered invalid.
ACKs for top commit:
jonatack:
ACK f7264fff0a
practicalswift:
cr ACK f7264fff0a: patch looks correct
theStack:
ACK f7264fff0a✔️
Tree-SHA512: 5300df2ffbbd69c40271b6d8df96cca98eb3e1ee76aba62c9c76025d083788ab1f1332775890c63b06e02ca593863a867cd53956bce5962383e8450487898669
Replace the 200ms polling loop with a faster and more efficient waiting
operation.
This was tried a few times before, but given up every time because
solutions use a condition variable which is not safe for use in signals
as they need to be reentrant.
On UNIX-ish OSes, use a safe way: a pipe. When shutdown is requested
write a dummy byte to the pipe. Waiting for shutdown is a matter of a
blocking read from the pipe.
On Windows, there are no signals so using a condition variable is safe.
8c09c0c1d1 fuzz: Avoid time-based "non-determinism" in fuzzing harnesses by using mocked GetTime() (practicalswift)
Pull request description:
Avoid time-based "non-determinism" in fuzzing harnesses by using mocked `GetTime()`.
Prior to this commit the fuzzing harnesses `banman`, `connman`, `net` and `rbf` had time-based "non-determinism". `addrman` is fixed in #20425. `process_message` and `process_messages` are left to fix: simply using mock time is not enough for them due to interaction with `IsInitialBlockDownload()`.
See [`doc/fuzzing.md`](https://github.com/bitcoin/bitcoin/blob/master/doc/fuzzing.md) for information on how to fuzz Bitcoin Core. Don't forget to contribute any coverage increasing inputs you find to the [Bitcoin Core fuzzing corpus repo](https://github.com/bitcoin-core/qa-assets).
Happy fuzzing :)
ACKs for top commit:
MarcoFalke:
review ACK 8c09c0c1d1
practicalswift:
> review ACK [8c09c0c](8c09c0c1d1)
Tree-SHA512: 32dfbead3dfd18cf4ff56dc2ea341aa977441b4e19a54879cf54fa5820c7e2b14b92c7e238d32fd785654f3b28cc82826ae66c03e94c292633c63c41196ba9a8
0475c8ba4d net: use std::chrono throughout maxOutbound logic (fanquake)
f805933e70 init: set nMaxOutboundLimit connection option directly (fanquake)
173d0d35f1 net: remove nMaxOutboundTimeframe from connection options (fanquake)
b117eb1486 net: remove SetMaxOutboundTimeframe (fanquake)
2f3f1aec1f net: remove SetMaxOutboundTarget (fanquake)
Pull request description:
Switch to using `std::chrono` types for the max outbound related logic.
Removes some unnecessary code from init.
ACKs for top commit:
jnewbery:
utACK 0475c8ba4d
MarcoFalke:
review ACK 0475c8ba4d🎭
Tree-SHA512: 5a6d5b61e0d4c08a235cfc0257dae65d09a5df019d8d230b1a58a3e2483ddf4a31efdefc885c4a02e4715e4180b0ed92ebc0a1c08b2bf476a391945114593514
03edb52eee qt: Remove redundant BitcoinGUI::setTrayIconVisible (Hennadii Stepanov)
17174f8328 gui: Replace "Hide tray icon" option with positive "Show tray icon" one (Hennadii Stepanov)
Pull request description:
This change makes easier both (1) using this option, and (2) reasoning about the code.
ACKs for top commit:
jonasschnelli:
utACK 03edb52eee
Tree-SHA512: 38e317492210d4fb13302dea383bd1f4f0ae1219d7ff2fdcb78607f15ac61a51969acaadb59b72c3f075b6356ef54368eb46fb49e6e1bd42db6d5804b97e232b
836a3dc02c Avoid weak-linked getauxval support on non-linux platforms (like macOS) (Jonas Schnelli)
41a413b317 Define correct symbols for getauxval (Jonas Schnelli)
Pull request description:
PR #20358 made use of the two preprocessor symbols `HAVE_STRONG_GETAUXVAL` as well as `HAVE_WEAK_GETAUXVAL`.
These symbols have not been defined in configure.ac. They where only passed selective as CRC32 CPPFLAGS in https://github.com/bitcoin/bitcoin/blob/master/src/Makefile.crc32c.include#L16.
PR #20358 would have broken the macOS build since `getauxval` is not supported on macOS (but weak-linking does pass).
This PR defines the two symbols correctly and reduces calls to `getauxval` to linux.
ACKs for top commit:
laanwj:
Code review ACK 836a3dc02c
jonatack:
utACK 836a3dc02c
Tree-SHA512: 6527f4a617b937f4c368a3cb1c162f1ac38a6f5e6341295554961eaf322906e9b27398a6f7b00819854ceebb5c828d3e6ce0a779edd769adc4053ce8beda3739
b316dcb758 [net processing] Tolerate sendheaders and sendcmpct messages before verack (John Newbery)
Pull request description:
BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
which stage the `sendheaders` or `sendcmpct` messages should be sent.
Therefore we should tolerate them being sent before the version-verack
handshake is complete.
ACKs for top commit:
MarcoFalke:
review ACK b316dcb758📒
jonatack:
Code review re-ACK b316dcb758 per `git range-diff b103fdcb 82d0a43 b316dcb`, rebase only
luke-jr:
utACK b316dcb758, only code movement from after verack check to before
Tree-SHA512: a8da3990a786a53c195a33cd278eb9d1b6b09e9a33717674d5bc6ef5629811189216f3eda1a3dd11f6972b9680c7b7a532bf7343b6e7440009dd831bef128e1d
BIP 130 (sendheaders) and BIP 152 (compact blocks) do not specify at
which stage the `sendheaders` or `sendcmpct` messages should be sent.
Therefore we should tolerate them being sent before the version-verack
handshake is complete.
The timeout interval for the send and recv buffers was changed from 90
minutes to 20 minutes in commit f1920e86 in 2013, except for peers that
did not support the pong message (where the recv buffer timeout remained
at 90 minutes). A few observations:
- for peers that support BIP 31 (pong messages), this recv buffer
timeout is almost redundant with the ping timeout. We send a ping
message every two minutes, and set a timeout of twenty minutes to
receive the pong response. If the recv buffer was really timing out,
then the pong response would also time out.
- BIP 31 is supported by all nodes of p2p version 60000 and higher, and
has been in widespread use since 2013. I'd be very surprised if there
are many nodes on the network that don't support pong messages.
- The recv buffer timeout is not specified in any p2p BIP. We're free to
set it at any value we want.
- A peer that doesn't support BIP 31 and hasn't sent any message to us
at all in 90 minutes is unlikely to be useful for us, and is more likely
to be evicted AttemptToEvictConnection() since it'll have the worst
possible ping time and isn't providing blocks/transactions.
Therefore, we remove this check, and sent the recv buffer timeout to 20
minutes for all peers. This removes the final p2p version dependent
logic from the net layer, so all p2p version data can move into the
net_processing layer.
Alternative approaches:
- Set the recv buffer timeout to 90 minutes for all peers. This almost
wouldn't be a behaviour change at all (pre-BIP 31 peers would still
have the same recv buffer timeout, and we can't ever reach a recv buffer
timeout higher than 21 minutes for post-BIP31 peers, because the pong
timeout would be hit first).
- Stop supporting peers that don't support BIP 31. BIP 31 has been in
use since 2012, and implementing it is trivial.
7fabe0f359 net: don't relay to the address' originator (Vasil Dimov)
Pull request description:
For each address to be relayed we "randomly" pick 2 nodes to send the
address to (in `RelayAddress()`). However we do not take into
consideration that it does not make sense to relay the address back to
its originator (`CNode::PushAddress()` will do nothing in that case).
This means that if the originator is among the "randomly" picked nodes,
then we will relay to one node less than intended.
Fix this by skipping the originating node when choosing candidates to
relay to.
ACKs for top commit:
sdaftuar:
ACK 7fabe0f359 (this time I looked at the test, and verified the test breaks in expected ways if I break the code).
jnewbery:
utACK 7fabe0f359 (only net_processing changes. I haven't reviewed the test changes)
jonatack:
re-ACK 7fabe0f359 per `git range-diff b76abae fd897f8 7fabe0f`, change since last review is rebase and more readable Doxygen documentation
Tree-SHA512: c6a9d11c7afc97ab4e8960513f6416648d4a8c0c64b713c145a7482a7b9e54946f81386a3351e3ec0011e5594ba5ccff4d10c6f656bb80680d9f0d0a63366165
f6360088de [net processing] Clarify UpdatedBlockTip() (John Newbery)
94d2cc35be [net processing] Remove unnecesary nNewHeight variable in UpdatedBlockTip() (John Newbery)
8b57013473 [net processing] Remove nStartingHeight check from block relay (John Newbery)
Pull request description:
nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
0.2.9, P2P version 209) with the comment "better prevention of inventory
relaying during initial download". At that time, there was no function
to determine whether the node was still in Initial Block Download, so to
prevent syncing nodes from relaying old blocks to their peers, a check
was added to never relay a block to a peer where the height was lower
than 2000 less than the peer's best block. That check was updated
several times in later commits to ensure that we weren't relaying blocks
before the latest checkpoint if the peer didn't provide a
startingheight. The checkpoint comparison was changed to compare with an
estimate of the highest block in commit eae82d8e.
In commit 202e0194, all block relay was gated on being out of Initial
Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
removed since "we already checked IsIBD()".
We can remove the check against nStartingHeight entirely. If the node is
out of Initial Block Download, then its tip height must have been within
24 hours of current time, so should not be more than ~144 blocks behind
the most work tip.
This simplifies moving block inventory state into the `Peer` object (#19829).
ACKs for top commit:
Sjors:
utACK f636008
jonatack:
ACK f6360088de
MarcoFalke:
ACK f6360088de💽
ariard:
Code Review ACK f636008
Tree-SHA512: 4959cf35f1dcde46f34bffec1375729a157e1b2a1fd8a8ca33da9771c3c89a6c43e7050cdeeab8d90bb507b0795703db8c8bc304a1a5065ef00aae7a6992ca4f
4b7b58b3fe Update net_processing WTXID documentation per BIP339 (Jon Atack)
Pull request description:
BIP339 currently states:
*The wtxidrelay message MUST be sent in response to a version message from a peer whose protocol version is >= 70016 and prior to sending a verack. A wtxidrelay message received after a verack message MUST be ignored or treated as invalid.*
ACKs for top commit:
MarcoFalke:
ACK 4b7b58b3fe
practicalswift:
ACK 4b7b58b3fe
RiccardoMasutti:
ACK 4b7b58b
Tree-SHA512: 58ca6b197618cc73c70aa5de0a2d9d89a68b4cad9d5a708278ef17a9d6854d4362bcc384b6d29696642924977204a8fc120b31e91e2d97b6072b7b0d41c9f2dc
a33442fdc7 Remove m_is_manual_connection from CNodeState (Antoine Riard)
Pull request description:
Currently, this member is only used to exclude MANUAL peers from discouragement
in MaybePunishNodeForBlock(). Manual connections are already protected in
MaybeDiscourageAndDisconnect(), independently from their network
processing behaviors.
ACKs for top commit:
MarcoFalke:
cr ACK a33442fdc7
promag:
Code review ACK a33442fdc7.
jnewbery:
utACK a33442fdc7
amitiuttarwar:
code review ACK a33442fdc7
Tree-SHA512: cfe3f3dfa131373e3299002d34ae9e22ca6e1a966831bab32fcf06ff1d08f06095b4ab020cc4d267f3ec05ae23fbdc22373382ab828b999c0db11b8c842a4f0c
It's not actually possible to change this value, so remove the
indirection of it being a conn option.
DEFAULT_MAX_UPLOAD_TIMEFRAME is a compile time constant.
fac39c1983 wallet: document that tx in CreateTransaction is purely an out-param (MarcoFalke)
faac31521b Remove unused and confusing CTransaction constructor (MarcoFalke)
Pull request description:
The constructor is confusing and dangerous (as explained in the TODO), fix that by removing it.
ACKs for top commit:
laanwj:
Code review ACK fac39c1983
promag:
Code review ACK fac39c1983.
theStack:
Code review ACK fac39c1983
Tree-SHA512: e0c8cffce8d8ee0166b8e1cbfe85ed0657611e26e2af0d69fde70eceaa5d75cbde3eb489af0428fe4fc431360b4c791fb1cc21b8dee7d4c7a4f17df00836229d
faaad1bbac p2p: Ignore version msgs after initial version msg (MarcoFalke)
fad68afcff p2p: Ignore non-version msgs before version msg (MarcoFalke)
Pull request description:
Handshake misbehaviour doesn't cost us more than any other unknown message, so it seems odd to treat it differently
ACKs for top commit:
jnewbery:
utACK faaad1bbac
practicalswift:
ACK faaad1bbac: patch looks correct
Tree-SHA512: 9f30c3b5c1f6604fd02cff878f10999956152419a3dd9825f8267cbdeff7d06787418b41c7fde8a00a5e557fe89204546e05d5689042dbf7b07fbb7eb95cddff
f3d870fc22 wallet: List all wallets in non-SQLite or non-BDB builds (Russell Yanofsky)
d70dc89e78 refactor: Consolidate redundant wallet database path and exists functions (Russell Yanofsky)
6a7a63644c refactor: Drop call to GetWalletEnv in wallet salvage code (Russell Yanofsky)
6ee9cbdd18 refactor: Replace ListWalletDir() function with ListDatabases() (Russell Yanofsky)
5aaeb6cf87 MOVEONLY: Move IsBDBFile, IsSQLiteFile, and ListWalletDir (Russell Yanofsky)
Pull request description:
This PR does not change behavior when bitcoin is built normally with both the SQLite and BDB libraries. It just makes non-SQLite and non-BDB builds more similar to the normal build. Specifically:
- It makes wallet directory lists always include all wallets so wallets don't appear missing depending on the build.
- It now triggers specific "Build does not support SQLite database format" and "Build does not support Berkeley DB database format" errors if a wallet can't be loaded instead of the more ambiguous and scary "Data is not in recognized format" error.
Both changes are implemented in the last commit. The previous commits are just refactoring cleanups that make the last commit possible and consolidate and reduce code.
ACKs for top commit:
achow101:
ACK f3d870fc22
promag:
Tested ACK f3d870fc22. Tested a --without-sqlite build with sqlite wallets.
Tree-SHA512: 029ad21559dbc338b5f351d05113c51bc25bce830f4f4e18bcd82287bc528275347a60249da65b91d252632aeb70b25d057bd59c704bfcaafb9f790bc5b59762
Previously, the compression of the .iso file to a .dmg file was done
outside of `make deploy' in order to use the faketime-wrapped version of
libdmg-hfsplus's DMG tool.
Specifying the faketime-wrapped version of the DMG tool to ./configure
fixes this and simplifies build scripts.
nStartingHeight was introduced in commit 7a47324c7 (Bitcoin version
0.2.9, P2P version 209) with the comment "better prevention of inventory
relaying during initial download". At that time, there was no function
to determine whether the node was still in Initial Block Download, so to
prevent syncing nodes from relaying old blocks to their peers, a check
was added to never relay a block to a peer where the height was lower
than 2000 less than the peer's best block. That check was updated
several times in later commits to ensure that we weren't relaying blocks
before the latest checkpoint if the peer didn't provide a
startingheight. The checkpoint comparison was changed to compare with an
estimate of the highest block in commit eae82d8e.
In commit 202e0194, all block relay was gated on being out of Initial
Block Download. In commit 0278fb5f, the comparison to nBlockEstimate was
removed since "we already checked IsIBD()".
We can remove the check against nStartingHeight entirely. If the node is
out of Initial Block Download, then its tip height must have been within
24 hours of current time, so should not be more than ~144 blocks behind
the most work tip.
b3a515c0be Clarify comments around outbound peer eviction (Suhas Daftuar)
daffaf03fb Periodically make block-relay connections and sync headers (Suhas Daftuar)
3cc8a7a0f5 Use conn_type to identify block-relay peers, rather than m_tx_relay == nullptr (Suhas Daftuar)
91d61952a8 Simplify and clarify extra outbound peer counting (Suhas Daftuar)
Pull request description:
To make eclipse attacks more difficult, regularly initiate outbound connections
and stay connected long enough to sync headers and potentially learn of new
blocks. If we learn a new block, rotate out an existing block-relay peer in
favor of the new peer.
This augments the existing outbound peer rotation that exists -- currently we
make new full-relay connections when our tip is stale, which we disconnect
after waiting a small time to see if we learn a new block. As block-relay
connections use minimal bandwidth, we can make these connections regularly and
not just when our tip is stale.
Like feeler connections, these connections are not aggressive; whenever our
timer fires (once every 5 minutes on average), we'll try to initiate a new
block-relay connection as described, but if we fail to connect we just wait for
our timer to fire again before repeating with a new peer.
ACKs for top commit:
ariard:
Code Review ACK b3a515c, only change since last time is dropping a useless `cs_main` taking. I manually tested a previous version of the PR, and not substantial change has been introduced since then which would alter behavior IMO.
jonatack:
Tested ACK b3a515c0be over several weeks, though this change and behavior could benefit from test coverage and other follow-ups (refactoring, etc.) described in the review feedback. I did not verify the behavior of `m_start_extra_block_relay_peers` only being enabled after initial chain sync. Since my last review, one unneeded `cs_main` lock was removed.
Tree-SHA512: 75fc6f8e8003e88e93f86b845caf2d30b8b9c0dbb0a6b8aabe4e24ea4f6327351f736a068a3b2720a8a581b789942a3a47f921e2afdb47e88bc50d078aa37b6f
e1e68b6305 test: Fix inconsistent lock order in wallet_tests/CreateWallet (Hennadii Stepanov)
cb23fe01c1 [skip ci] sync: Check precondition in LEAVE_CRITICAL_SECTION() macro (Hennadii Stepanov)
c5e3e74f70 sync: Improve CheckLastCritical() (Hennadii Stepanov)
Pull request description:
This PR:
- fixes#19049 that was caused by #16426
- removes `wallet_tests::CreateWallet` suppression from the `test/sanitizer_suppressions/tsan`
The example of the improved `CheckLastCritical()`/`LEAVE_CRITICAL_SECTION()` log (could be got when compiled without the last commit):
```
2020-09-20T08:34:28.429485Z [test] INCONSISTENT LOCK ORDER DETECTED
2020-09-20T08:34:28.429493Z [test] Current lock order (least recent first) is:
2020-09-20T08:34:28.429501Z [test] 'walletInstance->cs_wallet' in wallet/wallet.cpp:4007 (in thread 'test')
2020-09-20T08:34:28.429508Z [test] 'cs_wallets' in wallet/wallet.cpp:4089 (in thread 'test')
```
Currently, there are other "naked" `LEAVE_CRITICAL_SECTION()` in the code base:
b99a1633b2/src/rpc/mining.cpp (L698)b99a1633b2/src/checkqueue.h (L208)
ACKs for top commit:
MarcoFalke:
review ACK e1e68b6305💂
ryanofsky:
Code review ACK e1e68b6305. Just trivial rebase and suggested switch to BOOST_CHECK_EXCEPTION since last review
vasild:
ACK e1e68b630
Tree-SHA512: a627680eac3af4b4c02772473d68322ce8d3811bf6b035d3485ccc97d35755bef933cffabd3f20b126f89e3301eccecec3f769df34415fb7c426c967b6ce36e6
The current FreeBSD Build documentation is a little outdated and underwhelming. This PR updates the doc to
be more informative. It also adds new instructions for building the GUI and adding support for descriptor
wallets.
on vasild's recommendation: it is ok to point the user to download db5 which has an active port package
instead of building db4 from the provided script.
Co-authored-by: Vasil Dimov <vd@freebsd.org>
Currently, this member is only used to exclude MANUAL peers from discouragement
in MaybePunishNodeForBlock(). Manual connections are already protected in
MaybeDiscourageAndDisconnect(), independently from their network
processing behaviors.
