22b44fc696 p2p: improve checkaddrman logging with duration in milliseconds (Jon Atack)
ec65bed00e log, timer: add LOG_TIME_MILLIS_WITH_CATEGORY_MSG_ONCE macro (Jon Atack)
325da75a53 log, timer: allow not repeating log message on completion (Jon Atack)
Pull request description:
This patch:
- updates the `logging/timer.h::Timer` class to allow not repeating the log message on completion
- adds a `LOG_TIME_MILLIS_WITH_CATEGORY_MSG_ONCE` macro that prints the descriptive message when logging the start but not when logging the completion
- updates the checkaddrman logging to log the duration, and renames the function like the `-checkaddrman` configuration option in order to prefix every log message with `CheckAddrman` instead of the longer, less pleasant, and different-from-checkaddrman `ForceCheckAddrman` (the Doxygen documentation on the function already makes clear that it is unaffected by `m_consistency_check_ratio`).
before
```
2021-09-21T18:42:50Z [opencon] Addrman checks started: new 64864, tried 1690, total 66554
2021-09-21T18:42:50Z [opencon] Addrman checks completed successfully
```
after
```
2021-09-21T18:42:50Z [opencon] CheckAddrman: new 64864, tried 1690, total 66554 started
2021-09-21T18:42:50Z [opencon] CheckAddrman: completed (76.21ms)
```
To test, build and run bitcoind with `-debug=addrman -checkaddrman=<n>` for a value of `n` in the range of, say, 10 to 40.
ACKs for top commit:
laanwj:
Code review ACK 22b44fc696
Tree-SHA512: 658c0dfaaa9d07092e2418f2d05007c58cc35be6593f22b3c592ce793334a885dd92dacc46bdeddc9d37939cf11174660a094c07c0fa117fbb282953aa45a94d
0fdb619aaf [validation] Always call mempool.check() after processing a new transaction (John Newbery)
2c64270bbe [refactor] Don't call AcceptToMemoryPool() from outside validation.cpp (John Newbery)
92a3aeecf6 [validation] Add CChainState::ProcessTransaction() (John Newbery)
36167faea9 [logging/documentation] Remove reference to AcceptToMemoryPool from error string (John Newbery)
4c24142b1e [validation] Remove comment about AcceptToMemoryPool() (John Newbery)
5759fd12b8 [test] Don't set bypass_limits to true in txvalidation_tests.cpp (John Newbery)
497c9e2964 [test] Don't set bypass_limits to true in txvalidationcache_tests.cpp (John Newbery)
Pull request description:
Similarly to how #18698 added `ProcessNewBlock()` and `ProcessNewBlockHeaders()` methods to the `ChainstateManager` class, this PR adds a new `ProcessTransaction()` method. Code outside validation no longer calls `AcceptToMemoryPool()` directly, but calls through the higher-level `ProcessTransaction()` method. Advantages:
- The interface is simplified. Calling code no longer needs to know about the active chainstate or mempool object, since `AcceptToMemoryPool()` can only ever be called for the active chainstate, and that chainstate knows which mempool it's using. We can also remove the `bypass_limits` argument, since that can only be used internally in validation.
- responsibility for calling `CTxMemPool::check()` is removed from the callers, and run automatically by `ChainstateManager` every time `ProcessTransaction()` is called.
ACKs for top commit:
lsilva01:
tACK 0fdb619 on Ubuntu 20.04
theStack:
Code-review ACK 0fdb619aaf
ryanofsky:
Code review ACK 0fdb619aaf. Only changes since last review: splitting & joining commits, adding more explanations to commit messages, tweaking MEMPOOL_ERROR string, fixing up argument name comments.
Tree-SHA512: 0b395c2e3ef242f0d41d47174b1646b0a73aeece38f1fe29349837e6fb832f4bf8d57e1a1eaed82a97c635cfd59015a7e07f824e0d7c00b2bee4144e80608172
aa1a4c9204 Add file validation to savemempool RPC test (lsilva01)
871e64d22f Add filename to savemempool RPC result (lsilva01)
Pull request description:
Currently, if the user calls the `savemempool` RPC method, there is no way to know
where the file was created (unless the user knows internal implementation details).
This PR adds a return message stating the file name and path where the mempool was saved and changes `mempool_persist.py` to validate this new return message.
ACKs for top commit:
laanwj:
Code review ACK aa1a4c9204
Tree-SHA512: e8b1dd0a8976e5eb15f7476c9651e492d2c621a67e0b726721fa7a2ae0ddd272ee28b87a2d0c650bd635e07fa96bdefe77bece4deb6486ef3ee9a4f83423a840
041abfebe4 test: MiniWallet: add P2TR support and use it per default (Sebastian Falbesoner)
4a2edf2bf7 test: generate blocks to MiniWallet address in rpc_blockchain.py (Sebastian Falbesoner)
Pull request description:
Taproot activates in [about 19 days](https://taproot.watch/) (2716 blocks), and it'd be nice if we set a good example and also support it in our MiniWallet. This PR changes the default mode from P2WSH (segwit v0 output, bech32 address) to P2TR (segwit v1 output, bech32m address) transactions type with the _anyone-can-spend_ policy, i.e. a witness script of `OP_TRUE`. The transition is actually quite painless, one only needs one extra piece in the form of a internal public key that is passed in the control block on the witness stack, in order to trigger script-path spending. To keep things simple, the lowest possible valid x-only-public key with the value of 1 was chosen as internal key.
Since many tests expect to find outputs for the default scriptPubKey of MiniWallet in the pre-mined chain of the test framework, the generation address is also changed from `ADDRESS_BCRT1_P2WSH_OP_TRUE` to `create_deterministic_address_bcrt1_p2tr_op_true()[0]` accordingly (see method `BitcoinTestFramework._initialize_chain(...)`). Note that the pre-mined chain is cached locally, so you probably have to delete the `./test/cache` folder first for the tests to pass again.
In order to avoid unnecessary renames, the import of `ADDRESS_BCRT1_P2WSH_OP_TRUE` is eliminated in rpc_blockchain.py by generating blocks directly to the MiniWallet address by using the `self.generate(self.wallet, ...)` interface (see first commit).
ACKs for top commit:
laanwj:
Code review ACK 041abfebe4
Tree-SHA512: 876a5b0595333f9c96c68d5ecf2b4530aee2715aebb75a953f4f75ca12258bd7239210fcfa1ae044bee91489804c9c2f2a6a335bd46c3ac701873d32e3a4f49d
The previous diff touched most files in ./test/, so bump the headers to
avoid having to touch them again for a bump later.
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./test/
-END VERIFY SCRIPT-
037c9ee79b fix `XOnlyPubKey::IsFullyValid` comment reference (Sebastian Falbesoner)
Pull request description:
The method name `CreatePayToContract` doesn't exist, very likely it was a (local) working title that was renamed to `CreateTapTweak` later.
ACKs for top commit:
michaelfolkson:
ACK 037c9ee79b
sipa:
ACK 037c9ee79b
Tree-SHA512: ab2b6ca93b66aba83c725b62f39e9f9316f3bea6f75ef35f66a2ac18b22a0a69ff2069cadab0365b28b0af2d30ad5ee3d3022194ac2024a1cdbb81d106fca0bb
14cd7bf793 [test] call CheckPackage for package sanitization checks (glozow)
6876378365 MOVEONLY: move package unit tests to their own file (glozow)
c9b1439ca9 MOVEONLY: mempool checks to their own functions (glozow)
9e910d8152 scripted-diff: clean up MemPoolAccept aliases (glozow)
fd92b0c398 document workspace members (glozow)
3d3e4598b6 [validation] cache iterators to mempool conflicts (glozow)
36a8441912 [validation/rpc] cache + use vsize calculated in PreChecks (glozow)
8fa2936b34 [validation] re-introduce bool for whether a transaction is RBF (glozow)
cbb3598b5c [validation/refactor] store precomputed txdata in workspace (glozow)
0a79eaba72 [validation] case-based constructors for ATMPArgs (glozow)
Pull request description:
This contains the refactors and moves within #22674. There are no behavior changes, so it should be simpler to review.
ACKs for top commit:
ariard:
Code Review ACK 14cd7bf
jnewbery:
Code review ACK 14cd7bf793
laanwj:
Code review ACK 14cd7bf793, thanks for adding documentation and clarifying the code
t-bast:
Code Review ACK 14cd7bf793
Tree-SHA512: 580ed48b43713a3f9d81cd9b573ef6ac44efe5df2fc7b7b7036c232b52952b04bf5ea92940cf73739f4fbd54ecf980cef58032e8a2efe05229ad0b3c639de8a0
79fd28cacb Adds verification step to Schnorr and ECDSA signing (amadeuszpawlik)
Pull request description:
As detailed in #22435, BIP340 defines that during Schnorr signing a verification should be done. This is so that potentially corrupt signage does not leak information about private keys used during the process. This is not followed today as no such verification step is being done. The same is valid for ECDSA signing functions `Sign` and `SignCompact`.
This PR adds this missing verification step to `SignSchnorr`, `Sign` and `SignCompact`.
ACKs for top commit:
sipa:
utACK 79fd28cacb
laanwj:
Code review ACK 79fd28cacb
theStack:
re-ACK 79fd28cacb
Tree-SHA512: 8fefa26caea577ae8631cc16c4e2f4cc6cfa1c7cf51d45a4a34165636ee290950617a17a19b4237c6f7a841db0e40fd5c36ad12ef43da82507c0e9fb9375ab82
bd9c6ade46 wallet: Fixed Grammatical error in bdb.h (zealsham)
Pull request description:
A comment in bdb.h file in the wallet directory contains a grammatical error that makes the underlying code harder to reason about .
The comment which says `/** Indicate the a new database user has began using the database. */` should actually be
`/** indicate that a new database user has began using the database. */`. The former is quite confusing , and leaves you wondering what "the a new database user " is refering to . This pull request thus provides value to the bitcoin codebase by improving readability .
Top commit has no ACKs.
Tree-SHA512: db07cda39f89ac344be3497c884a8c6e4b48276afae18e931a9a8e5732c58eed20645ccd18d6b1212c763f64b1300477c1de26a00b5b3b24e6141ffae3658ca7
The method name `CreatePayToContract` doesn't exist, very likely it was
a (local) working title that was renamed to `CreateTapTweak` later.
Also mention `CheckTapTweak`.
facc352648 test: Implicitly sync after generate*, unless opted out (MarcoFalke)
Pull request description:
The most frequent failure in functional tests are intermittent races. Fixing such bugs is cumbersome because it involves:
* Noticing the failure
* Fetching and reading the log to determine the test case that failed
* Adding a `self.sync_all()` where it was forgotten
* Spamming out a pr and waiting for review, which is already sparse
Also, writing a linter to catch those is not possible, nor is review effective in finding these bugs prior to merge.
Fix all future intermittent races caused by a missing sync_block call by calling `sync_all` implicitly after each `generate*`, unless opted out. This ensures that the code is race-free (with regards to blocks) when the tests pass once, instead of our current approach where the code can never be guaranteed to be race-free.
There are some scripted-diff cleanups (see https://github.com/bitcoin/bitcoin/pull/22567), but they will be submitted in a follow-up to reduce the conflicts in this pull.
ACKs for top commit:
lsilva01:
tACK facc352 on Ubuntu 20.04
brunoerg:
tACK facc352648 on MacOS 11.6
Tree-SHA512: 046a40a066b4a3bd28a3077bd654fa8887442dd1f0ec6fd11671865809ef02376f126eb667a1320ebd67b6e372c78c00dbf8bd25d86ed86f1d9a25363103ed97
ea360d69e5 doc: remove redundant optionals from build-unix.md (fanquake)
b97185812e doc: remove Boost build note from build-unix.md (fanquake)
Pull request description:
We don't have build instructions for any other dependency, and in any case, this isn't something we should maintain in our docs. If someone wants to know how to build Boost, or any other dependency, they can look at the relevant website / external build documentation.
ACKs for top commit:
jarolrod:
ACK ea360d69e5
Tree-SHA512: c7389b5f051f79c728d8ea0725143affeb2c35b1e3c20d5cd441c6ac540d230698c47bf2c57feb12e7b6bffd2f337c1c83a9cd3a928ce5c479cdbbdf8f61fba4
We don't do this for any other dependency, and users are better looking
at the actual Boost site/docs. This isn't something we should need to
have in our build docs.
90f1f849e9 doc: Suggest `keys.openpgp.org` as keyserver in SECURITY.md (Tim Ruffing)
Pull request description:
`--recv-keys` without a `--keyserver` arg simply failed for me on a fresh Arch Linux installation, so I think it's a good idea to suggest a keyserver. OpenPGP ecosystem is broken in a number of ways, so the right way to approach this issue has some potential for bikeshedding. But the only thing that this PR does is to keep `SECURITY.md` in line with the instructions for builder keys, where there was agreement on switching to `keys.openpgp.org` (#22688).
ACKs for top commit:
MarcoFalke:
review ACK 90f1f849e9
laanwj:
Review ACK 90f1f849e9
hebasto:
ACK 90f1f849e9, agree with arguments above.
Zero-1729:
ACK 90f1f849e9
Tree-SHA512: 1ab20c837cd952aa32b57473772cbfd33411a08db6e88b951bce38f76a3c509c0e91d6944ec0ca5eac8d5eb4d98a5489276d55691328f2e2556b2640f8e7c108
420695c193 contrib: recognize CJDNS seeds as such (Vasil Dimov)
f9c28330a0 net: take the first 4 random bits from CJDNS addresses in GetGroup() (Vasil Dimov)
29ff79c0a2 net: relay CJDNS addresses even if we are not connected to CJDNS (Vasil Dimov)
d96f8d304c net: don't skip CJDNS from GetNetworkNames() (Vasil Dimov)
c2d751abba net: take CJDNS into account in CNetAddr::GetReachabilityFrom() (Vasil Dimov)
9b43b3b257 test: extend feature_proxy.py to test CJDNS (Vasil Dimov)
508eb258fd test: remove default argument of feature_proxy.py:node_test() (Vasil Dimov)
6387f397b3 net: recognize CJDNS addresses as such (Vasil Dimov)
e6890fcb44 net: don't skip CJDNS from GetNetworksInfo() (Vasil Dimov)
e9d90d3c11 net: introduce a new config option to enable CJDNS (Vasil Dimov)
78f456c576 net: recognize CJDNS from ParseNetwork() (Vasil Dimov)
de01e312b3 net: use -proxy for connecting to the CJDNS network (Vasil Dimov)
aedd02ef27 net: make it possible to connect to CJDNS addresses (Vasil Dimov)
Pull request description:
CJDNS overview
=====
CJDNS is like a distributed, shared VPN with multiple entry points where every participant can reach any other participant. All participants use addresses from the `fc00::/8` network (reserved IPv6 range). Installation and configuration is done outside of applications, similarly to VPN (either in the host/OS or on the network router).
Motivation
=====
Even without this PR it is possible to connect two Bitcoin Core nodes through CJDNS manually by using e.g. `-addnode` in environments where CJDNS is set up. However, this PR is necessary for address relay to work properly and automatic connections to be made to CJDNS peers. I.e. to make CJDNS a first class citizen network like IPv4, IPv6, Tor and I2P.
Considerations
=====
An address from the `fc00::/8` network, could mean two things:
1. Part of a local network, as defined in RFC 4193. Like `10.0.0.0/8`. Bitcoin Core could be running on a machine with such address and have peers with those (e.g. in a local network), but those addresses are not relayed to other peers because they are not globally routable on the internet.
2. Part of the CJDNS network. This is like Tor or I2P - if we have connectivity to that network then we could reach such peers and we do relay them to other peers.
So, Bitcoin Core needs to be able to tell which one is it when it encounters a bare `fc00::/8` address, e.g. from `-externalip=` or by looking up the machine's own addresses. Thus a new config option is introduced `-cjdnsreacable`:
* `-cjdnsreacable=0`: it is assumed a `fc00::/8` address is a private IPv6 (1.)
* `-cjdnsreacable=1`: it is assumed a `fc00::/8` address is a CJDNS one (2.)
After setting up CJDNS outside of Bitcoin Core, a node operator only needs to enable this option.
Addresses from P2P relay/gossip don't need that because they are properly tagged as IPv6 or as CJDNS.
For testing
=====
```
[fc32:17ea:e415:c3bf:9808:149d:b5a2:c9aa]:8333
[fc68:7026:cb27:b014:5910:e609:dcdb:22a2]:8333
[fcb3:dc50:e1ae:7998:7dc0:7fa6:4582:8e46]:8333
[fcc7:be49:ccd1:dc91:3125:f0da:457d:8ce]:8333
[fcf2:d9e:3a25:4eef:8f84:251b:1b4d:c596]:8333
```
ACKs for top commit:
dunxen:
ACK 420695c
jonatack:
re-ACK 420695c193 per `git range-diff 23ae793 4fbff39 420695c`
laanwj:
Code review ACK 420695c193
Tree-SHA512: 21559886271aa84671d52b120fa3fa5a50fdcf0fcb26e5b32049c56fab0d606438d19dd366a9c8ce612d3894237ae6d552ead3338b326487e3534399b88a317a
fa93ef5a8a refactor: Take Span in SetSeed (MarcoFalke)
Pull request description:
This makes calling code less verbose and less fragile. Also, by adding
the CKey::data() member function, it is now possible to call HexStr()
with a CKey object.
ACKs for top commit:
sipa:
utACK fa93ef5a8a
laanwj:
Code review ACK fa93ef5a8a
theStack:
Code-review ACK fa93ef5a8a
Tree-SHA512: 73fb999320719ad4b9ab5544018a7a083d140545c2807ee3582ecf7f441040a30b5157e85790b6b840af82f002a7faf30bd8162ebba5caaf2067391c43dc7e25
095f07744c ci: Do not print `git log` for empty COMMIT_RANGE (Hennadii Stepanov)
Pull request description:
On master (77a2f5d30c) a CI lint task [log](https://api.cirrus-ci.com/v1/task/4817858858319872/logs/lint.log) exceeds 20K lines.
This PR fixes this issue.
ACKs for top commit:
MarcoFalke:
cr ACK 095f07744c
Tree-SHA512: 89180018aeccf1599cdf218924cbab12dcbae0f6674bb90e13b64e342cdd908a880b885039c23f0d1d03493e55a94fe04abf39481616ae6550c6a759f5ca9a35
a04350b86c Open streams_test_tmp file in temporary folder (Martin Leitner-Ankerl)
Pull request description:
The tests `streams_tests/streams_buffered_file` and `streams_tests/streams_buffered_file_rand`
did not use a the temporary directory provided by `BasicTestingSetup`, so it was not possible
to execute multiple of them in parallel. This fixes that.
To reproduce, run
```sh
parallel --halt now,fail=1 './src/test/test_bitcoin --run_test=streams_tests/streams_buffered_file_rand' -- ::: {1..1000}
```
This executes the test 1000 times, one job per CPU. It works on that commit but mergebase fails quickly.
ACKs for top commit:
theStack:
Tested ACK a04350b86c
Tree-SHA512: 705b127e99a0fdbf33362283cc94036ac3aeff364a5e75b0f0a2490114bbaa44cf310ac13d43c254a4b51b72d4e856e9ee584904ce56b2f496b92d995ac7dc24
15fb57556e ci: Enable ccache for "ARM64 Android APK" job (Hennadii Stepanov)
7a777ec98c build: Fix `make apk` if ccache enabled (Hennadii Stepanov)
Pull request description:
On master (456c8d6cd8) `make apk` is broken for Android targets if the build system was configured with `--enable-ccache` explicitly or by default.
This PR fixes this bug, and enables `ccache` for "ARM64 Android APK" Cirrus CI job.
ACKs for top commit:
fanquake:
ACK 15fb57556e
Tree-SHA512: e3c94dccc9f07a712e0e01983462ee607122623b213173c74d17e91024243239de212a35e5db94036bfd0a347e727b133865c26c67d6bd279604416ea971ab94
78e36700a0 doc: remove mention of system univalue (fanquake)
Pull request description:
Should have been part of #22646.
ACKs for top commit:
hebasto:
ACK 78e36700a0
Tree-SHA512: a5d54d73526033825ce4467cc3c57c26064739eef546556975a4c6f1f5bea84004640acd426734f90f98bc7a76ec837d716aa31167f2bdce7ee3887ad92e3152
7eb5b25e6e doc: Mention that BerkeleyDB is for legacy wallet in build-unix (W. J. van der Laan)
Pull request description:
This updates build-unix for the descriptor wallet, and prepares for eventual legacy wallet deprecation.
- Move 'descriptor wallet' dependencies above legacy wallet deps both for Debian and Fedora.
- Explicitly mention 'legacy wallet' where referring to the BerkeleyDB wallet. Shorten BerkeleyDB instruction to a single paragraph.
ACKs for top commit:
lsilva01:
ACK 7eb5b25
hebasto:
ACK 7eb5b25e6e, I have reviewed the code and it looks OK, I agree it can be merged.
Tree-SHA512: 8a0859bfaf170a1c5a8b3e8f873d6b788f3b1cebd8d006569348c5b415d5ebbdfd56a5a294c509114975addb599da8a50c4be618b0545ff402065e948f692ba1
bb1c84082c Remove the build_msvc/testconsensus project (Aaron Clauson)
Pull request description:
The testconsensus project is not integral to the Bitcoin Core code. It was originally added as a quick and dirty demo of how to do a consensus check with the msvc build. There are better examples.
PR #23438 made a change that caused a compiler error with the buildmsvc/testconsensus code. Rather than leave it hanging around to incur potential bitrot, or further PR build failures, it should be removed.
ACKs for top commit:
hebasto:
ACK bb1c84082c, tested on Windows 10 Pro (20H2).
Tree-SHA512: d81b99eb09171b66c179961b15f0b2e2e97e5ee7f011f18667e890c90e3d169593ad9aedd05a8616e962212952048827b7159d3c2a2ecb7ac378136b80bf6b23
