fafe896a0b test: Set regtest.BIP66Height = 102 to speed up tests (MarcoFalke)
Pull request description:
No need to waste time by forcing creation of more than 1000 blocks to get the benefits of being able to test BIP 66. Also, reducing the height makes it more likely that (third-party) tests are conforming to BIP 66, which is enforced on mainnet for all new blocks.
ACKs for top commit:
GeneFerneau:
Concept + code review ACK [fafe896](fafe896a0b)
0xB10C:
crACK fafe896a0b
laanwj:
ACK fafe896a0b
Zero-1729:
tACK fafe896
kristapsk:
ACK fafe896a0b. Full functional test suite showed few second speed incrase on my laptop (although I didn't do proper benchmarking with multiple runs, just single `time ./test/functional/test_runner.py` on current master vs this PR).
theStack:
Tested ACK fafe896a0b
hg333:
tACK fafe896a0b
Tree-SHA512: 4bbee3c8587d612e74a59fde49b6439c1296f2fc27d3a7cf59a35e920f729fdd581c930290bd04def618f81412236676ddb99b4ceb4d80dfb9fd610b128a04b1
90b3e482e9 release: Release with separate SHA256SUMS and sig files (Carl Dong)
Pull request description:
This allows us to:
- remove the rfc4880 EOL hacks, and
- release with a SHA256SUMS.asc file that's a combination of all signer signatures
ACKs for top commit:
achow101:
ACK 90b3e482e9
laanwj:
Concept and code review ACK 90b3e482e9
Tree-SHA512: 5d5086063d303aa0cbd590e5fdf2ae8f555e25f4e43bf67545e33384449b990e94834c711622530ad0eb3dcc83f52746884a5081dadb0acff8dd799cfadafac7
5d37cc44f9 Generate doxygen documentation for test sources (Patrick Kamin)
Pull request description:
Fixes#19248
While searching for the documentation of the test utilities I realized they were excluded from doxygen. I agree with the statement in #19248. It's also helpful for new contributors to gain a broader understanding of the class dependencies visually (see BasicTestSetup)
ACKs for top commit:
laanwj:
ACK 5d37cc44f9
Tree-SHA512: 32f0abab2970c65621af5cee7f620c2653bd9688366e125543262bd078841e64a5a1e24cf0241e9f6ec538b8759e06108d5ff056449aa1c98d5f287deef18c86
fa1eddb1a3 Fix whitespace in touched files (MarcoFalke)
fa4e6afdae Remove unused CSubNet serialize code (MarcoFalke)
fa384fdd0b Ignore banlist.dat (MarcoFalke)
Pull request description:
The code to read `banlist.dat` should be removed eventually. The major release (22.x) can be used to translate a `banlist.dat` into a `banlist.json`. Thus, it is now possible to remove the reading code.
ACKs for top commit:
Zero-1729:
re-ACK fa1eddb1a3
laanwj:
concept and code review ACK fa1eddb1a3
vasild:
ACK fa1eddb1a3
jonatack:
Light code review utACK fa1eddb1a3
Tree-SHA512: e136193b7c0ba1d6d2e79c7fb4106ba4af75fa229ed7214675ee64e98e59bb4808779e7a8a09eecce62f7a5d4bc6e16b8a5ad4596129357c8fc5e3b88f214249
d2dffd5be4 doc: add info to i2p.md about IBD time and multiple networks (Jon Atack)
2962640c49 contrib, p2p: update I2P hardcoded seeds (Jon Atack)
Pull request description:
- Update the I2P seeds with the latest ones I can connect to (feedback welcome).
- Update the I2P doc with frequently asked and reported info: IBD speed and running with other networks.
ACKs for top commit:
tryphe:
ACK d2dffd5be4
duncandean:
ACK d2dffd5
willcl-ark:
ACK d2dffd5be4
laanwj:
ACK d2dffd5be4
vasild:
ACK d2dffd5be4
Tree-SHA512: 1edda0eb77b4e224b44961849b36c5a13e1eb14bdc50f909ec44d7fe926c259cada62404906594a1cf68ffd96254647ac49f24248214b42403ecebae45135e63
8a4f0fcd3f Document faster throughput configuration (Alex Groce)
Pull request description:
This is a small change to the fuzzing doc that I think might help more people improve the corpus coverage, which I think is low partly just due to lack of long, low-overhead, runs, in addition to the need to apply a more diverse set of fuzzers and coverage notions.
ACKs for top commit:
practicalswift:
ACK 8a4f0fcd3f
tryphe:
ACK 8a4f0fcd3f
Tree-SHA512: 0f1802f5c551d6ade7393cd2ac439ffd485786b17c4fd0f1a321f69f8ed0db1167ae04b5cae7bf904e89aba03e89b6d974bff564bfc6a78a571893719f323434
9b313dfef1 guix: Ensure EPOCH_SOURCE_DATE does not include GPG information (Andrew Chow)
43225f0a2a guix: Remove extra \r from all.SHA256SUMS line ending (Andrew Chow)
d080c27066 guix, doc: Add a note that codesigners need to rebuild after tagging (Andrew Chow)
4a466388a0 guix: Allow changing the base manifest in guix-verify (Andrew Chow)
33455c7696 guix: Make all.SHA256SUMS rather than codesigned.SHA256SUMS (Andrew Chow)
Pull request description:
`guix-verify` expects `all.SHA256SUMS` but `guix-attest` produces `codesigned.SHA256SUMS`. Since `all.SHA256SUMS` makes more sense (as the file contains all the sha256sums, not just the codesigned ones), `guix-attest` has been changed to output a file of that name.
As a quality of life improvement, `guix-verify` can take `SIGNER` and use the signer's manifest as the base to compare against. This makes it easier to compare a single person's attestations with everyone else's and can make it more obvious when one builder is clearly mismatching with everyone else.
Lastly `release-process.md` is updated with a note about a gotcha that can cause a mismatch in the codesigned attestation.
ACKs for top commit:
fanquake:
ACK 9b313dfef1
Tree-SHA512: 0d60627def38288dbd3059ad1e72cad224f9205da11b1a561c082ef28250a074df5cc5f2797c91a7be027bc486a3fda3319c2e496a8724e5b539337236c6f990
One of the issues observed during the 22.0rc1 release process was that a
codesigner's attestation mismatched non-codesigner attestations because
the guix-codesign step was performed prior to tagging the version in
bitcoin-detached-sigs.
Both added files are extended in the following commits.
doc/usdt.md is based on earlier work by laanwj.
Co-authored-by: W. J. van der Laan <laanwj@protonmail.com>
20edf4bcf6 rpc: Return block time in getblockchaininfo (João Barbosa)
Pull request description:
Return tip time in `getblockchaininfo`, for some use cases this can save a call to `getblock`.
ACKs for top commit:
naumenkogs:
ACK 20edf4bcf6
theStack:
re-ACK 20edf4bcf6
0xB10C:
ACK 20edf4bcf6
kristapsk:
ACK 20edf4bcf6
Zero-1729:
re-ACK 20edf4bcf6
Tree-SHA512: 29a920cfff1ef53e0af601c3f93f8f9171f3be47fc84b0fa293cb865b824976e8c1510b17b27d17daf0b8e658dd77d9dc388373395f0919fc4a23cd5019642d5
fac4814106 doc/release-process: Add torrent creation details (Carl Dong)
5d24cc3d82 guix/INSTALL: Guix installs init scripts in libdir (Carl Dong)
5da2ee49d5 guix/INSTALL: Add coreutils/inotify-dir-recreate troubleshooting (Carl Dong)
318c60700b guix: Adapt release-process.md to new Guix process (Carl Dong)
fcab35b229 guix-attest: Produce and sign normalized documents (Carl Dong)
c2541fd0ca guix: Overhaul README (Carl Dong)
46ce6ce378 tree-wide: Rename gitian-keys to builder-keys (Carl Dong)
fc4f8449f3 guix: Update various check_tools lists (Carl Dong)
263220a85c guix: Check for a sane services database (Carl Dong)
Pull request description:
Based on: #21462
Keeping the README in one file so that it's easy to search through. Will add more jumping links later so navigation is easier.
Current TODOs:
- [x] Shell installer option: prompt user to re-login for `/etc/profile.d` entry to be picked up
- [x] Binary tarball option: prompt user to create `/etc/profile.d` entry and re-login
- [x] Fanquake docker option: complete section
- [x] Arch Linux AUR option: prompt to start `guix-daemon-latest` unit after finishing "optional setup" section
- [x] Building from source option: Insert dependency tree diagram that I made
- [x] Building from source option: redo sectioning, kind of a mess right now
- [x] Optional setup: make clear which parts are only needed if building from source
- [x] Workaround 1 for GnuTLS: perhaps mention how to remove Guix build farm's key
- [x] Overall (after everything): Make the links work.
Note to self: wherever possible, tell user how to check that something is true rather than branching by installation option.
ACKs for top commit:
fanquake:
ACK fac4814106 - going to go ahead and merge this now. It's a lot of documentation, and could probably be nit-picked / improved further, however, that can continue over the next few weeks. I'm sure more (backportable) improvements / clarifications will be made while we progress through RCs towards a new release.
Tree-SHA512: dc46c0ecdfc67c7c7743ca26e4a603eb3f54adbf81be2f4c1f4c20577ebb84b5250b9c9ec89c0e9860337ab1c7cff94d7963c603287267deecfe1cd987fa070a
fa84caebc7 doc: added info to bitcoin.conf doc (Brian Liotti)
Pull request description:
Should probably be explicitly stated to not make modifications to the conf file while daemon is running. ref #11586
For example, if rpc credentials are modified while bitcoind is running, `bitcoin-cli stop` is unable to stop bitcoind until the original credentials are restored in `bitcoin.conf`
ACKs for top commit:
rajarshimaitra:
ACK fa84caebc7
Zero-1729:
ACK fa84caebc7
theStack:
ACK fa84caebc7🗄️
Tree-SHA512: f6ddffc25563c0b01e661b6abe43a7909938ad8eca38d6d0e2d4a2ce9fb850e51b54d950ef3118b74b6e340c64fe3f37205861720a2de2933db29782234869bb
a16378e501 doc: Remove unnecessary steps from translations update process (Wladimir J. van der Laan)
2584929823 doc: Add steps for transifex to release process (Wladimir J. van der Laan)
Pull request description:
Document how to update settings on and for the transifex website before and after branch-off of a new release.
(This is #21440, updated with the review feedback.)
ACKs for top commit:
laanwj:
ACK a16378e501
hebasto:
ACK a16378e501
Tree-SHA512: 9669cc3bc7ba056f913ee77c2ef9d9ee2313da947fc07f8cd955807c34c5d39e3af73587adfe734274ab2412a7dfb1e2dfe7ee4904ca28cfef9bf8061d26a573
4101ec9d2e doc: mention that we enforce port=0 in I2P (Vasil Dimov)
e0a2b390c1 addrman: reset I2P ports to 0 when loading from disk (Vasil Dimov)
41cda9d075 test: ensure I2P ports are handled as expected (Vasil Dimov)
4f432bd738 net: do not connect to I2P hosts on port!=0 (Vasil Dimov)
1f096f091e net: distinguish default port per network (Vasil Dimov)
aeac3bce3e net: change I2P seeds' ports to 0 (Vasil Dimov)
38f900290c net: change assumed I2P port to 0 (Vasil Dimov)
Pull request description:
_This is an alternative to https://github.com/bitcoin/bitcoin/pull/21514, inspired by https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-815049933. They are mutually exclusive. Just one of them should be merged._
Change assumed ports for I2P to 0 (instead of the default 8333) as this is closer to what actually happens underneath with SAM 3.1 (https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-812632520, https://github.com/bitcoin/bitcoin/pull/21514#issuecomment-816564719).
Don't connect to I2P peers with advertised port != 0 (we don't specify a port to our SAM 3.1 proxy and it always connects to port = 0).
Note, this change:
* Keeps I2P addresses with port != 0 in addrman and relays them to others via P2P gossip. There may be non-bitcoin-core-22.0 peers using SAM 3.2 and for them such addresses may be useful.
* Silently refuses to connect to I2P hosts with port != 0. This is ok for automatically chosen peers from addrman. Not so ok for peers provided via `-addnode` or `-connect` - a user who specifies `foo.b32.i2p:1234` (non zero port) may wonder why "nothing is happening".
Fixes#21389
ACKs for top commit:
laanwj:
Code review ACK 4101ec9d2e
jonatack:
re-ACK 4101ec9d2e per `git range-diff efff9c3 0b0ee03 4101ec9`, built with DDEBUG_ADDRMAN, did fairly extensive testing on mainnet both with and without a peers.dat / -dnsseeds=0 to test boostrapping.
Tree-SHA512: 0e3c019e1dc05e54f559275859d3450e0c735596d179e30b66811aad9d5b5fabe3dcc44571e8f7b99f9fe16453eee393d6e153454dd873b9ff14907d4e6354fe
e65d1d4986 doc: recommend `--disable-external-signer` in OpenBSD build guide (Sebastian Falbesoner)
Pull request description:
Building the master branch with the default build settings (i.e. with external signer support enabled) leads to the following errors on my OpenBSD 6.9 machine:
```
In file included from util/system.cpp:9:
In file included from /usr/local/include/boost/process.hpp:25:
In file included from /usr/local/include/boost/process/group.hpp:32:
/usr/local/include/boost/process/detail/posix/wait_group.hpp:38:17: error: no member named 'waitid' in the global namespace
ret = ::waitid(P_PGID, p.grp, &status, WEXITED | WNOHANG);
~~^
/usr/local/include/boost/process/detail/posix/wait_group.hpp:38:24: error: use of undeclared identifier 'P_PGID'
ret = ::waitid(P_PGID, p.grp, &status, WEXITED | WNOHANG);
^
/usr/local/include/boost/process/detail/posix/wait_group.hpp:38:48: error: use of undeclared identifier 'WEXITED'
ret = ::waitid(P_PGID, p.grp, &status, WEXITED | WNOHANG);
^
/usr/local/include/boost/process/detail/posix/wait_group.hpp:144:17: error: no member named 'waitid' in the global namespace
ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
~~^
/usr/local/include/boost/process/detail/posix/wait_group.hpp:144:24: error: use of undeclared identifier 'P_PGID'
ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
^
/usr/local/include/boost/process/detail/posix/wait_group.hpp:144:49: error: use of undeclared identifier 'WEXITED'
ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
^
/usr/local/include/boost/process/detail/posix/wait_group.hpp:144:59: error: use of undeclared identifier 'WSTOPPED'
ret = ::waitid(P_PGID, p.grp, &siginfo, WEXITED | WSTOPPED | WNOHANG);
^
7 errors generated.
```
This PR recommends passing `--disable-external-signer` in the OpenBSD build guide ([as suggested by laanwj](https://github.com/bitcoin/bitcoin/pull/22294#issuecomment-867452411)). The same commit also bumps the OpenBSD version mentioned in the header to 6.9 -- I recently used this document to setup a Bitcoin Core build on 6.9 and the description and all mentioned versions were still valid (before external signer support was enabled by default).
Would be nice if another OpenBSD user could confirm the build error.
ACKs for top commit:
laanwj:
ACK e65d1d4986
Tree-SHA512: c3ae7eca29cf42b4b52024477e1c3fb7242bbf9d809bc95f8fa08b2f9bf4bcfd4f22457d58569a208ac1d8e5fe41b270addd13d85a5bba0521a0f9e325288448
05f9770c1f doc: Clarify developer notes about constant naming (Russell Yanofsky)
Pull request description:
I'm pretty sure developer notes were intended to say constants should be upper case and variables should be lower case, but right now they are ambiguous about whether to write:
```c++
extern const int SYMBOL;
```
or:
```c++
extern const int g_symbol;
```
First convention above is better than the second convention because it tells you without having to look anything up that the value of `SYMBOL` won't change at runtime. Also I haven't seen other c++ projects using the second convention.
ACKs for top commit:
MarcoFalke:
cr ACK 05f9770c1f
practicalswift:
ACK 05f9770c1f
jarolrod:
ACK 05f9770c1f🥃
Tree-SHA512: 766d0e25d9db818d45df4ad6386987014f2053584cbced4b755ceef8bda6b7e2cfeb34eb8516423bd03b140faaf577614d5e3be2799f7eed0eb439187ab85323
dd49f1b52e doc: doc/README.md IRC updates (Jon Atack)
Pull request description:
Follow-up to https://github.com/bitcoin/bitcoin/pull/22109:
- update remaining link and reference from Freenode to Libera
- remove defunct #bitcoin-dev channel
- other minor fixups
ACKs for top commit:
fanquake:
ACK dd49f1b52e
Tree-SHA512: de20a8f3889028def7139be1cd0fa15daa3351ca987a16090d0caa6b4cd397e370400e2b3033ec776e270aa3a1ab085ada20fbdcc063ac88e2ad3b8351a596e7
957f358427 build: remove check for Boost Process header (fanquake)
df2c933217 build: remove workaround for Boost and std::atomic (fanquake)
2bf211696b build: set minimum required Boost to 1.64.0 (fanquake)
Pull request description:
Setting a newer minimum required Boost means we can remove the awkward header / compile check for Boost Process.
If we don't do this, we should at-least make Boost Process being missing no longer a failure, otherwise anyone building using Boost < 1.64.0 would have to pass `--disable-external-signer` as well.
The only system I can see that is affected here, (doesn't have new enough system packages) is Debian Oldstable. However, anyone compiling there, can use depends. They can also no-longer use the system GCC (6.0), and I'd assume would be using Clang 7, which would be the newest compiler available to them. It's extended, LTS support also end in 1 year from now, so anyone still using it should be considering upgrading.
Debian Buster (Stable) has 1.67+, Ubuntu Bionic has 1.65+, any of the BSDs, recent Fedora, macOS etc all also have well and truly new enough Boost versions available.
I think this is something we should just do for 22.0. If not, definitely for 23.0.
Fixes#22319. Compiling Bitcoin Core should work, as `windows.h` will be included.
Alternative to #22294.
Would also close#22269.
#19128 could be re-opened.
ACKs for top commit:
laanwj:
Tested ACK 957f358427 that this fixes#22269.
MarcoFalke:
review ACK 957f358427
Tree-SHA512: a8ffa7933dce8bf994892ef16664103d7b4e1008e52628e9becb918a7727232dfb51b23100a82dc2b60cd9af5877abc32dc2d3754a7e1b3ac5410a92fdf393f3
bb719a08db style: remove () from assert in rpc_setban.py (Vasil Dimov)
24b10ebda3 doc: fix grammar in doc/files.md (Vasil Dimov)
dd4e957dcd test: ensure banlist can be read from disk after restart (Vasil Dimov)
d197977ae2 banman: save the banlist in a JSON format on disk (Vasil Dimov)
Pull request description:
Save the banlist in `banlist.json` instead of `banlist.dat`.
This makes it possible to store Tor v3 entries in the banlist on disk
(and any other addresses that cannot be serialized in addrv1 format).
Only read `banlist.dat` if it exists and `banlist.json` does not exist (first start after an upgrade).
Supersedes https://github.com/bitcoin/bitcoin/pull/20904
Resolves https://github.com/bitcoin/bitcoin/issues/19748
ACKs for top commit:
jonatack:
Code review re-ACK bb719a08db per `git range-diff 6a67366 4b52c72 bb719a0`
achow101:
Code Review ACK bb719a08db
Tree-SHA512: fc135c3a1fe20bcf5d008ce6bea251b4135e56c78bf8f750b4bd8144c095b81ffe165133cdc7e4715875eec7e7c4e13ad9f5d2450b21102af063d7c8abf716b6
b9e76f1bf0 rpc: Add test for -rpcwaittimeout (Christian Decker)
f76cb10d7d rpc: Prefix rpcwaittimeout error with details on its nature (Christian Decker)
c490e17ef6 doc: Add release notes for the `-rpcwaittimeout` cli parameter (Christian Decker)
a7fcc8eb59 rpc: Add a `-rpcwaittimeout` parameter to limit time spent waiting (Christian Decker)
Pull request description:
Adds a new numeric `-rpcwaittimeout` that can be used to limit the
time we spend waiting on the RPC server to appear. This is used by
downstream projects to provide a bit of slack when `bitcoind`s RPC
interface is not available right away.
This makes the `-rpcwait` argument more useful, since we can now limit
how long we'll ultimately wait, before potentially giving up and reporting
an error to the caller. It was discussed in the context of the BTCPayServer
wanting to have c-lightning wait for the RPC interface to become available
but still have the option of giving up eventually ([4355]).
I checked with laanwj whether this is already possible ([comment]), and
whether this would be a welcome change. Initially I intended to repurpose
the (optional) argument to `-rpcwait`, however I decided against it since it
would potentially break existing configurations, using things like `rpcwait=1`,
or `rpcwait=true` (the former would have an unintended short timeout, when
old behavior was to wait indefinitely).
~Due to its simplicity I didn't implement a test for it yet, but if that's desired I
can provide one.~ Test was added during reviews.
[4355]: https://github.com/ElementsProject/lightning/issues/4355
[comment]: https://github.com/ElementsProject/lightning/issues/4355#issuecomment-768288261
ACKs for top commit:
laanwj:
Code review ACK b9e76f1bf0
promag:
ACK b9e76f1bf0.
Tree-SHA512: 3cd6728038ec7ca7c35c2e7ccb213bfbe963f99a49bb48bbc1e511c4dd23d9957c04f9af1f8ec57120e47b26eaf580b46817b099d5fc5083c98da7aa92db8638
Save the banlist in `banlist.json` instead of `banlist.dat`.
This makes it possible to store Tor v3 entries in the banlist on disk
(and any other addresses that cannot be serialized in addrv1 format).
Only read `banlist.dat` if it exists and `banlist.json` does not
exist (first start after an upgrade).
Supersedes https://github.com/bitcoin/bitcoin/pull/20904
Resolves https://github.com/bitcoin/bitcoin/issues/19748
2ad034a890 doc: update release notes with removal of tor v2 support (Jon Atack)
49938eee9c doc: update tor.md with removal of tor v2 support (Jon Atack)
Pull request description:
Follow-up documentation to #22050 that removed support for Tor version 2 hidden services from Bitcoin Core.
ACKs for top commit:
laanwj:
ACK 2ad034a890
Tree-SHA512: 0f13f9d1db7e11f1e3d9967b6d17b8dc3144b3ab3a258c706464c5e6ac5cbcf2ce2db4ea54be9939f05a82ebd1e7f325f50b435f9822c08b4f21ed4ac58de0af
Since the flag has been removed from the RPC command table in commit
ec6902d0ea (PR #11179), this guideline
is not relevant anymore and can be removed.
2667366aaa tests: check derivation of P2TR (Pieter Wuille)
7cedafc541 Add tr() descriptor (derivation only, no signing) (Pieter Wuille)
90fcac365e Add TaprootBuilder class (Pieter Wuille)
5f6cc8daa8 Add XOnlyPubKey::CreateTapTweak (Pieter Wuille)
2fbfb1becb Make consensus checking of tweaks in pubkey.* Taproot-specific (Pieter Wuille)
a4bf84039c Separate WitnessV1Taproot variant in CTxDestination (Pieter Wuille)
41839bdb89 Avoid dependence on CTxDestination index order (Pieter Wuille)
31df02a070 Change Solver() output for WITNESS_V1_TAPROOT (Pieter Wuille)
4b1cc08f9f Make XOnlyPubKey act like byte container (Pieter Wuille)
Pull request description:
This is a subset of #21365, to aide review.
This adds support `tr(KEY)` or `tr(KEY,SCRIPT)` or `tr(KEY,{{S1,{{S2,S3},...}},...})` descriptors, describing Taproot outputs with specified internal key, and optionally any number of scripts, in nested groups of 2 inside `{`/`}` if there are more than one. While it permits importing `tr(KEY)`, anything beyond that is just laying foundations for more features later.
Missing:
* Signing support (see #21365)
* Support for more interesting scripts inside the tree (only `pk(KEY)` is supported for now). In particular, a multisig policy based on the new `OP_CHECKSIGADD` opcode would be very useful.
* Inferring `tr()` descriptors from outputs (given sufficient information).
* `getaddressinfo` support.
* MuSig support. Standardizing that is still an ongoing effort, and is generally kind of useless without corresponding PSBT support.
* Convenient ways of constructing descriptors without spendable internal key (especially ones that arent't trivially recognizable as such).
ACKs for top commit:
Sjors:
utACK 2667366 (based on https://github.com/bitcoin/bitcoin/pull/21365#issuecomment-846945215 review, plus the new functional test)
achow101:
Code Review ACK 2667366aaa
lsilva01:
Tested ACK 2667366aaa
meshcollider:
utACK 2667366aaa
Tree-SHA512: 61046fef22c561228338cb178422f0b782ef6587ec8208d3ce2bd07afcff29a664b54b35c6b01226eb70b6540b43f6dd245043d09aa6cb6db1381b6042667e75
dafab2b1b3 doc: improve note on choosing posix mingw32 (Jarol Rodriguez)
Pull request description:
The current [windows build doc](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md) can lead someone to believe that the step where you must choose the posix mingw32 g++ compiler option is only for `Ubuntu 18.04`. It is only until you (or just me) go through the cross-compile process and realize that it's not building because you didn't set this option on > `Ubuntu 18.04`. Then you come back and read the footnotes and see: `Starting from Ubuntu Xenial 16.04...`.
This PR improves this portion of the doc by editing the text around this stage to state "this is now the next step". We could add a note saying `Ubuntu 18.04 and up`, but this is redundant as it's unlikely someone will be using < Ubuntu 18.04 since it's not officially supported by our build system. While here, some minor fixups and add some more guidance to completing this step.
**Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md#building-for-64-bit-windows)
**PR:** [render](dafab2b1b3/doc/build-windows.md (building-for-64-bit-windows))
ACKs for top commit:
laanwj:
ACK dafab2b1b3
hebasto:
ACK dafab2b1b3
Tree-SHA512: d8d50fa13c34be753700843183a1a4addad6447c813bfc039865ba11d5fa03c25206ed6f3782331df4c6a44448e3fec4138ae911b995b5cacc5b756bb5d74bb4
13650fe2e5 [policy] detect unsorted packages (glozow)
9ef643e21b [doc] add release note for package testmempoolaccept (glozow)
c4259f4b7e [test] functional test for packages in RPCs (glozow)
9ede34a6f2 [rpc] allow multiple txns in testmempoolaccept (glozow)
ae8e6df709 [policy] limit package sizes (glozow)
c9e1a26d1f [fuzz] add ProcessNewPackage call in tx_pool fuzzer (glozow)
363e3d916c [test] unit tests for ProcessNewPackage (glozow)
cd9a11ac96 [test] make submit optional in CreateValidMempoolTransaction (glozow)
2ef187941d [validation] package validation for test accepts (glozow)
578148ded6 [validation] explicit Success/Failure ctors for MempoolAcceptResult (glozow)
b88d77aec5 [policy] Define packages (glozow)
249f43f3cc [refactor] add option to disable RBF (glozow)
897e348f59 [coins/mempool] extend CCoinsViewMemPool to track temporary coins (glozow)
42cf8b25df [validation] make CheckSequenceLocks context-free (glozow)
Pull request description:
This PR enables validation dry-runs of packages through the `testmempoolaccept` RPC. The expectation is that the results returned from `testmempoolaccept` are what you'd get from test-then-submitting each transaction individually, in that order (this means the package is expected to be sorted in topological order, for now at least). The validation is also atomic: in the case of failure, it immediately halts and may return "unfinished" `MempoolAcceptResult`s for transactions that weren't fully validated. The API for 1 transaction stays the same.
**Motivation:**
- This allows you to test validity for transaction chains (e.g. with multiple spending paths and where you don't want to broadcast yet); closes#18480.
- It's also a first step towards package validation in a minimally invasive way.
- The RPC commit happens to close#21074 by clarifying the "allowed" key.
There are a few added restrictions on the packages, mostly to simplify the logic for areas that aren't critical to main package use cases:
- No package can have conflicts, i.e. none of them can spend the same inputs, even if it would be a valid BIP125 replacement.
- The package cannot conflict with the mempool, i.e. RBF is disabled.
- The total count of the package cannot exceed 25 (the default descendant count limit), and total size cannot exceed 101KvB (the default descendant size limit).
If you're looking for review comments and github isn't loading them, I have a gist compiling some topics of discussion [here](https://gist.github.com/glozow/c3acaf161c95bba491fce31585b2aaf7)
ACKs for top commit:
laanwj:
Code review re-ACK 13650fe2e5
jnewbery:
Code review ACK 13650fe2e5
ariard:
ACK 13650fe
Tree-SHA512: 8c5cbfa91a6c714e1c8710bb281d5ff1c5af36741872a7c5df6b24874d6272b4a09f816cb8a4c7de33ef8e1c2a2c252c0df5105b7802f70bc6ff821ed7cc1a2f
This adds a new descriptor with syntax e.g. tr(KEY,{S1,{{S2,S3},S4})
where KEY is a key expression for the internal key and S_i are
script expression for the leaves. They have to be organized in
nested {A,B} groups, with exactly two elements.
tr() only exists at the top level, and inside the script expressions
only pk() scripts are allowed for now.
fadd98d02f doc: Fix OSS-Fuzz links (MarcoFalke)
Pull request description:
* Add missing link to the coverage report
* Replace unqualified link with qualified one
ACKs for top commit:
practicalswift:
ACK fadd98d02f
jarolrod:
ACK fadd98d02f
Tree-SHA512: 5d5138288ae75d6a9ae536f307a63813ffe60c628a8c9a74b33f0d1be1a34fac2ff5eca9c9fe64be3139f2251736ba088c5f3731f0b3f50c70c5a5f1a89f0ea4
07bc22ef10 docs: improve make with parallel jobs description. (Klement Tan)
Pull request description:
Changed `use -jX here for parallelism` to `use "-j N" for N parallel jobs`
**Rationale**: In my opinion `use -jX here for parallelism` is quite ambiguous as it could be perceived as a single option without any argument. Ie running:
```sh
make -jX
```
Embarrassingly this caused me to be stuck for quite a long time until I opened the help menu for `make` but if I am the only one who faced this issue I would be happy to close this PR.
ACKs for top commit:
jarolrod:
ACK 07bc22ef10
Tree-SHA512: 2d119b6a461668906c63184b865d2cc9fb2f75abeba34e2e44bc1ef3bcb4adec4a49896ddaf3cc6a20c0095ad20d0de0908401b351eaca9443161d24d6b20d0b
11d6459b6e rpc: include_unsafe option for fundrawtransaction (t-bast)
Pull request description:
Allow RPC users to opt-in to unsafe inputs when funding a raw transaction.
Applications that need to manage a complex RBF flow (such as lightning nodes using anchor outputs) are very limited if they can only use safe inputs.
I also added this option to `send` and `walletcreatefundedpsbt` who internally delegate to `fundrawtransaction`.
Fixes#21299
ACKs for top commit:
laanwj:
Code review ACK 11d6459b6e
Tree-SHA512: 5e542a4febcfd6f41cf784678ff02ec9282eae2082c274983f72c5ea87b7ebbe1bd5fdc6a020d7a9d5996157754eb4966b8aeb6c1ceebf0b1519f735579b8bac
47c3ea021e doc: add OSS-Fuzz section to fuzzing.md doc (Adam Jonas)
Pull request description:
This adds documentation about [Bitcoin Core's participation](https://github.com/google/oss-fuzz/pull/5699/files) in Google's OSS-Fuzz program and adds the caveat that the project may not disclose vulnerabilities within the 90-day window described in the [program's disclosure guidelines](https://google.github.io/oss-fuzz/getting-started/bug-disclosure-guidelines/).
ACKs for top commit:
jonatack:
ACK 47c3ea021e
Tree-SHA512: 87bf0146fb74d1e4b3b8839e6c8f3d53046008a6d5b926ffe5b95be3c396a5e47e47967533422f60b04c4446482f49d210ada410b742f69781a7afde623d704d
ebd4be43cc doc: add release notes for 20867 (Antoine Poinsot)
5aa50ab9cc rpc/util: multisig: only check redeemScript size is <= 520 for P2SH (Antoine Poinsot)
063df9e897 test/functional: standardness sanity checks for P2(W)SH multisig (Antoine Poinsot)
ae0429d3af script: allow up to 20 keys in wsh() descriptors (Antoine Poinsot)
9fc68faf35 script: match multisigs with up to MAX_PUBKEYS_PER_MULTISIG keys (Antoine Poinsot)
Pull request description:
As described in https://github.com/bitcoin/bitcoin/issues/20620 multisigs are currently limited to 16 keys in descriptors and RPC helpers, even for P2WSH and P2SH-P2WSH.
This adds support for multisig with up to 20 keys (which are already standard) for Segwit v0 context for descriptors (`wsh()`, `sh(wsh())`) and RPC helpers.
Fixes https://github.com/bitcoin/bitcoin/issues/20620
ACKs for top commit:
meshcollider:
re-utACK ebd4be43cc
instagibbs:
re-ACK ebd4be43cc
Tree-SHA512: 36141f10a8288010d17d5c4fe8d24878bcd4533b88a8aba3a44fa8f74ceb3182d70fee01427e0ab7f53ce7fab46c88c1cd3ac3b18ab8a10bd4a6b8b74ed79e46
Allow RPC users to opt-in to unsafe inputs when funding a raw transaction.
Applications that need to manage a complex RBF flow (such as lightning
nodes using anchor outputs) are very limited if they can only use safe inputs.
Fixes#21299
06c43201a7 cli: use C++17 std::array class template argument deduction (CTAD) (Jon Atack)
edf3167151 addrinfo: raise helpfully on server error or incompatible server version (Jon Atack)
bb85cbc4f7 doc: add cli -addrinfo release note (Jon Atack)
5056a37624 cli: add -addrinfo command (Jon Atack)
db4d2c282a cli: create AddrinfoRequestHandler class (Jon Atack)
Pull request description:
While looking at issue #21351, it turned out that the problem was a lack of tor v3 addresses known to the node. It became clear (e.g. https://github.com/bitcoin/bitcoin/issues/21351#issuecomment-811004779) that a CLI command returning the number of addresses the node knows per network (with a tor v2 / v3 breakdown) would be very helpful. This patch adds that.
`-addrinfo` is useful to see if your node knows enough addresses in a network to use options like `-onlynet=<network>`, or to upgrade to the upcoming tor release that no longer supports tor v2, for which you'll need to be sure your node knows enough tor v3 peers.
```
$ bitcoin-cli --help | grep -A1 addrinfo
-addrinfo
Get the number of addresses known to the node, per network and total.
$ bitcoin-cli -addrinfo
{
"addresses_known": {
"ipv4": 14406,
"ipv6": 2511,
"torv2": 5563,
"torv3": 2842,
"i2p": 8,
"total": 25330
}
}
$ bitcoin-cli -addrinfo 1
error: -addrinfo takes no arguments
```
This can be manually tested, for example, with commands like this:
```
$ bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".onion")) | select(.address | length <= 22)) | .address' | wc -l
5563
$ bitcoin-cli getnodeaddresses 0 | jq '.[] | (select(.address | contains(".onion")) | select(.address | length > 22)) | .address' | wc -l
2842
$ bitcoin-cli getnodeaddresses 0 | jq '.[] | .address' | wc -l
25330
```
ACKs for top commit:
laanwj:
Tested ACK 06c43201a7
Tree-SHA512: b668b47718a4ce052aff218789f3da629bca730592c18fcce9a51034d95a0a65f8e6da33dd47443cdd8f60c056c02696db175b0fe09a688e4385a76c1d8b7aeb
223b1ba7d9 doc: Use CONFIG_SITE instead of --prefix (Hennadii Stepanov)
Pull request description:
The current examples of `--prefix=...` option usage to point `configure` script to appropriate `depends` directory is not [standard](https://www.gnu.org/prep/standards/html_node/Directory-Variables.html). This causes some [confusion](https://github.com/bitcoin/bitcoin/pull/16691) and a bit of inconvenience.
Consider a CentOS 7 32 bit system. Packages `libdb4-devel`, `libdb4-cxx-devel`, `miniupnpc-devel` and `zeromq-devel` are unavailable from repos. After recommended build with depends:
```
cd depends
make
cd ..
./autogen.sh
./configure --prefix=$PWD/depends/i686-pc-linux-gnu
make
```
a user is unable to `make install` compiled binaries neither locally (to `~/.local`) nor system-wide (to `/usr/local`) as `--prefix` is set already.
Meanwhile, the standard approach with using [`config.site`](https://www.gnu.org/software/automake/manual/html_node/config_002esite.html) files allows both possibilities:
```
cd depends
make
cd ..
./autogen.sh
CONFIG_SITE=$PWD/depends/i686-pc-linux-gnu/share/config.site ./configure --prefix ~/.local
make
make install
```
or
```
CONFIG_SITE=$PWD/depends/i686-pc-linux-gnu/share/config.site ./configure
make
sudo make install # install to /usr/local
```
Moreover, this approach is used in [Gitian descriptors](https://github.com/bitcoin/bitcoin/tree/master/contrib/gitian-descriptors) already.
ACKs for top commit:
practicalswift:
ACK 223b1ba7d9: patch looks correct
fanquake:
ACK 223b1ba7d9
Tree-SHA512: 46d97924f0fc7e95ee4566737cf7c2ae805ca500e5c49af9aa99ecc3acede4b00329bc727a110aa1b62618dfbf5d1ca2234e736f16fbdf96d6ece5f821712f54
88d4d5ff2f rpc: add help for enumeratesigners and walletdisplayaddress (Sjors Provoost)
b0db187e5b ci: use --enable-external-signer instead of --with-boost-process (Sjors Provoost)
b54b2e7b1a Move external signer out of wallet module (Sjors Provoost)
Pull request description:
In addition, this PR enables external signer testing on CI.
This PR moves the ExternalSigner class and RPC methods out of the wallet module.
The `enumeratesigners` RPC can be used without a wallet since #21417. With additional modifications external signers could be used without a wallet in general, e.g. via `signrawtransaction`.
The `signerdisplayaddress` RPC is ranamed to `walletdisplayaddress` because it requires wallet context. A future `displayaddress` RPC call without wallet context could take a descriptor argument.
This commit fixes a `rpc_help.py` failure when configured with `--disable-wallet`.
ACKs for top commit:
ryanofsky:
Code review ACK 88d4d5ff2f
fanquake:
ACK 88d4d5ff2f
Tree-SHA512: 3242a24e22313aed97eee32a520bfcb1c17495ba32a2b8e06a5e151e2611320e2da5ef35b572d84623af0a49a210d2f9377a2531250868d1a0ccf3e144352a97
d3b0b08b0f doc: release notes for new listbanned fields (Jarol Rodriguez)
60290d3f5e test: increase listbanned unit test coverage (Jon Atack)
3e978d1a5d rpc: add time_remaining field to listbanned (Jarol Rodriguez)
5456b34531 rpc: add ban_duration field to listbanned (Jarol Rodriguez)
c95c61657a doc: improve listbanned help (Jarol Rodriguez)
dd3c8eaa33 rpc: swap position of banned_until and ban_created fields (Jarol Rodriguez)
Pull request description:
This PR adds a `ban_duration` and `time_remaining` field to the `listbanned` RPC command. Thanks to jonatack, this PR also expands the `listbanned` test coverage to include these new fields
It's useful to keep track of `ban_duration` as this is another data point on which to sort banned peers. I found this helpful in adding additional context columns to the GUI `bantablemodel` as part of a follow-up PR. As [suggested](https://github.com/bitcoin/bitcoin/pull/21602#issuecomment-813486134) by jonatack, `time_remaining` is another useful user-centric data point.
Since a ban always expires after its created, the `ban_created` field is now placed before the `banned_until` field. This new ordering is more logical.
This PR also improves the `help listbanned` output by providing additional context to the descriptions of the `address`, `ban_created`, and `banned_until` fields.
**Master: listbanned**
```
[
{
"address": "1.2.3.4/32",
"banned_until": 1617691101,
"ban_created": 1617604701
},
{
"address": "135.181.41.129/32",
"banned_until": 1649140716,
"ban_created": 1617604716
}
]
```
**PR: listbanned**
```
[
{
"address": "1.2.3.4/32",
"ban_created": 1617775773,
"banned_until": 1617862173,
"ban_duration": 86400,
"time_remaining": 86392
},
{
"address": "3.114.211.172/32",
"ban_created": 1617753165,
"banned_until": 1618357965,
"ban_duration": 604800,
"time_remaining": 582184
}
]
```
ACKs for top commit:
jonatack:
re-ACK d3b0b08b0f
hebasto:
ACK d3b0b08b0f, tested on Linux Mint 20.1 (x86_64).
MarcoFalke:
review ACK d3b0b08b0f🕙
Tree-SHA512: 5b83ed2483344e546d57e43adc8a1ed7a1fff292124b14c86ca3a1aa2aec8b0f7198212fabff2c5145e7f726ca04ae567fe667b141254c7519df290cf63774e5
This commit moves the ExternalSigner class and RPC methods out of the wallet module.
The enumeratesigners RPC can be used without a wallet since #21417.
With additional modifications external signers could be used without a wallet in general, e.g. via signrawtransaction.
The signerdisplayaddress RPC is ranamed to walletdisplayaddress because it requires wallet context.
A future displayaddress RPC call without wallet context could take a descriptor argument.
This commit fixes a rpc_help.py failure when configured with --disable-wallet.
5c446784b1 rpc: improve getnodeaddresses help (Jon Atack)
1b9189866a rpc: simplify/constify getnodeaddresses code (Jon Atack)
3bb6e7b655 rpc: add network field to rpc getnodeaddresses (Jon Atack)
Pull request description:
This patch adds a network field to RPC `getnodeaddresses`, which is useful on its own, particularly with the addition of new networks like I2P and others in the future, and which I also found helpful for adding a new CLI command as a follow-up to this pull that calls `getnodeaddresses` and needs to know the network of each address.
While here, also improve the `getnodeaddresses` code and help.
```
$ bitcoin-cli -signet getnodeaddresses 3
[
{
"time": 1611564659,
"services": 1033,
"address": "2600:1702:3c30:734f:8f2e:744b:2a51:dfa5",
"port": 38333,
"network": "ipv6"
},
{
"time": 1617531931,
"services": 1033,
"address": "153.126.143.201",
"port": 38333,
"network": "ipv4"
},
{
"time": 1617473058,
"services": 1033,
"address": "nsgyo7begau4yecc46ljfecaykyzszcseapxmtu6adrfagfrrzrlngyd.onion",
"port": 38333,
"network": "onion"
}
]
$ bitcoin-cli help getnodeaddresses
getnodeaddresses ( count )
Return known addresses, which can potentially be used to find new nodes in the network.
Arguments:
1. count (numeric, optional, default=1) The maximum number of addresses to return. Specify 0 to return all known addresses.
Result:
[ (json array)
{ (json object)
"time" : xxx, (numeric) The UNIX epoch time when the node was last seen
"services" : n, (numeric) The services offered by the node
"address" : "str", (string) The address of the node
"port" : n, (numeric) The port number of the node
"network" : "str" (string) The network (ipv4, ipv6, onion, i2p) the node connected through
},
...
]
```
Future idea: allow passing `getnodeaddresses` a network (or networks) as an argument to return only addresses in that network.
ACKs for top commit:
laanwj:
Tested ACK 5c446784b1
jarolrod:
re-ACK 5c446784b1
promag:
Code review ACK 5c446784b1.
Tree-SHA512: ab0101f50c76d98c3204133b9f2ab6b7b17193ada31455ef706ad11afbf48f472fa3deb33e96028682369b35710ccd07d81863d2fd55c1485f32432f2b75efa8
90ae3d8ca6 doc: Add release notes for -deprecatedrpc=addresses and bitcoin-tx (Michael Dietz)
085b3a7299 rpc: deprecate `addresses` and `reqSigs` from rpc outputs (Michael Dietz)
Pull request description:
Considering the limited applicability of `reqSigs` and the confusing output of `1` in all cases except bare multisig, the `addresses` and `reqSigs` outputs are removed for all rpc commands.
1) add a new sane "address" field (for outputs that have an identifiable address, which doesn't include bare multisig)
2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact (with all weird/wrong behavior they have now)
3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely always.
Note: Some light refactoring done to allow us to very easily delete a few chunks of code (marked with TODOs) when we remove this deprecated behavior.
Using `IsDeprecatedRPCEnabled` in core_write.cpp caused some circular dependencies involving core_io
Circular dependencies were caused by rpc/util unnecessarily importing node/coinstats and node/transaction. Really what rpc/util needs are some fundamental type/helper-function definitions. So this was cleaned up to make more sense.
This fixes#20102.
ACKs for top commit:
MarcoFalke:
re-ACK 90ae3d8ca6📢
Tree-SHA512: 8ffb617053b5f4a8b055da17c06711fd19632e0037d71c4c8135e50c8cd7a19163989484e4e0f17a6cc48bd597f04ecbfd609aef54b7d1d1e76a784214fcf72a
dc4551c22c remove incompatibility release note for darkmode on macos (Sylvain Goumy)
303cfc6227 allow darkmode on macos build (Sylvain Goumy)
78f75a2d60 Allow icon colorization on mac os to better support dark mode (Uplab)
Pull request description:
Allow icons to be colorized on macOS to support native Dark mode color scheme.
Rendering on macOS Big Sur before PR:
Rendering on macOS Big Sur after PR:
Light mode stay visually unchanged.
<del>Note, that this currently only affect the build from source, as the macos dmg includes an attributes to force light color scheme on macos windows (see https://github.com/bitcoin/bitcoin/pull/14593). </del>
<del>But once all glitches are fixed, we will be able to remove this temporary fix. </del>
Edit: this PR is know including the removal of `NSRequiresAquaSystemAppearance` on Info.plist file so that the color fix is apply to every build.
Linked issues: #68#136
ACKs for top commit:
hebasto:
re-ACK dc4551c22c
jarolrod:
ACK dc4551c22c
Tree-SHA512: 1c3a4dec796063e61fcaf80112afc2b15c8669a1cd30ebd537cea96647c20215f8f80289719f905820bb0c490c8c1f94bfae4bb32f9c6d1fdd4e8f199ebb559f
246774e264 depends: fix Qt precompiled headers bug (Igor Cota)
8e7ad4146d depends: disable Qt Vulkan support on Android (Igor Cota)
ba46adaa1a CI: add Android APK build to cirrus (Igor Cota)
7563720e30 CI: add Android APK build script (Igor Cota)
ebfb10cb75 Qt: add Android packaging support (Igor Cota)
Pull request description:
This PR is the third and final piece of the basic Android support puzzle - it depends on https://github.com/bitcoin/bitcoin/pull/16110 and is related to https://github.com/bitcoin/bitcoin/pull/16883. It introduces an `android` directory under `qt` and a simple way to build an Android package of `bitcoin-qt`:
1. Build depends for Android as described in the [README](https://github.com/bitcoin/bitcoin/blob/master/depends/README.md)
2. Configure with one of the resulting prefixes
3. Run `make && make apk` in `src/qt`
The resulting APK files will be in `android/build/outputs/apk`. You can install them manually or with [adb](https://developer.android.com/studio/command-line/adb). One can also open the `android` directory in Android Studio for that integrated development and debugging experience. `BitcoinQtActivity` is your starting point.
Under the hood makefile `apk` target:
1. Renames the `bitcoin-qt` binary to `libbitcoin-qt.so` and copies it over to a folder under `android/libs` depending on which prefix and corresponding [ABI](https://developer.android.com/ndk/guides/abis.html#sa) `bitcoin-qt` was built for
2. Takes `libc++_shared.so` from the Android NDK and puts in the same place. It [must be included](https://developer.android.com/ndk/guides/cpp-support) in the APK
3. Extracts Qt for Android Java support files from the `qtbase` archive in `depends/sources` to `android/src`
There is also just a tiny bit of `ifdef`'d code to make the Qt Widgets menus usable. It's not pretty but it works and is a stepping stone towards https://github.com/bitcoin/bitcoin/pull/16883.
ACKs for top commit:
MarcoFalke:
cr ACK 246774e264
laanwj:
Code review ACK 246774e264
Tree-SHA512: ba30a746576a167545223c35a51ae60bb0838818779fc152c210f5af1413961b2a6ab6af520ff92cbc8dcd5dcb663e81ca960f021218430c1f76397ed4cead6c
1) add a new sane "address" field (for outputs that have an
identifiable address, which doesn't include bare multisig)
2) with -deprecatedrpc: leave "reqSigs" and "addresses" intact
(with all weird/wrong behavior they have now)
3) without -deprecatedrpc: drop "reqSigs" and "addresses" entirely,
always.
Introduce an android directory under qt and allow one to package bitcoin-qt for Android by running make apk.
Add bitcoin-qt Android build instructions.
bb3f79fbdc doc: Update libnatpmp info in dependencies.md (Hennadii Stepanov)
1a01a5dc8d doc: Update zlib info in dependencies.md (Hennadii Stepanov)
Pull request description:
Update docs according to the recent changes in the code:
- #21209 (zlib)
- #21376 (libnatpmp)
ACKs for top commit:
fanquake:
ACK bb3f79fbdc - thanks for keeping this updated.
Tree-SHA512: 48350ad07700aa071ad6c34e4c161aaadc050488fc068cf478e9781d632828187962a4384c1b67c2344145a2c00c3e16cddd09259130af8e9e86cd76cd32900d
c180c911b8 doc: revamp macOS build doc (Jarol Rodriguez)
Pull request description:
This PR makes the macOS build-docs more informative and adds in the following information:
- Proper descriptions and delineation of required/optional dependencies
- walk-through of optional dependencies
- configuration walk-through
- various other tidbits of information
This is a part of the efforts done in https://github.com/bitcoin/bitcoin/pull/20601 and https://github.com/bitcoin/bitcoin/pull/20610 to update the docs and introduce some consistency between them.
This update does not add instructions for arm-based M1 Macbooks as I do not have one to test with. It would be nice to have someone follow up with an update containing instructions for arm-based Macs.
**Before/Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-osx.md)
**After/PR:** [render](c180c911b8/doc/build-osx.md)
ACKs for top commit:
fanquake:
ACK c180c911b8 - I still think these are getting too verbose and we're duplicating information all over the place; dependencies, configure options, combinations of options etc. However if people are happy to maintain them, I guess it's fine for now, and this revamping has already happened for some of the other build READMEs.
Tree-SHA512: 1440046c723fe80d4158e4a429e3aa8bd93570acb84ad202d5d24c749ab9a89a3aca8b61b49e75e042a4bf4317acd632d3906e1b5808a9052e74209256528b45
57ff5a42ab doc: specify minimum HWI version (Sjors Provoost)
03308b2bfa rpc: don't require wallet for enumeratesigners (Sjors Provoost)
Pull request description:
HWI just released 2.0. See https://github.com/bitcoin-core/HWI/releases/tag/2.0.0
As of #16546 we already rely on features that are in 2.0 and not in the previous 1.* releases:
* `--chain` param
This shouldn't be a problem, because HWI 2.0 has been released before we release v22.
Misc improvements:
* document that HWI 2.0 is required
* drop wallet requirement for `enumeratesigners`
ACKs for top commit:
achow101:
Code Review ACK 57ff5a42ab
Tree-SHA512: 3fb6ba20894e52a116f89525a5f5a1f61d363ccd904e1cffd0e6d095640fc6d2edf0388cd6ae20f83bbc31e5f458255ec090b6e823798d426eba3e45b4336bf9
fab633d2db doc: Update fuzzing docs for afl-clang-lto (MarcoFalke)
Pull request description:
Update the docs to default to `afl-clang-lto`. The afl-gcc (and other afl legacy fuzz engines) are still supported, though discouraged.
ACKs for top commit:
fanquake:
ACK fab633d2db - seems to work for me. Compiled and ran some fuzzers using Clang 11 on Bionic. Set `llvm-config` so that `clang-11` would be used over `clang` (10).
jarolrod:
ACK fab633d2db, tested on Ubuntu Focal
Tree-SHA512: 3d1969c167bea45a9d691f3b757f51213d550c9c1b895bed1fcf3c2f7345791787cfb13c376291b94eb3181caf4ae3126f4d01c7cebda7b2bb1c40a1294e9a68
This pr makes the macOS build docs more informative and adds in the following information:
- Proper descriptions and delineation of required/optional dependencies
- walk-through of optional dependencies
- configuration walk-through
- various other tid-bits of information
4899fa3abd doc: Update build instructions for Fedora (Hennadii Stepanov)
Pull request description:
This PR updates build instructions for Fedora, as Fedora 33 has no `libdb4-devel` and `libdb4-cxx-devel` packages in its default repos.
ACKs for top commit:
practicalswift:
ACK 4899fa3abd
Tree-SHA512: 0c23e2ce0ea690ed5eeaa56514d4246e0057a77b7c71f28af4ee1e480521d465122f81cea37cc773ce2db4fc189d5ab3c8f8ffdd65f150cc006390aa1e2a4ac8
Brew has updated such that qt now refers to Qt 6.0.1. If builders
install this, configure will not work pick up qt. For now, install
qt@5 (5.15.2), until required build system and likely source changes
are made.
Introduce two new options to reach the I2P network:
* `-i2psam=<ip:port>` point to the I2P SAM proxy. If this is set then
the I2P network is considered reachable and we can make outgoing
connections to I2P peers via that proxy. We listen for and accept
incoming connections from I2P peers if the below is set in addition to
`-i2psam=<ip:port>`
* `-i2pacceptincoming` if this is set together with `-i2psam=<ip:port>`
then we accept incoming I2P connections via the I2P SAM proxy.
faa06ecc9c build: Bump minimum Qt version to 5.9.5 (Hennadii Stepanov)
Pull request description:
Close#20104.
ACKs for top commit:
laanwj:
Code review ACK faa06ecc9c
jarolrod:
ACK faa06ecc9c
fanquake:
ACK faa06ecc9c - this should be ok to do now.
Tree-SHA512: 7295472b5fd37ffb30f044e88c39d375a5a5187d3f2d44d4e73d0eb0c7fd923cf9949c2ddab6cddd8c5da7e375fff38112b6ea9779da4fecce6f024d05ba9c08
f75e0c1edd doc: add external-signer.md (Sjors Provoost)
d4b0107d68 rpc: send: support external signer (Sjors Provoost)
245b4457cf rpc: signerdisplayaddress (Sjors Provoost)
7ebc7c0215 wallet: ExternalSigner: add GetDescriptors method (Sjors Provoost)
fc5da520f5 wallet: add GetExternalSigner() (Sjors Provoost)
259f52cc33 test: external_signer wallet flag is immutable (Sjors Provoost)
2655197e1c rpc: add external_signer option to createwallet (Sjors Provoost)
2700f09c41 rpc: signer: add enumeratesigners to list external signers (Sjors Provoost)
07b7c940a7 rpc: add external signer RPC files (Sjors Provoost)
8ce7767071 wallet: add ExternalSignerScriptPubKeyMan (Sjors Provoost)
157ea7c614 wallet: add external_signer flag (Sjors Provoost)
f3e6ce78fb test: add external signer test (Sjors Provoost)
8cf543f96d wallet: add -signer argument for external signer command (Sjors Provoost)
f7eb7ecc67 test: framework: add skip_if_no_external_signer (Sjors Provoost)
87a97941f6 configure: add --enable-external-signer (Sjors Provoost)
Pull request description:
Big picture overview in [this gist](https://gist.github.com/Sjors/29d06728c685e6182828c1ce9b74483d).
This PR lets `bitcoind` call an arbitrary command `-signer=<cmd>`, e.g. a hardware wallet driver, where it can fetch public keys, ask to display an address, and sign a transaction (using PSBT under the hood).
It's design to work with https://github.com/bitcoin-core/HWI, which supports multiple hardware wallets. Any command with the same arguments and return values will work. It simplifies the manual procedure described [here](https://github.com/bitcoin-core/HWI/blob/master/docs/bitcoin-core-usage.md).
Usage is documented in [doc/external-signer.md](
https://github.com/Sjors/bitcoin/blob/2019/08/hww-box2/doc/external-signer.md), which also describes what protocol a different signer binary should conform to.
Use `--enable-external-signer` to opt in, requires Boost::Process:
```
Options used to compile and link:
with wallet = yes
with gui / qt = no
external signer = yes
```
It adds the following RPC methods:
* `enumeratesigners`: asks <cmd> for a list of signers (e.g. devices) and their master key fingerprint
* `signerdisplayaddress <address>`: asks <cmd> to display an address
It enhances the following RPC methods:
* `createwallet`: takes an additional `external_signer` argument and fetches keys from device
* `send`: automatically sends transaction to device and waits
Usage TL&DR:
* clone HWI repo somewhere and launch `bitcoind -signer=../HWI/hwi.py`
* check if you can see your hardware device: `bitcoin-cli enumeratesigners`
* create wallet and auto import keys `bitcoin-cli createwallet "hww" true true "" true true true`
* display address on device: `bitcoin-cli signerdisplayaddress ...`
* to spend, use `send` RPC and approve transaction on device
Prerequisites:
- [x] #21127 load wallet flags before everything else
- [x] #21182 remove mostly pointless BOOST_PROCESS macro
Potentially useful followups:
- GUI support: bitcoin-core/gui#4
- bumpfee support
- (automatically) verify (a subset of) keys on the device after import, through message signing
ACKs for top commit:
laanwj:
re-ACK f75e0c1edd
Tree-SHA512: 7db8afd54762295c1424c3f01d8c587ec256a72f34bd5256e04b21832dabd5dc212be8ab975ae3b67de75259fd569a561491945750492f417111dc7b6641e77f
25c57d6409 [doc] Add a note about where lock annotations should go. (Amiti Uttarwar)
ad5f01b960 [validation] Move the lock annotation from function definition to declaration (Amiti Uttarwar)
Pull request description:
Based on reviewing #21188
the first commit switches the lock annotations on `CheckInputScripts` to be on the function declaration instead of on the function definition. this ensures that all call sites are checked, not just ones that come after the definition.
the second commit adds a note to the developer-notes section to clarify where the annotations should be applied.
ACKs for top commit:
MarcoFalke:
ACK 25c57d6409🥘
promag:
Code review ACK 25c57d6409.
Tree-SHA512: 61b6ef856bf6c6016d535fbdd19daf57b9e59fe54a1f30d47282a071b9b9d60b2466b044ee57929e0320cb1bdef52e7a1687cacaa27031bbc43d058ffffe22ba
This option replaces --with-boost-process
This prepares external signer support to be disabled by default.
It adds a configure option to enable this feature and to check
if Boost::Process is present.
This also exposes ENABLE_EXTERNAL_SIGNER to the test suite via test/config.ini
c5da2749e2 build: actually stop configure if Boost isn't available (fanquake)
cad8b527ea build: explicitly install libboost-dev package (fanquake)
Pull request description:
If Boost is not found via AX_BOOST_BASE, we don't actually stop
configuring, only a warning is emitted:
```bash
checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version MINIMUM_REQUIRED_BOOST or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
```
Instead we usually fail when one of the other AX_BOOST_* macros fails to find a library. These macros are slowly being
removed, and in any case, it makes more sense to fail earlier if Boost is missing.
If Boost is unavailable, the failure now looks like:
```bash
checking for boostlib >= 1.58.0 (105800)... configure: We could not detect the boost libraries (version 1.58.0 or higher). If you have a staged boost library (still not installed) please specify $BOOST_ROOT in your environment and do not give a PATH to --with-boost option. If you are sure you have boost installed, then check your version number looking in <boost/version.hpp>. See http://randspringer.de/boost for more documentation.
configure: error: Boost is not available!
```
Note that we now just pass the version into AX_BOOST_BASE, which fixes it's display in the output (rather than showing `MINIMUM_REQUIRED_BOOST`).
This PR also has a commit that adds `libboost-dev` to our install instructions and CI. This package is currently installed as a side-effect of installing our other libboost-*-dev packages. However as those continue to disappear, it makes sense to install boost-dev explicitly.
ACKs for top commit:
laanwj:
Code review ACK c5da2749e2
MarcoFalke:
Concept ACK c5da2749e2
Tree-SHA512: f866062f9d7d3a2316b6c887f17c664b9cfff41fdc0cb99ca79d641240fb01a5ae0d34140e515bc465219e1b43d5ca84f7c55f48b9c5b45a80ff2795dafd072b
fd0be92cff doc: Add instructions on how to fuzz the P2P layer using Honggfuzz NetDriver (practicalswift)
Pull request description:
Add instructions on how to fuzz the P2P layer using [Honggfuzz NetDriver](http://blog.swiecki.net/2018/01/fuzzing-tcp-servers.html).
Honggfuzz NetDriver allows for very easy fuzzing of TCP servers such as Bitcoin Core without having to write any custom fuzzing harness. The `bitcoind` server process is largely fuzzed without modification.
This makes the fuzzing highly realistic: a bug reachable by the fuzzer is likely also remotely triggerable by an untrusted peer.
Top commit has no ACKs.
Tree-SHA512: 9e98cb30f00664c00c8ff9fd224ff9822bff3fd849652172df48dbaeade1dd1a5fc67ae53203f1966a1d4210671b35656009a2d8b84affccf3ddf1fd86124f6e
This package is currently installed as a side-effect of installing our
other libboost-*-dev packages. However as those continue to dissapear,
it makes sense to install boost dev explicitly.
aa929abf8d [docs] Update developer notes to discourage very long lines (John Newbery)
Pull request description:
Mandatory rules on line lengths are bad - there will always be cases where a longer line is more readable than the alternative.
However, very long lines for no good reason _do_ hurt readability. For example, this declaration in validation.h is 274 chars:
```c++
bool ConnectTip(BlockValidationState& state, const CChainParams& chainparams, CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock, ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool) EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool.cs);
```
That won't fit on one line without wrapping on my 27" monitor with a comfortable font size. Much easier to read is something like:
```c++
bool ConnectTip(BlockValidationState& state, const CChainParams& chainparams,
CBlockIndex* pindexNew, const std::shared_ptr<const CBlock>& pblock,
ConnectTrace& connectTrace, DisconnectedBlockTransactions& disconnectpool)
EXCLUSIVE_LOCKS_REQUIRED(cs_main, m_mempool.cs);
```
Therefore, _discourage_ (don't forbid) line lengths greater than 100 characters in our developer style guide.
100 chars is somewhat arbitrary. The old standard was 80, but that seems very limiting with modern displays.
ACKs for top commit:
fanquake:
ACK aa929abf8d - this is basically just something to point too when a PR has unreasonably long lines for no particularly reason.
practicalswift:
ACK aa929abf8d
amitiuttarwar:
ACK aa929abf8d
theStack:
ACK aa929abf8d
glozow:
ACK aa929abf8d
Tree-SHA512: 17f1b11f811137497ede8851ede93fa612dc622922b5ad7ac8f065ea026d9a718db5b92325754b74d24012b4d45c4e2cd5cd439a6a8d34bbabf5da927d783970
c943326d3c doc/bips: Add BIPs 43, 44, 49, and 84 (Luke Dashjr)
Pull request description:
If you don't like what they say, please suggest alternatives ;)
ACKs for top commit:
prusnak:
ACK c943326
Tree-SHA512: 7db93f8491289657ec45df30e557eb8572b35201eb29aed1b11bf3949924fce56b4e2d71e1f0acf5d24a01278c0dec99790d632f04c15117010c4ac564368d6b
060a2a64d4 ci: remove boost thread installation (fanquake)
06e1d7d81d build: don't build or use Boost Thread (fanquake)
7097add83c refactor: replace Boost shared_mutex with std shared_mutex in sigcache (fanquake)
8e55981ef8 refactor: replace Boost shared_mutex with std shared_mutex in cuckoocache tests (fanquake)
Pull request description:
This replaces `boost::shared_mutex` and `boost::unique_lock` with [`std::shared_mutex`](https://en.cppreference.com/w/cpp/thread/shared_mutex) & [`std::unique_lock`](https://en.cppreference.com/w/cpp/thread/unique_lock).
Even though [some concerns were raised](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-726214696) in #16684 with regard to `std::shared_mutex` being unsafe to use across some glibc versions, I still think this change is an improvement. As I mentioned in #21022, I also think trying to restrict standard library feature usage based on bugs in glibc is not only hard to do, but it's not currently clear exactly how we do that in practice (does it also extend to patching out use in our dependencies, should we be implementing more runtime checks for features we are using, when do we consider an affected glibc "old enough" not to worry about? etc). If you take a look through the [glibc bug tracker](https://sourceware.org/bugzilla/describecomponents.cgi?product=glibc) you'll no doubt find plenty of (active) bug reports for standard library code we already using. Obviously not to say we shouldn't try and avoid buggy code where possible.
Two other points:
[Cory mentioned in #21022](https://github.com/bitcoin/bitcoin/pull/21022#issuecomment-769274179):
> It also seems reasonable to me to worry that boost hits the same underlying glibc bug, and we've just not happened to trigger the right conditions yet.
Moving away from Boost to the standard library also removes the potential for differences related to Boosts configuration. Boost has multiple versions of `shared_mutex`, and what you end up using, and what it's backed by depends on:
* The version of Boost.
* The platform you're building for.
* Which version of `BOOST_THREAD_VERSION` is defined: (2,3,4 or 5) default=2. (see [here](https://www.boost.org/doc/libs/1_70_0/doc/html/thread/build.html#thread.build.configuration) for some of the differences).
* Is `BOOST_THREAD_V2_SHARED_MUTEX` defined? (not by default). If so, you might get the ["less performant, but more robust"](https://github.com/boostorg/thread/issues/230#issuecomment-475937761) version of `shared_mutex`.
A lot of these factors are eliminated by our use of depends, but users will have varying configurations. It's also not inconceivable to think that a distro, or some package manager might start defining something like `BOOST_THREAD_VERSION=3`. Boost tried to change the default from 2 to 3 at one point.
With this change, we no longer use Boost Thread, so this PR also removes it from depends, the build system, CI etc.
Previous similar PRs were #19183 & #20922. The authors are included in the commits here.
Also related to #21022 - pthread sanity checking.
ACKs for top commit:
laanwj:
Code review ACK 060a2a64d4
vasild:
ACK 060a2a64d4
Tree-SHA512: 572d14d8c9de20bc434511f20d3f431836393ff915b2fe9de5a47a02dca76805ad5c3fc4cceecb4cd43f3ba939a0508178c4e60e62abdbaaa6b3e8db20b75b03
e1604b3d50 doc: Replace tabs for spaces (Gunar C. Gessner)
98db48d349 doc: Fix markdown formatting (Gunar Gessner)
Pull request description:
Lines were being joined making it hard to read.
ACKs for top commit:
RandyMcMillan:
ACK e1604b3d50
Tree-SHA512: fd5a7c5e9a1cbbf0fbb13b5c30b87853c84751da7f0fad08151bda07f1933872ab51cad29a0c0a70ced48e60df6d83bff3f84c2f77d00d22723fae9a8c3534fc
fa362064e3 rpc: Return total fee in mempool (MarcoFalke)
Pull request description:
This avoids having to loop over the whole mempool to query each entry's fee
ACKs for top commit:
achow101:
ACK fa362064e3
glozow:
ACK fa362064e3🧸
jnewbery:
ACK fa362064e3
Tree-SHA512: e2fa1664df39c9e187f9229fc35764ccf436f6f75889c5a206d34fff473fc21efbf2bb143f4ca7895c27659218c22884d0ec4195e7a536a5a96973fc9dd82d08
To make release tags the `make-tag.py` script from the maintainer tools
should be used. This ensures that all the various occurences of the
version in different files match the tagged version before proceeding.
Also replace other "ping wumpus" references.
fc726e0138 doc, rpc: add missing signet mentions in network name lists (Sebastian Falbesoner)
Pull request description:
This small PR adds a few missing mentions of signet w.r.t. chain enumerations:
- RPC `getblockchaininfo`: result description for `"chain"`
- RPC `getmininginfo`: result description for `"chain"`
- REST interface documentation:
- default ports listing for each chain
- `"chain"` description for `chaininfo` endpoint result
The instances were identified via `git grep -i "main.*test.*reg"`.
ACKs for top commit:
ajtowns:
ACK fc726e0138 -- quick code review only
benthecarman:
ACK fc726e0138
Tree-SHA512: 62cdc6ef74fa10db75cc04b9eaf7367183f726b3fee3d21fdf741b3816669dd21508735e89da389ddac980f49773ab229263748d1399553375fefe4526361846
570e43fe72 guix: Print build params inside/outside of container (Carl Dong)
2f9d1fdde6 guix: Move DISTSRC determination to guix-build.sh (Carl Dong)
0b7cd07bb5 guix: Move OUTDIR determination+creation to guix-build.sh (Carl Dong)
d27ff8b86a guix: Add more sanity checks to guix-build.sh (Carl Dong)
57f9533146 guix: Add section headings to guix-build.sh (Carl Dong)
38b7b2ed72 genbuild: Specify rev-parse length (Carl Dong)
036dc740da docs: Point to contrib/guix/README.md in doc/guix.md (Carl Dong)
34f0fda2d3 guix: Small updates to README wording (Carl Dong)
402e3a5b1e guix: Update HOSTS README entry for new architectures (Carl Dong)
cfa7ceb21b guix: Remove README development environment section (Carl Dong)
93b6a8544a guix: Add ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS options (Carl Dong)
0f31e24703 guix: Add SUBSTITUTE_URLS option (Carl Dong)
444fcfca90 guix: Make guix honor MAX_JOBS setting (Carl Dong)
Pull request description:
After live-demo-ing a Guix build (which completed successfully!) on achow101's stream, I realized there were a few quality of life improvements which can be made to improve the user experience of our Guix build process. Here are a few of them.
Notable changes:
1. When `MAX_JOBS` is specified, both `guix time-machine` and `guix environment` will now build up to `MAX_JOBS` packages at a time when creating the build environment
2. The instructions for using substitutes were incorrect, and has now been replaced with a `SUBSTITUTE_URLS` environment variable, which works well with shell's IFS splitting rules
3. New `ADDITIONAL_GUIX_{COMMON,TIMEMACHINE}_FLAGS` options, for more granular customization of the build process.
4. README cleanup
ACKs for top commit:
fanquake:
ACK 570e43fe72 - lets move this forward.
Tree-SHA512: 4e8ab560522ade5efb5e8736aec0fb1a3f19ae9deb586c1ab87020816876f3f466a950b3f8c04d9fa1d072ae5ee780038c5c9063577049bdd9db17978e11c328
3e61b8c800 doc: Add explicit macdeployqtplus dependencies install step (Hennadii Stepanov)
Pull request description:
This PR adds to macOS docs an explicit step to install `macdeployqtplus` script dependencies that are not part of the [Python Standard Library](https://docs.python.org/3/library/index.html):
- https://pypi.org/project/ds-store/
- https://pypi.org/project/mac-alias/
This change is required on macOS 11 Big Sur:
- #20371
- #20878Close#20878.
ACKs for top commit:
fanquake:
ACK 3e61b8c800
Tree-SHA512: d177139ee142d47cb27ad878d721cafcd03403ef861965ff532d712da461416380ec5878f70accf223a552a1f1e65eedb1e0ad72cb7a96791f8a55536ce28645
