ff42d81383 guix: use clang-toolchain-15 for macOS compilation (fanquake)
94955b4b1d depends: use LLVM/Clang 15.0.6 for macOS cross-compile (fanquake)
Pull request description:
This will end up being a blocker for #28210, and is already part of #21778, even though an even newer LLVM/Clang combination is required (and still missing from upstream Guix). Seems straight-forward enough to just bump the macOS compiler to a more modern Clang.
ACKs for top commit:
TheCharlatan:
re-ACK ff42d81383
Tree-SHA512: 8af4b54c3a56abb3825c6470444a28e14e9c69820c09ec4a33acebb8ae434df9ae18163c088a582130cc68755293a7e2bde5d065763919d94064ff9b3f83730f
When estimating the maximum size of an input, we were assuming the
number of elements on the witness stack could be encode in a single
byte. This is a valid approximation for all the descriptors we support
(including P2WSH Miniscript ones), but may not hold anymore once we
support Miniscript within Taproot descriptors (since the max standard
witness stack size of 100 gets lifted).
It's a low-hanging fruit to account for it correctly, so just do it now.
Instead of using the dummysigner to compute a placeholder satisfaction,
infer a descriptor on the scriptPubKey of the coin being spent and use
the estimation of the satisfaction size given by the descriptor
directly.
Note this (almost, see next paragraph) exactly conserves the previous
behaviour. For instance CalculateMaximumSignedInputSize was previously
assuming the input to be spent in a transaction that spends at least one
Segwit coin, since it was always accounting for the serialization of the
number of witness elements.
In this commit we use a placeholder for the size of the serialization of
the witness stack size (1 byte). Since the logic in this commit is
already tricky enough to review, and that it is only a very tiny
approximation not observable through the existing tests, it is addressed
in the next commit.
It is sometimes useful to interface with multiple signing providers at
once. For instance when inferring a descriptor with solving information
being provided from multiple sources (see next commit).
Instead of inneficiently copying the information from one provider into
the other, introduce a new signing provider that takes a list of
pointers to existing providers.
In the wallet code, we are currently estimating the size of a signed
input by doing a dry run of the signing logic. This is unnecessary as
all outputs we are able to sign for can be represented by a descriptor,
and we can derive the size of a satisfaction ("signature") from the
descriptor itself directly.
In addition, this approach does not scale: getting the size of a
satisfaction through a dry run of the signing logic is only possible for
the most basic scripts.
This commit introduces the computation of the size of satisfaction per
descriptor. It's a bit intricate for 2 main reasons:
- We want to conserve the behaviour of the current dry-run logic used by
the wallet that sometimes assumes ECDSA signatures will be low-r,
sometimes not (when we don't create them).
- We need to account for the witness discount. A single descriptor may
sometimes benefit of it, sometimes not (for instance `pk()` if used as
top-level versus if used inside `wsh()`).
Similarly to how we compute the maximum stack size.
Also note how it would be quite expensive to recompute it recursively
by accounting for different ECDSA signature sizes. So we just assume
high-R everywhere. It's only a trivial difference anyways.
b3a93b409e test: add functional test for deadlock situation (Martin Zumsande)
3557aa4d0a test: add basic tests for sendmsgtopeer to rpc_net.py (Martin Zumsande)
a9a1d69391 rpc: add test-only sendmsgtopeer rpc (Martin Zumsande)
Pull request description:
This adds a `sendmsgtopeer` rpc (for testing only) that allows a node to send a message (provided in hex) to a peer.
While we would usually use a `p2p` object instead of a node for this in the test framework, that isn't possible in situations where this message needs to trigger an actual interaction of multiple nodes.
Use this rpc to add test coverage for the bug fixed in #27981 (that just got merged):
The test lets two nodes (almost) simultaneously send a single large (4MB) p2p message to each other, which would have caused a deadlock previously (making this test fail), but succeeds now.
As can be seen from the discussion in #27981, it was not easy to reproduce this bug without `sendmsgtopeer`. I would imagine that `sendmsgtopeer` could also be helpful in various other test constellations.
ACKs for top commit:
ajtowns:
ACK b3a93b409e
sipa:
ACK b3a93b409e
achow101:
ACK b3a93b409e
Tree-SHA512: 6e22e72402f3c4dd70cddb9e96ea988444720f7a164031df159fbdd48056c8ac77ac53def045d9208a3ca07437c7c8e34f8b4ebc7066c0a84d81cd53f2f4fa5f
c8e066461b doc: Improve documentation of rpcallowip rpchelp (willcl-ark)
Pull request description:
Closes#21070
v21.0 introduced a behaviour changed noted in #21070 where using a config value `rpcallowip=::0` no longer also permitted ipv4 ip addresses.
The rpc_bind.py functional test covers this new behaviour already by checking that the list of bind addresses exactly matches what is expected so this commit only updates the documentation.
ACKs for top commit:
achow101:
ACK c8e066461b
pinheadmz:
ACK c8e066461b
jonatack:
ACK c8e066461b
Tree-SHA512: 332060cf0df0427c6637a9fd1e0783ce0b0940abdb41b0df13f03bfbdc28af067cec8f0b1bbc4e47b3d54fa1b2f110418442b05b39d5e7c7e0b96744ddd7c003
bf26f978ff fuzz: coinselection, fix `m_cost_of_change` (brunoerg)
6d9b26d56a fuzz: coinselection, BnB should never produce change (brunoerg)
b2eb558407 fuzz: coinselection, compare `GetSelectedValue` with target (brunoerg)
0df0438c60 fuzz: coinselection, improve `ComputeAndSetWaste` (brunoerg)
1e351e5db1 fuzz: coinselection, add coverage for `Merge` (brunoerg)
f0244a8614 fuzz: coinselection, add coverage for `GetShuffledInputVector`/`GetInputSet` (brunoerg)
808618b8a2 fuzz: coinselection, add coverage for `AddInputs` (brunoerg)
90c4e6a241 fuzz: coinselection, add coverage for `EligibleForSpending` (brunoerg)
2a031cb2c2 fuzz: coinselection, add `CreateCoins` (brunoerg)
Pull request description:
This PR:
- Moves coin creation to its own function called `CreateCoins`.
- Add coverage for `EligibleForSpending`
- Add coverage for `AddInputs`: get result of each algorithm (srd, knapsack and bnb), call `CreateCoins` and add into them.
- Add coverage for `GetShuffledInputVector` and `GetInputSet` using the result of each algorithm (srd, knapsack and bnb).
- Add coverage for `Merge`: Call SRD with the new utxos and, if successful, try to merge with the previous SRD result.
ACKs for top commit:
murchandamus:
reACK with some minimal fuzzing bf26f978ff
achow101:
ACK bf26f978ff
furszy:
re-ACK bf26f97
Tree-SHA512: bdd2b0a39de37be0a9b21a7c51260b6b8abe538cc0ea74312eb658b90a121a1ae07306c09fb0e75e93b531ce9ea2402feb041b0d852902d07739257f792e64ab
8a3b6f3387 refactor: make Transport::ReceivedBytes just return success/fail (Pieter Wuille)
bb4aab90fd net: move message conversion to wire bytes from PushMessage to SocketSendData (Pieter Wuille)
a1a1060fd6 net: measure send buffer fullness based on memory usage (Pieter Wuille)
009ff8d650 fuzz: add bidirectional fragmented transport test (Pieter Wuille)
fb2c5edb79 net: make V1Transport implicitly use current chainparams (Pieter Wuille)
0de48fe858 net: abstract sending side of transport serialization further (Pieter Wuille)
649a83c7f7 refactor: rename Transport class receive functions (Pieter Wuille)
27f9ba23ef net: add V1Transport lock protecting receive state (Pieter Wuille)
93594e42c3 refactor: merge transport serializer and deserializer into Transport class (Pieter Wuille)
Pull request description:
This PR furthers the P2P message serialization/deserialization abstraction introduced in #16202 and #16562, in preparation for introducing the BIP324 v2 transport (making this part of #27634). However, nothing in this PR is BIP324-specific, and it contains a number of independently useful improvements.
The overall idea is to have a single object in every `CNode` (called `m_transport`) that is responsible for converting sent messages to wire bytes, and for converting received wire bytes back to messages, while having as little as possible knowledge about this conversion process in higher-level net code. To accomplish that, there is an abstract `Transport` class with (currently) a single `V1Transport` implementation.
Structurally, the above is accomplished by:
* Merging the `TransportDeserializer` and `TransportSerializer` classes into a single `Transport` class, which encompasses both the sending and receiving side. For `V1Transport` these two sides are entirely separate, but this assumption doesn't hold for the BIP324 transport where e.g. the sending encryption key depends on the DH key negotiation data received from the other side. Merging the two means a future `V2Transport` can handle all this interaction without callers needing to be aware.
* Removing the assumption that each message is sent using a computed header followed by (unmodified) data bytes. To achieve that, the sending side of `Transport` mirrors what the receiver side does: callers can set a message to be sent, then ask what bytes must be sent out, and then allowing them to transition to the next message.
* Adding internal locks to protect the sending and receiving state of the `V1Transport` implementation. I believe these aren't strictly needed (opinions welcome) as there is no real way to use `Transport` objects in a multi-threaded fashion without some form of external synchronization (e.g. "get next bytes to send" isn't meaningful to call from multiple threads at the same time without mechanism to control the order they'll actually get sent). Still, I feel it's cleaner to make the object responsible for its own consistency (as we definitely do not want the entire object to be under a single external GUARDED_BY, as that'd prevent simultaneous sending and receiving).
* Moving the conversion of messages to bytes on the sending side from `PushMessage` to `SocketSendData`, which is needed to deal with the fact that a transport may not immediately be able to send messages.
This PR is not a refactor, though some commits are. Among the semantic changes are:
* Changing the send buffer pushback mechanism to trigger based on the memory usage of the buffer rather than the amount of bytes to be sent. This is both closer to the desired behavior, and makes the buffering independent from transport details (which is why it's included here).
* When optimistic send is not applicable, the V1 message checksum calculation now runs in the net thread rather than the message handling thread. I believe that's generally an improvement, as the message handling thread is far more computationally bottlenecked already.
* The checksum calculation now runs under the `CNode::cs_vSend` lock, which does mean no two checksum calculations for messages sent to the same node can run in parallel, even if running in separate threads. Despite that limitation, having the checksum for non-optimistic sends moved in the net thread is still an improvement, I believe.
* Statistics for per-message-type sent bytes are now updated when the bytes are actually handed to the OS rather than in `PushMessage`. This is because the actual serialized sizes aren't known until they've gone through the transport object.
A fuzz test of the entire `V1Transport` is included. More elaborate rationale for each of the changes can be found in the commit messages.
ACKs for top commit:
theStack:
re-ACK 8a3b6f3387
vasild:
ACK 8a3b6f3387
dergoegge:
Code review ACK 8a3b6f3387
Tree-SHA512: 26e9a6df47f1dd3e3f3edb4874edf365728e5a8bbc9d0d4d71fb6000cb2dfde5574902c47ffcf825af6743922f2ff9d31a5a38942a196f4ca6669122e15e42e4
c00000df16 rpc: Add MaybeArg() and Arg() default helper (MarcoFalke)
Pull request description:
Currently the RPC method implementations have many issues:
* Default RPC argument values (and their optionality state) are duplicated in the documentation and the C++ code, with no checks to prevent them from going out of sync.
* Getting an optional RPC argument is verbose, using a ternary operator, or worse, a multi-line `if`.
Fix all issues by adding default helper that can be called via `self.Arg<int>(0)`. The helper needs a few lines of code in the `src/rpc/util.h` header file. Everything else will be implemented in the cpp file once and if an RPC method needs it.
There is also an `self.MaybeArg<int>(0)` helper that works on any arg to return the argument, the default, or a falsy value.
ACKs for top commit:
ajtowns:
reACK c00000df16
stickies-v:
re-ACK c00000df16
TheCharlatan:
re-ACK c00000df16
Tree-SHA512: e7ddcab3faa319bc53edbdf3f89ce83389d2c4e571d5db42401620ff105e522a4a0669dad08e08cde5fd05c790aec3b806f63261a9100c2778865a489e57381e
fab7f5c01d ci: Add missing docker.io prefix to CI_IMAGE_NAME_TAG (MarcoFalke)
Pull request description:
Currently, the CI system may pick the wrong (non-native) architecture due to the missing prefix.
For example, assuming the CI_IMAGE_NAME_TAG is `debian:bookworm` and the user has previously pulled an s390x image:
```
$ podman run --rm 'docker.io/s390x/debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
Now, `debian:bookworm` will refer to the same image:
```
$ podman run --rm 'debian:bookworm' dpkg --print-architecture
exec /usr/bin/dpkg: exec format error
```
However, `docker.io/debian:bookworm` works fine:
```
$ podman run --rm 'docker.io/debian:bookworm' dpkg --print-architecture
arm64
ACKs for top commit:
hebasto:
ACK fab7f5c01d.
Tree-SHA512: c423c5cd454a95fa3e67081411ca08d316b8c680a5bba49196c514b91df65d9cc46a47700cc00d9579327842615f98146d0ac50abb016616a9b17d042598dab6
360ac64b90 test: previous releases: speed up fetching sources with shallow clone (Sebastian Falbesoner)
Pull request description:
For the sake of building previous releases, fetching the whole history of the repository for each version seems to be overkill as it takes much more time, bandwidth and disk space than necessary. Create a shallow clone instead with history truncated to the one commit of the version tag, which is directly checked out in the same command. This has the nice side-effect that we can remove the extra `git checkout` step after as it's not needed anymore.
Note that it might look confusing to pass a _tag_ to a parameter named `--branch`, but the git-clone manpage explicitly states that this is supported.
ACKs for top commit:
MarcoFalke:
lgtm ACK 360ac64b90
Tree-SHA512: c885a695c1ea90895cf7a785540c24e8ef8d1d9ea78db28143837240586beb6dfb985b8b0b542d2f64e2f0ffdca7c65fc3d55f44b5e1b22cc5535bc044566f86
c4929cfa50 test: wallet_backup.py, fix intermittent failure in "restore using dumped wallet" (furszy)
Pull request description:
Aiming to fix#25652.
The failure arises because the test expects `init_wallet()` (the test framework function) to create a wallet with no keys. However, the function also imports the deterministic private key used to receive the coinbase coins.
This causes a race within the "restore using dumped wallet" case, where we intend to have a new wallet (with no existing keys) to test the 'importwallet()' RPC result.
The reason why this failure is intermittent is that it depends on other peers delivering the chain right after node2 startup and prior to the test 'node2.getbalance()' call and also the synchronization of the validation queue.
ACKs for top commit:
MarcoFalke:
lgtm ACK c4929cfa50
Tree-SHA512: 80faa590439305576086a7d6e328f2550c97b218771fc5eba0567feff78732a2605d028a30a368d50944ae3d25fdbd6d321fb97321791a356416f2b790999613
fa5cc3ccfb test: Fix intermittent issue in mempool_reorg (MarcoFalke)
Pull request description:
Currently the test case may fail intermittently, see https://github.com/bitcoin/bitcoin/issues/28313
Fix this by changing a number and reducing the failure rate a bit.
ACKs for top commit:
glozow:
ACK fa5cc3ccfb
Tree-SHA512: ff552111b434ca712c7dbdc5ba32a986a6fa4512cba5a756234eae69428063bf6ecfdc8f350ee84226ed4d3e4262b4639dbe49162a722e8da85f0d61e5690c51
fa15f7e082 ci: Remove no longer applicable section (MarcoFalke)
fa378bed56 ci: Start with clean env (MarcoFalke)
fa8c250c2f ci: Limit scope of some env vars (MarcoFalke)
Pull request description:
Starting with a clean `env` should help to avoid non-determinism, such as the one fixed in https://github.com/bitcoin/bitcoin/pull/27739#issuecomment-1564529747
ACKs for top commit:
dergoegge:
utACK fa15f7e082
Tree-SHA512: 716b264217557b6524dab92d5a2a8d61ecb982dff475bd0cf5a763070b4c5916cd5995e764eb5d67d9cf2428c29d5fc2f42b32941b54c7c3053123ce448171e5
Using the new time-machine results in warnings about consistently using
keyword arguments:
```bash
guix environment: warning: 'cross-kernel-headers' must be used with keyword arguments
guix environment: warning: 'cross-libc' must be used with keyword arguments
```
This is required for bumping the time-machine, for compatibility with
OpenSSL:
oscrypto: openssl backend, 1.2.1, /tmp/guix-build-python-oscrypto-1.2.1.drv-0/source/oscrypto
Traceback (most recent call last):
File "/tmp/guix-build-python-oscrypto-1.2.1.drv-0/source/oscrypto/_openssl/_libcrypto_ctypes.py", line 304, in <module>
libcrypto.EVP_PKEY_size.argtypes = [
File "/gnu/store/9dkl9fnidcdpw19ncw5pk0p7dljx7ijb-python-3.10.7/lib/python3.10/ctypes/__init__.py", line 387, in __getattr__
func = self.__getitem__(name)
File "/gnu/store/9dkl9fnidcdpw19ncw5pk0p7dljx7ijb-python-3.10.7/lib/python3.10/ctypes/__init__.py", line 392, in __getitem__
func = self._FuncPtr((name_or_ordinal, self))
AttributeError: /gnu/store/2hr7w64zhr6jjznidyc2xi40d5ynhj9c-openssl-3.0.8/lib/libcrypto.so.3: undefined symbol: EVP_PKEY_size. Did you mean: 'EVP_PKEY_free'?
806b75b213 guix: consolidate Linux GCC package (fanquake)
4415275f96 guix: consolidate glibc 2.27 package (fanquake)
Pull request description:
This is some refactoring to the Linux Guix build that facilitates bumping our Guix time-machine. Namely, avoiding `package-with-extra-configure-variable`, which is non-functional in the newer time-machine, see https://issues.guix.gnu.org/64436.
At the same time, consolidate our Linux GCC build into `linux-base-gcc`. Now that we only use `building-on`, remove `explicit-cross-configure`.
Split out of https://github.com/bitcoin/bitcoin/pull/27897. Most of the [[WIP] Linux commit](8335fc4775), minus anything GCC 12 related.
I'll also be splitting out the other changes we can do pre-timemachine bump, for easier review.
Similar/followup to #28294. Requirement for #28328.
Guix Build:
```bash
17463110d4b4721a7c188e71b1fc00c9b5b82227aa8342471390c17678e04a9a guix-build-806b75b21340/output/aarch64-linux-gnu/SHA256SUMS.part
0ca919ce568e7d4ffe44dda871d48963ca2988516068e75b1f30ca342d853d20 guix-build-806b75b21340/output/aarch64-linux-gnu/bitcoin-806b75b21340-aarch64-linux-gnu-debug.tar.gz
134afa263e4da6c8d7df79a7dd8e59911c1e643b53b7c285de9418d97fb06d5f guix-build-806b75b21340/output/aarch64-linux-gnu/bitcoin-806b75b21340-aarch64-linux-gnu.tar.gz
22ce318241084829e57f064bf47de57752151863aac545e643eea5dd8eee96fe guix-build-806b75b21340/output/arm-linux-gnueabihf/SHA256SUMS.part
a26fecfeb821040704ca70ea056bff796789ed9500d9575d8fa13a85b32143f6 guix-build-806b75b21340/output/arm-linux-gnueabihf/bitcoin-806b75b21340-arm-linux-gnueabihf-debug.tar.gz
213c84494835c81a40ebc5d38a62bb19cbee5b214b2a8aaed6d28746b245108e guix-build-806b75b21340/output/arm-linux-gnueabihf/bitcoin-806b75b21340-arm-linux-gnueabihf.tar.gz
ce1efcf6d3ca0e7422c5ce35f5e45e0770a3ae48173e061137daa7dc551e9d48 guix-build-806b75b21340/output/arm64-apple-darwin/SHA256SUMS.part
fc01aaeb4e4722d21fd60c78f1b5322c9875ec6fb4d244f4547a354e91a33ed7 guix-build-806b75b21340/output/arm64-apple-darwin/bitcoin-806b75b21340-arm64-apple-darwin-unsigned.dmg
632e4a243d3e4144313f53047499f91b7c9380a1a50f5846e1635d0a00fd202a guix-build-806b75b21340/output/arm64-apple-darwin/bitcoin-806b75b21340-arm64-apple-darwin-unsigned.tar.gz
8e694e4cd1bf45e6a586a0d8c19b675014f168f342f881a9ae0c4fbbda796914 guix-build-806b75b21340/output/arm64-apple-darwin/bitcoin-806b75b21340-arm64-apple-darwin.tar.gz
dad6e8475f13dac6c4f0b182f53dc330997e9e1e5cf4d46413655f319dcd9bff guix-build-806b75b21340/output/dist-archive/bitcoin-806b75b21340.tar.gz
32e8b6c7e7a7561e132c5f15e2151a51aad1c5004ab90a36a7e80f92c200ef6f guix-build-806b75b21340/output/powerpc64-linux-gnu/SHA256SUMS.part
9033e85e03bd12a3a19599735cfd44fcfdfb1bf1b632733341cec6a4f75ff86f guix-build-806b75b21340/output/powerpc64-linux-gnu/bitcoin-806b75b21340-powerpc64-linux-gnu-debug.tar.gz
72698691b27ec0ac17f21dce8551de0ca683dd00b5b9442ea7616fb56cca8c6b guix-build-806b75b21340/output/powerpc64-linux-gnu/bitcoin-806b75b21340-powerpc64-linux-gnu.tar.gz
ef7c6d7184249eb59fa67d6df91d1a567570b9fb026dbb8682763029decaacca guix-build-806b75b21340/output/powerpc64le-linux-gnu/SHA256SUMS.part
fc6bb5619ee76899a88c1dd62640b429ad8957bbdf821238038b41fc87d18eca guix-build-806b75b21340/output/powerpc64le-linux-gnu/bitcoin-806b75b21340-powerpc64le-linux-gnu-debug.tar.gz
0eceb969f41f6b8dba88f641e268590de7edf0008318c8051d9cb208fb15e7f7 guix-build-806b75b21340/output/powerpc64le-linux-gnu/bitcoin-806b75b21340-powerpc64le-linux-gnu.tar.gz
6f51a4791d87a610abd81cee83efa7f469e905829797bc2edac4fb95a2e0f3e4 guix-build-806b75b21340/output/riscv64-linux-gnu/SHA256SUMS.part
c978706988f31e65a7991ff7582d79b3d1df44249c14d9807d93c01bf3f5080d guix-build-806b75b21340/output/riscv64-linux-gnu/bitcoin-806b75b21340-riscv64-linux-gnu-debug.tar.gz
93aefe058025067550692adae59ead14228ac252a9e7cf8b55c8fb4189ece545 guix-build-806b75b21340/output/riscv64-linux-gnu/bitcoin-806b75b21340-riscv64-linux-gnu.tar.gz
862a53f6023bd1ca98a078ea540bba8ca9bfa335a9560f3d8d62ac873c2d5848 guix-build-806b75b21340/output/x86_64-apple-darwin/SHA256SUMS.part
8f632b42c94d061fa30364087e75bb8b04eb2ac5a0a988eacc37c5983669f01b guix-build-806b75b21340/output/x86_64-apple-darwin/bitcoin-806b75b21340-x86_64-apple-darwin-unsigned.dmg
ce62e76ca446a6316b31490e12463c0a641e15beef9bdae7acc8e5db057b433a guix-build-806b75b21340/output/x86_64-apple-darwin/bitcoin-806b75b21340-x86_64-apple-darwin-unsigned.tar.gz
f57b014818e3b1ec07d27c8224ec4ac0e5786dacd00639513b599c6138790ece guix-build-806b75b21340/output/x86_64-apple-darwin/bitcoin-806b75b21340-x86_64-apple-darwin.tar.gz
70e75f6f13795f968f91719d221673b687bf747f90d77912cbcb2c1ee45ec623 guix-build-806b75b21340/output/x86_64-linux-gnu/SHA256SUMS.part
30dec525364bb21a26cfe8bfff061d013c4ce849165aa67b06eb154019444862 guix-build-806b75b21340/output/x86_64-linux-gnu/bitcoin-806b75b21340-x86_64-linux-gnu-debug.tar.gz
d8b3a996f25fb948b3555d5750852aaf82f7051848586b9ba0f4d0d223226e4b guix-build-806b75b21340/output/x86_64-linux-gnu/bitcoin-806b75b21340-x86_64-linux-gnu.tar.gz
4259adec77912bab6494f71a2a95d98093b116c05fc9ad03069e92de4ce0248c guix-build-806b75b21340/output/x86_64-w64-mingw32/SHA256SUMS.part
0a2d5cab3fe94a86def0cc1b6efe9ac871839cbcdc05ad92686df1d2bdd154ea guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64-debug.zip
d2a1876333bdb1cd5b8b1d4a52bccd756ea2e992c291dac233e65beeb0c905fd guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64-setup-unsigned.exe
192ea38d70e12c23327ff811ea930b50ac31c9fb2bc8dcc9391ad585112322ff guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64-unsigned.tar.gz
474f88a1f4cc8900a7d8967909336d4122e449ce98cacaf2cacec340780ede0b guix-build-806b75b21340/output/x86_64-w64-mingw32/bitcoin-806b75b21340-win64.zip
```
ACKs for top commit:
TheCharlatan:
Nice cleanups, ACK 806b75b213
Tree-SHA512: cede797c3b9b88cc1588d0ff7ff9b2908316a8ba384d9087b16466aceeb2e0c194aa56e3023f6b6ce7ca8896a1b87ef56b966db198cc1712cb6ddc37fe684567
For the sake of building previous releases, fetching the whole history
of the repository for each version seems to be overkill as it takes much
more time, bandwidth and disk space than necessary. Create a shallow
clone instead with history truncated to the one commit of the version
tag, which is directly checked out in the same command. This has the
nice side-effect that we can remove the extra `git checkout` step after
as it's not needed anymore.
Note that it might look confusing to pass a _tag_ to a parameter named
`--branch`, but the git-clone manpage explicitly states that this is
supported.
This furthers transport abstraction by removing the assumption that a message
can always immediately be converted to wire bytes. This assumption does not hold
for the v2 transport proposed by BIP324, as no messages can be sent before the
handshake completes.
This is done by only keeping (complete) CSerializedNetMsg objects in vSendMsg,
rather than the resulting bytes (for header and payload) that need to be sent.
In SocketSendData, these objects are handed to the transport as permitted by it,
and sending out the bytes the transport tells us to send. This also removes the
nSendOffset member variable in CNode, as keeping track of how much has been sent
is now a responsability of the transport.
This is not a pure refactor, and has the following effects even for the current
v1 transport:
* Checksum calculation now happens in SocketSendData rather than PushMessage.
For non-optimistic-send messages, that means this computation now happens in
the network thread rather than the message handler thread (generally a good
thing, as the message handler thread is more of a computational bottleneck).
* Checksum calculation now happens while holding the cs_vSend lock. This is
technically unnecessary for the v1 transport, as messages are encoded
independent from one another, but is untenable for the v2 transport anyway.
* Statistics updates about per-message sent bytes now happen when those bytes
are actually handed to the OS, rather than at PushMessage time.
This more accurately captures the intent of limiting send buffer size, as
many small messages can have a larger overhead that is not counted with the
current approach.
It also means removing the dependency on the header size (which will become
a function of the transport choice) from the send buffer calculations.
This adds a simulation test, with two V1Transport objects, which send messages
to each other, with sending and receiving fragmented into multiple pieces that
may be interleaved. It primarily verifies that the sending and receiving side
are compatible with each other, plus a few sanity checks.
The rest of net.cpp already uses Params() to determine chainparams in many
places (and even V1Transport itself does so in some places).
Since the only chainparams dependency is through the message start characters,
just store those directly in the transport.
This makes the sending side of P2P transports mirror the receiver side: caller provides
message (consisting of type and payload) to be sent, and then asks what bytes must be
sent. Once the message has been fully sent, a new message can be provided.
This removes the assumption that P2P serialization of messages follows a strict structure
of header (a function of type and payload), followed by (unmodified) payload, and instead
lets transports decide the structure themselves.
It also removes the assumption that a message must always be sent at once, or that no
bytes are even sent on the wire when there is no message. This opens the door for
supporting traffic shaping mechanisms in the future.
Now that the Transport class deals with both the sending and receiving side
of things, make the receive side have function names that clearly indicate
they're about receiving.
* Transport::Read() -> Transport::ReceivedBytes()
* Transport::Complete() -> Transport::ReceivedMessageComplete()
* Transport::GetMessage() -> Transport::GetReceivedMessage()
* Transport::SetVersion() -> Transport::SetReceiveVersion()
Further, also update the comments on these functions to (among others) remove
the "deserialization" terminology. That term is better reserved for just the
serialization/deserialization between objects and bytes (see serialize.h), and
not the conversion from/to wire bytes as performed by the Transport.
Rather than relying on the caller to prevent concurrent calls to the
various receive-side functions of Transport, introduce a private m_cs_recv
inside the implementation to protect the lock state.
Of course, this does not remove the need for callers to synchronize calls
entirely, as it is a stateful object, and e.g. the order in which Receive(),
Complete(), and GetMessage() are called matters. It seems impossible to use
a Transport object in a meaningful way in a multi-threaded way without some
form of external synchronization, but it still feels safer to make the
transport object itself responsible for protecting its internal state.
This allows state that is shared between both directions to be encapsulated
into a single object. Specifically the v2 transport protocol introduced by
BIP324 has sending state (the encryption keys) that depends on received
messages (the DH key exchange). Having a single object for both means it can
hide logic from callers related to that key exchange and other interactions.
27b168b81f Update help text for spend and rawtransaction rpcs (Michael Tidwell)
Pull request description:
The "data" field without outputs was marked as "required" in the help docs when using bitcoin-cli. This field when left off worked as an intended optional OP_RETURN. closes#27828.
Motivation: It is hard to understand that "data" is actually optional for commands like `createpsbt` and `walletcreatefundedpsbt`.
ACKs for top commit:
achow101:
ACK 27b168b81f
Sjors:
tACK 27b168b81f
Tree-SHA512: 235e7ed4af69880880c04015b3f7de72c8f31ae035485c4c64c483e282948f3ea3f1eef16f15e260a1aaf21114150713516ba6a99967ccad9ecd91ff67cb0450
