13adbf733f remove unneeded environment option from cpp-subprocess (Sebastian Falbesoner)
2088777ba0 remove unneeded cwd option from cpp-subprocess (Sebastian Falbesoner)
03ffb09c31 remove unneeded bufsize option from cpp-subprocess (Sebastian Falbesoner)
79c3036373 remove unneeded close_fds option from cpp-subprocess (Sebastian Falbesoner)
62db8f8e5a remove unneeded session_leader option from cpp-subprocess (Sebastian Falbesoner)
80d008c66d remove unneeded defer_spawn option from cpp-subprocess (Sebastian Falbesoner)
cececad7b2 remove unneeded preexec function option from cpp-subprocess (Sebastian Falbesoner)
633e45b2e2 remove unneeded shell option from cpp-subprocess (Sebastian Falbesoner)
Pull request description:
The newly introduced cpp-subprocess library provides a good number of options for the `Popen` class:
0de63b8b46/src/util/subprocess.hpp (L1009-L1020)
Some of them are either not fully implemented (`shell`, missing an implementation on Windows), implemented in an ugly way (e.g. using "Impoverished, meager, needy, truly needy version of type erasure" for `preexec_func` according to the author's own words) or simply unlikely to be ever needed for our external signer use-case (`defer_spawn`). Instead of maintaining incomplete and/or unneeded code, I'd suggest to get rid of it and only keep support for options if there is a strong reason for it.
ACKs for top commit:
achow101:
ACK 13adbf733f
hebasto:
re-ACK 13adbf733f.
Tree-SHA512: 8270da27891cb659da2ef6062a23f4b86331859b15ac27b79ae7433b14f5bd7efaba621f2b3ba1953708d0f38377a8bd23ef1cc0f28b9c152ac8958dd9eec6b0
6d91cb781c test: add unit tests for `calculate_input_weight` (Sebastian Falbesoner)
f81fad5e0f test: introduce and use `calculate_input_weight` helper (Sebastian Falbesoner)
Pull request description:
Rather than manually estimating an input's weight by adding up all the involved components (fixed-size skeleton, compact-serialized lengths, and the actual scriptSig / witness stack items) we can simply take use of the serialization classes `CTxIn` / `CTxInWitness` instead, to achieve the same with significantly less code.
The new helper is used in the functional tests rpc_psbt.py and wallet_send.py, where the previous manual estimation code was
duplicated. Unit tests are added in the second commit.
ACKs for top commit:
kevkevinpal:
tACK [6d91cb7](6d91cb781c)
QureshiFaisal:
tACK [6d91cb7](6d91cb781c)
achow101:
ACK 6d91cb781c
AngusP:
tACK 6d91cb781c
rkrux:
tACK [6d91cb7](6d91cb781c)
Tree-SHA512: 04424e4d94d0e13745a9c11df2dd3697c98552bbb0e792c4af67ecbb66060adc3cc0cefc202cdee2d9db0baf85b8bedf2eb339ac4b316d986b5f10f6b70c5a33
You can use this tool to decode the utxo snapshot https://github.com/jrakibi/utxo-live
Here’s an overview of how it’s done:
The serialization forma for a UTXO in the snapshot is as follows:
1. Transaction ID (txid) - 32 bytes
2. Output Index (outnum)- 4 bytes
3. VARINT (code) - A varible-length integer encoding the height and whether the transaction is a coinbase. The format of this VARINT is (height << 1) | coinbase_flag.
4. VARINT (amount_v) - A variable-length integer that represents a compressed format of the output amount (in satoshis).
For the test cases mentioned:
* b"\x84\x58" - This value corresponds to a VARINT representing the height and coinbase flag. Once we decode this code, we can extract the height and coinbase using height = code_decoded >> 1 and coinbase = code_decoded & 0x01. In our case, with code_decoded = 728, it results in height = 364 and coinbase = 0.
* b"\xCA\xD2\x8F\x5A" - This byte sequence represents a compressed amount value. The decompression function takes this value and translates it into a full amount in satoshis. In our case, the decompression of this amount translates to a number larger than the maximum allowed value of coins (21 million BTC)
test:Validate UTXO snapshot with coin_height > base_height & amount > money_supply
test:Validate UTXO snapshot with coin_height > base_height & amount > money_supply
fa6c300a99 test: Fix intermittent timeout in p2p_tx_download.py (MarcoFalke)
Pull request description:
Currently the test passes, but may fail during shutdown, because blocks and transactions are synced with `NUM_INBOUND` * `self.num_nodes` peers, which may take a long time.
There is no need for this test to have this amount of inbounds.
So avoid the extraneous inbounds to speed up the test and avoid the intermittent test failures.
ACKs for top commit:
instagibbs:
ACK fa6c300a99
fjahr:
Thanks, ACK fa6c300a99
achow101:
ACK fa6c300a99
theStack:
ACK fa6c300a99
Tree-SHA512: 0a480fd1db293ed8571ae629557cf81d5a79ec883e9e635f22c8a7cf48427161249ad2180b66c67661306f696c977b8e06ad520bd11911f119c9c95b3ffc9134
c3e632b441 Bugfix: bitcoin-cli: Check length of peer.transport_protocol_type (Luke Dashjr)
Pull request description:
"v" would dereference beyond the string length, and "v10" would show as '1'
Turn both of these cases into a blank, like anything else unexpected currently is.
ACKs for top commit:
sipa:
utACK c3e632b441.
hernanmarino:
utACK c3e632b441
alfonsoromanz:
ACK c3e632b441
achow101:
ACK c3e632b441
Tree-SHA512: f641e4412521adae7c8c8e1f268bdaaa223d9048d8286e3df4b13905faaa0d601155ce581cd649f760cab2acc4122356fa94a44714f1f190845552100105eda0
The script provided for signature might be externally provided, for
instance by way of 'finalizepsbt'. Therefore the script might be
ill-crafted, so don't assume pubkeys are always 32 bytes.
Thanks to Niklas for finding this.
f2e3662e57 net: Decrease nMaxIPs when learning from DNS seeds (laanwj)
Pull request description:
Limit number of IPs learned from a single DNS seed to 32, to prevent the results from one DNS seed from dominating AddrMan. Note that the number of results from a UDP DNS query is bounded to 33 already, but it is possible for it to use TCP where a larger number of results can be returned.
Closes#16070.
ACKs for top commit:
Sjors:
utACK f2e3662e57
achow101:
ACK f2e3662e57
1440000bytes:
utACK f2e3662e57
mzumsande:
utACK f2e3662e57
Tree-SHA512: 3f108c2baba7adfedb8019daaf60aa00e628b38d3942e1319c7183a4683670be01929ced9e6372c8e983c902e8633f81fbef12d7cdcaadd7f77ed729c1019942
6b02c11d66 test: Fix intermittent issue in p2p_handshake.py (stratospher)
Pull request description:
When establishing outbound connections [`TestNode` --------> `P2PConnection`], `P2PConnection` listens for a single connection from `TestNode` on a [port which is fixed based on `p2p_idx`](312f54278f/test/functional/test_framework/p2p.py (L746)).
If we reuse the same port when disconnecting and establishing connections again, we might hit this scenario where:
- disconnection is done on python side for `P2PConnection`
- disconnection not complete on c++ side for `TestNode`
- we're trying to establish a new connection on same port again
Prevent this scenario from happening by ensuring disconnection on c++ side for TestNode as well.
One way to reproduce this on master would be adding a sleep statement before disconnection happens on c++ side.
```diff
diff --git a/src/net.cpp b/src/net.cpp
index e388f05b03..62507d1f39 100644
--- a/src/net.cpp
+++ b/src/net.cpp
@@ -2112,6 +2112,7 @@ void CConnman::SocketHandlerConnected(const std::vector<CNode*>& nodes,
if (!pnode->fDisconnect) {
LogPrint(BCLog::NET, "socket closed for peer=%d\n", pnode->GetId());
}
+ std::this_thread::sleep_for(std::chrono::milliseconds(1000));
pnode->CloseSocketDisconnect();
}
else if (nBytes < 0)
```
ACKs for top commit:
maflcko:
lgtm ACK 6b02c11d66
mzumsande:
Tested ACK 6b02c11d66
BrandonOdiwuor:
Tested ACK 6b02c11d66
theStack:
Tested ACK 6b02c11d66
glozow:
ACK 6b02c11d66
Tree-SHA512: 69509edb61ba45739fd585b6cc8a254f412975c124a5b5a52688288ecaaffd264dd76019b8290cc34c26c3ac2dfe477965ee5a11d7aabdd8e4d2a75229a4a068
21d0e6c7b7 doc: release notes for PR 27679 (Matthew Zipkin)
791dea204e test: cover unix sockets in zmq interface (Matthew Zipkin)
c87b0a0ff4 zmq: accept unix domain socket address for notifier (Matthew Zipkin)
Pull request description:
This is a follow-up to https://github.com/bitcoin/bitcoin/pull/27375, allowing ZMQ notifications to be published to a UNIX domain socket.
Fortunately, libzmq handles unix sockets already, all we really have to do to support it is allow the format in the actual option.
[libzmq](https://libzmq.readthedocs.io/en/latest/zmq_ipc.html) uses the prefix `ipc://` as opposed to `unix:` which is [used by Tor](https://gitlab.torproject.org/tpo/core/tor/-/blob/main/doc/man/tor.1.txt?ref_type=heads#L1475) and now also by [bitcoind](a85e5a7c9a/doc/release-notes-27375.md (L5)) so we need to switch that internally.
As far as I can tell, [LND](d20a764486/zmq.go (L38)) supports `ipc://` and `unix://` (notice the double slashes).
With this patch, LND can connect to bitcoind using unix sockets:
Example:
*bitcoin.conf*:
```
zmqpubrawblock=unix:/tmp/zmqsb
zmqpubrawtx=unix:/tmp/zmqst
```
*lnd.conf*:
```
bitcoind.zmqpubrawblock=ipc:///tmp/zmqsb
bitcoind.zmqpubrawtx=ipc:///tmp/zmqst
```
ACKs for top commit:
laanwj:
Code review ACK 21d0e6c7b7
tdb3:
crACK for 21d0e6c7b7. Changes lgtm. Will follow up with some testing within the next few days as time allows.
achow101:
ACK 21d0e6c7b7
guggero:
Tested and code review ACK 21d0e6c7b7
Tree-SHA512: ffd50222e80dd029d903e5ddde37b83f72dfec1856a3f7ce49da3b54a45de8daaf80eea1629a30f58559f4b8ded0b29809548c0638cd1c2811b2736ad8b73030
016ed248ba fuzz: explicitly cap the vsize of RBFs for diagram checks (Greg Sanders)
Pull request description:
In master we are hitting a case where vsize transactions much larger than max standard size are causing an overflow in not-yet-exposed RBF diagram checking code: https://github.com/bitcoin/bitcoin/pull/29757#issuecomment-2049220195
`ConsumeTxMemPoolEntry` is creating entries with tens of thousands of sigops cost, causing the resulting RBFs to be "overly large".
To fix this I cause the fuzz test to stop adding transactions to the mempool when we reach a potential overflow of `int32_t`.
ACKs for top commit:
glozow:
ACK 016ed248ba
marcofleon:
ACK 016ed248ba. I ran libFuzzer on `package_rbf` on the current master branch until the overflow was encountered. Then I built the PR branch and ran the fuzzer using the crash input.
Tree-SHA512: b3ffc98d2c4598eb3010edd58b9370aab1441aafbb1044c83b2b90c17dfe9135b8de9dba475dd0108863c1ffedede443cd978e95231a41cf1f0715629197fa51
60ca5d5508 test: p2p: add test for rejected tx request logic (`m_recent_rejects` filter) (Sebastian Falbesoner)
e9dc511a7e fixup: get all utxos up front in fill_mempool, discourage wallet mixing (glozow)
Pull request description:
Motivated by the discussion in #28970 (https://github.com/bitcoin/bitcoin/pull/28970#discussion_r1553911167), this PR adds test coverage for the logic around the `m_recent_rejects` filter, in particular that the filter is cleared after a new block comes in:
f0794cbd40/src/net_processing.cpp (L2199-L2206)
As expected, the second part of the test fails if the following patch is applied:
```diff
diff --git a/src/net_processing.cpp b/src/net_processing.cpp
index 6996af38cb..5cb1090e70 100644
--- a/src/net_processing.cpp
+++ b/src/net_processing.cpp
@@ -2202,7 +2202,7 @@ bool PeerManagerImpl::AlreadyHaveTx(const GenTxid& gtxid)
// or a double-spend. Reset the rejects filter and give those
// txs a second chance.
hashRecentRejectsChainTip = m_chainman.ActiveChain().Tip()->GetBlockHash();
- m_recent_rejects.reset();
+ //m_recent_rejects.reset();
}
const uint256& hash = gtxid.GetHash();
```
I'm still not sure in which file this test fits best, and if there is already test coverage for the first part of the test somewhere. Happy for any suggestions.
ACKs for top commit:
maflcko:
ACK 60ca5d5508🍳
glozow:
code review ACK 60ca5d5508
instagibbs:
ACK 60ca5d5508
Tree-SHA512: 9cab43858e8f84db04a708151e6775c9cfc68c20ff53096220eac0b2c406f31aaf9223e8e04be345e95bf0a3f6dd15efac50b0ebeb1582a48a4560b3ab0bcba5
If we reuse the same port when disconnecting and establishing connections
again, we might hit this scenario:
- disconnection is done on python side for P2PConnection
- disconnection is not complete on c++ side for TestNode
- we're trying to establish a new connection on same port again
Prevent this scenario from happening by ensuring disconnection on c++
side for TestNode as well.
- Mention which directories contain the respective unsigned tarballs
- Clarify that bitcoin.conf might not need to be updated
- Specify where to put historical release notes if there is already
something in release-notes.md
- Clarify what exactly is the problem with running guix-codesign more
than once
- Correct number: 6 codesigned attestations are needed before uploading
binaries
- Remove scp command which is outdated
- Remove server path which is outdated
- Specify that translations update should happen before branch-off, not
before each release candidate
- Mention that you should notify lists when RCs are available
- Put "Archive the release notes" as a separate step, since creating the
github release has a dependency on it.
- Put bitcoincore.org website updates as a separate step, since
updating packaging repos may have a dependency on it.
- Update "bitcoin-dev mailing list" to "bitcoin-dev group"
- Document that maintainers should create PRs to collect backports
- Remove section about not uploading `*-debug` files, reader should
upload all build artifacts.
- Torrent is created automatically, so delete instructions.
- Mention that server also generates ots file automatically.
Prior to this, when establishing a network connection via CConnman::ConnectNode,
if the connection needed address resolution, a single address would be picked
at random from the resolved addresses and our node will try to connect to it. However,
this would lead to the behavior of ConnectNode being unpredictable when the address
was resolved to various ips (e.g. the address resolving to IPv4 and IPv6, but we only
support one of them).
This patches the aforementioned behavior by going over all resolved IPs until we find one
we can connect to or until we exhaust them.
On systems where `int8_t` is defined as `char`, the
`{S,Uns}erialize(Stream&, signed char)` functions become undefined.
This change resolves the issue by testing
`{S,Uns}erialize(Stream&, int8_t)` instead.
No behavior change on systems where `int8_t` is defined as
`signed char`, which is the case for most other systems.
c6d1b8de89 gui: change example address from legacy (P2PKH) to bech32m (P2TR) (Sebastian Falbesoner)
Pull request description:
Legacy addresses are less and less common these days and not recommended to use, so it seems senseful to also reflect that in the example addresses and update to the most recent address / output type (bech32m / P2TR). Also, as I couldn't see any value in computing these at runtime, they are pre-generated. This was done with the following Python script, executed in `./test/functional` (it's also included in the commit body, though without the she-bang):
```python
#!/usr/bin/env python3
from test_framework.segwit_addr import CHARSET, decode_segwit_address, encode_segwit_address
from test_framework.messages import sha256
output_key = sha256(b'bitcoin dummy taproot output key')
for network, hrp in [('mainnet', 'bc'), ('signet', 'tb'), ('testnet', 'tb'), ('regtest', 'bcrt')]:
dummy_address = encode_segwit_address(hrp, 1, output_key)
while decode_segwit_address(hrp, dummy_address) != (None, None):
last_char = CHARSET[(CHARSET.index(dummy_address[-1]) + 1) % 32]
dummy_address = dummy_address[:-1] + last_char
print(f'{network:7} example address: {dummy_address}')
```
Note that the last bech32 character is modified in order to make the checksum fail.
master (mainnet):
PR (mainnet):
ACKs for top commit:
maflcko:
lgtm ACK c6d1b8de89
pablomartin4btc:
tACK c6d1b8de89
Tree-SHA512: a53c267a3e0d29b9c41bf043b123e7152fbf297e2322d74ce047ba2582b54768187162d462cc334e91a84874731c2e0793726ad44d9970c10ecfe70a1d4f3f1c
05416422d3 refactor: Avoid implicit-integer-sign-change in processNewTransaction (MarcoFalke)
321f105d08 refactor: Avoid implicit-signed-integer-truncation-or-sign-change in FreedesktopImage (MarcoFalke)
6d8eecd33a refactor: Avoid implicit-integer-sign-change in createTransaction (MarcoFalke)
Pull request description:
This is allowed by the language. However, the `integer` sanitizer complains about it. Thus, fix it, so that the `integer` sanitizer can be used in the future to catch unintended sign changes.
Fixes#805.
ACKs for top commit:
pablomartin4btc:
tACK 05416422d3
hebasto:
ACK 05416422d3, I have reviewed the code and it looks OK.
Tree-SHA512: eaa941479bd7bee196eb8b31d93b8e1db122410cf62e8ec4cbbec35cfd14cc766081c3df5dd14a228e21ad2678d8b8ba0d2649e5934c994a90ae96d8b264b4ce
This ensures that the index does not miss any 'new block' signals
occurring in-between reading the 'next block' and setting 'm_synced'.
Because, if this were to happen, the ignored blocks would never be
indexed, thus stalling the index forever.
0244416aac security: restrict abis in bitcoind.service (Charlie)
Pull request description:
[As noted here](https://www.freedesktop.org/software/systemd/man/systemd.exec.html#MemoryDenyWriteExecute=), it's a good idea to pair `MemoryDenyWriteExecute=true` with `SystemCallArchitectures=native` because `MemoryDenyWriteExecute` can be circumvented in some operating systems which support multiple ABIs like x86/x86-64.
This helps restrict the possible application binary interfaces (ABIs) that can be used when running bitcoind through systemd, reducing the attack surface area.
ACKs for top commit:
laanwj:
ACK 0244416aac . This is a sensible security feature.
0xB10C:
ACK 0244416aac
Tree-SHA512: 77a35b0674d8d67d857cd20ae1b8cd011f82d6f5ed21bc106cbe45bfa937e786ddc1bf7261e3bdb8c289df1224e91658760905d2c8f37cc4c6506ef8037ad158
24b67fa9f6 doc: Add example of mixing private and public keys in descriptors (Anton A)
Pull request description:
closes: #27414
ACKs for top commit:
achow101:
ACK 24b67fa9f6
alfonsoromanz:
Re ACK 24b67fa9f6
Tree-SHA512: 8c063f23199ac0ff35909f786a5b0de1b4a9b15d1e93bdcdac10cb4bd2002c12e99b6fb1c2e56d16971e7622b67d910b79088429df92c48279be2d7797049911
dd3e0fa125 build: Fix false positive `CHECK_ATOMIC` test for clang-15 (Hennadii Stepanov)
Pull request description:
On the master branch @ 0de63b8b46, a building `bitcoind` with clang-15 for `i686-pc-linux-gnu` fails to link:
```
CXXLD bitcoind
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `std::remove_volatile<double>::type std::__atomic_impl::load<double>(double const*, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:948: undefined reference to `__atomic_load'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `std::remove_volatile<double>::type std::__atomic_impl::load<double>(double const*, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:948: undefined reference to `__atomic_load'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-wallet.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-backup.o): in function `void std::__atomic_impl::store<double>(double*, std::remove_volatile<double>::type, std::memory_order)':
/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: /usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: undefined reference to `__atomic_store'
/usr/bin/ld: libbitcoin_wallet.a(libbitcoin_wallet_a-backup.o):/usr/bin/../lib/gcc/x86_64-linux-gnu/11/../../../../include/c++/11/bits/atomic_base.h:940: more undefined references to `__atomic_store' follow
clang: error: linker command failed with exit code 1 (use -v to see invocation)
```
due to false positive `CHECK_ATOMIC` test in the `configure` script.
This PR fixes this test.
ACKs for top commit:
maflcko:
review ACK dd3e0fa125
fanquake:
ACK dd3e0fa125
Tree-SHA512: b60acf8d83fc84cc3280d95028395d341ed9ed2fcf38ae0a101d50aa19cc35540e9763aa36668c4dc1e1bc7e1f33dbda0e662df39c9e414a284ef91d7fc55fba
