Commit graph

39904 commits

Author SHA1 Message Date
Ava Chow
e69796c79c
Merge bitcoin/bitcoin#28560: wallet, rpc: FundTransaction refactor
18ad1b9142 refactor: pass CRecipient to FundTransaction (josibake)
5ad19668db refactor: simplify `CreateRecipients` (josibake)
47353a608d refactor: remove out param from `ParseRecipients` (josibake)
f7384b921c refactor: move parsing to new function (josibake)
6f569ac903 refactor: move normalization to new function (josibake)
435fe5cd96 test: add tests for fundrawtx and sendmany rpcs (josibake)

Pull request description:

  ## Motivation

  The primary motivation for this PR is to enable `FundTransaction` to take a vector of `CRecipient` objects to allow passing BIP352 silent payment addresses to RPCs that use `FundTransaction` (e.g. `send`, `walletcreatefundedpsbt`). To do that, SFFO logic needs to be moved out of `FundTransaction` so the `CRecipient` objects with the correct SFFO information can be created and then passed to `FundTransaction`.

  As a secondary motivation, this PR moves the SFFO stuff closer to the caller, making the code cleaner and easier to understand. This is done by having a single function which parses RPC inputs for SFFO and consistently using the `set<int>` method for communicating SFFO.

  I'm also not convinced we need to pass a full `CMutableTx` object to `FundTransaction`, but I'm leaving that for a follow-up PR/discussion, as its not a blocker for silent payments.

ACKs for top commit:
  S3RK:
    reACK 18ad1b9142
  josibake:
    > According to my `range-diff` nothing changed. reACK [18ad1b9](18ad1b9142)
  achow101:
    ACK 18ad1b9142

Tree-SHA512: d61f017cf7d98489ef216475b68693fd77e7b53a26a6477dcd73e7e5ceff5036b2d21476e377839e710bb73644759d42c4f9f4b14ed96b3e56ed87b07aa6d1a7
2024-01-23 16:40:58 -05:00
Ava Chow
2f218c664b
Merge bitcoin/bitcoin#28921: multiprocess: Add basic type conversion hooks
6acec6b9ff multiprocess: Add type conversion code for UniValue types (Ryan Ofsky)
0cc74fce72 multiprocess: Add type conversion code for serializable types (Ryan Ofsky)
4aaee23921 test: add ipc test to test multiprocess type conversion code (Ryan Ofsky)

Pull request description:

  Add type conversion hooks to allow `UniValue` objects, and objects that have `CDataStream` `Serialize` and `Unserialize` methods to be used as arguments and return values in Cap'nProto interface methods. Also add unit test to verify the hooks are working and data can be round-tripped correctly.

  The non-test code in this PR was previously part of #10102 and has been split off for easier review, but the test code is new.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  achow101:
    ACK 6acec6b9ff
  dergoegge:
    reACK 6acec6b9ff

Tree-SHA512: 5d2cbc5215d488b876d34420adf91205dabf09b736183dcc85aa86255e3804c2bac5bab6792dacd585ef99a1d92cf29c8afb3eb65e4d953abc7ffe41994340c6
2024-01-23 16:22:29 -05:00
Ava Chow
874c8bdb9e
Merge bitcoin/bitcoin#29144: init: handle empty settings file gracefully
e9014042a6 settings: add auto-generated warning msg for editing the file manually (furszy)
966f5de99a init: improve corrupted/empty settings file error msg (furszy)

Pull request description:

  Small and simple issue reported [here](https://community.umbrel.com/t/bitcoin-docker-container-keeps-restarting/2144).

  Improving a confusing situation reported by users who did not understand why a
  settings parsing error occurred when the file was empty and did not know how to solve it.

  Empty setting file could be due (1) corruption or (2) an user manually cleaning up the file content.
  In both scenarios, the 'Unable to parse settings file' error does not help the user move forward.

ACKs for top commit:
  achow101:
    ACK e9014042a6
  hebasto:
    re-ACK e9014042a6.
  ryanofsky:
    Code review ACK e9014042a6. Just whitespace formatting changes and shortening a test string literal since last review
  shaavan:
    Code review ACK e9014042a6

Tree-SHA512: 2910654c6b9e9112de391eedb8e46980280f822fa3059724dd278db7436804dd27fae628d2003f2c6ac1599b07ac5c589af016be693486e949f558515e662bec
2024-01-23 15:14:03 -05:00
Ava Chow
6f732ffc3c
Merge bitcoin/bitcoin#28774: wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it
32a9f13cb8 wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it (Vasil Dimov)

Pull request description:

  `CWallet::GetEncryptionKey()` would return a reference to the internal
  `CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

  Returning a copy would be a shorter solution, but could have security
  implications of the master key remaining somewhere in the memory even
  after `CWallet::Lock()` (the current calls to
  `CWallet::GetEncryptionKey()` are safe, but that is not future proof).

  So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
  change the `GetEncryptionKey()` method to provide the encryption
  key to a given callback:
  `m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

  This silences the following (clang 18):

  ```
  wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
   3520 |     return vMasterKey;
        |            ^
  ```

  ---
  _Previously this PR modified both ArgsManager and wallet code. But the ArgsManager commit 856c88776f was merged in https://github.com/bitcoin/bitcoin/pull/29040 so now this only affects wallet code. The previous PR description was:_

  Avoid this unsafe pattern from `ArgsManager` and `CWallet`:

  ```cpp
  class A
  {
      Mutex mutex;
      Foo member GUARDED_BY(mutex);
      const Foo& Get()
      {
          LOCK(mutex);
          return member;
      } // callers of `Get()` will have access to `member` without owning the mutex.
  ```

ACKs for top commit:
  achow101:
    ACK 32a9f13cb8
  ryanofsky:
    Code review ACK 32a9f13cb8. This seems like a potentially real race condition, and the fix here is pretty simple.
  furszy:
    ACK 32a9f13c

Tree-SHA512: 133da84691642afc1a73cf14ad004a7266cb4be1a6a3ec634d131dca5dbcdef52522c1d5eb04f5b6c4e06e1fc3e6ac57315f8fe1e207b464ca025c2b4edefdc1
2024-01-23 15:05:23 -05:00
Ava Chow
7cb7759b25
Merge bitcoin/bitcoin#29272: wallet: fix coin selection tracing to return -1 when no change pos
d55fdb1a49 Move TRACEx parameters to seperate lines (Richard Myers)
2d58629ee6 wallet: fix coin selection tracing to return -1 when no change pos (Richard Myers)

Pull request description:

  This is a bugfix for from when [optional was introduced](758501b713)  for `change_pos` in the wallet. When optional `change_pos` is unset, we should return -1 and not 0.

  I added two new checks to the `test/functional/interface_usdt_coinselection.py` which adds coverage for the situations when `normal_create_tx_internal` and `aps_create_tx_internal` events occur with no change.

  You can reproduce this bug using the coin-selection-simulation scripts as described in [issue #16](https://github.com/achow101/coin-selection-simulation/issues/16). You can also run the `interface_usdt_coinselection.py` test  without the changes to `wallet/spend.cpp`.

ACKs for top commit:
  0xB10C:
    ACK d55fdb1a49
  achow101:
    ACK d55fdb1a49
  murchandamus:
    ACK d55fdb1a49

Tree-SHA512: 6efac3b756bdf51debbcb759dc3c4b7a4304626bc047b70025cec02f3a04937ace7712e9558ac71e560fd136005a98c518ac5bb4b90c3282d776beccd0de9749
2024-01-23 14:33:43 -05:00
fanquake
f1ab078ed7
Merge bitcoin/bitcoin#29276: depends: Update libmultiprocess library to fix C++20 macos build error
b8105b3ed7 depends: Update libmultiprocess library to fix C++20 macos build error (Ryan Ofsky)

Pull request description:

  Fixes #29248

  The std::result_of type was removed in c++20, but was being referenced in some old, unused code in the library. The issue was fixed in:

  - https://github.com/chaincodelabs/libmultiprocess/pull/91

  This update also includes other recent libmultiprocess changes to improve C++20 support and fix build issues:

  - https://github.com/chaincodelabs/libmultiprocess/pull/89
  - https://github.com/chaincodelabs/libmultiprocess/pull/90
  - https://github.com/chaincodelabs/libmultiprocess/pull/93

ACKs for top commit:
  fanquake:
    ACK b8105b3ed7.

Tree-SHA512: 2ca64b5fc27be752baba38df4b4faf62152e18c70ead6e0e063f1cb0c25dd5d924dec7ebfd7f8bbd651ae50eb35e8d8b591a9847c36f22558b5f5effccf56536
2024-01-23 17:06:57 +00:00
fanquake
8c9dceb962
Merge bitcoin/bitcoin#29291: Add test for negative transaction version w/ CSV to tx_valid.json
97181decf5 Add test for negative transaction version w/ CSV to tx_valid.json (Chris Stewart)

Pull request description:

  This PR adds a static test vector corresponding to the bug found in various implementations of the bitcoin protocol discovered by dergoegge

  For more information see:

  https://delvingbitcoin.org/t/disclosure-btcd-consensus-bugs-due-to-usage-of-signed-transaction-version/455

ACKs for top commit:
  darosior:
    ACK 97181decf5
  dergoegge:
    ACK 97181decf5

Tree-SHA512: 92bbcd3cd10a569757b4de91e1b2bcfebc2b75ddb0160be36d8e512a6fa4623cced1aba93bd1cc044962cd2b10e1d184ef109ccdfe3cfcf85cf4b9585d80d115
2024-01-23 16:53:37 +00:00
Ryan Ofsky
b8105b3ed7 depends: Update libmultiprocess library to fix C++20 macos build error
Fixes #29248

The std::result_of type was removed in c++20, but was being referenced in some
old, unused code in the library. The issue was fixed in:

https://github.com/chaincodelabs/libmultiprocess/pull/91 util: Drop Bind, BindTuple, ComposeFn, GetFn, and ThrowFn helpers

This update also includes other recent libmultiprocess changes to improve C++20
support and fix build issues:

https://github.com/chaincodelabs/libmultiprocess/pull/89 pkgconfig: Drop -std=c++17 compile flag
https://github.com/chaincodelabs/libmultiprocess/pull/90 pkgconfig: Use @CMAKE_INSTALL_LIBDIR@ variable
https://github.com/chaincodelabs/libmultiprocess/pull/93 Fix support for vector<bool> serialization with libc++
2024-01-22 11:47:13 -05:00
furszy
e9014042a6
settings: add auto-generated warning msg for editing the file manually
Hopefully, refraining users from modifying the file unless they are
certain about the potential consequences.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
furszy
966f5de99a
init: improve corrupted/empty settings file error msg
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
glozow
651fb034d8
Merge bitcoin/bitcoin#29260: refactor: remove CTxMemPool::queryHashes()
282b12ddb0 refactor: remove CTxMemPool::queryHashes() (stickies-v)

Pull request description:

  `CTxMemPool::queryHashes()` is only used in `MempoolToJSON()`, where it can just as easily be replaced with the more general `CTxMemPool::entryAll()`. No behaviour change, just cleans up the code.

ACKs for top commit:
  dergoegge:
    Code review ACK 282b12ddb0
  TheCharlatan:
    ACK 282b12ddb0
  glozow:
    ACK 282b12ddb0. Looks like there's no conflicts.

Tree-SHA512: 16160dec8e1f2457fa0f62dc96d2d2efd92c4bab810ecdb0e08918b8e85a667702c8e41421eeb4ea6abe92a5956a2a39a7a6368514973b78be0d22de2ad299b2
2024-01-22 10:03:57 +00:00
Richard Myers
d55fdb1a49
Move TRACEx parameters to seperate lines 2024-01-20 14:58:17 +01:00
Richard Myers
2d58629ee6
wallet: fix coin selection tracing to return -1 when no change pos 2024-01-20 14:56:41 +01:00
josibake
18ad1b9142
refactor: pass CRecipient to FundTransaction
Instead turning tx.vout into a vector of `CRecipient`, make `FundTransaction`
take a `CRecipient` vector directly. This allows us to remove SFFO logic from
the wrapper RPC `FundTransaction` since the `CRecipient` objects have already
been created with the correct SFFO values. This also allows us to remove
SFFO from both `FundTransaction` function signatures.

This sets us up in a future PR to be able to use these RPCs with BIP352
static payment codes.
2024-01-19 15:04:56 +01:00
josibake
5ad19668db
refactor: simplify CreateRecipients
Move validation logic out of `CreateRecipients` and instead take the
already validated outputs from `ParseOutputs` as an input.

Move SFFO parsing out of `CreateRecipients` into a new function,
`InterpretSubtractFeeFromOutputsInstructions`. This takes the SFFO instructions
from `sendmany` and `sendtoaddress` and turns them into a set of integers.
In a later commit, we will also move the SFFO parsing logic from
`FundTransaction` into this function.

Worth noting: a user can pass duplicate addresses and addresses that dont exist
in the transaction outputs as SFFO args to `sendmany` and `sendtoaddress`
without triggering a warning. This behavior is preserved in to keep this commit
strictly a refactor.
2024-01-19 15:04:56 +01:00
josibake
47353a608d
refactor: remove out param from ParseRecipients
Have `ParseRecipients` return a vector of `CRecipients` and rename to `CreateRecipients`.
2024-01-19 15:04:56 +01:00
josibake
f7384b921c
refactor: move parsing to new function
Move the parsing and validation out of `AddOutputs` into its own function,
`ParseOutputs`. This allows us to re-use this logic in `ParseRecipients` in a
later commit, where the code is currently duplicated.

The new `ParseOutputs` function returns a CTxDestination,CAmount tuples.
This allows the caller to then translate the validated outputs into
either CRecipients or CTxOuts.
2024-01-19 15:04:56 +01:00
josibake
6f569ac903
refactor: move normalization to new function
Move the univalue formatting logic out of AddOutputs and into its own function,
`NormalizeOutputs`. This allows us to re-use this logic in later commits.
2024-01-19 15:04:56 +01:00
josibake
435fe5cd96
test: add tests for fundrawtx and sendmany rpcs
If the serialized transaction passed to `fundrawtransaction` contains
duplicates, they will be deserialized and added to the transaction. Add
a test to ensure this behavior is not changed during the refactor.

A user can pass any number of duplicated and unrelated addresses as an
SFFO argument to `sendmany` and the RPC will not throw an error (note,
all the rest of the RPCs which take SFFO as an argument will error if
the user passes duplicates or specifies outputs not present in the
transaction). Add a test to ensure this behavior is not changed during
the refactor.
2024-01-19 15:04:56 +01:00
fanquake
03752444cd
Merge bitcoin/bitcoin#29249: depends: add NM output to gen_id
6ec2813cd8 depends: add NM output to gen_id (fanquake)

Pull request description:

  `NM` is part of the current toolset, and can be set by the user. Include it in `gen_id`.

ACKs for top commit:
  TheCharlatan:
    Re-ACK 6ec2813cd8

Tree-SHA512: 2ada61e03783f9eb441f285ef5da50557ad729cb52ce2d2c4b2c38103dab29920a26262d4545fd2ac7fbf1cedc4902cd2359833544fbc0debf829c12a63e9769
2024-01-19 13:17:16 +00:00
stickies-v
282b12ddb0
refactor: remove CTxMemPool::queryHashes()
Its only usage can easily be replaced with CTxMemPool::entryAll()
2024-01-18 21:54:56 +00:00
Ava Chow
5f3a0574c4
Merge bitcoin/bitcoin#29262: rpc: Fix race in loadtxoutset
5555d8db33 test: Use blocks_path where possible (MarcoFalke)
fa9108941f rpc: Fix race in loadtxoutset (MarcoFalke)

Pull request description:

  The tip may have advanced, also if it did not, there is no reason to
  have two variables point to the same block.

  Fixes https://github.com/bitcoin/bitcoin/pull/27596#discussion_r1344694600

ACKs for top commit:
  achow101:
    ACK 5555d8db33
  pablomartin4btc:
    ACK 5555d8db33
  BrandonOdiwuor:
    Code Review ACK 5555d8db33

Tree-SHA512: 23a82924a915b61bb1adab8ad20ec8914139c8ee647817af34ca27ee310a2e45833d8b285503e0feebe63e4667193d6d98cfcbbc1509bf40712225e04dd19e8b
2024-01-18 13:17:35 -05:00
Ava Chow
ac3901ebd0
Merge bitcoin/bitcoin#29228: test: Remove all-lint.py script
fa2b95cf3f test: Remove all-lint.py script (MarcoFalke)
fadb06c361 doc: move-only lint docs to one place (MarcoFalke)

Pull request description:

  Seems confusing to have a test runner that calls another runner (`all-lint.py`), which calls a subset of the lint tests.

  Fix that by just calling this subset of lint tests in the test runner directly, and remove the now unused `all-lint.py`.

  To run all lint checks locally, refer to the documentation: https://github.com/bitcoin/bitcoin/blob/master/test/lint/README.md#running-locally

ACKs for top commit:
  kevkevinpal:
    ACK [fa2b95c](fa2b95cf3f)
  achow101:
    ACK fa2b95cf3f
  TheCharlatan:
    ACK fa2b95cf3f
  pablomartin4btc:
    tACK fa2b95cf3f
  brunoerg:
    utACK fa2b95cf3f

Tree-SHA512: 43fac9acb4e9a6744d695dd49c7202e19ab4bf480f4cccff768647d0157a065f40e6ad70b9f6a65ba42048cc5fa9834365aa8e7aa0ed64c09e0cd4eb8dccb831
2024-01-18 13:02:15 -05:00
Vasil Dimov
32a9f13cb8
wallet: avoid returning a reference to vMasterKey after releasing the mutex that guards it
`CWallet::GetEncryptionKey()` would return a reference to the internal
`CWallet::vMasterKey`, guarded by `CWallet::cs_wallet`, which is unsafe.

Returning a copy would be a shorter solution, but could have security
implications of the master key remaining somewhere in the memory even
after `CWallet::Lock()` (the current calls to
`CWallet::GetEncryptionKey()` are safe, but that is not future proof).

So, instead of `EncryptSecret(m_storage.GetEncryptionKey(), ...)`
change the `GetEncryptionKey()` method to provide the encryption
key to a given callback:
`m_storage.WithEncryptionKey([](const CKeyingMaterial& k) { EncryptSecret(k, ...); })`

This silences the following (clang 18):

```
wallet/wallet.cpp:3520:12: error: returning variable 'vMasterKey' by reference requires holding mutex 'cs_wallet' [-Werror,-Wthread-safety-reference-return]
 3520 |     return vMasterKey;
      |            ^
```
2024-01-18 18:12:59 +01:00
fanquake
03c5b0064d
Merge bitcoin/bitcoin#29085: refactor: C++20: Use std::rotl
6044628543 crypto, hash: replace custom rotl32 with std::rotl (Fabian Jahr)

Pull request description:

  While exploring some C++20 changes and checking against our code I found this potential improvement:

  1. We can replace our custom implementation of `rotl32` in crypto/chacha20 with `std::rotl` from the [new `bit` header](https://en.cppreference.com/w/cpp/header/bit).

ACKs for top commit:
  fanquake:
    ACK 6044628543

Tree-SHA512: db55b366f20fca2ef62e5f10a838f8a709d531678c35c1dba20898754029c788a2fd47995208ed6d187cf814109a7ca397bc2c301504500aee79da04c95d6895
2024-01-18 09:40:44 +00:00
fanquake
3d52cedb49
Merge bitcoin/bitcoin#29251: contrib: Update clang-format-diff
52149b7a2c contrib: Fix clang-format-diff.py lint errors (TheCharlatan)
008e81e025 contrib: Latest clang-format-diff.py script (TheCharlatan)

Pull request description:

  This was taken from 900bb318b5/clang/tools/clang-format/clang-format-diff.py and is useful for systems where clang tools are shipped with a version suffix.

ACKs for top commit:
  maflcko:
    lgtm ACK 52149b7a2c  🌱

Tree-SHA512: cad720b283650e36c5b3ec597278112899ae6413a42c79b6296c58242000f32ae6ab7ed732a3a0f30f1f2586d7e3c0d1ef856a4821e28272bae6f428ed9497e9
2024-01-17 16:08:10 +00:00
fanquake
514268170b
Merge bitcoin/bitcoin#29133: refactor: Allow std::span construction from CKey
fa96d93711 refactor: Allow std::span construction from CKey (MarcoFalke)
999962d68d Add missing XOnlyPubKey::data() to get mutable data (MarcoFalke)

Pull request description:

  Is is possible to construct a `Span` from a reference to a `CKey`. However, the same is not possible with `std::span`.

  Fix that.

ACKs for top commit:
  shaavan:
    ReACK fa96d93711
  willcl-ark:
    ACK fa96d93711

Tree-SHA512: 44fccdce5f32bc16b44f3b1bd32e86d9eabfd09bca6abe79f2d6db0cb0b5e4aaeaff710f023cb21ccde9315d2007d55f1b43f29416e81bceeeabe3948f673d3a
2024-01-17 16:00:32 +00:00
MarcoFalke
5555d8db33
test: Use blocks_path where possible 2024-01-17 16:48:47 +01:00
MarcoFalke
fa9108941f
rpc: Fix race in loadtxoutset
The tip may have advanced, also if it did not, there is no reason to
have two variables point to the same block.
2024-01-17 16:48:42 +01:00
fanquake
c818607ed5
Merge bitcoin/bitcoin#29233: build: depends move macOS C(XX) FLAGS out of C & CXX
cbc9bf11fe build: move -mlinker-version to *FLAGS (fanquake)
42b2283765 depends: deduplicate use of mmacosx-version-min in macOS build (fanquake)

Pull request description:

  Move some C/CXX FLAGS out of C/CXX. The remaining flags are host/SDK related, and will need some more thought.
  This is more correct in any case, and simplifies future changes.
  Related to #21778.

ACKs for top commit:
  theuni:
    utACK cbc9bf11fe
  TheCharlatan:
    ACK cbc9bf11fe

Tree-SHA512: 373216c1de32375faddc161ecc09b14fed0e0994cbd5ed62c862c38a5aea80c7f1740f32f36b8a60ac1caf749309750d22164c50b89916f41838d6453296ac4a
2024-01-17 10:46:16 +00:00
TheCharlatan
52149b7a2c
contrib: Fix clang-format-diff.py lint errors
We assume to be using python3, so don't check for it. This removes a
type error on the line `from io import BytesIO as StringIO`.

Specify the encoding as "utf8" when opening a file.
2024-01-17 11:10:15 +01:00
Ava Chow
8106b268cd
Merge bitcoin/bitcoin#29239: rpc: Make v2transport default for addnode RPC when enabled
3ba815b42d Make v2transport default for addnode RPC when enabled (Pieter Wuille)

Pull request description:

  Since #29058, several types of manually configured connections will attempt v2 connections when `-v2transport` is enabled, except for the `addnode` RPC, as that one has an explicit argument to enable or disable.

  Make the default for that RPC match the `-v2transport` setting so the behavior matches that of other manual connections from a user perspective.

ACKs for top commit:
  achow101:
    ACK 3ba815b42d
  kristapsk:
    ACK 3ba815b42d
  theStack:
    Code-review ACK 3ba815b42d

Tree-SHA512: 31ef48cf1e533abb17866020378c004df929e626074dc98b3229fb60a66de58435e95c8fda8d1b463e1208aa39d1f42d239818e7e58595a3738089920598befc
2024-01-16 16:50:03 -05:00
Ava Chow
a3fb1f80ac
Merge bitcoin/bitcoin#28791: snapshots: don't core dump when running -checkblockindex after loadtxoutset
cdc6ac4126 snapshots: don't core dump when running -checkblockindex after `loadtxoutset` (Mark Friedenbach)

Pull request description:

  Transaction counts aren't known for block history loaded from a snapshot. If you start with `-checkblockindex` after loading a snapshot, the bitcoin daemon will core dump. The test suite does not check for this because all the snapshots have no non-coinbase transactions (all blocks prior to the snapshot are assumed to have `nTx = 1`).

  Recommend for backport to 26.x

ACKs for top commit:
  fjahr:
    utACK cdc6ac4126
  achow101:
    ACK cdc6ac4126
  pablomartin4btc:
    tACK cdc6ac4126

Tree-SHA512: f7488a85cc29056e2ac443ce8f34aea4dfde6ba246efce82235d6a4dca2dca4344f07b93c93424b4addcb83e4cb2ae49a3ebb37d89840d42d2aeea35904cab04
2024-01-16 15:02:53 -05:00
Ava Chow
5711da6588
Merge bitcoin/bitcoin#29213: doc, test: test and explain service flag handling
74ebd4d135 doc, test: Test and explain service flag handling (Martin Zumsande)

Pull request description:

  Service flags received from the peer-to-peer network are handled differently, depending on how we receive them.
  If received directly from an outbound peer the flags belong to, they replace existing flags.
  If received via gossip relay (so that anyone could send them), new flags are added, but existing ones but cannot be overwritten.

  Document that and add test coverage for it.

ACKs for top commit:
  achow101:
    ACK 74ebd4d135
  furszy:
    ACK 74ebd4d135
  brunoerg:
    utACK 74ebd4d135

Tree-SHA512: 604adc3304b8e3cb1a10dfd017025c10b029bebd3ef533f96bcb5856fee5d4396a9aed4949908b8e7ef267ad21320d1814dd80f88426330c5c9c2c529c497591
2024-01-16 13:35:45 -05:00
Ava Chow
27d935f58b
Merge bitcoin/bitcoin#29179: test: wallet rescan with reorged parent + IsFromMe child in mempool
df30247705 [test] import descriptor wallet with reorged parent + IsFromMe child in mempool (glozow)
c3d02be536 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool (Gloria Zhao)

Pull request description:

  Originally motivated by #29019, which reverts back to having `requestMempoolTransactions` emit `transactionAddedToMempool` in `mapTx` default order instead of `GetSortedDepthAndScore` order.

  It's important that these notifications happen in topological order, otherwise the wallet rescan may miss transactions that belong to it. Notably, checking whether a transaction `IsFromMe` requires knowing its inputs, which may be from a mempool parent.

  When using `mapTx` order, a parent may come later than its child if it was added from a block disconnected in a reorg.

  This PR adds a test for this case.

ACKs for top commit:
  achow101:
    ACK df30247705
  furszy:
    Code review ACK df30247705, nits can be disregarded.

Tree-SHA512: 2f1d9ef92313228adbbef94e634e5f7a9ec6e6a2c88e16aa343bdc95ffc9b9f9c82a569b412c9a3841db9d789e52f9283e8b9385731668d59355903e26e58a5d
2024-01-16 12:49:09 -05:00
fanquake
f1fcc9638c
Merge bitcoin/bitcoin#29170: contrib: add macho branch protection check
5335e454c0 contrib: add macho branch protection check (fanquake)

Pull request description:

  Followup to https://github.com/bitcoin/bitcoin/pull/28459. Add a sanity check that `bti` instructions are present in the arm macho binary, similar to our x86_64 check for control flow.

  Could do something similar for aarch64 linux in future, and maybe could use https://github.com/lief-project/LIEF/issues/975.

ACKs for top commit:
  TheCharlatan:
    ACK 5335e454c0

Tree-SHA512: 6cc8721209fe07fe07f0524ef6f114004e2b98844f73d31ff16547f7055c7cb4a5609480058c45ede21b457b2dea5357f1475eaa5063ea1f9772aa260f49039b
2024-01-16 15:33:41 +00:00
TheCharlatan
008e81e025
contrib: Latest clang-format-diff.py script
This was take from
900bb318b5/clang/tools/clang-format/clang-format-diff.py

Updating it introduces some new options. For example specifying the
clang-format binary, which is useful for systems where clang tools are
shipped with a version suffix.
2024-01-16 16:33:12 +01:00
MarcoFalke
fa96d93711
refactor: Allow std::span construction from CKey 2024-01-16 15:29:18 +01:00
fanquake
9fa8eda8af
Merge bitcoin/bitcoin#29230: doc: update -loglevel help to add info to the always logged levels
ec779a2b8e doc: add unconditional info loglevel following merge of PR 28318 (Jon Atack)

Pull request description:

  Commit ab34dc6012 of #28318 was an incomplete version of [`118c756` (#25203)](118c7567f6) from the `Severity-based logging` parent PR.

  Add the missing text to update the `-loglevel` help doc.

  While here, make the help text a little easier to understand.

  Can be tested by running:

  ```
  ./src/bitcoind -regtest -help-debug | grep -A12 loglevel=
  ```

  before
  ```
    -loglevel=<level>|<category>:<level>
         Set the global or per-category severity level for logging categories
         enabled with the -debug configuration option or the logging RPC:
         info, debug, trace (default=debug); warning and error levels are
         always logged.
  ```

  after
  ```
    -loglevel=<level>|<category>:<level>
         Set the global or per-category severity level for logging categories
         enabled with the -debug configuration option or the logging RPC.
         Possible values are info, debug, trace (default=debug). The
         following levels are always logged: error, warning, info.
  ```

ACKs for top commit:
  stickies-v:
    ACK ec779a2b8e

Tree-SHA512: 0c375e30a5a4c168ca7d97720e8c287f598216767afedae329824e09a480830faf8537b792c5c4bb647c68681c287fe3005c62093708ce85624e9a71c8245e42
2024-01-16 10:52:54 +00:00
MarcoFalke
999962d68d
Add missing XOnlyPubKey::data() to get mutable data
This is needed for consistency, and also to allow std::span construction
from XOnlyPubKey.
2024-01-16 10:58:57 +01:00
MarcoFalke
fa2b95cf3f
test: Remove all-lint.py script 2024-01-16 10:54:42 +01:00
MarcoFalke
fadb06c361
doc: move-only lint docs to one place
Can be reviewed with --color-moved=dimmed-zebra
2024-01-16 10:54:14 +01:00
fanquake
cbc9bf11fe
build: move -mlinker-version to *FLAGS
This doesn't need to exist in C & CXX.
2024-01-16 09:53:27 +00:00
fanquake
42b2283765
depends: deduplicate use of mmacosx-version-min in macOS build 2024-01-16 09:52:39 +00:00
fanquake
6ec2813cd8
depends: add NM output to gen_id 2024-01-16 09:44:09 +00:00
fanquake
2ac2821a74
Merge bitcoin/bitcoin#29185: build: remove --enable-lto
2d1b1c7dae build: remove --enable-lto (fanquake)

Pull request description:

  This has outlived its usefulness, doesn't gel well with newer compilers & `-flto` related options, i.e thin vs full, or `=auto`, and having `-flto` as the only option means that sometimes this just needs to be worked around, i.e in oss-fuzz:
  https://github.com/google/oss-fuzz/blob/master/projects/bitcoin-core/build.sh.

  While it was convenient when `-flto` was newer, support for `-flto` is now in all compilers we use, and there's also no-longer any real need for us to treat `-flto` different to any other optimization option.

  Remove it, to remove build complexity, and so there's no need to port a similar option to CMake.

  Note that the LTO option remains in depends, because we still a way to build packages that have LTO specific patches/options.

ACKs for top commit:
  TheCharlatan:
    ACK 2d1b1c7dae
  hebasto:
    ACK 2d1b1c7dae.

Tree-SHA512: 91812de7da35346f51850714a188fcffbac478bc8b348bf756c2555fcbde86ba622ac2fb77d294dea0378c741d3656f06121ef3a795aeed63fd170fc31bfa5af
2024-01-16 09:42:12 +00:00
Martin Zumsande
74ebd4d135 doc, test: Test and explain service flag handling
Service flags are handled differently, depending on whether
validated (if received from the peer) or unvalidated (received
via gossip relay).
2024-01-15 16:19:53 -05:00
fanquake
05c4c5a434
Merge bitcoin/bitcoin#29227: log mempool loading progress
eb78ea4eeb [log] mempool loading (glozow)

Pull request description:

  Motivated by #29193. Currently, we only log something (non-debug) when we fail to load the file and at the end of importing all the transactions. That means it's hard to tell what's happening if it's taking a long time to load.

  This PR adds a maximum of 10 new unconditional log lines:
  - When we start to load transactions.
  - Our progress percentage when it advances by at least 10% from the last time we logged. Percentage is based on the number of transactions.

  If there are lots of transactions in the mempool, the logs will look like this:
  ```
  2024-01-11T11:36:30.410726Z Loading 401 mempool transactions from disk...
  2024-01-11T11:36:30.423374Z Progress loading mempool transactions from disk: 10% (tried 41, 360 remaining)
  2024-01-11T11:36:30.435539Z Progress loading mempool transactions from disk: 20% (tried 81, 320 remaining)
  2024-01-11T11:36:30.447874Z Progress loading mempool transactions from disk: 30% (tried 121, 280 remaining)
  2024-01-11T11:36:30.460474Z Progress loading mempool transactions from disk: 40% (tried 161, 240 remaining)
  2024-01-11T11:36:30.473731Z Progress loading mempool transactions from disk: 50% (tried 201, 200 remaining)
  2024-01-11T11:36:30.487806Z Progress loading mempool transactions from disk: 60% (tried 241, 160 remaining)
  2024-01-11T11:36:30.501739Z Progress loading mempool transactions from disk: 70% (tried 281, 120 remaining)
  2024-01-11T11:36:30.516334Z Progress loading mempool transactions from disk: 80% (tried 321, 80 remaining)
  2024-01-11T11:36:30.531309Z Progress loading mempool transactions from disk: 90% (tried 361, 40 remaining)
  2024-01-11T11:36:30.549019Z  Imported mempool transactions from disk: 401 succeeded, 0 failed, 0 expired, 0 already there, 400 waiting for initial broadcast
  ```
  If there are 0 or 1 transactions, progress logs aren't printed.

ACKs for top commit:
  kevkevinpal:
    Concept ACK [eb78ea4](eb78ea4eeb)
  ismaelsadeeq:
    ACK eb78ea4eeb
  dergoegge:
    Code review ACK eb78ea4eeb
  theStack:
    re-ACK eb78ea4eeb
  mzumsande:
    tested ACK eb78ea4eeb

Tree-SHA512: ae4420986dc7bd5cb675a7ebc76b24c8ee60007f0296ed37e272f1c3415764d44963bea84c51948da319a65661dca8a95eac2a59bf7e745519b6fcafa09812cf
2024-01-15 15:20:18 +00:00
fanquake
17e33fb578
Merge bitcoin/bitcoin#29237: depends: Allow PATH with spaces in directory names.
4756114e50 [depends] Allow PATH with spaces in directory names. (Mark Friedenbach)

Pull request description:

  The goal of this PR is to help close https://github.com/bitcoin/bitcoin/pull/28733. I reverted the change on `depends/config.guess` based on the feedback provided in the previous PR. I've also incorporated the test mentioned by maflcko

ACKs for top commit:
  maflcko:
    lgtm ACK 4756114e50
  hebasto:
    ACK 4756114e50, successfully built depends on Ubuntu 22.04.
  TheCharlatan:
    ACK 4756114e50

Tree-SHA512: ee257f6efd235839156bc236384f08d77b91debc3c257168368a71e70742639f28a3289572b8693609c1109062dc9968e461103d1f4f5679906506e94b54e649
2024-01-15 13:14:32 +00:00
fanquake
28ccc7003a
Merge bitcoin/bitcoin#29241: doc: Add missing backtick in developer notes logging section
c003562120 doc: Add missing backtick in developer notes logging section (Fabian Jahr)

Pull request description:

  Newly added logging section from https://github.com/bitcoin/bitcoin/pull/28318 is missing a single backtick. Also fixes some minor punctuation errors in that section.

ACKs for top commit:
  jonatack:
    ACK c003562120
  alfonsoromanz:
    ACK c003562120

Tree-SHA512: 2f75f9472d212ce7c7ebf3e7404f86b3bd8c695f63e2a7447d7a55bb54dcdb5e3bfd15e8eac5b92efbdcf1216c4e8d699cae0250d021f72b9d1c32a7db91989d
2024-01-15 10:25:42 +00:00