Commit graph

41069 commits

Author SHA1 Message Date
Max Edwards
9eea51d905 ci: move Asan / LSan / USDT job to Github Actions
Moving it from Cirrus CI so it can be easier to maintain and used by forks
2024-06-12 14:20:25 +01:00
Max Edwards
4b527fa93b ci: add IPV6 network to ci container
Allows IPV6 functional tests to run inside the container
2024-06-08 17:09:33 +01:00
merge-script
a786fd2041
Merge bitcoin/bitcoin#30143: doc: Update NetBSD Build Guide
85e480a41a doc: Update NetBSD Build Guide (Hennadii Stepanov)

Pull request description:

  This PR updates the NetBSD Build Guide to reflect:
  - the recent NetBSD Release
  - GCC minimum supported version update (https://github.com/bitcoin/bitcoin/pull/28348 and https://github.com/bitcoin/bitcoin/pull/29091)
  - Python minimum supported version update (https://github.com/bitcoin/bitcoin/pull/28211)

  Also a smaller package set has been suggested:
  - `boost-headers` instead of the full `boost`
  - `qt5-qtbase qt5-qttools` instead of the full `qt5` (similar to https://github.com/bitcoin/bitcoin/pull/29932 and https://github.com/bitcoin/bitcoin/pull/29947).

ACKs for top commit:
  maflcko:
    utACK 85e480a41a

Tree-SHA512: 0848b6f169a00f2da78eea452fd116193aece853680f4e615ba1df654246ec2a9d1600a4cfb238eb9305b72bbe4b2873519bf1e2987eaaf48aba900c64e85a48
2024-05-21 10:26:49 +01:00
merge-script
8804ec736a
Merge bitcoin/bitcoin#30095: util: avoid using thread_local variable that has a destructor
d35ba1b3f1 util: avoid using thread_local variable that has a destructor (Vasil Dimov)

Pull request description:

  Store the thread name in a `thread_local` variable of type `char[]` instead of `std::string`. This avoids calling the destructor when the thread exits. This is a workaround for
  https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278701

  For type-safety, return `std::string` from
  `util::ThreadGetInternalName()` instead of `char[]`.

  As a side effect of this change, we no longer store a reference to a `thread_local` variable in `CLockLocation`. This was dangerous because if the thread quits while the reference still exists (in the global variable `lock_data`, see inside `GetLockData()`) then the reference will become dangling.

ACKs for top commit:
  laanwj:
    Code review ACK d35ba1b3f1
  hebasto:
    re-ACK d35ba1b3f1.
  theuni:
    utACK d35ba1b3f1

Tree-SHA512: a2a3bc4401654d6e99db5b9c46a7051855f5a26886142298662e681b78dd581ff4c6bebe42f649b8e1fb8a78d569c6117302db2cd6362e884a22f2a5839b7d43
2024-05-21 10:25:56 +01:00
merge-script
5acdc2b97d
Merge bitcoin/bitcoin#26606: wallet: Implement independent BDB parser
d51fbab4b3 wallet, test: Be able to always swap BDB endianness (Ava Chow)
0b753156ce test: Test bdb_ro dump of wallet without reset LSNs (Ava Chow)
c1984f1282 test: Test dumping dbs with overflow pages (Ava Chow)
fd7b16e391 test: Test dumps of other endian BDB files (Ava Chow)
6ace3e953f bdb: Be able to make byteswapped databases (Ava Chow)
d9878903fb Error if LSNs are not reset (Ava Chow)
4d7a3ae78e Berkeley RO Database fuzz test (TheCharlatan)
3568dce9e9 tests: Add BerkeleyRO to db prefix tests (Ava Chow)
70cfbfdadf wallettool: Optionally use BERKELEY_RO as format when dumping BDB wallets (Ava Chow)
dd57713f6e Add MakeBerkeleyRODatabase (Ava Chow)
6e50bee67d Implement handling of other endianness in BerkeleyRODatabase (Ava Chow)
cdd61c9cc1 wallet: implement independent BDB deserializer in BerkeleyRODatabase (Ava Chow)
ecba230979 wallet: implement BerkeleyRODatabase::Backup (Ava Chow)
0c8e728476 wallet: implement BerkeleyROBatch (Ava Chow)
756ff9b478 wallet: add dummy BerkeleyRODatabase and BerkeleyROBatch classes (Ava Chow)
ca18aea5c4 Add AutoFile::seek and tell (Ava Chow)

Pull request description:

  Split from #26596

  This PR adds `BerkeleyRODatabase` which is an independent implementation of a BDB file parser. It provides read only access to a BDB file, and can therefore be used as a read only database backend for wallets. This will be used for dumping legacy wallet records and migrating legacy wallets without the need for BDB itself.

  Wallettool's `dump` command is changed to use `BerkeleyRODatabase` instead of `BerkeleyDatabase` (and `CWallet` itself) to demonstrate that this parser works and to test it against the existing wallettool functional tests.

ACKs for top commit:
  josibake:
    reACK d51fbab4b3
  TheCharlatan:
    Re-ACK d51fbab4b3
  furszy:
    reACK d51fbab4b3
  laanwj:
    re-ACK d51fbab4b3
  theStack:
    ACK d51fbab4b3

Tree-SHA512: 1e7b97edf223b2974eed2e9eac1179fc82bb6359e0a66b7d2a0c8b9fa515eae9ea036f1edf7c76cdab2e75ad994962b134b41056ccfbc33b8d54f0859e86657b
2024-05-21 10:05:09 +01:00
Hennadii Stepanov
85e480a41a
doc: Update NetBSD Build Guide 2024-05-20 12:59:15 +01:00
glozow
ecd23656db
Merge bitcoin/bitcoin#30133: test: remove unneeded -maxorphantx=1000 settings
8950053636 test: remove unneeded `-maxorphantx=1000` settings (Sebastian Falbesoner)

Pull request description:

  It's unclear what the motivation for increasing the orphan pool is here, and it seems that this not needed at all. None of these tests involve orphan transactions explicitly, and if they would occur occasionally, there is no good reason to prefer a value of 1000 over the default of 100 (see DEFAULT_MAX_ORPHAN_TRANSACTIONS).

ACKs for top commit:
  maflcko:
    utACK 8950053636
  edilmedeiros:
    Tested ACK 8950053636
  AngusP:
    tACK 8950053636
  glozow:
    ACK 8950053636 From skimming the tests, it appears that none of these need a larger `-maxorphantx`.

Tree-SHA512: 81d4a4fb2ea92b97119f21cbc6c4b1240d863269932e6adf4982aead9726f20652523a4707add3ad38eb332d4452de41de6735265f22e62298f3b4b45de75a57
2024-05-20 09:47:31 +01:00
glozow
063bb2fbb5
Merge bitcoin/bitcoin#30066: test: add conflicting topology test case
9365baa489 test: add conflicting topology test case (Greg Sanders)

Pull request description:

  We want to ensure that even if topologies
  that are acceptable are relaxed, like
  removing package-not-child-with-unconfirmed-parents, that we don't end up accepting packages we shouldn't.

ACKs for top commit:
  glozow:
    reACK 9365baa489
  rkrux:
    reACK [9365baa](9365baa489)

Tree-SHA512: d58661064ca099ac0447c331a5020c74c0cdfe24259aa875592805bbd63de1bf23aa7ced9ff485fef90dc0602fcb997e631aaf1aa2e9805d2cf5f0e5c9b2f0e2
2024-05-20 09:27:37 +01:00
Sebastian Falbesoner
8950053636 test: remove unneeded -maxorphantx=1000 settings
It's unclear what the motivation for increasing the orphan pool is, and
it seems that this not needed at all. None of these tests involve orphan
transactions explicitly, and if they would occur occasionally, there is
no good reason to prefer a value of 1000 over the default of 100 (see
DEFAULT_MAX_ORPHAN_TRANSACTIONS).
2024-05-17 22:38:59 +02:00
Ava Chow
058af75874
Merge bitcoin/bitcoin#29817: kernel: De-globalize fReindex
b47bd95920 kernel: De-globalize fReindex (TheCharlatan)

Pull request description:

  fReindex is one of the last remaining globals exposed by the kernel library, so move it into the blockstorage class to reduce the amount of global mutable state and make the kernel library a bit less awkward to use.

  ---

  This pull request is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  achow101:
    ACK b47bd95920
  ryanofsky:
    Code review ACK b47bd95920. I rereviewed the whole PR, but the only change since last review was reverting the bugfix https://github.com/bitcoin/bitcoin/pull/29817#discussion_r1578327024 and make the change a pure refactoring.
  mzumsande:
    Code Review ACK b47bd95920
  stickies-v:
    ACK b47bd95920

Tree-SHA512: f7399d01f93bc0c0c7428fe95d19b9d29b4ed00a4f1deabca78fb0c4fecb434ec971e890feecb105938b5247c926850b1b7b4a4a9caa333a061e40777d0c8463
2024-05-17 15:50:56 -04:00
Ava Chow
4877fcdb42
Merge bitcoin/bitcoin#30048: crypto: add NUMS_H const
9408a04e42 tests, fuzz: use new NUMS_H const (josibake)
b946f8a4c5 crypto: add NUMS_H const (josibake)

Pull request description:

  Broken out from #28122

  ---

  [BIP341](https://github.com/bitcoin/bips/blob/master/bip-0341.mediawiki#constructing-and-spending-taproot-outputs) defines a NUMS point `H` as *H = lift_x(0x50929b74c1a04954b78b4b6035e97a5e078a5a0f28ec96d547bfee9ace803ac0)* which is [constructed](11af7015de/src/modules/rangeproof/main_impl.h (L16)) by taking the hash of the standard uncompressed encoding of the [secp256k1](https://www.secg.org/sec2-v2.pdf) base point G as X coordinate."

  Add this as a constant so it can be used in our codebase. My primary motivation is BIP352 specifies a special case for when taproot spends use `H` as the internal key, but outside of BIP352 it seems generally useful to have `H` in the codebase, for testing or other use cases.

ACKs for top commit:
  paplorinc:
    re-ACK 9408a04e42
  achow101:
    ACK 9408a04e42
  theStack:
    Code-review ACK 9408a04e42

Tree-SHA512: ad84492f5d635c0cb05bd82546079ded7e5138e95361f20d8285a9ad6e69c10ee2cc3fe46e16b46ef03c4253c8bee1051911c6b91264c90c3b1ad33a824bff4b
2024-05-17 14:10:51 -04:00
Ava Chow
d51fbab4b3 wallet, test: Be able to always swap BDB endianness 2024-05-16 15:03:13 -04:00
Ava Chow
0b753156ce test: Test bdb_ro dump of wallet without reset LSNs 2024-05-16 15:03:13 -04:00
Ava Chow
c1984f1282 test: Test dumping dbs with overflow pages 2024-05-16 15:03:13 -04:00
Ava Chow
fd7b16e391 test: Test dumps of other endian BDB files 2024-05-16 15:03:13 -04:00
Ava Chow
6ace3e953f bdb: Be able to make byteswapped databases
Byteswapped databases make it easier to test opening and deserializing
other endian databases.
2024-05-16 15:03:13 -04:00
Ava Chow
d9878903fb Error if LSNs are not reset 2024-05-16 15:03:13 -04:00
TheCharlatan
4d7a3ae78e Berkeley RO Database fuzz test 2024-05-16 15:03:13 -04:00
Ava Chow
3568dce9e9 tests: Add BerkeleyRO to db prefix tests 2024-05-16 15:03:13 -04:00
Ava Chow
70cfbfdadf wallettool: Optionally use BERKELEY_RO as format when dumping BDB wallets
In order to ease the transition to not having BDB, make the dump tool
use DatabaseFormmat::BERKELEY_RO when -withinternalbdb is set.
2024-05-16 15:03:13 -04:00
Ava Chow
dd57713f6e Add MakeBerkeleyRODatabase
Implements MakeBerkeleyRODatabase and adds DatabaseFormat::BERKELEY_RO
so that MakeDatabase can use BerkeleyRO as the backend database.
2024-05-16 15:03:13 -04:00
Ava Chow
6e50bee67d Implement handling of other endianness in BerkeleyRODatabase 2024-05-16 15:03:13 -04:00
Ava Chow
cdd61c9cc1 wallet: implement independent BDB deserializer in BerkeleyRODatabase
BerkeleyRODatabase is intended for use after BDB is removed, so it needs
to be able to read all of the records from a BDB file. Thus an
independent deserializer for BDB data files is implemented in it. This
deserializer is targeted towards the data files that Bitcoin Core
creates so it does not fully support all of BDB's features (e.g. other
database types, encryption, etc.).
2024-05-16 15:03:13 -04:00
Vasil Dimov
d35ba1b3f1
util: avoid using thread_local variable that has a destructor
Store the thread name in a `thread_local` variable of type `char[]`
instead of `std::string`. This avoids calling the destructor when
the thread exits. This is a workaround for
https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=278701

For type-safety, return `std::string` from
`util::ThreadGetInternalName()` instead of `char[]`.

As a side effect of this change, we no longer store a reference
to a `thread_local` variable in `CLockLocation`. This was
dangerous because if the thread quits while the reference still
exists (in the global variable `lock_data`, see inside `GetLockData()`)
then the reference will become dangling.
2024-05-16 18:16:46 +02:00
Ryan Ofsky
2f53f2273d
Merge bitcoin/bitcoin#29975: blockstorage: Separate reindexing from saving new blocks
e41667b720 blockstorage: Don't move cursor backwards in UpdateBlockInfo (Ryan Ofsky)
17103637c6 blockstorage: Rename FindBlockPos and have it return a FlatFilePos (Martin Zumsande)
d9e477c4dc validation, blockstorage: Separate code paths for reindex and saving new blocks (Martin Zumsande)
064859bbad blockstorage: split up FindBlockPos function (Martin Zumsande)
fdae638e83 doc: Improve doc for functions involved in saving blocks to disk (Martin Zumsande)
0d114e3cb2 blockstorage: Add Assume for fKnown / snapshot chainstate (Martin Zumsande)

Pull request description:

  `SaveBlockToDisk` / `FindBlockPos` are used for two purposes, depending on whether they are called during reindexing (`dbp` set,  `fKnown = true`) or in the "normal" case when adding new blocks (`dbp == nullptr`,  `fKnown = false`).
  The actual tasks are quite different
  - In normal mode, preparations for saving a new block are made, which is then saved: find the correct position on disk (maybe skipping to a new blk file), check for available disk space, update the blockfile info db, save the block.
  - during reindex, most of this is not necessary (the block is already on disk after all), only the blockfile info needs to rebuilt because reindex wiped the leveldb it's saved in.

  Using one function with many conditional statements for this leads to code that is hard to read / understand and bug-prone:
  - many code paths in `FindBlockPos` are conditional on `fKnown` or `!fKnown`
  - It's not really clear what actually needs to be done during reindex (we don't need to "save a block to disk" or "find a block pos" as the function names suggest)
  - logic that should be applied to only one of the two modes is sometimes applied to both (see first commit, or #27039)

  #24858 and #27039 were recent bugs directly related to the differences between reindexing and normal mode, and in both cases the simple fix took a long time to be reviewed and merged.

  This PR proposes to clean this code up by splitting out the reindex logic into a separate function (`UpdateBlockInfo`) which will be called directly from validation. As a result, `SaveBlockToDisk` and `FindBlockPos` only need to cover the non-reindex logic.

ACKs for top commit:
  paplorinc:
    ACK e41667b720
  TheCharlatan:
    Re-ACK e41667b720
  ryanofsky:
    Code review ACK e41667b720. Just improvements to comments since last review.

Tree-SHA512: a14ff9a0facf6b1e3c1cd724a2d19a79a25d4b48de64398fdd172671532a472bc10a20cbb64ac3a3e55814dcc877d0597a3e1699cabc4f9d9a86b439b6eaba20
2024-05-16 11:16:08 -04:00
Ryan Ofsky
75118a608f
Merge bitcoin/bitcoin#27101: Support JSON-RPC 2.0 when requested by client
cbc6c440e3 doc: add comments and release-notes for JSON-RPC 2.0 (Matthew Zipkin)
e7ee80dcf2 rpc: JSON-RPC 2.0 should not respond to "notifications" (Matthew Zipkin)
bf1a1f1662 rpc: Avoid returning HTTP errors for JSON-RPC 2.0 requests (Matthew Zipkin)
466b90562f rpc: Add "jsonrpc" field and drop null "result"/"error" fields (Matthew Zipkin)
2ca1460ae3 rpc: identify JSON-RPC 2.0 requests (Matthew Zipkin)
a64a2b77e0 rpc: refactor single/batch requests (Matthew Zipkin)
df6e3756d6 rpc: Avoid copies in JSONRPCReplyObj() (Matthew Zipkin)
09416f9ec4 test: cover JSONRPC 2.0 requests, batches, and notifications (Matthew Zipkin)
4202c170da test: refactor interface_rpc.py (Matthew Zipkin)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/2960

  Bitcoin Core's JSONRPC server behaves with a special blend of 1.0, 1.1 and 2.0 behaviors. This introduces compliance issues with more strict clients. There are the major misbehaviors that I found:
  - returning non-200 HTTP codes for RPC errors like "Method not found" (this is not a server error or an HTTP error)
  - returning both `"error"` and `"result"` fields together in a response object.
  - different error-handling behavior for single and batched RPC requests (batches contain errors in the response but single requests will actually throw HTTP errors)

  https://github.com/bitcoin/bitcoin/pull/15495 added regression tests after a discussion in https://github.com/bitcoin/bitcoin/pull/15381 to kinda lock in our RPC behavior to preserve backwards compatibility.

  https://github.com/bitcoin/bitcoin/pull/12435 was an attempt to allow strict 2.0 compliance behind a flag, but was abandoned.

  The approach in this PR is not strict and preserves backwards compatibility in a familiar bitcoin-y way: all old behavior is preserved, but new rules are applied to clients that opt in. One of the rules in the [JSON RPC 2.0 spec](https://www.jsonrpc.org/specification#request_object) is that the kv pair `"jsonrpc": "2.0"` must be present in the request. Well, let's just use that to trigger strict 2.0 behavior! When that kv pair is included in a request object, the [response will adhere to strict JSON-RPC 2.0 rules](https://www.jsonrpc.org/specification#response_object), essentially:

  - always return HTTP 200 "OK" unless there really is a server error or malformed request
  - either return `"error"` OR `"result"` but never both
  - same behavior for single and batch requests

  If this is merged next steps can be:

  - Refactor bitcoin-cli to always use strict 2.0
  - Refactor the python test framework to always use strict 2.0 for everything
  - Begin deprecation process for 1.0/1.1 behavior (?)

  If we can one day remove the old 1.0/1.1 behavior we can clean up the rpc code quite a bit.

ACKs for top commit:
  cbergqvist:
    re ACK cbc6c440e3
  ryanofsky:
    Code review ACK cbc6c440e3. Just suggested changes since the last review: changing uncaught exception error code from PARSE_ERROR to MISC_ERROR, renaming a few things, and adding comments.
  tdb3:
    re ACK for cbc6c440e3

Tree-SHA512: 0b702ed32368b34b29ad570d090951a7aeb56e3b0f2baf745bd32fdc58ef68fee6b0b8fad901f1ca42573ed714b150303829cddad4a34ca7ad847350feeedb36
2024-05-16 10:18:04 -04:00
TheCharlatan
b47bd95920
kernel: De-globalize fReindex
fReindex is one of the last remaining globals exposed by the kernel
library, so move it into the blockstorage class to reduce the amount of
global mutable state and make the kernel library a bit less awkward to
use.
2024-05-16 11:28:46 +02:00
merge-script
dd42a5ddea
Merge bitcoin/bitcoin#30085: p2p: detect addnode cjdns peers in GetAddedNodeInfo()
d0b047494c test: add GetAddedNodeInfo() CJDNS regression unit test (Jon Atack)
684da97070 p2p, bugfix: detect addnode cjdns peers in GetAddedNodeInfo() (Jon Atack)

Pull request description:

  Addnode peers connected to us via the cjdns network are currently not detected by `CConnman::GetAddedNodeInfo()`, i.e. `fConnected` is always false. This causes the following issues:

  - RPC `getaddednodeinfo` incorrectly shows them as not connected

  - `CConnman::ThreadOpenAddedConnections()` continually retries to connect them

  Fix the issue and add a unit regression test. Extracted from #28248. Suggest running the test with:

  `./src/test/test_bitcoin -t net_peer_connection_tests -l test_suite`

ACKs for top commit:
  mzumsande:
    utACK d0b047494c
  brunoerg:
    crACK d0b047494c
  pinheadmz:
    ACK d0b047494c

Tree-SHA512: a4d81425f79558f5792585611f3fe8ab999b82144daeed5c3ec619861c69add934c2b2afdad24c8488a0ade94f5ce8112f5555d60a1ce913d4f5a1cf5dbba55a
2024-05-16 11:18:26 +08:00
merge-script
ae2658caac
Merge bitcoin/bitcoin#30097: crypto: disable asan for sha256_sse4 with clang and -O0
141df0a288 crypto: disable asan for sha256_sse4 with clang and -O0 (Cory Fields)

Pull request description:

  Clang is unable to compile the Transform function for that combination of options.

  Fixes #29801.

ACKs for top commit:
  achow101:
    ACK 141df0a288

Tree-SHA512: d74fdac5840ad7524edfde069fb43ae75c31146e90ecc58bbc7912ff57a02b068547431b1766afeed782272c0b93b0b41a286c1cf26ec55ce332d94ce917d810
2024-05-16 08:40:34 +08:00
Ava Chow
71f0f2273f
Merge bitcoin/bitcoin#28929: serialization: Support for multiple parameters
8d491ae9ec serialization: Add ParamsStream GetStream() method (Ryan Ofsky)
951203bcc4 net: Simplify ParamsStream usage (Ryan Ofsky)
e6794e475c serialization: Accept multiple parameters in ParamsStream constructor (Ryan Ofsky)
cb28849a88 serialization: Reverse ParamsStream constructor order (Ryan Ofsky)
83436d14f0 serialization: Drop unnecessary ParamsStream references (Ryan Ofsky)
84502b755b serialization: Drop references to GetVersion/GetType (Ryan Ofsky)
f3a2b52376 serialization: Support for multiple parameters (Ryan Ofsky)

Pull request description:

  Currently it is only possible to attach one serialization parameter to a stream at a time. For example, it is not possible to set a parameter controlling the transaction format and a parameter controlling the address format at the same time because one parameter will override the other.

  This limitation is inconvenient for multiprocess code since it is not possible to create just one type of stream and serialize any object to it. Instead it is necessary to create different streams for different object types, which requires extra boilerplate and makes using the new parameter fields a lot more awkward than the older version and type fields.

  Fix this problem by allowing an unlimited number of serialization stream parameters to be set, and allowing them to be requested by type. Later parameters will still override earlier parameters, but only if they have the same type.

  For an example of different ways multiple parameters can be set, see the new [`with_params_multi`](40f505583f/src/test/serialize_tests.cpp (L394-L410)) unit test.

  This change requires replacing the `stream.GetParams()` method with a `stream.GetParams<T>()` method in order for serialization code to retrieve the desired parameters. The change is more verbose, but probably a good thing for readability because previously it could be difficult to know what type the `GetParams()` method would return, and now it is more obvious.

  ---

  This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/issues/28722).

ACKs for top commit:
  maflcko:
    ACK 8d491ae9ec 🔵
  sipa:
    utACK 8d491ae9ec
  TheCharlatan:
    ACK 8d491ae9ec

Tree-SHA512: 40b7041ee01c0372b1f86f7fd6f3b6af56ef24a6383f91ffcedd04d388e63407006457bf7ed056b0e37b4dec9ffd5ca006cb8192e488ea2c64678567e38d4647
2024-05-15 15:09:56 -04:00
merge-script
7a40f2a3f1
Merge bitcoin-core/gui#722: wallet: Allow user to navigate options while encrypting at creation
cccddc03f0 Wallet encrypt on create, allow to navigate options (Hernan Marino)

Pull request description:

  This fixes https://github.com/bitcoin-core/gui/issues/394.
  It adds a  "Go back" button to the "Confirm wallet encryption" window, allowing the users to change the password if they want to. It also adds a Cancel button to the "Wallet to be encrypted" window.
  Prior to this change users had no option to alter the password, and were forced to either go ahead with wallet creation or cancel the whole process. Also, at the final window, they were shown a warning but with no option to cancel.
  The new workflow for wallet encryption and creation is similar to the following:

  ![videoNavigation](https://user-images.githubusercontent.com/87907936/225705434-22d3c678-fa01-4079-ba10-ca5a0e8d3922.gif)

ACKs for top commit:
  alfonsoromanz:
    Re-Tested ACK cccddc03f0
  BrandonOdiwuor:
    re-Tested ACK cccddc03f0
  hebasto:
    ACK cccddc03f0, tested on Ubuntu 24.04.

Tree-SHA512: d2856d75f75acbd7d51ede62b4abd317f6ed6a9b890fe0b73b63b921b4b3d61b49477e35dc74466a056a9e8c0c1598df7601111d36c57ef18fdfdf0b18f503e6
2024-05-15 18:41:15 +01:00
Ryan Ofsky
33303b2b29
Merge bitcoin/bitcoin#30000: p2p: index TxOrphanage by wtxid, allow entries with same txid
0fb17bf61a [log] updates in TxOrphanage (glozow)
b16da7eda7 [functional test] attackers sending mutated orphans (glozow)
6675f6428d [unit test] TxOrphanage handling of same-txid-different-witness txns (glozow)
8923edfc1f [p2p] allow entries with the same txid in TxOrphanage (glozow)
c31f148166 [refactor] TxOrphanage::EraseTx by wtxid (glozow)
efcc593017 [refactor] TxOrphanage::HaveTx only by wtxid (glozow)
7e475b9648 [p2p] don't query orphanage by txid (glozow)

Pull request description:

  Part of #27463 in the "make orphan handling more robust" section.

  Currently the main map in `TxOrphanage` is indexed by txid; we do not allow 2 transactions with the same txid into TxOrphanage. This means that if we receive a transaction and want to store it in orphanage, we'll fail to do so if a same-txid-different-witness version of the tx already exists in the orphanage. The existing orphanage entry can stay until it expires 20 minutes later, or until we find that it is invalid.

  This means an attacker can try to block/delay us accepting an orphan transaction by sending a mutated version of the child ahead of time. See included test.

  Prior to #28970, we don't rely on the orphanage for anything and it would be relatively difficult to guess what transaction will go to a node's orphanage. After the parent(s) are accepted, if anybody sends us the correct transaction, we'll end up accepting it. However, this is a bit more painful for 1p1c: it's easier for an attacker to tell when a tx is going to hit a node's orphanage, and we need to store the correct orphan + receive the parent before we'll consider the package. If we start out with a bad orphan, we can't evict it until we receive the parent + try the 1p1c, and then we'll need to download the real child, put it in orphanage, download the parent again, and then retry 1p1c.

ACKs for top commit:
  AngusP:
    ACK 0fb17bf61a
  itornaza:
    trACK 0fb17bf61a
  instagibbs:
    ACK 0fb17bf61a
  theStack:
    ACK 0fb17bf61a
  sr-gi:
    crACK [0fb17bf](0fb17bf61a)
  stickies-v:
    ACK 0fb17bf61a

Tree-SHA512: edcbac7287c628bc27036920c2d4e4f63ec65087fbac1de9319c4f541515d669fc4e5fdc30c8b9a248b720da42b89153d388e91c7bf5caf4bc5b3b931ded1f59
2024-05-15 09:56:17 -04:00
Cory Fields
141df0a288 crypto: disable asan for sha256_sse4 with clang and -O0
Clang is unable to compile the Transform function for that combination of
options.
2024-05-15 13:50:25 +00:00
merge-script
42d5a1ff25
Merge bitcoin/bitcoin#30060: ci: Roll clang in test-each-commit task
fa90ad23c0 ci: Roll test-each-commit Ubuntu (MarcoFalke)
fa6c82dd9b ci: Remove clang version pin in test-each-commit (MarcoFalke)

Pull request description:

  Needed for https://github.com/bitcoin/bitcoin/pull/29077#issuecomment-2099704210

ACKs for top commit:
  hebasto:
    re-ACK fa90ad23c0.

Tree-SHA512: 753a3a77d967f308b5b5dd0bc7ea9f3268fc93c5ac978da3d79b85461cb1e994c6ac481888dc472b9a08be45ad0fad66ad3fda241a8955f999b7c2cb2a2b1f58
2024-05-15 16:31:27 +08:00
MarcoFalke
fa90ad23c0
ci: Roll test-each-commit Ubuntu 2024-05-15 09:53:04 +02:00
MarcoFalke
fa6c82dd9b
ci: Remove clang version pin in test-each-commit 2024-05-15 09:53:00 +02:00
merge-script
3d24189664
Merge bitcoin/bitcoin#30098: refactor: simplify FormatSubVersion using strprintf/Join
12d82817bf refactor: simplify `FormatSubVersion` using strprintf/Join (Sebastian Falbesoner)

Pull request description:

  Rather than using std::ostringstream and manually joining the comments, use strprintf and our own `Join` helper.

ACKs for top commit:
  maflcko:
    utACK 12d82817bf
  TheCharlatan:
    tACK 12d82817bf
  hebasto:
    ACK 12d82817bf, I have reviewed the code and it looks OK.
  tdb3:
    ACK for 12d82817bf.

Tree-SHA512: b9b965c4416a4c0c8727e3c4b40da4be04b14067200220492e9bed4fa35c1907fb5cdec2a30052b9e762f71da3d3cf042f43c96ab1f2523df5bb9920b44ea2a5
2024-05-15 12:48:37 +08:00
merge-script
695d80126f
Merge bitcoin/bitcoin#30074: contrib: use ENV flags in get_arch
b59a027d95 contrib: drop dead get_machine from test sym check (fanquake)
e6aba463ad contrib: use env_flags in get_arch (fanquake)

Pull request description:

  This isn't an issue right now (because the get_arch check is simple), but becomes one as soon as we want to use `lld` for linking, and need LDFLAGS (otherwise we call `ld` and fail, see it's usage in #21778). So I've split this out for review. It also makes sense to use the same flags for all compilation in these checks.

  Also drops some dead code in test-symbol-check.

ACKs for top commit:
  TheCharlatan:
    ACK b59a027d95

Tree-SHA512: d8afc4144815369aae63cf6dc6e983af46f208c7043d6ea5c9c811152649c256a8e67eb6864ea9d385d87b6b049fece07710a84b90da325da7fc3f05efcaacd6
2024-05-15 09:02:32 +08:00
Ava Chow
f5fc3190fb
Merge bitcoin/bitcoin#29086: refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition
cc67d33fda refactor: Simply include CTxMemPool::Options in CTxMemPool directly rather than duplicating definition (Luke Dashjr)

Pull request description:

  Instead of duplicating mempool options two places, just include the Options struct directly on the CTxMemPool

ACKs for top commit:
  achow101:
    ACK cc67d33fda
  kristapsk:
    cr utACK cc67d33fda
  jonatack:
    ACK cc67d33fda

Tree-SHA512: 9deb5ea6f85eeb1c7e04536cded65303b0ec459936a97e4f257aff2c50b0984a4ddbf69a4651f48455b9c80200a1fd24e9c74926874fdd9be436bbbe406251ce
2024-05-14 20:00:34 -04:00
Ryan Ofsky
e41667b720 blockstorage: Don't move cursor backwards in UpdateBlockInfo
Previously, it was possible to move the cursor back to an older file
during reindex if blocks are enocuntered out of order during reindex.
This would mean that MaxBlockfileNum() would be incorrect, and
a wrong DB_LAST_BLOCK could be written to disk.

This improves the logic by only ever moving the cursor forward (if possible)
but not backwards.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2024-05-14 14:54:27 -04:00
Martin Zumsande
17103637c6 blockstorage: Rename FindBlockPos and have it return a FlatFilePos
The new name reflects that it is no longer called with existing blocks
for which the position is already known.

Returning a FlatFilePos directly simplifies the interface.
2024-05-14 14:54:27 -04:00
Martin Zumsande
d9e477c4dc validation, blockstorage: Separate code paths for reindex and saving new blocks
By calling SaveBlockToDisk only when we actually want to save a new
block to disk. In the reindex case, we now call UpdateBlockInfo
directly from validation.

This commit doesn't change behavior.
2024-05-14 14:54:27 -04:00
Martin Zumsande
064859bbad blockstorage: split up FindBlockPos function
FindBlockPos does different things depending on whether the block is known
or not, as shown by the fact that much of the existing code is conditional on fKnown set or not.

If the block position is known (during reindex) the function only updates the block info
statistics. It doesn't actually find a block position in this case.

This commit removes fKnown and splits up these two code paths by introducing a separate function
for the reindex case when the block position is known.
It doesn't change behavior.
2024-05-14 14:54:26 -04:00
Martin Zumsande
fdae638e83 doc: Improve doc for functions involved in saving blocks to disk
In particular, document the flat file positions expected and
returned by functions better.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-05-14 13:49:34 -04:00
Matthew Zipkin
cbc6c440e3
doc: add comments and release-notes for JSON-RPC 2.0 2024-05-14 11:28:48 -04:00
Matthew Zipkin
e7ee80dcf2
rpc: JSON-RPC 2.0 should not respond to "notifications"
For JSON-RPC 2.0 requests we need to distinguish between
a missing "id" field and "id":null. This is accomplished
by making the JSONRPCRequest id property a
std::optional<UniValue> with a default value of
UniValue::VNULL.

A side-effect of this change for non-2.0 requests is that request which do not
specify an "id" field will no longer return "id": null in the response.
2024-05-14 11:28:43 -04:00
Ryan Ofsky
dbb3113082
Merge bitcoin/bitcoin#30083: kernel: Remove batchpriority from kernel library
d4b17c7d46 kernel: Remove batchpriority from kernel library (TheCharlatan)

Pull request description:

  The current usage of ScheduleBatchPriority is not transparent. Once the thread scheduling is changed, it remains unchanged for the remainder of the thread's lifetime. So move the call from `ImportBlocks` to the init code where it is clearer that its effect lasts for the entire lifetime of the thread.

  Users of the kernel library might not expect `ImportBlocks` to have an influence on the thread it is called in. Particularly since it is only a compile time option and cannot be controlled at runtime. With this patch users of the kernel library can now freely choose their own scheduling policy.

  This PR is easier reviewed with `git diff --color-moved-ws=ignore-all-space --color-moved=dimmed-zebra`

  ---
  This PR is part of the [libbitcoinkernel project](https://github.com/bitcoin/bitcoin/issues/27587).

ACKs for top commit:
  maflcko:
    ACK d4b17c7d46 📭
  ryanofsky:
    Code review ACK d4b17c7d46, just added suggested comment since last review
  hebasto:
    ACK d4b17c7d46, I have reviewed the code and it looks OK.

Tree-SHA512: cafedecd9affad58ddd7f30f68bba71291ca951bb186ff4b2da04b7f21f0b26e5e3143846d032b9e391bd5ce6c7466b97aa3758d2a85ebd7353eb8b69139641a
2024-05-14 11:20:33 -04:00
Matthew Zipkin
bf1a1f1662
rpc: Avoid returning HTTP errors for JSON-RPC 2.0 requests
Avoid returning HTTP status errors for non-batch JSON-RPC 2.0 requests if the
RPC method failed but the HTTP request was otherwise valid. Batch requests
already did not return HTTP errors previously.
2024-05-14 11:15:54 -04:00
Matthew Zipkin
466b90562f
rpc: Add "jsonrpc" field and drop null "result"/"error" fields
Only for JSON-RPC 2.0 requests.
2024-05-14 10:39:43 -04:00
Matthew Zipkin
2ca1460ae3
rpc: identify JSON-RPC 2.0 requests 2024-05-14 10:32:43 -04:00