Commit graph

38305 commits

Author SHA1 Message Date
Martin Leitner-Ankerl
c8839ec5cd refactor: use "if constexpr" in prevector's Serialize()
This gets rid of unnecessarily creating a temporary object T() to call
the right function.
2023-08-03 10:32:25 +02:00
Martin Leitner-Ankerl
1403d181c1 refactor: use fold expressions instead of recursive calls in UnserializeMany()
Instead of recursively calling `UnserializeMany` and peeling off one
argument at a time, use a fold expression. This simplifies the code,
makes it most likely faster because it reduces the number of function
calls, and compiles faster because there are fewer template
instantiations.
2023-08-03 10:31:33 +02:00
Martin Leitner-Ankerl
bd08a008b4 refactor: use fold expressions instead of recursive calls in SerializeMany()
Instead of recursively calling `SerializeMany` and peeling off one
argument at a time, use a fold expression. This simplifies the code,
makes it most likely faster because it reduces the number of function
calls, and compiles faster because there are fewer template
instantiations.
2023-08-03 10:30:02 +02:00
Hennadii Stepanov
703b758e18
qa: Close SQLite connection properly
Connection object used as context manager only commits or rollbacks
transactions, so the connection object should be closed manually.

Fixes the following error on Windows:
```
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: ...
```
2023-08-02 19:29:01 +01:00
fanquake
2fa60f0b68
Merge bitcoin/bitcoin#27452: test: cover addrv2 anchors by adding TorV3 to CAddress in messages.py
ba8ab4fc54 test: cover addrv2 support in anchors.dat with a TorV3 address (Matthew Zipkin)
b4bee4bbf4 test: add keep_alive option to socks5 proxy in test_framework (Matthew Zipkin)
5aaf988ccc test: cover TorV3 address in p2p_addrv2_relay (Matthew Zipkin)
80f64a3d40 test: add support for all networks in CAddress in messages.py (brunoerg)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27140

  Adds test coverage for https://github.com/bitcoin/bitcoin/pull/20516 to ensure that https://github.com/bitcoin/bitcoin/issues/20511 is completed and may be closed.

  This PR adds a test case to `feature_anchors.py` where an onion v3 address is set as a blocks-only relay peer and then shutdown, ensuring that the address is saved to anchors.dat in addrv2 format. We then ensure that bitcoin attempts to reconnect to that anchor address on restart.

  To compute the addrv2 serialization of the onion v3 address, I added logic to `CAddress` in `messages.py`. This new logic is covered by extending `p2p_addrv2_relay.py` to include an onion v3 address. Future work will be adding coverage for ipv6, torv2 and cjdns in these modules and also `feature_proxy.py`

  Also includes de/serialization unit test for `CAddress` in test framework.

ACKs for top commit:
  jonatack:
    ACK ba8ab4fc54
  brunoerg:
    crACK ba8ab4fc54
  willcl-ark:
    ACK ba8ab4fc54

Tree-SHA512: 7220e30d7cb975903d9ac575a7215a08e8f784c24c5741561affcbde12fb92cbf8704cb42e66494b788ba6ed4bb255fb0cc327e4f2190fae50c0ed9f336c0ff0
2023-08-02 12:57:30 +01:00
fanquake
2dea6c5ca0
Merge bitcoin/bitcoin#27572: test: dedup file hashing using sha256sum_file helper
2c0c6f4477 test: dedup file hashing using `sha256sum_file` helper (Sebastian Falbesoner)

Pull request description:

  Rather than doing the open/read/hash-steps manually in the affected functional tests, we can just use the `sha256sum_file` helper from the utils module instead.

  Note that for the tool_wallet.py test, the used hash is changed from sha1 to sha256, but as the only purpose is to detect file content changes, this doesn't matter. Also, the optimization using `memoryview` is overkill here, as the opened file has only a size of 24KiB and determining the hash via the helper doesn't take longer than a few hundred micro-seconds on my machine.

ACKs for top commit:
  kristapsk:
    ACK 2c0c6f4477

Tree-SHA512: 64fe21650b56a50e9f1a95f6ef27d88d8bfbb621e5be456f327ef8dbb5596b529d03976c200f3fd68da48cc427de9f257b403f3228e38cf1df918006674fac68
2023-08-02 11:53:35 +01:00
TheCharlatan
afc534df9a
refactor: Wrap DestroyDB in dbwrapper helper
Wrap leveldb::DestroyDB in a helper function without exposing
leveldb-specifics.

Also, add missing optional include.

The context of this commit is an effort to decouple the dbwrapper header
file from leveldb includes. To this end, the includes are moved to the
dbwrapper implementation file. This is done as part of the kernel
project to reduce the number of required includes for users of the
kernel.
2023-08-01 22:14:15 +02:00
fanquake
1b5cbf71df
Merge bitcoin/bitcoin#28144: test: fix intermittent failure in p2p_getaddr_caching.py
8a20f765cc test: drop duplicate getaddrs from p2p_getaddr_caching (Martin Zumsande)
feb0096139 test: fix intermittent failure in p2p_getaddr_caching (Martin Zumsande)

Pull request description:

  Fixes #28133

  In the consistency check, it's not enough to check that our address/port is unique, only the combination of source and target must be unique. Otherwise, the OS may reuse ports for connections to different `-addrbind`, which was happening in the failed runs.

  While at it, the second commit cleans up duplicate `getaddr` messages in `p2p_getaddr_caching.py` that do nothing but generate `Ignoring repeated "getaddr"` log messages (and cleans up some whitespace the python linter complains about).

ACKs for top commit:
  vasild:
    ACK 8a20f765cc

Tree-SHA512: eabe4727d7887f729074076f6333a918bba8cb34b8e3baaa83f167b441b0daa24f7c4824abcf03a9538a2ef14b2d826ff19aeffcb93a6c20735253a9678aac9c
2023-08-01 16:59:37 +01:00
fanquake
eb95368e0c
Merge bitcoin/bitcoin#28166: test, rpc: invalid sighashtype coverage
90c8f79e94 test: remove redundant test values (Jon Atack)
c3f203387d test: use common assert_signing_completed_successfully helper (Jon Atack)
647d95aae9 test: add coverage for passing an invalid sighashtype (Jon Atack)

Pull request description:

  Add test coverage for passing an invalid sighashtype to RPCs signrawtransactionwithwallet, signrawtransactionwithkey, walletprocesspsbt, and descriptorprocesspsbt.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 90c8f79e94 🎥
  brunoerg:
    light crACK 90c8f79e94

Tree-SHA512: 3861658487edd0d9a377390acf3d43f45c3dd9e324894f0fdb8f5312b618301a55479b1f70c61daee0b20785e768ffde6fa5abe6af190b73c0d0e017f3976704
2023-08-01 16:56:15 +01:00
fanquake
ceda819886
Merge bitcoin/bitcoin#28060: util: Teach AutoFile how to XOR
fa633aa690 streams: Teach AutoFile how to XOR (MarcoFalke)
000019e158 Add AutoFile::detail_fread member function (MarcoFalke)
fa7724bc9d refactor: Modernize AutoFile (MarcoFalke)
fa8d227d58 doc: Remove comments that just repeat what the code does (MarcoFalke)
fafe2ca0ce refactor: Remove redundant file check from AutoFile shift operators (MarcoFalke)
9999a49b32 Extract util::Xor, Add key_offset option, Add bench (MarcoFalke)

Pull request description:

  This allows `AutoFile` to roll an XOR pattern while reading or writing to the underlying file.

  This is needed for https://github.com/bitcoin/bitcoin/pull/28052, but can also be used in any other place.

  Also, there are tests, so I've split this up from the larger pull to make review easier, hopefully.

ACKs for top commit:
  Crypt-iQ:
    crACK fa633aa
  willcl-ark:
    Lightly tested ACK fa633aa690
  jamesob:
    reACK fa633aa690 ([`jamesob/ackr/28060.4.MarcoFalke.util_add_xorfile`](https://github.com/jamesob/bitcoin/tree/ackr/28060.4.MarcoFalke.util_add_xorfile))

Tree-SHA512: 6d66cad0a089a096d3f95e4f2b28bce80b349d4b76f53d09dc9a0bea4fc1b7c0652724469c37971ba27728c7d46398a4c1d289c252af4c0f83bb2fcbc6f8e90b
2023-08-01 16:38:06 +01:00
MarcoFalke
fa474397b5
ci: Add missing linux-headers package to ASan task
Otherwise the task will throw in skip_if_no_python_bcc.

Also, adjust CI_CONTAINER_CAP for all needed permissions.
2023-08-01 17:33:36 +02:00
MarcoFalke
fabaa85c01
ci: Move ASan USDT to persistent_worker 2023-08-01 17:33:33 +02:00
fanquake
e5a9f2fb62
Merge bitcoin/bitcoin#28194: test: python E721 and flake8 updates
bee2d57a65 script: update flake8 to 6.1.0 (Jon Atack)
38c3fd846b test: python E721 updates (Jon Atack)

Pull request description:

  Update our functional tests per [E721](https://www.flake8rules.com/rules/E721.html) enforced by [flake8 6.1.0](https://flake8.pycqa.org/en/latest/release-notes/6.1.0.html), and update our CI lint task to use that release.  This makes the following linter output on current master with flake8 6.1.0 green.

  ```
  $ ./test/lint/lint-python.py ; ./test/lint/lint-spelling.py
  test/functional/p2p_invalid_locator.py:35:16: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:34:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:64:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  src/test/fuzz/descriptor_parse.cpp:88: occurences ==> occurrences
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/spelling.ignore-words.txt
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bee2d57a65

Tree-SHA512: f3788a543ca98e44eeeba1d06c32f1b11eec95d4aef068aa1b6b5c401261adfa3fb6c6d6c769f3fe6839d78e74a310d5c926867e7c367d6513a53d580fd376f3
2023-08-01 09:42:07 +01:00
fanquake
fadad10126
Merge bitcoin/bitcoin#28131: test: Add UBSan -fsanitize=integer suppressions for src/secp256k1 subtree
a7477744c5 Add UBSan `-fsanitize=integer` suppressions for `src/secp256k1` subtree (Hennadii Stepanov)

Pull request description:

  Required for https://github.com/bitcoin/bitcoin/pull/27991 (see the [comment](https://github.com/bitcoin/bitcoin/pull/27991#issuecomment-1611472816)) and for the upcoming CMake-based build system.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK a7477744c5

Tree-SHA512: 602fa3ad22d3b0f6981a51358677d2347c92c4c9f59626b497af10f7ba828ede37227d8ee717f089bf33bde5efe0854d53acc89bea46f0955e62b7f22c454d05
2023-08-01 09:40:36 +01:00
fanquake
b6c66f3091
Merge bitcoin/bitcoin#28003: doc: cleanup release process doc
bd5ae6c663 doc: misc changes in release-process (fanquake)
d99ba3cc01 doc: filter out merge-script from list of authors (josibake)
472d6f79b9 doc: remove generate changelog section from release-process.md (fanquake)
5555ecb80e doc: remove note to update bips.md version number (fanquake)

Pull request description:

  Collection of changes to to simplify / correct the release-process documentation.
  I think we could still simplify this further.
  For example, we could remove the guix building docs, and defer to `contrib/guix`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bd5ae6c663

Tree-SHA512: 44bd12dd4f09380daee03fa79f01f9f7e8e2d5cc2fd5ff8c9e85ab54b4ea2b8a35df30ce3bdecfb4ff056cf52822be771ed3419613f68929c122750b1f8c89f9
2023-08-01 09:40:07 +01:00
fanquake
8535802f1d
Merge bitcoin/bitcoin#28070: test: Drop 22.x node from TxindexCompatibilityTest
fafe43cb6c scripted-diff: Use blocks_path where possible (MarcoFalke)
fa060c15fb test: Add blocks_path property to TestNode (MarcoFalke)
faba4fc325 test: Drop 22.x node from TxindexCompatibilityTest (MarcoFalke)
fa7f65b0f8 test: Use clean chain in MempoolCompatibilityTest (MarcoFalke)

Pull request description:

  The node in this test was never really needed, because the compatibility tests shouldn't be used to test previous releases. (The test suite of the previous release itself should be used for that). So remove it.

  Also, other test changes. (See individual commits)

ACKs for top commit:
  theStack:
    Code-review ACK fafe43cb6c

Tree-SHA512: 289f54695bf5310663ab38ebf1aa457f53d0aafae56e6657be0e75bf96b303165bad417dc7eaf4c40f0639aa92ce139e5bacb318a2eabab1f8e23a811cabe0cc
2023-08-01 09:38:49 +01:00
Ryan Ofsky
f4f1d6d230
Merge bitcoin/bitcoin#27746: Rework validation logic for assumeutxo
a733dd79e2 Remove unused function `reliesOnAssumedValid` (Suhas Daftuar)
d4a11abb19 Cache block index entry corresponding to assumeutxo snapshot base blockhash (Suhas Daftuar)
3556b85022 Move CheckBlockIndex() from Chainstate to ChainstateManager (Suhas Daftuar)
0ce805b632 Documentation improvements for assumeutxo (Ryan Ofsky)
768690b7ce Fix initialization of setBlockIndexCandidates when working with multiple chainstates (Suhas Daftuar)
d43a1f1a2f Tighten requirements for adding elements to setBlockIndexCandidates (Suhas Daftuar)
d0d40ea9a6 Move block-storage-related logic to ChainstateManager (Suhas Daftuar)
3cfc75366e test: Clear block index flags when testing snapshots (Suhas Daftuar)
272fbc370c Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot (Suhas Daftuar)
10c05710ce Add wrapper for adding entries to a chainstate's block index candidates (Suhas Daftuar)
471da5f6e7 Move block-arrival information / preciousblock counters to ChainstateManager (Suhas Daftuar)
1cfc887d00 Remove CChain dependency in node/blockstorage (Suhas Daftuar)
fe86a7cd48 Explicitly track maximum block height stored in undo files (Suhas Daftuar)

Pull request description:

  This PR proposes a clean up of the relationship between block storage and the chainstate objects, by moving the decision of whether to store a block on disk to something that is not chainstate-specific.  Philosophically, the decision of whether to store a block on disk is related to validation rules that do not require any UTXO state; for anti-DoS reasons we were using some chainstate-specific heuristics, and those have been reworked here to achieve the proposed separation.

  This PR also fixes a bug in how a chainstate's `setBlockIndexCandidates` was being initialized; it should always have all the HAVE_DATA block index entries that have more work than the chain tip.  During startup, we were not fully populating `setBlockIndexCandidates` in some scenarios involving multiple chainstates.

  Further, this PR establishes a concept that whenever we have 2 chainstates, that we always know the snapshotted chain's base block and the base block's hash must be an element of our block index. Given that, we can establish a new invariant that the background validation chainstate only needs to consider blocks leading to that snapshotted block entry as potential candidates for its tip. As a followup I would imagine that when writing net_processing logic to download blocks for the background chainstate, that we would use this concept to only download blocks towards the snapshotted entry as well.

ACKs for top commit:
  achow101:
    ACK a733dd79e2
  jamesob:
    reACK a733dd79e2 ([`jamesob/ackr/27746.5.sdaftuar.rework_validation_logic`](https://github.com/jamesob/bitcoin/tree/ackr/27746.5.sdaftuar.rework_validation_logic))
  Sjors:
    Code review ACK a733dd79e2.
  ryanofsky:
    Code review ACK a733dd79e2. Just suggested changes since the last review. There are various small things that could be followed up on, but I think this is ready for merge.

Tree-SHA512: 9ec17746f22b9c27082743ee581b8adceb2bd322fceafa507b428bdcc3ffb8b4c6601fc61cc7bb1161f890c3d38503e8b49474da7b5ab1b1f38bda7aa8668675
2023-07-31 16:18:20 -04:00
Jon Atack
bee2d57a65 script: update flake8 to 6.1.0
and touch up the spelling returned by lint-spelling.py
2023-07-31 12:14:06 -06:00
Jon Atack
38c3fd846b test: python E721 updates 2023-07-31 12:13:46 -06:00
MarcoFalke
fa69e3a95c
Remove unused MessageStartChars parameters from BlockManager methods 2023-07-31 14:32:57 +02:00
fanquake
44b05bf3fe
Merge bitcoin/bitcoin#28091: fuzz: use ConnmanTestMsg in connman
ecfe507e07 fuzz: use `ConnmanTestMsg` in `connman` (brunoerg)

Pull request description:

  Fixes #27980

  Using `ConnmanTestMsg` we can add nodes and be
  more effective fuzzing functions like `DisconnectNode`,
  `FindNode`, `GetNodeStats` and other ones.

ACKs for top commit:
  MarcoFalke:
    review ACK ecfe507e07
  dergoegge:
    utACK ecfe507e07

Tree-SHA512: 97c363b422809f2e9755c082d1102237347abfab72c7baca417bd8975f8a595ddf3a085f8353dbdb9f17fb98fbfe830792bfc0b83451168458018faf6c239efa
2023-07-31 11:43:39 +01:00
fanquake
e92013e178
Merge bitcoin/bitcoin#28188: ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE
79ceb161db ci: Use documented `CCACHE_MAXSIZE` instead of `CCACHE_SIZE` (Hennadii Stepanov)

Pull request description:

  This PR aims to:
  1) Remove our own `CCACHE_SIZE` environment variable that violates Ccache's `CCACHE_*` namespace.
  2) Introduce the `CCACHE_MAXSIZE` environment variable that is documented since [v3.3](https://ccache.dev/manual/3.3.html), which makes its usage consistent with other ones, such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 79ceb161db

Tree-SHA512: 13c8a3a3b2337191cab32a3e1c21c19dc465cd4fa9267b2999ca2fde5cca0c03811f520cd3940959aa57ea9cf0251db325df56a90fca85069d04ce2141ec7044
2023-07-31 11:19:55 +01:00
fanquake
80800361b1
Merge bitcoin/bitcoin#28181: qa, doc: Fix comment
ab498d913c qa, doc: Fix comment (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up for:
  - https://github.com/bitcoin/bitcoin/pull/9956
  - https://github.com/bitcoin/bitcoin/pull/10096

ACKs for top commit:
  RandyMcMillan:
    ACK ab498d913c

Tree-SHA512: 267ae52c961ee79e172f27cb1587282ac5cf7ec929a136db6feed3de4f319a74b462befd9b99711081db8a5c30a513f72366364068700418b273a31958b31b1d
2023-07-31 10:51:50 +01:00
Hennadii Stepanov
79ceb161db
ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE
This change aims to:
1) Remove our own `CCACHE_SIZE` environment variable that violates
Ccache's `CCACHE_*` namespace.
2) Introduce the `CCACHE_MAXSIZE` environment variable that is
documented since v3.3, which makes its usage consistent with other ones,
such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.
2023-07-30 21:36:56 +01:00
fanquake
64440bb733
Merge bitcoin/bitcoin#28118: test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC
fabef121b0 refactor: Use EnsureAnyNodeContext (MarcoFalke)
fa1640617e test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC (MarcoFalke)

Pull request description:

  There should be no risk or downside in adding a call to `SyncWithValidationInterfaceQueue` here. In fact, it will make tests less brittle. For example,

  * If one sets the timeouts in `test/functional/feature_fee_estimation.py` to `0`, on `master` the test will fail and here it will pass.
  * It may avoid a rare (theoretic) intermittent issue in https://github.com/bitcoin/bitcoin/pull/28108/files#r1268966663

ACKs for top commit:
  TheCharlatan:
    ACK fabef121b0
  furszy:
    Code review ACK fabef121. Convinced by checking all current tests usages.

Tree-SHA512: c9e9a536a8721d1b3f267a66b40578b34948892301affdcad121ef8e02bf17037305d0dd53aa94b1b064753e66f9cfb31823b916b707a9d812627f502b818003
2023-07-30 11:21:40 +01:00
Hennadii Stepanov
ab498d913c
qa, doc: Fix comment
This change is a follow-up for:
- https://github.com/bitcoin/bitcoin/pull/9956
- https://github.com/bitcoin/bitcoin/pull/10096
2023-07-30 11:10:59 +01:00
fanquake
4c57e53a61
Merge bitcoin/bitcoin#28138: ci: Keep system env vars as-is (bugfix)
fabc04a4d9 ci: Keep system env vars as-is (MarcoFalke)
fa8dcdcc8b ci: Set PATH inside the CI env (MarcoFalke)
fac229ab1f ci: Remove P_CI_DIR and --workdir (MarcoFalke)

Pull request description:

  This fixes a bug where the `$PATH` from the host is used inside the container. This will lead to bugs when the `$PATH` is different. For example on a host of Fedora 38, and a container of `debian:bullseye`.

  This can be tested with the `FILE_ENV=./ci/test/00_setup_env_arm.sh` CI env. On master:

  ```
  Error: crun: executable file `bash` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
  ```

  On this pull:

  (everything passes)

ACKs for top commit:
  TheCharlatan:
    lgtm ACK fabc04a4d9

Tree-SHA512: 51d2affed91624d0e5b43a3ee1e696313f934e05fde6b5a19e8ac4e6666c1e7b667a444bf3de3f77190bcd00e81efb7576196afb0f6b5e788d1a50e7ddb28d7e
2023-07-28 15:56:14 +01:00
fanquake
42a9110899
Merge bitcoin/bitcoin#28162: refactor: Revert additional univalue check in ParseSighashString
06199a995f refactor: Revert addition of univalue sighash string check (TheCharlatan)
0b47c16215 doc: Correct release-notes for sighashtype exceptions (TheCharlatan)

Pull request description:

  This is a follow up for #28113.

  The string type check is already done by the rpc parser / RPCHelpMan. Re-doing it is adding dead code. Instead, throwing an exception when the assumption does not hold is the already correct behavior. Pointed out in this [comment](https://github.com/bitcoin/bitcoin/pull/28113/files#r1274568557).

  Also correct the release note for the correct sighashtype exception change. There is no change in the handling of non-string sighashtype arugments. Pointed out in this [comment](https://github.com/bitcoin/bitcoin/pull/28113/files#r1274567555).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 06199a995f
  jonatack:
    Tested ACK 06199a995f
  stickies-v:
    ACK 06199a995f

Tree-SHA512: 3faa6b3d2247624c0973df8d79c09fbf1f90ffb99f1be484e359b528f485c31affea45976759bd206e4c81cbb54ebba5ad0ef4127d1deacbfe2a58153fcc94ee
2023-07-28 12:29:55 +01:00
fanquake
8aa77a77e6
Merge bitcoin/bitcoin#28168: refactor: Remove unused raw-pointer read helper from univalue
fa940f41ea Remove unused raw-pointer read helper from univalue (MarcoFalke)

Pull request description:

  The helpers are unused outside of tests and redundant with the existing `bool read(std::string_view raw);`.

  Fix both issues by removing them.

  Also, simplify the tests code by removing a `std::string` constructor where possible.

ACKs for top commit:
  stickies-v:
    utACK fa940f41ea
  TheCharlatan:
    tACK fa940f41ea

Tree-SHA512: 60c154c1046f01551335af79bf820a6104844f63e89977271b4336b3cd06ac3bab1379e18b7bc61d12bef7446029e91c16541ddecf9e88bc8bc897fc1f6ee2c8
2023-07-28 12:04:49 +01:00
Andrew Chow
cbf385058b
Merge bitcoin/bitcoin#27888: Fuzz: a more efficient descriptor parsing target
131314b62e fuzz: increase coverage of the descriptor targets (Antoine Poinsot)
90a24741e7 fuzz: add a new, more efficient, descriptor parsing target (Antoine Poinsot)
d60229ede5 fuzz: make the parsed descriptor testing into a function (Antoine Poinsot)

Pull request description:

  The current descriptor parsing fuzz target requires valid public or private keys to be provided. This is unnecessary as we are only interested in fuzzing the descriptor parsing logic here (other targets are focused on fuzzing keys serializations). And it's pretty inefficient, especially for formats that need a checksum (`xpub`, `xprv`, WIF).

  This introduces a new target that mocks the keys as an index in a list of precomputed keys. Keys are represented as 2 hex characters in the descriptor. The key type (private, public, extended, ..) is deterministically based on this one-byte value. Keys are deterministically generated at target initialization. This is much more efficient and also largely reduces the size of the seeds.
  TL;DR: for instance instead of requiring the fuzzer to generate a `pk(xpub6DdBu7pBoyf7RjnUVhg8y6LFCfca2QAGJ39FcsgXM52Pg7eejUHLBJn4gNMey5dacyt4AjvKzdTQiuLfRdK8rSzyqZPJmNAcYZ9kVVEz4kj)` to parse a valid descriptor, it just needs to generate a `pk(03)`.

  Note we only mock the keys themselves, not the entire descriptor key expression. As we want to fuzz the real code that parses the rest of the key expression (origin, derivation paths, ..).

  This is a target i used for reviewing #17190 and #27255, and figured it was worth PR'ing on its own since the added complexity for mocking the keys is minimal and it could help prevent introducing bugs to the descriptor parsing logic much more efficiently.

ACKs for top commit:
  MarcoFalke:
    re-ACK 131314b62e  🐓
  achow101:
    ACK 131314b62e

Tree-SHA512: 485a8d6a0f31a3a132df94dc57f97bdd81583d63507510debaac6a41dbbb42fa83c704ff3f2bd0b78c8673c583157c9a3efd79410e5e79511859e1470e629118
2023-07-27 13:48:12 -04:00
Andrew Chow
272c4f3f10
Merge bitcoin/bitcoin#28148: refactor: consistently use ApplyArgsManOptions for PeerManager::Options
8a3159728a refactor: deduplicate ignores_incoming_txs (stickies-v)
5f41afcc46 refactor: set ignore_incoming_txs in ApplyArgsManOptions (stickies-v)

Pull request description:

  Consistently use `ApplyArgsManOptions` for `PeerManager::Options`, and initialize `PeerManager::Options` early to avoid reading `"-blocksonly"` twice. Suggested in https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1268400386 and also requested in https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1273346189.

  No behaviour change, but the [`TestingSetup`](e35fb7bc48/src/test/util/setup_common.cpp (L255-L256)) is now also able to access `"-blocksonly"`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 8a3159728a
  achow101:
    ACK 8a3159728a
  TheCharlatan:
    ACK 8a3159728a
  dergoegge:
    utACK 8a3159728a

Tree-SHA512: 6cb489d79ac2a87e8faedb76c96973ab3fc597426f274a90a3ffd0bc5fe3f2b25db9c7ec2e55a0c806c2bcbc0fdded6e228adb43d2cd81f14fd6552863847698
2023-07-27 11:33:47 -04:00
Jon Atack
90c8f79e94 test: remove redundant test values
as they are parsed identically.

See AmountFromValue() / ParseFixedPoint() / UniValue#getValStr()
2023-07-27 06:54:32 -06:00
Jon Atack
c3f203387d test: use common assert_signing_completed_successfully helper 2023-07-27 06:37:55 -06:00
Jon Atack
647d95aae9 test: add coverage for passing an invalid sighashtype
in RPCs descriptorprocesspbst, walletprocesspbst, signrawtransactionwithkey,
and signrawtransactionwithwallet.
2023-07-27 06:37:54 -06:00
MarcoFalke
fa940f41ea
Remove unused raw-pointer read helper from univalue 2023-07-27 14:24:52 +02:00
fanquake
dfe2dc1d84
Merge bitcoin/bitcoin#28164: test: remove unused code in wallet_fundrawtransaction
108c6255bc test: remove unused `totalOut` code (brunoerg)
0fc3deee9a test: remove unecessary `decoderawtransaction` calls (brunoerg)

Pull request description:

  This PR removes in `wallet_fundrawtransaction`:
  - unecessary variables/calls to `decoderawtransaction`
  - unused `totalOut` variable and its related code (`totalOut` is used in some functions to test change, in other ones its value is not used)

ACKs for top commit:
  kevkevinpal:
    utACK [108c625](108c6255bc)
  MarcoFalke:
    lgtm ACK 108c6255bc

Tree-SHA512: c352524f3633146117534c79bd1a24523a7068f13a17d0b8a425cc3c85d62cb769a79ea60db8b075b137da2a0cc43142c43a23ca5af89246ff86cd824e37cf17
2023-07-27 12:25:18 +01:00
fanquake
04f66ce500
Merge bitcoin/bitcoin#28092: ci: document that -Wreturn-type has been fixed upstream (mingw-w64)
08eb5f1b67 ci: document that -Wreturn-type has been fixed upstream (Windows) (fanquake)

Pull request description:

  `noreturn` attributes have been added to the mingw-w64 headers, 1690994f51, meaning that [from 11.0.0 onwards](https://www.mingw-w64.org/changelog/), you'll no-longer see `-Wreturn-type` warnings when using `assert(false)`.

  Add -Wno-return-type to the Windows CI, where is should have been all
  along, and document why it's required. This can be dropped when we are
  using the fixed version of the mingw-w64 headers there.

  Drop the -Werror -Wno-return-type special case from our build system.
  -Wreturn-type is on by default in Clang and GCC.

  The new mingw-w64 header behaviour can be checked on Ubuntu mantic, [which ships with 11.0.0](https://packages.ubuntu.com/mantic/mingw-w64), using:
  ```cpp
  #include <cassert>

  int f(){ assert(false); }

  int main() {
  return 0;
  }
  ```

  On Mantic (with 11.0.0):
  ```bash
  x86_64-w64-mingw32-g++ test.cpp -Wreturn-type
  # nada
  ```

  On Lunar ([with 10.0.0](https://packages.ubuntu.com/lunar/mingw-w64)):
  ```bash
  x86_64-w64-mingw32-g++ test.cpp -Wreturn-type
  test.cpp: In function 'int f()':
  test.cpp:3:25: warning: no return statement in function returning non-void [-Wreturn-type]
      3 | int f(){ assert(false); }
        |                         ^
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 08eb5f1b67

Tree-SHA512: 9cd4310a96abd87bf8ceb37949ad0259fe4adee3367c604f4c4ad521a0cf09bdcc5dd305db19a0f45ce74c85178b0d739e2fca5ad0fc841ac935523a23b28a7f
2023-07-27 11:21:49 +01:00
TheCharlatan
06199a995f
refactor: Revert addition of univalue sighash string check
This check is already done by the rpc parser. Re-doing it is adding dead
code. Instead, throwing an exception when the assumption does not hold
is the already correct behavior.

To make the fuzz test more accurate and not swallow all runtime errors,
add a check that the passed in UniValue sighash argument is either a
string or null.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2023-07-27 09:36:05 +02:00
TheCharlatan
0b47c16215
doc: Correct release-notes for sighashtype exceptions 2023-07-27 09:16:11 +02:00
Pieter Wuille
1c7582ead6 tests: add decryption test to bip324_tests 2023-07-26 17:09:27 -04:00
Pieter Wuille
990f0f8da9 Add BIP324Cipher, encapsulating key agreement, derivation, and stream/AEAD ciphers
Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-07-26 17:09:23 -04:00
Pieter Wuille
c91cedf281 crypto: support split plaintext in ChaCha20Poly1305 Encrypt/Decrypt 2023-07-26 17:00:22 -04:00
Pieter Wuille
af2b44c76e bench: add benchmark for FSChaCha20Poly1305
Add a benchmark for FSChaCha20Poly1305 encryption, so the overhead of key
generation and authentication can be observed for various message sizes.
2023-07-26 16:55:05 -04:00
Pieter Wuille
aa8cee9334 crypto: add FSChaCha20Poly1305, rekeying wrapper around ChaCha20Poly1305
This adds the FSChaCha20Poly1305 AEAD as specified in BIP324, a wrapper
around the ChaCha20Poly1305 AEAD (as specified in RFC8439 section 2.8) which
automatically rekeys every N messages, and automatically increments the nonce
every message.
2023-07-26 16:55:05 -04:00
Pieter Wuille
0fee267792 crypto: add FSChaCha20, a rekeying wrapper around ChaCha20
This adds the FSChaCha20 stream cipher as specified in BIP324, a
wrapper around the ChaCha20 stream cipher (specified in RFC8439
section 2.4) which automatically rekeys every N messages, and
manages the nonces used for encryption.

Co-authored-by: dhruv <856960+dhruv@users.noreply.github.com>
2023-07-26 16:55:05 -04:00
Pieter Wuille
9ff0768bdc crypto: add the ChaCha20Poly1305 AEAD as specified in RFC8439
This adds an implementation of the ChaCha20Poly1305 AEAD exactly matching
the version specified in RFC8439 section 2.8, including tests and official
test vectors.
2023-07-26 16:55:00 -04:00
Pieter Wuille
9fd085a1a4 crypto: remove outdated variant of ChaCha20Poly1305 AEAD
Remove the variant of ChaCha20Poly1305 AEAD that was previously added in
anticipation of BIP324 using it. BIP324 was updated to instead use rekeying
wrappers around otherwise unmodified versions of the ChaCha20 stream cipher
and the ChaCha20Poly1305 AEAD as specified in RFC8439.
2023-07-26 16:51:51 -04:00
brunoerg
108c6255bc test: remove unused totalOut code
In `wallet_fundrawtransaction`, `totalOut` is used in
some functions to check if the change is correct. In
other ones, it has been created but never used.
2023-07-26 16:05:37 -03:00
brunoerg
0fc3deee9a test: remove unecessary decoderawtransaction calls
In `wallet_fundrawtransaction`, there are some unecessary
variables/calls to `decoderawtransaction`. They have not
been used.
2023-07-26 15:57:43 -03:00
fanquake
f57e724a80
Merge bitcoin/bitcoin#28127: refactor: Remove C-style const-violating cast, Use reinterpret_cast
fa9108f85a refactor: Use reinterpret_cast where appropriate (MarcoFalke)
3333f950d4 refactor: Avoid casting away constness (MarcoFalke)
fa6394dd10 refactor: Remove unused C-style casts (MarcoFalke)

Pull request description:

  Using a C-style cast to convert pointer types to a byte-like pointer type has many issues:

  * It may accidentally and silently throw away `const`.
  * It forces reviewers to check that it doesn't accidentally throw away `const`.

  For example, on current master a `const char*` is cast to `unsigned char*` (without `const`), see d23fda0584/src/span.h (L273) . This can lead to UB, and the only reason why it didn't lead to UB is because the return type added back the `const`. (Obviously this would break if the return type was deduced via `auto`)

  Fix all issues by adding back the `const` and using `reinterpret_cast` where appropriate.

ACKs for top commit:
  darosior:
    re-utACK fa9108f85a
  hebasto:
    re-ACK fa9108f85a.
  john-moffett:
    ACK fa9108f85a

Tree-SHA512: 87f6e4b574f9bd96d4e0f2a0631fd0a9dc6096e5d4f1b95042fe9f197afc2fe9a24e333aeb34fed11feefcdb184a238fe1ea5aff10d580bb18d76bfe48b76a10
2023-07-26 16:03:39 +01:00