Commit graph

38244 commits

Author SHA1 Message Date
fanquake
7de23cceb8
refactor: fix unterminated LogPrintf()s 2023-08-03 17:52:24 +01:00
fanquake
0a1029aa29
lint: remove /* Continued */ markers from codebase 2023-08-03 17:52:24 +01:00
fanquake
910007995d
lint: remove lint-logs.py 2023-08-03 17:52:24 +01:00
fanquake
d86a83d6b8
lint: drop DIR_IWYU global 2023-08-03 17:52:24 +01:00
fanquake
da3816e4e8
Merge bitcoin/bitcoin#27832: doc: Clarify -datacarriersize, add -datacarriersize=2 tests
faafc35a77 doc: Clarify that -datacarriersize applies to the full raw scriptPubKey, not the data push (MarcoFalke)
55550e7fe7 test: Add -datacarriersize=2 tests (MarcoFalke)

Pull request description:

  Clarify with a test that `-datacarriersize` applies to the serialized size of the scriptPubKey, not the size of the pushed data. So for example,

  * `-datacarriersize=2` will reject a `raw(6a01aa)`, even though only one byte is pushed
  * `-datacarriersize=0` (or `-datacarrier=0`) will reject a `raw(6a)`, even though no byte is pushed
  * `-datacarriersize=0` (or `-datacarrier=0`) will reject a `raw(6a00)`, even though zero bytes are pushed

ACKs for top commit:
  ajtowns:
    ACK faafc35a77
  instagibbs:
    ACK faafc35a77

Tree-SHA512: f01ace02798f596ac2a02461e9f2a6ef91b3b37c976ea0b3bc860e2d3efb0ace0fd8b779dd18249cee7f84ebbe5fd21d8506afd3a15edadc00b843ff3b4aacc7
2023-08-03 17:46:43 +01:00
fanquake
61849f0464
Merge bitcoin/bitcoin#27918: fuzz: addrman, avoid ConsumeDeserializable when possible
025fda0a76 fuzz: addrman, avoid `ConsumeDeserializable` when possible (brunoerg)

Pull request description:

  Using specific functions like `ConsumeService`, `ConsumeAddress` and `ConsumeNetAddr` may be more effective than using `ConsumeDeserializable`. They always return some value while `ConsumeDeserializable` may return `std::nullopt`.

  E.g.: In this part of the code, if `op_net_addr` is `std::nullopt`,  we basically generated the addresses (if so) unnecessarily, because we won't be able to use them:
  ```cpp
  std::vector<CAddress> addresses;
  LIMITED_WHILE(fuzzed_data_provider.ConsumeBool(), 10000) {
      const std::optional<CAddress> opt_address = ConsumeDeserializable<CAddress>(fuzzed_data_provider);
      if (!opt_address) {
          break;
      }
      addresses.push_back(*opt_address);
  }
  const std::optional<CNetAddr> opt_net_addr = ConsumeDeserializable<CNetAddr>(fuzzed_data_provider);
  if (opt_net_addr) {
      addr_man.Add(addresses, *opt_net_addr, std::chrono::seconds{ConsumeTime(fuzzed_data_provider, 0, 100000000)});
  }
  ```

  Also, if we are not calling `Add` effectively, it would also be affect other functions that may "depend" on it.

ACKs for top commit:
  dergoegge:
    Code review ACK 025fda0a76

Tree-SHA512: 02450bec0b084c15ba0cd1cbdfbac067c8fea4ccf27be0c86d54e020f029a6c749a16d8e0558f9d6d35a7ca9db8916f180c872f09474702b5591129e9be0d192
2023-08-03 17:32:46 +01:00
glozow
7c66a4b610
Merge bitcoin/bitcoin#28059: refactor: Make more transaction size variables signed
92de74ef18 refactor: Make more transaction size variables signed (Hennadii Stepanov)

Pull request description:

  This PR is a continuation of https://github.com/bitcoin/bitcoin/pull/23962 and it:
  - gets rid of two static casts,
  - addresses https://github.com/bitcoin/bitcoin/pull/23962#issuecomment-1593289706,
  - is useful for https://github.com/bitcoin/bitcoin/pull/25972, see the failed ARM and multiprocess CI jobs.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 92de74ef18  🥔
  glozow:
    ACK 92de74ef18

Tree-SHA512: 84225961af8e08439664e75661b98fe86560217e891e5633a28316bf248d88df317a0c6b5a5f6b03feb2b0e0fd40a1f91dd4a85a0610d567470805bf47a84487
2023-08-03 11:45:51 +01:00
fanquake
532bd1f2e7
Merge bitcoin/bitcoin#28204: qa: Close SQLite connection properly
703b758e18 qa: Close SQLite connection properly (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up for https://github.com/bitcoin/bitcoin/pull/26462 that introduced a bug on Windows:
  ```
  >test\functional\wallet_descriptor.py
  ...
  PermissionError: [WinError 32] The process cannot access the file because it is being used by another process:
  ...
  ```

  From `sqlite3` Python module [docs](https://docs.python.org/3/library/sqlite3.html#how-to-use-the-connection-context-manager):
  > `Connection` object used as context manager only commits or rollbacks transactions, so the connection object should be closed manually.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 703b758e18
  theStack:
    utACK 703b758e18

Tree-SHA512: 35b1403507be06d1fc04e7e07ff56af5bcfe5013024671f0c1d9f3c41aacc4c777bcc6376ce82d720394e27450415d50ff5d5834ed388ec3f21503f86f1a42a5
2023-08-03 10:02:19 +01:00
Hennadii Stepanov
703b758e18
qa: Close SQLite connection properly
Connection object used as context manager only commits or rollbacks
transactions, so the connection object should be closed manually.

Fixes the following error on Windows:
```
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: ...
```
2023-08-02 19:29:01 +01:00
fanquake
2fa60f0b68
Merge bitcoin/bitcoin#27452: test: cover addrv2 anchors by adding TorV3 to CAddress in messages.py
ba8ab4fc54 test: cover addrv2 support in anchors.dat with a TorV3 address (Matthew Zipkin)
b4bee4bbf4 test: add keep_alive option to socks5 proxy in test_framework (Matthew Zipkin)
5aaf988ccc test: cover TorV3 address in p2p_addrv2_relay (Matthew Zipkin)
80f64a3d40 test: add support for all networks in CAddress in messages.py (brunoerg)

Pull request description:

  Closes https://github.com/bitcoin/bitcoin/issues/27140

  Adds test coverage for https://github.com/bitcoin/bitcoin/pull/20516 to ensure that https://github.com/bitcoin/bitcoin/issues/20511 is completed and may be closed.

  This PR adds a test case to `feature_anchors.py` where an onion v3 address is set as a blocks-only relay peer and then shutdown, ensuring that the address is saved to anchors.dat in addrv2 format. We then ensure that bitcoin attempts to reconnect to that anchor address on restart.

  To compute the addrv2 serialization of the onion v3 address, I added logic to `CAddress` in `messages.py`. This new logic is covered by extending `p2p_addrv2_relay.py` to include an onion v3 address. Future work will be adding coverage for ipv6, torv2 and cjdns in these modules and also `feature_proxy.py`

  Also includes de/serialization unit test for `CAddress` in test framework.

ACKs for top commit:
  jonatack:
    ACK ba8ab4fc54
  brunoerg:
    crACK ba8ab4fc54
  willcl-ark:
    ACK ba8ab4fc54

Tree-SHA512: 7220e30d7cb975903d9ac575a7215a08e8f784c24c5741561affcbde12fb92cbf8704cb42e66494b788ba6ed4bb255fb0cc327e4f2190fae50c0ed9f336c0ff0
2023-08-02 12:57:30 +01:00
fanquake
2dea6c5ca0
Merge bitcoin/bitcoin#27572: test: dedup file hashing using sha256sum_file helper
2c0c6f4477 test: dedup file hashing using `sha256sum_file` helper (Sebastian Falbesoner)

Pull request description:

  Rather than doing the open/read/hash-steps manually in the affected functional tests, we can just use the `sha256sum_file` helper from the utils module instead.

  Note that for the tool_wallet.py test, the used hash is changed from sha1 to sha256, but as the only purpose is to detect file content changes, this doesn't matter. Also, the optimization using `memoryview` is overkill here, as the opened file has only a size of 24KiB and determining the hash via the helper doesn't take longer than a few hundred micro-seconds on my machine.

ACKs for top commit:
  kristapsk:
    ACK 2c0c6f4477

Tree-SHA512: 64fe21650b56a50e9f1a95f6ef27d88d8bfbb621e5be456f327ef8dbb5596b529d03976c200f3fd68da48cc427de9f257b403f3228e38cf1df918006674fac68
2023-08-02 11:53:35 +01:00
fanquake
1b5cbf71df
Merge bitcoin/bitcoin#28144: test: fix intermittent failure in p2p_getaddr_caching.py
8a20f765cc test: drop duplicate getaddrs from p2p_getaddr_caching (Martin Zumsande)
feb0096139 test: fix intermittent failure in p2p_getaddr_caching (Martin Zumsande)

Pull request description:

  Fixes #28133

  In the consistency check, it's not enough to check that our address/port is unique, only the combination of source and target must be unique. Otherwise, the OS may reuse ports for connections to different `-addrbind`, which was happening in the failed runs.

  While at it, the second commit cleans up duplicate `getaddr` messages in `p2p_getaddr_caching.py` that do nothing but generate `Ignoring repeated "getaddr"` log messages (and cleans up some whitespace the python linter complains about).

ACKs for top commit:
  vasild:
    ACK 8a20f765cc

Tree-SHA512: eabe4727d7887f729074076f6333a918bba8cb34b8e3baaa83f167b441b0daa24f7c4824abcf03a9538a2ef14b2d826ff19aeffcb93a6c20735253a9678aac9c
2023-08-01 16:59:37 +01:00
fanquake
eb95368e0c
Merge bitcoin/bitcoin#28166: test, rpc: invalid sighashtype coverage
90c8f79e94 test: remove redundant test values (Jon Atack)
c3f203387d test: use common assert_signing_completed_successfully helper (Jon Atack)
647d95aae9 test: add coverage for passing an invalid sighashtype (Jon Atack)

Pull request description:

  Add test coverage for passing an invalid sighashtype to RPCs signrawtransactionwithwallet, signrawtransactionwithkey, walletprocesspsbt, and descriptorprocesspsbt.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 90c8f79e94 🎥
  brunoerg:
    light crACK 90c8f79e94

Tree-SHA512: 3861658487edd0d9a377390acf3d43f45c3dd9e324894f0fdb8f5312b618301a55479b1f70c61daee0b20785e768ffde6fa5abe6af190b73c0d0e017f3976704
2023-08-01 16:56:15 +01:00
fanquake
ceda819886
Merge bitcoin/bitcoin#28060: util: Teach AutoFile how to XOR
fa633aa690 streams: Teach AutoFile how to XOR (MarcoFalke)
000019e158 Add AutoFile::detail_fread member function (MarcoFalke)
fa7724bc9d refactor: Modernize AutoFile (MarcoFalke)
fa8d227d58 doc: Remove comments that just repeat what the code does (MarcoFalke)
fafe2ca0ce refactor: Remove redundant file check from AutoFile shift operators (MarcoFalke)
9999a49b32 Extract util::Xor, Add key_offset option, Add bench (MarcoFalke)

Pull request description:

  This allows `AutoFile` to roll an XOR pattern while reading or writing to the underlying file.

  This is needed for https://github.com/bitcoin/bitcoin/pull/28052, but can also be used in any other place.

  Also, there are tests, so I've split this up from the larger pull to make review easier, hopefully.

ACKs for top commit:
  Crypt-iQ:
    crACK fa633aa
  willcl-ark:
    Lightly tested ACK fa633aa690
  jamesob:
    reACK fa633aa690 ([`jamesob/ackr/28060.4.MarcoFalke.util_add_xorfile`](https://github.com/jamesob/bitcoin/tree/ackr/28060.4.MarcoFalke.util_add_xorfile))

Tree-SHA512: 6d66cad0a089a096d3f95e4f2b28bce80b349d4b76f53d09dc9a0bea4fc1b7c0652724469c37971ba27728c7d46398a4c1d289c252af4c0f83bb2fcbc6f8e90b
2023-08-01 16:38:06 +01:00
fanquake
e5a9f2fb62
Merge bitcoin/bitcoin#28194: test: python E721 and flake8 updates
bee2d57a65 script: update flake8 to 6.1.0 (Jon Atack)
38c3fd846b test: python E721 updates (Jon Atack)

Pull request description:

  Update our functional tests per [E721](https://www.flake8rules.com/rules/E721.html) enforced by [flake8 6.1.0](https://flake8.pycqa.org/en/latest/release-notes/6.1.0.html), and update our CI lint task to use that release.  This makes the following linter output on current master with flake8 6.1.0 green.

  ```
  $ ./test/lint/lint-python.py ; ./test/lint/lint-spelling.py
  test/functional/p2p_invalid_locator.py:35:16: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:34:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  test/functional/test_framework/siphash.py:64:12: E721 do not compare types, for exact checks use `is` / `is not`, for instance checks use `isinstance()`
  src/test/fuzz/descriptor_parse.cpp:88: occurences ==> occurrences
  ^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/spelling.ignore-words.txt
  ```

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bee2d57a65

Tree-SHA512: f3788a543ca98e44eeeba1d06c32f1b11eec95d4aef068aa1b6b5c401261adfa3fb6c6d6c769f3fe6839d78e74a310d5c926867e7c367d6513a53d580fd376f3
2023-08-01 09:42:07 +01:00
fanquake
fadad10126
Merge bitcoin/bitcoin#28131: test: Add UBSan -fsanitize=integer suppressions for src/secp256k1 subtree
a7477744c5 Add UBSan `-fsanitize=integer` suppressions for `src/secp256k1` subtree (Hennadii Stepanov)

Pull request description:

  Required for https://github.com/bitcoin/bitcoin/pull/27991 (see the [comment](https://github.com/bitcoin/bitcoin/pull/27991#issuecomment-1611472816)) and for the upcoming CMake-based build system.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK a7477744c5

Tree-SHA512: 602fa3ad22d3b0f6981a51358677d2347c92c4c9f59626b497af10f7ba828ede37227d8ee717f089bf33bde5efe0854d53acc89bea46f0955e62b7f22c454d05
2023-08-01 09:40:36 +01:00
fanquake
b6c66f3091
Merge bitcoin/bitcoin#28003: doc: cleanup release process doc
bd5ae6c663 doc: misc changes in release-process (fanquake)
d99ba3cc01 doc: filter out merge-script from list of authors (josibake)
472d6f79b9 doc: remove generate changelog section from release-process.md (fanquake)
5555ecb80e doc: remove note to update bips.md version number (fanquake)

Pull request description:

  Collection of changes to to simplify / correct the release-process documentation.
  I think we could still simplify this further.
  For example, we could remove the guix building docs, and defer to `contrib/guix`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK bd5ae6c663

Tree-SHA512: 44bd12dd4f09380daee03fa79f01f9f7e8e2d5cc2fd5ff8c9e85ab54b4ea2b8a35df30ce3bdecfb4ff056cf52822be771ed3419613f68929c122750b1f8c89f9
2023-08-01 09:40:07 +01:00
fanquake
8535802f1d
Merge bitcoin/bitcoin#28070: test: Drop 22.x node from TxindexCompatibilityTest
fafe43cb6c scripted-diff: Use blocks_path where possible (MarcoFalke)
fa060c15fb test: Add blocks_path property to TestNode (MarcoFalke)
faba4fc325 test: Drop 22.x node from TxindexCompatibilityTest (MarcoFalke)
fa7f65b0f8 test: Use clean chain in MempoolCompatibilityTest (MarcoFalke)

Pull request description:

  The node in this test was never really needed, because the compatibility tests shouldn't be used to test previous releases. (The test suite of the previous release itself should be used for that). So remove it.

  Also, other test changes. (See individual commits)

ACKs for top commit:
  theStack:
    Code-review ACK fafe43cb6c

Tree-SHA512: 289f54695bf5310663ab38ebf1aa457f53d0aafae56e6657be0e75bf96b303165bad417dc7eaf4c40f0639aa92ce139e5bacb318a2eabab1f8e23a811cabe0cc
2023-08-01 09:38:49 +01:00
Ryan Ofsky
f4f1d6d230
Merge bitcoin/bitcoin#27746: Rework validation logic for assumeutxo
a733dd79e2 Remove unused function `reliesOnAssumedValid` (Suhas Daftuar)
d4a11abb19 Cache block index entry corresponding to assumeutxo snapshot base blockhash (Suhas Daftuar)
3556b85022 Move CheckBlockIndex() from Chainstate to ChainstateManager (Suhas Daftuar)
0ce805b632 Documentation improvements for assumeutxo (Ryan Ofsky)
768690b7ce Fix initialization of setBlockIndexCandidates when working with multiple chainstates (Suhas Daftuar)
d43a1f1a2f Tighten requirements for adding elements to setBlockIndexCandidates (Suhas Daftuar)
d0d40ea9a6 Move block-storage-related logic to ChainstateManager (Suhas Daftuar)
3cfc75366e test: Clear block index flags when testing snapshots (Suhas Daftuar)
272fbc370c Update CheckBlockIndex invariants for chains based on an assumeutxo snapshot (Suhas Daftuar)
10c05710ce Add wrapper for adding entries to a chainstate's block index candidates (Suhas Daftuar)
471da5f6e7 Move block-arrival information / preciousblock counters to ChainstateManager (Suhas Daftuar)
1cfc887d00 Remove CChain dependency in node/blockstorage (Suhas Daftuar)
fe86a7cd48 Explicitly track maximum block height stored in undo files (Suhas Daftuar)

Pull request description:

  This PR proposes a clean up of the relationship between block storage and the chainstate objects, by moving the decision of whether to store a block on disk to something that is not chainstate-specific.  Philosophically, the decision of whether to store a block on disk is related to validation rules that do not require any UTXO state; for anti-DoS reasons we were using some chainstate-specific heuristics, and those have been reworked here to achieve the proposed separation.

  This PR also fixes a bug in how a chainstate's `setBlockIndexCandidates` was being initialized; it should always have all the HAVE_DATA block index entries that have more work than the chain tip.  During startup, we were not fully populating `setBlockIndexCandidates` in some scenarios involving multiple chainstates.

  Further, this PR establishes a concept that whenever we have 2 chainstates, that we always know the snapshotted chain's base block and the base block's hash must be an element of our block index. Given that, we can establish a new invariant that the background validation chainstate only needs to consider blocks leading to that snapshotted block entry as potential candidates for its tip. As a followup I would imagine that when writing net_processing logic to download blocks for the background chainstate, that we would use this concept to only download blocks towards the snapshotted entry as well.

ACKs for top commit:
  achow101:
    ACK a733dd79e2
  jamesob:
    reACK a733dd79e2 ([`jamesob/ackr/27746.5.sdaftuar.rework_validation_logic`](https://github.com/jamesob/bitcoin/tree/ackr/27746.5.sdaftuar.rework_validation_logic))
  Sjors:
    Code review ACK a733dd79e2.
  ryanofsky:
    Code review ACK a733dd79e2. Just suggested changes since the last review. There are various small things that could be followed up on, but I think this is ready for merge.

Tree-SHA512: 9ec17746f22b9c27082743ee581b8adceb2bd322fceafa507b428bdcc3ffb8b4c6601fc61cc7bb1161f890c3d38503e8b49474da7b5ab1b1f38bda7aa8668675
2023-07-31 16:18:20 -04:00
Jon Atack
bee2d57a65 script: update flake8 to 6.1.0
and touch up the spelling returned by lint-spelling.py
2023-07-31 12:14:06 -06:00
Jon Atack
38c3fd846b test: python E721 updates 2023-07-31 12:13:46 -06:00
fanquake
44b05bf3fe
Merge bitcoin/bitcoin#28091: fuzz: use ConnmanTestMsg in connman
ecfe507e07 fuzz: use `ConnmanTestMsg` in `connman` (brunoerg)

Pull request description:

  Fixes #27980

  Using `ConnmanTestMsg` we can add nodes and be
  more effective fuzzing functions like `DisconnectNode`,
  `FindNode`, `GetNodeStats` and other ones.

ACKs for top commit:
  MarcoFalke:
    review ACK ecfe507e07
  dergoegge:
    utACK ecfe507e07

Tree-SHA512: 97c363b422809f2e9755c082d1102237347abfab72c7baca417bd8975f8a595ddf3a085f8353dbdb9f17fb98fbfe830792bfc0b83451168458018faf6c239efa
2023-07-31 11:43:39 +01:00
fanquake
e92013e178
Merge bitcoin/bitcoin#28188: ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE
79ceb161db ci: Use documented `CCACHE_MAXSIZE` instead of `CCACHE_SIZE` (Hennadii Stepanov)

Pull request description:

  This PR aims to:
  1) Remove our own `CCACHE_SIZE` environment variable that violates Ccache's `CCACHE_*` namespace.
  2) Introduce the `CCACHE_MAXSIZE` environment variable that is documented since [v3.3](https://ccache.dev/manual/3.3.html), which makes its usage consistent with other ones, such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 79ceb161db

Tree-SHA512: 13c8a3a3b2337191cab32a3e1c21c19dc465cd4fa9267b2999ca2fde5cca0c03811f520cd3940959aa57ea9cf0251db325df56a90fca85069d04ce2141ec7044
2023-07-31 11:19:55 +01:00
fanquake
80800361b1
Merge bitcoin/bitcoin#28181: qa, doc: Fix comment
ab498d913c qa, doc: Fix comment (Hennadii Stepanov)

Pull request description:

  This PR is a follow-up for:
  - https://github.com/bitcoin/bitcoin/pull/9956
  - https://github.com/bitcoin/bitcoin/pull/10096

ACKs for top commit:
  RandyMcMillan:
    ACK ab498d913c

Tree-SHA512: 267ae52c961ee79e172f27cb1587282ac5cf7ec929a136db6feed3de4f319a74b462befd9b99711081db8a5c30a513f72366364068700418b273a31958b31b1d
2023-07-31 10:51:50 +01:00
Hennadii Stepanov
79ceb161db
ci: Use documented CCACHE_MAXSIZE instead of CCACHE_SIZE
This change aims to:
1) Remove our own `CCACHE_SIZE` environment variable that violates
Ccache's `CCACHE_*` namespace.
2) Introduce the `CCACHE_MAXSIZE` environment variable that is
documented since v3.3, which makes its usage consistent with other ones,
such as `CCACHE_DIR` and `CCACHE_NOHASHDIR`.
2023-07-30 21:36:56 +01:00
fanquake
64440bb733
Merge bitcoin/bitcoin#28118: test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC
fabef121b0 refactor: Use EnsureAnyNodeContext (MarcoFalke)
fa1640617e test: Add SyncWithValidationInterfaceQueue to mockscheduler RPC (MarcoFalke)

Pull request description:

  There should be no risk or downside in adding a call to `SyncWithValidationInterfaceQueue` here. In fact, it will make tests less brittle. For example,

  * If one sets the timeouts in `test/functional/feature_fee_estimation.py` to `0`, on `master` the test will fail and here it will pass.
  * It may avoid a rare (theoretic) intermittent issue in https://github.com/bitcoin/bitcoin/pull/28108/files#r1268966663

ACKs for top commit:
  TheCharlatan:
    ACK fabef121b0
  furszy:
    Code review ACK fabef121. Convinced by checking all current tests usages.

Tree-SHA512: c9e9a536a8721d1b3f267a66b40578b34948892301affdcad121ef8e02bf17037305d0dd53aa94b1b064753e66f9cfb31823b916b707a9d812627f502b818003
2023-07-30 11:21:40 +01:00
Hennadii Stepanov
ab498d913c
qa, doc: Fix comment
This change is a follow-up for:
- https://github.com/bitcoin/bitcoin/pull/9956
- https://github.com/bitcoin/bitcoin/pull/10096
2023-07-30 11:10:59 +01:00
fanquake
4c57e53a61
Merge bitcoin/bitcoin#28138: ci: Keep system env vars as-is (bugfix)
fabc04a4d9 ci: Keep system env vars as-is (MarcoFalke)
fa8dcdcc8b ci: Set PATH inside the CI env (MarcoFalke)
fac229ab1f ci: Remove P_CI_DIR and --workdir (MarcoFalke)

Pull request description:

  This fixes a bug where the `$PATH` from the host is used inside the container. This will lead to bugs when the `$PATH` is different. For example on a host of Fedora 38, and a container of `debian:bullseye`.

  This can be tested with the `FILE_ENV=./ci/test/00_setup_env_arm.sh` CI env. On master:

  ```
  Error: crun: executable file `bash` not found in $PATH: No such file or directory: OCI runtime attempted to invoke a command that was not found
  ```

  On this pull:

  (everything passes)

ACKs for top commit:
  TheCharlatan:
    lgtm ACK fabc04a4d9

Tree-SHA512: 51d2affed91624d0e5b43a3ee1e696313f934e05fde6b5a19e8ac4e6666c1e7b667a444bf3de3f77190bcd00e81efb7576196afb0f6b5e788d1a50e7ddb28d7e
2023-07-28 15:56:14 +01:00
fanquake
42a9110899
Merge bitcoin/bitcoin#28162: refactor: Revert additional univalue check in ParseSighashString
06199a995f refactor: Revert addition of univalue sighash string check (TheCharlatan)
0b47c16215 doc: Correct release-notes for sighashtype exceptions (TheCharlatan)

Pull request description:

  This is a follow up for #28113.

  The string type check is already done by the rpc parser / RPCHelpMan. Re-doing it is adding dead code. Instead, throwing an exception when the assumption does not hold is the already correct behavior. Pointed out in this [comment](https://github.com/bitcoin/bitcoin/pull/28113/files#r1274568557).

  Also correct the release note for the correct sighashtype exception change. There is no change in the handling of non-string sighashtype arugments. Pointed out in this [comment](https://github.com/bitcoin/bitcoin/pull/28113/files#r1274567555).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 06199a995f
  jonatack:
    Tested ACK 06199a995f
  stickies-v:
    ACK 06199a995f

Tree-SHA512: 3faa6b3d2247624c0973df8d79c09fbf1f90ffb99f1be484e359b528f485c31affea45976759bd206e4c81cbb54ebba5ad0ef4127d1deacbfe2a58153fcc94ee
2023-07-28 12:29:55 +01:00
fanquake
8aa77a77e6
Merge bitcoin/bitcoin#28168: refactor: Remove unused raw-pointer read helper from univalue
fa940f41ea Remove unused raw-pointer read helper from univalue (MarcoFalke)

Pull request description:

  The helpers are unused outside of tests and redundant with the existing `bool read(std::string_view raw);`.

  Fix both issues by removing them.

  Also, simplify the tests code by removing a `std::string` constructor where possible.

ACKs for top commit:
  stickies-v:
    utACK fa940f41ea
  TheCharlatan:
    tACK fa940f41ea

Tree-SHA512: 60c154c1046f01551335af79bf820a6104844f63e89977271b4336b3cd06ac3bab1379e18b7bc61d12bef7446029e91c16541ddecf9e88bc8bc897fc1f6ee2c8
2023-07-28 12:04:49 +01:00
Andrew Chow
cbf385058b
Merge bitcoin/bitcoin#27888: Fuzz: a more efficient descriptor parsing target
131314b62e fuzz: increase coverage of the descriptor targets (Antoine Poinsot)
90a24741e7 fuzz: add a new, more efficient, descriptor parsing target (Antoine Poinsot)
d60229ede5 fuzz: make the parsed descriptor testing into a function (Antoine Poinsot)

Pull request description:

  The current descriptor parsing fuzz target requires valid public or private keys to be provided. This is unnecessary as we are only interested in fuzzing the descriptor parsing logic here (other targets are focused on fuzzing keys serializations). And it's pretty inefficient, especially for formats that need a checksum (`xpub`, `xprv`, WIF).

  This introduces a new target that mocks the keys as an index in a list of precomputed keys. Keys are represented as 2 hex characters in the descriptor. The key type (private, public, extended, ..) is deterministically based on this one-byte value. Keys are deterministically generated at target initialization. This is much more efficient and also largely reduces the size of the seeds.
  TL;DR: for instance instead of requiring the fuzzer to generate a `pk(xpub6DdBu7pBoyf7RjnUVhg8y6LFCfca2QAGJ39FcsgXM52Pg7eejUHLBJn4gNMey5dacyt4AjvKzdTQiuLfRdK8rSzyqZPJmNAcYZ9kVVEz4kj)` to parse a valid descriptor, it just needs to generate a `pk(03)`.

  Note we only mock the keys themselves, not the entire descriptor key expression. As we want to fuzz the real code that parses the rest of the key expression (origin, derivation paths, ..).

  This is a target i used for reviewing #17190 and #27255, and figured it was worth PR'ing on its own since the added complexity for mocking the keys is minimal and it could help prevent introducing bugs to the descriptor parsing logic much more efficiently.

ACKs for top commit:
  MarcoFalke:
    re-ACK 131314b62e  🐓
  achow101:
    ACK 131314b62e

Tree-SHA512: 485a8d6a0f31a3a132df94dc57f97bdd81583d63507510debaac6a41dbbb42fa83c704ff3f2bd0b78c8673c583157c9a3efd79410e5e79511859e1470e629118
2023-07-27 13:48:12 -04:00
Andrew Chow
272c4f3f10
Merge bitcoin/bitcoin#28148: refactor: consistently use ApplyArgsManOptions for PeerManager::Options
8a3159728a refactor: deduplicate ignores_incoming_txs (stickies-v)
5f41afcc46 refactor: set ignore_incoming_txs in ApplyArgsManOptions (stickies-v)

Pull request description:

  Consistently use `ApplyArgsManOptions` for `PeerManager::Options`, and initialize `PeerManager::Options` early to avoid reading `"-blocksonly"` twice. Suggested in https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1268400386 and also requested in https://github.com/bitcoin/bitcoin/pull/27499#discussion_r1273346189.

  No behaviour change, but the [`TestingSetup`](e35fb7bc48/src/test/util/setup_common.cpp (L255-L256)) is now also able to access `"-blocksonly"`.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 8a3159728a
  achow101:
    ACK 8a3159728a
  TheCharlatan:
    ACK 8a3159728a
  dergoegge:
    utACK 8a3159728a

Tree-SHA512: 6cb489d79ac2a87e8faedb76c96973ab3fc597426f274a90a3ffd0bc5fe3f2b25db9c7ec2e55a0c806c2bcbc0fdded6e228adb43d2cd81f14fd6552863847698
2023-07-27 11:33:47 -04:00
Jon Atack
90c8f79e94 test: remove redundant test values
as they are parsed identically.

See AmountFromValue() / ParseFixedPoint() / UniValue#getValStr()
2023-07-27 06:54:32 -06:00
Jon Atack
c3f203387d test: use common assert_signing_completed_successfully helper 2023-07-27 06:37:55 -06:00
Jon Atack
647d95aae9 test: add coverage for passing an invalid sighashtype
in RPCs descriptorprocesspbst, walletprocesspbst, signrawtransactionwithkey,
and signrawtransactionwithwallet.
2023-07-27 06:37:54 -06:00
MarcoFalke
fa940f41ea
Remove unused raw-pointer read helper from univalue 2023-07-27 14:24:52 +02:00
fanquake
dfe2dc1d84
Merge bitcoin/bitcoin#28164: test: remove unused code in wallet_fundrawtransaction
108c6255bc test: remove unused `totalOut` code (brunoerg)
0fc3deee9a test: remove unecessary `decoderawtransaction` calls (brunoerg)

Pull request description:

  This PR removes in `wallet_fundrawtransaction`:
  - unecessary variables/calls to `decoderawtransaction`
  - unused `totalOut` variable and its related code (`totalOut` is used in some functions to test change, in other ones its value is not used)

ACKs for top commit:
  kevkevinpal:
    utACK [108c625](108c6255bc)
  MarcoFalke:
    lgtm ACK 108c6255bc

Tree-SHA512: c352524f3633146117534c79bd1a24523a7068f13a17d0b8a425cc3c85d62cb769a79ea60db8b075b137da2a0cc43142c43a23ca5af89246ff86cd824e37cf17
2023-07-27 12:25:18 +01:00
fanquake
04f66ce500
Merge bitcoin/bitcoin#28092: ci: document that -Wreturn-type has been fixed upstream (mingw-w64)
08eb5f1b67 ci: document that -Wreturn-type has been fixed upstream (Windows) (fanquake)

Pull request description:

  `noreturn` attributes have been added to the mingw-w64 headers, 1690994f51, meaning that [from 11.0.0 onwards](https://www.mingw-w64.org/changelog/), you'll no-longer see `-Wreturn-type` warnings when using `assert(false)`.

  Add -Wno-return-type to the Windows CI, where is should have been all
  along, and document why it's required. This can be dropped when we are
  using the fixed version of the mingw-w64 headers there.

  Drop the -Werror -Wno-return-type special case from our build system.
  -Wreturn-type is on by default in Clang and GCC.

  The new mingw-w64 header behaviour can be checked on Ubuntu mantic, [which ships with 11.0.0](https://packages.ubuntu.com/mantic/mingw-w64), using:
  ```cpp
  #include <cassert>

  int f(){ assert(false); }

  int main() {
  return 0;
  }
  ```

  On Mantic (with 11.0.0):
  ```bash
  x86_64-w64-mingw32-g++ test.cpp -Wreturn-type
  # nada
  ```

  On Lunar ([with 10.0.0](https://packages.ubuntu.com/lunar/mingw-w64)):
  ```bash
  x86_64-w64-mingw32-g++ test.cpp -Wreturn-type
  test.cpp: In function 'int f()':
  test.cpp:3:25: warning: no return statement in function returning non-void [-Wreturn-type]
      3 | int f(){ assert(false); }
        |                         ^
  ```

ACKs for top commit:
  TheCharlatan:
    ACK 08eb5f1b67

Tree-SHA512: 9cd4310a96abd87bf8ceb37949ad0259fe4adee3367c604f4c4ad521a0cf09bdcc5dd305db19a0f45ce74c85178b0d739e2fca5ad0fc841ac935523a23b28a7f
2023-07-27 11:21:49 +01:00
TheCharlatan
06199a995f
refactor: Revert addition of univalue sighash string check
This check is already done by the rpc parser. Re-doing it is adding dead
code. Instead, throwing an exception when the assumption does not hold
is the already correct behavior.

To make the fuzz test more accurate and not swallow all runtime errors,
add a check that the passed in UniValue sighash argument is either a
string or null.

Co-authored-by: stickies-v <stickies-v@protonmail.com>
2023-07-27 09:36:05 +02:00
TheCharlatan
0b47c16215
doc: Correct release-notes for sighashtype exceptions 2023-07-27 09:16:11 +02:00
brunoerg
108c6255bc test: remove unused totalOut code
In `wallet_fundrawtransaction`, `totalOut` is used in
some functions to check if the change is correct. In
other ones, it has been created but never used.
2023-07-26 16:05:37 -03:00
brunoerg
0fc3deee9a test: remove unecessary decoderawtransaction calls
In `wallet_fundrawtransaction`, there are some unecessary
variables/calls to `decoderawtransaction`. They have not
been used.
2023-07-26 15:57:43 -03:00
fanquake
f57e724a80
Merge bitcoin/bitcoin#28127: refactor: Remove C-style const-violating cast, Use reinterpret_cast
fa9108f85a refactor: Use reinterpret_cast where appropriate (MarcoFalke)
3333f950d4 refactor: Avoid casting away constness (MarcoFalke)
fa6394dd10 refactor: Remove unused C-style casts (MarcoFalke)

Pull request description:

  Using a C-style cast to convert pointer types to a byte-like pointer type has many issues:

  * It may accidentally and silently throw away `const`.
  * It forces reviewers to check that it doesn't accidentally throw away `const`.

  For example, on current master a `const char*` is cast to `unsigned char*` (without `const`), see d23fda0584/src/span.h (L273) . This can lead to UB, and the only reason why it didn't lead to UB is because the return type added back the `const`. (Obviously this would break if the return type was deduced via `auto`)

  Fix all issues by adding back the `const` and using `reinterpret_cast` where appropriate.

ACKs for top commit:
  darosior:
    re-utACK fa9108f85a
  hebasto:
    re-ACK fa9108f85a.
  john-moffett:
    ACK fa9108f85a

Tree-SHA512: 87f6e4b574f9bd96d4e0f2a0631fd0a9dc6096e5d4f1b95042fe9f197afc2fe9a24e333aeb34fed11feefcdb184a238fe1ea5aff10d580bb18d76bfe48b76a10
2023-07-26 16:03:39 +01:00
fanquake
f033a981ed
Merge bitcoin/bitcoin#28139: test: create wallet specific for test_locked_wallet case
c648bdbda2 test: create wallet specific for test_locked_wallet case (furszy)

Pull request description:

  Coming from https://github.com/bitcoin/bitcoin/pull/28089#discussion_r1265478128.

  Several test cases are relying on the node1 default wallet, which thanks to 'test_locked_wallet' is encrypted.
  And can be only accessed within a specific timeframe (100ms), a duration internally set by the same test.

  This situation introduces a potential race condition, where other tests must complete their operations within
  the specified 100ms window to pass (otherwise the wallet gets re-locked and they fail).

  This can be seen running the test in valgrind (https://github.com/bitcoin/bitcoin/pull/28089), where other test cases fail due the wallet re-locking
  itself after the 100ms.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK c648bdbda2
  ishaanam:
    utACK c648bdbda2

Tree-SHA512: 01cde5a4a0cb3405adb9ea3c1f73841f3fa237d1162268ed06f0d49ca38541006b423a029e0b5e5955e1aa7e018c4600d894e555a68cf17ff60a4b8be58f4aa9
2023-07-26 10:19:56 +01:00
fanquake
c2ff87e1fa
Merge bitcoin/bitcoin#28150: test: Avoid intermittent issues due to async events in validationinterface_tests
faca9a3d5a test: Avoid intermittent issues due to async events in validationinterface_tests (MarcoFalke)

Pull request description:

  Currently the tests have many issues:

  * They setup the genesis block, even though it is not needed
  * They queue an async `UpdatedBlockTip` even, which causes intermittent issues: https://github.com/bitcoin/bitcoin/issues/28146#issuecomment-1650064645

  Fix all issues by trimming down the setup to just `ChainTestingSetup`.

ACKs for top commit:
  Crypt-iQ:
    tACK faca9a3d5a

Tree-SHA512: 4449040330f89bbaf5ce5b2052417c160b451c373987fdf1069596c07834ed81f0aea1506d53c7d2cd21062b27332d30679285dae194b272fd0cb9ce5ded32cf
2023-07-26 09:58:33 +01:00
fanquake
4517e2f4d4
Merge bitcoin/bitcoin#28147: suppressions: note that type:ClassName::MethodName should be used
d0c6cc4abe suppressions: note that 'type:ClassName::MethodName' should be used (fanquake)

Pull request description:

  Now that the symbolizer is back in play, suppressions can once-again be targeted to functions, rather than file-wide.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK d0c6cc4abe
  hebasto:
    ACK d0c6cc4abe

Tree-SHA512: fb65398eae18a6ebc5f8414275c568cf2664ab5357c2b3160f3bf285b67bc3af788225c5dba3c824c0e098627789450bec775375f52529d71c6ef700a9632d65
2023-07-26 09:42:50 +01:00
fanquake
8fba5dfc10
Merge bitcoin/bitcoin#27529: test: fix feature_addrman.py on big-endian systems
53c990ad34 test: fix `feature_addrman.py` on big-endian systems (Sebastian Falbesoner)

Pull request description:

  The test `feature_addrman.py` currently serializes the addrdb without specifying endianness for `int`s, so the machine's native byte order is used (see https://docs.python.org/3/library/struct.html#byte-order-size-and-alignment) and the generated `peers.dat` would be invalid on big-endian systems (our internal (de)serializers always use little-endian, see `ser_{read,write}data32`). Fix this by explicitly specifying little-endian serialization via the `<` character in `struct.pack(...)`.

  This is not detected by CI as we unfortunately don't run functional tests on big-endian systems there (I think we definitely should!).

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 53c990ad34   🔚

Tree-SHA512: 513af6f1f785a713e7a8ef3a57fcd3fe2520a7d537f63a9c8e1f4bdea4c2f605fd4c35001623d6b13458883dbc256f24943684ab8f224055c22bf8d8eeee5fe2
2023-07-26 09:42:20 +01:00
fanquake
95d523fabb
Merge bitcoin/bitcoin#28145: valgrind: add suppression for bug 472219
50f7214e09 valgrind: add suppression for bug 472219 (fanquake)

Pull request description:

  Now that https://bugs.kde.org/show_bug.cgi?id=472219 has been fixed upstream in:

  https://sourceware.org/git/?p=valgrind.git;a=commit;h=6ce0979884a8f246c80a098333ceef1a7b7f694d

  Add a supression to ignore the bug until we are using a fixed version of Valgrind.

  Related to #28072.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 50f7214e09

Tree-SHA512: 1030f3709195250350fd9c558420a9b1773fb54fdb323e0452a46eeb69ec6d60b5df50bde617c12d917e16dde07db64dee1b0101ddd4eda6161261fc7f6d4474
2023-07-26 09:36:21 +01:00
fanquake
54fe963a53
Merge bitcoin/bitcoin#28035: test: Ignore UTF-8 errors in assert_debug_log
fa3d72960b lint: Ignore check_fileopens failure on **kwargs (MarcoFalke)
fa6bb85cd2 test: Ignore UTF-8 errors in assert_debug_log (MarcoFalke)
fa63326fbc test: Fix debug_log_size helper (MarcoFalke)

Pull request description:

  Fix two bugs, see commit messages.

ACKs for top commit:
  theStack:
    utACK fa3d72960b

Tree-SHA512: 4a29bdf954bf62bb7676c2a41b03ad017bc86d535b2bd912c96bd41d1621beb06d840b53c211480ad51974e8b293bbae620060d2528d269159f32c0b44e47712
2023-07-26 09:35:51 +01:00
Andrew Chow
32c15237b6
Merge bitcoin/bitcoin#27930: util: Don't derive secure_allocator from std::allocator
07c59eda00 Don't derive secure_allocator from std::allocator (Casey Carter)

Pull request description:

  Giving the C++ Standard Committee control of the public interface of your type means they will break it. C++23 adds a new `allocate_at_least` member to `std::allocator`. Very bad things happen when, say, `std::vector` uses `allocate_at_least` from `secure_allocator`'s base to allocate memory which it then tries to free with `secure_allocator::deallocate`.

  (Discovered by microsoft/STL#3712, which will be reverted by microsoft/STL#3819 before it ships.)

ACKs for top commit:
  jonatack:
    re-ACK 07c59eda00 no change since my previous ACK apart from squashing the commits
  achow101:
    ACK 07c59eda00
  john-moffett:
    ACK 07c59eda00 Reviewed and tested. Performance appears unaffected in my environment.

Tree-SHA512: 23606c40414d325f5605a9244d4dd50907fdf5f2fbf70f336accb3a2cb98baa8acd2972f46eab1b7fdec1d28a843a96b06083cd2d09791cda7c90ee218e5bbd5
2023-07-25 18:54:29 -04:00