Commit graph

6019 commits

Author SHA1 Message Date
fanquake
11b436a66a
Merge bitcoin/bitcoin#29343: test: fix wallet_import_rescan unrounded minimum amount
26ad2aeb29 test: fix wallet_import_rescan unrounded minimum amount (stickies-v)

Pull request description:

  Addresses https://github.com/bitcoin/bitcoin/pull/29283#discussion_r1468842089.

  Fixes a `JSONRPCException: Invalid amount (-3)` exception by ensuring the amount sent to `sendtoaddress` is rounded to 8 decimals.

  See https://cirrus-ci.com/task/5562947183837184?logs=ci#L2559

  Note: since `round` can also round down, `min_amount` is not _exactly_ guaranteed, but this is not a problem for the current usage. I've added a docstring to highlight this.

ACKs for top commit:
  sr-gi:
    ACK [26ad2ae](26ad2aeb29)

Tree-SHA512: 82ce16447f30535f17fa73336f7e4f74639e33215a228294b9b8005b8050a760b90a3726de279cce98c7e439f09104172b74072be3a300dbd461bf0c3f54b954
2024-01-31 09:59:50 +00:00
glozow
78c06a38c4
Merge bitcoin/bitcoin#29067: test: Treat msg_version.relay as unsigned, Remove struct packing in messages.py
55556a64a8 test: Remove struct import from messages.py (MarcoFalke)
fa3fa86dda scripted-diff: test: Use int from_bytes and to_bytes over struct packing (MarcoFalke)
fafc0d68ee test: Use int from_bytes and to_bytes over struct packing (MarcoFalke)
fa3886b7c6 test: Treat msg_version.relay as unsigned (MarcoFalke)

Pull request description:

  `struct` has many issues in messages.py:

  * For unpacking, it requires to specify the length a second time, even when it is already clear from the `f.read(num_bytes)` context.
  * For unpacking, it is designed to support a long format string and returning a tuple of many values. However, except for 3 instances in `messages.py`, usually only a single value is unpacked and all those cases require an `[0]` access.
  * For packing and unpacking of a single value, the format string consists of characters that may be confusing and may need to be looked up in the documentation, as opposed to using easy to understand self-documenting code.

  I presume the above issues lead to accidentally treat `msg_version.relay` as a "signed bool", which is fine, but confusing.

  Fix all issues by using the built-in `int` helpers `to_bytes` and `from_bytes` via a scripted diff.

  Review notes:

  * `struct.unpack` throws an error if the number of bytes passed is incorrect. `int.from_bytes` doesn't know about "missing" bytes and treats an empty byte array as `int(0)`. "Extraneous" bytes should never happen, because all `read` calls are limited in this file. If it is important to keep this error behavior, a helper `int_from_stream(stream, num_bytes, bytes, byteorder, *, **kwargs)` can be added, which checks the number of bytes read from the stream.
  * For `struct.pack` and `int.to_bytes` the error behavior is the same, although the error messages are not identical.

ACKs for top commit:
  stickies-v:
    ACK 55556a64a8
  theStack:
    re-ACK 55556a64a8

Tree-SHA512: 1cef8cdfd763fb424ed4b8be850a834b83fd0ef47fbea626a29784eb4f4832d44e42c4fe05b298b6070a933ef278b0222289a9955a97c86707e091e20bbb247a
2024-01-30 12:00:47 +00:00
Ava Chow
411ba32af2
Merge bitcoin/bitcoin#24748: test/BIP324: functional tests for v2 P2P encryption
bc9283c441 [test] Add functional test to test early key response behaviour in BIP 324 (stratospher)
ffe6a56d75 [test] Check whether v2 TestNode performs downgrading (stratospher)
ba737358a3 [test] Add functional tests to test v2 P2P behaviour (stratospher)
4115cf9956 [test] Ignore BIP324 decoy messages (stratospher)
8c054aa04d [test] Allow inbound and outbound connections supporting v2 P2P protocol (stratospher)
382894c3ac  [test] Reconnect using v1 P2P when v2 P2P terminates due to magic byte mismatch (stratospher)
a94e350ac0 [test] Build v2 P2P messages (stratospher)
bb7bffed79 [test] Use lock for sending P2P messages in test framework (stratospher)
5b91fb14ab [test] Read v2 P2P messages (stratospher)
05bddb20f5 [test] Perform initial v2 handshake (stratospher)
a049d1bd08 [test] Introduce EncryptedP2PState object in P2PConnection (stratospher)
b89fa59e71 [test] Construct class to handle v2 P2P protocol functions (stratospher)
8d6c848a48 [test] Move MAGIC_BYTES to messages.py (stratospher)
595ad4b168 [test/crypto] Add ECDH (stratospher)
4487b80517 [rpc/net] Allow v2 p2p support in addconnection (stratospher)

Pull request description:

  This PR introduces support for v2 P2P encryption(BIP 324) in the existing functional test framework and adds functional tests for the same.

  ### commits overview
  1. introduces a new class `EncryptedP2PState` to store the keys, functions for performing the initial v2 handshake and encryption/decryption.
  3. this class is used by `P2PConnection` in inbound/outbound connections to perform the initial v2 handshake before the v1 version handshake. Only after the initial v2 handshake is performed do application layer P2P messages(version, verack etc..) get exchanged. (in a v2 connection)
      - `v2_state` is the object of class `EncryptedP2PState` in `P2PConnection` used to store its keys, session-id etc.
      - a node [advertising](https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md#advertising-to-support-v2-p2p) support for  v2 P2P is different from a node actually [supporting v2 P2P](https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md#supporting-v2-p2p) (differ when false advertisement of services occur)
          - introduce a boolean variable `supports_v2_p2p` in `P2PConnection` to denote if it supports v2 P2P.
          - introduce a boolean variable `advertises_v2_p2p` to denote whether `P2PConnection` which mimics peer behaviour advertises V2 P2P support. Default option is `False`.
      - In the test framework, you can create Inbound and Outbound connections to `TestNode`
          1. During **Inbound Connections**, `P2PConnection` is the initiator [`TestNode` <--------- `P2PConnection`]
              - Case 1:
                  - if the `TestNode` advertises/signals v2 P2P support (means `self.nodes[i]` set up with `"-v2transport=1"`), different behaviour will be exhibited based on whether:
                      1. `P2PConnection` supports v2 P2P
                      2. `P2PConnection` does not support v2 P2P
                 - In a real world scenario, the initiator node would intrinsically know if they support v2 P2P based on whatever code they choose to run. However, in the test scenario where we mimic peer behaviour, we have no way of knowing if `P2PConnection` should support v2 P2P or not. So `supports_v2_p2p` boolean variable is used as an option to enable support for v2 P2P in `P2PConnection`.
                - Since the `TestNode` advertises v2 P2P support (using "-v2transport=1"), our initiator `P2PConnection` would send:
                  1. (if the `P2PConnection` supports v2 P2P) ellswift + garbage bytes to initiate the connection
                  2. (if the `P2PConnection` does not support v2 P2P) version message to initiate the connection
             - Case 2:
                  - if the `TestNode` doesn't signal v2 P2P support; `P2PConnection` being the initiator would send version message to initiate a connection.
         2. During **Outbound Connections** [TestNode --------> P2PConnection]
             - initiator `TestNode` would send:
                  - (if the `P2PConnection` advertises v2 P2P) ellswift + garbage bytes to initiate the connection
                  - (if the `P2PConnection` advertises v2 P2P) version message to initiate the connection
            - Suppose `P2PConnection` advertises v2 P2P support when it actually doesn't support v2 P2P (false advertisement scenario)
                 - `TestNode` sends ellswift + garbage bytes
                 - `P2PConnection` receives but can't process it and disconnects.
                 - `TestNode` then tries using v1 P2P and sends version message
                 - `P2PConnection` receives/processes this successfully and they communicate on v1 P2P

  4. the encrypted P2P messages follow a different format - 3 byte length + 1-13 byte message_type + payload + 16 byte MAC
  5. includes support for testing decoy messages and v2 connection downgrade(using false advertisement - when a v2 node makes an outbound connection to a node which doesn't support v2 but is advertised as v2 by some malicious
  intermediary)

  ### run the tests
  * functional test - `test/functional/p2p_v2_encrypted.py` `test/functional/p2p_v2_earlykeyresponse.py`

  I'm also super grateful to @ dhruv for his really valuable feedback on this branch.
  Also written a more elaborate explanation here - https://github.com/stratospher/blogosphere/blob/main/integration_test_bip324.md

ACKs for top commit:
  naumenkogs:
    ACK bc9283c441
  mzumsande:
    Code Review ACK bc9283c441
  theStack:
    Code-review ACK bc9283c441
  glozow:
    ACK bc9283c441

Tree-SHA512: 9b54ed27e925e1775e0e0d35e959cdbf2a9a1aab7bcf5d027e66f8b59780bdd0458a7a4311ddc7dd67657a4a2a2cd5034ead75524420d58a83f642a8304c9811
2024-01-29 12:31:31 -05:00
fanquake
759195040a
Merge bitcoin/bitcoin#29329: fuzz: Print coverage summary after run_once
fab97d81ce fuzz: Print coverage summary after run_once (MarcoFalke)

Pull request description:

  This can be used to quickly check the coverage effects of a code change or qa-assets change.

ACKs for top commit:
  dergoegge:
    ACK fab97d81ce

Tree-SHA512: 0ac913c14698f39e76e0e7bf124f182220031796d6443edb34c6e4615e128157cf746da661b216c4640a41964e977249712445ca9c005b1b4a3737adabdb4a7d
2024-01-29 16:24:51 +00:00
MarcoFalke
fab97d81ce
fuzz: Print coverage summary after run_once 2024-01-29 15:24:29 +01:00
stickies-v
26ad2aeb29
test: fix wallet_import_rescan unrounded minimum amount
Fixes a `JSONRPCException: Invalid amount (-3)` exception by
ensuring the amount sent to `sendtoaddress` is rounded to 8
decimals.

See https://cirrus-ci.com/task/5562947183837184?logs=ci#L2559
2024-01-29 11:45:08 +01:00
MarcoFalke
55556a64a8
test: Remove struct import from messages.py 2024-01-29 11:12:15 +01:00
MarcoFalke
fa3fa86dda
scripted-diff: test: Use int from_bytes and to_bytes over struct packing
-BEGIN VERIFY SCRIPT-
 sed -i --regexp-extended 's!struct.unpack\("(|<|>)B", (.*)\)\[0\]!int.from_bytes(\2, "little")!g'               ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.unpack\("<(H|I|Q)", (.*)\)\[0\]!int.from_bytes(\2, "little")!g'              ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.unpack\("<(h|i|q)", (.*)\)\[0\]!int.from_bytes(\2, "little", signed=True)!g' ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.unpack\(">(H|I|Q)", (.*)\)\[0\]!int.from_bytes(\2, "big")!g'                 ./test/functional/test_framework/messages.py

 sed -i --regexp-extended 's!struct.pack\("<?B", (.*)\)!\1.to_bytes(1, "little")!g'             ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.pack\("<I", (.*)\)!\1.to_bytes(4, "little")!g'              ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.pack\("<i", (.*)\)!\1.to_bytes(4, "little", signed=True)!g' ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.pack\("<Q", (.*)\)!\1.to_bytes(8, "little")!g'              ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.pack\("<q", (.*)\)!\1.to_bytes(8, "little", signed=True)!g' ./test/functional/test_framework/messages.py
 sed -i --regexp-extended 's!struct.pack\(">H", (.*)\)!\1.to_bytes(2, "big")!g'                 ./test/functional/test_framework/messages.py
-END VERIFY SCRIPT-
2024-01-29 11:11:54 +01:00
MarcoFalke
fafc0d68ee
test: Use int from_bytes and to_bytes over struct packing
This is done in prepration for the scripted diff, which can not deal
with the 0 literal int.
2024-01-29 11:10:59 +01:00
MarcoFalke
fa3886b7c6
test: Treat msg_version.relay as unsigned
The C++ code treats bool as uint8_t, so the python tests should as well.

This also allows to simplify the code, because converting an empty byte
array to int gives int(0).

>>> int.from_bytes(b'')
0
2024-01-29 11:09:35 +01:00
Ava Chow
ff0eac055f
Merge bitcoin/bitcoin#29283: test: ensure output is large enough to pay for its fees
3bfc5bd36e test: ensure output is large enough to pay for its fees (stickies-v)

Pull request description:

  Fixes a (rare) intermittency issue in wallet_import_rescan.py

  Since we [use](03752444cd/test/functional/wallet_import_rescan.py (L296)) `subtract_fee_from_outputs=[0]` in the `send` command, the output amount must at least be as large as the fee we're paying.

  Example in CI: https://api.cirrus-ci.com/v1/task/6107972259020800/logs/ci.log

  ```
  2024-01-18T22:16:12.383000Z TestFramework (INFO): Test that the mempool is rescanned as well if the rescan parameter is set to true
  2024-01-18T22:16:20.187000Z TestFramework (ERROR): JSONRPC error
  Traceback (most recent call last):
    File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/test_framework.py", line 131, in main
      self.run_test()
    File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/wallet_import_rescan.py", line 292, in run_test
      child = self.nodes[1].send(
    File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/coverage.py", line 50, in __call__
      return_val = self.auth_service_proxy_instance.__call__(*args, **kwargs)
    File "/ci_container_base/ci/scratch/build/bitcoin-x86_64-pc-linux-gnu/test/functional/test_framework/authproxy.py", line 129, in __call__
      raise JSONRPCException(response['error'], status)
  test_framework.authproxy.JSONRPCException: The transaction amount is too small to pay the fee (-4)
  ```

  Can be reproduced locally by forcing usage of the lowest possible value produced by `get_rand_amount()` ([thanks furszy](https://github.com/bitcoin/bitcoin/pull/29283#pullrequestreview-1832956095)):

  <details>
  <summary>git diff on 5f3a0574c4</summary>

  ```diff
  diff --git a/test/functional/wallet_import_rescan.py b/test/functional/wallet_import_rescan.py
  index 7f01d23941..925849d5c0 100755
  --- a/test/functional/wallet_import_rescan.py
  +++ b/test/functional/wallet_import_rescan.py
  @@ -270,7 +270,7 @@ class ImportRescanTest(BitcoinTestFramework):
                   address_type=variant.address_type.value,
               ))
               variant.key = self.nodes[1].dumpprivkey(variant.address["address"])
  -            variant.initial_amount = get_rand_amount() * 2
  +            variant.initial_amount = Decimal(str(round(AMOUNT_DUST, 8))) * 2
               variant.initial_txid = self.nodes[0].sendtoaddress(variant.address["address"], variant.initial_amount)
               variant.confirmation_height = 0
               variant.timestamp = timestamp

  ```
  </details>

ACKs for top commit:
  achow101:
    ACK 3bfc5bd36e
  glozow:
    utACK 3bfc5bd36e, didn't experience this issue but in theory a minimum of `AMOUNT_DUST` could be too low to pay the fees
  furszy:
    utACK 3bfc5bd36

Tree-SHA512: 821ab94a510772e90528b2cef368bbf70309d8fd1dcda53dce75dd1bf91622358e80fea4d9fc68249b9d598892306c66f6c843b4a6855a9f9a9175f7b41109c6
2024-01-26 18:33:46 -05:00
fanquake
cf937b2068
fuzz: also set MSAN_SYMBOLIZER_PATH 2024-01-26 13:56:09 +00:00
fanquake
e3b68b3b83
Merge bitcoin/bitcoin#28875: build: Pass sanitize flags to instrument libsecp256k1 code
cbea49c0d3 build: Pass sanitize flags to instrument `libsecp256k1` code (Hennadii Stepanov)

Pull request description:

  This PR is a revived https://github.com/bitcoin/bitcoin/pull/27991 with an addressed [comment](https://github.com/bitcoin/bitcoin/pull/27991#discussion_r1252148488).

  Fixes https://github.com/bitcoin/bitcoin/issues/27990.

  Might be tested as follows:
  ```
  $ ./autogen.sh && ./configure --enable-fuzz --with-sanitizers=fuzzer CC=clang-13 CXX=clang++-13
  $ make clean > /dev/null && make
  $ objdump --disassemble=secp256k1_xonly_pubkey_serialize src/test/fuzz/fuzz | grep __sanitizer_cov
   1953bd0:e8 bb c6 05 ff       call   9b0290 <__sanitizer_cov_trace_const_cmp8>
   1953d32:e8 69 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
   1953d58:e8 43 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
   1953d82:e8 19 c4 05 ff       call   9b01a0 <__sanitizer_cov_trace_pc_indir>
  ```

ACKs for top commit:
  fanquake:
    ACK cbea49c0d3
  dergoegge:
    reACK cbea49c0d3

Tree-SHA512: 801994e75b711d20eaf0d675f378da07d693f4a7de026efd93860f5f1deabed855a83eca3561725263e4fe605fcc5f91eb73c021ec91c831864e6deb575e3885
2024-01-26 11:31:34 +00:00
stratospher
bc9283c441 [test] Add functional test to test early key response behaviour in BIP 324
- A node initiates a v2 connection by sending 64 bytes ellswift
- In BIP 324 "The responder waits until one byte is received which does not match the
  V1_PREFIX (16 bytes consisting of the network magic followed by "version\x00\x00\x00\x00\x00".)"
- It's possible that the 64 bytes ellswift sent by an initiator starts with a prefix of V1_PREFIX
- Example form of 64 bytes ellswift could be:
	4 bytes network magic + 60 bytes which aren't prefixed with remaining V1_PREFIX
- We test this behaviour:
	- when responder receives 4 byte network magic -> no response received by initiator
	- when first mismatch happens -> response received by initiator
2024-01-25 11:12:15 +05:30
stratospher
ffe6a56d75 [test] Check whether v2 TestNode performs downgrading 2024-01-25 11:10:50 +05:30
stratospher
ba737358a3 [test] Add functional tests to test v2 P2P behaviour 2024-01-25 11:10:50 +05:30
stratospher
4115cf9956 [test] Ignore BIP324 decoy messages
Also allow P2PConnection::send_message() to send decoy messages for
writing tests.
2024-01-25 11:10:50 +05:30
stratospher
8c054aa04d [test] Allow inbound and outbound connections supporting v2 P2P protocol
- Add an optional `supports_v2_p2p` parameter to specify if the inbound
and outbound connections support v2 P2P protocol.
- In the `addconnection_callback` which gets called when creating
outbound connections, call the `addconnection` RPC with v2 P2P protocol
support enabled.
2024-01-25 11:10:50 +05:30
stratospher
382894c3ac [test] Reconnect using v1 P2P when v2 P2P terminates due to magic byte mismatch
- When a v2 TestNode makes an outbound connection to a P2PInterface node
which doesn't support v2 but is advertised as v2 by some malicious
intermediary, the TestNode sends 64 bytes ellswift. The v1 node doesn't
understand this and disconnects. Then the v2 TestNode reconnects by
sending a v1/version message.
2024-01-25 11:10:48 +05:30
stratospher
a94e350ac0 [test] Build v2 P2P messages 2024-01-25 11:09:52 +05:30
stratospher
bb7bffed79 [test] Use lock for sending P2P messages in test framework
Messages are built, encrypted and sent over the socket in v2
connections. If a race condition happens between python's main
thread and p2p thread with both of them trying to send a message,
it's possible that the messages get encrypted with wrong keystream.

Messages are built and sent over the socket in v1 connections.
So there's no problem if messages are sent in the wrong order.

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
Co-authored-by: theStack <sebastian.falbesoner@gmail.com>
2024-01-25 11:09:52 +05:30
stratospher
5b91fb14ab [test] Read v2 P2P messages 2024-01-25 11:09:52 +05:30
stratospher
05bddb20f5 [test] Perform initial v2 handshake 2024-01-25 11:09:52 +05:30
stratospher
a049d1bd08 [test] Introduce EncryptedP2PState object in P2PConnection
Instantiate this object when the connection supports v2 P2P transport
protocol.

- When a P2PConnection is opened, perform initiate_v2_handshake() if the
connection is an initiator. application layer messages are only sent after
the initial v2 handshake is over (for both initiator and responder).
2024-01-25 11:09:50 +05:30
fanquake
ea4ddd8652
Merge bitcoin/bitcoin#29304: fuzz: Exit and log stderr for parse_test_list errors
9d09c873a5 fuzz: Exit and log stderr for parse_test_list errors (dergoegge)

Pull request description:

  We should log all errors that occur when attempting to print the harness list in the fuzz test runner.

ACKs for top commit:
  maflcko:
    lgtm ACK 9d09c873a5

Tree-SHA512: 50471b732c8cbe287dacba14487e7c8a5826f146432d93aa3bb55d063a8ba158d01641d6cb1360241dd4cd54ef5e045b0412f9cc34d06c181134921d1f1ceced
2024-01-24 15:14:16 +00:00
dergoegge
9d09c873a5 fuzz: Exit and log stderr for parse_test_list errors 2024-01-24 11:42:30 +00:00
stratospher
b89fa59e71 [test] Construct class to handle v2 P2P protocol functions
The class `EncryptedP2PState` stores the 4 32-byte keys, session id,
garbage terminators, whether it's an initiator/responder, whether the
initial handshake has been completed etc.. It also contains functions
to perform the v2 handshake and to encrypt/decrypt p2p v2 messages.

- In an inbound connection to TestNode, P2PConnection is the initiator
and `initiate_v2_handshake()`, `complete_handshake()`, `authenticate_handshake()`
are called on it. [ TestNode <----------------- P2PConnection ]

- In an outbound connection from TestNode, P2PConnection is the responder
and `respond_v2_handshake()`, `complete_handshake()`, `authenticate_handshake()`
are called on it. [ TestNode -----------------> P2PConnection ]
2024-01-24 11:51:47 +05:30
Ava Chow
e69796c79c
Merge bitcoin/bitcoin#28560: wallet, rpc: FundTransaction refactor
18ad1b9142 refactor: pass CRecipient to FundTransaction (josibake)
5ad19668db refactor: simplify `CreateRecipients` (josibake)
47353a608d refactor: remove out param from `ParseRecipients` (josibake)
f7384b921c refactor: move parsing to new function (josibake)
6f569ac903 refactor: move normalization to new function (josibake)
435fe5cd96 test: add tests for fundrawtx and sendmany rpcs (josibake)

Pull request description:

  ## Motivation

  The primary motivation for this PR is to enable `FundTransaction` to take a vector of `CRecipient` objects to allow passing BIP352 silent payment addresses to RPCs that use `FundTransaction` (e.g. `send`, `walletcreatefundedpsbt`). To do that, SFFO logic needs to be moved out of `FundTransaction` so the `CRecipient` objects with the correct SFFO information can be created and then passed to `FundTransaction`.

  As a secondary motivation, this PR moves the SFFO stuff closer to the caller, making the code cleaner and easier to understand. This is done by having a single function which parses RPC inputs for SFFO and consistently using the `set<int>` method for communicating SFFO.

  I'm also not convinced we need to pass a full `CMutableTx` object to `FundTransaction`, but I'm leaving that for a follow-up PR/discussion, as its not a blocker for silent payments.

ACKs for top commit:
  S3RK:
    reACK 18ad1b9142
  josibake:
    > According to my `range-diff` nothing changed. reACK [18ad1b9](18ad1b9142)
  achow101:
    ACK 18ad1b9142

Tree-SHA512: d61f017cf7d98489ef216475b68693fd77e7b53a26a6477dcd73e7e5ceff5036b2d21476e377839e710bb73644759d42c4f9f4b14ed96b3e56ed87b07aa6d1a7
2024-01-23 16:40:58 -05:00
Ava Chow
874c8bdb9e
Merge bitcoin/bitcoin#29144: init: handle empty settings file gracefully
e9014042a6 settings: add auto-generated warning msg for editing the file manually (furszy)
966f5de99a init: improve corrupted/empty settings file error msg (furszy)

Pull request description:

  Small and simple issue reported [here](https://community.umbrel.com/t/bitcoin-docker-container-keeps-restarting/2144).

  Improving a confusing situation reported by users who did not understand why a
  settings parsing error occurred when the file was empty and did not know how to solve it.

  Empty setting file could be due (1) corruption or (2) an user manually cleaning up the file content.
  In both scenarios, the 'Unable to parse settings file' error does not help the user move forward.

ACKs for top commit:
  achow101:
    ACK e9014042a6
  hebasto:
    re-ACK e9014042a6.
  ryanofsky:
    Code review ACK e9014042a6. Just whitespace formatting changes and shortening a test string literal since last review
  shaavan:
    Code review ACK e9014042a6

Tree-SHA512: 2910654c6b9e9112de391eedb8e46980280f822fa3059724dd278db7436804dd27fae628d2003f2c6ac1599b07ac5c589af016be693486e949f558515e662bec
2024-01-23 15:14:03 -05:00
Ava Chow
7cb7759b25
Merge bitcoin/bitcoin#29272: wallet: fix coin selection tracing to return -1 when no change pos
d55fdb1a49 Move TRACEx parameters to seperate lines (Richard Myers)
2d58629ee6 wallet: fix coin selection tracing to return -1 when no change pos (Richard Myers)

Pull request description:

  This is a bugfix for from when [optional was introduced](758501b713)  for `change_pos` in the wallet. When optional `change_pos` is unset, we should return -1 and not 0.

  I added two new checks to the `test/functional/interface_usdt_coinselection.py` which adds coverage for the situations when `normal_create_tx_internal` and `aps_create_tx_internal` events occur with no change.

  You can reproduce this bug using the coin-selection-simulation scripts as described in [issue #16](https://github.com/achow101/coin-selection-simulation/issues/16). You can also run the `interface_usdt_coinselection.py` test  without the changes to `wallet/spend.cpp`.

ACKs for top commit:
  0xB10C:
    ACK d55fdb1a49
  achow101:
    ACK d55fdb1a49
  murchandamus:
    ACK d55fdb1a49

Tree-SHA512: 6efac3b756bdf51debbcb759dc3c4b7a4304626bc047b70025cec02f3a04937ace7712e9558ac71e560fd136005a98c518ac5bb4b90c3282d776beccd0de9749
2024-01-23 14:33:43 -05:00
stratospher
8d6c848a48 [test] Move MAGIC_BYTES to messages.py
This avoids circular dependency happening when importing MAGIC_BYTES.
Before,
	p2p.py <--import for EncryptedP2PState-- v2_p2p.py
	  |					    ^
	  |				            |
	  └---------import for MAGIC_BYTES----------┘
Now, MAGIC_BYTES are kept separately in messages.py

Co-authored-by: Martin Zumsande <mzumsande@gmail.com>
2024-01-23 22:04:55 +05:30
stratospher
595ad4b168 [test/crypto] Add ECDH
Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2024-01-23 22:04:55 +05:30
furszy
e9014042a6
settings: add auto-generated warning msg for editing the file manually
Hopefully, refraining users from modifying the file unless they are
certain about the potential consequences.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
furszy
966f5de99a
init: improve corrupted/empty settings file error msg
The preceding "Unable to parse settings file" message lacked
the necessary detail and guidance for users on what steps to
take next in order to resolve the startup error.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2024-01-22 10:50:03 -03:00
Richard Myers
2d58629ee6
wallet: fix coin selection tracing to return -1 when no change pos 2024-01-20 14:56:41 +01:00
stickies-v
3bfc5bd36e
test: ensure output is large enough to pay for its fees
Fixes a (rare) intermittency issue in wallet_import_rescan.

Since we use `subtract_fee_from_outputs=[0]` in the `send` command,
the output amount must at least be as large as the fee we're paying.
2024-01-19 14:19:25 +00:00
josibake
435fe5cd96
test: add tests for fundrawtx and sendmany rpcs
If the serialized transaction passed to `fundrawtransaction` contains
duplicates, they will be deserialized and added to the transaction. Add
a test to ensure this behavior is not changed during the refactor.

A user can pass any number of duplicated and unrelated addresses as an
SFFO argument to `sendmany` and the RPC will not throw an error (note,
all the rest of the RPCs which take SFFO as an argument will error if
the user passes duplicates or specifies outputs not present in the
transaction). Add a test to ensure this behavior is not changed during
the refactor.
2024-01-19 15:04:56 +01:00
Hennadii Stepanov
cbea49c0d3
build: Pass sanitize flags to instrument libsecp256k1 code
Also a new UBSan suppression has been added.
2024-01-19 10:08:41 +00:00
Ava Chow
5f3a0574c4
Merge bitcoin/bitcoin#29262: rpc: Fix race in loadtxoutset
5555d8db33 test: Use blocks_path where possible (MarcoFalke)
fa9108941f rpc: Fix race in loadtxoutset (MarcoFalke)

Pull request description:

  The tip may have advanced, also if it did not, there is no reason to
  have two variables point to the same block.

  Fixes https://github.com/bitcoin/bitcoin/pull/27596#discussion_r1344694600

ACKs for top commit:
  achow101:
    ACK 5555d8db33
  pablomartin4btc:
    ACK 5555d8db33
  BrandonOdiwuor:
    Code Review ACK 5555d8db33

Tree-SHA512: 23a82924a915b61bb1adab8ad20ec8914139c8ee647817af34ca27ee310a2e45833d8b285503e0feebe63e4667193d6d98cfcbbc1509bf40712225e04dd19e8b
2024-01-18 13:17:35 -05:00
Ava Chow
ac3901ebd0
Merge bitcoin/bitcoin#29228: test: Remove all-lint.py script
fa2b95cf3f test: Remove all-lint.py script (MarcoFalke)
fadb06c361 doc: move-only lint docs to one place (MarcoFalke)

Pull request description:

  Seems confusing to have a test runner that calls another runner (`all-lint.py`), which calls a subset of the lint tests.

  Fix that by just calling this subset of lint tests in the test runner directly, and remove the now unused `all-lint.py`.

  To run all lint checks locally, refer to the documentation: https://github.com/bitcoin/bitcoin/blob/master/test/lint/README.md#running-locally

ACKs for top commit:
  kevkevinpal:
    ACK [fa2b95c](fa2b95cf3f)
  achow101:
    ACK fa2b95cf3f
  TheCharlatan:
    ACK fa2b95cf3f
  pablomartin4btc:
    tACK fa2b95cf3f
  brunoerg:
    utACK fa2b95cf3f

Tree-SHA512: 43fac9acb4e9a6744d695dd49c7202e19ab4bf480f4cccff768647d0157a065f40e6ad70b9f6a65ba42048cc5fa9834365aa8e7aa0ed64c09e0cd4eb8dccb831
2024-01-18 13:02:15 -05:00
fanquake
03c5b0064d
Merge bitcoin/bitcoin#29085: refactor: C++20: Use std::rotl
6044628543 crypto, hash: replace custom rotl32 with std::rotl (Fabian Jahr)

Pull request description:

  While exploring some C++20 changes and checking against our code I found this potential improvement:

  1. We can replace our custom implementation of `rotl32` in crypto/chacha20 with `std::rotl` from the [new `bit` header](https://en.cppreference.com/w/cpp/header/bit).

ACKs for top commit:
  fanquake:
    ACK 6044628543

Tree-SHA512: db55b366f20fca2ef62e5f10a838f8a709d531678c35c1dba20898754029c788a2fd47995208ed6d187cf814109a7ca397bc2c301504500aee79da04c95d6895
2024-01-18 09:40:44 +00:00
MarcoFalke
5555d8db33
test: Use blocks_path where possible 2024-01-17 16:48:47 +01:00
Ava Chow
8106b268cd
Merge bitcoin/bitcoin#29239: rpc: Make v2transport default for addnode RPC when enabled
3ba815b42d Make v2transport default for addnode RPC when enabled (Pieter Wuille)

Pull request description:

  Since #29058, several types of manually configured connections will attempt v2 connections when `-v2transport` is enabled, except for the `addnode` RPC, as that one has an explicit argument to enable or disable.

  Make the default for that RPC match the `-v2transport` setting so the behavior matches that of other manual connections from a user perspective.

ACKs for top commit:
  achow101:
    ACK 3ba815b42d
  kristapsk:
    ACK 3ba815b42d
  theStack:
    Code-review ACK 3ba815b42d

Tree-SHA512: 31ef48cf1e533abb17866020378c004df929e626074dc98b3229fb60a66de58435e95c8fda8d1b463e1208aa39d1f42d239818e7e58595a3738089920598befc
2024-01-16 16:50:03 -05:00
Ava Chow
27d935f58b
Merge bitcoin/bitcoin#29179: test: wallet rescan with reorged parent + IsFromMe child in mempool
df30247705 [test] import descriptor wallet with reorged parent + IsFromMe child in mempool (glozow)
c3d02be536 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool (Gloria Zhao)

Pull request description:

  Originally motivated by #29019, which reverts back to having `requestMempoolTransactions` emit `transactionAddedToMempool` in `mapTx` default order instead of `GetSortedDepthAndScore` order.

  It's important that these notifications happen in topological order, otherwise the wallet rescan may miss transactions that belong to it. Notably, checking whether a transaction `IsFromMe` requires knowing its inputs, which may be from a mempool parent.

  When using `mapTx` order, a parent may come later than its child if it was added from a block disconnected in a reorg.

  This PR adds a test for this case.

ACKs for top commit:
  achow101:
    ACK df30247705
  furszy:
    Code review ACK df30247705, nits can be disregarded.

Tree-SHA512: 2f1d9ef92313228adbbef94e634e5f7a9ec6e6a2c88e16aa343bdc95ffc9b9f9c82a569b412c9a3841db9d789e52f9283e8b9385731668d59355903e26e58a5d
2024-01-16 12:49:09 -05:00
MarcoFalke
fa2b95cf3f
test: Remove all-lint.py script 2024-01-16 10:54:42 +01:00
MarcoFalke
fadb06c361
doc: move-only lint docs to one place
Can be reviewed with --color-moved=dimmed-zebra
2024-01-16 10:54:14 +01:00
glozow
df30247705 [test] import descriptor wallet with reorged parent + IsFromMe child in mempool
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.

Co-authored-by: furszy <matiasfurszyfer@protonmail.com>
2024-01-12 14:51:16 +00:00
Pieter Wuille
3ba815b42d Make v2transport default for addnode RPC when enabled 2024-01-12 09:31:31 -05:00
glozow
cd603361a4
Merge bitcoin/bitcoin#28885: mempool / rpc: followup to getprioritisedtransactions and delete a mapDeltas entry when delta==0
0eebd6fe7d test: Assert that a new tx with a delta of 0 is never added (kevkevin)
cfdbcd19b3 rpc: exposing modified_fee in getprioritisedtransactions (kevkevin)
252a86729a rpc: renaming txid -> transactionid (kevkevin)
2fca6c2dd0 rpc: changed prioritisation-map -> "" (kevkevin)
3a118e19e1 test: Directly constructing 2 entry map for getprioritisedtransactions (kevkevin)

Pull request description:

  In this PR I am addressing some comments in https://github.com/bitcoin/bitcoin/pull/27501 as a followup.
  - changed `prioritisation-map` in the `RPCResult` to `""`
  - Directly constructing 2 entry map for getprioritisedtransactions in functional tests
  - renamed `txid` to `transactionid` in `RPCResult` to be more consistent with naming elsewhere
  - exposed the `modified_fee` field instead of having it be a useless arg
  - Created a new test that asserts when `prioritisedtransaction` is called with a fee_delta of 0 it is not added to mempool

ACKs for top commit:
  glozow:
    reACK 0eebd6fe7d, only change is the doc suggestion

Tree-SHA512: e99056e37a8b1cfc511d87c83edba7c928b50d2cd6c2fd7c038976779850677ad37fddeb2b983e8bc007ca8567eb21ebb78d7eae9b773657c2b297299993ec05
2024-01-12 12:03:52 +00:00
Gloria Zhao
c3d02be536 [test] rescan legacy wallet with reorged parent + IsFromMe child in mempool
Test that wallet rescans process transactions topologically, even if a
parent's entry into the mempool is later than that of its child.
This behavior is important because IsFromMe requires the ability to look
up a transaction's inputs.
2024-01-12 09:54:57 +00:00