Commit graph

36850 commits

Author SHA1 Message Date
fanquake
4398cfb22b
Merge bitcoin/bitcoin#27173: valgrind: remove libsecp256k1 suppression
29b62c01c8 valgrind: remove libsecp256k1 suppression (fanquake)

Pull request description:

  I am no-longer been able to recreate this issue, atleast after the most recent libsecp256k1 changes. Can someone else confirm?

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 29b62c01c8
  sipa:
    utACK 29b62c01c8

Tree-SHA512: e61e5b88ac2af3c779f23d999938bec4497f6433a029c07dd57a9481fe9cc104d8d8f63586910b29f41e66bbf0ed094bc7539c0df84754a1783c4b1b15af072e
2023-02-28 16:56:29 +00:00
glozow
a8080c0def
Merge bitcoin/bitcoin#23897: refactor: Move calculation logic out from CheckSequenceLocksAtTip()
75db62ba4c refactor: Move calculation logic out from `CheckSequenceLocksAtTip()` (Hennadii Stepanov)
3bc434f459 refactor: Add `CalculateLockPointsAtTip()` function (Hennadii Stepanov)

Pull request description:

  This PR is follow up for bitcoin/bitcoin#22677 and bitcoin/bitcoin#23683.

  On master (013daed9ac) it is not obvious that `CheckSequenceLocksAtTip()` function can modify its `LockPoints* lp` parameter which leads to https://github.com/bitcoin/bitcoin/pull/22677#discussion_r762040101.

  This PR:
  - separates the lockpoint calculate logic from `CheckSequenceLocksAtTip()` function into a new `CalculateLockPointsAtTip()` one
  - cleans up the `CheckSequenceLocksAtTip()` function interface
  - makes code easier to reason about (hopefully)

ACKs for top commit:
  achow101:
    ACK 75db62ba4c
  stickies-v:
    re-ACK 75db62b

Tree-SHA512: 072c3fd9cd1e1b0e0bfc8960a67b01c80a9f16d6778f374b6944ade03a020415ce8b8ab2593b0f5e787059c8cf90af798290b4c826785d41955092f6e12e7486
2023-02-28 16:53:02 +00:00
Andrew Chow
8303f11e10
Merge bitcoin/bitcoin#27170: refactor: Stop using gArgs global in system.cpp
9a9d5da11f refactor: Stop using gArgs global in system.cpp (Ryan Ofsky)
b20b34f5b3 refactor: Use new GetConfigFilePath function (Ryan Ofsky)

Pull request description:

  Most of the code in `util/system.cpp` that was hardcoded to use the global `ArgsManager` instance `gArgs` has been changed to stop using it (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a few hardcoded references to `gArgs` remain. This commit removes the last ones so these functions aren't reading or writing global state.

  Noticed these `gArgs` references while reviewing #27073

ACKs for top commit:
  achow101:
    ACK 9a9d5da11f
  stickies-v:
    ACK 9a9d5da11
  willcl-ark:
    tACK 9a9d5da11

Tree-SHA512: 2c74b0d5fc83e9ed2ec6562eb26ec735512f75db8876a11a5d5f04e6cdbe0cd8beec19894091aa2cbf29319194d2429ccbf8036f5520ecc394f6fe89a0079a7b
2023-02-28 11:01:21 -05:00
fanquake
9384536eb3
Merge bitcoin/bitcoin#27174: ci: bump lint task to bookworm for git v2.38
a984beeca1 [ci] change lint to bookworm for git v2.38 (glozow)

Pull request description:

  Since 5497c14, verify-commits.py uses `git merge-tree` which requires git v2.38 or later. Fix the lint jobs on master (e.g. https://cirrus-ci.com/task/4971007513985024).

ACKs for top commit:
  achow101:
    ACK a984beeca1
  hebasto:
    re-ACK a984beeca1

Tree-SHA512: dd50598aefb6f9c86cf221dea27fcc521e335cb182f7a3abcb3215d3991794354085be3e07c133ab74ca8b957e3a6acc43722899165957b3d898867d6253ebdc
2023-02-28 15:44:19 +00:00
fanquake
c37fb251f5
Merge bitcoin/bitcoin#27176: docs: GetDataDirNet and GetDataDirBase don't create datadir
fb0dbe9423 docs: GetDataDirNet and GetDataDirBase don't create datadir (stickies-v)

Pull request description:

  Since #27073, the behaviour of `GetDataDir()` [changed](https://github.com/bitcoin/bitcoin/pull/27073/files#diff-19427b0dd1a791adc728c82e88f267751ba4f1c751e19262cac03cccd2822216L435-L443) to only return the datadir path, but not create it if non-existent. This also changed the behaviour of `GetDataDirNet()` and `GetDataDirBase()` but the docs do not yet reflect that.

ACKs for top commit:
  TheCharlatan:
    ACK fb0dbe9423
  theStack:
    ACK fb0dbe9423
  willcl-ark:
    ACK fb0dbe942

Tree-SHA512: 3f10f4871df59882f3649c6d3b2362cae2f8a01ad0bd0c636c5608b0d177d279a2e8712930b819d6d3912e91fa6447b9e54507c33d8afe427f7f39002b013bfb
2023-02-28 15:34:23 +00:00
Andrew Chow
bb136aaf2c
Merge bitcoin/bitcoin#26533: prune: scan and unlink already pruned block files on startup
3141eab9c6 test: add functional test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
e252909e56 test: add unit test for ScanAndUnlinkAlreadyPrunedFiles (Andrew Toth)
77557dda4a prune: scan and unlink already pruned block files on startup (Andrew Toth)

Pull request description:

  There are a few cases where we can mark a block and undo file as pruned in our block index, but not actually remove the files from disk.
  1. If we call `FindFilesToPrune` or `FindFilesToPruneManual` and crash before `UnlinkPrunedFiles`.
  2. If on Windows there is an open file handle to the file somewhere else when calling `fs::remove` in `UnlinkPrunedFiles` (https://en.cppreference.com/w/cpp/filesystem/remove, https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-deletefilew#remarks). This could be from another process, or if we are calling `ReadBlockFromDisk`/`ReadRawBlockFromDisk` without having a lock on `cs_main` (which has been allowed since ccd8ef65f9).

  This PR mitigates this by scanning all pruned block files on startup after `LoadBlockIndexDB` and unlinking them again.

ACKs for top commit:
  achow101:
    ACK 3141eab9c6
  pablomartin4btc:
    re-ACK with added functional test 3141eab9c6.
  furszy:
    Code review ACK 3141eab9
  theStack:
    Code-review ACK 3141eab9c6

Tree-SHA512: 6c73bc57838ad1b7e5d441af3c4d6bf4c61c4382e2b86485e57fbb74a61240710c0ceeceb8b4834e610ecfa3175c6955c81ea4b2285fee11ca6383f472979d8d
2023-02-28 09:54:10 -05:00
glozow
a984beeca1
[ci] change lint to bookworm for git v2.38
Since 5497c14, verify-commits.py requires git merge-tree which is only
available in git v2.38 or later.
2023-02-28 14:42:48 +00:00
stickies-v
fb0dbe9423
docs: GetDataDirNet and GetDataDirBase don't create datadir
Since #27073, the behaviour of GetDataDir changed to only return
the datadir path, but not create it. This also changed the behaviour
of GetDataDirNet and GetDataDirBase but the docs do not yet reflect
that.
2023-02-28 12:52:42 +00:00
fanquake
29b62c01c8
valgrind: remove libsecp256k1 suppression 2023-02-28 10:45:57 +00:00
fanquake
519ec2650e
Merge bitcoin/bitcoin#27157: init: Return ChainstateLoadStatus::INTERRUPTED when verification was interrupted.
c5825e14f8 doc: add explanation for fail_on_insufficient_dbcache (Ryan Ofsky)
7dff7da4f5 init: Return more fitting ChainStateLoadStatus if verification was interrupted (Martin Zumsande)

Pull request description:

  This addresses two outstanding comments by ryanofsky from #25574:
  * return `ChainstateLoadStatus::INTERRUPTED` instead of `ChainstateLoadStatus::SUCCESS`  if verification was stopped by an interrupt. This would coincide with straightforward expectation, and it avoids a misleading [log entry](c5825e14f8/src/init.cpp (L1526)) in `init` for the block index load time (because that would include the verificiation, which didn't complete). It shouldn't affect node behavior otherwise because the shutdown signal would be caught in init anyway. In test, this would lead to an assert ([link](c5825e14f8/src/test/util/setup_common.cpp (L230))), which also makes more sense because benign interrupts are not expected there during init.
  This can be tested by setting a large value for `-checkblocks`, interrupting the node during block verification and observing the log.
   https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110050930
  * add documentation for `require_full_verification` https://github.com/bitcoin/bitcoin/pull/25574#discussion_r1110031541

ACKs for top commit:
  MarcoFalke:
    thanks lgtm ACK c5825e14f8

Tree-SHA512: ca1c71a1b046d30083337dd9ef6d52e66fa1ac8c4ecd807716e4aa6a894179a81df41caee916fa30997fd6e0b284412a3c8f2919d19c29d826fb580ffb89fd73
2023-02-28 10:40:24 +00:00
fanquake
e60a58f191
Merge bitcoin/bitcoin#27118: depends: harden libevent
ff4a73aea2 depends: use FORTIFY_SOURCE=3 with libevent (fanquake)

Pull request description:

  Use `FORTIFY_SOURCE=3` when building libevent in depends. I've upstreamed a change to switch libevent from using =2 to =3 as well: https://github.com/libevent/libevent/pull/1418.

  Solves half of #27038, by giving us some fortified funcs in `bitcoin-cli`.

ACKs for top commit:
  TheCharlatan:
    ACK ff4a73aea2

Tree-SHA512: eaf692ec92b288f0cb524c011fc81529f58efa4c43d418a7b3ae7108eba2bccba708a81a28ac6d063267be80ca615637c6e3fccc02497d7367af2eaae0e8d812
2023-02-28 10:28:34 +00:00
Ryan Ofsky
9a9d5da11f refactor: Stop using gArgs global in system.cpp
Most of the code in util/system.cpp that was hardcoded to use the global
ArgsManager instance `gArgs` has been changed to work with explicit ArgsManager
instances (for example in https://github.com/bitcoin/bitcoin/pull/20092). But a
few hardcoded references to `gArgs` remain. This commit removes the last ones
so these functions aren't reading or writing global state.
2023-02-27 14:21:13 -05:00
Ryan Ofsky
b20b34f5b3 refactor: Use new GetConfigFilePath function
New function was introduced by willcl-ark <will@256k1.dev> in commit
56e370fbb9 from
https://github.com/bitcoin/bitcoin/pull/27073 and removes some duplicate code.
2023-02-27 14:14:58 -05:00
Andrew Chow
710cab1d43
Merge bitcoin/bitcoin#26032: wallet: skip R-value signature grinding for external signers
807de2cebd wallet: skip R-value grinding for external signers (Sjors Provoost)
72b763e452 wallet: annotate bools in descriptor SPKM FillPSBT() (Sjors Provoost)

Pull request description:

  When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.

  In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.

  Suggested testing:
  1. On master, launch with `-signet` and create an external signer wallet using e.g. a Trezor and HWI, see [guide](https://github.com/bitcoin/bitcoin/blob/master/doc/external-signer.md#example-usage) (with the GUI it should "just work" once you have the HWI path configured).
  2. Create a few addresses and fund them from the faucet: https://signet.bc-2.jp/ (wait for confirmation)
  3. Create another address, and now send the entire wallet to it, set the fee to 1 sat/byte
  4. Most likely this transaction never gets broadcast and you won't see it on the [signet explorer](https://explorer.bc-2.jp)

  5. With this PR, try again.
  6. Check the explorer and inspect the transaction. Each input witness starts with either `30440220` (R has 32 bytes) or `30440221` (R has 33 bytes). See this explainer for [DER encoding](https://bitcoin.stackexchange.com/questions/92680/what-are-the-der-signature-and-sec-format).

  Fixes #26030

ACKs for top commit:
  S3RK:
    ACK 807de2cebd
  achow101:
    ACK 807de2cebd
  furszy:
    ACK 807de2ce
  ishaanam:
    utACK 807de2cebd

Tree-SHA512: 64f626a3030ef0ab1e43af86d8fba113151512561baf425e6e5182af53df3a64fa9e85c7f67bf4ed15b5ad6e5d5afc7fbba8b6e1f3bad388e48db51cb9446074
2023-02-27 12:37:46 -05:00
fanquake
82793f1984
Merge bitcoin/bitcoin#27146: Fix various libbitcoinkernel DLL build problems
5da7c0b3e3 build: allow libitcoinkernel dll builds now that exports are fixed (Cory Fields)
130490aef9 build: always build bitcoin-chainstate against static libbitcoinkernel (Cory Fields)
545a74ef32 build: fix bitcoin-chainstate when libbitcoinkernel is static (Cory Fields)
9c253d2398 build: don't define DLL_EXPORT for windows (Cory Fields)

Pull request description:

  Fixes #25008.
  Fixes #19772.

  1. Fixup the build defines so that exports are clean.
  2. Work around a libtool issue wrt dependency calculation
  3. Simplify everything by only ever building in-tree bitcoin-chainstate against a static libbitcoinkernel
  4. Remove Windows-only hack that disabled dll creation

ACKs for top commit:
  TheCharlatan:
    ACK 5da7c0b3e3

Tree-SHA512: 61bab457e13842946387240da703d313509af30d4ca3371a19a26a5ef1716e4d7107b09567323041b549ab1fc97a064aa1d6992406936ab9c491a616bc7f4e7f
2023-02-27 14:41:47 +00:00
fanquake
a2877f7ad3
Merge bitcoin/bitcoin#25227: Handle invalid hex encoding in ParseHex
faab273e06 util: Return empty vector on invalid hex encoding (MarcoFalke)
fa3549a77b test: Add hex parse unit tests (MarcoFalke)

Pull request description:

  Seems a bit confusing to happily accept random bytes and pretend they are hex encoded strings.

ACKs for top commit:
  stickies-v:
    re-ACK faab273e06

Tree-SHA512: a808135f744f50aece03d4bf5a71481c7bdca1fcdd0d5b113abdb0c8b382bf81cafee6d17c239041fb49b59f4e19970f24a475378e7f711c3a47d6438de2bdab
2023-02-27 14:27:50 +00:00
glozow
873dcc1910
Merge bitcoin/bitcoin#27058: contrib: Improve verify-commits.py to work with maintainers leaving
14fac808bd verify-commits: Mention git v2.38.0 requirement (Andrew Chow)
bb86887527 verify-commits: Skip checks for commits older than trusted roots (Andrew Chow)
5497c14830 verify-commits: Use merge-tree in clean merge check (Andrew Chow)
76923bfa09 verify-commits: Remove all allowed commit exceptions (Andrew Chow)
53b07b2b47 verify-commits: Move trusted-keys valid sig check into verify-commits itself (Andrew Chow)

Pull request description:

  Currently the `verify-commits.py` script does not work well with maintainers giving up their commit access. If a key is removed from `trusted-keys`, any commits it signed previously will fail to verify, however keys cannot be kept in the list as it would allow that person to continue to push new commits. Furthermore, the `trusted-keys` used depends on the working tree which `verify-commits.py` itself may be modifying. When the script is run, the `trusted-keys` may be the one that is intended to be used, but the script may change the tree to a different commit with a different `trusted-keys` and use that instead!

  To resolve these issues, I've updated `verify-commits.py` to load the `trusted-keys` file and check the keys itself rather than delegating that to `gpg.sh` (which previously read in `trusted-keys`). This avoids the issue with the tree changing.

  I've also updated the script so that it stops modifying the tree. It would do this for the clean merge check where it would checkout each individual commit and attempt to reapply the merges, and then checking out the commit given as a cli arg. `git merge-tree` lets us do basically that but without modifying the tree. It will give us the object id for the resulting tree which we can compare against the object id of the tree in the merge commit in question. This also appears to be quite a bit faster.

  Lastly I've removed all of the exception commits in `allow-revsig-commits`, `allow-incorrect-sha512-commits`, and `allow-unclean-merge-commits` since all of these predate the commits in `trusted-git-root` and `trusted-sha512-root`. I've also updated the script to skip verification of commits that predate `trusted-git-root`, and skip sha512 verification for those that predate `trusted-sha512-root`.

ACKs for top commit:
  Sjors:
    ACK 14fac808bd
  glozow:
    Concept ACK 14fac808bd

Tree-SHA512: f9b0c6e1f1aecb169cdd6c833b8871b15e31c2374dc589858df0523659b294220d327481cc36dd0f92e9040d868eee6a8a68502f3163e05fa751f9fc2fa8832a
2023-02-27 13:17:48 +00:00
MarcoFalke
faab273e06
util: Return empty vector on invalid hex encoding 2023-02-27 13:39:55 +01:00
MarcoFalke
fa3549a77b
test: Add hex parse unit tests 2023-02-27 13:35:51 +01:00
Ryan Ofsky
c5825e14f8 doc: add explanation for fail_on_insufficient_dbcache 2023-02-24 15:11:27 -05:00
Martin Zumsande
7dff7da4f5 init: Return more fitting ChainStateLoadStatus if verification was interrupted
This also avoids a misleading block index loadtime log entry in init.

Co-authored-by: Ryan Ofsky <ryan@ofsky.org>
2023-02-24 15:09:24 -05:00
glozow
be2e748f37
Merge bitcoin/bitcoin#27154: doc: mention sanitizer suppressions in developer docs
84ca5b349e doc: mention sanitizer suppressions in developer docs (fanquake)

Pull request description:

  Should be enough to close #17834.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 84ca5b349e

Tree-SHA512: 233c688a3cef1006c9a00f7b7a52fd6ee0ec150367e5e56904b6f1bbdca21b9217c69f8fcf653a4943613d12c3178a39f761b25eb24fc1954a563cfb1f832f5e
2023-02-24 13:58:04 +00:00
fanquake
84ca5b349e
doc: mention sanitizer suppressions in developer docs
Should be enough to close #17834.
2023-02-24 12:16:37 +00:00
glozow
6758bd7de7
Merge bitcoin/bitcoin#27135: Remove MarcoFalke fingerprint, update trusted-git-root
fab17f08e2 Revert "[contrib] verify-commits: Add MarcoFalke fingerprint" (MarcoFalke)

Pull request description:

  This reverts commit fa24329334.

  The commit may be signed by my key, but I haven't checked it. Also, I haven't checked the new `contrib/verify-commits/trusted-git-root`.

ACKs for top commit:
  achow101:
    ACK fab17f08e2
  glozow:
    ACK fab17f08e2

Tree-SHA512: 485fb302f7e42704412afffd6c09a031f63df18f259b27282b8373d5bf95b0ec72426cec476d88bf23e793a6e1dae4c1df2059645961806e34b50448ebf1862a
2023-02-24 10:27:25 +00:00
fanquake
cb25c21921
Merge bitcoin/bitcoin#27151: util: Remove duplicate include
e8462690a9 util: Remove duplicate include (Andrew Chow)

Pull request description:

  Duplicate `#include <utility>` is upsetting the linter.

ACKs for top commit:
  davidgumberg:
    ACK e8462690a9
  theStack:
    ACK e8462690a9
  john-moffett:
    ACK e8462690a9

Tree-SHA512: 9e45d8f6a2dd5efcb8eb1c3c440d94b16490dbd63255784cb39863767fa07227e06da112a150ef337ef89e2e305b60b00d5b1c12ff7e1e9c02f6648ed97fac8c
2023-02-24 09:03:02 +00:00
Andrew Chow
e8462690a9 util: Remove duplicate include
Duplicate `#include <utility>` is upsetting the linter.
2023-02-23 17:58:40 -05:00
Andrew Chow
1258af40c0
Merge bitcoin/bitcoin#27073: Convert ArgsManager::GetDataDir to a read-only function
64c105442c util: make GetDataDir read-only & create datadir.. (willcl-ark)
56e370fbb9 util: add ArgsManager datadir helper functions (willcl-ark)

Pull request description:

  Fixes #20070

  Currently `ArgsManager::GetDataDir()` ensures it will always return a datadir by creating one if necessary. The function is shared between `bitcoind` `bitcoin-qt` and `bitcoin-cli` which results in the undesirable behaviour described in #20070.

  This PR splits out the part of the function which creates directories and adds it as a standalone function, only called as part of `bitcoind` and `bitcoin-qt` init, but not `bitcoin-cli`.

  `ReadConfigFiles`' behavior is changed to use the absolute path of the config file in error and warning messages instead of a relative path.

  This was inadvertantly the form being tested [here](73966f75f6/test/functional/feature_config_args.py (L287)), whilst we were _not_ testing that a relative path was returned by the message even though we passed a relative path in as argument.

ACKs for top commit:
  achow101:
    ACK 64c105442c
  hebasto:
    re-ACK 64c105442c, only comments have been adjusted as requsted since my previous [review](https://github.com/bitcoin/bitcoin/pull/27073#pullrequestreview-1307435890).
  TheCharlatan:
    Re-ACK 64c105442c
  ryanofsky:
    Code review ACK 64c105442c. Only comment changes since last review

Tree-SHA512: b129501346071ad62551c9714492b21536d0558a94117d97218e255ef4e948d00df899a4bc2788faea27d3b1f20fc6136ef9d03e6a08498d926d9ad8688d6c96
2023-02-23 16:41:14 -05:00
Andrew Chow
c033720b2b
Merge bitcoin/bitcoin#16195: util: Use void* throughout support/lockedpool.h
f36d1d5b89 Use void* throughout support/lockedpool.h (Jeffrey Czyz)

Pull request description:

  Replace uses of char* with void* in Arena's member variables. Instead,
  cast to char* where needed in the implementation.

  Certain compiler environments disallow std::hash<char*> specializations
  to prevent hashing the pointer's value instead of the string contents.
  Thus, compilation fails when std::unordered_map is keyed by char*.

  Explicitly using void* is a workaround in such environments. For
  consistency, void* is used throughout all member variables similarly to
  the public interface.

  Changes to this code are covered by src/test/allocator_tests.cpp.

ACKs for top commit:
  achow101:
    ACK f36d1d5b89
  theStack:
    Code-review ACK f36d1d5b89
  jonatack:
    ACK f36d1d5b89 review, debug build, unit tests, checked clang 15 raises "error: arithmetic on a pointer to void"  without the conversions here from the generic void* pointer back to char*

Tree-SHA512: f9074e6d29ef78c795a512a6e00e9b591e2ff34165d09b73eae9eef25098c59e543c194346fcd4e83185a39c430d43744b6f7f9d1728a132843c67bd27ea5189
2023-02-23 15:44:42 -05:00
Andrew Chow
b7702bd546
Merge bitcoin/bitcoin#25943: rpc: Add a parameter to sendrawtransaction which sets a maximum value for unspendable outputs.
7013da07fb Add release note for PR#25943 (David Gumberg)
04f270b435 Add test for unspendable transactions and parameter 'maxburnamount' to sendrawtransaction. (David Gumberg)

Pull request description:

  This PR adds a user configurable, zero by default parameter — `maxburnamount` — to `sendrawtransaction`. This PR makes bitcoin core reject transactions that contain unspendable outputs which exceed `maxburnamount`.  closes #25899.

  As a result of this PR, `sendrawtransaction` will by default block 3 kinds of transactions:

  1. Those that begin with `OP_RETURN` - (datacarriers)
  2. Those whose lengths exceed the script limit.
  3. Those that contain invalid opcodes.

  The user is able to configure a `maxburnamount` that will override this check and allow a user to send a potentially unspendable output into the mempool.

  I see two legitimate use cases for this override:
  1. Users that deliberately use `OP_RETURN` for datacarrier transactions that embed data into the blockchain.
  2.  Users that refuse to update, or are unable to update their bitcoin core client would be able to make use of new opcodes that their client doesn't know about.

ACKs for top commit:
  glozow:
    reACK 7013da07fb
  achow101:
    re-ACK 7013da07fb

Tree-SHA512: f786a796fb71a587d30313c96717fdf47e1106ab4ee0c16d713695e6c31ed6f6732dff6cbc91ca9841d66232166eb058f96028028e75c1507324426309ee4525
2023-02-23 13:57:38 -05:00
Sjors Provoost
807de2cebd
wallet: skip R-value grinding for external signers
When producing a dummy signature for the purpose of estimating the transaction fee, do not assume an external signer performs R-value grinding on the signature.

In particular, this avoids a scenario where the fee rate is 1 sat / vbyte and a transaction with a 72 byte signature is not accepted into our mempool.

This commit also  drops the nullptr default for CCoinControl arguments for functions that it touches. This is because having a boolean argument right next to an optional pointer is error prone.

Co-Authored-By: S3RK <1466284+S3RK@users.noreply.github.com>
2023-02-23 18:30:12 +01:00
Sjors Provoost
72b763e452
wallet: annotate bools in descriptor SPKM FillPSBT() 2023-02-23 11:46:29 +01:00
fanquake
32f9ce0f52
Merge bitcoin/bitcoin#27124: docs: add ramdisk guide for running tests on OSX
2f84ad7b9e docs: add ramdisk guide for running tests on OSX (Matthew Zipkin)

Pull request description:

  Using a ramdisk on OSX sped up the test suite by about 5x (using default `jobs=4`) on my M1 macbook pro running macOS Monterey 12.3.1. This PR adds the relevant OSX commands following the Linux directions.

  Default:
  ```
  8204 s (accumulated)
  Runtime: 2104 s
  ```

  following commands from the PR:
  ```
  1606 s (accumulated)
  Runtime: 421 s
  ```

  ramdisk + `jobs=32`:
  ```
  2090 s (accumulated)
  Runtime: 85 s
  ```

ACKs for top commit:
  jonatack:
    ACK 2f84ad7b9e
  willcl-ark:
    ACK 2f84ad7b9e
  brunoerg:
    utACK 2f84ad7b9e

Tree-SHA512: 37a9903c8ac2cbfaa91e7e73fc96ef65042ff4b15763d452af7b8615255adf03429ad01cf85265a99dd569290c1d69c05a393d616868c05c190b60b053820786
2023-02-23 10:04:37 +00:00
willcl-ark
64c105442c
util: make GetDataDir read-only & create datadir..
.. only in bitcoind and bitcoin-qt

This changes behaviour of GetConfigFilePath which now always returns the
absolute path of the provided -conf argument.
2023-02-23 08:38:35 +00:00
willcl-ark
56e370fbb9
util: add ArgsManager datadir helper functions
* Add ArgsManager::EnsureDataDir()
  Creates data directory if it doesn't exist
* Add ArgsManager::GetConfigFilePath()
  Return config file path (read-only)
2023-02-23 08:37:27 +00:00
Cory Fields
5da7c0b3e3 build: allow libitcoinkernel dll builds now that exports are fixed
Symbol visibility issues are not actually fixed yet because we have not yet
defined an api and exported symbols, but everything is now in place for that.
2023-02-22 21:23:10 +00:00
Cory Fields
130490aef9 build: always build bitcoin-chainstate against static libbitcoinkernel
Building binaries against our uninstalled shared libs is impractical. Instead,
to test them, we'll need to work on a runtime shared-lib execution harness.
2023-02-22 21:23:10 +00:00
Cory Fields
545a74ef32 build: fix bitcoin-chainstate when libbitcoinkernel is static
Libtool is unable to calculate dependencies correctly so give it some help.
2023-02-22 20:56:11 +00:00
Andrew Chow
832fa2d238
Merge bitcoin/bitcoin#25574: validation: Improve error handling when VerifyDB dosn't finish successfully
0af16e7134 doc: add release note for #25574 (Martin Zumsande)
57ef2a4812 validation: report if pruning prevents completion of verification (Martin Zumsande)
0c7785bb25 init, validation: Improve handling if VerifyDB() fails due to insufficient dbcache (Martin Zumsande)
d6f781f1cf validation: return VerifyDBResult::INTERRUPTED if verification was interrupted (Martin Zumsande)
6360b5302d validation: Change return value of VerifyDB to enum type (Martin Zumsande)

Pull request description:

  `VerifyDB()` can fail to complete due to insufficient dbcache at the level 3 checks. This PR improves the error handling in this case in the following ways:
  - The rpc `-verifychain` now returns false if the check can't be completed due to insufficient cache
  - During init, we only log a warning if the default values for `-checkblocks` and `-checklevel` are taken and the check doesn't complete. However, if the user actively specifies one of these args, we return with an InitError if we can't complete the check.

  This PR also changes `-verifychain` RPC to return `false` if the verification didn't finish due to missing block data (pruning) or due to being interrupted by the node being shutdown.

  Previously, this PR also included a fix for a possible assert during verification - this was done in #27009 (now merged).

ACKs for top commit:
  achow101:
    ACK 0af16e7134
  ryanofsky:
    Code review ACK 0af16e7134. Only small suggested changes since the last review, like renaming some of the enum values. I did leave more suggestions, but they are not very important and could be followups
  john-moffett:
    ACK 0af16e7134
  MarcoFalke:
    lgtm re-ACK 0af16e7134 🎚

Tree-SHA512: 84b4f767cf9bfbafef362312757c9bf765b41ae3977f4ece840e40c52a2266b1457832df0cdf70440be0aac2168d9b58fc817238630b0b6812f3836ca950bc0e
2023-02-22 14:19:44 -05:00
Cory Fields
9c253d2398 build: don't define DLL_EXPORT for windows
This fixes libbitcoinkernel dll linking.
2023-02-22 19:04:21 +00:00
fanquake
9f6ef0c156
Merge bitcoin/bitcoin#27143: test: Replace 0xC0 constant
c3b4b5a142 test: Replace 0xC0 constant (roconnor-blockstream)

Pull request description:

  Instead it should be the named constant `LEAF_VERSION_TAPSCRIPT`.

ACKs for top commit:
  instagibbs:
    ACK c3b4b5a142
  theStack:
    ACK c3b4b5a142

Tree-SHA512: c00be584ea2d0e7c01bf5620da0da1f37e5b5298ef95df48d91d137c8c542f5d91be158d45392cf2ba8874bf27bd12924e2eed395773b49d091e3028de3356a2
2023-02-22 18:11:26 +00:00
Matthew Zipkin
2f84ad7b9e
docs: add ramdisk guide for running tests on OSX 2023-02-22 13:04:23 -05:00
Andrew Chow
5e55534586
Merge bitcoin/bitcoin#27068: wallet: SecureString to allow null characters
4bbf5ddd44 Detailed error message for passphrases with null chars (John Moffett)
b4bdabc223 doc: Release notes for 27068 (John Moffett)
4b1205ba37 Test case for passphrases with null characters (John Moffett)
00a0861181 Pass all characters to SecureString including nulls (John Moffett)

Pull request description:

  `SecureString` is a `std::string` specialization with a secure allocator. However, in practice it's treated like a C- string (no explicit length and null-terminated). This can cause unexpected and potentially insecure behavior. For instance, if a user enters a passphrase with embedded null characters (which is possible through Qt and the JSON-RPC), it will ignore any characters after the first null, potentially giving the user a false sense of security.

  Instead of assigning to `SecureString` via `std::string::c_str()`, assign it via a `std::string_view` of the original. This explicitly captures the size and still doesn't make any extraneous copies in memory.

  Note to reviewers, the following all compile identically in recent `GCC` (x86-64 and ARM64) with `-O2` (and `-std=c++17`):

  ```C++
  std::string orig_string;
  std::cin >> orig_string;
  SecureString s;
  s.reserve(100);
  // The following all compile identically
  s = orig_string;
  s = std::string_view{orig_string};
  s.assign(std::string_view{orig_string});
  s.assign(orig_string.data(), orig_string.size());
  ```

  So it's largely a matter of preference. However, one thing to keep in mind is that we want to avoid making unnecessary copies of any sensitive data in memory.

  Something like `SecureString s{orig_string};` is still invalid and probably unwanted in our case, since it'd get treated as a short string and optimized away from the secure allocator. I presume that's the reason for the `reserve()` calls.

  Fixes #27067.

ACKs for top commit:
  achow101:
    re-ACK 4bbf5ddd44
  stickies-v:
    re-ACK [4bbf5dd](4bbf5ddd44)
  furszy:
    utACK 4bbf5ddd

Tree-SHA512: 47a96905a82ca674b18076a20a388123beedf70e9de73e42574ea68afbb434734e56021835dd9b148cdbf61709926b487cc95e9021d9bc534a7c93b3e143d2f7
2023-02-22 13:02:16 -05:00
fanquake
174f022f68
Merge bitcoin/bitcoin#27144: kernel: add missing include
49d01f32c9 kernel: add missing include (Cory Fields)

Pull request description:

  This syncs the cs_main definition/declaration.

  Noticed when experimenting with the external visibility of `cs_main`.

  Specifically, this is needed for the following to work as intended:
  ```c++
  __attribute__ ((visibility ("default"))) extern RecursiveMutex cs_main;
  ```

ACKs for top commit:
  fanquake:
    ACK 49d01f32c9

Tree-SHA512: ea0dbcf81959566f949d76c7dcd1e33de53e613519500c863bfb0ac8209665b1c12cff2daa7890d03b76debc4d046339ee7b3231adb71b128e9d5a8fa3132b6c
2023-02-22 18:01:48 +00:00
fanquake
30874a7cc9
Merge bitcoin/bitcoin#26837: I2P network optimizations
3c1de032de i2p: use consistent number of tunnels with i2pd and Java I2P (Vasil Dimov)
801b405f85 i2p: lower the number of tunnels for transient sessions (Vasil Dimov)
b906b64eb7 i2p: reuse created I2P sessions if not used (Vasil Dimov)

Pull request description:

  * Reuse an I2P transient session instead of discarding it if we failed to connect to the desired peer. This means we never used the generated address (destination), whose creation is not cheap. This does not mean that we will use the same address for more than one peer.
  * Lower the number of tunnels for transient sessions.
  * Explicitly specify the number of tunnels for persistent sessions instead of relying on the defaults which differ between I2P routers. This way we get consistent behavior with all routers.

  Alleviates: https://github.com/bitcoin/bitcoin/issues/26754

  (I have not tested this with i2pd, yet)

ACKs for top commit:
  jonatack:
    ACK 3c1de032de
  mzumsande:
    Light ACK 3c1de032de

Tree-SHA512: 477b4b9a5755e6a9a46bc0f7b268fa419dff4414e25445c750ae913f7552d9e2313f2aca4e3b70067b8390c2d0c2d68ec459f331765e939fc84139e454031cd4
2023-02-22 17:58:41 +00:00
fanquake
c6e65a102c
Merge bitcoin/bitcoin#27137: test: Raise PRNG seed log to INFO
4d84eaec82 Raise PRNG seed log to INFO. (roconnor-blockstream)

Pull request description:

  Some build infrastructure, such as Nix, will delete failed builds by default, keeping only the log (stdout/stderr) of the failed build.

  For flaky tests, it would be very helpful to have the PRNG seed in the default log in order to redo the failed test.

  By simply raising the PRNG seed logging to INFO, we can, by default, record the seed in the log of every build.

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 4d84eaec82
  theStack:
    ACK 4d84eaec82

Tree-SHA512: 3ccb4a4e7639a3babc3b2a6456a6d0bffc090da34e4545b317f7bfbed4e9950d1b38ea5b2a90c37ccb49b3454bdeff03a6aaf86770b9c4dd14b26320aba50b94
2023-02-22 17:51:39 +00:00
fanquake
63893d5eab
Merge bitcoin/bitcoin#26595: wallet: be able to specify a wallet name and passphrase to migratewallet
9486509be6 wallet, rpc: Update migratewallet help text for encrypted wallets (Andrew Chow)
aaf02b5721 tests: Tests for migrating wallets by name, and providing passphrase (Andrew Chow)
7fd125b27d wallet: Be able to unlock the wallet for migration (Andrew Chow)
6bdbc5ff59 rpc: Allow users to specify wallet name for migratewallet (Andrew Chow)
dbfa345403 wallet: Allow MigrateLegacyToDescriptor to take a wallet name (Andrew Chow)

Pull request description:

  `migratewallet` currently operates on wallets that are already loaded, however this is not necessarily required, and in the future, not possible once the legacy wallet is removed. So we need to also be able to give the wallet name to migrate.

  Additionally, the passphrase is required when migrating a wallet. Since a wallet may not be loaded when we migrate, and as we currently unload wallets when migrating, we need the passphrase to be given to `migratewallet` in order to migrate encrypted wallets.

  Fixes #27048

ACKs for top commit:
  john-moffett:
    reACK 9486509be6
  pinheadmz:
    ACK 9486509be6
  furszy:
    ACK 9486509b

Tree-SHA512: 35e2ba69a148e129a41e20d7fb99c4cab7947b1b7e7c362f4fd06ff8ac6e79e476e07207e063ba5b80e1a33e2343f4b4f1d72d7930ce80c34571c130d2f5cff4
2023-02-22 17:48:23 +00:00
Cory Fields
49d01f32c9 kernel: add missing include
This syncs the cs_main definition/declaration.

Noticed when experimenting with the external visibility of cs_main.
2023-02-22 15:46:21 +00:00
roconnor-blockstream
c3b4b5a142 test: Replace 0xC0 constant
Instead it should be the named constant `LEAF_VERSION_TAPSCRIPT`.
2023-02-22 10:26:07 -05:00
fanquake
8b4dc94734
Merge bitcoin/bitcoin#27117: fuzz: avoid redundant dup key checks when creating Miniscript nodes
c1b7bd047f fuzz: avoid redundant dup key checks when creating Miniscript nodes (Antoine Poinsot)

Pull request description:

  I thought i had done that already in #24149, but it must have slipped through the rebase. It's a 2x speed improvement against the existing corpora and will probably be much more as we extend them with larger nodes.

ACKs for top commit:
  sipa:
    ACK c1b7bd047f

Tree-SHA512: 9e6ceb6254183964b6c5538e21ba6321df95a68acb343a15a6ecfef5c51a1980d2627df5aeef9aef1db41656e18cc4f3bc96e6f24314d12fa60368b04a350001
2023-02-22 09:37:07 +00:00
fanquake
0c579203d2
Merge bitcoin/bitcoin#25867: lint: enable E722 do not use bare except
61bb4e783b lint: enable E722 do not use bare except (Leonardo Lazzaro)

Pull request description:

  Improve test code and enable E722 lint check.

   If you want to catch all exceptions that signal program errors, use except Exception: (bare except is equivalent to except BaseException:).

  Reference: https://peps.python.org/pep-0008/#programming-recommendations

ACKs for top commit:
  MarcoFalke:
    lgtm ACK 61bb4e783b

Tree-SHA512: c7497769d5745fa02c78a20f4a0e555d8d3996d64af6faf1ce28e22ac1d8be415b98e967294679007b7bda2a9fd04031a9d140b24201e00257ceadeb5c5d7665
2023-02-22 09:28:09 +00:00