Commit graph

487 commits

Author SHA1 Message Date
Andrew Chow
65d7c31b3f
Merge bitcoin/bitcoin#25789: test: clean and extend availablecoins_tests coverage
9622fe64b8 test: move coins result test to wallet_tests.cpp (furszy)
f69347d058 test: extend and simplify availablecoins_tests (furszy)
212ccdf2c2 wallet: AvailableCoins, add arg to include/skip locked coins (furszy)

Pull request description:

  Negative PR with extended test coverage :).

  1) Cleaned duplicated code and added coverage for the 'AvailableCoins' incremental result.

  2) The class `AvailableCoinsTestingSetup` inside `availablecoins_tests.cpp` is a plain copy
  of `ListCoinsTestingSetup` that is inside `wallet_tests.cpp`.

      So, deleted the file and moved the `BasicOutputTypesTest` test case to `wallet_tests.cpp`.

  3) Added arg to include/skip locked coins from the `AvailableCoins` result. This is needed for point (1) as otherwise the wallet will spend the coins that we recently created due its closeness to the recipient amount.
  Note: this last point comes from #25659 where I'm using the same functionality to clean/speedup another flow as well.

ACKs for top commit:
  achow101:
    ACK 9622fe64b8
  theStack:
    ACK 9622fe64b8
  aureleoules:
    reACK 9622fe64b8, nice cleanup!

Tree-SHA512: 1ed9133120bfe8815455d1ad317bb0ff96e11a0cc34ee8098716ab9b001749168fa649212b2fa14b330c1686cb1f29039ff1f88ae306db68881b0428c038f388
2023-01-03 12:52:40 -05:00
MarcoFalke
b9028b2e26
Merge bitcoin/bitcoin#26481: bench: Suppress output when running with -sanity-check option
f1e89597c8 test: Drop no longer required bench output redirection (Hennadii Stepanov)
4dbcdf26a3 bench: Suppress output when running with `-sanity-check` option (Hennadii Stepanov)

Pull request description:

  This change allows to simplify CI tests, and makes it easier to integrate the `bench_bitcoin` binary into CMake custom [targets](https://cmake.org/cmake/help/latest/command/add_custom_target.html) or [commands](https://cmake.org/cmake/help/latest/command/add_custom_command.html), as `COMMAND` does not support output redirection.

ACKs for top commit:
  aureleoules:
    tACK f1e89597c8. Ran as expected and is more practical than using an output redirection.

Tree-SHA512: 29086d428cccedcfd031c0b4514213cbc1670e35f955e8fd35cee212bc6f9616cf9f20d0cb984495390c4ae2c50788ace616aea907d44e0d6a905b9dda1685d8
2022-12-29 11:42:03 +01:00
furszy
9622fe64b8
test: move coins result test to wallet_tests.cpp
The class `AvailableCoinsTestingSetup` inside `availablecoins_tests.cpp` is a plain copy
of `ListCoinsTestingSetup` that is inside wallet_tests.cpp.
2022-12-14 11:16:01 -03:00
furszy
ee7a984f85
refactor: unify test/util/wallet.h with wallet/test/util.h
files share the same purpose, and we shouldn't have wallet code
inside the test directory.

This later is needed to use wallet util functions in the bench
and test binaries without be forced to duplicate them.
2022-11-21 17:30:00 -03:00
MacroFake
fa4ec1be51
test: Split overly large util_tests.cpp file 2022-11-14 14:22:43 +01:00
Hennadii Stepanov
f1e89597c8
test: Drop no longer required bench output redirection 2022-11-10 16:26:44 +00:00
Andrew Chow
fabc031048
Merge bitcoin/bitcoin#26158: bench: add "priority level" to the benchmark framework
3e9d0bea8d build: only run high priority benchmarks in 'make check' (furszy)
466b54bd4a bench: surround main() execution with try/catch (furszy)
3da7cd2a76 bench: explicitly make all current benchmarks "high" priority (furszy)
05b8c76232 bench: add "priority level" to the benchmark framework (furszy)
f1593780b8 bench: place benchmark implementation inside benchmark namespace (furszy)

Pull request description:

  This is from today's meeting, a simple "priority level" for the benchmark framework.

  Will allow us to run certain benchmarks while skip non-prioritized ones in `make check`.

  By default, `bench_bitcoin` will run all the benchmarks. `make check`will only run the high priority ones,
  and have marked all the existent benchmarks as "high priority" to retain the current behavior.

  Could test it by modifying any benchmark priority to something different from "high", and
  run `bench_bitcoin -priority-level=high` and/or `bench_bitcoin -priority-level=medium,low`
  (the first command will skip the modified bench while the second one will include it).

  Note: the second commit could be avoided by having a default arg value for the priority
  level but.. an explicit set in every `BENCHMARK` macro call makes it less error-prone.

ACKs for top commit:
  kouloumos:
    re-ACK 3e9d0bea8d
  achow101:
    ACK 3e9d0bea8d
  theStack:
    re-ACK 3e9d0bea8d
  stickies-v:
    re-ACK 3e9d0bea8d

Tree-SHA512: ece59bf424c5fc1db335f84caa507476fb8ad8c6151880f1f8289562e17023aae5b5e7de03e8cbba6337bf09215f9be331e9ef51c791c43bce43f7446813b054
2022-10-20 11:05:03 -04:00
furszy
3e9d0bea8d
build: only run high priority benchmarks in 'make check' 2022-10-20 10:21:05 -03:00
Gleb Naumenko
b99ee9d22d test: Add unit tests for reconciliation negotiation 2022-10-17 12:35:44 +03:00
glozow
cc12b8947b
Merge bitcoin/bitcoin#24858: incorrect blk file size calculation during reindex results in recoverable blk file corruption
bcb0cacac2 reindex, log, test: fixes #21379 (mruddy)

Pull request description:

  Fixes #21379.

  The blocks/blk?????.dat files are mutated and become increasingly malformed, or corrupt, as a result of running the re-indexing process.
  The mutations occur after the re-indexing process has finished, as new blocks are appended, but are a result of a re-indexing process miscalculation that lingers in the block manager's `m_blockfile_info` `nSize` data until node restart.
  These additions to the blk files are non-fatal, but also not desirable.
  That is, this is a form of data corruption that the reading code is lenient enough to process (it skips the extra bytes), but it adds some scary looking log messages as it encounters them.

  The summary of the problem is that the re-index process double counts the size of the serialization header (magic message start bytes [4 bytes] + length [4 bytes] = 8 bytes) while calculating the blk data file size (both values already account for the serialization header's size, hence why it is over accounted).

  This bug manifests itself in a few different ways, after re-indexing, when a new block from a peer is processed:
  1. If the new block will not fit into the last blk file processed while re-indexing, while remaining under the 128MiB limit, then the blk file is flushed to disk and truncated to a size that is 8 greater than it should be. The truncation adds zero bytes (see `FlatFileSeq::Flush` and `TruncateFile`).
  1. If the last blk file processed while re-indexing has logical space for the new block under the 128 MiB limit:
      1. If the blk file was not already large enough to hold the new block, then the zeros are, in effect, added by `fseek` when the file is opened for writing. Eight zero bytes are added to the end of the last blk file just before the new block is written. This happens because the write offset is 8 too great due to the miscalculation. The result is 8 zero bytes between the end of the last block and the beginning of the next block's magic + length + block.
      1. If the blk file was already large enough to hold the new block, then the current existing file contents remain in the 8 byte gap between the end of the last block and the beginning of the next block's magic + length + block. Commonly, when this occcurs, it is due to the blk file containing blocks that are not connected to the block tree during reindex and are thus left behind by the reindex process and later overwritten when new blocks are added. The orphaned blocks can be valid blocks, but due to the nature of concurrent block download, the parent may not have been retrieved and written by the time the node was previously shutdown.

ACKs for top commit:
  LarryRuane:
    tested code-review ACK bcb0cacac2
  ryanofsky:
    Code review ACK bcb0cacac2. This is a disturbing bug with an easy fix which seems well-worth merging.
  mzumsande:
    ACK bcb0cacac2 (reviewed code and did some testing, I agree that it fixes the bug).
  w0xlt:
    tACK bcb0cacac2

Tree-SHA512: acc97927ea712916506772550451136b0f1e5404e92df24cc05e405bb09eb6fe7c3011af3dd34a7723c3db17fda657ae85fa314387e43833791e9169c0febe51
2022-10-12 14:13:54 -04:00
fanquake
079cf88c0d
refactor: move Boost datetime usage to wallet
This means we don't need datetime in a --disable-wallet build, and it
isn't included in the kernel.
2022-10-01 11:41:53 +01:00
fanquake
12de8f6262
build: extract $(BOOST_CPPFLAGS) from $(BITCOIN_INCLUDES)
This leaves $(BITCOIN_INCLUDES) as internal dependencies, and gives
finer control over Boost includes.
2022-09-13 17:15:17 +01:00
furszy
e06676377d
wallet: coverage for loading an unknown descriptor
Previously, this was crashing the wallet.
2022-09-09 15:35:31 -03:00
Suhas Daftuar
0b6aa826b5 Add unit test for HeadersSyncState 2022-08-29 08:10:35 -04:00
Pieter Wuille
84852bb6bb Add bitdeque, an std::deque<bool> analogue that does bit packing. 2022-08-23 11:34:10 -04:00
Andrew Chow
c3b099ace0 wallet, tests: Test bumpfee's max input weight calculation 2022-08-19 14:37:36 -04:00
Andrew Chow
243d7bde78
Merge bitcoin/bitcoin#25798: build: fix cleanup of test logs
5474f5c356 build: fix cleanup of test logs (fanquake)

Pull request description:

  `make clean` currently looks for `test_name.cpp.log`, when it should be `test_name.log`, meaning .log files are left after running `make clean`.

  Also fixes #21705. `make distcheck` seems to work fine after the logs files are properly cleaned up:
  ```bash
  ./autogen.sh && ./configure && make distcheck -j9
  ....
  make[1]: Leaving directory '/home/ubuntu/bitcoin/bitcoin-23.99.0/_build/sub'
  if test -d "bitcoin-23.99.0"; then find "bitcoin-23.99.0" -type d ! -perm -200 -exec chmod u+w {} ';' && rm -rf "bitcoin-23.99.0" || { sleep 5 && rm -rf "bitcoin-23.99.0"; }; else :; fi
  =================================================
  bitcoin-23.99.0 archives ready for distribution:
  bitcoin-23.99.0.tar.gz
  =================================================
  ```

  Probably broken in #19385 / #24715.

  Guix Build (x86_64):
  ```bash
  c33306c2ae55bc0e037a1050bd0813fd7654f21fefd0e7df089a541118b629bc  guix-build-5474f5c356c5/output/aarch64-linux-gnu/SHA256SUMS.part
  f3cf5b8366e27155f3a369ab0d017074912506c43b4010054a72e5c3ae8cab2c  guix-build-5474f5c356c5/output/aarch64-linux-gnu/bitcoin-5474f5c356c5-aarch64-linux-gnu-debug.tar.gz
  48f618300f63533c50c31395959737103bb0279972b989cc5417adbf338a5c9f  guix-build-5474f5c356c5/output/aarch64-linux-gnu/bitcoin-5474f5c356c5-aarch64-linux-gnu.tar.gz
  6b3e0ceefc84dfad48aec3a9ea8ae98a427775242370234709605855f593dc88  guix-build-5474f5c356c5/output/arm-linux-gnueabihf/SHA256SUMS.part
  36a38f4d0d2d0fee51256ee9c610cde014055cf18b5b852c8b3235ef218b461e  guix-build-5474f5c356c5/output/arm-linux-gnueabihf/bitcoin-5474f5c356c5-arm-linux-gnueabihf-debug.tar.gz
  3acb46a786323068fe34eaa8b2f7bff428d35367677e456b78fc00db09da8adf  guix-build-5474f5c356c5/output/arm-linux-gnueabihf/bitcoin-5474f5c356c5-arm-linux-gnueabihf.tar.gz
  aaeb126cca3cbbb1d21be2cbb36b3e16a8c110199a5e61172ee55c9c913fb717  guix-build-5474f5c356c5/output/arm64-apple-darwin/SHA256SUMS.part
  0135d47c34dfbfbcd68181292fc810a1be4efaa0ff9fab05f9f33415167fc82c  guix-build-5474f5c356c5/output/arm64-apple-darwin/bitcoin-5474f5c356c5-arm64-apple-darwin-unsigned.dmg
  261ae9f70238ecbd8fedf9b3e02f34601abe42c4a23b25d0571d04d0881fdaba  guix-build-5474f5c356c5/output/arm64-apple-darwin/bitcoin-5474f5c356c5-arm64-apple-darwin-unsigned.tar.gz
  de716e0f425bae38cfda2690e86dbfa6831b6d37dd8a7638ca14334eecbddf99  guix-build-5474f5c356c5/output/arm64-apple-darwin/bitcoin-5474f5c356c5-arm64-apple-darwin.tar.gz
  2e2e1d5cd009ec5c69ce22124afe90ff8c15e9cb546df818b2305ff96efa9c81  guix-build-5474f5c356c5/output/dist-archive/bitcoin-5474f5c356c5.tar.gz
  85dd41584a2c7715b16508dff0f51bbb20b3891a1901a1781ecc37bc1cfd97c8  guix-build-5474f5c356c5/output/powerpc64-linux-gnu/SHA256SUMS.part
  49ae951e9acb34fede0ead4d53679ddc041bfb6d60646efaca2a99cf87972be9  guix-build-5474f5c356c5/output/powerpc64-linux-gnu/bitcoin-5474f5c356c5-powerpc64-linux-gnu-debug.tar.gz
  2efcb3fbb6702bff30bbf05d83b9849f390c67a1c363c883d71f365f4bee7ef6  guix-build-5474f5c356c5/output/powerpc64-linux-gnu/bitcoin-5474f5c356c5-powerpc64-linux-gnu.tar.gz
  cea9e8e2499932ef6e20bba7a6b3408e2cca6fcd1875c1890293dd745add6942  guix-build-5474f5c356c5/output/powerpc64le-linux-gnu/SHA256SUMS.part
  7616659242a1f15b7d6f9f54382dfb52d0bbdca701e1fb3d48fbe7bb590e6213  guix-build-5474f5c356c5/output/powerpc64le-linux-gnu/bitcoin-5474f5c356c5-powerpc64le-linux-gnu-debug.tar.gz
  d9d4eeaf6f9d1272000aa598c8461afc330a8e65f263d45b0eab222f8ddfec71  guix-build-5474f5c356c5/output/powerpc64le-linux-gnu/bitcoin-5474f5c356c5-powerpc64le-linux-gnu.tar.gz
  f83ee11c35001d34f9fba7d1e9e201a4cc6fa5e44bd51e13dabffcb35324348c  guix-build-5474f5c356c5/output/riscv64-linux-gnu/SHA256SUMS.part
  c441e56f23f224122ed064cdb57364fb129f5c9d50c5e8173952ce649b46bdb8  guix-build-5474f5c356c5/output/riscv64-linux-gnu/bitcoin-5474f5c356c5-riscv64-linux-gnu-debug.tar.gz
  319db8af21a4d3c7bbdf54c315ad70bacf7fba1f2559408188d90c9ba60ca63c  guix-build-5474f5c356c5/output/riscv64-linux-gnu/bitcoin-5474f5c356c5-riscv64-linux-gnu.tar.gz
  9ac536a04d7e500f87b1f7dfb60e1e84cde2c192d3dffb89c308b864e9b9d583  guix-build-5474f5c356c5/output/x86_64-apple-darwin/SHA256SUMS.part
  19e70f13fb4bf82375f7ca882a23e831f84729278e643cf5911182bdababa893  guix-build-5474f5c356c5/output/x86_64-apple-darwin/bitcoin-5474f5c356c5-x86_64-apple-darwin-unsigned.dmg
  673a20e9457af3d0a89ea6721f84c6136132d3fbe469b7371bf14ce688b567d0  guix-build-5474f5c356c5/output/x86_64-apple-darwin/bitcoin-5474f5c356c5-x86_64-apple-darwin-unsigned.tar.gz
  74162d53faffc4372ae4587cde395fe078b5c440c43c5a4ad8b8b890e9546255  guix-build-5474f5c356c5/output/x86_64-apple-darwin/bitcoin-5474f5c356c5-x86_64-apple-darwin.tar.gz
  d751d50427d7abcbe9ac1daf087dc3addb7e4e6b90bb4c3ef6c31f8e54cac25e  guix-build-5474f5c356c5/output/x86_64-linux-gnu/SHA256SUMS.part
  49951b53172d4fe193d7ffc15b04a4bc058a3209653982b65912f2b221305dd4  guix-build-5474f5c356c5/output/x86_64-linux-gnu/bitcoin-5474f5c356c5-x86_64-linux-gnu-debug.tar.gz
  93d4c4b07202a9171a72b6887e3931e53b3a8c22433b26521e2cb2a0c942f52a  guix-build-5474f5c356c5/output/x86_64-linux-gnu/bitcoin-5474f5c356c5-x86_64-linux-gnu.tar.gz
  b4b79e3578b6ffbb3075aacff969fb201b386e35a2d9d597047b61ff14bdfbfb  guix-build-5474f5c356c5/output/x86_64-w64-mingw32/SHA256SUMS.part
  8260f5c9a38cc577dff2143f00c465d117aa9835b3633365289d6807d7e46e7c  guix-build-5474f5c356c5/output/x86_64-w64-mingw32/bitcoin-5474f5c356c5-win64-debug.zip
  b20fbf02ddf617b86e5d5f88346ed97f9d169cd56904684ba3ca5f03ea85f008  guix-build-5474f5c356c5/output/x86_64-w64-mingw32/bitcoin-5474f5c356c5-win64-setup-unsigned.exe
  16e9d1b817a832bfb0be2b9065440245a5d04b3aae8e34ff0f43f20c5dd7047f  guix-build-5474f5c356c5/output/x86_64-w64-mingw32/bitcoin-5474f5c356c5-win64-unsigned.tar.gz
  da766e257d10cf8890a530babc6100039c69ae7ed8e4f969eb612b4a411dd88f  guix-build-5474f5c356c5/output/x86_64-w64-mingw32/bitcoin-5474f5c356c5-win64.zip
  ```

ACKs for top commit:
  adam2k:
    ACK 5474f5c356
  achow101:
    ACK 5474f5c356
  jarolrod:
    ACK 5474f5c356

Tree-SHA512: d5595c96d0386b5ee9b98d9499770a00d8b751500020bf92f75c88e688640a50cfa5ebe7c26dea8cc5473b16a6adb83ec39891dd18d32ef59df5cf48d7091d6c
2022-08-11 11:57:12 -04:00
fanquake
d8b26abed9
build: move raw rule into Makefile.am
The same rule is used by the tests and benchmarks to generate headers,
and currently causes #25501. Just deduplicate the code into Makefile.am.
2022-08-10 13:36:33 +01:00
fanquake
5474f5c356
build: fix cleanup of test logs
make clean currently looks for test.cpp.log, when it should be test.log.
2022-08-07 12:13:51 +01:00
Ryan Ofsky
a23cca56c0 refactor: Replace BResult with util::Result
Rename `BResult` class to `util::Result` and update the class interface to be
more compatible with `std::optional` and with a full-featured result class
implemented in https://github.com/bitcoin/bitcoin/pull/25665. Motivation for
this change is to update existing `BResult` usages now so they don't have to
change later when more features are added in #25665.

This change makes the following improvements originally implemented in #25665:

- More explicit API. Drops potentially misleading `BResult` constructor that
  treats any bilingual string argument as an error. Adds `util::Error`
  constructor so it is never ambiguous when a result is being assigned an error
  or non-error value.

- Better type compatibility. Supports `util::Result<bilingual_str>` return
  values to hold translated messages which are not errors.

- More standard and consistent API. `util::Result` supports most of the same
  operators and methods as `std::optional`. `BResult` had a less familiar
  interface with `HasRes`/`GetObj`/`ReleaseObj` methods. The Result/Res/Obj
  naming was also not internally consistent.

- Better code organization. Puts `src/util/` code in the `util::` namespace so
  naming reflects code organization and it is obvious where the class is coming
  from. Drops "B" from name because it is undocumented what it stands for
  (bilingual?)

- Has unit tests.
2022-08-03 07:33:01 -04:00
Andrew Chow
1abbae65eb
Merge bitcoin/bitcoin#24584: wallet: avoid mixing different OutputTypes during coin selection
71d1d13627 test: add unit test for AvailableCoins (josibake)
da03cb41a4 test: functional test for new coin selection logic (josibake)
438e04845b wallet: run coin selection by `OutputType` (josibake)
77b0707206 refactor: use CoinsResult struct in SelectCoins (josibake)
2e67291ca3 refactor: store by OutputType in CoinsResult (josibake)

Pull request description:

  # Concept

  Following https://github.com/bitcoin/bitcoin/pull/23789, Bitcoin Core wallet will now generate a change address that matches the payment address type. This improves privacy by not revealing which of the outputs is the change at the time of the transaction in scenarios where the input address types differ from the payment address type. However, information about the change can be leaked in a later transaction. This proposal attempts to address that concern.

  ## Leaking information in a later transaction

  Consider the following scenario:

  ![mix input types(1)](https://user-images.githubusercontent.com/7444140/158597086-788339b0-c698-4b60-bd45-9ede4cd3a483.png)

  1. Alice has a wallet with bech32 type UTXOs and pays Bob, who gives her a P2SH address
  2. Alice's wallet generates a P2SH change output, preserving her privacy in `txid: a`
  3. Alice then pays Carol, who gives her a bech32 address
  4. Alice's wallet combines the P2SH UTXO with a bech32 UTXO and `txid: b` has two bech32 outputs

  From a chain analysis perspective, it is reasonable to infer that the P2SH input in `txid: b` was the change from `txid: a`. To avoid leaking information in this scenario, Alice's wallet should avoid picking the P2SH output and instead fund the transaction with only bech32 Outputs. If the payment to Carol can be funded with just the P2SH output, it should be preferred over the bech32 outputs as this will convert the P2SH UTXO to bech32 UTXOs via the payment and change outputs of the new transaction.

  **TLDR;** Avoid mixing output types, spend non-default `OutputTypes` when it is economical to do so.

  # Approach

  `AvailableCoins` now populates a struct, which makes it easier to access coins by `OutputType`. Coin selection tries to find a funding solution by each output type and chooses the most economical by waste metric. If a solution can't be found without mixing, coin selection runs over the entire wallet, allowing mixing, which is the same as the current behavior.

  I've also added a functional test (`test/functional/wallet_avoid_mixing_output_types.py`) and unit test (`src/wallet/test/availablecoins_tests.cpp`.

ACKs for top commit:
  achow101:
    re-ACK 71d1d13627
  aureleoules:
    ACK 71d1d13627.
  Xekyo:
    reACK 71d1d13627 via `git range-diff master 6530d19 71d1d13`
  LarryRuane:
    ACK 71d1d13627

Tree-SHA512: 2e0716efdae5adf5479446fabc731ae81d595131d3b8bade98b64ba323d0e0c6d964a67f8c14c89c428998bda47993fa924f3cfca1529e2bd49eaa4e31b7e426
2022-07-28 18:16:51 -04:00
glozow
c320cddb1b
[unit tests] individual RBF Rules in isolation
Test each component of the RBF policy in isolation. Unlike the RBF
functional tests, these do not rely on things like RPC results, mempool
submission, etc.
2022-07-28 12:05:05 +01:00
josibake
71d1d13627
test: add unit test for AvailableCoins
test that UTXOs are bucketed correctly after
running AvailableCoins
2022-07-19 18:42:21 +02:00
fanquake
70d807c355
refactor: integrate no_nul into univalue unitester 2022-07-18 09:15:18 +01:00
chinggg
6eb0909cb7 fuzz: add low-level target for txorphanage 2022-07-06 22:13:54 +08:00
MarcoFalke
fa7cbc6e5c
build: Remove negated --enable-fuzz checks from build system 2022-06-22 08:12:23 +02:00
fanquake
695ca641a4
Merge bitcoin/bitcoin#24860: Miniscript integration follow-ups
f3a50c9dfe miniscript: rename IsSane and IsSaneSubexpression to prevent misuse (Antoine Poinsot)
c5fe5163dc miniscript: nit: don't return after assert(false) (Antoine Poinsot)
7bbaca9d8d miniscript: explicit the threshold size computation in multi() (Antoine Poinsot)
8323e4249d miniscript: add an OpCode typedef for readability (Antoine Poinsot)
7a549c6c59 miniscript: mark nodes with duplicate keys as insane (Antoine Poinsot)
8c0f8bf7bc fuzz: add a Miniscript target for string representation roundtripping (Antoine Poinsot)
be34d5077b fuzz: rename and improve the Miniscript Script roundtrip target (Antoine Poinsot)
7eb70f0ac0 miniscript: tiny doc fixups (Antoine Poinsot)
5cea85f12c miniscript: split ValidSatisfactions from IsSane (Antoine Poinsot)
a0f064dc14 miniscript: introduce a CheckTimeLocksMix helper (Antoine Poinsot)
ed45ee3882 miniscript: use optional instead of bool/outarg (Antoine Poinsot)
1ab8d89fd1 miniscript: make equality operator non-recursive (Antoine Poinsot)
5922c662c0 scripted-diff: miniscript: rename 'nodetype' variables to 'fragment' (Antoine Poinsot)
c5f65db0f0 miniscript: remove a workaround for a GCC 4.8 bug (Antoine Poinsot)

Pull request description:

  The Miniscript repository and the Miniscript integration PR here have been a moving target for the past months, and some final cleanups were done there that were not included here. I initially intended to add some small followup commits to #24148 but i think there are enough of them to be worth a followup PR on its own.

  Some parts of the code did not change since it was initially written in 2019, and the code could use some modernization. (Use std::optional instead of out args, remove old compiler workarounds).
  We refactored the helpers to be more meaningful, and also did some renaming. A new fuzz target was also added and both were merged in a single file. 2 more will be added in #24149 that will be contained in this file too.

  The only behaviour change in this PR is to rule out Miniscript with duplicate keys from sane Miniscripts. In a P2WSH context, signatures can be rebounded (Miniscript does not use CODESEPARATOR) and it's reasonable to assume that reusing keys across the Script drops the malleability guarantees.
  It was previously assumed such Miniscript would never exist in the first place since a compiler should never create them. We finally agreed that if one were to exist (say, written by hand or from a buggy compiler) it would be very confusing if an imported Miniscript descriptor (after #24148) with duplicate keys was deemed sane (ie, "safe to use") by Bitcoin Core. We now check for duplicate keys in the constructor.

  This is (still) joint work with Pieter Wuille. (Actually he entirely authored the cleanups and code modernization.)

ACKs for top commit:
  sipa:
    utACK f3a50c9dfe (with the caveat that a lot of it is my own code)
  sanket1729:
    code review ACK f3a50c9dfe. Did not review the fuzz tests.

Tree-SHA512: c043325e4936fe25e8ece4266b46119e000c6745f88cea530fed1edf01c80f03ee6f9edc83b6e9d42ca01688d184bad16bfd967c5bb8037744e726993adf3deb
2022-06-04 20:54:20 +01:00
fanquake
46a890960e
build: pass bdb cppflags only where needed
Move bdb cppflags out of the catch-all BITCOIN_INCLUDES, and pass them
only where they are needed, which is in libbitcoin_node/wallet and the
tests.
2022-05-31 06:37:43 +01:00
laanwj
652b54e532 bench: Add --sanity-check flag, use it in make check
The benchmarks are run as part of `make check` for a minimum sanity
check. The actual results are being ignored. So only run them for one
iteration.

This makes the `bench_bitcoin` part take 2m00 instead of 5m20 here.
Which is still too long (imo), but this needs to be solved in the
`WalletLoading*` benchmarks which take that long per iteration.
2022-05-17 11:32:25 +02:00
mruddy
bcb0cacac2 reindex, log, test: fixes #21379
This fixes a blk file size calculation made during reindex that results in increased blk file malformity.
The fix is to avoid double counting the size of the serialization header during reindex.
This adds a unit test to reproduce the bug before the fix and to ensure that it does not recur.
These changes include a log message change also so as to not be as alarming. This is a common and recoverable
data corruption. These messages can now be filtered by the debug log reindex category.
2022-05-07 07:11:29 -04:00
Antoine Poinsot
be34d5077b
fuzz: rename and improve the Miniscript Script roundtrip target
Parse also key hashes using the Key type. Make this target the first of
the 4 Miniscript fuzz targets in a single `miniscript` file.

Co-authored-by: Pieter Wuille <pieter.wuille@gmail.com>
2022-04-28 16:44:42 +02:00
fanquake
dd17c42a16
Merge bitcoin/bitcoin#24322: [kernel 1/n] Introduce initial libbitcoinkernel
035fa1f07a build: Remove LIBTOOL_APP_LDFLAGS for bitcoin-chainstate (Cory Fields)
3f0595095d docs: Add libbitcoinkernel_la_SOURCES explanation (Carl Dong)
94ad45deb2 ci: Build libbitcoinkernel (Carl Dong)
26b2e7ffb3 build: Extract the libbitcoinkernel library (Carl Dong)
1df44dd20c b-cs: Define G_TRANSLATION_FUN in bitcoinkernel.cpp (Carl Dong)
83a0bb7cc9 build: Separate lib_LTLIBRARIES initialization (Carl Dong)
c1e16cb31f build: Create .la library for bitcoincrypto (Carl Dong)
8bdfe057c7 build: Create .la library for leveldb (Carl Dong)
05d1525b6d build: Create .la library for crc32c (Carl Dong)
64caf94479 build: Remove vestigial LIBLEVELDB_SSE42 (Carl Dong)
1392e8e2d8 build: Don't add unrelated libs to LIBTEST_* (Carl Dong)

Pull request description:

  Part of: #24303

  This PR introduces a `libbitcoinkernel` static library linking in the minimal list of files necessary to use our consensus engine as-is. `bitcoin-chainstate` introduced in #24304 now will link against `libbitcoinkernel`.

  Most of the changes are related to the build system.

  Please read the commit messages for more details.

ACKs for top commit:
  theuni:
    This may be my favorite PR ever. It's a privilege to ACK 035fa1f07a.

Tree-SHA512: b755edc3471c7c1098847e9b16ab182a6abb7582563d9da516de376a770ac7543c6fdb24238ddd4d3d2d458f905a0c0614b8667aab182aa7e6b80c1cca7090bc
2022-04-28 15:14:32 +01:00
John Newbery
b8f17fbcb4 [tests] Move TxOrphange tests to orphange_tests.cpp 2022-04-25 08:37:01 +01:00
Carl Dong
64caf94479 build: Remove vestigial LIBLEVELDB_SSE42
- LIBLEVELDB_SSE42_INT was defined, but never referenced anywhere
- LIBLEVELDB_SSE42 is referenced, but never defined anywhere

Apparently leveldb used to have platform-specific crc32 code before it
got split off into a separate lib.
2022-04-11 16:56:34 -04:00
MarcoFalke
27cfaeed1e
Merge bitcoin/bitcoin#24098: rest: Use query parameters to control resource loading
54b39cfb34 Add release notes (stickies-v)
f959fc0397 Update /<count>/ endpoints to use a '?count=' query parameter instead (stickies-v)
a09497614e Add GetQueryParameter helper function (stickies-v)
fff771ee86 Handle query string when parsing data format (stickies-v)
c1aad1b3b9 scripted-diff: rename RetFormat to RESTResponseFormat (stickies-v)
9f1c54787c Refactoring: move declarations to rest.h (stickies-v)

Pull request description:

  In RESTful APIs, [typically](https://rapidapi.com/blog/api-glossary/parameters/query/) path parameters  (e.g. `/some/unique/resource/`) are used to represent resources, and query parameters (e.g. `?sort=asc`) are used to control how these resources are being loaded through e.g. sorting, pagination, filtering, ...

  As first [discussed in #17631](https://github.com/bitcoin/bitcoin/pull/17631#discussion_r733031180), the [current REST api](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md) contains two endpoints `/headers/` and `/blockfilterheaders/` that rather unexpectedly use path parameters to control how many (filter) headers are returned in the response. While this is no critical issue, it is unintuitive and we are still early enough to easily phase this behaviour out and ensure new endpoints (if any) do not have to stick to non-standard behaviour just for internal consistency.

  In this PR, a new `HTTPRequest::GetQueryParameter` method is introduced to easily parse query parameters, as well as two new `/headers/` and `/blockfilterheaders/` endpoints that use a count query parameter are introduced. The old path parameter-based endpoints are kept without too much overhead, but the documentation now points to the new query parameter-based endpoints as the default interface to encourage standardness.

  ## Behaviour change
  ### New endpoints and default values
  `/headers/` and `/blockfilterheaders/` now have 2 new endpoints that contain query parameters (`?count=<count>`) instead of path parameters (`/<count>/`), as described in REST-interface.md. Since query parameters can easily have default values, I have set this at 5 for both endpoints.

  **headers**
  `GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  **blockfilterheaders**
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  ### Some previously invalid API calls are now valid
  API calls that contained query strings in the URI could not be parsed prior to this PR. This PR changes behaviour in that previously invalid calls (e.g. `GET /rest/headers/5/somehash.json?someunusedparam=foo`) would now become valid, as the query parameters are properly parsed, and discarded if unused.
  For example, prior to this PR, adding an irrelevant `someparam` parameter would be illegal:
  ```
  GET /rest/headers/5/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ->
  Invalid hash: 0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ```
  **This behaviour change affects all rest endpoints, not just the 2 new ones introduced here.**

  *(Note: I'd be open to implementing additional logic to refuse requests containing unrecognized query parameters to minimize behaviour change, but for the endpoints that we currently have I don't really see the point for that added complexity. E.g. I don't see any scenarios where misspelling a parameter could lead to harmful outcomes)*

  ## Using the REST API

  To run the API HTTP server, start a bitcoind instance with the `-rest` flag enabled. To use the
  `blockfilterheaders` endpoint, you'll also need to set `-blockfilterindex=1`:
  ```
  ./bitcoind -signet -rest -blockfilterindex=1
  ```

  As soon as bitcoind is fully up and running, you should be able to query the API, for example by
  using curl on the command line: ```curl "127.0.0.1:38332/rest/chaininfo.json"```.
  To more easily parse the JSON output, you can also use tools like 'jq' or `json_pp`, e.g.:
  ```
  curl -s "localhost:38332/rest/blockfilterheaders/basic/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?count=2" | json_pp .
  ```

  ## To do
  - [x] update `doc/release-notes`

  ## Feedback
  This is my first PR (hooray!). Please don't hold back on any feedback/comments/nits/... you may have, big or small, whether they are code, process, language, ... related. I welcome private messages too if there's anything you don't want to clutter the PR with. I'm here to learn and am grateful for everyone's input.

ACKs for top commit:
  stickies-v:
    I've had to push a tiny doc update to `REST-interface.md` (`git range-diff 219d728 9aac438 54b39cf`) since this was not merged for v23, but since there are no significant changes beyond theStack and jnewbery's ACKs I think this PR is now ready to be considered for merging? @MarcoFalke
  jnewbery:
    ACK 54b39cfb34
  theStack:
    re-ACK 54b39cfb34

Tree-SHA512: 3b393ffde34f25605ca12c0b1300799a19684b816a1d03aed38b0f5439df47bfe6a589ffbcd7b83fd2def6c9d00a1bae5e45b1d18df4ae998c617c709990f83f
2022-04-06 09:25:56 +02:00
laanwj
d492dc1cda
Merge bitcoin/bitcoin#24147: Miniscript integration
2da94a4c6f fuzz: add a fuzz target for Miniscript decoding from Script (Antoine Poinsot)
f8369996e7 Miniscript: ops limit and stack size computation (Pieter Wuille)
2e55e88f86 Miniscript: conversion from script (Pieter Wuille)
1ddaa66eae Miniscript: type system, script creation, text notation, tests (Pieter Wuille)
4fe29368c0 script: expose getter for CScriptNum, add a BuildScript helper (Antoine Poinsot)
f4e289f384 script: move CheckMinimalPush from interpreter to script.h (Antoine Poinsot)
31ec6ae92a script: make IsPushdataOp non-static (Antoine Poinsot)

Pull request description:

  Miniscript is a language for writing (a subset of) Bitcoin Scripts in a structured way.

  Miniscript permits:
  - To safely extend the Output Descriptor language to many more scripting features thanks to the typing system (composition).
  - Statical analysis of spending conditions, maximum spending cost of each branch, security properties, third-party malleability.
  - General satisfaction of any correctly typed ("valid" [0]) Miniscript. The satisfaction itself is also analyzable.
  - To extend the possibilities of external signers, because of all of the above and since it carries enough metadata.

  Miniscript guarantees:
  - That for any statically-analyzed as "safe" [0] Script, a witness can be constructed in the bounds of the consensus and standardness rules (standardness complete).
  - That unless the conditions of the Miniscript are met, no witness can be created for the Script (consensus sound).
  - Third-party malleability protection for the satisfaction of a sane Miniscript, which is too complex to summarize here.

  For more details around Miniscript (including the specifications), please refer to the [website](https://bitcoin.sipa.be/miniscript/).

  Miniscript was designed by Pieter Wuille, Andrew Poelstra and Sanket Kanjalkar.
  This PR is an updated and rebased version of #16800. See [the commit history of the Miniscript repository](https://github.com/sipa/miniscript/commits/master) for details about the changes made since September 2019 (TL;DR: bugfixes, introduction of timelock conflicts in the type system, `pk()` and `pkh()` aliases, `thresh_m` renamed to `multi`, all recursive algorithms were made non-recursive).

  This PR is also the first in a series of 3:
  - The first one (here) integrates the backbone of Miniscript.
  - The second one (#24148) introduces support for Miniscript in Output Descriptors, allowing for watch-only support of Miniscript Descriptors in the wallet.
  - The third one (#24149) implements signing for these Miniscript Descriptors, using Miniscript's satisfaction algorithm.

  Note to reviewers:
  - Miniscript is currently defined only for P2WSH. No Taproot yet.
  - Miniscript is different from the policy language (a high-level logical representation of a spending policy). A policy->Miniscript compiler is not included here.
  - The fuzz target included here is more interestingly extended in the 3rd PR to check a script's satisfaction against `VerifyScript`. I think it could be further improved by having custom mutators as we now have for multisig (see https://github.com/bitcoin/bitcoin/issues/23105). A minified corpus of Miniscript Scripts is available at https://github.com/bitcoin-core/qa-assets/pull/85.

  [0] We call "valid" any correctly-typed Miniscript. And "safe" any sane Miniscript, ie one whose satisfaction isn't malleable, which requires a key for any spending path, etc..

ACKs for top commit:
  jb55:
    ACK 2da94a4c6f
  laanwj:
    Light code review ACK 2da94a4c6f (mostly reviewed the changes to the existing code and build system)

Tree-SHA512: d3ef558436cfcc699a50ad13caf1e776f7d0addddb433ee28ef38f66ea5c3e581382d8c748ccac9b51768e4b95712ed7a6112b0e3281a6551e0f325331de9167
2022-04-05 13:22:09 +02:00
Andrew Chow
1021e4cc68
Merge bitcoin/bitcoin#24602: fuzz: add target for coinselection algorithms
21520b9551 fuzz: add target for coinselection (Martin Zumsande)

Pull request description:

  This adds a fuzz target for the coinselection algorithms by creating random `OutputGroup`s and running all three coin selection algorithms for them.
  It does not fuzz higher-level wallet logic for selecting eligible coins (as in `SelectCoins()`), thought it probably would make sense to have a fuzz target for that too.

ACKs for top commit:
  achow101:
    ACK 21520b9551
  vasild:
    ACK 21520b9551

Tree-SHA512: c763003cf5ff5317f929d3d0b2f06fa739ae41dd642042d9a5c5c96e6cb9b349a6c7aeabc77bc2b846d12c8bcb60e07ee20a9f38539429c65723ab76aeee6b2e
2022-03-31 13:09:17 -04:00
Martin Zumsande
21520b9551 fuzz: add target for coinselection
This creates random OutputGroups and runs the
existing coinselection algorithms for them.
2022-03-30 17:17:37 +02:00
Hennadii Stepanov
8b517fae7e
build, refactor: Replace tabs with spaces 2022-03-30 12:16:05 +02:00
Hennadii Stepanov
dc0774cbdf
build, test: Fix test logfile name
Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
2022-03-30 12:15:53 +02:00
MarcoFalke
d2615312c1
Merge bitcoin/bitcoin#19385: test: Change default test logging directory
f8cba0d911 test: Change default test logging directory (Yancy Ribbens)

Pull request description:

  This PR changes the default test log location request here: https://github.com/bitcoin/bitcoin/issues/17224.  Instead of using the location of the makefile [automatic variable](https://www.gnu.org/software/make/manual/make.html#Automatic-Variables) `$<` I extract just the basename and then prepend a new location `./test`.  This is done because `$<` represents the variable name AND location of the prerequisite here.

Top commit has no ACKs.

Tree-SHA512: f0fbc530cf0e14c284b4bbf6671c145b1d7a2e1f5561f5c5d09f0cbe88b98e620e763bbbf2dfa9aeeec3dcc9b0127939e105e14c7e4f6660c7c19663622a393d
2022-03-25 13:30:26 +01:00
Antoine Poinsot
2da94a4c6f
fuzz: add a fuzz target for Miniscript decoding from Script 2022-03-17 14:09:09 +01:00
Pieter Wuille
1ddaa66eae
Miniscript: type system, script creation, text notation, tests
More information about Miniscript can be found at https://bitcoin.sipa.be/miniscript/ (the
website source is hosted at https://github.com/sipa/miniscript/).
This commit defines all fragments, their composition, parsing from
string representation and conversion to Script.

Co-Authored-By: Antoine Poinsot <darosior@protonmail.com>
Co-Authored-By: Sanket Kanjalkar <sanket1729@gmail.com>
Co-Authored-By: Samuel Dobson <dobsonsa68@gmail.com>
2022-03-17 14:09:07 +01:00
MarcoFalke
3617d22562
Merge bitcoin/bitcoin#14752: tests: Unit tests for IsPayToWitnessScriptHash and IsWitnessProgram
bce9aaf31e Unit tests for IsWitnessProgram and IsP2WSH. (Daniel Kraft)

Pull request description:

  This adds basic unit tests for `CScript::IsPayToWitnessScriptHash` and `CScript::IsWitnessProgram`, similar to the existing tests for `CScript::IsPayToScriptHash`.  These tests are probably not super important given the other existing tests for segwit related code, but may be useful in catching some errors early.

  This implements #14737.

ACKs for top commit:
  aureleoules:
    tACK bce9aaf31e (`make check)`.

Tree-SHA512: 3cff5efc4ac53079289c72bfba8b1937bc103baadd32bb1fba41e78017f65f9cca17678c3202ad0711eae42b351d4132d9ed9b4e2dc07d138298691a09c4e822
2022-03-16 17:47:56 +01:00
stickies-v
a09497614e
Add GetQueryParameter helper function
Easily get the query parameter from the URI, with optional default value.
2022-03-10 12:01:54 +01:00
stickies-v
fff771ee86
Handle query string when parsing data format
URLs may contain a query string (prefixed with '?') and this should be ignored when parsing
the data format.

To facilitate testing this functionality, ParseDataFormat has been made non-static.
2022-03-10 12:01:53 +01:00
MarcoFalke
fae3f17823
fuzz: Split script formatting from script fuzz target 2022-02-15 12:19:34 +01:00
fanquake
2037a3b6c1
build: header-only Boost 2022-02-13 20:59:07 +00:00
fanquake
39e66e938f
build: use header-only Boost unit test 2022-02-13 20:59:02 +00:00
Russell Yanofsky
172096e9dd scripted-diff: Rename libbitcoin_server.a to libbitcoin_node.a
Goal along with namespacing PR #23497 is to have src/node/ code in
node:: namespace in libbitcoin_node.a library

-BEGIN VERIFY SCRIPT-
bash -c ' # Bash shell needed for brace expansion {a,b}
git mv build_msvc/libbitcoin_{server,node}
git mv build_msvc/libbitcoin_node/libbitcoin_{server,node}.vcxproj.in
ren() { git grep -l "$1" src build_msvc | xargs sed -i "s/$1/$2/g"; }
ren LIBBITCOIN_{SERVER,NODE}
ren libbitcoin_{server,node}
'
-END VERIFY SCRIPT-
2021-12-20 10:53:01 -05:00