bitcoin/contrib/devtools
Ava Chow 011a895a82
Merge bitcoin/bitcoin#29015: kernel: Streamline util library
c7376babd1 doc: Clarify distinction between util and common libraries in libraries.md (Ryan Ofsky)
4f74c59334 util: Move util/string.h functions to util namespace (Ryan Ofsky)
4d05d3f3b4 util: add TransactionError includes and namespace declarations (Ryan Ofsky)
680eafdc74 util: move fees.h and error.h to common/messages.h (Ryan Ofsky)
02e62c6c9a common: Add PSBTError enum (Ryan Ofsky)
0d44c44ae3 util: move error.h TransactionError enum to node/types.h (Ryan Ofsky)
9bcce2608d util: move spanparsing.h to script/parsing.h (Ryan Ofsky)
6dd2ad4792 util: move spanparsing.h Split functions to string.h (Ryan Ofsky)
23cc8ddff4 util: move HexStr and HexDigit from util to crypto (TheCharlatan)
6861f954f8 util: move util/message to common/signmessage (Ryan Ofsky)
cc5f29fbea build: move memory_cleanse from util to crypto (Ryan Ofsky)
5b9309420c build: move chainparamsbase from util to common (Ryan Ofsky)
ffa27af24d test: Add check-deps.sh script to check for unexpected library dependencies (Ryan Ofsky)

Pull request description:

  Remove `fees.h`, `errors.h`, and `spanparsing.h` from the util library. Specifically:

  - Move `Split` functions from `util/spanparsing.h` to `util/string.h`, using `util` namespace for clarity.
  - Move remaining spanparsing functions to `script/parsing.h` since they are used for descriptor and miniscript parsing.
  - Combine `util/fees.h` and `util/errors.h` into `common/messages.h` so there is a place for simple functions that generate user messages to live, and these functions are not part of the util library.

  Motivation for this change is that the util library is a dependency of the kernel, and we should remove functionality from util that shouldn't be called by kernel code or kernel applications. These changes should also improve code organization and make functions easier to discover. Some of these same moves are (or were) part of #28690, but did not help with code organization, or made it worse, so it is better to move them and clean them up in the same PR so code only has to change one time.

ACKs for top commit:
  achow101:
    ACK c7376babd1
  TheCharlatan:
    Re-ACK c7376babd1
  hebasto:
    re-ACK c7376babd1.

Tree-SHA512: 5bcef16c1255463b1b69270548711e7ff78ca0dd34e300b95e3ca1ce52ceb34f83d9ddb2839e83800ba36b200de30396e504bbb04fa02c6d0c24a16d06ae523d
2024-06-12 17:12:54 -04:00
..
bitcoin-tidy tidy: remove C compiler check 2024-03-22 13:48:00 +00:00
iwyu ci, iwyu: Drop backported mappings 2024-01-05 16:01:52 +00:00
check-deps.sh test: Add check-deps.sh script to check for unexpected library dependencies 2024-05-16 11:16:08 -04:00
circular-dependencies.py test: use built-in collection types for type hints (Python 3.9 / PEP 585) 2023-10-25 01:10:21 +02:00
clang-format-diff.py contrib: Fix clang-format-diff.py lint errors 2024-01-17 11:10:15 +01:00
copyright_header.py scripted-diff: Bump copyright headers 2022-12-24 23:49:50 +00:00
gen-bitcoin-conf.sh script: add script to generate example bitcoin.conf 2022-05-02 15:44:39 +02:00
gen-manpages.py contrib: list other binaries in manpage output 2024-03-07 13:07:53 +00:00
headerssync-params.py headerssync: update params for 26.x 2023-10-05 11:36:03 +01:00
README.md docs: fix typo 2023-10-10 15:06:47 +08:00
security-check.py contrib: add macho branch protection check 2024-01-03 17:28:39 +00:00
split-debug.sh.in gitian: Improve error handling 2019-03-07 17:21:46 +01:00
symbol-check.py refactor: rename (macho) ld64 to lld 2024-06-10 13:20:54 +01:00
test-security-check.py build: switch to using lld for macOS builds 2024-05-22 08:51:33 +01:00
test-symbol-check.py contrib: drop dead get_machine from test sym check 2024-05-10 00:13:50 +08:00
test_deterministic_coverage.sh [refactor] Make signals optional in mempool and chainman 2024-02-15 13:28:45 +01:00
test_utxo_snapshots.sh rpc: Do not wait for headers inside loadtxoutset 2024-01-30 18:09:58 +01:00
utils.py test: use built-in collection types for type hints (Python 3.9 / PEP 585) 2023-10-25 01:10:21 +02:00
utxo_snapshot.sh Merge bitcoin/bitcoin#28852: script, assumeutxo: Enhance validations in utxo_snapshot.sh 2023-12-04 17:07:14 -05:00

Contents

This directory contains tools for developers working on this repository.

clang-format-diff.py

A script to format unified git diffs according to .clang-format.

Requires clang-format, installed e.g. via brew install clang-format on macOS, or sudo apt install clang-format on Debian/Ubuntu.

For instance, to format the last commit with 0 lines of context, the script should be called from the git root folder as follows.

git diff -U0 HEAD~1.. | ./contrib/devtools/clang-format-diff.py -p1 -i -v

copyright_header.py

Provides utilities for managing copyright headers of The Bitcoin Core developers in repository source files. It has three subcommands:

$ ./copyright_header.py report <base_directory> [verbose]
$ ./copyright_header.py update <base_directory>
$ ./copyright_header.py insert <file>

Running these subcommands without arguments displays a usage string.

Produces a report of all copyright header notices found inside the source files of a repository. Useful to quickly visualize the state of the headers. Specifying verbose will list the full filenames of files of each category.

Updates all the copyright headers of The Bitcoin Core developers which were changed in a year more recent than is listed. For example:

// Copyright (c) <firstYear>-<lastYear> The Bitcoin Core developers

will be updated to:

// Copyright (c) <firstYear>-<lastModifiedYear> The Bitcoin Core developers

where <lastModifiedYear> is obtained from the git log history.

This subcommand also handles copyright headers that have only a single year. In those cases:

// Copyright (c) <year> The Bitcoin Core developers

will be updated to:

// Copyright (c) <year>-<lastModifiedYear> The Bitcoin Core developers

where the update is appropriate.

Inserts a copyright header for The Bitcoin Core developers at the top of the file in either Python or C++ style as determined by the file extension. If the file is a Python file and it has #! starting the first line, the header is inserted in the line below it.

The copyright dates will be set to be <year_introduced>-<current_year> where <year_introduced> is according to the git log history. If <year_introduced> is equal to <current_year>, it will be set as a single year rather than two hyphenated years.

If the file already has a copyright for The Bitcoin Core developers, the script will exit.

gen-manpages.py

A small script to automatically create manpages in ../../doc/man by running the release binaries with the -help option. This requires help2man which can be found at: https://www.gnu.org/software/help2man/

With in-tree builds this tool can be run from any directory within the repository. To use this tool with out-of-tree builds set BUILDDIR. For example:

BUILDDIR=$PWD/build contrib/devtools/gen-manpages.py

headerssync-params.py

A script to generate optimal parameters for the headerssync module (src/headerssync.cpp). It takes no command-line options, as all its configuration is set at the top of the file. It runs many times faster inside PyPy. Invocation:

pypy3 contrib/devtools/headerssync-params.py

gen-bitcoin-conf.sh

Generates a bitcoin.conf file in share/examples/ by parsing the output from bitcoind --help. This script is run during the release process to include a bitcoin.conf with the release binaries and can also be run by users to generate a file locally. When generating a file as part of the release process, make sure to commit the changes after running the script.

With in-tree builds this tool can be run from any directory within the repository. To use this tool with out-of-tree builds set BUILDDIR. For example:

BUILDDIR=$PWD/build contrib/devtools/gen-bitcoin-conf.sh

security-check.py and test-security-check.py

Perform basic security checks on a series of executables.

symbol-check.py

A script to check that release executables only contain certain symbols and are only linked against allowed libraries.

For Linux this means checking for allowed gcc, glibc and libstdc++ version symbols. This makes sure they are still compatible with the minimum supported distribution versions.

For macOS and Windows we check that the executables are only linked against libraries we allow.

Example usage:

find ../path/to/executables -type f -executable | xargs python3 contrib/devtools/symbol-check.py

If no errors occur the return value will be 0 and the output will be empty.

If there are any errors the return value will be 1 and output like this will be printed:

.../64/test_bitcoin: symbol memcpy from unsupported version GLIBC_2.14
.../64/test_bitcoin: symbol __fdelt_chk from unsupported version GLIBC_2.15
.../64/test_bitcoin: symbol std::out_of_range::~out_of_range() from unsupported version GLIBCXX_3.4.15
.../64/test_bitcoin: symbol _ZNSt8__detail15_List_nod from unsupported version GLIBCXX_3.4.15

circular-dependencies.py

Run this script from the root of the source tree (src/) to find circular dependencies in the source code. This looks only at which files include other files, treating the .cpp and .h file as one unit.

Example usage:

cd .../src
../contrib/devtools/circular-dependencies.py {*,*/*,*/*/*}.{h,cpp}