bitcoin/src/test
MarcoFalke 1a43cd3f74
Merge #19709: test: Fix 'make cov' with clang
35cd2da623 test: Fix 'make cov' with clang (Hennadii Stepanov)

Pull request description:

  This is a follow up of #19688.

  With this PR it is possible to do the following:
  ```
  $ ./autogen.sh
  $ ./configure --enable-lcov CC=clang CXX=clang++
  $ make
  $ make cov
  ```

  Currently, on master (8a85377cd0), `make cov` fails to `Processing src/test/test_bitcoin-util_tests.gcda`.

ACKs for top commit:
  vasild:
    ACK 35cd2da
  Crypt-iQ:
    ACK 35cd2da

Tree-SHA512: aaf56118e2644064e9738a8279889c617db5805c5c804c904469b24c496bd609f9c5fc2aebcf1a422f8a5ed2eb38bd6e76b484680310b55c36d922b73a4c33cf
2020-08-14 14:57:49 +02:00
..
data wallet: Fix typo in comments; Simplify assert 2020-07-11 14:24:36 +02:00
fuzz Merge #19316: [net] Cleanup logic around connection types 2020-08-12 10:01:44 +08:00
util Merge #19098: test: Remove duplicate NodeContext hacks 2020-08-07 08:07:37 +02:00
addrman_tests.cpp [addrman] Specify max addresses and pct when calling GetAddresses() 2020-08-12 09:22:07 +01:00
allocator_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
amount_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
arith_uint256_tests.cpp test: Move boost/stdlib includes last 2020-04-17 06:36:04 -04:00
base32_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
base58_tests.cpp tests: Add tests for base58-decoding of std::string:s containing non-base58 characters 2019-12-12 11:01:56 +00:00
base64_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
bech32_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
bip32_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
blockchain_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
blockencodings_tests.cpp Convert blockencodings_tests to new serialization 2020-03-30 16:10:30 -07:00
blockfilter_index_tests.cpp scripted-diff: Replace EnsureChainman with Assert in unit tests 2020-06-15 07:39:26 -04:00
blockfilter_tests.cpp test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
bloom_tests.cpp test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
bswap_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
checkqueue_tests.cpp refactor: Specify boost/thread/thread.hpp explicitly 2020-06-04 10:05:54 -04:00
coins_tests.cpp refactor: Replace HexStr(o.begin(), o.end()) with HexStr(o) 2020-06-24 18:41:45 +02:00
compilerbug_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
compress_tests.cpp test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
crypto_tests.cpp refactor: Make HexStr take a span 2020-08-06 19:41:43 +02:00
cuckoocache_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
dbwrapper_tests.cpp Merge #18317: Serialization improvements step 6 (all except wallet/gui) 2020-05-20 07:30:29 -04:00
denialofservice_tests.cpp [net/refactor] Introduce an enum to distinguish type of connection 2020-08-07 17:18:16 -07:00
descriptor_tests.cpp wallet: Fix typo in comments; Simplify assert 2020-07-11 14:24:36 +02:00
flatfile_tests.cpp test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
fs_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
getarg_tests.cpp scripted-diff: replace gArgs with argsman 2020-05-28 06:22:19 -07:00
hash_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
interfaces_tests.cpp test: Verify findCommonAncestor always initializes outputs 2020-04-15 17:07:44 -04:00
key_io_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
key_tests.cpp Make Hash[160] consume range-like objects 2020-07-30 13:57:54 -07:00
limitedmap_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
logging_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
main.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
Makefile
mempool_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
merkle_tests.cpp Make Hash[160] consume range-like objects 2020-07-30 13:57:54 -07:00
merkleblock_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
miner_tests.cpp scripted-diff: Replace EnsureChainman with Assert in unit tests 2020-06-15 07:39:26 -04:00
multisig_tests.cpp scripted-diff: TxoutType C++11 scoped enum class 2020-06-21 06:41:55 -04:00
net_tests.cpp [net/refactor] Remove fInbound flag from CNode 2020-08-07 17:18:16 -07:00
netbase_tests.cpp Merge #18991: Cache responses to GETADDR to prevent topology leaks 2020-08-03 14:48:52 +02:00
pmt_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
policy_fee_tests.cpp test: Add FeeFilterRounder test 2020-06-19 09:26:59 -04:00
policyestimator_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
pow_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
prevector_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
raii_event_tests.cpp Merge #16564: test: Always define the raii_event_tests test suite 2020-05-31 17:55:55 -04:00
random_tests.cpp Merge #18781: Add templated GetRandDuration<> 2020-05-15 08:58:49 -04:00
README.md doc: Explain new test logging 2020-03-31 17:11:47 -04:00
ref_tests.cpp Add util::Ref class as temporary alternative for c++17 std::any 2020-05-13 16:20:13 -04:00
reverselock_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
rpc_tests.cpp refactor: Pass NodeContext to RPC and REST methods through util::Ref 2020-05-13 16:20:13 -04:00
sanity_tests.cpp test: fix message for ECC_InitSanityCheck test 2020-04-30 16:57:46 +08:00
scheduler_tests.cpp Merge #19090: refactor: Misc scheduler cleanups 2020-06-25 09:24:58 -04:00
script_p2sh_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
script_standard_tests.cpp Make CHash256/CHash160 output to Span 2020-07-30 13:57:54 -07:00
script_tests.cpp Make CHash256/CHash160 output to Span 2020-07-30 13:57:54 -07:00
scriptnum10.h scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
scriptnum_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
serialize_tests.cpp Make Hash[160] consume range-like objects 2020-07-30 13:57:54 -07:00
settings_tests.cpp refactor: Make HexStr take a span 2020-08-06 19:41:43 +02:00
sighash_tests.cpp refactor: Replace HexStr(o.begin(), o.end()) with HexStr(o) 2020-06-24 18:41:45 +02:00
sigopcount_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
skiplist_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
streams_tests.cpp test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
sync_tests.cpp test: Add LockStackEmpty() 2020-08-02 16:42:39 +03:00
system_tests.cpp [util] add RunCommandParseJSON 2020-07-31 13:38:10 +02:00
timedata_tests.cpp Make GetWarnings() return bilingual_str 2020-06-10 15:01:20 +03:00
torcontrol_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
transaction_tests.cpp Do not turn OP_1NEGATE in scriptSig into 0x0181 in signing code 2020-07-15 15:29:22 +12:00
txindex_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
txvalidation_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00
txvalidationcache_tests.cpp Merge #18675: tests: Don't initialize PrecomputedTransactionData in txvalidationcache tests 2020-04-19 06:18:21 -04:00
uint256_tests.cpp test: Move boost/stdlib includes last 2020-04-17 06:36:04 -04:00
util_tests.cpp test: Fix 'make cov' with clang 2020-08-14 12:18:47 +03:00
util_threadnames_tests.cpp test: Set -logthreadnames in unit tests 2020-05-21 08:13:23 -04:00
validation_block_tests.cpp scripted-diff: Replace EnsureChainman with Assert in unit tests 2020-06-15 07:39:26 -04:00
validation_chainstate_tests.cpp test: Simplify cs_main locks 2020-07-29 08:00:54 +02:00
validation_chainstatemanager_tests.cpp test: Simplify cs_main locks 2020-07-29 08:00:54 +02:00
validation_flush_tests.cpp Pass mempool pointer to GetCoinsCacheSizeState 2020-07-29 12:30:11 +02:00
validation_tests.cpp scripted-diff: test: Move setup_common to test library 2019-11-06 11:56:41 -05:00
validationinterface_tests.cpp test: Remove const to work around compiler error on xenial 2020-05-14 18:40:57 +02:00
versionbits_tests.cpp scripted-diff: Bump copyright headers 2020-04-16 13:33:09 -04:00

Unit tests

The sources in this directory are unit test cases. Boost includes a unit testing framework, and since Bitcoin Core already uses Boost, it makes sense to simply use this framework rather than require developers to configure some other framework (we want as few impediments to creating unit tests as possible).

The build system is set up to compile an executable called test_bitcoin that runs all of the unit tests. The main source file for the test library is found in util/setup_common.cpp.

Compiling/running unit tests

Unit tests will be automatically compiled if dependencies were met in ./configure and tests weren't explicitly disabled.

After configuring, they can be run with make check.

To run the unit tests manually, launch src/test/test_bitcoin. To recompile after a test file was modified, run make and then run the test again. If you modify a non-test file, use make -C src/test to recompile only what's needed to run the unit tests.

To add more unit tests, add BOOST_AUTO_TEST_CASE functions to the existing .cpp files in the test/ directory or add new .cpp files that implement new BOOST_AUTO_TEST_SUITE sections.

To run the GUI unit tests manually, launch src/qt/test/test_bitcoin-qt

To add more GUI unit tests, add them to the src/qt/test/ directory and the src/qt/test/test_main.cpp file.

Running individual tests

test_bitcoin has some built-in command-line arguments; for example, to run just the getarg_tests verbosely:

test_bitcoin --log_level=all --run_test=getarg_tests -- DEBUG_LOG_OUT

log_level controls the verbosity of the test framework, which logs when a test case is entered, for example. The DEBUG_LOG_OUT after the two dashes redirects the debug log, which would normally go to a file in the test datadir (BasicTestingSetup::m_path_root), to the standard terminal output.

... or to run just the doubledash test:

test_bitcoin --run_test=getarg_tests/doubledash

Run test_bitcoin --help for the full list.

Adding test cases

To add a new unit test file to our test suite you need to add the file to src/Makefile.test.include. The pattern is to create one test file for each class or source file for which you want to create unit tests. The file naming convention is <source_filename>_tests.cpp and such files should wrap their tests in a test suite called <source_filename>_tests. For an example of this pattern, see uint256_tests.cpp.

Logging and debugging in unit tests

make check will write to a log file foo_tests.cpp.log and display this file on failure. For running individual tests verbosely, refer to the section above.

To write to logs from unit tests you need to use specific message methods provided by Boost. The simplest is BOOST_TEST_MESSAGE.

For debugging you can launch the test_bitcoin executable with gdbor lldb and start debugging, just like you would with any other program:

gdb src/test/test_bitcoin