bitcoin/test/functional/test_framework
MarcoFalke 3d6752779f
Merge #17633: tests: Add option --valgrind to run the functional tests under Valgrind
5db506ba59 tests: Add option --valgrind to run nodes under valgrind in the functional tests (practicalswift)

Pull request description:

  What is better than fixing bugs? Fixing entire bug classes of course! :)

  Add option `--valgrind` to run the functional tests under Valgrind.

  Regular functional testing under Valgrind would have caught many of the uninitialized reads we've seen historically.

  Let's kill this bug class once and for all: let's never use an uninitialized value ever again. Or at least not one that would be triggered by running the functional tests! :)

  My hope is that this addition will make it super-easy to run the functional tests under Valgrind and thus increase the probability of people making use of it :)

  Hopefully `test/functional/test_runner.py --valgrind` will become a natural part of the pre-release QA process.

  **Usage:**

  ```
  $ test/functional/test_runner.py --help
  …
    --valgrind            run nodes under the valgrind memory error detector:
                          expect at least a ~10x slowdown, valgrind 3.14 or
                          later required
  ```

  **Live demo:**

  First, let's re-introduce a memory bug by reverting the recent P2P uninitialized read bug fix from PR #17624 ("net: Fix an uninitialized read in ProcessMessage(…, "tx", …) when receiving a transaction we already have").

  ```
  $ git diff
  diff --git a/src/consensus/validation.h b/src/consensus/validation.h
  index 3401eb64c..940adea33 100644
  --- a/src/consensus/validation.h
  +++ b/src/consensus/validation.h
  @@ -114,7 +114,7 @@ inline ValidationState::~ValidationState() {};

   class TxValidationState : public ValidationState {
   private:
  -    TxValidationResult m_result = TxValidationResult::TX_RESULT_UNSET;
  +    TxValidationResult m_result;
   public:
       bool Invalid(TxValidationResult result,
                    const std::string &reject_reason="",
  ```

  Second, let's test as normal without Valgrind:

  ```
  $ test/functional/p2p_segwit.py -l INFO
  2019-11-28T09:30:42.810000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test__fc8q3qo
  …
  2019-11-28T09:31:57.187000Z TestFramework (INFO): Subtest: test_non_standard_witness_blinding (Segwit active = True)
  …
  2019-11-28T09:32:08.265000Z TestFramework (INFO): Tests successful
  ```

  Third, let's test with `--valgrind` and see if the test fail (as we expect) when the unitialized value is used:

  ```
  $ test/functional/p2p_segwit.py -l INFO --valgrind
  2019-11-28T09:32:33.018000Z TestFramework (INFO): Initializing test directory /tmp/bitcoin_func_test_gtjecx2l
  …
  2019-11-28T09:40:36.702000Z TestFramework (INFO): Subtest: test_non_standard_witness_blinding (Segwit active = True)
  2019-11-28T09:40:37.813000Z TestFramework (ERROR): Assertion failed
  ConnectionRefusedError: [Errno 111] Connection refused
  ```

ACKs for top commit:
  MarcoFalke:
    ACK 5db506ba59
  jonatack:
    ACK 5db506ba59

Tree-SHA512: 2eaecacf4da166febad88b2a8ee6d7ac2bcd38d4c1892ca39516b6343e8f8c8814edf5eaf14c90f11a069a0389d24f0713076112ac284de987e72fc5f6cc3795
2019-12-10 13:30:37 -05:00
..
__init__.py Rename rpc-tests directory to functional 2017-03-20 10:40:31 -04:00
address.py rpc: Add generatetodescriptor 2019-10-30 10:01:32 -04:00
authproxy.py test: Properly log named args in authproxy 2019-04-08 18:38:54 -04:00
bignum.py Remove unused variables and/or function calls 2017-08-28 15:18:14 +02:00
blocktools.py test: Properly serialize BIP34 coinbase height 2019-07-09 14:12:33 -04:00
coverage.py Update copyright headers to 2018 2018-07-27 07:15:02 -04:00
descriptors.py Descriptor checksum 2019-02-15 22:36:05 -08:00
key.py Add comments to Python ECDSA implementation 2019-04-18 13:23:26 -07:00
messages.py p2p: Remove BIP61 reject messages 2019-10-02 10:39:14 -04:00
mininode.py Remove network_event_loop instance in close() 2019-11-03 20:33:49 +01:00
netutil.py scripted-diff: Update copyright in ./test 2019-03-02 10:58:35 -05:00
script.py [test] rename SegwitVersion1SignatureHash() 2019-10-14 17:13:05 -04:00
script_util.py test: fix "tx-size-small" errors after default address change 2019-10-14 15:03:11 +02:00
segwit_addr.py Use BIP173 addresses in segwit.py test 2017-09-28 17:29:01 -07:00
siphash.py Update copyright headers to 2018 2018-07-27 07:15:02 -04:00
socks5.py scripted-diff: Update copyright in ./test 2019-03-02 10:58:35 -05:00
test_framework.py Merge #17633: tests: Add option --valgrind to run the functional tests under Valgrind 2019-12-10 13:30:37 -05:00
test_node.py tests: Add option --valgrind to run nodes under valgrind in the functional tests 2019-12-01 19:53:30 +00:00
test_shell.py TestShell: Return self from setup() 2019-11-05 12:55:52 +01:00
util.py [qa] Add shrinkdebugfile=0 to regtest bitcoin.conf 2019-10-31 13:54:07 -04:00
wallet_util.py test: add rpc getaddressinfo labels test coverage 2019-11-24 23:08:38 +01:00