Commit graph

4398 commits

Author SHA1 Message Date
fanquake
28ce05d06f
Merge #19260: p2p: disconnect peers that send filterclear + update existing filter msg disconnect logic
3a10d935ac [p2p/refactor] move disconnect logic and remove misbehaving (gzhao408)
ff8c430c65 [test] test disconnect for filterclear (gzhao408)
1c6b787e03 [netprocessing] disconnect node that sends filterclear (gzhao408)

Pull request description:

  Nodes that don't have bloomfilters turned on (i.e. no `NODE_BLOOM` service) should disconnect peers that send them `filterclear` P2P messages.

  Non-bloomfilter nodes already disconnect peers for [`filteradd` and `filterload`](19e919217e/src/net_processing.cpp (L2218)), but #8709 removed `filterclear` so it could be used to reset tx relay. This isn't needed now because using `feefilter` message is much better for this purpose (See #19204).

  Also refactors existing disconnect logic for `filteradd` and `filterload` into respective message handlers and removes banning for them.

ACKs for top commit:
  jnewbery:
    Code review ACK 3a10d935ac
  naumenkogs:
    utACK 3a10d93
  gillichu:
    tested ACK: quick test_runner on macOS [`3a10d93`](3a10d935ac)
  MarcoFalke:
    re-ACK 3a10d935ac only change is replacing false with true 🚝

Tree-SHA512: 7aad8b3c0b0e776a47ad52544f0c1250feb242320f9a2962542f5905042f77e297a1486f8cdc3bf0fb93cd00c1ab66a67b2ec426eb6da3fe4cda56b5e623620f
2020-06-16 16:09:22 +08:00
Andrew Chow
951bca61d7 tests: feature_backwards_compatibility.py test 0.16 up/downgrade 2020-06-15 11:01:57 -04:00
MarcoFalke
fa6ef701ad
util: Add Assert identity function
The utility is primarily useful to dereference pointer types, which are
known to be not null at that time.

For example, the ArgsManager is known to exist when the wallets are
started. Instead of silently relying on that assumption, Assert can be
used to abort the program and avoid UB should the assumption ever be
violated.
2020-06-15 07:39:08 -04:00
gzhao408
ff8c430c65 [test] test disconnect for filterclear 2020-06-14 11:47:49 -07:00
John Newbery
49236be099 [tests] Don't import asyncio to test magic bytes 2020-06-13 10:49:31 -04:00
Martin Ankerl
78c312c983 Replace current benchmarking framework with nanobench
This replaces the current benchmarking framework with nanobench [1], an
MIT licensed single-header benchmarking library, of which I am the
autor. This has in my opinion several advantages, especially on Linux:

* fast: Running all benchmarks takes ~6 seconds instead of 4m13s on
  an Intel i7-8700 CPU @ 3.20GHz.

* accurate: I ran e.g. the benchmark for SipHash_32b 10 times and
  calculate standard deviation / mean = coefficient of variation:

  * 0.57% CV for old benchmarking framework
  * 0.20% CV for nanobench

  So the benchmark results with nanobench seem to vary less than with
  the old framework.

* It automatically determines runtime based on clock precision, no need
  to specify number of evaluations.

* measure instructions, cycles, branches, instructions per cycle,
  branch misses (only Linux, when performance counters are available)

* output in markdown table format.

* Warn about unstable environment (frequency scaling, turbo, ...)

* For better profiling, it is possible to set the environment variable
  NANOBENCH_ENDLESS to force endless running of a particular benchmark
  without the need to recompile. This makes it to e.g. run "perf top"
  and look at hotspots.

Here is an example copy & pasted from the terminal output:

|             ns/byte |              byte/s |    err% |        ins/byte |        cyc/byte |    IPC |       bra/byte |   miss% |     total | benchmark
|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|---------------:|--------:|----------:|:----------
|                2.52 |      396,529,415.94 |    0.6% |           25.42 |            8.02 |  3.169 |           0.06 |    0.0% |      0.03 | `bench/crypto_hash.cpp RIPEMD160`
|                1.87 |      535,161,444.83 |    0.3% |           21.36 |            5.95 |  3.589 |           0.06 |    0.0% |      0.02 | `bench/crypto_hash.cpp SHA1`
|                3.22 |      310,344,174.79 |    1.1% |           36.80 |           10.22 |  3.601 |           0.09 |    0.0% |      0.04 | `bench/crypto_hash.cpp SHA256`
|                2.01 |      496,375,796.23 |    0.0% |           18.72 |            6.43 |  2.911 |           0.01 |    1.0% |      0.00 | `bench/crypto_hash.cpp SHA256D64_1024`
|                7.23 |      138,263,519.35 |    0.1% |           82.66 |           23.11 |  3.577 |           1.63 |    0.1% |      0.00 | `bench/crypto_hash.cpp SHA256_32b`
|                3.04 |      328,780,166.40 |    0.3% |           35.82 |            9.69 |  3.696 |           0.03 |    0.0% |      0.03 | `bench/crypto_hash.cpp SHA512`

[1] https://github.com/martinus/nanobench

* Adds support for asymptotes

  This adds support to calculate asymptotic complexity of a benchmark.
  This is similar to #17375, but currently only one asymptote is
  supported, and I have added support in the benchmark `ComplexMemPool`
  as an example.

  Usage is e.g. like this:

  ```
  ./bench_bitcoin -filter=ComplexMemPool -asymptote=25,50,100,200,400,600,800
  ```

  This runs the benchmark `ComplexMemPool` several times but with
  different complexityN settings. The benchmark can extract that number
  and use it accordingly. Here, it's used for `childTxs`. The output is
  this:

  | complexityN |               ns/op |                op/s |    err% |          ins/op |          cyc/op |    IPC |     total | benchmark
  |------------:|--------------------:|--------------------:|--------:|----------------:|----------------:|-------:|----------:|:----------
  |          25 |        1,064,241.00 |              939.64 |    1.4% |    3,960,279.00 |    2,829,708.00 |  1.400 |      0.01 | `ComplexMemPool`
  |          50 |        1,579,530.00 |              633.10 |    1.0% |    6,231,810.00 |    4,412,674.00 |  1.412 |      0.02 | `ComplexMemPool`
  |         100 |        4,022,774.00 |              248.58 |    0.6% |   16,544,406.00 |   11,889,535.00 |  1.392 |      0.04 | `ComplexMemPool`
  |         200 |       15,390,986.00 |               64.97 |    0.2% |   63,904,254.00 |   47,731,705.00 |  1.339 |      0.17 | `ComplexMemPool`
  |         400 |       69,394,711.00 |               14.41 |    0.1% |  272,602,461.00 |  219,014,691.00 |  1.245 |      0.76 | `ComplexMemPool`
  |         600 |      168,977,165.00 |                5.92 |    0.1% |  639,108,082.00 |  535,316,887.00 |  1.194 |      1.86 | `ComplexMemPool`
  |         800 |      310,109,077.00 |                3.22 |    0.1% |1,149,134,246.00 |  984,620,812.00 |  1.167 |      3.41 | `ComplexMemPool`

  |   coefficient |   err% | complexity
  |--------------:|-------:|------------
  |   4.78486e-07 |   4.5% | O(n^2)
  |   6.38557e-10 |  21.7% | O(n^3)
  |   3.42338e-05 |  38.0% | O(n log n)
  |   0.000313914 |  46.9% | O(n)
  |     0.0129823 | 114.4% | O(log n)
  |     0.0815055 | 133.8% | O(1)

  The best fitting curve is O(n^2), so the algorithm seems to scale
  quadratic with `childTxs` in the range 25 to 800.
2020-06-13 12:24:18 +02:00
MarcoFalke
f154071ec8
Merge #19177: test: Fix and clean p2p_invalid_messages functional tests
af2a145e57 Refactor resource exhaustion test (Troy Giorshev)
5c4648d17b Fix "invalid message size" test (Troy Giorshev)
ff1e7b8844 Move size limits to module-global (Troy Giorshev)
57890abf2c Remove two unneeded tests (Troy Giorshev)

Pull request description:

  This PR touches only the p2p_invalid_messages.py functional test module.  There are two main goals accomplished here.  First, it fixes the "invalid message size" test, which previously made a message that was invalid for multiple reasons.  Second, it refactors the file into a single consistent style.  This file appears to have originally had two authors, with different styles and some test duplication.

  It should now be easier and quicker to understand this module, anticipating the upcoming [BIP324](https://github.com/bitcoin/bitcoin/pull/18242) and [AltNet](https://github.com/bitcoin/bitcoin/issues/18989) changes.

  This should probably go in ahead of #19107, but the two are not strictly related.

ACKs for top commit:
  jnewbery:
    ACK af2a145e57
  MarcoFalke:
    re-ACK af2a145e57 🍦

Tree-SHA512: 9b57561e142c5eaefac5665f7355c8651670400b4db1a89525d2dfdd20e872d6873c4f6175c4222b6f5a8e5210cf5d6a52da69b925b673a2e2ac30a15d670d1c
2020-06-12 15:29:53 -04:00
MarcoFalke
5af16a4db7
Merge #16756: test: Connection eviction logic tests
45eff751c6 Add functional test for P2P eviction logic of inbound peers (Martin Zumsande)

Pull request description:

  This adds a functional test for the eviction logic for inbound peers, which is triggered when the number of maximum connections is exceeded.

  The functional test covers eviction protection for peers that  have sent us blocks or txns recently, or that have faster pings. I couldn't find a way to test the logic of `CConnman::AttemptToEvictConnection` that is based on netgroup (see #14210 for related discussion)

  Fixes #16660 (at least partially).

  [Edit: Earlier, this PR also contained a unit test, which was removed after the discussion]

ACKs for top commit:
  jonatack:
    ACK 45eff751c6
  naumenkogs:
    Tested ACK 45eff75
  fjahr:
    re-ACK 45eff751c6
  andrewtoth:
    re-ACK 45eff751c6

Tree-SHA512: 177208ab6f30dc62da1cc5f51e654f7c9770d8c6b42aca6ae7ecb30e29d3096e04d75739578e7d149a0f29dd92652b4a707e93c0f1be8aa7ed315e6ec3ab07a4
2020-06-12 14:48:20 -04:00
MarcoFalke
19e919217e
Merge #19250: wallet: Make RPC help compile-time static
fadf6bd04f refactor: Remove unused request.fHelp (MarcoFalke)
fad889cbf0 wallet: Make RPC help compile-time static (MarcoFalke)

Pull request description:

  Currently calling `help` on a wallet RPC method will either return `help: unknown command: getnewaddress` or the actual help. This runtime dependency of the help is a bug that complicates any tool that relies on documentation. Also, the code that enables the bug is overly complicated and confusing.

  The fix is split into two commits:

  * First, a commit that can be reviewed with the `--color-moved=dimmed-zebra` option and tested with the included test.
  * Second, a commit that removes the complicated and confusing code.

ACKs for top commit:
  achow101:
    re-ACK fadf6bd04f
  promag:
    Tested ACK fadf6bd04f.

Tree-SHA512: 65d4ff400467f57cb8415c30ce30f814dc76c5c157308b7a7409c59ac9db629e65dfba31cd9c389cfe60a008d3d87787ea0a0e0f2671fd65fd190543c915493d
2020-06-12 13:04:37 -04:00
Sjors Provoost
a4a3fc4cd2
doc: improve subtree check instructions 2020-06-12 13:31:42 +02:00
MarcoFalke
b33136b6ba
Merge #19083: test: msg_mempool, fRelay, and other bloomfilter tests
dca73941eb scripted-diff: rename node to peer for mininodes (gzhao408)
0474ea25af [test] fix race conditions and test in p2p_filter (gzhao408)
4ef80f0827 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect (gzhao408)
497a619386 [test] add BIP 37 test for node with fRelay=false (gzhao408)
e8acc60156 [test] add mempool msg test for node with bloomfilter enabled (gzhao408)

Pull request description:

  This PR adds a few tests that are bloomfilter-related, including behavior for when bloomfilters are turned _off_:
  1. Tests p2p message `msg_mempool`: a node that has `peerbloomfilters` enabled should send its mempool (disabled behavior already tested [here](https://github.com/bitcoin/bitcoin/blob/master/test/functional/p2p_mempool.py)).
  2. Tests that bloomfilter peers with [`fRelay=False`](https://github.com/bitcoin/bips/blob/master/bip-0037.mediawiki#extensions-to-existing-messages) in the `version` message should not receive any invs until they set the filter. The rest is the same as what’s already tested in `p2p_filter.py`.
  3. Tests that peers get disconnected if they send `filterload` or `filteradd` p2p messages to a node with bloom filters disabled.
  4. Refactor: renames p2p_mempool.py to p2p_nobloomfilter_messages.py.
  5. Fixes race conditions in p2p_filter.py

ACKs for top commit:
  MarcoFalke:
    ACK dca73941eb only changes is restoring accidentally deleted test 🍮
  jonatack:
    ACK dca73941eb modulo a few nits if you retouch, happy to re-ACK if you take any of them but don't feel obliged to.

Tree-SHA512: 442aeab0755cb8b830251ea170d1d5e6da8ac9029b3276d407a20ee3d588cc61b77b8842368de18c244056316b8c63b911776d6e106bc7c023439ab915b27ad3
2020-06-11 14:35:08 -04:00
MarcoFalke
85f7db2284
Merge #19239: tests: move generate_wif_key to wallet_util.py
3a83a01694 [tests] move generate_wif_key to wallet_util.py (John Newbery)
b216b0b71f [tests] sort imports in rpc_createmultisig.py (John Newbery)
e38081846d Revert "[TESTS] Move base58 to own module to break circular dependency" (John Newbery)

Pull request description:

  generate_wif_key is a wallet utility function. Move it from the EC key module to the wallet util module.

  This fixes the circular dependency issue in #17977

ACKs for top commit:
  MarcoFalke:
    ACK 3a83a01694 🍪

Tree-SHA512: 24985dffb75202721ccc0c6c5b52f1fa5d1ce7963bccde24389feb913cab4dad0c265274ca67892c46c8b64e6a065a0f23263a89be4fb9134dfefbdbe5c7238a
2020-06-11 12:49:38 -04:00
MarcoFalke
fad889cbf0
wallet: Make RPC help compile-time static 2020-06-11 12:38:36 -04:00
fanquake
45a6811d36
Merge #19206: test: Remove leftover comment in mining_basic
fa98e10d5e test: Remove leftover comment in mining_basic (MarcoFalke)
faedb50d89 test: pep-8 mining_basic (MarcoFalke)

Pull request description:

  Remove an accidental leftover comment from #19082, which no longer applies and thus might be confusing

ACKs for top commit:
  adamjonas:
    code review ACK fa98e10

Tree-SHA512: c7f7f8f579b3c6e92f45769be0a7af1a421438a3f5524db5278b2269511a9e0e08f44e3836afb26727644035897ee51ff8296d13ce23030549e7403f57b40e40
2020-06-11 16:38:11 +08:00
John Newbery
3a83a01694 [tests] move generate_wif_key to wallet_util.py
generate_wif_key is a wallet utility function. Move
it from the EC key module to the wallet util module.
2020-06-10 12:10:02 -04:00
John Newbery
b216b0b71f [tests] sort imports in rpc_createmultisig.py 2020-06-10 11:56:44 -04:00
John Newbery
e38081846d Revert "[TESTS] Move base58 to own module to break circular dependency"
This reverts commit c75de5da5f.
2020-06-10 11:54:25 -04:00
gzhao408
dca73941eb scripted-diff: rename node to peer for mininodes
-BEGIN VERIFY SCRIPT-
sed -i 's/FilterNode/P2PBloomFilter/g' test/functional/p2p_filter.py;
sed -i 's/filter_node/filter_peer/g' test/functional/p2p_filter.py;
-END VERIFY SCRIPT-
2020-06-10 07:28:45 -07:00
gzhao408
0474ea25af [test] fix race conditions and test in p2p_filter
-grab mininode_lock for every access to mininode attributes,
otherwise there are race conditions
2020-06-10 07:28:04 -07:00
gzhao408
4ef80f0827 [test] sending invalid msgs to node with bloomfilters=0 causes disconnect
-A node with bloomfilters disabled should disconnect peers that send
msg_mempool, msg_filterload, or msg_filteradd.
-Renamed the test because it now has a wider scope and msg_mempool's
actual functionality makes more sense for p2p_filter.py.
2020-06-10 07:28:04 -07:00
gzhao408
497a619386 [test] add BIP 37 test for node with fRelay=false
A node with fRelay=False should not receive any invs until filter is set using
filterload; all other behavior should be identical.
2020-06-10 07:28:04 -07:00
gzhao408
e8acc60156 [test] add mempool msg test for node with bloomfilter enabled
-msg_mempool is currently only tested with bloomfilter disabled
(node is disconnected) in p2p_mempool.py
-msg_mempool should get mempool txns in response when bloomfilter
is enabled
-edit test that doesn't test msg_mempool as intended
2020-06-10 07:27:58 -07:00
MarcoFalke
6762a627ec
Merge #19230: [TESTS] Move base58 to own module to break circular dependency
c75de5da5f [TESTS] Move base58 to own module to break circular dependency (Pieter Wuille)

Pull request description:

  I encountered difficulties with the test framework in #17977. This fixes them, and I think the change is generally useful.

ACKs for top commit:
  laanwj:
    Code review ACK c75de5da5f
  MarcoFalke:
    ACK c75de5da5f according to --color-moved=dimmed-zebra this is a move-only apart from the imports 👒

Tree-SHA512: 9e0493de3e279074f0c70e92c959b73ae30479ad6f2083a3c6bbf4b0191d65ef94854559a5b7c904f5dadc5e93129ed00f6dc0a8ccce6ba7921cd45f7119f74b
2020-06-10 10:23:58 -04:00
Wladimir J. van der Laan
bc933aeaf0
Merge #19226: test: Add BerkeleyDatabase tsan suppression
fa7b46cc91 test: Add BerkeleyDatabase tsan suppression (MarcoFalke)

Pull request description:

  Suppresses/Fixes #19211 for now

ACKs for top commit:
  laanwj:
    ACK fa7b46cc91
  practicalswift:
    ACK fa7b46cc91 -- patch looks correct

Tree-SHA512: 749e606caf0f140c1a190e3273ff81d220daa3eb004ba2b2078e6b3c5b6ac196bd5fc91ef42841412cfd4fe1e2a8694fc2a28268fde8485db90076593fc51dc7
2020-06-10 15:25:13 +02:00
Ivan Metlushko
16d4b3fd6d test: mempool.dat compatibility between versions 2020-06-10 11:06:03 +07:00
Pieter Wuille
c75de5da5f [TESTS] Move base58 to own module to break circular dependency
This breaks the script->key->address->script dependency cycle.
2020-06-09 17:50:50 -07:00
TrentZ
6fc641644f
change blacklist to blocklist
Let's use a more appropriate and clear word and discard the usage of the blacklist. Blocklist is clear. Happy for everyone.
2020-06-09 15:08:41 -05:00
MarcoFalke
fa7b46cc91
test: Add BerkeleyDatabase tsan suppression 2020-06-09 14:15:04 -04:00
MarcoFalke
297466b49f
Merge #18826: Expose txinwitness for coinbase in JSON form from RPC
34645c4dd0 Test txinwitness is accessible on coinbase vin (Rod Vagg)
3e4421070a Expose txinwitness for coinbase in JSON form (Rod Vagg)

Pull request description:

  ## Rationale

  The CLI can provide you with everything about transactions and blocks that you need to reconstruct the block structure and raw block itself **except** for the witness commitment nonce which is stored in the `scriptWitness` of the coinbase and is not printed. You could manually parse the raw `"hex"` fields for transactions if you really wanted to, but this seems to defeat the point of having a JSONification of the raw block/transaction data.

  Without the nonce you can't:

  1. calculate and validate the witness commitment yourself, you can generate the witness tx merkle root but you don't have the nonce to combine it with
  2. reconstruct the raw block form because you don't have `scriptWitness` stack associated with the coinbase (although you know how big it will be and can guess the common case of `[0x000...000]`)

  I'm building some archiving tooling for block data and being able to do a validated two-way conversion is very helpful.

  ## What

  This PR simply makes the `txinwitness` field not dependent on whether we are working with the coinbase or not. So you get it for the coinbase as well as the rest.

  ## Examples

  Common case of a `[0x000...000]` nonce: 00000000000000000000140a7289f3aada855dfd23b0bb13bb5502b0ca60cdd7

  ```json
        "vin": [
          {
            "coinbase": "0368890904c1fe8d5e2f706f6f6c696e2e636f6d2ffabe6d6d5565843a681160cf7b08b1b74ac90a719e6d6ab28c16d336b924f0dc2fcabdc6010000000000000051bf2ad74af345dbe642154b2658931612a70d195e007add0100ffffffff",
            "txinwitness": [
              "0000000000000000000000000000000000000000000000000000000000000000"
            ],
            "sequence": 4294967295
          }
        ],
  ...
  ```

  Novel nonce value: 000000000000000000008c31945b2012258366cc600a3e9a3ee0598e8f797731

  ```json
        "vin": [
          {
            "coinbase": "031862082cfabe6d6d80c099b5e21f4c186d54eb292e17026932e52b1b807fa1380574c5adc1c843450200000000000000",
            "txinwitness": [
              "5b5032506f6f6c5d5b5032506f6f6c5d5b5032506f6f6c5d5b5032506f6f6c5d"
            ],
            "sequence": 4294967295
          }
        ],
  ...
  ```

  ## Alternatives

  This field could be renamed for the coinbase, `"witnessnonce"` perhaps. It could also be omitted when null/zero (`0x000...000`).

  ## Tests

  This didn't break any tests and I couldn't find an obvious way to include a test for this. If this is desired I'd apreicate some pointers.

ACKs for top commit:
  MarcoFalke:
    ACK 34645c4dd0

Tree-SHA512: b192facc1dfd210a5ec3f0d5d1ac6d0cae81eb35be15eaa71f60009a538dd6a79ab396f218434e7e998563f7f0df2c396cc925cb91619f6841c5a67806148c85
2020-06-08 10:18:42 -04:00
MarcoFalke
b3ec1fe811
Merge #18890: test: disconnect_nodes should warn if nodes were already disconnected
34e641a564 test: Remove unnecessary disconnect_nodes call in rpc_psbt.py (Danny Lee)
e6e7abd51a test: remove redundant two-way disconnect_nodes calls (Danny Lee)
a9bd1f9adf test: warn if nodes not connected before disconnect_nodes (Danny Lee)

Pull request description:

  There's no harm in calling `disconnect_nodes` for nodes that weren't connected (in this case it's a no-op).  However, detecting this case and logging a warning can help ensure that tests are behaving as expected.

  In addition, since `disconnect_nodes` works bidirectionally, I removed all instances of this pattern:

  ```
  disconnect_nodes(self.nodes[0], 1)
  disconnect_nodes(self.nodes[1], 0)
  ```

ACKs for top commit:
  MarcoFalke:
    review ACK 34e641a564 👔
  amitiuttarwar:
    ACK 34e641a564. Thanks for this test improvement!

Tree-SHA512: 344855ceb46c012d43c13d7c09f44d32dcb7645706d10ae1e4645d9edca54c6c6c13fee26b79480755cdfcdf39b4b5770b36bb03ce71ba002d5be8a27fe008af
2020-06-08 09:13:22 -04:00
Troy Giorshev
af2a145e57 Refactor resource exhaustion test
This is a simple refactor of the specified test.  It is now brought in
line with the rest of the tests in the module.  This should make things
easier to debug, as all of the tests are now grouped together at the
top.
2020-06-08 08:52:39 -04:00
MarcoFalke
fa98e10d5e
test: Remove leftover comment in mining_basic 2020-06-08 08:10:37 -04:00
MarcoFalke
faedb50d89
test: pep-8 mining_basic
Can be reviewed with the git options
--word-diff-regex=. --ignore-all-space  -U0
2020-06-08 08:10:23 -04:00
Jon Atack
a8507c99da
rpc: remove deprecated getaddressinfo labels: purpose 2020-06-08 10:38:31 +02:00
Jon Atack
645a8653c8
rpc: remove deprecated getaddressinfo label field 2020-06-08 10:38:29 +02:00
Troy Giorshev
5c4648d17b Fix "invalid message size" test
This test originally made a message with an invalid stated length, and
an invalid checksum.  This was because only the header was changed, but
the checksum stayed the same.  This was fine for now because we check
the header first to see if it has a valid stated size, and we disconnect
if it does not, so we never end up checking for the checksum.  If this
behavior was to change, this test would become a problem.  (Indeed I
discovered this when playing around with this behavior).  By instead
creating a message with an oversized payload from the start, we create a
message with an invalid stated length but a valid checksum, as intended.

Additionally, this takes advantage to the newly module-global
VALID_DATA_LIMIT as opposed to the magic 0x02000000.  Yes, 4MB < 32MiB,
but at the moment when receiving a message we check both, so this makes
the test tighter.
2020-06-08 00:55:34 -04:00
Troy Giorshev
ff1e7b8844 Move size limits to module-global
As well, this renames those variables to match PEP8 and this clears up
the comment relating to VALID_DATA_LIMIT.

Admittedly, this commit is mainly to make the following ones cleaner.
2020-06-08 00:54:52 -04:00
Troy Giorshev
57890abf2c Remove two unneeded tests
Test 1 is a duplicate of test_size() later in the file.  Inexplicably,
this test does not work on macOS, whereas test_size() does.

Test 2 is problematic for two reasons.  First, it always fails with an
invalid checksum, which is probably not what was intended.  Second, it's
not defined at this layer what the behavior should be.  Hypothetically,
if this test was fixed so that it gave messages with valid checksums,
then the message would pass successfully thought the network layer and
fail only in the processing layer.  A priori the network layer has no
idea what the size of a message "actually" is.

The "Why does behavior change at 78 bytes" is because of the following:

print(len(node.p2p.build_message(msg))) # 125
=> Payload size = 125 - 24 = 101
If we take 77 bytes, then there are 101 - 77 = 24 left
That's exactly the size of a header
So, bitcoind deserializes the header and rejects it for some other reason
(Almost always an invalid size (too large))
But, if we take 78 bytes, then there are 101 - 78 = 23 left
That's not enough to fill a header, so the socket stays open waiting for
more data.  That's why we sometimes have to push additional data in
order for the peer to disconnect.

Additionally, both of these tests use the "conn" variable.  For fun, go
look at where it's declared.  (Hint: test_large_inv().  Don't we all
love python's idea of scope?)
2020-06-08 00:54:43 -04:00
Martin Zumsande
45eff751c6 Add functional test for P2P eviction logic of inbound peers 2020-06-06 22:30:29 +02:00
MarcoFalke
b1b1739944
Merge #18968: doc: noban precludes maxuploadtarget disconnects
fa9604c46f doc: noban precludes maxuploadtarget disconnects (MarcoFalke)
fa3999fe35 net: Reformat excessively long if condition into multiple lines (MarcoFalke)

Pull request description:

  Whitelisting has been replaced by permission flags, so properly document this. See also #10131

ACKs for top commit:
  hebasto:
    ACK fa9604c46f, I have reviewed the code and it looks OK, I agree it can be merged.
  ariard:
    ACK fa9604c

Tree-SHA512: 5aee917ab9817719f01ec155487542118e17fa3d145ae7e4bc0e872b2cec39cde9e7fbdee2ae77e9a52700dd8bcc366de4224152e08e709d44d08e0d2f19c613
2020-06-06 09:51:21 -04:00
fanquake
17cfa52d38
Merge #19172: test: Do not swallow flake8 exit code
5d77549d8b doc: Add mypy to test dependencies (Hennadii Stepanov)
7dda912e1c test: Do not swallow flake8 exit code (Hennadii Stepanov)

Pull request description:

  After #18210 the `flake8` exit code in `test/lint/lint-python.sh` just not used that makes the linter broken.

  This PR:
  - combines exit codes of `flake8` and `mypy` into the  `test/lint/lint-python.sh` exit code
  - documents `mypy` as the test dependency

ACKs for top commit:
  MarcoFalke:
    Approach ACK 5d77549d8b, fine with me
  practicalswift:
    ACK 5d77549d8b

Tree-SHA512: e948ba04dc4d73393967ebf3c6a26c40d428d33766382a0310fc64746cb7972e027bd62e7ea76898b742a656cf7d0fcda2fdd61560a21bfd7be249cea27f3d41
2020-06-06 14:25:08 +08:00
John Newbery
62068381a3 [tests] Make mininode_lock non-reentrant
There's no need for mininode_lock to be reentrant.
Use a simpler non-recursive lock.
2020-06-05 11:01:54 -04:00
John Newbery
c67c1f2c03 [tests] Don't call super twice in P2PTxInvStore.on_inv() 2020-06-05 10:59:40 -04:00
John Newbery
9d80762fa0 [tests] Don't acquire mininode_lock twice in wait_for_broadcast() 2020-06-05 10:58:25 -04:00
John Newbery
edae6075aa [tests] Only acquire lock once in p2p_compactblocks.py 2020-06-05 10:57:13 -04:00
Hennadii Stepanov
5d77549d8b
doc: Add mypy to test dependencies 2020-06-05 16:16:16 +03:00
Hennadii Stepanov
7dda912e1c
test: Do not swallow flake8 exit code 2020-06-05 16:15:31 +03:00
fanquake
b55b5b6c3d
Merge #19164: ci: tsan with wallet
fa7e002d52 ci: tsan with wallet (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK fa7e002d52 -- patch looks correct and Travis is happy
  hebasto:
    ACK fa7e002d52, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 1138459bbef72f402f32dae1e28d96f174901d4248d959b538b973747c8b06f221ecd81a386a1f915c0a2faaefb9fb8f11e3be39e6c5e2468bf9ae43d9f97757
2020-06-05 16:17:27 +08:00
fanquake
4ede05d421
Merge #18758: Remove unused boost/thread
89f9fef1f7 refactor: Specify boost/thread/thread.hpp explicitly (Hennadii Stepanov)
fad8c890f5 txdb: Remove unused boost/thread (MarcoFalke)
faa958bc28 txindex: Remove unused boost/thread (MarcoFalke)

Pull request description:

  There are predefined interruption points for `boost::thread`: https://www.boost.org/doc/libs/1_71_0/doc/html/thread/thread_management.html#interruption_points

  However, non-boost threads such as `std::thread` or the `main()` thread can obviously not be interrupted. So remove all unused boost/thread from methods that are never executed in a `boost::thread`.

  Most of them were accompanied by a `ShutdownRequested` anyway. So even if the current thread was a `boost::thread`, the interruption point would be redundant. (We only interrupt threads during shutdown)

ACKs for top commit:
  fanquake:
    ACK 89f9fef1f7
  hebasto:
    ACK 89f9fef1f7, tested on Linux Mint 19.3 (x86_64), verified shutdown in different scenarios.

Tree-SHA512: 17221dadedf2d107e5bda9e4f371cc4f8ffce6ad27cae41aa2b8f1150d8f1adf23d396585ca4a2dd25b1dc6f0d5c81fecd950d8557966ccb45a6d4a85a331d90
2020-06-05 11:01:39 +08:00
MarcoFalke
fa7e002d52
ci: tsan with wallet 2020-06-04 18:26:01 -04:00