Commit graph

1752 commits

Author SHA1 Message Date
Wladimir J. van der Laan
2c2d988062
Merge #10546: Remove 33 unused Boost includes
49de096 Remove unused Boost includes (practicalswift)

Tree-SHA512: ff2dad94f9eeb1dc50fcffd0e94e1686be04e4e5bc45f58ae955d630c15cd25bb8f5583d0aa1f2f263b5a723be79747ef3c6e6b417c7be7787c0abc8d1874019
2017-06-11 15:37:15 +02:00
Pieter Wuille
76f268b9bd
Merge #10521: Limit variable scope
90593ed92 Limit variable scope (practicalswift)

Tree-SHA512: 4719e303688a31aefbe1d239e86b21dd3c2045524e08bd628c6ba0c6c2a97de14d04305b9beafe0b1dcde7229793e6663168953f192e88ed409be5c30fd2a9a9
2017-06-09 13:09:24 -07:00
practicalswift
49de096c2a Remove unused Boost includes 2017-06-09 10:25:26 +02:00
Wladimir J. van der Laan
29f80cd230
Merge #10545: Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of
3fb81a8 Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of (practicalswift)

Tree-SHA512: 63a9ac9ec5799472943dce1cd92a4b14e7f1fe12758a5fc4b1efceaf2c85a4ba71dad5ccc50813527f18b192e7714c076e2478ecd6ca0d452b24e88416f872f7
2017-06-08 20:26:40 +02:00
Russell Yanofsky
3ff1fa8c4a Use override keyword on CCoinsView overrides 2017-06-08 09:28:28 -04:00
Wladimir J. van der Laan
9c248e39f2
Merge #10481: Decodehextx scripts sanity check
ac4e438 Sanity check transaction scripts in DecodeHexTx (Andrew Chow)
5b75c47 Add a valid opcode sanity check to CScript (Andrew Chow)

Tree-SHA512: a516e95c274c9d131123150c798cae8bed75925e8a4d59469967dd7f1d49f7f8161e26afb61024b821bd8dcdffdfd0d19b8dcad20b39b1106820326d8d56904d
2017-06-08 13:36:48 +02:00
Wladimir J. van der Laan
71ab6e5538
Merge #10547: [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution}
227ae9b [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution} (practicalswift)

Tree-SHA512: 1bde6c8b9498051fa2eae4913eb1f5411adea8dea1511c0df859aea57a2a7db6f5839945ddf2eccdddfa322bceacad35a5d875742db7d15e40dbea83185307bb
2017-06-08 12:39:43 +02:00
practicalswift
30c2d9db48 [tests] Remove unused function InsecureRandBytes(size_t len) 2017-06-08 09:43:54 +02:00
Wladimir J. van der Laan
6c2d81f34d
Merge #10524: [tests] Remove printf(...)
0abc588 [tests] Remove printf(...) (practicalswift)

Tree-SHA512: a56fa1edce050f5a44a53842223fc99e4e8ade413047e04f2064faeb5c4a418864a94471853d5c26a20608d1dcc18b9226d7fe15172f79a66cc5cf6e1443f7e9
2017-06-08 09:43:09 +02:00
Pieter Wuille
e801084dec
Merge #10321: Use FastRandomContext for all tests
e94584858 scripted-diff: Use new naming style for insecure_rand* functions (Pieter Wuille)
2fcd9cc86 scripted-diff: Use randbits/bool instead of randrange where possible (Pieter Wuille)
2ada67852 Use randbits instead of ad-hoc emulation in prevector tests (Pieter Wuille)
5f0b04eed Replace rand() & ((1 << N) - 1) with randbits(N) (Pieter Wuille)
3ecabae36 Replace more rand() % NUM by randranges (Pieter Wuille)
efee1db21 scripted-diff: use insecure_rand256/randrange more (Pieter Wuille)
1119927df Add various insecure_rand wrappers for tests (Pieter Wuille)
124d13a58 Merge test_random.h into test_bitcoin.h (Pieter Wuille)
90620d66c scripted-diff: Rename cuckoo tests' local rand context (Pieter Wuille)
37e864eb9 Add FastRandomContext::rand256() and ::randbytes() (Pieter Wuille)

Tree-SHA512: d09705a3ec718ae792f7d66a75401903ba7b9c9d3fc36669d6e3b9242f0194738106be26baefc8a8e3fa6df7c9a35978c71c0c430278a028b331df23a3ea3070
2017-06-07 15:12:14 -07:00
Andrew Chow
5b75c47784 Add a valid opcode sanity check to CScript
Added a function in CScript that checks if the script contains valid opcodes.

Add a test for that function
2017-06-07 12:40:01 -07:00
Pieter Wuille
e945848582 scripted-diff: Use new naming style for insecure_rand* functions
-BEGIN VERIFY SCRIPT-
sed -i 's/\<insecure_randbits(/InsecureRandBits(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_randbool(/InsecureRandBool(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_randrange(/InsecureRandRange(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_randbytes(/InsecureRandBytes(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_rand256(/InsecureRand256(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<insecure_rand(/InsecureRand32(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
sed -i 's/\<seed_insecure_rand(/SeedInsecureRand(/g' src/test/*.cpp src/test/*.h src/wallet/test/*.cpp
-END VERIFY SCRIPT-
2017-06-07 12:08:39 -07:00
practicalswift
0abc58890c [tests] Remove printf(...) 2017-06-07 20:42:54 +02:00
practicalswift
227ae9b34d [tests] Use FastRandomContext instead of boost::random::{mt19937,uniform_int_distribution} 2017-06-07 20:38:03 +02:00
Pieter Wuille
2fcd9cc86b scripted-diff: Use randbits/bool instead of randrange where possible
-BEGIN VERIFY SCRIPT-
sed -i 's/insecure_randbits(1)/insecure_randbool()/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(2)/insecure_randbool()/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(4)/insecure_randbits(2)/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(32)/insecure_randbits(5)/g' src/test/*_tests.cpp
sed -i 's/insecure_randrange(256)/insecure_randbits(8)/g' src/test/*_tests.cpp
-END VERIFY SCRIPT-
2017-06-07 11:35:16 -07:00
Pieter Wuille
2ada678521 Use randbits instead of ad-hoc emulation in prevector tests 2017-06-07 11:35:16 -07:00
Pieter Wuille
5f0b04eedc Replace rand() & ((1 << N) - 1) with randbits(N) 2017-06-07 11:35:13 -07:00
Pieter Wuille
3ecabae363 Replace more rand() % NUM by randranges 2017-06-07 11:34:55 -07:00
Matt Corallo
b5fea8d0cc Cache full script execution results in addition to signatures
This adds a new CuckooCache in validation, caching whether all of a
transaction's scripts were valid with a given set of script flags.

Unlike previous attempts at caching an entire transaction's
validity, which have nearly universally introduced consensus
failures, this only caches the validity of a transaction's
scriptSigs. As these are pure functions of the transaction and
data it commits to, this should be much safer.

This is somewhat duplicative with the sigcache, as entries in the
new cache will also have several entries in the sigcache. However,
the sigcache is kept both as ATMP relies on it and because it
prevents malleability-based DoS attacks on the new higher-level
cache. Instead, the -sigcachesize option is re-used - cutting the
sigcache size in half and using the newly freed memory for the
script execution cache.

Transactions which match the script execution cache never even have
entries in the script check thread's workqueue created.

Note that the cache is indexed only on the script execution flags
and the transaction's witness hash. While this is sufficient to
make the CScriptCheck() calls pure functions, this introduces
dependancies on the mempool calculating things such as the
PrecomputedTransactionData object, filling the CCoinsViewCache, etc
in the exact same way as ConnectBlock. I belive this is a reasonable
assumption, but should be noted carefully.

In a rather naive benchmark (reindex-chainstate up to block 284k
with cuckoocache always returning true for contains(),
-assumevalid=0 and a very large dbcache), this connected blocks
~1.7x faster.
2017-06-07 11:02:36 -04:00
practicalswift
3fb81a8480 Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of 2017-06-06 21:29:16 +02:00
Wladimir J. van der Laan
300f8e7a82
Merge #9909: tests: Add FindEarliestAtLeast test for edge cases
cf66f04 Add FindEarliestAtLeast test for edge cases (Russell Yanofsky)

Tree-SHA512: bed2e9b2eefe0014414f1be8f0ba236b1dcc6744a670ffa546100c62817dd06a117cc62cdff80c4988d2b374cae9a57f2bb135fc582ffb51ca2d34ab07bb3fe0
2017-06-06 15:51:45 +02:00
Pieter Wuille
efee1db21a scripted-diff: use insecure_rand256/randrange more
-BEGIN VERIFY SCRIPT-
sed -i "s/\<GetRandHash(/insecure_rand256(/" src/test/*_tests.cpp
sed -i "s/\<GetRand(/insecure_randrange(/" src/test/*_tests.cpp src/test/test_bitcoin.cpp
sed -i 's/\<insecure_rand() % \([0-9]\+\)/insecure_randrange(\1)/g' src/test/*_tests.cpp
-END VERIFY SCRIPT-
2017-06-05 12:45:44 -07:00
Pieter Wuille
1119927df0 Add various insecure_rand wrappers for tests 2017-06-05 12:44:44 -07:00
Pieter Wuille
124d13a58c Merge test_random.h into test_bitcoin.h 2017-06-05 12:44:44 -07:00
Pieter Wuille
90620d66c9 scripted-diff: Rename cuckoo tests' local rand context
-BEGIN VERIFY SCRIPT-
sed -i 's/insecure_rand/local_rand_ctx/g' src/test/cuckoocache_tests.cpp
-END VERIFY SCRIPT-
2017-06-05 12:44:44 -07:00
Pieter Wuille
37e864eb9f Add FastRandomContext::rand256() and ::randbytes()
FastRandomContext now provides all functionality that the real Rand* functions
provide.
2017-06-05 12:44:44 -07:00
Jorge Timón
7c00c26726
scripted-diff: Fully remove BOOST_FOREACH
-BEGIN VERIFY SCRIPT-
sed -i 's/BOOST_FOREACH *(\(.*\),/for (\1 :/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
-END VERIFY SCRIPT-
2017-06-05 20:10:50 +02:00
Wladimir J. van der Laan
e103b3ff1e
Merge #10514: Bugfix: missing == 0 after randrange
9aa215b Bugfixes: missing == 0 after randrange (Pieter Wuille)

Tree-SHA512: 160657ac09553f23ad7a3966c753a30ba938ce6f7ccfd34a4ef0d05d73d712362f7eef97e44a96e37a181b8347caa9d8e1584cc4485f69674ab2de3d8a247373
2017-06-05 16:09:59 +02:00
Wladimir J. van der Laan
a7e3c2814c rpc: Add listen address to incoming connections in getpeerinfo
This adds the listening address on which incoming connections were received to the
CNode and CNodeStats structures.

The address is reported in `getpeerinfo`.

This can be useful for distinguishing connections received on different listening ports
(e.g. when using a different listening port for Tor hidden service connections)
or different networks.
2017-06-05 13:35:30 +02:00
practicalswift
90593ed92c Limit variable scope 2017-06-05 00:52:36 +02:00
Pieter Wuille
9aa215bff9 Bugfixes: missing == 0 after randrange 2017-06-02 13:47:59 -07:00
Pieter Wuille
589827975f scripted-diff: various renames for per-utxo consistency
Thanks to John Newberry for pointing these out.

-BEGIN VERIFY SCRIPT-
sed -i 's/\<GetCoins\>/GetCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoins\>/HaveCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoinsInCache\>/HaveCoinInCache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<IsPruned\>/IsSpent/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<FetchCoins\>/FetchCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<CoinsEntry\>/CoinEntry/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxnToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<fHadTxInCache\>/had_coin_in_cache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<coinbaseids\>/coinbase_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<disconnectedids\>/disconnected_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<duplicateids\>/duplicate_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<oldcoins\>/old_coin/g' src/test/coins_tests.cpp
sed -i 's/\<origcoins\>/orig_coin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
-END VERIFY SCRIPT-
2017-06-01 13:15:25 -07:00
Pieter Wuille
73de2c1ff3 Rename CCoinsCacheEntry::coins to coin 2017-06-01 13:15:25 -07:00
Pieter Wuille
ce23efaa5c Extend coins_tests 2017-06-01 12:59:41 -07:00
Pieter Wuille
5083079688 Switch CCoinsView and chainstate db from per-txid to per-txout
This patch makes several related changes:
* Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...)
  to be COutPoint/Coin-based rather than txid/CCoins-based.
* Changes the chainstate db to a new incompatible format that is also
  COutPoint/Coin based.
* Implements reconstruction code for hash_serialized_2.
* Adapts the coins_tests unit tests (thanks to Russell Yanofsky).

A side effect of the new CCoinsView model is that we can no longer
use the (unreliable) test for transaction outputs in the UTXO set
to determine whether we already have a particular transaction.
2017-06-01 12:59:38 -07:00
Pieter Wuille
961e483979 Switch tests from ModifyCoins to AddCoin/SpendCoin 2017-06-01 12:43:12 -07:00
Pieter Wuille
8b3868c1b4 Switch CScriptCheck to use Coin instead of CCoins 2017-06-01 11:56:06 -07:00
Matt Corallo
c87b957a32 Only pass things committed to by tx's witness hash to CScriptCheck
This clarifies a bit more the ways in which the new script execution
cache could break consensus in the future if additional data from
the CCoins object were to be used as a part of script execution.

After this change, any such consensus breaks should be very visible
to reviewers, hopefully ensuring no such changes can be made.
2017-06-01 11:56:06 -07:00
Pieter Wuille
f68cdfe92b Switch from per-tx to per-txout CCoinsViewCache methods in some places 2017-06-01 11:56:06 -07:00
Pieter Wuille
bd83111a0f Optimization: Coin&& to ApplyTxInUndo
This avoids a prevector copy in ApplyTxInUndo.
2017-06-01 11:56:06 -07:00
Pieter Wuille
cb2c7fdac2 Replace CTxInUndo with Coin
The earlier CTxInUndo class now holds the same information as the Coin
class. Instead of duplicating functionality, replace CTxInUndo with a
serialization adapter for Coin.
2017-06-01 11:56:06 -07:00
practicalswift
5a9b508279 [trivial] Add end of namespace comments 2017-05-31 22:21:25 +02:00
Pieter Wuille
d342424301 Remove/ignore tx version in utxo and undo
This makes the following changes:
* In undo data and the chainstate database, the transaction nVersion
  field is removed from the data structures, always written as 0, and
  ignored when reading.
* The definition of hash_serialized in gettxoutsetinfo is changed to no
  longer incude the nVersion field. It is renamed to hash_serialized_2
  to avoid confusion. The new definition also includes transaction
  height and coinbase information, as this information was missing
  before.

This depends on having a CHashVerifier-based undo data checksum
verifier.

Apart from changing the definition of serialized_hash, downgrading
after using this patch is supported, as no release ever used the value
of nVersion field in UTXO entries.
2017-05-26 13:27:50 -07:00
Pieter Wuille
7e00322906 Add specialization of SipHash for 256 + 32 bit data
We'll need a version of SipHash for tuples of 256 bits and 32 bits
data, when CCoinsViewCache switches from using txids to COutPoints as
keys.
2017-05-26 13:24:25 -07:00
Pieter Wuille
f54580e7e4 error() in disconnect for disk corruption, not inconsistency
The error() function unconditionally reports an error. It should only
be used for actually exception situations, and not for the type of
inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal
with.

This also makes a subtle semantics change: in ApplyTxInUndo, when a
record with metadata is encountered (indicating it is the last spend
from a tx), don't wipe the CCoins record if it wasn't empty at that
point. This makes sure that UTXO operations never affect any other
UTXOs (including those from the same tx).
2017-05-26 13:24:25 -07:00
practicalswift
f3ba869734 [tests] Add libFuzzer support.
See http://llvm.org/docs/LibFuzzer.html#fuzzer-usage for usage instructions.
2017-05-22 17:21:38 +02:00
Wladimir J. van der Laan
217b416c72
Merge #10415: [tests] Speed up fuzzing by ~200x when using afl-fuzz
693247b [test] Speed up fuzzing by ~200x when using afl-fuzz (practicalswift)

Tree-SHA512: 95922fc2616b8cb00dd531ed1140a52bbda4e04292dd8c1c60a8f49dbf6ccb797a18b61180b3fb68d695456b478a1f5ae7fda47e8ecee41dd65555487aef40a3
2017-05-22 16:56:58 +02:00
Wladimir J. van der Laan
6b99dafe42
Merge #10429: tests: fix spurious addrman test failure
6b51b0a tests: fix spurious addrman test failure (Cory Fields)

Tree-SHA512: 3d41723b1a31ff459d950331ffea7f383e4ef6187990be6a634978bead0c29d7c096f68e7edb6d4dc56069c1fe8a6f12a6daf573cb1e542b15d000eaa54ad288
2017-05-19 15:51:35 +02:00
practicalswift
693247b82b [test] Speed up fuzzing by ~200x when using afl-fuzz
Enable the `afl-clang-fast++` features deferred forkserver (`__AFL_INIT`) and persistent mode (`__AFL_LOOP(1000)`).

Before this patch:

```
$ afl-fuzz -i input -o output -m512 -- src/test/test_bitcoin_fuzzy
[*] Validating target binary...
[!] WARNING: The target binary is pretty slow! See /usr/local/share/doc/afl/perf_tips.txt.
[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 1072 to 1072 bits (average: 1072.00 bits)
        Exec timing : 20.4k to 20.4k us (average: 20.4k us)
…
exec speed : 57.58/sec (slow!)
exec speed : 48.35/sec (slow!)
exec speed : 53.78/sec (slow!)
```

After this patch:

```
$ afl-fuzz -i input -o output -m512 -- src/test/test_bitcoin_fuzzy
[*] Validating target binary...
[+] Persistent mode binary detected.
[+] Deferred forkserver binary detected.
[+] Here are some useful stats:

    Test case count : 1 favored, 0 variable, 1 total
       Bitmap range : 24 to 24 bits (average: 24.00 bits)
        Exec timing : 114 to 114 us (average: 114 us)
…
exec speed : 15.9k/sec
exec speed : 13.1k/sec
exec speed : 15.1k/sec
```
2017-05-19 07:28:46 +02:00
Cory Fields
6b51b0a076 tests: fix spurious addrman test failure
When inserting two addresses of the same class, from the same source, they have
a 1/64 chance of colliding.
2017-05-19 00:00:44 -04:00