Commit graph

323 commits

Author SHA1 Message Date
Pieter Wuille
0580ee08ff Adapt memory usage estimation for flushing 2017-06-26 10:45:48 -07:00
Pieter Wuille
013a56aa1a Non-atomic flushing using the blockchain as replay journal 2017-06-26 10:45:48 -07:00
Wladimir J. van der Laan
f3f1e2e7d3
Merge #9544: [trivial] Add end of namespace comments. Improve consistency.
5a9b508 [trivial] Add end of namespace comments (practicalswift)

Tree-SHA512: 92b0fcae4d1d3f4da9e97569ae84ef2d6e09625a5815cd0e5f0eb6dd2ecba9852fa85c184c5ae9de5117050330ce995e9867b451fa8cd5512169025990541a2b
2017-06-26 13:40:26 +02:00
Wladimir J. van der Laan
d609fd85ca
Merge #9176: Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts()
b324b28 Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts() (Jorge Timón)

Tree-SHA512: 9ef561a026960dc69df52603f4d60246c913bdd127465e8238f0954c6bbbc6b8b3b039bab70df38be1cbf5d06788fb631a2236b49c0167633d54366103c7c055
2017-06-25 10:26:01 +02:00
Wladimir J. van der Laan
6bef7ca8bc
Merge #10633: doc: Fix various typos
0a5a6b9 Fixed multiple typos (Dimitris Tsapakidis)

Tree-SHA512: 57748710bcbc03945b160db5e95bd686a2c64605f25d5e11d8ed9d0e1be3b3bf287a63588dc6eb33d0cef4ff17c765fda7c226d667a357acc539c8fcf2b9bb7e
2017-06-22 20:46:37 +02:00
Matt Corallo
a3543af3cc Better document CheckInputs parameter meanings 2017-06-22 12:21:09 -04:00
Matt Corallo
309ee1ae7b Update -maxsigcachesize doc clarify init logprints for it 2017-06-22 12:21:09 -04:00
Matt Corallo
b014668e27 Add CheckInputs wrapper CCoinsViewMemPool -> non-consensus-critical
This wraps CheckInputs in ATMP's cache-inputs call to check that
each scriptPubKey the CCoinsViewCache provides is the one which
was committed to by the input's transaction hash.
2017-06-22 12:21:09 -04:00
Dimitris Tsapakidis
0a5a6b90bc Fixed multiple typos
A few "a->an" and "an->a".
"Shows, if the supplied default SOCKS5 proxy" -> "Shows if the supplied default SOCKS5 proxy". Change made on 3 occurrences.
"without fully understanding the ramification of a command" -> "without fully understanding the ramifications of a command".
Removed duplicate words such as "the the".
2017-06-22 19:18:10 +03:00
Jorge Timón
3eff827f89
scripted-diff: Remove BOOST_REVERSE_FOREACH
-BEGIN VERIFY SCRIPT-
sed -i 's/BOOST_REVERSE_FOREACH(\(.*\), \(.*\))/for (\1 : reverse_iterate(\2))/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ;
-END VERIFY SCRIPT-
2017-06-22 03:48:50 +02:00
Jorge Timón
300851ec16
Introduce src/reverse_iterator.hpp and include it...
...where it will be needed

Taken from https://gist.github.com/arvidsson/7231973 with small
modifications to fit the bitcoin core project
2017-06-22 03:48:42 +02:00
Pieter Wuille
b3eb0d6485
Merge #10537: Few Minor per-utxo assert-semantics re-adds and tweak
9417d7a33 Be much more agressive in AccessCoin docs. (Matt Corallo)
f58349ca8 Restore some assert semantics in sigop cost calculations (Matt Corallo)
3533fb4d3 Return a bool in SpendCoin to restore pre-per-utxo assert semantics (Matt Corallo)
ec1271f2b Remove useless mapNextTx lookup in CTxMemPool::TrimToSize. (Matt Corallo)

Tree-SHA512: 158a4bce063eac93e1d50709500a10a7cb1fb3271f10ed445d701852fce713e2bf0da3456088e530ab005f194ef4a2adf0c7cb23226b160cecb37a79561f29ca
2017-06-20 18:27:45 -07:00
Pieter Wuille
efbcf2b1d5
Merge #10503: Use REJECT_DUPLICATE for already known and conflicted txn
d9bec888f Use REJECT_DUPLICATE for already known and conflicted txn (Pieter Wuille)

Tree-SHA512: adc6dc5caed731c3fd5c8784e8820a074e320360cdb5579c5b9299f9799dd99de60b7382d336ab1909dab8b23e744456d78aa0c3b1c8dd1af3d1b779314cf8fa
2017-06-20 18:00:21 -07:00
Pieter Wuille
1ad3d4e126
Merge #10502: scripted-diff: Remove BOOST_FOREACH, Q_FOREACH and PAIRTYPE
1238f13cf scripted-diff: Remove PAIRTYPE (Jorge Timón)
18dc3c396 scripted-diff: Remove Q_FOREACH (Jorge Timón)
7c00c2672 scripted-diff: Fully remove BOOST_FOREACH (Jorge Timón)
a5410ac5e Small preparations for Q_FOREACH, PAIRTYPE and #include <boost/foreach.hpp> removal (Jorge Timón)

Tree-SHA512: d3ab4a173366402e7dcef31608977b757d4aa07abbbad2ee1bcbcfa311e994a4552f24e5a55272cb22c2dcf89a4b0495e02e9d9aceae4b08c0bab668f20e324c
2017-06-13 18:05:58 -07:00
Wladimir J. van der Laan
b7296bcea0
Merge #10550: Don't return stale data from CCoinsViewCache::Cursor()
3ff1fa8 Use override keyword on CCoinsView overrides (Russell Yanofsky)
24e44c3 Don't return stale data from CCoinsViewCache::Cursor() (Russell Yanofsky)

Tree-SHA512: 08699dae0925ffb9c018f02612ac6b7eaf73ec331e2f4f934f1fe25a2ce120735fa38596926e924897c203f7470e99f0a99cf70d2ce31ff428b105e16583a861
2017-06-12 16:29:59 +02:00
Wladimir J. van der Laan
2c2d988062
Merge #10546: Remove 33 unused Boost includes
49de096 Remove unused Boost includes (practicalswift)

Tree-SHA512: ff2dad94f9eeb1dc50fcffd0e94e1686be04e4e5bc45f58ae955d630c15cd25bb8f5583d0aa1f2f263b5a723be79747ef3c6e6b417c7be7787c0abc8d1874019
2017-06-11 15:37:15 +02:00
Andrew Chow
c45cbaf69f Fix stopatheight
Moves stopatheight check into the ActivateBestChain loop so that the block height is actually checked and stopped at the appropriate time.
2017-06-09 13:40:45 -07:00
Matt Corallo
3533fb4d33 Return a bool in SpendCoin to restore pre-per-utxo assert semantics
Since its free to do so, assert that Spends succeeded when we expect
them to.
2017-06-09 13:10:05 -04:00
practicalswift
49de096c2a Remove unused Boost includes 2017-06-09 10:25:26 +02:00
Russell Yanofsky
24e44c354d Don't return stale data from CCoinsViewCache::Cursor()
CCoinsViewCache doesn't actually support cursor iteration returning the
current contents of the cache, so raise an error when the cursor method is
called instead of returning a cursor that iterates over stale data.

Also update the gettxoutsetinfo RPC which was relying on the old behavior to be
explicit about which view it is returning data about.
2017-06-07 13:00:11 -04:00
Matt Corallo
eada04e778 Do not print soft-fork-script warning with -promiscuousmempool 2017-06-07 11:02:36 -04:00
Matt Corallo
b5fea8d0cc Cache full script execution results in addition to signatures
This adds a new CuckooCache in validation, caching whether all of a
transaction's scripts were valid with a given set of script flags.

Unlike previous attempts at caching an entire transaction's
validity, which have nearly universally introduced consensus
failures, this only caches the validity of a transaction's
scriptSigs. As these are pure functions of the transaction and
data it commits to, this should be much safer.

This is somewhat duplicative with the sigcache, as entries in the
new cache will also have several entries in the sigcache. However,
the sigcache is kept both as ATMP relies on it and because it
prevents malleability-based DoS attacks on the new higher-level
cache. Instead, the -sigcachesize option is re-used - cutting the
sigcache size in half and using the newly freed memory for the
script execution cache.

Transactions which match the script execution cache never even have
entries in the script check thread's workqueue created.

Note that the cache is indexed only on the script execution flags
and the transaction's witness hash. While this is sufficient to
make the CScriptCheck() calls pure functions, this introduces
dependancies on the mempool calculating things such as the
PrecomputedTransactionData object, filling the CCoinsViewCache, etc
in the exact same way as ConnectBlock. I belive this is a reasonable
assumption, but should be noted carefully.

In a rather naive benchmark (reindex-chainstate up to block 284k
with cuckoocache always returning true for contains(),
-assumevalid=0 and a very large dbcache), this connected blocks
~1.7x faster.
2017-06-07 11:02:36 -04:00
Matt Corallo
6d22b2b17b Pull script verify flags calculation out of ConnectBlock 2017-06-07 11:02:36 -04:00
Mario Dian
24980a3e40 Make functions in validation.cpp static and pass chainparams
Fix bugs as per PR comment

Change bool static to static bool
2017-06-06 22:21:30 +08:00
Jorge Timón
1238f13cf6
scripted-diff: Remove PAIRTYPE
-BEGIN VERIFY SCRIPT-
sed -i 's/PAIRTYPE(\([^,]*\), \([^\)]*\))/std::pair<\1, \2>/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
sed -i ':a;N;$!ba;s/#define std::pair<t1, t2>    std::pair<t1, t2>\n//' ./src/utilstrencodings.h ;
-END VERIFY SCRIPT-
2017-06-05 20:14:53 +02:00
Jorge Timón
7c00c26726
scripted-diff: Fully remove BOOST_FOREACH
-BEGIN VERIFY SCRIPT-
sed -i 's/BOOST_FOREACH *(\(.*\),/for (\1 :/' ./src/*.h ./src/*.cpp ./src/*/*.h ./src/*/*.cpp ./src/*/*/*.h ./src/*/*/*.cpp ;
-END VERIFY SCRIPT-
2017-06-05 20:10:50 +02:00
Pieter Wuille
d9bec888f1 Use REJECT_DUPLICATE for already known and conflicted txn 2017-06-02 00:18:47 -07:00
Pieter Wuille
1088b02f0c
Merge #10195: Switch chainstate db and cache to per-txout model
589827975 scripted-diff: various renames for per-utxo consistency (Pieter Wuille)
a5e02bc7f Increase travis unit test timeout (Pieter Wuille)
73de2c1ff Rename CCoinsCacheEntry::coins to coin (Pieter Wuille)
119e552f7 Merge CCoinsViewCache's GetOutputFor and AccessCoin (Pieter Wuille)
580b02309 [MOVEONLY] Move old CCoins class to txdb.cpp (Pieter Wuille)
8b25d2c0c Upgrade from per-tx database to per-txout (Pieter Wuille)
b2af357f3 Reduce reserved memory space for flushing (Pieter Wuille)
41aa5b79a Pack Coin more tightly (Pieter Wuille)
97072d668 Remove unused CCoins methods (Pieter Wuille)
ce23efaa5 Extend coins_tests (Pieter Wuille)
508307968 Switch CCoinsView and chainstate db from per-txid to per-txout (Pieter Wuille)
4ec0d9e79 Refactor GetUTXOStats in preparation for per-COutPoint iteration (Pieter Wuille)
13870b56f Replace CCoins-based CTxMemPool::pruneSpent with isSpent (Pieter Wuille)
05293f3cb Remove ModifyCoins/ModifyNewCoins (Pieter Wuille)
961e48397 Switch tests from ModifyCoins to AddCoin/SpendCoin (Pieter Wuille)
8b3868c1b Switch CScriptCheck to use Coin instead of CCoins (Pieter Wuille)
c87b957a3 Only pass things committed to by tx's witness hash to CScriptCheck (Matt Corallo)
f68cdfe92 Switch from per-tx to per-txout CCoinsViewCache methods in some places (Pieter Wuille)
000391132 Introduce new per-txout CCoinsViewCache functions (Pieter Wuille)
bd83111a0 Optimization: Coin&& to ApplyTxInUndo (Pieter Wuille)
cb2c7fdac Replace CTxInUndo with Coin (Pieter Wuille)
422634e2f Introduce Coin, a single unspent output (Pieter Wuille)
7d991b55d Store/allow tx metadata in all undo records (Pieter Wuille)
c3aa0c119 Report on-disk size in gettxoutsetinfo (Pieter Wuille)
d34242430 Remove/ignore tx version in utxo and undo (Pieter Wuille)
7e0032290 Add specialization of SipHash for 256 + 32 bit data (Pieter Wuille)
e484652fc Introduce CHashVerifier to hash read data (Pieter Wuille)
f54580e7e error() in disconnect for disk corruption, not inconsistency (Pieter Wuille)
e66dbde6d Add SizeEstimate to CDBBatch (Pieter Wuille)

Tree-SHA512: ce1fb1e40c77d38915cd02189fab7a8b125c7f44d425c85579d872c3bede3a437760997907c99d7b3017ced1c2de54b2ac7223d99d83a6658fe5ef61edef1de3
2017-06-01 16:20:27 -07:00
Pieter Wuille
589827975f scripted-diff: various renames for per-utxo consistency
Thanks to John Newberry for pointing these out.

-BEGIN VERIFY SCRIPT-
sed -i 's/\<GetCoins\>/GetCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoins\>/HaveCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<HaveCoinsInCache\>/HaveCoinInCache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<IsPruned\>/IsSpent/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<FetchCoins\>/FetchCoin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<CoinsEntry\>/CoinEntry/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxnToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<vHashTxToUncache\>/coins_to_uncache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<fHadTxInCache\>/had_coin_in_cache/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<coinbaseids\>/coinbase_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<disconnectedids\>/disconnected_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<duplicateids\>/duplicate_coins/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
sed -i 's/\<oldcoins\>/old_coin/g' src/test/coins_tests.cpp
sed -i 's/\<origcoins\>/orig_coin/g' src/*.cpp src/*.h src/*/*.cpp src/*/*.h
-END VERIFY SCRIPT-
2017-06-01 13:15:25 -07:00
Pieter Wuille
b2af357f39 Reduce reserved memory space for flushing
As the maximum amount of data that can be pulled into the cache due to
a block validation is much lower now (at most one CCoin entry per input
and per output), reduce the conservative estimate used to determine
flushing time.
2017-06-01 13:15:06 -07:00
Pieter Wuille
5083079688 Switch CCoinsView and chainstate db from per-txid to per-txout
This patch makes several related changes:
* Changes the CCoinsView virtual methods (GetCoins, HaveCoins, ...)
  to be COutPoint/Coin-based rather than txid/CCoins-based.
* Changes the chainstate db to a new incompatible format that is also
  COutPoint/Coin based.
* Implements reconstruction code for hash_serialized_2.
* Adapts the coins_tests unit tests (thanks to Russell Yanofsky).

A side effect of the new CCoinsView model is that we can no longer
use the (unreliable) test for transaction outputs in the UTXO set
to determine whether we already have a particular transaction.
2017-06-01 12:59:38 -07:00
Pieter Wuille
8b3868c1b4 Switch CScriptCheck to use Coin instead of CCoins 2017-06-01 11:56:06 -07:00
Matt Corallo
c87b957a32 Only pass things committed to by tx's witness hash to CScriptCheck
This clarifies a bit more the ways in which the new script execution
cache could break consensus in the future if additional data from
the CCoins object were to be used as a part of script execution.

After this change, any such consensus breaks should be very visible
to reviewers, hopefully ensuring no such changes can be made.
2017-06-01 11:56:06 -07:00
Pieter Wuille
f68cdfe92b Switch from per-tx to per-txout CCoinsViewCache methods in some places 2017-06-01 11:56:06 -07:00
Pieter Wuille
bd83111a0f Optimization: Coin&& to ApplyTxInUndo
This avoids a prevector copy in ApplyTxInUndo.
2017-06-01 11:56:06 -07:00
Pieter Wuille
cb2c7fdac2 Replace CTxInUndo with Coin
The earlier CTxInUndo class now holds the same information as the Coin
class. Instead of duplicating functionality, replace CTxInUndo with a
serialization adapter for Coin.
2017-06-01 11:56:06 -07:00
Karl-Johan Alm
a473eff8f9
[bench] Replace 0.00(000)1 with MICRO/MILLI #defines in validation.cpp. 2017-06-01 11:43:04 +09:00
Karl-Johan Alm
5f850b0311
[bench] Include ms/blk stats in Connect* benchmarks. 2017-06-01 11:43:04 +09:00
practicalswift
5a9b508279 [trivial] Add end of namespace comments 2017-05-31 22:21:25 +02:00
Pieter Wuille
f94b7d5bfa
Merge #10464: Introduce static DoWarning (simplify UpdateTip)
f28514576 Introduce static DoWarning (simplify UpdateTip) (Jorge Timón)

Tree-SHA512: 58b39f00bc921d47d199ef1cc15e9e36f918ec86e9182de46dc2f24fa9474853a92b18d330a274a3ea7c9b001681fdfa3d86aa2bbebb5c92ac8c5a6f7690dfc1
2017-05-30 17:23:30 -07:00
Wladimir J. van der Laan
acd9957b72
Merge #9208: Improve DisconnectTip performance
c1235e3 Add RecursiveDynamicUsage overload for std::shared_ptr (Russell Yanofsky)
71f1903 Store disconnected block transactions outside mempool during reorg (Suhas Daftuar)
9decd64 [qa] Relax assumptions on mempool behavior during reorg (Suhas Daftuar)

Tree-SHA512: c160ad853a5cd060d0307af7606a0c77907497ed7033c9599b95e73d83f68fdfcd4214bd8a83db1c5b7a58022722b9de1ed2e6ea2e02f38a7b6c717f079dd0c6
2017-05-30 18:43:03 +02:00
Jorge Timón
f285145764
Introduce static DoWarning (simplify UpdateTip) 2017-05-27 06:33:39 +02:00
Pieter Wuille
7d991b55db Store/allow tx metadata in all undo records
Previously, transaction metadata (height, coinbase or not, and before
the previous commit also nVersion) was only stored for undo records
that correspond to the last output of a transaction being spent.

This only saves 2 bytes per undo record. Change this to storing this
information for every undo record, and stop complaining for having it
in non-last output spends. This means that undo dat written with
this patch won't be readable by older versions anymore.
2017-05-26 13:33:39 -07:00
Pieter Wuille
d342424301 Remove/ignore tx version in utxo and undo
This makes the following changes:
* In undo data and the chainstate database, the transaction nVersion
  field is removed from the data structures, always written as 0, and
  ignored when reading.
* The definition of hash_serialized in gettxoutsetinfo is changed to no
  longer incude the nVersion field. It is renamed to hash_serialized_2
  to avoid confusion. The new definition also includes transaction
  height and coinbase information, as this information was missing
  before.

This depends on having a CHashVerifier-based undo data checksum
verifier.

Apart from changing the definition of serialized_hash, downgrading
after using this patch is supported, as no release ever used the value
of nVersion field in UTXO entries.
2017-05-26 13:27:50 -07:00
Pieter Wuille
e484652fc3 Introduce CHashVerifier to hash read data
This is necessary later, when we drop the nVersion field from the undo
data. At that point deserializing and reserializing the data won't
roundtrip anymore, and thus that approach can't be used to verify
checksums anymore.

With this CHashVerifier approach, we can deserialize while hashing the
exact serialized form that was used. This is both more efficient and
more correct in that case.
2017-05-26 13:24:25 -07:00
Pieter Wuille
f54580e7e4 error() in disconnect for disk corruption, not inconsistency
The error() function unconditionally reports an error. It should only
be used for actually exception situations, and not for the type of
inconsistencies that ApplyTxInUndo/DisconnectBlock can graciously deal
with.

This also makes a subtle semantics change: in ApplyTxInUndo, when a
record with metadata is encountered (indicating it is the last spend
from a tx), don't wipe the CCoins record if it wasn't empty at that
point. This makes sure that UTXO operations never affect any other
UTXOs (including those from the same tx).
2017-05-26 13:24:25 -07:00
Pieter Wuille
f2f7e97e8c
Merge #10347: Use range-based for loops (C++11) when looping over vector elements
211adc0 Use range-based for loops (C++11) when looping over vector elements (practicalswift)

Tree-SHA512: 0e007f20dcef99d3c7a1036265e00f689d69f42e02fd82dd8389f45b52d31947e5f9388de2610d3d9bd9f554915ce0d35ebce561e5ae3a9013956d0ee4937145
2017-05-23 14:08:20 -07:00
Wladimir J. van der Laan
46771514fa
Merge #9571: RPC: getblockchaininfo returns BIP signaling statistics
557c9a6 RPC: getblockchaininfo: BIP9 stats (Matthew Zipkin)

Tree-SHA512: ecf0bf47f04f92becc77acc649fdfa270e768939acce42df39d30069398d40d9a30539862f7c307e08239f78d5c58c470ca5f6e717d2ab8e24db9be0dd7bec0c
2017-05-23 19:12:29 +02:00
practicalswift
211adc074a Use range-based for loops (C++11) when looping over vector elements 2017-05-19 09:56:16 +02:00
Jorge Timón
b324b28209
Globals: Pass Consensus::Params through CBlockTreeDB::LoadBlockIndexGuts() 2017-05-18 22:23:10 +02:00