Commit graph

299 commits

Author SHA1 Message Date
Antoine Poinsot
86ff2cf202
Remove the remaining fee estimation globals
This moves the CBlockPolicyEstimator to the NodeContext, which get rids
of two globals and allows us to conditionally create the
CBlockPolicyEstimator (and to remove a circular dep).

Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-12-03 12:56:37 +01:00
Andrew Chow
95e61c1cf2 Move Hashers to util/hasher.{cpp/h}
Move the hashers that we use for hash tables to a common place.

Moved hashers:
- SaltedTxidHasher
- SaltedOutpointHasher
- FilterHeaderHasher
- SignatureCacheHasher
- BlockHasher
2020-11-10 14:33:37 -05:00
MarcoFalke
fa4234d877
test: Mock IBD in net_processing fuzzers 2020-11-07 07:50:59 +01:00
John Newbery
b048b275d9 [validation] Remove absurdfee from accepttomempool
Mempool behavior should not be user-specific.
Checking that txfee is acceptable should be
the responsibility of the wallet or client, not
the mempool.
2020-10-05 04:55:01 -07:00
Hennadii Stepanov
01511776ac
Add local thread pool to CCheckQueue 2020-09-24 06:55:33 +03:00
MarcoFalke
1b313cacc9
Merge #19927: validation: Reduce direct g_chainman usage
72a1d5c6f3 validation: Remove review-only comments + assertions (Carl Dong)
3756853b15 docs: Move FindFilesToPrune{,Manual} doxygen comment (Carl Dong)
485899a93c style: Make FindFilesToPrune{,Manual} match style guide (Carl Dong)
3f5b5f3f6d validation: Move FindFilesToPrune{,Manual} to BlockManager (Carl Dong)
f8d4975ab3 validation: Move PruneOneBlockFile to BlockManager (Carl Dong)
74f73c783d validation: Pass in chainman to UnloadBlockIndex (Carl Dong)
4668ded6d6 validation: Move ~CMainCleanup logic to ~BlockManager (Carl Dong)

Pull request description:

  This PR paves the way for de-globalizing `g_chainman` entirely by removing the usage of `g_chainman` in the following functions/methods:
  - `~CMainCleanup`
  - `CChainState::FlushStateToDisk`
  - `UnloadBlockIndex`

  The remaining direct uses of `g_chainman` are as follows:
  1. In initialization codepaths:
  	- `AppTests`
  	- `AppInitMain`
  	- `TestingSetup::TestingSetup`
  2. `::ChainstateActive`
  3. `LookupBlockIndex`
  	- Note: `LookupBlockIndex` is used extensively throughout the codebase and require a much larger set of changes, therefore I've left it out of this initial PR

ACKs for top commit:
  MarcoFalke:
    re-ACK 72a1d5c6f3 👚
  jnewbery:
    utACK 72a1d5c6f3

Tree-SHA512: 944a4fa8405eecf39706ff944375d6824373aaeea849d11473f08181eff26b12f70043a8348a5b08e6e9021b243b481842fbdfbc7c3140ca795fce3688b7f5c3
2020-09-23 20:35:54 +02:00
Wladimir J. van der Laan
8c5f68118c
Merge #18267: BIP-325: Signet [consensus]
8258c4c007 test: some sanity checks for consensus logic (Anthony Towns)
e47ad375bf test: basic signet tests (Karl-Johan Alm)
4c189abdc4 test: add small signet fuzzer (practicalswift)
ec9b25d046 test: signet network selection tests (Karl-Johan Alm)
3efe298dcc signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bca4e qt: update QT to support signet network (Karl-Johan Alm)
a8de47a1c9 consensus: add signet validation (Karl-Johan Alm)
e8990f1214 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b7cd add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7dad validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of #16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c007 per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c007
  MarcoFalke:
    Approach ACK 8258c4c007 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
2020-09-21 22:33:00 +02:00
Carl Dong
3756853b15
docs: Move FindFilesToPrune{,Manual} doxygen comment
[META] This is a pure comment commit.

They belong in the member declarations in the header file.
2020-09-21 13:30:21 -04:00
Carl Dong
3f5b5f3f6d
validation: Move FindFilesToPrune{,Manual} to BlockManager
[META] No behaviour change is intended in this commit.
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

Also stop FindFilesToPrune{,Manual} from unnecessary reaching for
::ChainActive() by passing in the necessary information.
2020-09-21 13:27:44 -04:00
codeShark149
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept
Return fee and vsize if tx would pass ATMP.
2020-09-15 18:01:32 -07:00
Carl Dong
f8d4975ab3
validation: Move PruneOneBlockFile to BlockManager
[META] This is a pure refactor commit.

Move PruneBlockFile to BlockManager because:
1. PruneOneBlockFile only acts on BlockManager
2. Eliminates the need for callers (FindFilesToPrune{,Manual}) to have a
   reference to the larger ChainstateManager, just a reference to
   BlockManager is enough. See following commits.
2020-09-15 14:13:44 -04:00
Carl Dong
74f73c783d
validation: Pass in chainman to UnloadBlockIndex 2020-09-15 14:11:34 -04:00
Carl Dong
4668ded6d6
validation: Move ~CMainCleanup logic to ~BlockManager
~CMainCleanup:
1. Is vestigial
2. References the g_chainman global (we should minimize g_chainman refs)
3. Only acts on g_chainman.m_blockman
4. Does the same thing as BlockManager::Unload
2020-09-14 10:42:45 -04:00
Karl-Johan Alm
a2147d7dad
validation: move GetWitnessCommitmentIndex to consensus/validation 2020-09-10 10:47:40 +09:00
MarcoFalke
fafb381af8
Remove mempool global 2020-09-05 16:24:56 +02:00
MarcoFalke
eeee1104d7
Remove mempool global from init
Can be reviewed with the git diff options

--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space --ignore-all-space
2020-09-05 16:24:08 +02:00
MarcoFalke
bab4cce1b0
Merge #19668: Do not hide compile-time thread safety warnings
ea74e10acf doc: Add best practice for annotating/asserting locks (Hennadii Stepanov)
2ee7743fe7 sync.h: Make runtime lock checks require compile-time lock checks (Anthony Towns)
23d71d171e Do not hide compile-time thread safety warnings (Hennadii Stepanov)
3ddc150857 Add missed thread safety annotations (Hennadii Stepanov)
af9ea55a72 Use LockAssertion utility class instead of AssertLockHeld() (Hennadii Stepanov)

Pull request description:

  On the way of transit from `RecursiveMutex` to `Mutex` (see #19303) it is crucial to have run-time `AssertLockHeld()` assertion that does _not_ hide compile-time Clang Thread Safety Analysis warnings.

  On master (65e4ecabd5) using `AssertLockHeld()` could hide Clang Thread Safety Analysis warnings, e.g., with the following patch applied:
  ```diff
  --- a/src/txmempool.h
  +++ b/src/txmempool.h
  @@ -607,7 +607,7 @@ public:
       void addUnchecked(const CTxMemPoolEntry& entry, setEntries& setAncestors, bool validFeeEstimate = true) EXCLUSIVE_LOCKS_REQUIRED(cs, cs_main);

       void removeRecursive(const CTransaction& tx, MemPoolRemovalReason reason) EXCLUSIVE_LOCKS_REQUIRED(cs);
  -    void removeForReorg(const CCoinsViewCache* pcoins, unsigned int nMemPoolHeight, int flags) EXCLUSIVE_LOCKS_REQUIRED(cs, cs_main);
  +    void removeForReorg(const CCoinsViewCache* pcoins, unsigned int nMemPoolHeight, int flags) EXCLUSIVE_LOCKS_REQUIRED(cs_main);
       void removeConflicts(const CTransaction& tx) EXCLUSIVE_LOCKS_REQUIRED(cs);
       void removeForBlock(const std::vector<CTransactionRef>& vtx, unsigned int nBlockHeight) EXCLUSIVE_LOCKS_REQUIRED(cs);

  ```
  Clang compiles the code without any thread safety warnings.

  See "Add missed thread safety annotations" commit for the actual thread safety warnings that are fixed in this PR.

ACKs for top commit:
  MarcoFalke:
    ACK ea74e10acf 🎙
  jnewbery:
    ACK ea74e10acf
  ajtowns:
    ACK ea74e10acf

Tree-SHA512: 8cba996e526751a1cb0e613c0cc1b10f027a3e9945fbfb4bd30f6355fd36b9f9c2e1e95ed3183fc254b42df7c30223278e18e5bdb5e1ef85db7fef067595d447
2020-09-01 08:18:26 +02:00
Anthony Towns
2ee7743fe7
sync.h: Make runtime lock checks require compile-time lock checks 2020-08-29 20:46:47 +03:00
MarcoFalke
fa0572d0f3
Pass mempool reference to chainstate constructor 2020-08-28 10:42:04 +02:00
MarcoFalke
fae8c28dae
Pass mempool pointer to GetCoinsCacheSizeState 2020-07-29 12:30:11 +02:00
MarcoFalke
fac674db20
Pass mempool pointer to UnloadBlockIndex 2020-07-29 12:29:51 +02:00
MarcoFalke
2f71a1ea35
Merge #18637: coins: allow cache resize after init
f19fdd47a6 test: add test for CChainState::ResizeCoinsCaches() (James O'Beirne)
8ac3ef4699 add ChainstateManager::MaybeRebalanceCaches() (James O'Beirne)
f36aaa6392 Add CChainState::ResizeCoinsCaches (James O'Beirne)
b223111da2 txdb: add CCoinsViewDB::ChangeCacheSize (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  In the assumeutxo implementation draft (#15056), once a UTXO snapshot is loaded, a new chainstate object is created after initialization. This means that we have to reclaim some of the cache that we've allocated to the original chainstate (per `dbcache=`) to repurpose for the snapshot chainstate.

  Furthermore, it makes sense to have different cache allocations depending on which chainstate is more active. While the snapshot chainstate is working to get to the network tip (and the background validation chainstate is idle), it makes sense that the snapshot chainstate should have the majority of cache allocation. And contrariwise once the snapshot has reached network tip, most of the cache should be given to the background validation chainstate.

  This set of changes (detailed in the commit messages) allows us to dynamically resize the various coins caches. None of the functionality introduced here is used at the moment, but will be in the next AU PR (which introduces `ActivateSnapshot`).

  `ChainstateManager::MaybeRebalanceCaches()` defines the (somewhat normative) cache allocations between the snapshot and background validation chainstates. I'd be interested in feedback if anyone has thoughts on the proportions I've set there.

ACKs for top commit:
  ajtowns:
    weak utACK f19fdd47a6 -- didn't find any major problems, but not super confident that I didn't miss anything
  fjahr:
    Code review ACK f19fdd4
  ryanofsky:
    Code review ACK f19fdd47a6. Only change since last review is constructor cleanup (no change in behavior). I think the suggestions here from ajtowns and others are good, but shouldn't delay merging the PR (and hold up assumeutxo)

Tree-SHA512: fffb7847fb6993dd4a1a41cf11179b211b0b20b7eb5f7cf6266442136bfe9d43b830bbefcafd475bfd4af273f5573500594aa41fff03e0ed5c2a1e8562ff9269
2020-07-29 07:53:19 +02:00
MarcoFalke
fa5979d12f
rpc: Avoid useless mempool query in gettxoutproof 2020-07-26 16:44:07 +02:00
Jon Atack
06059b0c2a
net: rename DEFAULT_BANSCORE_THRESHOLD to DISCOURAGEMENT_THRESHOLD
and move it from validation to net processing.
2020-07-11 19:41:24 +02:00
Jon Atack
1d4024bca8
net: remove -banscore configuration option 2020-07-11 19:41:21 +02:00
MarcoFalke
fab5586122
doc: Use precise permission flags where possible 2020-07-10 15:37:42 +02:00
James O'Beirne
8ac3ef4699 add ChainstateManager::MaybeRebalanceCaches()
Aside from in unittests, this method is unused at the moment. It will be used
in upcoming commits that enable utxo snapshot activation.
2020-07-01 14:44:28 -04:00
James O'Beirne
f36aaa6392 Add CChainState::ResizeCoinsCaches
Also adds CCoinsViewCache::ReallocateCache() to attempt to free
memory that the cacheCoins's allocator may be hanging onto when
downsizing the cache.

Adds `CChainState::m_coins{tip,db}_cache_size_bytes` data members
so that we can reference cache size on a per-chainstate basis for
flushing.
2020-07-01 14:44:28 -04:00
MarcoFalke
fa0dfdf447
refactor: Remove confusing BlockIndex global 2020-06-29 20:28:47 -04:00
MarcoFalke
faba65e696
Add ChainstateManager::ActiveChainstate 2020-06-19 09:27:00 -04:00
Calvin Kim
501e6ab4e7 doc: Add documentation for 'checklevel' argument in 'verifychain' RPC call 2020-06-07 17:50:22 +09:00
MarcoFalke
fab6b9d18f
validation: Mark g_chainman DEPRECATED 2020-05-21 09:56:25 -04:00
MarcoFalke
fa1d97b256
validation: Make ProcessNewBlock*() members of ChainstateManager 2020-05-21 09:56:16 -04:00
MarcoFalke
fa24d49098
validation: Make PruneOneBlockFile() a member of ChainstateManager 2020-05-21 09:56:16 -04:00
MarcoFalke
fa84b1cd84
validation: Make LoadBlockIndex() a member of ChainstateManager 2020-05-21 09:55:59 -04:00
MarcoFalke
fa7b626d7a
node: Add chainman alias for g_chainman 2020-05-21 09:55:51 -04:00
Hennadii Stepanov
1dab574edf
refactor: Pass SynchronizationState enum to GUI
Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-05-19 02:49:32 +03:00
Wladimir J. van der Laan
42fd503819
Merge #18786: init: Remove boost from ThreadImport
faec3dc2ad init: Remove boost from ThreadImport (MarcoFalke)

Pull request description:

  Can be tested by calling `-reindex` or `-loadblock` and then pressing `CTRL`+`C`.

  Should print something like:

  ```
  ...
  2020-04-27T19:34:31Z [loadblk] Reindexing block file blk00005.dat...
  ^C2020-04-27T19:34:32Z [loadblk] Shutdown requested. Exit ThreadImport
  2020-04-27T19:34:32Z [qt-init] Interrupting HTTP server
  ...
  ```

ACKs for top commit:
  laanwj:
    Code review ACK faec3dc2ad
  hebasto:
    ACK faec3dc2ad, tested on Linux Mint 19.3 (x86_64) both `bitcoind` and `bitcoin-qt` binaries.

Tree-SHA512: e105af18d98296d82ec99f48e478cf44577e3c32f7e4b47617a7bc7cbf71d6becb92722f229a1be38d58ad29712704509ad9740d8ab8cd3104cf90057664b437
2020-05-04 16:06:42 +02:00
fanquake
06442549f8
validation: Add minimum witness commitment size constant
Per BIP 141, the witness commitment structure is atleast 38 bytes,
OP_RETURN (0x6a) + 36 (0x24) + 4 byte header (0xaa21a9ed) + 32 byte
SHA256 hash. It can be longer, however any additional data has no
consensus meaning.
2020-04-29 11:20:22 +08:00
MarcoFalke
faec3dc2ad
init: Remove boost from ThreadImport 2020-04-27 15:35:26 -04:00
John Newbery
507b36dd1b [validation] Move all const declarations to top of validation.h 2020-04-23 12:54:06 -04:00
John Newbery
0109622b08 [validation] Move validation-only consts to validation.cpp 2020-04-23 12:54:06 -04:00
John Newbery
b8580cacc7 [net processing] Move net processing consts to net_processing.cpp 2020-04-23 12:54:03 -04:00
James O'Beirne
c9017ce3bc protect g_chainman with cs_main
I'd previously attempted to create a specialized lock for ChainstateManager,
but it turns out that because that lock would be required for functions like
ChainActive() and ChainstateActive(), it created irreconcilable lock inversions
since those functions are used so broadly throughout the codebase.

Instead, I'm just using cs_main to protect the contents of g_chainman.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-03-17 14:07:58 -04:00
James O'Beirne
4ae29f5f0c use ChainstateManager to initialize chainstate
This allows us to easily initialize multiple chainstates on startup in future
commits. It retires the g_chainstate global in lieu of g_chainman.
2020-03-17 14:03:40 -04:00
James O'Beirne
5b690f0aae refactor: move RewindBlockIndex to CChainState
This is in preparation for multiple chainstate initialization in init.
2020-03-17 14:03:40 -04:00
James O'Beirne
89cdf4d569 validation: introduce unused ChainstateManager
ChainstateManager is responsible for creating and managing multiple
chainstates, and will provide a high-level interface for accessing the
appropriate chainstate based upon a certain use.

Incorporates feedback from Marco Falke. Additional documentation written
by Russ Yanofsky.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2020-03-17 13:27:39 -04:00
James O'Beirne
8e2ecfe249 validation: add CChainState.m_from_snapshot_blockhash
This parameter is unused, but in future commits will allow ChainstateManager to
differentiate between chainstates created from a UTXO snapshot from those that
weren't.
2020-03-17 13:27:39 -04:00
MarcoFalke
e09c701e01 scripted-diff: Bump copyright of files changed in 2020
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-01-15 02:18:00 +07:00
MarcoFalke
6cbe620964 scripted-diff: Replace CCriticalSection with RecursiveMutex
-BEGIN VERIFY SCRIPT-
 # Delete outdated alias for RecursiveMutex
 sed -i -e '/CCriticalSection/d'                 ./src/sync.h
 # Replace use of outdated alias with RecursiveMutex
 sed -i -e 's/CCriticalSection/RecursiveMutex/g' $(git grep -l CCriticalSection)
-END VERIFY SCRIPT-
2020-01-15 01:43:46 +07:00
James O'Beirne
b17e91d842 refactoring: introduce CChainState::GetCoinsCacheSizeState
This separates out some logic for detecting how full the coins cache is from
FlushStateToDisk. We'll want to reuse this logic when deciding when to flush
the coins cache during UTXO snapshot activation.
2019-12-12 11:20:17 -05:00
Hennadii Stepanov
98fbd1cdff
Use correct C++11 header for std::swap() 2019-11-29 21:23:25 +02:00
Neha Narula
e9a27cf338 refactor: Remove unused COINBASE_FLAGS
Commit d449772cf6 stopped setting
COINBASE_FLAGS, and it looks like it hasn't been used since P2SH.
Update the help string to remove "flags", which is not specified in
BIP 22.
2019-11-20 19:06:52 -05:00
MarcoFalke
7d14e35f3f
Merge #17342: refactor: Clean up nScriptCheckThreads
5506ecfe7a [refactor] Replace global int nScriptCheckThreads with bool (John Newbery)
d9957623b4 [tests] Don't use TestingSetup in the checkqueue_tests (John Newbery)

Pull request description:

  The meaning of this value is confusing. Refactor it and add comments.

ACKs for top commit:
  sipa:
    ACK 5506ecfe7a
  promag:
    ACK 5506ecfe7a, only change was addressing my nits.
  laanwj:
    Code review ACK 5506ecfe7a
  MarcoFalke:
    ACK 5506ecfe7a 🥐

Tree-SHA512: 78536727c98d2c23f3c0f3f169131474fef9a4486ae65029011caf06eab30f6f70ff73a65b2fb04a5d969fc1150858d1c6ea4767f04d48c1eea6b829316d0e63
2019-11-07 10:07:11 -05:00
John Newbery
5506ecfe7a [refactor] Replace global int nScriptCheckThreads with bool
The global nScriptCheckThreads int is confusing and is only needed for
its int-ness in AppInitMain. Move all `-par` parsing logic there and
replace the int nScriptCheckThreads with a bool
g_parallel_script_checks.

Also tidy up logic and improve comments.
2019-11-06 15:04:50 -05:00
Wladimir J. van der Laan
b05b28183c
Merge #16899: UTXO snapshot creation (dumptxoutset)
92b2f5306b test: add dumptxoutset RPC test (James O'Beirne)
c1ccbc3dde devtools: add utxo_snapshot.sh (James O'Beirne)
57cf74c991 rpc: add dumptxoutset (James O'Beirne)
92fafb3a7d coinstats: add coins_count (James O'Beirne)
707fde7b9b add unused SnapshotMetadata class (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  This changeset defines the serialization format for UTXO snapshots and adds an RPC command for creating them, `dumptxoutset`. It also adds a convenience script for generating and verifying snapshots at a certain height, since that requires doing a hacky rewind of the chain via `invalidateblock`.

  All of this is unused at the moment.

ACKs for top commit:
  laanwj:
    ACK 92b2f5306b

Tree-SHA512: 200dff87767f157d627e99506ec543465d9329860a6cd49363081619c437163a640a46d008faa92b1f44fd403bfc7a7c9e851c658b5a4849efa9a34ca976bf31
2019-11-05 19:40:18 +01:00
John Newbery
3004d5a12d [validation] Remove fMissingInputs from AcceptToMemoryPool()
Handle this failure in the same way as all other failures: call Invalid()
with the reasons for the failure.
2019-10-29 15:46:45 -04:00
John Newbery
c428622a5b [validation] Remove unused first_invalid parameter from ProcessNewBlockHeaders()
No callers use the returned value in first_invalid. Remove it from the
function signature and don't set it in the function.
2019-10-29 15:46:45 -04:00
John Newbery
a27a2957ed [validation] Add CValidationState subclasses
Split CValidationState into TxValidationState and BlockValidationState
to store validation results for transactions and blocks respectively.
2019-10-29 15:46:45 -04:00
James O'Beirne
707fde7b9b add unused SnapshotMetadata class 2019-10-29 13:55:10 -04:00
Wladimir J. van der Laan
b688b859db
Merge #17004: validation: Remove REJECT code from CValidationState
9075d13153 [docs] Add release notes for removal of REJECT reasons (John Newbery)
04a2f326ec [validation] Fix REJECT message comments (John Newbery)
e9d5a59e34 [validation] Remove REJECT code from CValidationState (John Newbery)
0053e16714 [logging] Don't log REJECT code when transaction is rejected (John Newbery)
a1a07cfe99 [validation] Fix peer punishment for bad blocks (John Newbery)

Pull request description:

  We no longer send BIP 61 REJECT messages, so there's no need to set
  a REJECT code in the CValidationState object.

  Note that there is a minor bug fix in p2p behaviour here. Because the
  call to `MaybePunishNode()` in `PeerLogicValidation::BlockChecked()` only
  previously happened if the REJECT code was > 0 and < `REJECT_INTERNAL`,
  then there are cases were `MaybePunishNode()` can get called where it
  wasn't previously:

  - when `AcceptBlockHeader()` fails with `CACHED_INVALID`.
  - when `AcceptBlockHeader()` fails with `BLOCK_MISSING_PREV`.

  Note that `BlockChecked()` cannot fail with an 'internal' reject code. The
  only internal reject code was `REJECT_HIGHFEE`, which was only set in
  ATMP.

  This reverts a minor bug introduced in 5d08c9c579.

ACKs for top commit:
  ariard:
    ACK 9075d13, changes since last reviewed are splitting them in separate commits to ease understanding and fix nits
  fjahr:
    ACK 9075d13153, confirmed diff to last review was fixing nits in docs/comments.
  ryanofsky:
    Code review ACK 9075d13153. Only changes since last review are splitting the main commit and updating comments

Tree-SHA512: 58e8a1a4d4e6f156da5d29fb6ad6a62fc9c594bbfc6432b3252e962d0e9e10149bf3035185dc5320c46c09f3e49662bc2973ec759679c0f3412232087cb8a3a7
2019-10-24 10:49:45 +02:00
practicalswift
084e17cebd Remove unused includes 2019-10-15 22:56:43 +00:00
John Newbery
e9d5a59e34 [validation] Remove REJECT code from CValidationState
We no longer send BIP 61 REJECT messages, so there's no need to set
a REJECT code in the CValidationState object.
2019-10-10 13:31:53 -04:00
MarcoFalke
fa25f43ac5
p2p: Remove BIP61 reject messages 2019-10-02 10:39:14 -04:00
MarcoFalke
fa607c2292
validation: Make GetWitnessCommitmentIndex public 2019-09-24 11:16:05 -04:00
James O'Beirne
3cf36736e5 refactoring: move ReplayBlocks under CChainState 2019-09-17 09:45:38 -04:00
James O'Beirne
bcf73d3b84 refactoring: move LoadChainTip to CChainState method 2019-09-17 09:45:38 -04:00
James O'Beirne
f5809d5b13 doc: fix CChainState::ActivateBestChain doc 2019-09-17 09:45:38 -04:00
MarcoFalke
fa912a8ad5
doc: move-only ActivateBestChain doxygen comment to header 2019-09-10 15:48:23 +03:00
MarcoFalke
fa99efd054
doc: ActivateBestChainStep return value 2019-09-03 07:32:33 -04:00
nicolas.dorier
ce7eac3cb0
[Fix] The default whitelistrelay should be true 2019-08-17 00:43:22 +09:00
MarcoFalke
1bf2ff2bf8
Merge #16060: Bury bip9 deployments
e78aaf41f4 [docs] Add release notes for burying bip 9 soft fork deployments (John Newbery)
8319e738f9 [tests] Add coverage for the content of getblockchaininfo.softforks (James O'Beirne)
0328dcdcfc [Consensus] Bury segwit deployment (John Newbery)
1c93b9b31c [Consensus] Bury CSV deployment height (John Newbery)
3862e473f0 [rpc] Tidy up reporting of buried and ongoing softforks (John Newbery)

Pull request description:

  This hardcodes CSV and segwit activation heights, similar to the BIP 90 buried deployments for BIPs 34, 65 and 66.

  CSV and segwit have been active for over 18 months. Hardcoding the activation height is a code simplification, makes it easier to understand segwit activation status, and reduces technical debt.

  This was originally attempted by jl2012 in #11398 and again by me in #12360.

ACKs for top commit:
  ajtowns:
    ACK e78aaf41f4 ; checked diff to previous acked commit, checked tests still work
  ariard:
    ACK e78aaf4, check diff, run the tests again and successfully activated csv/segwit heights on mainnet as expected.
  MarcoFalke:
    ACK e78aaf41f4 (still didn't check if the mainnet block heights are correct, but the code looks good now)

Tree-SHA512: 7e951829106e21a81725f7d3e236eddbb59349189740907bb47e33f5dbf95c43753ac1231f47ae7bee85c8c81b2146afcdfdc11deb1503947f23093a9c399912
2019-08-15 16:02:10 -04:00
James O'Beirne
582d2cd747 Cover UTXO set access with lock annotations
i.e. any CoinsViews members. Adds a lock acquisition to `gettxoutsetinfo` RPC
to comply with added annotations.

Co-authored-by: Russell Yanofsky <russ@yanofsky.org>
2019-08-15 11:19:40 -04:00
James O'Beirne
5693530685 refactor: have CCoins* data managed under CChainState
This change encapsulates UTXO set data within CChainState instances, removing
global data `pcoinsTip` and `pcoinsviewdb`. This is necessary if we want to
maintain multiple chainstates with their own rendering of the UTXO set.

We introduce a class CoinsViews which consolidates the construction of a
CCoins* hierarchy. Construction of its various pieces (db, coinscatcher,
in-memory cache) is split up so that we avoid flushing bad state to disk if
startup is interrupted.

We also introduce `CChainState::CanFlushToDisk()` which tells us when it is
safe to flush the chainstate based on this partial construction.

This commit could be broken into smaller pieces, but it would require more
ephemeral diffs to, e.g., temporarily change CCoinsViewDB's constructor
invocations.

Other changes:

- A parameter has been added to the CCoinsViewDB constructor that allows the
  name of the corresponding leveldb directory to be specified.

Thanks to Russell Yanofsky and Marco Falke for helpful feedback.
2019-08-15 11:04:10 -04:00
John Newbery
0328dcdcfc [Consensus] Bury segwit deployment
Hardcode segwit deployment height to 481824 for mainnet.
2019-08-14 15:52:52 -04:00
James O'Beirne
fae6ab6aed refactor: pcoinsTip -> CChainState::CoinsTip()
This aliasing makes subsequent commits easier to review; eventually CoinsTip()
will return the CCoinsViewCache managed by CChainState.
2019-08-06 13:13:06 -04:00
Wladimir J. van der Laan
51a6e2c419
Merge #15681: [mempool] Allow one extra single-ancestor transaction per package
50cede3f5a [mempool] Allow one extra single-ancestor transaction per package (Matt Corallo)

Pull request description:

  This implements the proposed policy change from [1], which allows
  certain classes of contract protocols involving revocation
  punishments to use CPFP. Note that some such use-cases may still
  want some form of one-deep package relay, though even this alone
  may greatly simplify some lightning fee negotiation.

  [1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html

ACKs for top commit:
  ajtowns:
    ACK 50cede3f5a -- looked over code again, compared with previous commit, compiles, etc.
  sdaftuar:
    ACK 50cede3f5a
  ryanofsky:
    utACK 50cede3f5a. Changes since last review: adding EXTRA_DESCENDANT_TX_SIZE_LIMIT constant, changing max ancestor size from 1,000,000 to nLimitAncestorSize constant (101,000), fixing test comment and getting rid of unused test node.

Tree-SHA512: b052c2a0f384855572b4579310131897b612201214b5abbb225167224e4f550049e300b471dbf320928652571e92ca2d650050b7cf39ac92b3bc1d2bcd386c1c
2019-07-19 20:00:12 +02:00
fanquake
59ce537a49
Merge #16152: Disable bloom filtering by default.
bead32e31e Add release notes for DEFAULT_BLOOM change (Matt Corallo)
f27309f55c Move DEFAULT_PEERBLOOMFILTERS from validation.h to net_processing.h (Matt Corallo)
5efcb77283 Disable bloom filtering by default. (Matt Corallo)

Pull request description:

  BIP 37 bloom filters have been well-known to be a significant DoS
  target for some time. However, in order to provide continuity for
  SPV clients relying on it, the NODE_BLOOM service flag was added,
  and left as a default, to ensure sufficient nodes exist with such a
  flag.

  NODE_BLOOM is, at this point, well-established and, as long as
  there exist 0.18 nodes with default config (which I'd anticipate
  will be true for many years), will be available from some peers. By
  that time, the continued slowdown of BIP 37-based filtering will
  likely have rendered it useless (though this is already largely the
  case). Further, BIP 37 was deliberately never updated to support
  witness-based filtering as newer wallets are expected to migrate to
  some yet-to-be-network-exposed filters.

ACKs for top commit:
  jnewbery:
    ACK bead32e31e
  kallewoof:
    ACK bead32e31e

Tree-SHA512: ecd901898e8efe1a7c82b471af0acc2373c2282ac633eb58d9aae7c35deda1999d0f79fb0485e6cecbda7246aeda00206cd82c7fa36866e2ac64705ba93f9390
2019-07-19 17:33:56 +08:00
Matt Corallo
50cede3f5a [mempool] Allow one extra single-ancestor transaction per package
This implements the proposed policy change from [1], which allows
certain classes of contract protocols involving revocation
punishments to use CPFP. Note that some such use-cases may still
want some form of one-deep package relay, though even this alone
may greatly simplify some lightning fee negotiation.

[1] https://lists.linuxfoundation.org/pipermail/bitcoin-dev/2018-November/016518.html
2019-07-09 15:46:25 -04:00
James O'Beirne
682a1d0f20 refactoring: remove mapBlockIndex global
in lieu of ::BlockIndex().
2019-07-08 11:33:13 -04:00
James O'Beirne
55d525ab90 refactoring: make pindexBestInvalid internal to validation.cpp
There's no need to have this member live on CChainState since it's only used
in validation.cpp.
2019-07-08 11:33:13 -04:00
James O'Beirne
4ed55dfcd7 refactoring: add block_index_candidates arg to LoadBlockIndex
Prevents BlockManager from having to reference ChainstateActive()
within one of its methods which improves encapsulation and makes
testing easier.
2019-07-08 11:33:12 -04:00
James O'Beirne
613c46fe9e refactoring: move block metadata structures into BlockManager
Separate out the management of chain-agnostic block metadata from any given
CChainState instance. This allows us to avoid duplicating data like
`mapBlockIndex` unnecessarily for multiple chainstates.

This also adds a CChainState constructor that accepts and sets m_blockman.
Ultimately this reference will point to a BlockMan instance that
is shared across CChainStates.

This commit can be decomposed into smaller commits if necessary.
2019-07-08 11:33:12 -04:00
Wladimir J. van der Laan
c6e42f1ca9
Merge #14193: validation: Add missing mempool locks
fa2b083c3f [test] Add test to check mempool consistency in case of reorgs (MarcoFalke)
fabeb1f613 validation: Add missing mempool locks (MarcoFalke)
fa0c9dbf91 txpool: Make nTransactionsUpdated atomic (MarcoFalke)

Pull request description:

  Take the mempool read lock during reorgs, so that we don't accidentally read an inconsistent mempool.

ACKs for top commit:
  laanwj:
    code review ACK fa2b083c3f
  ryanofsky:
    utACK fa2b083c3f [EDIT: was ~e284e422e75189794e24fe482819d8b1407857c3~, from bad copy and paste]. Changes since last review: rebase after #15976, adding vTxHashes lock annotation, adding new commit dropping mempool lock for nTransactionsUpdated and making it atomic to avoid deadlock between mempool lock and g_best_block_mutex

Tree-SHA512: cfe7777993589087753e000e3736d79d320dca412383fb77b56bef8946a04049722bf888c11b6f722adf677165185c7e58b4a269f7c5fa25e84dda375f6c8a7d
2019-07-02 16:29:08 +02:00
Matt Corallo
8053e5cdad Remove -mempoolreplacement to prevent needless block prop slowness.
At this point there is no reasonable excuse to disable opt-in RBF,
and, unlike when this option was added, there are now significant
issues created when disabling it (in the form of compact block
reconstruction failures). Further, it breaks a lot of modern wallet
behavior.
2019-06-08 09:32:33 -04:00
MarcoFalke
fabeb1f613
validation: Add missing mempool locks 2019-06-07 11:07:09 +02:00
Matt Corallo
f27309f55c Move DEFAULT_PEERBLOOMFILTERS from validation.h to net_processing.h 2019-06-06 10:01:13 -04:00
Matt Corallo
5efcb77283 Disable bloom filtering by default.
BIP 37 bloom filters have been well-known to be a significant DoS
target for some time. However, in order to provide continuity for
SPV clients relying on it, the NODE_BLOOM service flag was added,
and left as a default, to ensure sufficient nodes exist with such a
flag.

NODE_BLOOM is, at this point, well-established and, as long as
there exist 0.18 nodes with default config (which I'd anticipate
will be true for many years), will be available from some peers. By
that time, the continued slowdown of BIP 37-based filtering will
likely have rendered it useless (though this is already largely the
case). Further, BIP 37 was deliberately never updated to support
witness-based filtering as newer wallets are expected to migrate to
some yet-to-be-network-exposed filters.
2019-06-06 10:01:13 -04:00
Wladimir J. van der Laan
5d37c1bde0
Merge #15976: refactor: move methods under CChainState (pt. 1)
403e677c9 refactoring: IsInitialBlockDownload -> CChainState (James O'Beirne)
3ccbc376d refactoring: FlushStateToDisk -> CChainState (James O'Beirne)
4d6688603 refactoring: introduce ChainstateActive() (James O'Beirne)
d7c97edee move-only: make the CChainState interface public (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal

  ---

  This changeset starts moving functionality intimately related to CChainState into methods. Parameterizing these functions by a particular CChainState is necessary for the use of multiple chainstates simultaneously (e.g. for asynchronous background validation).

  In this change, we
  - make the CChainState interface public - since other units will start to invoke its methods directly,
  - introduce `::ChainstateActive()`, the CChainState equivalent for `::ChainActive()`,
  - and move `IsInitialBlockDownload()` and `FlushStateToDisk()` into methods on CChainState.

  Independent of assumeutxo, these changes better encapsulate chainstate behavior and allow easier use from a testing context.

  There are more methods that we'll move in the future, but they require other substantial changes (i.e. moving ownership of the `CCoinsView*` hierarchy into CChainState) so we'll save them for future PRs.

  ---

  The first move-only commit is most easily reviewed with `git diff ... --color-moved=dimmed_zebra`.

ACKs for commit 403e67:
  Empact:
    utACK 403e677c9e no need to address my nits herein
  Sjors:
    utACK 403e677
  ryanofsky:
    utACK 403e677c9e. Only change since previous review is removing global state comment as suggested.
  MarcoFalke:
    utACK 403e677c9e, though the diff still seems a bit bloated with some unnecessary changes in the second commit.
  promag:
    utACK 403e677 and rebased with current [master](c7cfd20a7).

Tree-SHA512: 6fcf260bb2dc201361170c0b4547405366f5f331fcc3a2bac29b24442814b7b244ca1b58aac5af716885f9a130c343b544590dff780da0bf835c7c5b3ccb2257
2019-06-05 11:56:23 +02:00
James O'Beirne
403e677c9e refactoring: IsInitialBlockDownload -> CChainState
We introduce CChainState.m_cached_finished_ibd because the static state it
replaces would've been shared across all CChainState instances.
2019-05-16 09:06:54 -04:00
James O'Beirne
3ccbc376dd refactoring: FlushStateToDisk -> CChainState
Also renames global methods for clarity:

- ::FlushStateToDisk() -> CChainState::ForceFlushStateToDisk()
  - This performs an unconditional flush.

- ::PruneAndFlush() -> CChainState::PruneAndFlush()
2019-05-16 09:06:54 -04:00
James O'Beirne
4d6688603b refactoring: introduce ChainstateActive()
To be used once we move global functions (e.g. FlushStateToDisk()) into
CChainState methods.

Thanks to Marco Falke for suggestions
2019-05-16 09:06:54 -04:00
James O'Beirne
d7c97edeea move-only: make the CChainState interface public
along with DisconnectResult, and CBlockIndexWorkComparator.

The CChainState interface needs to be known to the rest of the system because
many global functions will move to CChainState methods. This is to allow
other parts of the system to be parameterized per chainstate instance
instead of assuming a single global.
2019-05-16 09:05:07 -04:00
MarcoFalke
fa3c651143
[refactor] interfaces: Add missing LockAnnotation for cs_main 2019-05-13 14:46:01 -04:00
MarcoFalke
49c1aa5f83
Merge #15971: validation: Add compile-time checking for negative locking requirement in LimitValidationInterfaceQueue
62d50ef308 Add LOCKS_EXCLUDED(cs_main) to LimitValidationInterfaceQueue(...) which does AssertLockNotHeld(cs_main) (practicalswift)

Pull request description:

  This PR adds compile-time checking for negative locking requirements that follow from the run-time locking requirement `AssertLockNotHeld(cs_main)` in `LimitValidationInterfaceQueue(...)`.

  Changes:
  * Add `LOCKS_EXCLUDED(cs_main)` to `LimitValidationInterfaceQueue(...)` which does `AssertLockNotHeld(cs_main)`
  * Add `LOCKS_EXCLUDED(cs_main)` to `CChainState::ActivateBestChain(…)`, `CChainState:: InvalidateBlock(…)` and `CChainState::RewindBlockIndex(…)` which all call `LimitValidationInterfaceQueue(...)` which does `AssertLockNotHeld(cs_main)`
  * Add `LOCKS_EXCLUDED(cs_main)` to `InvalidateBlock(…)` which calls `CChainState::InvalidateBlock(...)` which in turn calls `LimitValidationInterfaceQueue(...)` which  does `AssertLockNotHeld(cs_main)`
  * Add `LOCKS_EXCLUDED(cs_main)` to `RewindBlockIndex(…)` which calls `CChainState::RewindBlockIndex(...)` which in turn calls `LimitValidationInterfaceQueue(...)` which does `AssertLockNotHeld(cs_main)`

ACKs for commit 62d50e:
  MarcoFalke:
    utACK 62d50ef308

Tree-SHA512: 73d092ccd08c851ae3c5d60370c369fc030c5793f5507e2faccb6f91c851ddc0ce059fbea3899f2856330d7a8c78f2ac6a2988e8268b03154f946be9e60e3be1
2019-05-08 09:19:41 -04:00
practicalswift
62d50ef308 Add LOCKS_EXCLUDED(cs_main) to LimitValidationInterfaceQueue(...) which does AssertLockNotHeld(cs_main)
Add LOCKS_EXCLUDED(cs_main) to functions calling LimitValidationInterfaceQueue(...) which does AssertLockNotHeld(cs_main)

Add LOCKS_EXCLUDED(cs_main) to functions calling CChainState::InvalidateBlock(...) which calls LimitValidationInterfaceQueue(...) which in turn does AssertLockNotHeld(cs_main)

Add LOCKS_EXCLUDED(cs_main) to functions calling CChainState::RewindBlockIndex(...) which calls LimitValidationInterfaceQueue(...) which in turn does AssertLockNotHeld(cs_main)
2019-05-08 10:31:54 +02:00
James O'Beirne
486c1eea86 refactoring: remove unused chainActive 2019-05-03 15:03:05 -04:00
James O'Beirne
631940aab2 scripted-diff: replace chainActive -> ::ChainActive()
Though at the moment ChainActive() simply references `g_chainstate.m_chain`,
doing this change now clears the way for multiple chainstate usage and allows
us to script the diff.

-BEGIN VERIFY SCRIPT-
git grep -l "chainActive" | grep -E '(h|cpp)$' | xargs sed -i '/chainActive =/b; /extern CChain& chainActive/b; s/\(::\)\{0,1\}chainActive/::ChainActive()/g'
-END VERIFY SCRIPT-
2019-05-03 15:02:54 -04:00
James O'Beirne
a3a609079c refactoring: introduce unused ChainActive()
in preparation for the following scripted-diff commit.
2019-05-03 14:38:38 -04:00
MarcoFalke
12aa2ac988
Merge #15323: rpc: Expose g_is_mempool_loaded via getmempoolinfo
effe81f750 Move g_is_mempool_loaded into CTxMemPool::m_is_loaded (Ben Woosley)
bb8ae2c419 rpc: Expose g_is_mempool_loaded via getmempoolinfo and /rest/mempool/info.json (Ben Woosley)

Pull request description:

  And use it to fix a race condition in mempool_persist.py:
  https://travis-ci.org/Empact/bitcoin/jobs/487577243

  Since e.g. getrawmempool returns errors based on this status, this
  enables users to test it for readiness.

  Fixes #12863

ACKs for commit effe81:
  MarcoFalke:
    utACK effe81f750
  jnewbery:
    utACK effe81f750

Tree-SHA512: 74328b0c17a97efb8a000d4ee49b9a673c2b6dde7ea30c43a6a2eff961a233351c9471f9a42344412135786c02bdf2ee1b2526651bb8fed68bd94d2120c4ef86
2019-05-01 10:06:15 -04:00