Commit graph

692 commits

Author SHA1 Message Date
fanquake
d82b2c6e65
Merge #19898: log: print unexpected version warning in validation log category
62dba9628d log: print unexpected version warning in validation log category (nthumann)

Pull request description:

  Fixes #19603: As suggested by practicalswift, instead of always printing `<n> of the last 100 blocks have unexpected version` as a warning appended to UpdateTip, it is now printed in the validation log category and therefore only visible with `-debug=validation` enabled.

  Before:
  `2020-09-06T15:56:00Z UpdateTip: new best=00000000000000000001b2872e107a98b57913120e5c6c87ce2715a34c40adf8 height=646969 version=0x20400000 log2_work=92.261571 tx=565651941 date='2020-09-06T10:35:36Z' progress=0.999888 cache=32.2MiB(237417txo) warning='72 of last 100 blocks have unexpected version'`
  After:
  `2020-09-06T16:31:26Z UpdateTip: new best=0000000000000000000b3bd786dc42745dd7be4a8c695500a04518cb9e2f4dc1 height=646971 version=0x20000000 log2_work=92.261607 tx=565655901 date='2020-09-06T10:57:19Z' progress=0.999883 cache=3.8MiB(27550txo)`
  `2020-09-06T16:31:26Z 71 of last 100 blocks have unexpected version`

  Ran unit & functional tests, confirmed that the warning is now only printed when validation category is enabled.

ACKs for top commit:
  theStack:
    ACK 62dba9628d
  MarcoFalke:
    re-ACK 62dba96
  practicalswift:
    ACK 62dba9628d -- only change since last ACK is `s/nUpgraded/num_unexpected_version/`
  hebasto:
    re-ACK 62dba9628d, https://github.com/bitcoin/bitcoin/pull/19898#pullrequestreview-483158708 is resolved now.

Tree-SHA512: 2100ca7d6d3fd67c92e81d75162d2506d6f1ecf1761d5180d76663fac06771b35e5c4235ebe1a00731b5f7db82db3cd19328627929c8f22912df592686ba51d3
2020-09-29 20:41:11 +08:00
MarcoFalke
1b313cacc9
Merge #19927: validation: Reduce direct g_chainman usage
72a1d5c6f3 validation: Remove review-only comments + assertions (Carl Dong)
3756853b15 docs: Move FindFilesToPrune{,Manual} doxygen comment (Carl Dong)
485899a93c style: Make FindFilesToPrune{,Manual} match style guide (Carl Dong)
3f5b5f3f6d validation: Move FindFilesToPrune{,Manual} to BlockManager (Carl Dong)
f8d4975ab3 validation: Move PruneOneBlockFile to BlockManager (Carl Dong)
74f73c783d validation: Pass in chainman to UnloadBlockIndex (Carl Dong)
4668ded6d6 validation: Move ~CMainCleanup logic to ~BlockManager (Carl Dong)

Pull request description:

  This PR paves the way for de-globalizing `g_chainman` entirely by removing the usage of `g_chainman` in the following functions/methods:
  - `~CMainCleanup`
  - `CChainState::FlushStateToDisk`
  - `UnloadBlockIndex`

  The remaining direct uses of `g_chainman` are as follows:
  1. In initialization codepaths:
  	- `AppTests`
  	- `AppInitMain`
  	- `TestingSetup::TestingSetup`
  2. `::ChainstateActive`
  3. `LookupBlockIndex`
  	- Note: `LookupBlockIndex` is used extensively throughout the codebase and require a much larger set of changes, therefore I've left it out of this initial PR

ACKs for top commit:
  MarcoFalke:
    re-ACK 72a1d5c6f3 👚
  jnewbery:
    utACK 72a1d5c6f3

Tree-SHA512: 944a4fa8405eecf39706ff944375d6824373aaeea849d11473f08181eff26b12f70043a8348a5b08e6e9021b243b481842fbdfbc7c3140ca795fce3688b7f5c3
2020-09-23 20:35:54 +02:00
Gregory Sanders
e76fc2b84d Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas
Using the zmq notifications to avoid excessive mempool polling can be difficult
given the current notifications available. It announces all transactions
being added to mempool or included in blocks, but announces no evictions
and gives no indication if the transaction is in the mempool or a block.

Block notifications for zmq are also substandard, in that it only announces
block tips, while all block transactions are still announced.

This commit adds a unified stream which can be used to closely track mempool:

1) getrawmempool to fill out mempool knowledge
2) if txhash is announced, add or remove from set
based on add/remove flag
3) if blockhash is announced, get block txn list,
remove from those transactions local view of mempool
4) if we drop a sequence number, go to (1)

The mempool sequence number starts at the value 1, and
increments each time a transaction enters the mempool,
or is evicted from the mempool for any reason, including
block inclusion. The mempool sequence number is published
via ZMQ for any transaction-related notification.

These features allow for ZMQ/RPC consumer to track mempool
state in a more exacting way, without unnecesarily polling
getrawmempool. See interface_zmq.py::test_mempool_sync for
example usage.
2020-09-22 11:34:30 -04:00
Wladimir J. van der Laan
8c5f68118c
Merge #18267: BIP-325: Signet [consensus]
8258c4c007 test: some sanity checks for consensus logic (Anthony Towns)
e47ad375bf test: basic signet tests (Karl-Johan Alm)
4c189abdc4 test: add small signet fuzzer (practicalswift)
ec9b25d046 test: signet network selection tests (Karl-Johan Alm)
3efe298dcc signet: hard-coded parameters for Signet Global Network VI (2020-09-07) (Karl-Johan Alm)
c7898bca4e qt: update QT to support signet network (Karl-Johan Alm)
a8de47a1c9 consensus: add signet validation (Karl-Johan Alm)
e8990f1214 add signet chain and accompanying parameters (Karl-Johan Alm)
404682b7cd add signet basic support (signet.cpp) (Karl-Johan Alm)
a2147d7dad validation: move GetWitnessCommitmentIndex to consensus/validation (Karl-Johan Alm)

Pull request description:

  This PR is a part of BIP-325 (https://github.com/bitcoin/bips/blob/master/bip-0325.mediawiki), and is a sub-PR of #16411.

  * Signet consensus (this)
  * Signet RPC tools (pending)
  * Signet utility scripts (contrib/signet) (pending)

ACKs for top commit:
  jonatack:
    re-ACK 8258c4c007 per `git diff dbeea65 8258c4c`, only change since last review is updated `-signet*` config option naming.
  fjahr:
    re-ACK 8258c4c
  laanwj:
    ACK 8258c4c007
  MarcoFalke:
    Approach ACK 8258c4c007 🌵

Tree-SHA512: 5d158add96755910837feafa8214e13695b769a6aec3a2da753cf672618bef377fac43b0f4b772a87b25dd9f0c1c9b29f2789785d7a7d47a155cdcf48f7c975d
2020-09-21 22:33:00 +02:00
Carl Dong
72a1d5c6f3
validation: Remove review-only comments + assertions
[META] This is a followup to "validation: Move FindFilesToPrune{,Manual}
       to BlockManager" removing comments and assertions meant only to
       show that the change is correct.
2020-09-21 13:30:27 -04:00
Carl Dong
3756853b15
docs: Move FindFilesToPrune{,Manual} doxygen comment
[META] This is a pure comment commit.

They belong in the member declarations in the header file.
2020-09-21 13:30:21 -04:00
Carl Dong
485899a93c
style: Make FindFilesToPrune{,Manual} match style guide
[META] This is a pure style commit.
2020-09-21 13:28:08 -04:00
Carl Dong
3f5b5f3f6d
validation: Move FindFilesToPrune{,Manual} to BlockManager
[META] No behaviour change is intended in this commit.
[META] This commit should be followed up by removing the comments and
       assertions meant only to show that the change is correct.

Also stop FindFilesToPrune{,Manual} from unnecessary reaching for
::ChainActive() by passing in the necessary information.
2020-09-21 13:27:44 -04:00
fanquake
c30f79d418
Merge #19940: rpc: Return fee and vsize from testmempoolaccept
23c35bf005 [test] add get_vsize util for more programmatic testing (gzhao408)
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept (codeShark149)

Pull request description:

  From #19093 and resolves #19057.

  Difference from #19093: return `vsize` and `fees` object (similar to `getmempoolentry`) when the test accept is successful. Updates release-notes.md.

ACKs for top commit:
  jnewbery:
    utACK 23c35bf005
  fjahr:
    utACK 23c35bf
  instagibbs:
    reACK 23c35bf005

Tree-SHA512: dcb81b7b817a4684e9076bc5d427a6f2d549d2edc66544e718260c4b5f8f1d5ae1d47b754175e9f0c8a3bd8371ce116c2dca0583588d513a7d733d5d614f2b04
2020-09-19 15:04:03 +08:00
Karl-Johan Alm
a8de47a1c9
consensus: add signet validation 2020-09-18 09:37:57 +09:00
codeShark149
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept
Return fee and vsize if tx would pass ATMP.
2020-09-15 18:01:32 -07:00
fanquake
1c4f59728c
Merge #19879: [p2p] miscellaneous wtxid followups
a8a64acaf3 [BroadcastTransaction] Remove unsafe move operator (Amiti Uttarwar)
125c038126 [p2p] Remove dead code (Amiti Uttarwar)
fc66d0a65c [p2p] Check for nullptr before dereferencing pointer (Adam Jonas)
cb79b9dbf4 [mempool] Revert unbroadcast set to tracking just txid (Amiti Uttarwar)

Pull request description:

  Addresses some outstanding review comments from #18044

  - reverts unbroadcast txids to a set instead of a map (simpler, communicates intent better, takes less space, no efficiency advantages of map)
  - adds safety around two touchpoints (check for nullptr before dereferencing pointer, remove an inaccurate std::move operator)
  - removes some dead code

  Links to comments on wtxid PR: [1](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r460495254) [2](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r460496023) [3](https://github.com/bitcoin/bitcoin/pull/18044#discussion_r463532611)

  thanks to jnewbery & adamjonas for flagging these ! !

ACKs for top commit:
  sdaftuar:
    utACK a8a64acaf3
  naumenkogs:
    utACK a8a64acaf3
  jnewbery:
    utACK a8a64acaf3

Tree-SHA512: 7be669cb30cc17fb9e06b50e636ef7887c6a27354697987e4e4d38dba4b8f50e175647587430cd9bc3295bec01ce8b1e6639a50a4249d8fff9b1ca1b9ead3277
2020-09-16 06:30:57 +08:00
Carl Dong
f8d4975ab3
validation: Move PruneOneBlockFile to BlockManager
[META] This is a pure refactor commit.

Move PruneBlockFile to BlockManager because:
1. PruneOneBlockFile only acts on BlockManager
2. Eliminates the need for callers (FindFilesToPrune{,Manual}) to have a
   reference to the larger ChainstateManager, just a reference to
   BlockManager is enough. See following commits.
2020-09-15 14:13:44 -04:00
Carl Dong
74f73c783d
validation: Pass in chainman to UnloadBlockIndex 2020-09-15 14:11:34 -04:00
Carl Dong
4668ded6d6
validation: Move ~CMainCleanup logic to ~BlockManager
~CMainCleanup:
1. Is vestigial
2. References the g_chainman global (we should minimize g_chainman refs)
3. Only acts on g_chainman.m_blockman
4. Does the same thing as BlockManager::Unload
2020-09-14 10:42:45 -04:00
Karl-Johan Alm
a2147d7dad
validation: move GetWitnessCommitmentIndex to consensus/validation 2020-09-10 10:47:40 +09:00
nthumann
62dba9628d
log: print unexpected version warning in validation log category
Instead of printing "<n> of the last 100 blocks have unexpected version"
as a warning appended to UpdateTip, it is now printed in the validation
log category.
2020-09-09 20:57:06 +02:00
MarcoFalke
fafb381af8
Remove mempool global 2020-09-05 16:24:56 +02:00
MarcoFalke
eeee1104d7
Remove mempool global from init
Can be reviewed with the git diff options

--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space --ignore-all-space
2020-09-05 16:24:08 +02:00
Amiti Uttarwar
cb79b9dbf4 [mempool] Revert unbroadcast set to tracking just txid
When I originally implemented the unbroadcast set in 18038, it just tracked
txids. After 18038 was merged, I offered a patch to 18044 to make the
unbroadcast changes compatible with wtxid relay. In this patch, I updated
`unbroadcast_txids` to a map of txid -> wtxid. Post merge review comments shed
light on the fact that this update was unnecessary, and distracting. So, this
commit updates the unbroadcast ids back to a set.
2020-09-04 14:29:29 -07:00
MarcoFalke
fa0572d0f3
Pass mempool reference to chainstate constructor 2020-08-28 10:42:04 +02:00
Wladimir J. van der Laan
b75f2ad72d
Merge #19660: refactor: Make HexStr take a span
0a8aa626dd refactor: Make HexStr take a span (Wladimir J. van der Laan)

Pull request description:

  Make `HexSt`r take a span of bytes, instead of an awkward pair of templated iterators. This simplifies most of the uses.

ACKs for top commit:
  elichai:
    Code review ACK 0a8aa626dd
  hebasto:
    re-ACK 0a8aa626dd
  jonatack:
    re-ACK 0a8aa626dd

Tree-SHA512: 6e178ece5cbac62119c857a10299b1e85422938084c3f03063e17119a5129e0c28016e05a6fabaa4c271a7e0a37c7cd89fa47c435ee19b38a5acfe80d00de992
2020-08-09 15:35:58 +02:00
fanquake
6d8543504d
Merge #19620: Add txids with non-standard inputs to reject filter
9f88ded82b test addition of unknown segwit spends to txid reject filter (Gregory Sanders)
7989901c7e Add txids with non-standard inputs to reject filter (Suhas Daftuar)

Pull request description:

  Our policy checks for non-standard inputs depend only on the non-witness
  portion of a transaction: we look up the scriptPubKey of the input being
  spent from our UTXO set (which is covered by the input txid), and the p2sh
  checks only rely on the scriptSig portion of the input.

  Consequently it's safe to add txids of transactions that fail these checks to
  the reject filter, as the witness is irrelevant to the failure. This is helpful
  for any situation where we might request the transaction again via txid (either
  from txid-relay peers, or if we might fetch the transaction via txid due to
  parent-fetching of orphans).

  Further, in preparation for future witness versions being deployed on the
  network, ensure that WITNESS_UNKNOWN transactions are rejected in
  AreInputsStandard(), so that transactions spending v1 (or greater) witness
  outputs will fall into this category of having their txid added to the reject
  filter.

ACKs for top commit:
  ajtowns:
    ACK 9f88ded82b - code review
  jnewbery:
    Code review ACK 9f88ded82b
  ariard:
    Code Review/Tested ACK 9f88ded
  naumenkogs:
    utACK 9f88ded82b
  jonatack:
    ACK 9f88ded82b

Tree-SHA512: 1e93c0a5b68cb432524780ffc0093db893911fdfed9e2ed17f888e59114cc75d2a07062aefad4e5ce2e87c9270886117a8abb3c78fb889c9b9f31967f1777148
2020-08-07 07:34:27 +08:00
Wladimir J. van der Laan
0a8aa626dd refactor: Make HexStr take a span
Make HexStr take a span of bytes, instead of an awkward pair of
templated iterators.
2020-08-06 19:41:43 +02:00
Suhas Daftuar
7989901c7e Add txids with non-standard inputs to reject filter
Our policy checks for non-standard inputs depend only on the non-witness
portion of a transaction: we look up the scriptPubKey of the input being
spent from our UTXO set (which is covered by the input txid), and the p2sh
checks only rely on the scriptSig portion of the input.

Consequently it's safe to add txids of transactions that fail these checks to
the reject filter, as the witness is irrelevant to the failure. This is helpful
for any situation where we might request the transaction again via txid (either
from txid-relay peers, or if we might fetch the transaction via txid due to
parent-fetching of orphans).

Further, in preparation for future witness versions being deployed on the
network, ensure that WITNESS_UNKNOWN transactions are rejected in
AreInputsStandard(), so that transactions spending v1 (or greater) witness
outputs will fall into this category of having their txid added to the reject
filter.
2020-08-04 13:29:40 -04:00
Pieter Wuille
02c4cc5c5d Make CHash256/CHash160 output to Span 2020-07-30 13:57:54 -07:00
Pieter Wuille
e549bf8a9a Make CHash256 and CHash160 consume Spans 2020-07-30 13:57:53 -07:00
MarcoFalke
fae8c28dae
Pass mempool pointer to GetCoinsCacheSizeState 2020-07-29 12:30:11 +02:00
MarcoFalke
fac674db20
Pass mempool pointer to UnloadBlockIndex 2020-07-29 12:29:51 +02:00
MarcoFalke
2f71a1ea35
Merge #18637: coins: allow cache resize after init
f19fdd47a6 test: add test for CChainState::ResizeCoinsCaches() (James O'Beirne)
8ac3ef4699 add ChainstateManager::MaybeRebalanceCaches() (James O'Beirne)
f36aaa6392 Add CChainState::ResizeCoinsCaches (James O'Beirne)
b223111da2 txdb: add CCoinsViewDB::ChangeCacheSize (James O'Beirne)

Pull request description:

  This is part of the [assumeutxo project](https://github.com/bitcoin/bitcoin/projects/11):

  Parent PR: #15606
  Issue: #15605
  Specification: https://github.com/jamesob/assumeutxo-docs/tree/master/proposal

  ---

  In the assumeutxo implementation draft (#15056), once a UTXO snapshot is loaded, a new chainstate object is created after initialization. This means that we have to reclaim some of the cache that we've allocated to the original chainstate (per `dbcache=`) to repurpose for the snapshot chainstate.

  Furthermore, it makes sense to have different cache allocations depending on which chainstate is more active. While the snapshot chainstate is working to get to the network tip (and the background validation chainstate is idle), it makes sense that the snapshot chainstate should have the majority of cache allocation. And contrariwise once the snapshot has reached network tip, most of the cache should be given to the background validation chainstate.

  This set of changes (detailed in the commit messages) allows us to dynamically resize the various coins caches. None of the functionality introduced here is used at the moment, but will be in the next AU PR (which introduces `ActivateSnapshot`).

  `ChainstateManager::MaybeRebalanceCaches()` defines the (somewhat normative) cache allocations between the snapshot and background validation chainstates. I'd be interested in feedback if anyone has thoughts on the proportions I've set there.

ACKs for top commit:
  ajtowns:
    weak utACK f19fdd47a6 -- didn't find any major problems, but not super confident that I didn't miss anything
  fjahr:
    Code review ACK f19fdd4
  ryanofsky:
    Code review ACK f19fdd47a6. Only change since last review is constructor cleanup (no change in behavior). I think the suggestions here from ajtowns and others are good, but shouldn't delay merging the PR (and hold up assumeutxo)

Tree-SHA512: fffb7847fb6993dd4a1a41cf11179b211b0b20b7eb5f7cf6266442136bfe9d43b830bbefcafd475bfd4af273f5573500594aa41fff03e0ed5c2a1e8562ff9269
2020-07-29 07:53:19 +02:00
MarcoFalke
fa5979d12f
rpc: Avoid useless mempool query in gettxoutproof 2020-07-26 16:44:07 +02:00
Wladimir J. van der Laan
ccef10261e
Merge #18044: Use wtxid for transaction relay
0a4f1422cd Further improve comments around recentRejects (Suhas Daftuar)
0e20cfedb7 Disconnect peers sending wtxidrelay message after VERACK (Suhas Daftuar)
cacd85209e test: Use wtxid relay generally in functional tests (Fabian Jahr)
8d8099e97a test: Add tests for wtxid tx relay in segwit test (Fabian Jahr)
9a5392fdf6 test: Update test framework p2p protocol version to 70016 (Fabian Jahr)
dd78d1d641 Rename AddInventoryKnown() to AddKnownTx() (Suhas Daftuar)
4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason (Suhas Daftuar)
97141ca442 Delay getdata requests from peers using txid-based relay (Suhas Daftuar)
46d78d47de Add p2p message "wtxidrelay" (Suhas Daftuar)
2d282e0cba ignore non-wtxidrelay compliant invs (Anthony Towns)
ac88e2eb61 Add support for tx-relay via wtxid (Suhas Daftuar)
8e68fc246d Add wtxids to recentRejects instead of txids (Suhas Daftuar)
144c385820 Add wtxids of confirmed transactions to bloom filter (Suhas Daftuar)
85c78d54af Add wtxid-index to orphan map (Suhas Daftuar)
08b39955ec Add a wtxid-index to mapRelay (Suhas Daftuar)
60f0acda71 Just pass a hash to AddInventoryKnown (Suhas Daftuar)
c7eb6b4f1f Add wtxid to mempool unbroadcast tracking (Amiti Uttarwar)
2b4b90aa8f Add a wtxid-index to the mempool (Suhas Daftuar)

Pull request description:

  Using txids (a transaction's hash, without witness) for transaction relay is problematic, post-segwit -- if a peer gives us a segwit transaction that fails policy checks, it could be because the txid associated with the transaction is definitely unacceptable to our node (regardless of the witness), or it could be that the transaction was malleated and with a different witness, the txid could be accepted to our mempool.

  We have a bloom filter of recently rejected transactions, whose purpose is to help us avoid redownloading and revalidating transactions that fail to be accepted, but because of this potential for witness malleability to interfere with relay of valid transactions, we do not use the filter for segwit transactions.  This issue is discussed at some length in #8279.  The effect of this is that whenever a segwit transaction that fails policy checks is relayed, a node would download that transaction from every peer announcing it, because it has no way presently to cache failure.  Historically this hasn't been a big problem, but if/when policy for accepting segwit transactions were to change (eg taproot, or any other change), we could expect older nodes talking to newer nodes to be wasting bandwidth because of this.

  As discussed in that issue, switching to wtxid-based relay solves this problem -- by using an identifier for a transaction that commits to all the data in our relay protocol, we can be certain if a transaction that a peer is announcing is one that we've already tried to process, or if it's something new.  This PR introduces support for wtxid-based relay with peers that support it (and remains backwards compatible with peers that use txids for relay, of course).

  Apart from code correctness, one issue to be aware of is that by downloading from old and new peers alike, we should expect there to be some bandwidth wasted, because sometimes we might download the same transaction via txid-relay as well as wtxid-relay.  The last commit in this PR implements a heuristic I want to analyze, which is to just delay relay from txid-relay peers by 2 seconds, if we have at least 1 wtxid-based peer.  I've just started running a couple nodes with this heuristic so I can measure how well it works, but I'm open to other ideas for minimizing that issue.  In the long run, I think this will be essentially a non-issue, so I don't think it's too big a concern, we just need to bite the bullet and deal with it during upgrade.

  Finally, this proposal would need a simple BIP describing the changes, which I haven't yet drafted.  However, review and testing of this code in the interim would be welcome.

  To do items:
  - [x] Write BIP explaining the spec here (1 new p2p message for negotiating wtxid-based relay, along with a new INV type)
  - [ ] Measure and evaluate a heuristic for minimizing how often a node downloads the same transaction twice, when connected to old and new nodes.

ACKs for top commit:
  naumenkogs:
    utACK 0a4f1422cd
  laanwj:
    utACK 0a4f1422cd

Tree-SHA512: d8eb8f0688cf0cbe9507bf738e143edab1f595551fdfeddc2b6734686ea26e7f156b6bfde38bad8bbbe8bec1857c7223e1687f8f018de7463dde8ecaa8f450df
2020-07-22 20:58:55 +02:00
Wladimir J. van der Laan
1397afc5ec
Merge #19526: log: Avoid treating remote misbehvior as local system error
fa56eda58e log: Avoid treating remote misbehvior as local system error (MarcoFalke)
fa492895b5 refactor: Switch ValidationState mode to C++11 enum class (MarcoFalke)

Pull request description:

  When logging failures of `CheckBlockHeader` (high-hash), they are always logged as system error. This is problematic for several reasons:

  * Submitting a blockheader that fails `CheckBlockHeader` over RPC will result in a debug log line that starts with `ERROR`. Proper behaviour should be to log not anything and instead only return the failure reason to the RPC user. This pull does not fix this issue entirely, but is a good first step in the right direction.

  * A misbehaving peer that sends us an invalid block header that fails `CheckBlockHeader` will result in a debug log line that starts with `ERROR`. Proper behavior should be to log the remote peer misbehavior if logging for that category was enabled. This pull fixes this issue for `CheckBlockHeader` and other functions can be adjusted as well if needed in follow-ups. This should be a good first step in the right direction.

ACKs for top commit:
  practicalswift:
    re-ACK fa56eda58e

Tree-SHA512: 9793191f5cb57bdff7c93926e94877e8ca2ef89dcebcf9eb155899c733961839ec7c3f9b9f001dc082ada4234fe6e75f6df431301678d6822325840771166d77
2020-07-22 19:48:55 +02:00
MarcoFalke
65a54d684f
Merge #18984: Remove unnecessary input blockfile SetPos
5fa067a27d Remove unnecessary blockfile SetPos (Tom Harding)

Pull request description:

  Nothing could have changed the position since we retrieved it a few statements earlier. This dates from commit 16d5194165.

ACKs for top commit:
  LarryRuane:
    ACK 5fa067a27d

Tree-SHA512: 459cc7226e186c231ffb67f0613f550e8eb940f1b8933c3bc4a4e8dd519c8d5d45884e8cfd9347039dab90a093644bbbb31be063baed1c6fc7984b6cb4f17c9f
2020-07-21 11:28:35 +02:00
Suhas Daftuar
4eb515574e Make TX_WITNESS_STRIPPED its own rejection reason
Previously, TX_WITNESS_MUTATED could be returned during transaction validation
for either transactions that had a witness that was non-standard, or for
transactions that had no witness but were invalid due to segwit validation
rules.

However, for txid/wtxid-relay considerations, net_processing distinguishes the
witness stripped case separately, because it affects whether a wtxid should be
able to be added to the reject filter. It is safe to add the wtxid of a
witness-mutated transaction to the filter (as that wtxid shouldn't collide with
the txid, and hence it wouldn't interfere with transaction relay from
txid-relay peers), but it is not safe to add the wtxid (== txid) of a
witness-stripped transaction to the filter, because that would interfere with
relay of another transaction with the same txid (but different wtxid) when
relaying from txid-relay peers.

Also updates the comment explaining this logic, and explaining that we can get
rid of this complexity once there's a sufficient deployment of wtxid-relaying
peers on the network.
2020-07-19 02:10:42 -04:00
Amiti Uttarwar
c7eb6b4f1f Add wtxid to mempool unbroadcast tracking 2020-07-18 19:00:01 -04:00
MarcoFalke
fa56eda58e
log: Avoid treating remote misbehvior as local system error 2020-07-15 14:53:58 +02:00
MarcoFalke
b26d62c49a
Merge #18990: log: Properly log txs rejected from mempool
fa9f20b647 log: Properly log txs rejected from mempool (MarcoFalke)

Pull request description:

  Currently `CheckTxInputs` rejections from the mempool are the only rejections that log directly and unconditionally to debug.log instead of leaving it to the caller. This has multiple issues:

  * A rejected RPC transaction will log a redundant failure reason to debug log. All other failures are merely reported to the RPC user.
  * A rejected p2p transaction will log the failure twice. Once with the `MEMPOOLREJ` flag, and once unconditionally.
  * A rejected orphan transaction will log no failure.

  Fix all issues by simply returning the state to the caller, like it is done for all other rejections.

  The patch includes whitespace fixups to highlight relevant parts of the codebase and simplify review.

ACKs for top commit:
  naumenkogs:
    utACK fa9f20b647
  rajarshimaitra:
    Concept ACK. Compiled and ran tests. `fa9f20b`
  jnewbery:
    code review ACK fa9f20b647

Tree-SHA512: 86cc17b2a9239c01c4fc3f254ad48ee1d3883266966b9811030176338b9ac3deaea7ea5babfb8bbf739d7440154e30011fede8f9313175f199d4a062af6494f7
2020-07-14 16:15:07 +02:00
Wladimir J. van der Laan
9a3c7afe29
Merge #19317: Add a left-justified width field to log2_work component for a uniform debug.log output
c858302280 Change format of log2_work for uniform output (zero-padded) (jmorgan)

Pull request description:

  Motivation:
  It's jarring to watch the output of `tail -f ~/btcdata/debug.log` scroll by and very frequently see columns not lining up correctly because `log2_work` somtimes has less precision than 8 digits.

  Current display:
  ```
  2020-06-18T02:54:42Z UpdateTip: new best=0000000000000000107f877e4920643f9fb06090fa7551cd1cdd83b857f520aa height=382038 version=0x00000003 log2_work=83.558653 tx=90953616 date='2015-11-04T17:11:44Z' progress=0.166675 cache=117.6MiB(966410txo)
  2020-06-18T02:54:51Z UpdateTip: new best=0000000000000000019a4de585d30d1a8cc13c7a1972d11b4945635c9556acb5 height=382039 version=0x00000003 log2_work=83.55868 tx=90955936 date='2015-11-04T17:19:39Z' progress=0.166679 cache=117.9MiB(968799txo)
  ```

  Display with this commit:
  ```
  2020-06-18T02:54:42Z UpdateTip: new best=0000000000000000107f877e4920643f9fb06090fa7551cd1cdd83b857f520aa height=382038 version=0x00000003 log2_work=83.558653 tx=90953616 date='2015-11-04T17:11:44Z' progress=0.166675 cache=117.6MiB(966410txo)
  2020-06-18T02:54:51Z UpdateTip: new best=0000000000000000019a4de585d30d1a8cc13c7a1972d11b4945635c9556acb5 height=382039 version=0x00000003 log2_work=83.55868  tx=90955936 date='2015-11-04T17:19:39Z' progress=0.166679 cache=117.9MiB(968799txo)
  ```

ACKs for top commit:
  practicalswift:
    ACK c858302280 -- patch looks great :)
  achow101:
    ACK c858302280
  laanwj:
    Tested ACK c858302280

Tree-SHA512: 16cbe419c4993ad51019c676e8ca409ef1025b803cc598437c780dd7ca003d7e4ad421f451e9a374e0070ee9b3ee601b7aba849e1f346798f9321d1bce5c4401
2020-07-09 16:03:27 +02:00
MarcoFalke
5ec19df687
Merge #19277: util: Add Assert identity function
fab80fef61 refactor: Remove unused EnsureChainman (MarcoFalke)
fa34587f1c scripted-diff: Replace EnsureChainman with Assert in unit tests (MarcoFalke)
fa6ef701ad util: Add Assert identity function (MarcoFalke)
fa457fbd33 move-only: Move NDEBUG compile time check to util/check (MarcoFalke)

Pull request description:

  The utility function is primarily useful to dereference pointer types, which are known to be not null at that time.

  For example, the ArgsManager is known to exist when the wallets are started: https://github.com/bitcoin/bitcoin/pull/18923/files#diff-fdb2a1a1d8bc790fcddeb6cf5a42ac55R503 . Instead of silently relying on that assumption, `Assert` can be used to abort the program and avoid UB should the assumption ever be violated.

ACKs for top commit:
  promag:
    Tested ACK fab80fef61.
  ryanofsky:
    Code review ACK fab80fef61

Tree-SHA512: 830fba10152ba17d47c4dd42809c7e26f9fe6d38e17a2d5b3f054fd644a5c4c9841286ac421ec9bb28cea9f5faeb659740fcf00de6cc589d423fee7694c42d16
2020-07-04 08:44:45 -04:00
MarcoFalke
915ac8a861
Merge #19413: refactor: Remove confusing BlockIndex global
fa0dfdf447 refactor: Remove confusing BlockIndex global (MarcoFalke)

Pull request description:

  The global `::BlockIndex()` is problematic for several reasons:

  * It returns a mutable reference to the block tree, without the appropriate lock annotation (`m_block_index` is guarded by `cs_main`). The current code is fine, but in the future this might lead to accidental races and data corruption.
  * The rpc server shouldn't rely on node globals, but rather a context that is passed in to the RPC method.
  * Tests might want to spin up their own block tree, and thus should also not rely on a single global.

  Fix all issues by removing the global

ACKs for top commit:
  promag:
    Code review ACK fa0dfdf447.
  jonatack:
    re-ACK fa0dfdf

Tree-SHA512: 8f158fc5e1c67e73588a21c25677b3fa0fe442313b13ec24b87054806c59607d6ba0c062a865ce3e0ee568706bd0d1faa84febda21aff5bcd65dab172f74c52f
2020-07-03 07:38:16 -04:00
James O'Beirne
8ac3ef4699 add ChainstateManager::MaybeRebalanceCaches()
Aside from in unittests, this method is unused at the moment. It will be used
in upcoming commits that enable utxo snapshot activation.
2020-07-01 14:44:28 -04:00
James O'Beirne
f36aaa6392 Add CChainState::ResizeCoinsCaches
Also adds CCoinsViewCache::ReallocateCache() to attempt to free
memory that the cacheCoins's allocator may be hanging onto when
downsizing the cache.

Adds `CChainState::m_coins{tip,db}_cache_size_bytes` data members
so that we can reference cache size on a per-chainstate basis for
flushing.
2020-07-01 14:44:28 -04:00
Wladimir J. van der Laan
bb588669f9
Merge #19331: build: Do not include server symbols in wallet
faca73000f ci: Install fixed version of clang-format for linters (MarcoFalke)
fa4695da4c build: Sort Makefile.am after renaming file (MarcoFalke)
cccc2784a3 scripted-diff: Move ui_interface to the node lib (MarcoFalke)
fa72ca6a9d qt: Remove unused includes (MarcoFalke)
fac96e6450 wallet: Do not include server symbols (MarcoFalke)
fa0f6c58c1 Revert "Fix link error with --enable-debug" (MarcoFalke)

Pull request description:

  This reverts a hacky workaround from commit b83cc0f, which only happens to work due to compiler optimizations. Then, it actually fixes the linker error.

  The underlying problem is that the wallet includes symbols from the server (ui_interface), which usually results in linker failures. Though, in this specific case the linker failures have not been observed (unless `-O0`) because our compilers were smart enough to strip unused symbols.

  Fix the underlying problem by creating a new header-only with the needed symbol and move ui_interface to node to clarify that this is part of libbitcoin_server.

ACKs for top commit:
  Sjors:
    ACK faca730
  laanwj:
    ACK faca73000f
  hebasto:
    re-ACK faca73000f, since the [previous](https://github.com/bitcoin/bitcoin/pull/19331#pullrequestreview-434420539) review:

Tree-SHA512: e9731f249425aaea50b6db5fc7622e10078cf006721bb87989cac190a2ff224412f6f8a7dd83efd018835302337611f5839e29e15bef366047ed591cef58dfb4
2020-07-01 15:38:18 +02:00
MarcoFalke
fa0dfdf447
refactor: Remove confusing BlockIndex global 2020-06-29 20:28:47 -04:00
MarcoFalke
cccc2784a3
scripted-diff: Move ui_interface to the node lib
-BEGIN VERIFY SCRIPT-

 # Move files
 git mv src/ui_interface.h                                          src/node/ui_interface.h
 git mv src/ui_interface.cpp                                        src/node/ui_interface.cpp
 sed -i -e 's/BITCOIN_UI_INTERFACE_H/BITCOIN_NODE_UI_INTERFACE_H/g' src/node/ui_interface.h

 # Adjust includes and makefile
 sed -i -e 's|ui_interface|node/ui_interface|g' $(git grep -l ui_interface)

 # Sort includes
 git diff -U0 | clang-format-diff -p1 -i -v

-END VERIFY SCRIPT-
2020-06-27 11:49:28 -04:00
jmorgan
c858302280 Change format of log2_work for uniform output (zero-padded) 2020-06-21 17:23:26 -04:00
MarcoFalke
faba65e696
Add ChainstateManager::ActiveChainstate 2020-06-19 09:27:00 -04:00
MarcoFalke
fa02b47313
refactor: Use AbortError in FatalError
This is needed for consistency with AbortNode
2020-06-16 10:51:50 -04:00
MarcoFalke
4b30c41b4e
Merge #18927: Pass bilingual_str argument to AbortNode()
5527be0627 refactor: Add AbortError alias (Hennadii Stepanov)
d924f2a596 Drop MSG_NOPREFIX flag (Hennadii Stepanov)
083daf7fba Pass bilingual_str argument to AbortNode() (Hennadii Stepanov)
d1cca129b4 refactor: Use bilingual_str::empty() (Hennadii Stepanov)

Pull request description:

  This PR is a [followup](https://github.com/bitcoin/bitcoin/issues/16218#issuecomment-625919724) of #16224, and it adds `bilingual_str` type argument support to the `AbortNode()` functions.

ACKs for top commit:
  MarcoFalke:
    ACK 5527be0627 👟

Tree-SHA512: bf8b15b14912b1f672e6e588fffa1e6eb6f00b4b23d15d0ced7f18fbdf76919244427feb7217007fe29617049308e13def893a03a87358db819cca9692f59905
2020-06-16 08:53:02 -04:00