86b47fa741 speed up Unserialize_impl for prevector (Akio Nakamura)
Pull request description:
The unserializer for prevector uses `resize()` for reserve the area, but it's prefer to use `reserve()` because `resize()` have overhead to call its constructor many times.
However, `reserve()` does not change the value of `_size` (a private member of prevector).
This PR make the logic of read from stream to callback function, and prevector handles initilizing new values with that call-back and ajust the value of `_size`.
The changes are as follows:
1. prevector.h
Add a public member function named 'append'.
This function has 2 params, number of elemenst to append and call-back function that initilizing new appended values.
2. serialize.h
In the following two function:
- `Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&)`
- `Unserialize_impl(Stream& is, prevector<N, T>& v, const V&)`
Make a callback function from each original logic of reading values from stream, and call prevector's `append()`.
3. test/prevector_tests.cpp
Add a test for `append()`.
## A benchmark result is following:
[Machine]
MacBook Pro (macOS 10.13.3/i7 2.2GHz/mem 16GB/SSD)
[result]
DeserializeAndCheckBlockTest => 22% faster
DeserializeBlockTest => 29% faster
[before PR]
# Benchmark, evals, iterations, total, min, max, median
DeserializeAndCheckBlockTest, 60, 160, 94.4901, 0.0094644, 0.0104715, 0.0098339
DeserializeBlockTest, 60, 130, 65.0964, 0.00800362, 0.00895134, 0.00824187
[After PR]
# Benchmark, evals, iterations, total, min, max, median
DeserializeAndCheckBlockTest, 60, 160, 77.1597, 0.00767013, 0.00858959, 0.00805757
DeserializeBlockTest, 60, 130, 49.9443, 0.00613926, 0.00691187, 0.00635527
ACKs for top commit:
laanwj:
utACK 86b47fa741
Tree-SHA512: 62ea121ccd45a306fefc67485a1b03a853435af762607dae2426a87b15a3033d802c8556e1923727ddd1023a1837d0e5f6720c2c77b38196907e750e15fbb902
After this commit, the only remaining output is:
$ test/lint/lint-spelling.sh
src/test/base32_tests.cpp:14: fo ==> of, for
src/test/base64_tests.cpp:14: fo ==> of, for
^ Warning: codespell identified likely spelling errors. Any false positives? Add them to the list of ignored words in test/lint/lint-spelling.ignore-words.txt
Note:
* I ignore several valid alternative spellings
* homogenous is present in tinyformat, hence should be addressed upstream
* process' is correct only if there are plural processes
67f4e9c522 Include core_io.h from core_read.cpp (practicalswift)
eca9767673 Make reasoning about dependencies easier by not including unused dependencies (practicalswift)
Pull request description:
Make reasoning about dependencies easier by not including unused dependencies.
Please note that the removed headers are _not_ "transitively included" by other still included headers. Thus the removals are real.
As an added bonus this change means less work for the preprocessor/compiler. At least 51 393 lines of code no longer needs to be processed:
```
$ git diff -u HEAD~1 | grep -E '^\-#include ' | cut -f2 -d"<" | cut -f1 -d">" | \
sed 's%^%src/%g' | xargs cat | wc -l
51393
```
Note that 51 393 is the lower bound: the real number is likely much higher when taking into account transitively included headers :-)
ACKs for commit 67f4e9:
Tree-SHA512: 0c8868aac59813f099ce53d5307eed7962dd6f2ff3546768ef9e5c4508b87f8210f1a22c7e826c3c06bebbf28bdbfcf1628ed354c2d0fdb9a31a42cefb8fdf13
8be3f3063 netaddress: Update CNetAddr for ORCHIDv2 (Carl Dong)
Pull request description:
```
The original ORCHID prefix was deprecated as of 2014-03, the new
ORCHIDv2 prefix was allocated by RFC7343 as of 2014-07. We did not
consider the original ORCHID prefix routable, and I don't see any reason
to consider the new one to be either.
```
Would like to know if people think this kind of thing is even worth keeping the codebase updated for. Perhaps it'd be nice to write a devtool to pull the csv from [here](https://www.iana.org/assignments/iana-ipv6-special-registry/iana-ipv6-special-registry.xhtml) and generate the code.
ACKs for commit 8be3f3:
laanwj:
utACK 8be3f3063
ryanofsky:
utACK 8be3f30633. Only change since last review is rebasing after #15718 merge.
Tree-SHA512: 7c93317f597b1a6c1443e12dd690010392edb9d72a479a8201970db7d3444fbb99a80b98026caad6fbfbebb455ab4035d2dde79bc9263bfd1d0398cd218392e1
58e291cfa Add test for GCC bug 90348 (Pieter Wuille)
Pull request description:
This adds a test for GCC bug 90348 (https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90348), using a test case extracted from our own `sha256d64` test in crypto_tests.cpp, which was failing on some platforms.
This is based on top of #15983 to make sure the bug doesn't trigger (it does in some Travis configurations without it).
ACKs for commit 58e291:
Tree-SHA512: 4dc9084e92dd143a53930e42bb68e33d922a2a2b891406b259d3a0bed4511dcc49e7447a7a8e4eb793a26e3eacb188ca293b71e0e061f9b3230f8e7fcfd29525
This commit adds comments referencing multiple CVEs both in production and test code.
CVEs covered in this commit:
CVE-2010-5137
CVE-2010-5139
CVE-2010-5141
CVE-2012-1909
CVE-2012-2459
CVE-2012-3789
CVE-2018-17144
fa013664ae util: Add type safe GetTime (MarcoFalke)
Pull request description:
There are basically two ways to get the time in Bitcoin Core:
* get the system time (via `GetSystemTimeInSeconds` or `GetTime{Millis,Micros}`)
* get the mockable time (via `GetTime`)
Both return the same type (a plain int). This can lead to (test-only) bugs such as 99464bc38e.
Fix that by deprecating `GetTime` and adding a `GetTime<>` that returns the mockable time in a non-int type. The new util function is currently unused, but new code should it where possible.
ACKs for commit fa0136:
promag:
utACK fa013664.
Tree-SHA512: efab9c463f079fd8fd3030c479637c7b1e8be567a881234bd0f555c8f87e518e3b43ef2466128103db8fc40295aaf24e87ad76d91f338c631246fc703477e95c
2c448d6bc7 parameterize hard coded numbers referring to miner conf window (Jordan Baczuk)
Pull request description:
Replace hard coded values (eg. 2016) with `mainnetParams.nMinerConfirmationWindow` where appropriate. This parameterizes hard coded values in the unit test that refer to the `Miner Confirmation Window`, which currently is `2016`. This includes values not exactly 2016 but which were derived from it. Also changed `int` to `uint32_t` where appropriate to avoid compiler warnings. This makes one source of truth, and also helps people who might be adjusting this value in testing so the unit tests don't break.
ACKs for commit 2c448d:
Tree-SHA512: 9262e0b89c1baf7857b49fe2221b2b00f948f61317b321c4871a9182a86d6f8aadeb59d6b133e8a213cc9b31b4a417888fb1ad31caef16ccbbab1de33c4b8459
8794a4b3ae QA: add test for HKDF HMAC_SHA256 L32 (Jonas Schnelli)
551d489416 Add HKDF HMAC_SHA256 L=32 implementations (Jonas Schnelli)
3b64f852e4 QA: add test for CKey::Negate() (Jonas Schnelli)
463921bb64 CKey: add method to negate the key (Jonas Schnelli)
Pull request description:
This adds a limited implementation of `HKDF` (defined by rfc5869) that supports only HMAC-SHA256 and length output of 32 bytes (will be required for v2 transport protocol).
This PR also includes a method to negate a private key which is useful to enforce public keys starting with 0x02 (or 0x03) (a requirement for the v2 transport protocol). The new `CKey::Negate()` method is pretty much a wrapper around `secp256k1_ec_privkey_negate()`.
Including tests.
This is a subset of #14032 and a pre-requirement for the v2 transport protocol.
ACKs for commit 8794a4:
Tree-SHA512: 5341929dfa29f5da766ec3612784baec6a3ad69972f08b5a985a8aafdae4dae36f104a2b888d1f5d1f33561456bd111f960d7e32c2cc4fd18e48358468f26c1a
The original ORCHID prefix was deprecated as of 2014-03, the new
ORCHIDv2 prefix was allocated by RFC7343 as of 2014-07. We did not
consider the original ORCHID prefix routable, and I don't see any reason
to consider the new one to be either.
f6bb11fd37 Add test for ArgsManager::GetChainName (Russell Yanofsky)
4b331159df Add unit test NextString, ForEachNoDup functions (Russell Yanofsky)
05bfee3451 util_SettingsMerge test cleanup (Russell Yanofsky)
Pull request description:
There was some test coverage previously, but it was limited and didn't test conflicting and negated arguments.
ACKs for commit f6bb11:
MarcoFalke:
re-utACK f6bb11fd37
Tree-SHA512: d03596614dc48584c7a9440117b107c6abb23fd4c7fa15fb4015351ec3de08b2656bc956ce05310663675672343d7a6aff35421657f29172080c7005045680b0
2dfe27517 Add ChaCha20 bench (Jonas Schnelli)
2bc2b8b49 Add ChaCha20 encryption option (XOR) (Jonas Schnelli)
Pull request description:
The current ChaCha20 implementation does not support message encryption (it can only output the keystream which is sufficient for the RNG).
This PR adds the actual XORing of the `plaintext` with the `keystream` in order to return the desired `ciphertext`.
Required for v2 message transport protocol.
ACKs for commit 2dfe27:
jnewbery:
Looks good. utACK 2dfe275171.
jnewbery:
utACK 2dfe275171
sipa:
utACK 2dfe275171
ryanofsky:
utACK 2dfe275171. Changes since last review are just renaming the Crypt method, adding comments, and simplifying the benchmark.
Tree-SHA512: 84bb234da2ca9fdc44bc29a786d9dd215520f81245270c1aef801ef66b6091b7793e2eb38ad6dbb084925245065c5dce9e5582f2d0fa220ab3e182d43412d5b5
78e407ad0c GetKeyBirthTimes should return key ids, not destinations (Gregory Sanders)
70946e7fee Replace CScriptID and CKeyID in CTxDestination with dedicated types (Gregory Sanders)
Pull request description:
The current usage seems to be an overloading of meanings. `CScriptID` is used in the wallet as a lookup key, as well as a destination, and `CKeyID` likewise. Instead, have all destinations be dedicated types.
New types:
`CScriptID`->`ScriptHash`
`CKeyID`->`PKHash`
ACKs for commit 78e407:
ryanofsky:
utACK 78e407ad0c. Only changes are removing extra CScriptID()s and fixing the test case.
Sjors:
utACK 78e407a
meshcollider:
utACK 78e407ad0c
Tree-SHA512: 437f59fc3afb83a40540da3351507aef5aed44e3a7f15b01ddad6226854edeee762ff0b0ef336fe3654c4cd99a205cef175211de8b639abe1130c8a6313337b9
486c1eea86 refactoring: remove unused chainActive (James O'Beirne)
631940aab2 scripted-diff: replace chainActive -> ::ChainActive() (James O'Beirne)
a3a609079c refactoring: introduce unused ChainActive() (James O'Beirne)
1b6e6fcfd2 rename: CChainState.chainActive -> m_chain (James O'Beirne)
Pull request description:
This is part of the assumeutxo project:
Parent PR: #15606
Issue: #15605
Specification: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal
---
This change refactors the `chainActive` reference into a `::ChainActive()` call. It also distinguishes `CChainState`'s `CChain` data member as `m_chain` instead of the current `chainActive`, which makes it easily confused with the global data.
The active chain must be obtained via function because its reference will be swapped at some point during runtime after loading a UTXO snapshot.
This change, though lengthy, should be pretty easy to review since most of it is contained within a scripted-diff. Once merged, the parent PR should be easier to review.
ACKs for commit 486c1e:
Sjors:
utACK 486c1ee
promag:
utACK 486c1ee.
practicalswift:
utACK 486c1eea86
Tree-SHA512: 06ed8f9e77f2d25fc9bea0ba86436d80dbbce90a1e8be23e37ec4eeb26060483e60b4a5c4fba679cb1867f61e3921c24abeb9cabdfb4d0a9b1c4ddd77b17456a
Remove testcase generating code from util_SettingsMerge so it can be reused in
new tests.
The hash value expected in util_SettingsMerge changes as a result of this, but
only because the testcases are generated in a different order, not because any
cases are added or removed. It is possible to verify this with:
SETTINGS_MERGE_TEST_OUT=new.txt test/test_bitcoin --run_test=util_tests/util_SettingsMerge
git checkout HEAD~1
make test/test_bitcoin
SETTINGS_MERGE_TEST_OUT=old.txt test/test_bitcoin --run_test=util_tests/util_SettingsMerge
diff -u <(sort old.txt) <(sort new.txt)
The new output is a little more readable, with simpler testcases sorted first.
Followup to #15869. Treat "-wallet" as the network-specific argument in test
instead of "-server", to make test output clearer and be more consistent with
bitcoind. Update embedded hash to match changed output from this.
0ff1c2a838 Separate reason for premature spends (coinbase/locktime) (Suhas Daftuar)
54470e767b Assert validation reasons are contextually correct (Suhas Daftuar)
2120c31521 [refactor] Update some comments in validation.cpp as we arent doing DoS there (Matt Corallo)
12dbdd7a41 [refactor] Drop unused state.DoS(), state.GetDoS(), state.CorruptionPossible() (Matt Corallo)
aa502b88d1 scripted-diff: Remove DoS calls to CValidationState (Matt Corallo)
7721ad64f4 [refactor] Prep for scripted-diff by removing some \ns which annoy sed. (Matt Corallo)
5e78c5734b Allow use of state.Invalid() for all reasons (Matt Corallo)
6b34bc6b6f Fix handling of invalid headers (Suhas Daftuar)
ef54b486d5 [refactor] Use Reasons directly instead of DoS codes (Matt Corallo)
9ab2a0412e CorruptionPossible -> BLOCK_MUTATED (Matt Corallo)
6e55b292b0 CorruptionPossible -> TX_WITNESS_MUTATED (Matt Corallo)
7df16e70e6 LookupBlockIndex -> CACHED_INVALID (Matt Corallo)
c8b0d22698 [refactor] Drop redundant nDoS, corruptionPossible, SetCorruptionPossible (Matt Corallo)
34477ccd39 [refactor] Add useful-for-dos "reason" field to CValidationState (Matt Corallo)
6a7f8777a0 Ban all peers for all block script failures (Suhas Daftuar)
7b999103e2 Clean up banning levels (Matt Corallo)
b8b4c80146 [refactor] drop IsInvalid(nDoSOut) (Matt Corallo)
8818729013 [refactor] Refactor misbehavior ban decisions to MaybePunishNode() (Matt Corallo)
00e11e61c0 [refactor] rename stateDummy -> orphan_state (Matt Corallo)
f34fa719cf Drop obsolete sigops comment (Matt Corallo)
Pull request description:
This is a rebase of #11639 with some fixes for the last few comments which were not yet addressed.
The original PR text, with some strikethroughs of text that is no longer correct:
> This cleans up an old main-carryover - it made sense that main could decide what DoS scores to assign things because the DoS scores were handled in a different part of main, but now validation is telling net_processing what DoS scores to assign to different things, which is utter nonsense. Instead, we replace CValidationState's nDoS and CorruptionPossible with a general ValidationInvalidReason, which net_processing can handle as it sees fit. I keep the behavior changes here to a minimum, but in the future we can utilize these changes for other smarter behavior, such as disconnecting/preferring to rotate outbound peers based on them providing things which are invalid due to SOFT_FORK because we shouldn't ban for such cases.
>
> This is somewhat complementary with, though obviously conflicts heavily with #11523, which added enums in place of DoS scores, as well as a few other cleanups (which are still relevant).
>
> Compared with previous bans, the following changes are made:
>
> Txn with empty vin/vout or null prevouts move from 10 DoS
> points to 100.
> Loose transactions with a dependency loop now result in a ban
> instead of 10 DoS points.
> ~~BIP68-violation no longer results in a ban as it is SOFT_FORK.~~
> ~~Non-SegWit SigOp violation no longer results in a ban as it
> considers P2SH sigops and is thus SOFT_FORK.~~
> ~~Any script violation in a block no longer results in a ban as
> it may be the result of a SOFT_FORK. This should likely be
> fixed in the future by differentiating between them.~~
> Proof of work failure moves from 50 DoS points to a ban.
> Blocks with timestamps under MTP now result in a ban, blocks
> too far in the future continue to not result in a ban.
> Inclusion of non-final transactions in a block now results in a
> ban instead of 10 DoS points.
Note: The change to ban all peers for consensus violations is actually NOT the change I'd like to make -- I'd prefer to only ban outbound peers in those situations. The current behavior is a bit of a mess, however, and so in the interests of advancing this PR I tried to keep the changes to a minimum. I plan to revisit the behavior in a followup PR.
EDIT: One reviewer suggested I add some additional context for this PR:
> The goal of this work was to make net_processing aware of the actual reasons for validation failures, rather than just deal with opaque numbers instructing it to do something.
>
> In the future, I'd like to make it so that we use more context to decide how to punish a peer. One example is to differentiate inbound and outbound peer misbehaviors. Another potential example is if we'd treat RECENT_CONSENSUS_CHANGE failures differently (ie after the next consensus change is implemented), and perhaps again we'd want to treat some peers differently than others.
ACKs for commit 0ff1c2:
jnewbery:
utACK 0ff1c2a838
ryanofsky:
utACK 0ff1c2a838. Only change is dropping the first commit (f3883a321bf4ab289edcd9754b12cae3a648b175), and dropping the temporary `assert(level == GetDoS())` that was in 35ee77f2832eaffce30042e00785c310c5540cdc (now c8b0d22698)
Tree-SHA512: e915a411100876398af5463d0a885920e44d473467bb6af991ef2e8f2681db6c1209bb60f848bd154be72d460f039b5653df20a6840352c5f7ea5486d9f777a3
Though at the moment ChainActive() simply references `g_chainstate.m_chain`,
doing this change now clears the way for multiple chainstate usage and allows
us to script the diff.
-BEGIN VERIFY SCRIPT-
git grep -l "chainActive" | grep -E '(h|cpp)$' | xargs sed -i '/chainActive =/b; /extern CChain& chainActive/b; s/\(::\)\{0,1\}chainActive/::ChainActive()/g'
-END VERIFY SCRIPT-
This is a first step towards cleaning up our DoS interface - make
validation return *why* something is invalid, and let net_processing
figure out what that implies in terms of banning/disconnection/etc.
Behavior change: peers will now be banned for providing blocks
with premature coinbase spends.
Co-authored-by: Anthony Towns <aj@erisian.com.au>
Suhas Daftuar <sdaftuar@gmail.com>
151f3e9cf1 Add settings merge test to prevent regresssions (Russell Yanofsky)
Pull request description:
Test-only change. Motivation: I'm trying to clean up settings code and add support for read/write settings without changing existing behavior, but current tests are very scattershot and don't actually cover a lot of current behavior.
ACKs for commit 151f3e:
jonasschnelli:
utACK 151f3e9cf1.
MarcoFalke:
utACK 151f3e9cf1
Tree-SHA512: f9062f078da02855cdbdcae37d0cea5684e82adbe5c701a8eb042ee4a57d899f0ffb6a9db3bcf58b639dff22b2b2d8a75f9a7917402df58904036753d65a1e3e
This work is prerequisite to attaching thread names to log lines and deadlock
debug utilities. This code allows setting of an "internal" threadname per
thread on platforms where thread_local is available.
This commit also moves RenameThread() out of a more general module and adds a
numeric suffix to disambiguate between threads with the same name. It
explicitly names a few main threads using the new util::ThreadRename().
765c0b364d refactor: combine Chain::findFirstBlockWithTime/findFirstBlockWithTimeAndHeight (Antoine Riard)
Pull request description:
As suggested in #14711, pass height to CChain::FindEarliestAtLeast to
simplify Chain interface by combining findFirstBlockWithTime and
findFirstBlockWithTimeAndHeight into one
ACKs for commit 765c0b:
jnewbery:
utACK 765c0b364d. Nice work @ariard!
ryanofsky:
utACK 765c0b364d. Looks good, thanks for implementing the suggestion!
Tree-SHA512: 63f98252a93da95f08c0b6325ea98f717aa9ae4036d17eaa6edbec68e5ddd65672d66a6af267b80c36311fffa9b415a47308e95ea7718b300b685e23d4e9e6ec
c7efb652f3 blockfilter: Update BIP 158 test vectors. (Jim Posen)
19308c9e21 rpc: Add getblockfilter RPC method. (Jim Posen)
ff35105096 init: Add CLI option to enable block filter index. (Jim Posen)
accc8b8b18 index: Access functions for global block filter indexes. (Jim Posen)
2bc90e4e7b test: Unit test for block filter index reorg handling. (Jim Posen)
6bcf0998c0 test: Unit tests for block index filter. (Jim Posen)
b5e8200db7 index: Implement lookup methods on block filter index. (Jim Posen)
75a76e3619 index: Implement block filter index with write operations. (Jim Posen)
2ad2338ef9 serialize: Serialization support for big-endian 32-bit ints. (Jim Posen)
ba6ff9a6f7 blockfilter: Functions to translate filter types to/from names. (Jim Posen)
62b7a4f094 index: Ensure block locator is not stale after chain reorg. (Jim Posen)
4368384f1d index: Allow atomic commits of index state to be extended. (Jim Posen)
Pull request description:
This introduces a new BlockFilterIndex class, which is required for BIP 157 support.
The index is uses the asynchronous BaseIndex infrastructure driven by the ValidationInterface callbacks. Filters are stored sequentially in flat files and the disk location of each filter is indexed in LevelDB along with the filter hash and header. The index is designed to ensure persistence of filters reorganized out of the main chain to simplify the BIP 157 net implementation.
Stats (block height = 565500):
- Syncing the index from scratch takes 45m
- Total index size is 3.8 GiB
ACKs for commit c7efb6:
MarcoFalke:
utACK c7efb652f3
ryanofsky:
Slightly tested ACK c7efb652f3 (I just rebuilt the index with the updated PR and tested the RPC). Changes since last review: rebase, fixed compile errors in internal commits, new comments, updated error messages, tweaked cache size logic, renamed commit method, renamed constants and globals, fixed whitespace, extra BlockFilterIndex::Init error check.
Tree-SHA512: f8ed7a9b6f76df45933aa5eba92b27b3af83f6df2ccb3728a5c89eec80f654344dc14f055f6f63eb9b3a7649dd8af6553fe14969889e7e2fd2f8461574d18f28
fad7c33342 refactor: Add handleNotifications method to wallet (MarcoFalke)
fa46ac3127 bench: Add wallet_balance benchmarks (MarcoFalke)
Pull request description:
ACKs for commit fad7c3:
ryanofsky:
utACK fad7c33342. I might squash or rearrange the commits to avoid adding code in one commit that just gets deleted in the next one. But overall this looks good and the cleanup is nice.
Tree-SHA512: 231faac168cbe9bb0ab4bf10ac1d5b042c610364406d75061fba27f1e9d16c71867e74cc4606e9f42659aa980d7133c00e29fcc18bbba7da2fa7a80178b3246c
Adds the following util units and adds them to libbitcoin_util:
- `util/url.cpp` takes `urlDecode` from `httpserver.cpp`
- `util/error.cpp` takes `TransactionErrorString` from
`node/transaction.cpp` and `AmountHighWarn` and `AmountErrMsg` from
`ui_interface.cpp`
- `util/fees.cpp` takes `StringForFeeReason` and `FeeModeFromString` from `policy/fees.cpp`
- `util/rbf.cpp` takes `SignalsOptInRBF` from `policy/rbf.cpp`
- 'util/validation.cpp` takes `FormatStateMessage` and `strMessageMagic` from 'validation.cpp`
Moves the following utility methods to rpc/util and moves that unit to
libbitcoin_common so they can be accessed by all libraries.
- `RPCTypeCheck`
- `RPCTypeCheckArgument`
- `RPCTypeCheckObj`
- `AmountFromValue`
- `ParseHashV``ParseHashO`
- `ParseHexV`
- `ParseHexO`
- `HelpExampleCli`
- `HelpExampleRpc`
This moves the following policy settings functions and globals to a new
src/policy/settings unit in lib_server:
- `incrementalRelayFee`
- `dustRelayFee`
- `nBytesPerSigOp`
- `fIsBareMultisigStd`
These settings are only required by the node and should not be accessed
by other libraries.
CheckTransaction is a context-free function that does not require access
to the blockchain or mempool. Move it from src/consensus/tx_verify in
lib_server to a new unit src/consensus/tx_check in lib_consensus so that
it can be called by non-server libraries.
f6ee177f7 Remove unused AES-128 code (practicalswift)
Pull request description:
Remove unused AES-128 code.
As far as I can tell this AES-128 code has never been in use in the project (outside of testing/benchmarking).
The AES-256 code is used in `CCrypter::Encrypt`/`CCrypter::Decrypt` (`src/wallet/crypter.cpp`).
Trivia: 0.15% of the project's C++ LOC count (excluding dependencies) is trimmed off:
```
$ LOC_BEFORE=$(git grep -I "" HEAD~1 -- "*.cpp" "*.h" ":(exclude)src/leveldb/" ":(exclude)src/secp256k1/" ":(exclude)src/univalue/" | wc -l)
$ LOC_AFTER=$(git grep -I "" -- "*.cpp" "*.h" ":(exclude)src/leveldb/" ":(exclude)src/secp256k1/" ":(exclude)src/univalue/" | wc -l)
$ bc <<< "scale=4; ${LOC_AFTER}/${LOC_BEFORE}"
.9985
```
:-)
Tree-SHA512: 9588a3cd795a89ef658b8ee7323865f57723cb4ed9560c21de793f82d35e2835059e7d6d0705e99e3d16bf6b2a444b4bf19568d50174ff3776caf8a3168f5c85
As suggested in #14711, pass height to CChain::FindEarliestAtLeast to
simplify Chain interface by combining findFirstBlockWithTime and
findFirstBlockWithTimeAndHeight into one
Extend findearliestatleast_edge_test in consequence
e9d5e97561 Poly1305: tolerate the intentional unsigned wraparound in poly1305.cpp (Jonas Schnelli)
b34bf302f2 Add Poly1305 bench (Jonas Schnelli)
03be7f48fa Add Poly1305 implementation (Jonas Schnelli)
Pull request description:
This adds a currently unused Poly1305 implementation including test vectors from RFC7539.
Required for BIP151 (and related to #15512).
Tree-SHA512: f8c1ad2f686b980a7498ca50c517e2348ac7b1fe550565156f6c2b20faf764978e4fa6b5b1c3777a16e7a12e2eca3fb57a59be9c788b00d4358ee80f2959edb1
d358466de Remove remaining wallet accesses to node globals (Russell Yanofsky)
b1b2b2389 Remove use of CCoinsViewMemPool::GetCoin in wallet code (Russell Yanofsky)
4e4d9e9f8 Remove use of CRPCTable::appendCommand in wallet code (Russell Yanofsky)
91868e628 Remove use CValidationInterface in wallet code (Russell Yanofsky)
Pull request description:
This PR is the last in a chain of PRs (#14437, #14711, and #15288) that make the wallet code access node state through an abstract [`Chain`](https://github.com/ryanofsky/bitcoin/blob/pr/wipc-sep/src/interfaces/chain.h) class in [`src/interfaces/`](https://github.com/ryanofsky/bitcoin/tree/pr/wipc-sep/src/interfaces) instead of using global variables like `cs_main`, `chainActive`, and `g_connman`. After this PR, wallet code no longer accesses global variables declared outside the wallet directory, and no longer calls functions accessing those globals (as verified by the `hide-globals` script in #10244).
This PR and the previous PRs have been refactoring changes that do not affect behavior. Previous PRs have consisted of lots of mechanical changes like:
```diff
- wtx.nTimeReceived = GetAdjustedTime();
+ wtx.nTimeReceived = m_chain->getAdjustedTime();
```
This PR is smaller, but less mechanical. It replaces last few bits of wallet code that access node state directly (through `CValidationInterface`, `CRPCTable`, and `CCoinsViewMemPool` interfaces) with code that uses the `Chain` interface.
These changes allow followup PR #10102 (multiprocess gui & wallet PR) to work without any significant updates to wallet code. Additionally they:
* Provide a single place to describe the interface between wallet and node code.
* Can make better wallet testing possible, because the `Chain` object consists of virtual methods that can be overloaded for mocking. (This could be used to test edge cases in the rescan code, for example).
Tree-SHA512: e6291d8a3c50bdff18a9c8ad11e729beb30b5b7040d7aaf31ba678800b4a97b2dd2be76340b1e5c01fe2827d67d37ed1bb4c8380cf8ed653aadfea003e9b22e7
Problem:
- Nothing uses the `fspaces` argument to `HexStr()` besides unit
tests. This argument results in extra complexity and a small
performance decrease within the function for branch evalulation.
Solution:
- Remove unused `fspaces` option.
82c3b3f8e0 Remove sharp edge (uninitialized m_filter_type) when using the compiler-generated constructor for BlockFilter (practicalswift)
Pull request description:
Remove sharp edge (uninitialised member `m_filter_type`) when using the compiler-generated constructor for `BlockFilter`.
Before (but after added test):
```
$ src/test/test_bitcoin -t blockfilter_tests/blockfilter_basic_test
Running 1 test case...
test/blockfilter_tests.cpp(118): error: in "blockfilter_tests/blockfilter_basic_test": check default_ctor_block_filter_1.GetFilterType() == default_ctor_block_filter_2.GetFilterType() has failed [ != ]
*** 1 failure is detected in the test module "Bitcoin Test Suite"
```
After:
```
$ src/test/test_bitcoin -t blockfilter_tests/blockfilter_basic_test
Running 1 test case...
*** No errors detected
```
Tree-SHA512: 21d41f036b0bf12adcf1a788d84747353f2023cb85fd8ea6c97222967032e8bf54e7910cadb45dfcecd78e5b5dca86685f78cad0596b6d1a08f910ebf20d90aa
faa9b88199 fuzz: Link BasicTestingSetup (shared with unit tests) (MarcoFalke)
fa85468cd2 test: Move main_tests to validation_tests (MarcoFalke)
fa02b22245 test: Remove useless test_bitcoin_main.cpp (MarcoFalke)
fab2daa026 test: Add missing LIBBITCOIN_ZMQ to test_test_bitcoin_LDADD (MarcoFalke)
Pull request description:
Link against BasicTestingSetup in the fuzz tests, so we can fuzz against validation.
Also include a commit to remove test_bitcoin_main.cpp. That file may or may not overwrite globals in the link stage depending on the link order. This is confusing and useless anyway: The unit tests should never `std::exit` in the middle of the run (especially with success as exit code), since it will skip all test modules afterward.
Also include a commit to remove some unused forward declarations and move the main_tests to validation_tests, since main was long ago split into net_processing and validation.
Tree-SHA512: bdd34c87505450ec106d632f6664aadcbdac7c198172a77da55fab75b274f869ae1a8d06573ba2aff4cb186be9c7a34b7697894ab6f9c82b392f769c9135f36c
04cca33094 Style cleanup. (Jim Posen)
4c01e4e159 flatfile: Unit tests for FlatFileSeq methods. (Jim Posen)
65a489e93d scripted-diff: Rename CBlockDiskPos to FlatFilePos. (Jim Posen)
d6d8a78f26 Move CDiskBlockPos from chain to flatfile. (Jim Posen)
e0380933e3 validation: Refactor file flush logic into FlatFileSeq. (Jim Posen)
992404b31e validation: Refactor block file pre-allocation into FlatFileSeq. (Jim Posen)
e2d2abb99f validation: Refactor OpenDiskFile into method on FlatFileSeq. (Jim Posen)
9183d6ef65 validation: Extract basic block file logic into FlatFileSeq class. (Jim Posen)
62e7addb63 util: Move CheckDiskSpace to util. (Jim Posen)
Pull request description:
This cleans up and refactors block file helpers so that they may be used by the block filter indexer. Per [design discussion](https://github.com/bitcoin/bitcoin/pull/14121#issuecomment-451252591) about storing BIP 157 block filters, it has been suggested that they are stored in the same way as block and undo data. This refactor is sufficient to simplify file operations for this use case, though in the future perhaps more pruning-related logic ought to be moved into the new classes.
The basic abstraction is a `FlatFileSeq` which manages access to a sequence of numbered files into which raw data is written.
Tree-SHA512: b2108756777f2dad8964a1a2ef2764486e708a4a4a8cfac47b5de8bcb0625388964438eb096b10cfd9ea39212c299b5cb32fa943e768db2333cf49ea7def157e
1. Fix lack of warning by collecting all section names by moving
m_config_sections.clear() to ArgsManager::ReadConfigFiles().
2. Add info(file name, line number) to warning message.
3. Add a test code to confirm this situation.
3. Do clear() in ReadConfigString().
fd637be8d2 Add checksums to descriptors.md (Pieter Wuille)
be62903c41 Make descriptor checksums mandatory in deriveaddresses and importmulti (Pieter Wuille)
b52cb63688 Add getdescriptorinfo to compute checksum (Pieter Wuille)
3b40bff988 Descriptor checksum (Pieter Wuille)
Pull request description:
This adds support for a descriptor-specific 8-character checksum.
Descriptors may optionally be suffixed with a `#` plus these 8 checksum characters. Any descriptor that contains a `#` at the end must be followed by a valid checksum. If the `#` is missing entirely, it is valid without checksum.
All RPCs are updated to report descriptors that include the checksum. On input, they are optional except in `deriveaddress` and `importmulti`, which require descriptors which include a checksum.
A new RPC is also added to analyse descriptors (`getdescriptorinfo`), which can be used to compute the checksum for a descriptor without.
Tree-SHA512: a8294b09155eb6c67fbc178b5e2d3fbc0e9bec8b6de57a13f8835550d51c2cb32a428b3c9a188ded42b454d594e9305edbd4797906b755de77a8f33c79165f6b
ef0b01217a tests: Make updatecoins_simulation_test deterministic (practicalswift)
Pull request description:
Make test `updatecoins_simulation_test` deterministic.
Can be verified using `contrib/test_deterministic_coverage.sh` introduced in #15296.
Related:
* #15296: "tests: Add script checking for deterministic line coverage in unit tests"
* #15324: "test: Make bloom tests deterministic"
* #14343: "coverage reports non-deterministic"
Tree-SHA512: 3466e28a42dd3735effb8542044d88e8350a470729d4a4f02abce9d6367de6568d698131469ba154d3dc76d448bacb360b7aefd066bb5b91408c0be375dd3ecb
The unserializer for prevector uses resize() for reserve the area,
but it's prefer to use reserve() because resize() have overhead
to call its constructor many times.
However, reserve() does not change the value of "_size"
(a private member of prevector).
This PR introduce resize_uninitialized() to prevector that similar to
resize() but does not call constructor, and added elements are
explicitly initialized in Unserialize_imple().
The changes are as follows:
1. prevector.h
Add a public member function named 'resize_uninitialized'.
This function processes like as resize() but does not call constructors.
So added elemensts needs explicitly initialized after this returns.
2. serialize.h
In the following two function:
Unserialize_impl(Stream& is, prevector<N, T>& v, const unsigned char&)
Unserialize_impl(Stream& is, prevector<N, T>& v, const V&)
Calls resize_uninitialized() instead of resize()
3. test/prevector_tests.cpp
Add a test for resize_uninitialized().
223de8d94d Document RNG design in random.h (Pieter Wuille)
f2e60ca985 Use secure allocator for RNG state (Pieter Wuille)
cddb31bb0a Encapsulate RNGState better (Pieter Wuille)
152146e782 DRY: Implement GetRand using FastRandomContext::randrange (Pieter Wuille)
a1f252eda8 Sprinkle some sweet noexcepts over the RNG code (Pieter Wuille)
4ea8e50837 Remove hwrand_initialized. (Pieter Wuille)
9d7032e4f0 Switch all RNG code to the built-in PRNG. (Pieter Wuille)
16e40a8b56 Integrate util/system's CInit into RNGState (Pieter Wuille)
2ccc3d3aa3 Abstract out seeding/extracting entropy into RNGState::MixExtract (Pieter Wuille)
aae8b9bf0f Add thread safety annotations to RNG state (Pieter Wuille)
d3f54d1c82 Rename some hardware RNG related functions (Pieter Wuille)
05fde14e3a Automatically initialize RNG on first use. (Pieter Wuille)
2d1cc50939 Don't log RandAddSeedPerfmon details (Pieter Wuille)
6a57ca91da Use FRC::randbytes instead of reading >32 bytes from RNG (Pieter Wuille)
Pull request description:
This does not remove OpenSSL, but makes our own PRNG the 'main' one; for GetStrongRandBytes, the OpenSSL RNG is still used (indirectly, by feeding its output into our PRNG state).
It includes a few policy changes (regarding what entropy is seeded when).
Before this PR:
* GetRand*:
* OpenSSL
* GetStrongRand*:
* CPU cycle counter
* Perfmon data (on Windows, once 10 min)
* /dev/urandom (or equivalent)
* rdrand (if available)
* From scheduler when idle:
* CPU cycle counter before and after 1ms sleep
* At startup:
* CPU cycle counter before and after 1ms sleep
After this PR:
* GetRand*:
* Stack pointer (which indirectly identifies thread and some call stack information)
* rdrand (if available)
* CPU cycle counter
* GetStrongRand*:
* Stack pointer (which indirectly identifies thread and some call stack information)
* rdrand (if available)
* CPU cycle counter
* /dev/urandom (or equivalent)
* OpenSSL
* CPU cycle counter again
* From scheduler when idle:
* Stack pointer (which indirectly identifies thread and some call stack information)
* rdrand (if available)
* CPU cycle counter before and after 1ms sleep
* Perfmon data (on Windows, once every 10 min)
* At startup:
* Stack pointer (which indirectly identifies thread and some call stack information)
* rdrand (if available)
* CPU cycle counter
* /dev/urandom (or equivalent)
* OpenSSL
* CPU cycle counter again
* Perfmon data (on Windows, once every 10 min)
The interface of random.h is also simplified, and documentation is added.
This implements most of #14623.
Tree-SHA512: 0120e19bd4ce80a509b5c180a4f29497d299ce8242e25755880851344b825bc2d64a222bc245e659562fb5463fb7c70fbfcf003616be4dc59d0ed6534f93dd20
b9dafe7d9f Fix remaining compiler warnings (MSVC). Move disabling of specific warnings from /nowarn to project file. (practicalswift)
Pull request description:
Fix remaining compiler warnings (MSVC).
Before:
```
$ msbuild /p:TrackFileAccess=false /p:CLToolExe=clcache.exe build_msvc\bitcoin.sln /m /v:q /nowarn:C4244;C4267;C4715 /nologo
…\script\script.cpp(272): warning C4018: '>': signed/unsigned mismatch
…\test\allocator_tests.cpp(147): warning C4312: 'reinterpret_cast': conversion from 'int' to 'void *' of greater size
…\boost\test\tools\old\impl.hpp(107): warning C4805: '==': unsafe mix of type 'const Left' and type 'const Right' in operation
…\test\crypto_tests.cpp(535): warning C4334: '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?)
…\test\script_tests.cpp(188): warning C4805: '==': unsafe mix of type 'int' and type 'bool' in operation
…\test\script_tests.cpp(190): warning C4805: '==': unsafe mix of type 'int' and type 'bool' in operation
…\test\script_tests.cpp(191): warning C4805: '==': unsafe mix of type 'int' and type 'bool' in operation
$
```
After:
```
$ msbuild /p:TrackFileAccess=false /p:CLToolExe=clcache.exe build_msvc\bitcoin.sln /m /v:q /nowarn:C4244;C4267;C4715;C4805 /nologo
$
```
Tree-SHA512: 5b30334d3804e869779e77dad75a799e8e5e7eb2e08634cd40035cce140edd623cbb6c8b5806d2158c3df97888d3ea9ff4b8b6a5a83de3fe2cb361e29588c115
fa5e373365 validation: Add cs_main locking annotations (MarcoFalke)
fa5c346c5a doc: Add comment to cs_main and mempool::cs (MarcoFalke)
fafe941bdd test: Add missing validation locks (MarcoFalke)
fac4558462 sync: Add RecursiveMutex type alias (MarcoFalke)
Pull request description:
Both the chain state and the transaction pool are validation specific, but access to them is protected by two locks. The two locks have the following semantics:
* Writing to the chain state or adding transactions to the transaction pool -> Take both `cs_main` and `mempool::cs`
* Reading either or removing transactions from the the transaction pool -> Take only the appropriate lock
Tree-SHA512: 6f6e612ffc391904c6434a79a4f3f8de1b928bf0a3e3434b73561037b395e2b40a70a5a4bd8472dd230e9eacc8e5d5374c904a3c509910cf3971dd7ff59a626c
3a0e76fc12 Replace remaining 0 with nullptr in Qt code (Ben Woosley)
9096276e0b Don't use zero as null pointer constant (-Wzero-as-null-pointer-constant) (practicalswift)
Pull request description:
This corrects all violations of `-Wzero-as-null-pointer-constant` identified in the Qt codebase.
These changes are extracted from #15112 as suggested by @MarcoFalke to ease review. This is in service of enabling `-Wzero-as-null-pointer-constant`, which should eliminate this as a concern going forward.
Note there are 2 non-Qt changes: `src/test/allocator_tests.cpp` and `src/wallet/db.cpp`.
Tree-SHA512: 206bd668802147ba42bc413c2d7d259cb59aca9ec1da74a6bf2ca3932e60ae492faacbc61bcee0fd6b4b49a4d59d075b7e5404f0526b36c47718f9b0587e7768
d6b076c17b Drop IsLimited in favor of IsReachable (Ben Woosley)
Pull request description:
These two methods have had the same meaning, but inverted, since
110b62f069. Having one name for a single
concept simplifies the code.
This is a follow-up to #15051.
/cc #7553
Tree-SHA512: 347ceb9e2a55ea06f4c01226411c7bbcade09dd82130e4c59d0824ecefd960875938022edbe5d4bfdf12b0552c9b4cb78b09a688284d707119571daf4eb371b4
Unfortunately, `std::string` elements are (bare) chars. As these
are the most likely type to be passed to these functions, make them use
char instead of unsigned char. This avoids some casts.
8931a95bec Include util/strencodings.h which is required for IsSpace(...) (practicalswift)
7c9f790761 Update KNOWN_VIOLATIONS: Remove fixed violations (practicalswift)
587924f000 Use IsSpace(...) instead of boost::is_space (practicalswift)
c5fd143edb Use ToLower(...) instead of std::tolower (practicalswift)
e70cc8983c Use IsDigit(...) instead of std::isdigit (practicalswift)
Pull request description:
* Use `ToLower(...)` instead of `std::tolower`. `std::tolower` is locale dependent.
* Use `IsDigit(...)` instead of `std::isdigit`. Some implementations (e.g. Microsoft in 1252 codepage) may classify single-byte characters other than `[0-9]` as digits.
* Update `KNOWN_VIOLATIONS`: Remove fixed violations.
* ~~Replace use of locale dependent Boost trim (`boost::trim`) with locale independent `TrimString`.~~
* Use` IsSpace(...)` instead of `boost::is_space`
Tree-SHA512: defed016136b530b723fa185afdbd00410925a748856ba3afa4cee60f61a67617e30f304f2b9991a67b5fe075d9624f051e14342aee176f45fbc024d59e1aa82
6dc4593db1 IsReachable is the inverse of IsLimited (DRY). Includes unit tests (marcaiaf)
Pull request description:
IsReachable is the inverse of IsLimited, but the implementation is duplicated (DRY)
- Changed the implementation accordingly.
- Added unit tests to document behavior and relationship
- My modification in net.cpp applies only to IsReachable.
- Applied clang-format-diffpy
Created new pull request to avoid the mess with:
https://github.com/bitcoin/bitcoin/pull/15044
Checked with supposedly conflicting PRs mentioned in the old PR. No conflicts with the specific changes in this PR.
Tree-SHA512: b132dec6cc2c788ebe4f63f228d78f441614e156743b17adebc990de0180a5872874d2724c86eeaa470b4521918bd137b0e33ebcaae77c5efc1f0d56104f6c87
fa2510d5c1 Use C++11 default member initializers (MarcoFalke)
Pull request description:
Changes:
* Remove unused constructors that leave some members uninitialized
* Remove manual initialization in each constructor and prefer C++11 default member initializers
This is not a stylistic change, but a change that avoids bugs such as:
* fix uninitialized read when stringifying an addrLocal #14728
* qt: Initialize members in WalletModel #12426
* net: correctly initialize nMinPingUsecTime #6636
* ...
Tree-SHA512: 0f896f3b9fcc464d5fc7525f7c86343ef9ce9fb13425fbc68e9a9728fd8710c2b4e2fd039ee08279ea41ff20fd92b7185cf5cca95a0bcb6a5340a1e6f03cae6b
ed12d5df1b index: Fix for indexers skipping genesis block. (Jim Posen)
Pull request description:
This fixes a bug where indexers would skip processing of the genesis block. Preserves the current behavior of omitting genesis block transaction from the index.
Tree-SHA512: 092fd3d629bf1ef279566217c668cc913a8b8e012d811d0e544231894c49a0c0c179537ac4727c39b9bf407479541745d79c4e118db6f0795a2b848d0fe62cbf
4f4993fe2a Remove UBSan suppression (practicalswift)
958e1a307e streams: Remove unused seek(size_t) (practicalswift)
Pull request description:
Fix broken `streams_vector_reader` test. Remove unused `seek(size_t)`.
Before this change the test `streams_vector_reader` triggered an unintended unsigned integer wraparound. It tried so seek using a negative value in `reader.seek(-6)`.
Changes in this PR:
* Fix broken `VectorReader::seek(size_t)` test case
* Remove unused `seek(size_t)`
Tree-SHA512: 6c6affd680626363eef9e496748f2f86a522325abab9d6b13161f41125cdc29ceb36c2c1509c90b8ff108d606df7629e55e094cc2b6253b05a892b81ce176b71
6b25f29a91 Use std::vector API for construction of test data. (Daniel Kraft)
Pull request description:
For constructing test scripts, use `std::vector` and, in particular, `std::vector::insert` to insert 20 zero bytes rather than listing the full array of bytes explicitly. This makes the code easier to read and makes it immediately obvious what the structure of the data is, without having to count the zeros to understand it.
Of course, that is a matter of taste - so if you disagree that the change makes the code easier to read, let me know.
This has been split out of #14752.
Tree-SHA512: af82d447f0077259049f1da2d6f86a6c29723c6e17bd342e9a9ecf37b13bddff40643af95c8b3a3260765a5591713d31ca8a45a5a0c20a12c139aee53ea150da
b9f226b41f rpc: Remove cs_main lock from blockToJSON and blockHeaderToJSON (João Barbosa)
343b98cbcd rpc: Specify chain tip instead of chain in GetDifficulty (João Barbosa)
54dc13b6a2 rpc: Fix SoftForkMajorityDesc and SoftForkDesc signatures (João Barbosa)
Pull request description:
Motivated by https://github.com/bitcoin/bitcoin/pull/11913#discussion_r157798157, this pull makes `blockToJSON` and `blockheaderToJSON` free of `cs_main` locks.
Locking `cs_main` was required to access `chainActive` in order to check if the block was in the chain and to retrieve the next block index.
With the this approach, `CBlockIndex::GetAncestor()` is used in a way to check if the block belongs to the specified chain tip and, at the same time, get the next block index.
Tree-SHA512: a6720ace0182c19033bbed1a404f729d793574db8ab16e0966ffe412145611e32c30aaab02975d225df6d439d7b9ef2070e732b16137a902b0293c8cddfeb85f
75778a0724 test: Correct ineffectual WithOrVersion from transactions_tests (Ben Woosley)
Pull request description:
`WithOrVersion` uses `|` to combine the versions, and `|` with 0 is a no-op.
NicolasDorier / sipa do you recall why the version is being overridden here?
Introduced in ab48c5e721
Last updated 81e3228fcb
Tree-SHA512: 2aea925497bab2da973f17752410a6759d67181a57c3b12a685d184fbfcca2984c45b702ab0bd641d75e086696a0424f1bf77c5578ca765d6882dc03b42d5f9a
For constructing test scripts, use std::vector and, in particular,
std::vector::insert to insert 20 zero bytes rather than listing the full
array of bytes explicitly. This makes the code easier to read and makes
it immediately obvious what the structure of the data is, without having
to count the zeros to understand it.
WithOrVersion uses | to combine the versions, and | with 0 is a no-op.
Instead I run it with PROTOCOL_VERSION and 0 separately, as the original
code only tested PROTOCOL_VERSION but apparently only intended to test
version 0.
Introduced in ab48c5e721
Last updated 81e3228fcb
cb53b825c2 scripted-diff: Replace boost::bind with std::bind (Chun Kuan Lee)
2196c51821 refactor: Use boost::scoped_connection in signal/slot, also prefer range-based loop instead of std::transform (Chun Kuan Lee)
Pull request description:
Replace boost::bind with std::bind
- In `src/rpc/server.cpp`, replace `std::transform` with simple loop.
- In `src/validation.cpp`, store the `boost::signals2::connection` object and use it to disconnect.
- In `src/validationinterface.cpp`, use 2 map to store the `boost::signals2::scoped_connection` object.
Tree-SHA512: 6653cbe00036fecfc495340618efcba6d7be0227c752b37b81a27184433330f817e8de9257774e9b35828026cb55f11ee7f17d6c388aebe22c4a3df13b5092f0
e4ed8ce2c8 blockfilter: Remove default clause in switch statement. (Jim Posen)
c30620983d blockfilter: Additional constructors for BlockFilter. (Jim Posen)
20b812993a blockfilter: Refactor GCS params into struct. (Jim Posen)
Pull request description:
These commits have been split out of #14121 because they are fairly independent and that PR is very large.
Tree-SHA512: b9643b159e114df50a295f433e807afe6082db55a2a3a17401c1509b850c71bf5011ab3638863b46663709726be4445be6fde1dec514aec7696135497a9f0183
d2ce315fbf [docs] add release note for change to GBT (John Newbery)
0025c9eae4 [mining] segwit option must be set in GBT (John Newbery)
Pull request description:
Calling getblocktemplate without the segwit rule specified is most
likely a client error, since it results in lower fees for the miner.
Prevent this client error by failing getblocktemplate if called without
the segwit rule specified.
Of the previous 1000 blocks (measured at block [551591 (hash 0x...173c811)](https://blockstream.info/block/000000000000000000173c811e79858808abc3216af607035973f002bef60a7a)), 991 included segwit transactions.
Tree-SHA512: 7933b073d72683c9ab9318db46a085ec19a56a14937945c73f783ac7656887619a86b74db0bdfcb8121df44f63a1d6a6fb19e98505b2a26a6a8a6e768e442fee
c84c2b8c92 tests: Test for expected return values when calling functions returning a success code (practicalswift)
Pull request description:
Test for expected return values when calling functions returning a success code (instead of discarding the return values).
**Note to reviewers:** The following commands can be used to verify that the only text fragments added in this PR are `BOOST_CHECK(`, `!` and `)` :
```
$ git diff HEAD~1 | grep -E '^[\-][^\-]' | cut -b2- > before.txt
$ git diff HEAD~1 | grep -E '^[\+][^\+]' | cut -b2- > after.txt
$ cat after.txt | sed 's/BOOST_CHECK(//g' | sed 's/));/);/g' | tr -d '!' > after-sed.txt
$ diff -u before.txt after-sed.txt
$
```
Tree-SHA512: ff0863ef2046a2eda3c44e9c6b9aedfe167881f2fa58db29fef859416831233ef6502a3a11fd2322bc1a924db83df8d4a5c5879298007f2a7b085e2a7286af70
fa694f706c test: Add tests for truncated scripts (MarcoFalke)
Pull request description:
Previously not covered by any test
Tree-SHA512: 9f99659bdf3947271074938456a2fe64f5b39fc868e9aa474cec199a536ae5d7428f1cfa7f361936b71b09ee4c426261e6b25668fa77b8416b30dbe4ddb357f0
8db0c3d42b Removed implicit CTransaction conversion from benchmaks (lucash-dev)
ed61abedb2 Removed implicit CTransaction constructor from tests (lucash-dev)
Pull request description:
This PR was split from #14906 and is a prerequisite for it.
It updates tests and benchmarks, removing all implicit calls to `CTransaction(CMutableTransaction&)` constructors. This will make possible making the constructor explicit in the next PR.
The original rationale for making the constructor explicit:
- Conversion constructors should not be explicit unless there's a strong reason for it (in the opinion of, for example, https://google.github.io/styleguide/cppguide.html, and https://isocpp.github.io/CppCoreGuidelines/CppCoreGuidelines#Ro-conversion. Let me know your take on this).
- This particular conversion is very costly -- it implies a serialization plus hash of the transaction.
- Even though `CTransaction` and `CMutableTransaction` represent the same data, they have very different use cases and performance properties.
- Making it explicit allows for easier reasoning of performance trade-offs.
- There has been previous performance issues caused by unneeded use of this implicit conversion.
- This PR creates a map for places to look for possible refactoring and performance gains (this benefit still holds if the PR is not merged).
Tree-SHA512: de8073aa6ff8a3153bcbe10818616677ecf9598e4978d8a0b4c39a262e71c36be5679cec08554c760d1f011ba6d37350318248eef15f6d9b86f9e4462b2de0d2
bd3b0361d Add stop_block out arg to ScanForWalletTransactions (Ben Woosley)
3002d6cf3 Return a status enum from ScanForWalletTransactions (Ben Woosley)
bb24d6865 Make CWallet::ScanForWalletTransactions args and return value const (Ben Woosley)
Pull request description:
Return the failed block as an out arg.
Fixes#11450.
/cc #12275
Tree-SHA512: 6a523e5425ebfe24e664a942ae21c797ccc1281c25b1bf8d02ad95c19dae343fd8051985ef11853474de7628fd6bed5f15190fbc087c3466ce6fdecab37d72a9
26879509f Add comments to descriptor tests (Pieter Wuille)
82df4c64f Add descriptor expansion cache (Pieter Wuille)
1eda33aab [refactor] Combine the ToString and ToPrivateString implementations (Pieter Wuille)
24d3a7b3a [refactor] Use DescriptorImpl internally, permitting access to new methods (Pieter Wuille)
6be0fb4b3 [refactor] Add a base DescriptorImpl with most common logic (Pieter Wuille)
Pull request description:
This patch modifies the internal `Descriptor` class to optionally construct and use an "expansion cache". Such a cache is a byte array that encodes all information necessary to expand a `Descriptor` a second time without access to private keys, and without the need to perform expensive BIP32 derivations. For all currently defined descriptors, the cache simply contains a concatenation of all public keys used.
This is motivated by the goal of importing a descriptor into the wallet and using it as a replacement for the keypool, where it would be impossible to expand descriptors if they use hardened derivation.
Tree-SHA512: f531a0a82ec1eecc30b78ba8a31724d1249826b028cc3543ad32372e1aedd537f137ab03dbffc222c5df444d5865ecd5cec754c1ae1d4989b6e9baeaffade32a
Calling getblocktemplate without the segwit rule specified is most
likely a client error, since it results in lower fees for the miner.
Prevent this client error by failing getblocktemplate if called without
the segwit rule specified.
cf4b0327ed Use std::numeric_limits<UNSIGNED>::max()) instead of (UNSIGNED)-1 (practicalswift)
6b82fc59eb Use const in COutPoint class (Hennadii Stepanov)
Pull request description:
Refactoring:
- all cases of using `(uint32_t) -1` in `COutPoint` class are replaced with const;
- also all remaining instances of `(UNSIGNED)-1` transformed to `std::numeric_limits<UNSIGNED>::max()` (by @practicalswift).
Tree-SHA512: fc7fe9838b6e5136d8b97ea3d6f64c4aaa1215f4369832df432cab017396620bb6e30520a64180ceab6de222562ac11eab243a78dfa5a658ba018835a34caa19
bf2e01097 uint256: Remove unnecessary crypto/common.h use (Karl-Johan Alm)
Pull request description:
This is an alternative to #13242 which keeps the `ReadLE64` part, but moves the `crypto/common.h` dependency into `crypto/common.h` as a function outside of `uint256`.
**Reason:** this change will remove dependencies for `uint256` to `crypto/common.h`, `compat/endian.h`, and `compat/byteswap.h`.
This PR removes the need to update tests to be endian-aware/-independent, but keeps the (arguably dubious) `ReadLE64` part (which was only introduced to fix the tests, not for any functionality).
Tree-SHA512: 78b35123cdb185b3b3ec59aba5ca8a5db72624d147f2d6a5484ffa5ce626a72f782a01dc6893fc8f5619b03e2eae7b5a03b0df5d43460f3bda428e719e188aec
109699dd33 Add release notes (Pieter Wuille)
b65326b562 Add matching descriptors to scantxoutset output + tests (Pieter Wuille)
16203d5df7 Add descriptors to listunspent and getaddressinfo + tests (Pieter Wuille)
9b2a25b13f Add tests for InferDescriptor and Descriptor::IsSolvable (Pieter Wuille)
225bf3e3b0 Add Descriptor::IsSolvable() to distinguish addr/raw from others (Pieter Wuille)
4d78bd93b5 Add support for inferring descriptors from scripts (Pieter Wuille)
Pull request description:
This PR adds functionality to convert a script to a descriptor, given a `SigningProvider` with the relevant information about public keys and redeemscripts/witnessscripts.
The feature is exposed in `listunspent`, `getaddressinfo`, and `scantxoutset` whenever these calls are applied to solvable outputs/addresses.
This is not very useful on its own, though when we add RPCs to import descriptors, or sign PSBTs using descriptors, these strings become a compact and standalone way of conveying everything necessary to sign an output (excluding private keys).
Unit tests and rudimentary RPC tests are included (more relevant tests can be added once RPCs support descriptors).
Fixes#14503.
Tree-SHA512: cb36b84a3e0200375b7e06a98c7e750cfaf95cf5de132cad59f7ec3cbd201f739427de0dc108f515be7aca203652089fbf5f24ed283d4553bddf23a3224ab31f
b7b36decaf fix uninitialized read when stringifying an addrLocal (Kaz Wesley)
8ebbef0169 add test demonstrating addrLocal UB (Kaz Wesley)
Pull request description:
Reachable from either place where SetIP is used when all of:
- our best-guess addrLocal for a peer is IPv4
- the peer tells us it's reaching us at an IPv6 address
- NET logging is enabled
In that case, SetIP turns an IPv4 address into an IPv6 address without
setting the scopeId, which is subsequently read in GetSockAddr during
CNetAddr::ToStringIP and passed to getnameinfo. Fix by ensuring every
constructor initializes the scopeId field with something.
Tree-SHA512: 8f0159750995e08b985335ccf60a273ebd09003990bcf2c3838b550ed8dc2659552ac7611650e6dd8e29d786fe52ed57674f5880f2e18dc594a7a863134739e3
b08af10fb2 disallow oversized CBlockHeaderAndShortTxIDs (Kaz Wesley)
6bed4b374d fix a deserialization overflow edge case (Kaz Wesley)
051faf7e9d add a test demonstrating an overflow in a deserialization edge case (Kaz Wesley)
Pull request description:
A specially-constructed BlockTransactionsRequest can cause `offset` to wrap in deserialization. In the current code, there is not any way this could be dangerous; but disallowing it reduces the potential for future surprises.
Tree-SHA512: 1aaf7636e0801a905ed8807d0d1762132ac8b4421a600c35fb6d5e5033c6bfb587d8668cd9f48c7a08a2ae793a677b7649661e3ae248ab4f8499ab7b6ede483c
9cc0230cfc Add NODISCARD to all {Decode,Parse}[...](...) functions returning bool. Sort includes. (practicalswift)
579497e77a tests: Explicitly ignore the return value of DecodeBase58(...) (practicalswift)
145fe95ec7 tests: Check return value of ParseParameters(...) (practicalswift)
7c5bc2a523 miner: Default to DEFAULT_BLOCK_MIN_TX_FEE if unable to parse -blockmintxfee (practicalswift)
Pull request description:
Changes in this PR:
* ~~Add linter to make sure the return value of `Parse[...](...)` is checked~~
* Add `__attribute__((warn_unused_result))` to all `{Decode,Parse}[...](...)` functions returning `bool`
* Fix violations
Context:
* #13712: `wallet: Fix non-determinism in ParseHDKeypath(...). Avoid using an uninitialized variable in path calculation.` would have been prevented by this
Tree-SHA512: 41a97899f2d5a26584235fa02b1ebfb4faacd81ea97e927022955a658fa7e15d07a1443b4b7635151a43259a1adf8f2f4de3c1c75d7b5f09f0d5496463a1dae6
081accb875 Pass chain locked variables where needed (Russell Yanofsky)
79d579f4e1 Remove uses of cs_main in wallet code (Russell Yanofsky)
ea961c3d72 Remove direct node->wallet calls in init.cpp (Russell Yanofsky)
8db11dd0b1 Pass chain and client variables where needed (Russell Yanofsky)
7e2e62cf7c Add skeleton chain and client classes (Russell Yanofsky)
Pull request description:
This creates an incomplete [`Chain`](https://github.com/ryanofsky/bitcoin/blob/pr/wipc-sep/src/interfaces/chain.h) interface in [`src/interfaces/`](https://github.com/ryanofsky/bitcoin/tree/pr/wipc-sep/src/interfaces) and begins to update wallet code to use it.
#10973 builds on this, changing the wallet to use the new interface to access chain state, instead of using CBlockIndex pointers and global variables like `chainActive`.
Tree-SHA512: 6ef05a4d8ebf57f2ad71835e4d970c9c59e34057e39e48cee76b887492c2fee907e3f6a74a9861e5a9f97cdc6823f4865ebc41ec556ab371ebca1b664c20dbea
535203075e Avoid using numeric_limits for sequence numbers and lock times (Russell Yanofsky)
bafb921507 Remove duplicated code (Hennadii Stepanov)
e4dc39b3bc Replace platform dependent type with proper const (Hennadii Stepanov)
Pull request description:
Switches to named constants, because numeric_limits calls can be harder to read and less portable.
Change was suggested by jamesob in https://github.com/bitcoin/bitcoin/pull/10973#discussion_r213473620
There are no changes in behavior except on some platforms we don't support (ILP64, IP16L32, I16LP32), where `SignalsOptInRBF` and `MutateTxAddInput` functions would now work correctly.
Tree-SHA512: 3f5c6393c260551f65a0edfba55ef7eb3625232eec8d85b1457f26e144aa0b90c7ef5f44b2fd2f7d9be3c3bcb301030a9f5473c21b3bac566cc59b8c8780737c
This commit does not change behavior. All it does is pass new function
parameters.
It is easiest to review this change with:
git log -p -n1 -U0 --word-diff-regex=.
fa511e8dad Pass tx pool reference into CheckSequenceLocks (MarcoFalke)
Pull request description:
`CheckSequenceLocks` is called from ATMP and the member function `CTxMemPool::removeForReorg` without passing in the tx pool object that is used in those function's scope and instead using the global `::mempool` instance.
This fix should be refactoring only, since currently there is only one (global) tx pool in normal operation. Though, it fixes hard to track down issues in future settings where more than one mempool exists at a time. (E.g. for tests, rpc or p2p tx relay purposes)
Tree-SHA512: f0804588c7d29bb6ff05ec14f22a16422b89ab31ae714f38cd07f811d7dc7907bfd14e799c4c1c3121144ff22711019bbe9212b39e2fd4531936a4119950fa49
29ed2d64f6 Improve CAmount tests (Hennadii Stepanov)
Pull request description:
This provides:
- more `MoneyRange` tests;
- new `CFeeRate` constructor tests with zero byte size;
- explicit using of the `CAmount` type.
Tree-SHA512: ca0ad6ccb37909a2a5c11034dc07b316a84c32fb40c6f8b6cfc28ebec72a1de157f31d22e767ae80d70ed06d7296f23870cc5ed0689f34a754ae763d50e23d43
43c7fbb1e7 Make MSVC compiler read the source code using utf-8 (Chun Kuan Lee)
f86a571edb tests: Add test case for std::ios_base::ate (Chun Kuan Lee)
a554cc901a Move boost/std fstream to fsbridge (Chun Kuan Lee)
86eb3b3f1a utils: Add fsbridge fstream function wrapper (Chun Kuan Lee)
Pull request description:
If compiled with mingw, use glibc++ extension `stdio_filebuf` to open the file by `FILE*` instead of filename.
In other condition, we can use boost::fstream.
Tree-SHA512: b5dbd83e347fb9b2a0c8b1c2c7bd71a272e839ec0617883b2a0ec12506ae9e825373cf6e95b9bcc91d7edc85bf51580a7716b56a9ecaad776bc3ae61638cb3da
fa84723e73 amount: Move CAmount CENT to unit test header (MarcoFalke)
Pull request description:
`CAmount` is currently not type-safe. Exporting a constant (`CENT`) that is commonly not referred to by that name might be confusing. `CENT` is only used in two places prior to this commit (`ParseMoney` and `MIN_CHANGE`). So replace these with constants relative to `COIN` and move `CENT` to the unit test header.
Tree-SHA512: 5273e96d8664ced6ae211abde2e20bc763e6e99f89404eec02c621f29e1d235e5f9b1ade933743843fae16fc24b643f883deda9221e3d9fd31229d2ab63a914f
9c5af58d51 Consolidate redundant implementations of ParseHashStr (Ben Woosley)
Pull request description:
This change:
* adds a length check to all calls to `ParseHashStr`, appropriate given its use to populate
a 256-bit number from a hex str
* allows the caller to handle the failure, which allows for the more
appropriate `JSONRPCError` on failure in `prioritisetransaction` rpc
Relative to #14288
Tree-SHA512: baa791147e5ceb3c30c70df3981aaf807bf7d4a90a0be3625540b59aa4b9a9d303a452bfef18bf167cbb833ef9591b4ef5948bf4a1ce67b421d804ae8d20ea53
This change:
* adds a length check to ParseHashStr, appropriate given its use to populate
a 256-bit number from a hex str.
* allows the caller to handle the failure, which allows for the more
appropriate JSONRPCError on failure in prioritisetransaction rpc
b6718e373e tests: Use MakeUnique to construct objects owned by unique_ptrs (practicalswift)
Pull request description:
A subset of #14211 ("Use MakeUnique to construct objects owned by unique_ptrs") as suggested by @MarcoFalke in https://github.com/bitcoin/bitcoin/pull/14211#issuecomment-423324019.
Use `MakeUnique` to construct objects owned by `unique_ptr`s.
Rationale:
* `MakeUnique` ensures exception safety in complex expressions.
* `MakeUnique` gives a more concise statement of the construction.
Tree-SHA512: 1228ae6ce7beb178d79142c4e936b728178ccaa8aa35c6d8feeb33d1a667abfdd010c59996a9d833594611e913877ce5794e75953d11d9b1fdbac04aa491d9cf
893628be01 Drop minor GetSerializeSize template (Ben Woosley)
da74db0940 Drop unused GetType() from CSizeComputer (Ben Woosley)
Pull request description:
Based on conversation in #13462, it seems the serialization `GetType` has very narrow use/effect. In every case except for `CAddress`, which specifically relates to a network peer's address, not a wallet address etc., the serialized representation of an object is irrespective of its destination / type.
This removes the unused `GetType` method from `CSizeComputer` as a step to further narrowing that use.
Tree-SHA512: e72b8e9e5160396691e05aeaee3aba5a57935a75bd5005cfcc7fb51c936f3d1728a397f999da5c36696506dd815fafa5c738f3894df8864f25f91f639eba9c3d
b2f49bd732 Integration of property based testing into Bitcoin Core (Chris Stewart)
Pull request description:
This PR is a subset of the changes in #8469. It's meant to be easier to review. This PR contains all of the build instructions needed for travis to pass. It includes one property call `key_properties.cpp` along with a generator file called `crypto_gen.{h,cpp}`.
Tree-SHA512: 895c9d9273dcd29f696b1de8dfe1ee843095831bf1f68472844181278850bec36b20f0ba7e51e796112c5cc75cd24759f9f1771906503bbf3af16f627e18c6c9
f34c8c466a Make objects in range declarations immutable by default. Avoid unnecessary copying of objects in range declarations. (practicalswift)
Pull request description:
Make objects in range declarations immutable by default.
Rationale:
* Immutable objects are easier to reason about.
* Prevents accidental or hard-to-notice change of value.
Tree-SHA512: cad69d35f0cf8a938b848e65dd537c621d96fe3369be306b65ef0cd1baf6cc0a9f28bc230e1e383d810c555a6743d08cb6b2b0bd51856d4611f537a12e5abb8b
8ecaee13f7 Increase signal to noise in appveyor build output by reducing the MSVC warning count from 12 to 4 (12 is assuming the changes in #14086 are also implemented). (practicalswift)
Pull request description:
Remove unreferenced local variables:
Increase signal to noise in appveyor build output by reducing the MSVC warning count from 12 to 4. 12 is the number of MSVC warnings under our current appveyor setup assuming the changes in #14086 are also implemented.
This makes it easier to spot errors or more important warnings in the verbose appveyor output. MSVC warnings are good, so having access to them in a noise free way (read: without trivial warnings) via appveyor without having to use Windows is really valuable.
See https://github.com/bitcoin/bitcoin/pull/14086#issuecomment-416610313 plus discussion for context.
Before:
```
c:\projects\bitcoin\src\script\script.cpp(272): warning C4018: '>': signed/unsigned mismatch [C:\projects\bitcoin\build_msvc\libbitcoinconsensus\libbitcoinconsensus.vcxproj]
c:\projects\bitcoin\src\rest.cpp(467): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\libbitcoin_server\libbitcoin_server.vcxproj]
c:\projects\bitcoin\src\test\allocator_tests.cpp(147): warning C4312: 'reinterpret_cast': conversion from 'int' to 'void *' of greater size [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\coins_tests.cpp(511): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\coins_tests.cpp(524): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\coins_tests.cpp(722): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\coins_tests.cpp(783): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\crypto_tests.cpp(535): warning C4334: '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?) [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\dbwrapper_tests.cpp(265): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\net_tests.cpp(118): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\net_tests.cpp(151): warning C4101: 'e': unreferenced local variable [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\scheduler_tests.cpp(57): warning C4305: 'argument': truncation from 'int' to 'bool' [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
```
After:
```
c:\projects\bitcoin\src\script\script.cpp(272): warning C4018: '>': signed/unsigned mismatch [C:\projects\bitcoin\build_msvc\libbitcoinconsensus\libbitcoinconsensus.vcxproj]
c:\projects\bitcoin\src\test\allocator_tests.cpp(147): warning C4312: 'reinterpret_cast': conversion from 'int' to 'void *' of greater size [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\crypto_tests.cpp(535): warning C4334: '<<': result of 32-bit shift implicitly converted to 64 bits (was 64-bit shift intended?) [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
c:\projects\bitcoin\src\test\scheduler_tests.cpp(57): warning C4305: 'argument': truncation from 'int' to 'bool' [C:\projects\bitcoin\build_msvc\test_bitcoin\test_bitcoin.vcxproj]
```
Tree-SHA512: 5051134126c570b8421d57c710f1f1b977600398d2b5e69f8a8bd766b3696f992bf4e3459643b99a6b7e08dee1adc92985ee4d0d52b20755954415cb6f23f2fb
Call sync.h primitives "locks" and "mutexes" instead of "blocks" and "waitable
critical sections" to match current coding conventions and c++11 standard
names.
This PR does not rename the "CCriticalSection" class (though this could be done
as a followup) because it is used everywhere and would swamp the other changes
in this PR. Plain mutexes should mostly be preferred instead of recursive
mutexes in new code anyway.
-BEGIN VERIFY SCRIPT-
set -x
set -e
ren() { git grep -l $1 | xargs sed -i s/$1/$2/; }
ren CCriticalBlock UniqueLock
ren CWaitableCriticalSection Mutex
ren CConditionVariable std::condition_variable
ren cs_GenesisWait g_genesis_wait_mutex
ren condvar_GenesisWait g_genesis_wait_cv
perl -0777 -pi -e 's/.*typedef.*condition_variable.*\n\n?//g' src/sync.h
-END VERIFY SCRIPT-
9c4dc597dd Use LOCK macros for non-recursive locks (Russell Yanofsky)
1382913e61 Make LOCK, LOCK2, TRY_LOCK work with CWaitableCriticalSection (Russell Yanofsky)
ba1f095aad MOVEONLY Move AnnotatedMixin declaration (Russell Yanofsky)
41b88e9337 Add unit test for DEBUG_LOCKORDER code (Russell Yanofsky)
Pull request description:
Make LOCK macros work with non-recursive mutexes, and use wherever possible for better deadlock detection.
Also add unit test for DEBUG_LOCKORDER code.
Tree-SHA512: 64ef209307f28ecd0813a283f15c6406138c6ffe7f6cbbd084161044db60e2c099a7d0d2edcd1c5e7770a115e9b931b486e86c9a777bdc96d2e8a9f4dc192942
ca1a093127 Add regression test: Don't assert(...) with side effects (practicalswift)
4c3c9c3869 Don't assert(...) with side effects (practicalswift)
Pull request description:
Don't `assert(...)` with side effects.
From the developer notes:
> **Assertions should not have side-effects**
>
> Rationale: Even though the source code is set to refuse to compile with assertions disabled, having side-effects in assertions is unexpected and makes the code harder to understand
These assertions were introduced quite recently (in #14069 which was merged two days ago) and since this is a recurring thing (see #13534 – "Don't assert(foo()) where foo() has side effects" from May) I added a simple regression test for the most obvious common side effect.
Tree-SHA512: be65db9d8d5d0f5752152ba73fe3fbb0531880f156d3cd7dfdf1752709979b63214e46ae64b1adbe1e09fa121278f4087f4ae49bff16cf8f5aec16ea6bde3650
fa587773e5 scripted-diff: Remove unused first argument to addUnchecked (MarcoFalke)
fe5c49766c tx pool: Use the entry's hash instead of the one passed to addUnchecked (MarcoFalke)
ddd395f968 Mark CTxMemPoolEntry members that should not be modified const (MarcoFalke)
Pull request description:
Several years ago the transaction hash was not cached. For optimization the hash was instead passed into `addUnchecked` to avoid re-calculating it. See f77654a0e9
Passing in the hash is now redundant and the argument can safely be removed.
Tree-SHA512: 0206b65c7a014295f67574120e8c5397bf1b1bd70c918ae1360ab093676f7f89a6f084fd2c7000a141baebfe63fe6f515559e38c4ac71810ba64f949f9c0467f
b193d5a443 Removes the Boost case_conv.hpp dependency. (251)
7a208d9fad Implements custom tolower and toupper functions. (251)
e2ba043b8d Implements ParseNetwork unit test. (251)
Pull request description:
This pull request removes the `boost/algorithm/string/case_conv.hpp` dependency from the project.
`boost/algorithm/string/case_conv.hpp` is included for the `boost::to_lower` and `boost::to_upper` template functions.
We can replace the calls to these functions with straightforward alternative implementations that use the C++ Standard Library, because the functions are called with `std::string` objects that use standard 7-bit ASCII characters as argument.
The refactored implementation should work without the explicit `static_cast<unsigned char>` cast and `unsigned char` lambda return type. Both have been added defensively and to be explicit. Especially in case of the former, behaviour is undefined (potentially result in a crash) if the `std::toupper` argument is not an `unsigned char`.
A potential alternative, maybe even preferred, implementation to address the `boost::to_lower` function call in `ParseNetwork(std::string)` could have been:
```c++
if (net == "ipv4" || net == "IPv4") return NET_IPV4;
if (net == "ipv6" || net == "IPv6") return NET_IPV6;
```
This alternative implementation would however change the external behaviour of `ParseNetwork(std::string)`.
This pull requests includes a unit test to validate the implementation of `ParseNetwork(std::string)` prior and after the removal of the `case_conv.hpp` dependency.
`boost/algorithm/string/case_conv.hpp` has been removed from the `EXPECTED_BOOST_INCLUDES` in `test/lint/lint-includes.sh` because it is no longer required.
Tree-SHA512: d803ae709f2368a3efb223097384a722436955bce0c44a1a5cffd0abb3164be0cce85ba0e9ebd9408166df3f1a95ea0c0d29e3a2534af2fae206c0419d67fde9
This commit implements custom equivalents for the C and C++ `tolower` and `toupper` Standard Library functions.
In addition it implements a utility function to capitalize the first letter of a string.
1ac3c983bf Mark single-argument constructors "explicit" (practicalswift)
Pull request description:
Mark single-argument constructors `explicit`.
Rationale:
* Avoid unexpected implicit promotions.
From the developer notes:
> **By default, declare single-argument constructors explicit.**
> Rationale: This is a precaution to avoid unintended conversions that might arise when single-argument constructors are used as implicit conversion functions.
Tree-SHA512: 7901ed5be808c9d0ecb5ca501e1bc0395987fe1b7941b8548cebac2ff08a14f7dab61fab374a69b9ba29a9295a04245c814325c7f95b97ae558af0780f111dfa
update copyright headers
attempt to fix linting errors
Fixing issue with make check classifying generator files as actual unit tests
Wrapping gen files in ENABLE_PROPERTY_TESTS macro
Make macro better
faf4a9b674 qa: Stop txindex thread before calling destructor (MarcoFalke)
Pull request description:
Same as #13894, but for the tests.
Tree-SHA512: a21d9f8ad8dc9703217d1808cb14bd969903c364fe30bbdc0dd2df170ddc0cbaba98b0bde28bc21ff1319222aaf6cb4f1b2c45cd6b236fe3c645a92eab6bacba
254c85b687 bench: Benchmark GCS filter creation and matching. (Jim Posen)
f33b717a85 blockfilter: Optimization on compilers with int128 support. (Jim Posen)
97b64d67da blockfilter: Unit test against BIP 158 test vectors. (Jim Posen)
a4afb9cadb blockfilter: Additional helper methods to compute hash and header. (Jim Posen)
cd09c7925b blockfilter: Serialization methods on BlockFilter. (Jim Posen)
c1855f6052 blockfilter: Construction of basic block filters. (Jim Posen)
53e7874e07 blockfilter: Simple test for GCSFilter construction and Match. (Jim Posen)
558c536e35 blockfilter: Implement GCSFilter Match methods. (Jim Posen)
cf70b55005 blockfilter: Implement GCSFilter constructors. (Jim Posen)
c454f0ac63 blockfilter: Declare GCSFilter class for BIP 158 impl. (Jim Posen)
9b622dc722 streams: Unit tests for BitStreamReader and BitStreamWriter. (Jim Posen)
fe943f99bf streams: Implement BitStreamReader/Writer classes. (Jim Posen)
87f2d9ee43 streams: Unit test for VectorReader class. (Jim Posen)
947133dec9 streams: Create VectorReader stream interface for vectors. (Jim Posen)
Pull request description:
This implements the compact block filter construction in [BIP 158](https://github.com/bitcoin/bips/blob/master/bip-0158.mediawiki). The code is not used anywhere in the Bitcoin Core code base yet. The next step towards [BIP 157](https://github.com/bitcoin/bips/blob/master/bip-0157.mediawiki) support would be to create an indexing module similar to `TxIndex` that constructs the basic and extended filters for each validated block.
### Filter Sizes
[Here](https://gateway.ipfs.io/ipfs/QmRqaAAQZ5ZX5eqxP7J2R1MzFrc2WDdKSWJEKtQzyawqog) is a CSV of filter sizes for blocks in the main chain.
As you can see below, the ratio of filter size to block size drops after the first ~150,000 blocks:
The reason for the relatively large filter sizes is that Golomb-coded sets only achieve good compression with a sufficient number of elements. Empirically, the average element size with 100 elements is 14% larger than with 10,000 elements.
The ratio of filter size to block size is computed without witness data for basic filters. Here is a summary table of filter size ratios *for blocks after height 150,000*:
| Stat | Filter Type |
|-------|--------------|
| Weighted Size Ratio Mean | 0.0198 |
| Size Ratio Mean | 0.0224 |
| Size Ratio Std Deviation | 0.0202 |
| Mean Element Size (bits) | 21.145 |
| Approx Theoretical Min Element Size (bits) | 21.025 |
Tree-SHA512: 2d045fbfc3fc45490ecb9b08d2f7e4dbbe7cd8c1c939f06bbdb8e8aacfe4c495cdb67c820e52520baebbf8a8305a0efd8e59d3fa8e367574a4b830509a39223f
ddddce0e46 util: Replace boost::signals2 with std::function (MarcoFalke)
Pull request description:
This removes the `#include <boost/signals2/signal.hpp>` from `util.h` (hopefully speeding up the build time and reducing the memory usage further after #13634)
The whole translation interface is replaced by a function `G_TRANSLATION_FUN` that is set to nullptr in units that don't need translation. (Thus only set in the gui)
Tree-SHA512: 087c717358bbed8bdb409463e225239d667f1ced381abb10e7cd31a41dcdd2cebe20b43c2ee86f0f8e55d53301f75e963f07421a99a7ff4c0cad2c6a375c5ab1
984d72ec65 Return the script type from Solver (Ben Woosley)
Pull request description:
Because false is synonymous with TX_NONSTANDARD, this conveys the same
information and makes the handling explicitly based on script type,
simplifying each call site.
Prior to this change it was common for the return value to be ignored, or for the
return value and TX_NONSTANDARD to be redundantly handled.
Tree-SHA512: 31864f856b8cb75f4b782d12678070e8b1cfe9665c6f57cfb25e7ac8bcea8a22f9a78d7c8cf0101c841f2a612400666fb91798bffe88de856e98b873703b0965
6ad0328f1c Don't assert(foo()) where foo has side effects (practicalswift)
Pull request description:
Don't `assert(foo())` where `foo` has side effects.
From `assert(3)`:
> If the macro `NDEBUG` is defined at the moment `<assert.h>` was last included, the macro `assert()` generates no code, and hence does nothing at all.
Bitcoin currently cannot be compiled without assertions, but we shouldn't rely on that.
Tree-SHA512: 28cff0c6d1c2fb612ca58c9c94142ed01c5cfd0a2fecb8e59cdb6c270374b215d952ed3491d921d84dc1b439fa49da4f0e75e080f6adcbc6b0e08be14e54c170
4b7091a842 Replace median fee rate with feerate percentiles (Marcin Jachymiak)
Pull request description:
Currently, the `medianfeerate` statistic is calculated from the feerate of the middle transaction of a list of transactions sorted by feerate.
This PR instead uses the value of the 50th percentile weight unit in the block, and also calculates the feerate at the 10th, 25th, 75th, and 90th percentiles. This more accurately corresponds with what is generally meant by median feerate.
Tree-SHA512: 59255e243df90d7afbe69839408c58c9723884b8ab82c66dc24a769e89c6d539db1905374a3f025ff28272fb25a0b90e92d8101103e39a6d9c0d60423a596714
e306be7429 Use 72 byte dummy signatures when watching only inputs may be used (Andrew Chow)
48b1473c89 Use 71 byte signature for DUMMY_SIGNATURE_CREATOR (Andrew Chow)
18dfea0dd0 Always create 70 byte signatures with low R values (Andrew Chow)
Pull request description:
When creating signatures for transactions, always make one which has a 32 byte or smaller R and 32 byte or smaller S value. This results in signatures that are always less than 71 bytes (32 byte R + 32 byte S + 6 bytes DER + 1 byte sighash) with low R values. In most cases, the signature will be 71 bytes.
Because R is not mutable in the same way that S is, a low R value can only be found by trying different nonces. RFC 6979 for deterministic nonce generation has the option to specify additional entropy, so we simply use that and add a uin32_t counter which we increment in order to try different nonces. Nonces are sill deterministically generated as the nonce used will the be the first one where the counter results in a nonce that results in a low R value. Because different nonces need to be tried, time to produce a signature does increase. On average, it takes twice as long to make a signature as two signatures need to be created, on average, to find one with a low R.
Having a fixed size signature makes size calculations easier and also saves half a byte of transaction size, on average.
DUMMY_SIGNATURE_CREATOR has been modified to produce 71 byte dummy signatures instead of 72 byte signatures.
Tree-SHA512: 3cd791505126ce92da7c631856a97ba0b59e87d9c132feff6e0eef1dc47768e81fbb38bfbe970371bedf9714b7f61a13a5fe9f30f962c81734092a4d19a4ef33
Removes medianfeerate result from getblockstats.
Adds feerate_percentiles which give the feerate of the 10th, 25th, 50th,
75th, and 90th percentile weight unit in the block.
When extra entropy is not specified by the caller, CKey::Sign will
now always create a signature that has a low R value and is at most
70 bytes. The resulting signature on the stack will be 71 bytes when
the sighash byte is included.
Using low R signatures means that the resulting DER encoded signature
will never need to have additional padding to account for high R
values.
3fc20632a3 qt: Set BLOCK_CHAIN_SIZE = 220 (DrahtBot)
2b6a2f4a28 Regenerate manpages (DrahtBot)
eb7daf4d60 Update copyright headers to 2018 (DrahtBot)
Pull request description:
Some trivial maintenance to avoid having to do it again after the 0.17 branch off.
(The scripts to do this are in `./contrib/`)
Tree-SHA512: 16b2af45e0351b1c691c5311d48025dc6828079e98c2aa2e600dc5910ee8aa01858ca6c356538150dc46fe14c8819ed8ec8e4ec9a0f682b9950dd41bc50518fa
They should also work with any other mutex type which std::unique_lock
supports.
There is no change in behavior for current code that calls these macros with
CCriticalSection mutexes.
f6b7fc349c Support h instead of ' in hardened descriptor paths (Pieter Wuille)
fddea672eb Add experimental warning to scantxoutset (Jonas Schnelli)
6495849bfd [QA] Extend tests to more combinations (Pieter Wuille)
1af237faef [QA] Add xpub range tests in scantxoutset tests (Jonas Schnelli)
151600bb49 Swap in descriptors support into scantxoutset (Pieter Wuille)
0652c3284f Descriptor tests (Pieter Wuille)
fe8a7dcd78 Output descriptors module (Pieter Wuille)
e54d76044b Add simple FlatSigningProvider (Pieter Wuille)
29943a904a Add more methods to Span class (Pieter Wuille)
Pull request description:
As promised, here is an implementation of my output descriptor concept (https://gist.github.com/sipa/e3d23d498c430bb601c5bca83523fa82) and integration within the `scantxoutset` RPC that was just added through #12196.
It changes the RPC to use descriptors for everything; I hope the interface is simple enough to encompass all use cases. It includes support for P2PK, P2PKH, P2WPKH, P2SH, P2WSH, multisig, xpubs, xprvs, and chains of keys - combined in every possible way.
Tree-SHA512: 63b54a96e7a72f5b04a8d645b8517d43ecd6a65a41f9f4e593931ce725a8845ab0baa1e9db6a7243190d8ac841f6e7e2f520d98c539312d78f7fd687d2c7b88f
cbeaa91dbb Update ValidationInterface() documentation to explicitly specify threading and memory model (Jesse Cohen)
b296b425a7 Update documentation for SingleThreadedSchedulerClient() to specify the memory model (Jesse Cohen)
9994d01d8b Add Unit Test for SingleThreadedSchedulerClient (Jesse Cohen)
Pull request description:
As discussed in #13023 I've split this test out into a separate pr
This test (and documentation update) makes explicit the guarantee (previously undefined, but implied by the 'SingleThreaded' in `SingleThreadedSchedulerClient()`) - that callbacks pushed to the `SingleThreadedSchedulerClient()` obey the single threaded model for memory and execution - specifically, the callbacks are executed fully and in order, and even in cases where a subsequent callback is executed by a different thread, sequential consistency of memory for all threads executing these callbacks is maintained.
Maintaining memory consistency should make the api more developer friendly - especially for users of the validationinterface. To the extent that there are performance implications from this decision, these are not currently present in practice because all use of this scheduler happens on a single thread currently, furthermore the lock should guarantee consistency across callback executions even when callbacks are executed by multiple threads (as the test does).
Tree-SHA512: 5d95a7682c402e5ad76b05bc9dfbca99ca64105f62ab9e78f6fc0f6ea8c5277aa399fbb94298e35cc677b0c2181ff17259584bb7ae230e38aa68b85ecbc22856
12dd101345 scripted-diff: Remove trailing whitespaces (João Barbosa)
Pull request description:
The script test/lint/lint-whitespace.sh should prevent new cases.
This happens in some pulls where the code editor and the author 'git add's them, so this would fix it all.
Tree-SHA512: bcdd3472fcd01a2754e52212c7db1de2fdc422728b06785481954a27162fb72001cb73708329cc56e95bcc5e45c1348ebc4eacc2ccfa6aa12413c7ec450b6a33
e3245f2e7b Removes Boost predicate.hpp dependency (251)
Pull request description:
This pull request removes the `boost/algorithm/string/predicate.hpp` dependency from the project.
To replace the the `predicate.hpp` dependency from the project the function calls to `boost::algorithm::starts_with` and `boost::algorithm::ends_with` have been replaced with respectively C++11's `std::basic_string::front` and `std::basic_string::back` function calls.
Refactors that were not required, but have been done anyways:
- The Boost function `all` was implicitly made available via the `predicate.hpp` header. Instead of including the appropriate header, function calls to `all` have been replaced with function calls to `std::all_of`.
- The `boost::algorithm::is_digit` predicate has been replaced with a custom `IsDigit` function that is locale independent and ASCII deterministic.
Tree-SHA512: 22dda6adfb4d7ac0cabac8cc33e8fb8330c899805acc1ae4ede402c4b11ea75a399414b389dfaa3650d23b47f41351b4650077af9005d598fbe48d5277bdc320
Because false is synonymous with TX_NONSTANDARD, this conveys the same
information and makes the handling explicitly based on script type,
simplifying each call site.
Prior to this change it was common for the return value to be ignored,
or for the return value and TX_NONSTANDARD to be redundantly handled.
This is a squashed commit that squashes the following commits:
This commit removes the `boost/algorithm/string/predicate.hpp` dependenc
from the project by replacing the function calls to `boost::algorithm::starts_with`
`boost::algorithm::ends_with` and `all` with respectively C++11'
`std::basic_string::front`, `std::basic_string::back`, `std::all_of` function calls
This commit replaces `boost::algorithm::is_digit` with a locale independent isdigi
function, because the use of the standard library's `isdigit` and `std::isdigit
functions is discoraged in the developer notes
417b6c1d29 bitcoinconsensus: invalid flags should be set to bitcoinconsensus_error type, add test cases covering bitcoinconsensus error codes (Thomas Kerin)
Pull request description:
A check was added to the bitcoinconsensus verify_script codepath to ensure that callers only used _exposed_ interpreter flags. I think this error should be written to `bitcoinconsensus_err* err` and not returned by verify_script?
I modified the check so it indicates the error using *err like the others, and added tests covering the error codes.
Tree-SHA512: 8ab370e56956a7d4740f83475e6078774affd663ac92383a02b85295da550f1b4f7a7a68f32ed5c5bcb39d98e2f15ec0b76de8399887e7763eb7c1e21d131093
fa43a4138b bench_bitcoin: Avoid read/write to default datadir (MarcoFalke)
ea80b81e2e test_bitcoin: Avoid read/write to default datadir (MarcoFalke)
Pull request description:
tests should never read or write and potentially corrupt the default datadir, so try to avoid it.
Tree-SHA512: ee446ff4bf59da2aed38c2e4758581d6103e9d4c35a118497e9ec21d566ba33d913e160c2d7ba2ea6f937f000343ecea3816154bd87ee47f64f5b0cf9e88f6e0
3339ba28e9 Make g_enable_bip61 a member variable of PeerLogicValidation (Jesse Cohen)
6690a28606 Restrict as much as possible in net_processing to translation unit (Jesse Cohen)
1d4df02b7e [move-only] Move things only referenced in net_processing out of header file (Jesse Cohen)
02bbc05310 Rescope g_enable_bip61 to net_processing (Jesse Cohen)
Pull request description:
As part of a larger effort to decouple net_processing and validation a bit, these are a bunch of simple scope cleanups. I've moved things out of the header file that are only referenced in net_processing and added static (or anonymous namespace) modifiers to everything possible in net_processing.
There are a handful of functions which could be static except that they are exposed for the sake of unit testing - these are explicitly commented. There has been some discussion of a compile time annotation, but no conclusion has been reached on that yet.
This is somewhat related to other prs #12934#13413#13407 and will be followed by prs that reduce reliance on cs_main to synchronize data structures which are translation unit local to net_processing
Tree-SHA512: 46c9660ee4e06653feb42ba92189565b0aea17aac2375c20747c0d091054c63829cbf66d2daddf65682b58ce1d6922e23aefea051a7f2c8abbb6db253a609082
075429a482 Use common SetDataDir method to create temp directory in tests. (winder)
Pull request description:
Took a stab at #12574
Created a `getPath` method which can be used with the `TestingSetup` fixture to create a temp directory. Updated tests using temp directories to use this method.
I tried setting up a `BOOST_GLOBAL_FIXTURE` to create a truly global path for all tests but was getting linker errors when including `boost/test/unit_test.hpp` in `test_bitcoin.cpp`. Even if I had gotten the linking to work, it looks like `make check` invokes the test binary a bunch of times, so it may not have worked anyway.
Tree-SHA512: b51d0f5fada5d652ccc9362596cf98a742aa47f5daf94f189b5f034d8c035c85d095377befdcff7fb4247154d5160e8c500d70f554a2158e2c185a9d24f694f1
f95989b3ed Fix AreInputsStandard test to reference the proper scriptPubKey (Ben Woosley)
Pull request description:
This value doesn't affect the outcome of the test, because the values are
properly set on line 351 (https://github.com/bitcoin/bitcoin/pull/13565/files#diff-b7061098b41bd31ef5db043705441133R351), but this makes the test values internally coherent.
Tree-SHA512: 5a5fda843475abd91f6c366315536d3573e70420d7c6abeebd74a54939d4de774c33faad4560d1fd4b2c35006224d9e7b3a8c925fe9926013586fd1f7aa886cc
b81560029 Remove CombineSignatures and replace tests (Andrew Chow)
ed94c8b55 Replace CombineSignatures with ProduceSignature (Andrew Chow)
0422beb9b Make SignatureData able to store signatures and scripts (Andrew Chow)
b6edb4f5e Inline Sign1 and SignN (Andrew Chow)
Pull request description:
Currently CombineSignatures is used to create the final scriptSig or an input. However ProduceSignature is capable of doing this itself. Using both CombineSignatures and ProduceSignature results in code duplication which is unnecessary.
To move the scriptSig construction to ProduceSignatures, the SignatureData class contains two maps to hold pubkeys mapped to signatures, and script ids mapped to scripts. DataFromTransaction is extended to be able to extract signatures, their public keys, and scripts from existing ScriptSigs.
The SignaureData are then passed down to SignStep which can use the aforementioned maps to get the signatures, pubkeys, and scripts that it needs, falling back to the actual SigningProvider and SignatureCreator if the data are not available in the SignatureData.
Additionally, Sign1 and SignN have been removed and their functionality inlined into SignStep since Sign1 is really just a wrapper around CreateSig.
Since ProduceSignature can produce the final scriptSig or scriptWitness by using SignatureData which has extracted data from the transaction, CombineSignatures is unnecessary as ProduceSignature is able to replicate all of CombineSignatures' functionality.
This also furthers BIP 174 support and begins moving towards a BIP 174 style backend.
The tests have also been updated to use the new combining methodology.
Tree-SHA512: 78cd58a4ebe37f79229bd5eee2958a0bb45cd7f36d0e993eee13ff685b3665dd76ef2dfd5f47d34678995bb587f5594100ee5f6c09b1c69ee96d3684d470d01e
07c493f2d scripted-diff: Replace NET_TOR with NET_ONION (wodry)
Pull request description:
This is a follow-up to #13532, where @promag already asked if this renaming would make sense.
If network shall be named _Onion_ instead of _Tor_ (like in the option `onlynet`), renaming the network enum NET_TOR to NET_ONION maybe would make sense and be stringent.
Change was produced with the following script:
```
#!/bin/bash
for file in $(grep --exclude-dir='.git' --files-with-matches --binary-files=without-match --recursive NET_TOR bitcoin/)
do
sed --in-place --expression='s/NET_TOR/NET_ONION/g' $file
done
```
_Tor_ is used at many other places in the code, though.
Tree-SHA512: 4ffdeca8115031465eb64e1c76694fb77b5900c4ea465d3c13d9b6b75a1eb04c45913f83cdc8bdbef28936aeec4655f1d4905b3b98407da3263632a2128a8d23
bb582a59c Add P2WSH destination helper and use it instead of manual hashing (Pieter Wuille)
eaba1c111 Add additional unit tests for invalid IsMine combinations (Pieter Wuille)
e6b9730c4 Do not expose invalidity from IsMine (Pieter Wuille)
Pull request description:
This improves the handling of INVALID in IsMine:
* Extra INVALID conditions were added to `IsMine` (following https://github.com/bitcoin/bitcoin/pull/13142/files#r185349057), but these were untested. Add unit tests for them.
* In https://github.com/bitcoin/bitcoin/pull/13142#issuecomment-386396975 it was suggested to merge `isInvalid` into the return status. This PR takes a different approach, and removes the `isInvalid` entirely. It was only ever used inside tests, as normal users of IsMine don't care about the reason for non-mine-ness, only whether it is or not. As the unit tests are extensive enough, it seems sufficient to have a black box text (with tests for both compressed and uncompressed keys).
Some addition code simplification is done as well.
Tree-SHA512: 3267f8846f3fa4e994f57504b155b0e1bbdf13808c4c04dab7c6886c2c0b88716169cee9c5b350513297e0ca2a00812e3401acf30ac9cde5d892f9fb59ad7fef
In addition to having the scriptSig and scriptWitness, have SignatureData
also be able to store just the signatures (pubkeys mapped to sigs) and
scripts (script ids mapped to scripts).
Also have DataFromTransaction be able to extract signatures and scripts
from the scriptSig and scriptWitness of an input to put them in SignatureData.
Adds a new SignatureChecker which takes a SignatureData and puts pubkeys
and signatures into it when it successfully verifies a signature.
Adds a new field in SignatureData which stores whether the SignatureData
was complete. This allows us to also update the scriptSig and
scriptWitness to the final one when updating a SignatureData with another
one.
-BEGIN VERIFY SCRIPT-
sed --in-place'' --expression='s/NET_TOR/NET_ONION/g' $(git grep -I --files-with-matches 'NET_TOR')
-END VERIFY SCRIPT-
The --in-place'' hack is required for sed on macOS to edit files in-place without passing a backup extension.
This value doesn't affect the outcome of the test, because the values are
properly set on line 351, but this makes the test values internally coherent.
9fdf05d70c tests: Fix lock-order-inversion (potential deadlock) in DoS_tests. Reported by TSAN. (practicalswift)
Pull request description:
Fix lock-order-inversion (potential deadlock) in `DoS_tests`. Reported by Clang's TSAN.
Makes `src/test/test_bitcoin` pass also when compiled with TreadSanitizer (`./configure --with-sanitizers=thread` with `clang`).
Tree-SHA512: 41403bb7b6e26bdf1b830b5699e27c637d522bae1799d2a19ed4b68b21b2555438b42170d8b1189613beb32a69b76a65175d29a83f5f4e493896c3d0d94ae26d
e56771365b Do not use uppercase characters in source code filenames (practicalswift)
419a1983ca docs: Add a note about the source code filename naming convention (practicalswift)
Pull request description:
Add a note about the source code filename naming convention.
Tree-SHA512: 8d329bd9e19bcd26e74b0862fb0bc2369b46095dbd3e69d34859908632763abd7c3d00ccc44ee059772ad4bae4460c2bcc1c0e22fd9d8876d57e5fcd346cea4b
16e3cd380a Clarify include recommendation (practicalswift)
6d10f43738 Enforce the use of bracket syntax includes ("#include <foo.h>") (practicalswift)
906bee8e5f Use bracket syntax includes ("#include <foo.h>") (practicalswift)
Pull request description:
When analysing includes in the project it is often assumed that the preferred bracket include syntax (`#include <foo.h>`) mentioned in `developer-docs.md` is used consistently. @sipa:s excellent circular dependencies script [`circular-dependencies.py`](50c69b7801/contrib/devtools/circular-dependencies.py) (#13228) is an example of a script making this reasonable assumption.
This PR enables automatic Travis checking of the include syntax making sure that the bracket syntax includes (`#include <foo.h>`) is used consistently.
Tree-SHA512: a414921aabe8e487ebed42f3f1cbd02fecd1add385065c1f2244cd602c31889e61fea5a801507ec501ef9bd309b05d3c999f915cec1c2b44f085bb0d2835c182
2acd1d6716 Drop uint 256 not operator (Ben Woosley)
Pull request description:
All the other operators are integer or bitwise operations, and this is unused
apart from tests.
Note attempting to call `!` on `arith_uint256` results in a build error after this change:
```
test/arith_uint256_tests.cpp:201:17: error: invalid argument type 'const arith_uint256' to unary expression
BOOST_CHECK(!ZeroL);
```
Tree-SHA512: 5791b643f426dac9829e9499d678786f1ad294edb2d840879252a1b642bda55941632114f64048660a5991a984aeba49eeb5dfe64ba0a6275cbe7b1c049d7095
ec3073a274 index: Move index DBs into index/ directory. (Jim Posen)
89eddcd365 index: Remove TxIndexDB from public interface of TxIndex. (Jim Posen)
2318affd27 MOVEONLY: Move BaseIndex to its own file. (Jim Posen)
f376a49241 index: Generalize logged statements in BaseIndex. (Jim Posen)
61a1226d87 index: Extract logic from TxIndex into reusable base class. (Jim Posen)
e5af5fc6fb db: Make reusable base class for index databases. (Jim Posen)
9b0ec1a7f9 db: Remove obsolete methods from CBlockTreeDB. (Jim Posen)
Pull request description:
This refactors most of the logic in TxIndex into a reusable base class for other indices. There are two commits moving code between files, which may be be more easily reviewed using `git diff --color-moved` (https://blog.github.com/2018-04-05-git-217-released/).
The motivation for this is to support BIP 157 by indexing block filters.
<!-- Reviewable:start -->
---
This change is [<img src="https://reviewable.io/review_button.svg" height="34" align="absmiddle" alt="Reviewable"/>](https://reviewable.io/reviews/bitcoin/bitcoin/13243)
<!-- Reviewable:end -->
Tree-SHA512: 0857f04df2aa920178dab2eb8e57984d8eb4d5010deca9971190358479e05b6672ccca2a08af0a7ac9fe02afb947be84cf35a3693204d0667263c6add2959cbf
The code was converting CMutableTransaction to CTransaction multiple times, which implies recalculating the hash multiple times. This commit fixes this by reusing a single CTransaction.
ebec7317ca Drop the chain argument to GetDifficulty (Ben Woosley)
Pull request description:
By dropping the chain argument to `GetDifficulty`. `GetDifficulty` was called in two ways:
* with a guaranteed non-null blockindex
* with no argument
Change the latter case to be provided `chainActive.Tip()` explicitly.
Introduced in: #11748
Tree-SHA512: f2c97014be185f3e3de92db15848548650e4a67fab20a41bcfa851c5c63c245915cbe9380f84d9da2081e8756d31a41de417db1d35cfecf41ddb4f25070eb525
6aa33feadb Drop UpdateTransaction in favor of UpdateInput (Ben Woosley)
Pull request description:
Updating the input explicitly requires the caller to present a mutable
input, which more clearly communicates the effects and intent of the call
(and, often, the enclosing loop).
In most cases, this input is already immediately available and need not be
looked up.
Tree-SHA512: 8c7914a8b7ae975d8ad0e9d760e3c5da65776a5f79d060b8ffb6b3ff7a32235f71ad705f2185b368d9263742d7796bb562395d22b806d90e8502d8c496011e57
fa36aa7965 wallet: Prevent segfault when sending to unspendable witness (MarcoFalke)
Pull request description:
Previously we wouldn't care about the `txnouttype`, but after 4e91820531 we `switch` on the type.
Tree-SHA512: 6b597aba80cb43881671ad7b3a4ad97753864e8005a05c23fdd8ee79953483c08f241b5c392a9b494298eadc5cfba895b0480d916ef4f11d122fd6196f31b84a
4defdfab94 [MOVEONLY] Move unused Merkle branch code to tests (Pieter Wuille)
4437d6e1f3 8-way AVX2 implementation for double SHA256 on 64-byte inputs (Pieter Wuille)
230294bf5f 4-way SSE4.1 implementation for double SHA256 on 64-byte inputs (Pieter Wuille)
1f0e7ca09c Use SHA256D64 in Merkle root computation (Pieter Wuille)
d0c9632883 Specialized double sha256 for 64 byte inputs (Pieter Wuille)
57f34630fb Refactor SHA256 code (Pieter Wuille)
0df017889b Benchmark Merkle root computation (Pieter Wuille)
Pull request description:
This introduces a framework for specialized double-SHA256 with 64 byte inputs. 4 different implementations are provided:
* Generic C++ (reusing the normal SHA256 code)
* Specialized C++ for 64-byte inputs, but no special instructions
* 4-way using SSE4.1 intrinsics
* 8-way using AVX2 intrinsics
On my own system (AVX2 capable), I get these benchmarks for computing the Merkle root of 9001 leaves (supported lengths / special instructions / parallellism):
* 7.2 ms with varsize/naive/1way (master, non-SSE4 hardware)
* 5.8 ms with size64/naive/1way (this PR, non-SSE4 capable systems)
* 4.8 ms with varsize/SSE4/1way (master, SSE4 hardware)
* 2.9 ms with size64/SSE4/4way (this PR, SSE4 hardware)
* 1.1 ms with size64/AVX2/8way (this PR, AVX2 hardware)
Tree-SHA512: efa32d48b32820d9ce788ead4eb583949265be8c2e5f538c94bc914e92d131a57f8c1ee26c6f998e81fb0e30675d4e2eddc3360bcf632676249036018cff343e
If an unknown option is given via either the command line args or
the conf file, throw an error and exit
Update tests for ArgsManager knowing args
Ignore unknown options in the config file for bitcoin-cli
Fix tests and bitcoin-cli to match actual options used
97c112d4ca Declare TorReply parsing functions in torcontrol_tests (Ben Woosley)
Pull request description:
These methods are standalone string parsing methods which were included
into test via an include of torcontrol.cpp, which is bad practice.
~~Splitting them out reveals that they were the only torcontrol.cpp
methods under test, so the test file is renamed tor_reply_tests.cpp.~~
Introduced in #10408
Tree-SHA512: 8ff11a9c900a88f910a73dfe16f43581a567e9d60e9298a8a963fc9dd7cffb4d97a644da677610aafb7d89f1dd1cede9afeae2c6344305e021a9a322dbcea0ac
fac1223a56 Cache witness hash in CTransaction (MarcoFalke)
faab55fbb1 Make CMutableTransaction constructor explicit (MarcoFalke)
Pull request description:
This speeds up:
* compactblocks (v2)
* ATMP
* validation and miner (via `BlockWitnessMerkleRoot`)
* sigcache (see also unrelated #13204)
* rpc and rest (nice, but irrelevant)
This presumably slows down rescan, which uses a `CTransaction` and its `GetHash`, but never uses the `GetWitnessHash`. The slow down is proportional to the number of witness transactions in the rescan window. I.e. early in the chain there should be no measurable slow down. Later in the chain, there should be a slow down, but acceptable given the speedups in the modules mentioned above.
Tree-SHA512: 443e86acfcceb5af2163e68840c581d44159af3fd1fce266cab3504b29fcd74c50812b69a00d41582e7e1c5ea292f420ce5e892cdfab691da9c24ed1c44536c7
60ebc7da4c trivial: Mark overrides as such. (Daniel Kraft)
Pull request description:
This trivial change adds the `override` keyword to some methods that override virtual base class / interface methods. This ensures that any future changes to the interface's method signatures which are not correctly mirrored in the subclasses will break at compile time with a clear error message, rather than at runtime.
Tree-SHA512: cc1bfa5f03b5e29d20e3eab07b0b5fa2f77b47f79e08263dbff43e4f463e9dd8f4f537e2c8c9b6cb3663220dcf40cfd77723cd9fcbd623c9efc90a4cd44facfc
This removes the need to include rpc/blockchain.cpp in order to put
GetDifficulty under test. GetDifficulty was called in two ways:
* with a guaranteed non-null blockindex
* with no argument
Change the latter case to be provided chainActive.Tip() explicitly.
This trivial change adds the "override" keyword to some methods of
subclasses meant to override interface methods. This ensures that any
future change to the interface' method signatures which are not correctly
mirrored in the subclass will break at compile time with a clear error message,
rather than fail at runtime (which is harder to debug).
Updating the input explicitly requires the caller to present a mutable
input, which more clearly communicates the effects and intent of the method.
In most cases, this input is already immediately available and need not be
looked up.
dd435ad Add unit tests for signals generated by ProcessNewBlock() (Jesse Cohen)
a3ae8e6 Fix concurrency-related bugs in ActivateBestChain (Jesse Cohen)
ecc3c4a Do not unlock cs_main in ABC unless we've actually made progress. (Matt Corallo)
Pull request description:
Originally this PR was just to add tests around concurrency in block validation - those tests seem to have uncovered another bug in ActivateBestChain - this now fixes that bug and adds tests.
ActivateBestChain (invoked after a new block is validated) proceeds in steps - acquiring and releasing cs_main while incrementally disconnecting and connecting blocks to sync to the most work chain known (FindMostWorkChain()). Every time cs_main is released the result of FindMostWorkChain() can change - but currently that value is cached across acquisitions of cs_main and only refreshed when an invalid chain is explored. It needs to be refreshed every time cs_main is reacquired. The test added in 6094ce7304 will occasionally fail without the commit fixing this issue 26bfdbaddb
Original description below
--
After a bug discovered where UpdatedBlockTip() notifications could be triggered out of order (#12978), these unit tests check certain invariants about these signals.
The scheduler test asserts that a SingleThreadedSchedulerClient processes callbacks fully and sequentially.
The block validation test generates a random chain and calls ProcessNewBlock from multiple threads at random and in parallel. ValidationInterface callbacks verify that the ordering of BlockConnected BlockDisconnected and UpdatedBlockTip events occur as expected.
Tree-SHA512: 4102423a03d2ea28580c7a70add8a6bdb22ef9e33b107c3aadef80d5af02644cdfaae516c44933924717599c81701e0b96fbf9cf38696e9e41372401a5ee1f3c
After a recent bug discovered in callback ordering in MainSignals,
this test checks invariants in ordering of
BlockConnected / BlockDisconnected / UpdatedChainTip signals
60f61f9 Tighten up bech32::Decode(); add tests. (murrayn)
Pull request description:
Just a few minor optimizations to bech32::Decode():
1) optimize the order and logic of the conditionals
2) get rid of subsequent '(c < 33 || c > 126)' check which is redundant (already performed above)
3) add a couple more bech32 tests (mixed-case)
Tree-SHA512: e41af834c8f6b7d34c22c28b724df42c60f72e00df616e70a12efbc4271d15d80627fe1bc36845caf29f615c238499a566298a863cbe119fef457287231053c8
chain.h does not actually depend on the methods defined in pow.h, just its
include of consensus/params.h, which is standalone and can be included instead.
Confirmed by inspection and successful build.
159c32d1f1 Add assertion to guide static analyzers. Clang Static Analyzer needs this guidance. (practicalswift)
fd447a6efe Fix dead stores. Values were stored but never read. Limit scope. (practicalswift)
Pull request description:
Fix Clang Static Analyzer warnings reported by @kallewoof in #12961:
* Fix dead stores. Values were stored but never read.
* Add assertion to guide static analyzers. See #12961 for details.
Tree-SHA512: 83dbec821f45217637316bee978e7543f2d2caeb7f7b0b3aec107fede0fff8baa756da8f6b761ae0d38537740839ac9752f6689109c38a4b05c0c041aaa3a1fb
364bae5 qa: Pad scriptPubKeys to get minimum sized txs (MarcoFalke)
7485488 Policy to reject extremely small transactions (Johnson Lau)
0f8719b Add transaction tests for constant scriptCode (Johnson Lau)
9dabfe4 Add constant scriptCode policy in non-segwit scripts (Johnson Lau)
Pull request description:
This disables `OP_CODESEPARATOR` in non-segwit scripts (even in an unexecuted branch), and makes a positive `FindAndDelete` result invalid. This ensures that the `scriptCode` serialized in `SignatureHash` is always the same as the script passing to the `EvalScript`.
Tree-SHA512: a0552cb920294d130251c48053fa2ff1fbdd26332e62b52147d918837852750f0ce35ce2cd1cbdb86588943312f8154ccb4925e850dbb7c2254bc353070cd5f8
This disables OP_CODESEPARATOR in non-segwit scripts (even in an unexecuted branch), and makes a positive FindAndDelete result invalid. This ensures that the scriptCode serialized in SignatureHash() is always the same as the script passing to the EvalScript.
7d0f80b Use anonymous namespace instead of static functions (Pieter Wuille)
b61fb71 Mention removal of bare multisig IsMine in release notes (Pieter Wuille)
9c2a8b8 Do not treat bare multisig as IsMine (Pieter Wuille)
08f3228 Optimization: only test for witness scripts at top level (Pieter Wuille)
3619735 Track difference between scriptPubKey and P2SH execution in IsMine (Pieter Wuille)
ac6ec62 Switch to a private version of SigVersion inside IsMine (Pieter Wuille)
19fc973 Do not expose SigVersion argument to IsMine (Pieter Wuille)
fb1dfbb Remove unused IsMine overload (Pieter Wuille)
952d821 Make CScript -> CScriptID conversion explicit (Pieter Wuille)
Pull request description:
Currently our wallet code will treat bare multisig outputs (meaning scriptPubKeys with multiple public keys + `OP_CHECKMULTISIG` operator in it) as ours without the user asking for it, as long as all private keys in it are in our wallet.
This is a pointless feature. As it only works when all private keys are in one place, it's useless compared to single key outputs (P2PK, P2PKH, P2WPKH, P2SH-P2WPKH), and worse in terms of space, cost, UTXO size, and ability to test (due to lack of address format for them).
Furthermore, they are problematic in that producing a list of all `scriptPubKeys` we accept is not tractable (it involves all combinations of all public keys that are ours). In further wallet changes I'd like to move to a model where all scriptPubKeys that are treated as ours are explicit, rather than defined by whatever keys we have. The current behavior of the wallet is very hard to model in such a design, so I'd like to get rid of it.
I think there are two options:
* Remove it entirely (do not ever accept bare multisig outputs as ours, unless watched)
* Only accept bare multisig outputs in situations where the P2SH version of that output would also be acceptable
This PR implements the first option. The second option was explored in #12874.
Tree-SHA512: 917ed45b3cac864cee53e27f9a3e900390c576277fbd6751b1250becea04d692b3b426fa09065a3399931013bd579c4f3dbeeb29d51d19ed0c64da75d430ad9a
54a5a21 [MOVEONLY] Turn CScript::GetOp2 into a function and move to cpp (Pieter Wuille)
6a7456a [MOVEONLY] Move CSCript::FindAndDelete to interpreter (Pieter Wuille)
33a8ecf Delete unused non-const-iterator CSCript::GetOp overloads (Pieter Wuille)
2fb168b Make iterators in CScript::FindAndDelete const (Pieter Wuille)
Pull request description:
This PR moves `FindAndDelete` and `GetOp2` out of CScript (the first is only used inside the interpreter and moved there, the second does not actually depend on any script specifics and works on any vector). Furthermore, all non-const-iterator versions of GetOp are replaced by const ones, removing a number of methods in the process.
The longer term goal here is making the script interpreter independent from the CScript representation.
Note for reviewers: both `FindAndDelete` and `GetScriptOp` are consensus critical.
Tree-SHA512: c4ccf91c0b33c37cff0d474aa8dd2dab25b5b7655e2ed69a9b15e29daf0a67b21d51c23e1defb3a72ec762bd6138de96f69c6db1fb9c1fe1e976e421261aedb7
Such outputs can still be watched, and signed for, but they aren't treated as valid payments.
That means they won't cause transactions to appear in listtransactions, their outputs to be
shown under listunspent, or affect balances.
Seems providing at least minimal visibility to the failure is a good practice.
The only remaining ignored state is in LoadExternalBlockFile, where logging
would likely be spammy.
fae58eca93 tests: Avoid copies of CTransaction (MarcoFalke)
Pull request description:
Avoid the copy (or move) constructor of `CTransaction` in test code, whereever a simple reference can be used instead.
Tree-SHA512: 8ef2077a277d6182996f4671722fdc01a90909ae7431c1e52604aab8ed028910615028caf9b4cb07a9b15fdc04939dea2209cc3189dde7d38271256d9fe1076c
c25321f Add config changes to release notes (Anthony Towns)
5e3cbe0 [tests] Unit tests for -testnet/-regtest in [test]/[regtest] sections (Anthony Towns)
005ad26 ArgsManager: special handling for -regtest and -testnet (Anthony Towns)
608415d [tests] Unit tests for network-specific config entries (Anthony Towns)
68797e2 ArgsManager: Warn when ignoring network-specific config setting (Anthony Towns)
d1fc4d9 ArgsManager: limit some options to only apply on mainnet when in default section (Anthony Towns)
8a9817d [tests] Use regtest section in functional tests configs (Anthony Towns)
30f9407 [tests] Unit tests for config file sections (Anthony Towns)
95eb66d ArgsManager: support config file sections (Anthony Towns)
4d34fcc ArgsManager: drop m_negated_args (Anthony Towns)
3673ca3 ArgsManager: keep command line and config file arguments separate (Anthony Towns)
Pull request description:
The weekly meeting on [2017-12-07](http://www.erisian.com.au/meetbot/bitcoin-core-dev/2017/bitcoin-core-dev.2017-12-07-19.00.log.html) discussed allowing options to bitcoin to have some sensitivity to what network is in use. @theuni suggested having sections in the config file:
<cfields> an alternative to that would be sections in a config file. and on the
cmdline they'd look like namespaces. so, [testnet] port=5. or -testnet::port=5.
This approach is (more or less) supported by `boost::program_options::detail::config_file_iterator` -- when it sees a `[testnet]` section with `port=5`, it will treat that the same as "testnet.port=5". So `[testnet] port=5` (or `testnet.port=5` without the section header) in bitcoin.conf and `-testnet.port=5` on the command line.
The other aspect to this question is possibly limiting some options so that there is no possibility of accidental cross-contamination across networks. For example, if you're using a particular wallet.dat on mainnet, you may not want to accidentally use the same wallet on testnet and risk reusing keys.
I've set this up so that the `-addnode` and `-wallet` options are `NETWORK_ONLY`, so that if you have a bitcoin.conf:
wallet=/secret/wallet.dat
upnp=1
and you run `bitcoind -testnet` or `bitcoind -regtest`, then the `wallet=` setting will be ignored, and should behave as if your bitcoin.conf had specified:
upnp=1
[main]
wallet=/secret/wallet.dat
For any `NETWORK_ONLY` options, if you're using `-testnet` or `-regtest`, you'll have to add the prefix to any command line options. This was necessary for `multiwallet.py` for instance.
I've left the "default" options as taking precedence over network specific ones, which might be backwards. So if you have:
maxmempool=200
[regtest]
maxmempool=100
your maxmempool will still be 200 on regtest. The advantage of doing it this way is that if you have `[regtest] maxmempool=100` in bitcoin.conf, and then say `bitcoind -regtest -maxmempool=200`, the same result is probably in line with what you expect...
The other thing to note is that I'm using the chain names from `chainparamsbase.cpp` / `ChainNameFromCommandLine`, so the sections are `[main]`, `[test]` and `[regtest]`; not `[mainnet]` or `[testnet]` as might be expected.
Thoughts? Ping @MeshCollider @laanwj @jonasschnelli @morcos
Tree-SHA512: f00b5eb75f006189987e5c15e154a42b66ee251777768c1e185d764279070fcb7c41947d8794092b912a03d985843c82e5189871416995436a6260520fb7a4db
be67831 Make DummySignatureCreator a singleton (Pieter Wuille)
190b8d2 Make BaseSignatureCreator a pure interface (Pieter Wuille)
Pull request description:
* Removes the `m_provider` field from `BaseSignatureCreator`. Instead both a `SigningProvider` (which provides keys and scripts) and a `BaseSignatureCreator` (which implements the transaction-specific (or other) signing logic) are passed into and down in `ProduceSignature`, making the two concepts orthogonal.
* Makes `BaseSignatureCreator` a pure interface without constructor, making it easier to implement new derivations of it (for example for message signing).
* As `DummySignatureCreator` now becomes a stateless object, turn it into a singleton `DUMMY_SIGNATURE_CREATOR`.
Tree-SHA512: 5f1f4512e4ea7d02a31df7b9ede55008efa716c5b74a2630ca1c2fc6599584d8bf5f5641487266127f4b3788033803539fbd22b03ef1219c83c10da2d3da3dcd
