Commit graph

17141 commits

Author SHA1 Message Date
fanquake
a1e0359618
Merge #19986: refactor: clean up -Wlogical-op warning in wallet/scriptpubkeyman.cp
95fedd33a2 refactor: Clean up -Wlogical-op warning (maskoficarus)

Pull request description:

  This is a quick patch that fixes #19912 . This change prevents a -Wlogical-op warning that occurs because we're treating a const int value as a boolean. There's no sense checking if a non-zero constant has a value, so I've removed the check.

  #18836 also addresses the same warning, but has a larger scope and will require more review. This pull request will act as a patch to prevent this compile warning until 18836 is merged.

ACKs for top commit:
  MarcoFalke:
    review ACK 95fedd33a2
  hebasto:
    ACK 95fedd33a2, tested on Linux Mint 20 (x86_64):

Tree-SHA512: 13a9d7f7cb472f4c22a01ca2f9771a75267ff769bdae9d0dc6b2c7f3b05369f6dfa859be2b172b39c15ede6c44cddf556380b3565e42850faa65ccd3fe6e175b
2020-10-19 11:07:11 +08:00
fanquake
c92aa8357c
Merge #19911: net: guard vRecvGetData with cs_vRecv and orphan_work_set with g_cs_orphans
da0988daf1 scripted-diff: rename vRecvGetData (Neha Narula)
ba951812ec Guard vRecvGetData (now in net processing) with its own mutex (Neha Narula)
2d9f2fca43 Move vRecvGetData to net processing (Neha Narula)
673247b58c Lock before checking if orphan_work_set is empty; indicate it is guarded (Neha Narula)
8803aee668 Move m_orphan_work_set to net_processing (Neha Narula)
9c47cb29f9 [Rename only] Rename orphan_work_set to m_orphan_work_set. (Neha Narula)

Pull request description:

  Add annotations to guard `vRecvGetData` and `orphan_work_set` and fix up places where they were accessed without a lock. There is no current data race because they happen to be accessed by only one thread, but this might not always be the case.

  Original discussion: https://github.com/bitcoin/bitcoin/pull/18861#discussion_r451778445

ACKs for top commit:
  MarcoFalke:
    review ACK da0988daf1 🐬
  jnewbery:
    Code review ACK da0988daf1
  hebasto:
    ACK da0988daf1, I have reviewed the code and it looks correct, I agree it can be merged.

Tree-SHA512: 31cadd319ddc9273a87e77afc4db7339fd636e816b5e742eba5cb32927ac5cc07a672b2268d2d38a75a0f1b17d93836adab9acf7e52f26ea9a43f54efa57257e
2020-10-19 10:25:38 +08:00
Hennadii Stepanov
76bbcc414f
test: Fix -Wunused-function warning if configured --without-libs
This is a move-only change.
2020-10-18 19:06:01 +03:00
fanquake
5d644778da
Merge #20169: Taproot follow-up: Make ComputeEntrySchnorr and ComputeEntryECDSA const to clarify contract
51365674e8 script: Make ComputeEntrySchnorr and ComputeEntryECDSA const to clarify contract (practicalswift)

Pull request description:

  Make `ComputeEntrySchnorr` and `ComputeEntryECDSA` `const` to clarify contract.

ACKs for top commit:
  benthecarman:
    ACK 51365674e8
  theStack:
    ACK 51365674e8 👌
  sipa:
    utACK 51365674e8

Tree-SHA512: 0f7a72bf6df7a97d21045ead9db398d2a9527c358aeeb894dec34a5386da4cc316e2f3326716e960ef8aa47bf73b99d1f92bb6d45dfa7871c84624bcad8a79f1
2020-10-17 22:46:56 +08:00
Jon Atack
c8abbc9d1f
p2p: declare Announcement::m_state as uint8_t, add getter/setter
to silence these Travis CI GCC compiler warnings:

txrequest.cpp:73:21: warning: ‘{anonymous}::Announcement::m_state’ is
too small to hold all values of ‘enum class {anonymous}::State’
     State m_state : 3;
                     ^
The warnings are based on the maximum value held by the underlying uint8_t
enumerator type, though the intention of the bitfield declaration is the
maximum declared enumerator value.

The warning been silenced in GCC 8.4+ and 9.3+ according to
https://gcc.gnu.org/bugzilla/show_bug.cgi?id=61414
2020-10-16 23:33:44 +02:00
MarcoFalke
9e8d2bd076
Merge bitcoin-core/gui#97: Relax GUI freezes during IBD (when using wallets)
0d9d2a1f7c Only update the updateSmartFeeLabel once in sync (Jonas Schnelli)

Pull request description:

  Calling `updateSmartFeeLabel` and therefore `estimateSmartFee` is pointless during IBD.

  GUI freezes appear because `estimateSmartFee` competes with `processBlock` for the `m_cs_fee_estimator` lock leading to multiple seconds of blocking the GUI thread in `updateSmartFeeLabel`.

ACKs for top commit:
  ryanofsky:
    Code review ACK 0d9d2a1f7c. Clever fix. Didn't test but I remember I could reproduce the startup issue easily before by putting a sleep in estimateSmartFee.
  promag:
    Code review ACK 0d9d2a1f7c.
  hebasto:
    ACK 0d9d2a1f7c, tested on Linux Mint 20 (x86_64) with `QT_FATAL_WARNINGS=1` and `-debug=qt`.

Tree-SHA512: 85ec2266f06ddd7b523e24d2a462f10ed965d5b4d479005263056f81b7fe49996e1568dafb84658af406e9202ed3bfa846d59c10bb951e0f97cee230e30fafd5
2020-10-16 14:08:35 +02:00
Wladimir J. van der Laan
2947ae6f85
Merge #20035: signet: Fix uninitialized read in validation
fa723e3d43 Initialize default-initialized uint256 consensus params to zero explicitly (MarcoFalke)
fa729cdb2c doc: Move assumed-values doxygen comments to header (MarcoFalke)
fa64892b82 signet: Fix uninitialized read in validation (MarcoFalke)

Pull request description:

ACKs for top commit:
  practicalswift:
    re-ACK fa723e3d43: patch still looks correct
  kallewoof:
    ReACK fa723e3d43
  theStack:
    re-ACK fa723e3d43 🍐

Tree-SHA512: db562bcc15af23bbcbf485f0bbf7564c64c144a4368230fd7682e8861d9500f6f5240351e31c560140df43b2e8456eafd9d27d1e8dd682b20afcc279a39dc329
2020-10-16 11:12:08 +02:00
practicalswift
51365674e8 script: Make ComputeEntrySchnorr and ComputeEntryECDSA const to clarify contract 2020-10-16 06:26:46 +00:00
MarcoFalke
fa723e3d43
Initialize default-initialized uint256 consensus params to zero explicitly 2020-10-16 06:29:23 +02:00
fanquake
cbb5f3a2d5
Merge #19836: rpc: Properly deserialize txs with witness before signing
3333077823 rpc: Adjust witness-tx deserialize error message (MarcoFalke)
cccc752569 rpc: Properly deserialize txs with witness before signing (MarcoFalke)

Pull request description:

  Signing a transaction can only happen when the transaction has inputs. A transaction with inputs can always be deserialized as witness-transaction. If `try_no_witness` decoding is attempted, this will lead to rare intermittent failures.

  Fixes #18803

ACKs for top commit:
  achow101:
    ACK 3333077823
  ajtowns:
    ACK 3333077823

Tree-SHA512: 73f8a5cdfe03fb0e68908d2fa09752c346406f455694a020ec0dd1267ef8f0a583b8e84063ea74aac127106dd193b72623ca6d81469a94b3f5b3c766ebf2c42b
2020-10-16 12:05:26 +08:00
Wladimir J. van der Laan
9ad7cd2887
Merge #20090: [doc] Tiny followups to new getpeerinfo connection type field
41dca087b7 [trivial] Extract connection type doc into file where it is used. (Amiti Uttarwar)
3069b56a45 [doc] Improve help for getpeerinfo connection_type field. (Amiti Uttarwar)

Pull request description:

  two commits addressing small followups from #19725

  * first commit adds a clarification in the release notes that this field shouldn't be expected to be stable (suggested by sdaftuar in https://github.com/bitcoin/bitcoin/pull/19725#issuecomment-697421878)

  * second commit moves the `CONNECTION_TYPE_DOC` object out of the header file to reduce the size of the binary (suggested by MarcoFalke in https://github.com/bitcoin/bitcoin/pull/19725#discussion_r495467895, he tested and found a decrease of 10kB)

ACKs for top commit:
  achow101:
    ACK 41dca087b7
  laanwj:
    Code review ACK 41dca087b7

Tree-SHA512: a555df978b4341fbe05deeb40a8a655f0d3c5c1c0adcc1737fd2cf61b204a5a24a301ca0c2b5a3616554d4abf8c57074d22dbda5a50d8450bc22c57679424985
2020-10-15 20:45:56 +02:00
Wladimir J. van der Laan
9855422e65
Merge #17428: p2p: Try to preserve outbound block-relay-only connections during restart
a490d074b3 doc: Add anchors.dat to files.md (Hennadii Stepanov)
0a85e5a7bc p2p: Try to connect to anchors once (Hennadii Stepanov)
5543c7ab28 p2p: Fix off-by-one error in fetching address loop (Hennadii Stepanov)
4170b46544 p2p: Integrate DumpAnchors() and ReadAnchors() into CConnman (Hennadii Stepanov)
bad16aff49 p2p: Add CConnman::GetCurrentBlockRelayOnlyConns() (Hennadii Stepanov)
c29272a157 p2p: Add ReadAnchors() (Hennadii Stepanov)
567008d2a0 p2p: Add DumpAnchors() (Hennadii Stepanov)

Pull request description:

  This is an implementation of #17326:
  - all (currently 2) outbound block-relay-only connections (#15759) are dumped to `anchors.dat` file
  - on restart a node tries to connect to the addresses from `anchors.dat`

  This PR prevents a type of eclipse attack when an attacker exploits a victim node restart to force it to connect to new, probably adversarial, peers.

ACKs for top commit:
  jnewbery:
    code review ACK a490d074b3
  laanwj:
    Code review ACK a490d074b3

Tree-SHA512: 0f5098a3882f2814be1aa21de308cd09e6654f4e7054b79f3cfeaf26bc02b814ca271497ed00018d199ee596a8cb9b126acee8b666a29e225b08eb2a49b02ddd
2020-10-15 20:19:55 +02:00
Jon Atack
9a75e1e569
rpc: update GetNetworksInfo() to not return unsupported networks 2020-10-15 19:21:41 +02:00
Jon Atack
ba8997fb2e
net: update GetNetworkName() with all enum Network cases 2020-10-15 19:21:38 +02:00
Wladimir J. van der Laan
0d22482353
Merge #20002: net, rpc, cli: expose peer network in getpeerinfo; simplify/improve -netinfo
6272604bef refactor: enable -netinfo to add future networks (i2p, cjdns) (Jon Atack)
82fd40216c refactor: promote some -netinfo localvars to class members (Jon Atack)
5133fab37e cli: simplify -netinfo using getpeerinfo network field (Jon Atack)
4938a109ad rpc, test: expose CNodeStats network in RPC getpeerinfo (Jon Atack)
6df7882029 net: add peer network to CNodeStats (Jon Atack)

Pull request description:

  This PR:

  - builds on #19991 and #19998
  - exposes peer networks via a new getpeerinfo `network` field ("ipv4", "ipv6", or "onion"), and adds functional tests
  - updates -netinfo to use getpeerinfo `network` rather than detecting the peer networks client-side
  - refactors -netinfo to easily add future networks

ACKs for top commit:
  laanwj:
    ACK 6272604bef

Tree-SHA512: 28883487585135ceaaf84ce09131f2336e3193407f2e3df0960e3f4ac340f500ab94ffecb9d06a4c49bc05e3cca4f914ea4379860bea0bd5df2f834f74616015
2020-10-15 17:44:38 +02:00
Ivan Metlushko
538be4219a wallet: fix importdescriptor silent fail 2020-10-15 18:02:58 +07:00
Wladimir J. van der Laan
711ddce943
Merge #20131: test: Remove unused nVersion=1 in p2p tests
faad92fe1c test: Remove unused nVersion=1 in p2p tests (MarcoFalke)

Pull request description:

  After commit ddefb5c0b7 nVersion is no
  longer used in p2p logic when sending messages. Only when receiving
  messages, but in this test no messages are received.

ACKs for top commit:
  laanwj:
    Code review ACK faad92fe1c
  fanquake:
    ACK faad92fe1c

Tree-SHA512: 9a7029187aaa5a7929a4a2199646131ff1ea72df6a855ce7022dd3bb2647dd525356dbc5e460c77007eebcdeab400a689db8cb77e8239af3b539c117a4e0d16e
2020-10-15 12:01:11 +02:00
MarcoFalke
560dea9b26
Merge #19770: RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions")
5b57dc5458 RPC: getpeerinfo: Wrap long help line for bytesrecv_per_msg (Luke Dashjr)
d681a28219 RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") (Luke Dashjr)

Pull request description:

  If we were going to continue support for "whitelisted", we should have probably made it true if any permission flag was set, rather than only if "default permissions" were used.

  This corrects the description, and deprecates it.

ACKs for top commit:
  laanwj:
    ACK 5b57dc5458

Tree-SHA512: a2e2137f8be8110357c1b2fef2c923fa8c7c4a49b0b2b3a2d78aedf12f8ed5cc7e140018a21b37e6ec7770ed4007542aeef7ad4558973901b107e8e0f81d6003
2020-10-15 11:49:34 +02:00
Wladimir J. van der Laan
e3b474c548
Merge #20140: Restore compatibility with old CSubNet serialization
886be97af5 Ignore incorrectly-serialized banlist.dat entries (Pieter Wuille)
883cea7dea Restore compatibility with old CSubNet serialization (Pieter Wuille)

Pull request description:

  #19628 changed CSubNet for IPv4 netmasks, using the first 4 bytes of `netmask` rather than the last 4 to store the actual mask. Unfortunately, CSubNet objects are serialized on disk in banlist.dat, breaking compatibility with existing banlists (and bringing them into an inconsistent state where entries reported in `listbanned` cannot be removed).

  Fix this by reverting to the old format (just for serialization). Also add a sanity check to the deserializer so that nonsensical banlist.dat entries are ignored (which would otherwise be possible if someone added IPv4 entries after #19628 but without this PR).

  Reported by Greg Maxwell.

ACKs for top commit:
  laanwj:
    Code review ACK 886be97af5
  vasild:
    ACK 886be97af

Tree-SHA512: d3fb91e8ecd933406e527187974f22770374ee2e12a233e7870363f52ecda471fb0b7bae72420e8ff6b6b1594e3037a5115984c023dbadf38f86aeaffcd681e7
2020-10-15 11:44:36 +02:00
MarcoFalke
fa729cdb2c
doc: Move assumed-values doxygen comments to header 2020-10-15 11:28:13 +02:00
MarcoFalke
fa64892b82
signet: Fix uninitialized read in validation 2020-10-15 11:27:43 +02:00
Wladimir J. van der Laan
3956165903
Merge #17775: DecodeHexTx: Try case where txn has inputs first
27fc6a38f8 DecodeHexTx: Break out transaction decoding logic into own function (Gregory Sanders)
6020ce3c01 DecodeHexTx: Try case where txn has inputs first (Gregory Sanders)

Pull request description:

  Alternative/complementary to https://github.com/bitcoin/bitcoin/pull/17773 to avoid random `decoderawtransaction` failures. Most cases this is used now is on complete transactions, especially with the uptake of PSBT.

ACKs for top commit:
  ajtowns:
    ACK 27fc6a38f8
  achow101:
    ACK 27fc6a38f8

Tree-SHA512: 0a836d7c9951bf7d2764507788dbcc871d520f1ea9b77d6b22f051f4d6224ed779aba0e4f28c5c165040095ee0c70b67080c39164d82de61b19158f7ae6fddb2
2020-10-15 10:55:44 +02:00
Wladimir J. van der Laan
3caee16946
Merge #19953: Implement BIP 340-342 validation (Schnorr/taproot/tapscript)
0e2a5e448f tests: dumping and minimizing of script assets data (Pieter Wuille)
4567ba034c tests: add generic qa-asset-based script verification unit test (Pieter Wuille)
f06e6d0345 tests: functional tests for Schnorr/Taproot/Tapscript (Pieter Wuille)
3c226639eb tests: add BIP340 Schnorr signature support to test framework (Pieter Wuille)
206fb180ec --- [TAPROOT] Tests --- (Pieter Wuille)
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342) (Pieter Wuille)
e9a021d7e6 Make Taproot spends standard + policy limits (Pieter Wuille)
865d2c37e2 --- [TAPROOT] Regtest activation and policy --- (Pieter Wuille)
72422ce396 Implement Tapscript script validation rules (BIP 342) (Johnson Lau)
330de894a9 Use ScriptExecutionData to pass through annex hash (Pieter Wuille)
8bbed4b7ac Implement Taproot validation (BIP 341) (Pieter Wuille)
0664f5fe1f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340) (Pieter Wuille)
5de246ca81 Implement Taproot signature hashing (BIP 341) (Johnson Lau)
9eb590894f Add TaggedHash function (BIP 340) (Pieter Wuille)
450d2b2371 --- [TAPROOT] BIP340/341/342 consensus rules --- (Pieter Wuille)
5d62e3a68b refactor: keep spent outputs in PrecomputedTransactionData (Pieter Wuille)
8bd2b4e784 refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script (Pieter Wuille)
107b57df9f scripted-diff: put ECDSA in name of signature functions (Pieter Wuille)
f8c099e220 --- [TAPROOT] Refactors --- (Pieter Wuille)

Pull request description:

  This is an implementation of the Schnorr/taproot consensus rules proposed by BIPs [340](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), [341](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki), and [342](https://github.com/bitcoin/bips/blob/master/bip-0340.mediawiki).

  See the list of commits [below](https://github.com/bitcoin/bitcoin/pull/19953#issuecomment-691815830). No signing or wallet support of any kind is included, as testing is done entirely through the Python test framework.

  This is a successor to https://github.com/bitcoin/bitcoin/pull/17977 (see discussion following [this comment](https://github.com/bitcoin/bitcoin/pull/17977#issuecomment-682285983)), and will have further changes squashed/rebased. The history of this PR can be found in #19997.

ACKs for top commit:
  instagibbs:
    reACK 0e2a5e448f
  benthecarman:
    reACK 0e2a5e4
  kallewoof:
    reACK 0e2a5e448f
  jonasnick:
    ACK 0e2a5e448f almost only looked at bip340/libsecp related code
  jonatack:
    ACK 0e2a5e448f modulo the last four commits (tests) that I plan to finish reviewing tomorrow
  fjahr:
    reACK 0e2a5e448f
  achow101:
    ACK 0e2a5e448f

Tree-SHA512: 1b00314450a2938a22bccbb4e177230cf08bd365d72055f9d526891f334b364c997e260c10bc19ca78440b6767712c9feea7faad9a1045dd51a5b96f7ca8146e
2020-10-15 10:22:35 +02:00
Samuel Dobson
8ed37f6c84
Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04 Run dumpwallet for legacy wallets only in  wallet_backup.py (Andrew Chow)
6c6639ac9f Include sqlite3 in documentation (Andrew Chow)
f023b7cac0 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866 Set and check the sqlite user version (Andrew Chow)
9d3d2d263c Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8e walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225 Determine wallet file type based on file magic (Andrew Chow)
6045f77003 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4e Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7 Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2 Add SetupSQLStatements (Andrew Chow)
6636a2608a Implement SQLiteBatch::Close (Andrew Chow)
93825352a3 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372b Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe125 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df82580 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e Add libsqlite3 (Andrew Chow)

Pull request description:

  This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.

  For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

  We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.

  I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.

ACKs for top commit:
  Sjors:
    re-utACK c4a29d0a90
  promag:
    Tested ACK c4a29d0a90.
  fjahr:
    reACK c4a29d0a90
  S3RK:
    Re-review ACK c4a29d0a90
  meshcollider:
    re-utACK c4a29d0a90
  hebasto:
    re-ACK c4a29d0a90, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
  ryanofsky:
    Code review ACK c4a29d0a90. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
  jonatack:
    ACK c4a29d0a90, debug builds and test runs after rebase to latest master @ c2c4dbaebd, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.

Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
2020-10-15 20:12:29 +13:00
fanquake
f2e6d14430
Merge #20147: Update libsecp256k1 (endomorphism, test improvements)
52380bf304 Squashed 'src/secp256k1/' changes from 8ab24e8dad..c6b6b8f1bb (Pieter Wuille)

Pull request description:

  This updates the libsecp256k1 subtree to the latest master, which includes:

  * Enabling the GLV endomorphism optimization by default (and removing support for the non-GLV EC multiplication)
  * Added a proof for the correctness of the lambda split algorithm by roconnor-blockstream (other code was relying on the fact that it always outputs 128 bit results, which isn't at all obvious).
  * Improved exhaustive tests, in particular for the Schnorr signature module
  * Various other testing and CI improvements

ACKs for top commit:
  fanquake:
    ACK 9e5626d2a8 - performed a squash and checked that the changes were the same. The non-endomorphism code has now been ripped out.
  benthecarman:
    ACK 9e5626d

Tree-SHA512: 50fda5f3f934ee525f01cfc15e4f5efbc5261a97f2b77fe1b3453ee0edcf1281ad74ab4532a2fe1fe907652dd47023beff8cf3d73bf34f65ac914a694b9e7110
2020-10-15 11:27:47 +08:00
fanquake
661fe5d65c
Merge #20146: net: Send post-verack handshake messages at most once
fa1f6f237d net: Send post-verack handshake messages at most once (MarcoFalke)

Pull request description:

  There is no need to send `SENDHEADERS` and `SENDCMPCT` messages as a reply to each `VERACK` that is received. For alive checks, a `PING`/`PONG` can be used.

ACKs for top commit:
  jonatack:
    Concept ACK fa1f6f237d this is the only code section that sets `fCurrentlyConnected` and `fSuccessfullyConnected` to true. Could add a test. I did not verify if this code is actually being called repeatedly post initial verack; was it?
  hebasto:
    ACK fa1f6f237d, I have reviewed the code and it looks OK, I agree it can be merged.
  naumenkogs:
    ACK fa1f6f237d
  laanwj:
    Code review ACK fa1f6f237d

Tree-SHA512: c841d5d3807254a49463bbcfac3b32881b34a9d3206899544c86322c20988e17ad2ae243cba227fd3825a914f0cb2584451edda2414aecee6d5e3f5a0636f08a
2020-10-15 07:59:19 +08:00
João Barbosa
5e737a0092 rpc, wallet: Expose database format in getwalletinfo 2020-10-14 21:47:42 +01:00
Pieter Wuille
52380bf304 Squashed 'src/secp256k1/' changes from 8ab24e8dad..c6b6b8f1bb
c6b6b8f1bb Merge #830: Rip out non-endomorphism code + dependencies
c582abade1 Consistency improvements to the comments
63c6b71616 Reorder comments/function around scalar_split_lambda
2edc514c90 WNAF of lambda_split output has max size 129
4232e5b7da Rip out non-endomorphism code
ebad8414b0 Check correctness of lambda split without -DVERIFY
fe7fc1fda8 Make lambda constant accessible
9d2f2b44d8 Add tests to exercise lambda split near bounds
9aca2f7f07 Add secp256k1_split_lambda_verify
acab934d24 Detailed comments for secp256k1_scalar_split_lambda
76ed922a5f Increase precision of g1 and g2
6173839c90 Switch to our own memcmp function
63150ab4da Merge #827: Rename testrand functions to have test in name
c5257aed0b Merge #821: travis: Explicitly set --with-valgrind
bb1f54280f Merge #818: Add static assertion that uint32_t is unsigned int or wider
a45c1fa63c Rename testrand functions to have test in name
5006895bd6 Merge #808: Exhaustive test improvements + exhaustive schnorrsig tests
4eecb4d6ef travis: VALGRIND->RUN_VALGRIND to avoid confusion with WITH_VALGRIND
66a765c775 travis: Explicitly set --with-valgrind
d7838ba6a6 Merge #813: Enable configuring Valgrind support
7ceb0b7611 Merge #819: Enable -Wundef warning
8b7dcdd955 Add exhaustive test for extrakeys and schnorrsig
08d7d89299 Make pubkey parsing test whether points are in the correct subgroup
87af00b511 Abstract out challenge computation in schnorrsig
63e1b2aa7d Disable output buffering in tests_exhaustive.c
39f67dd072 Support splitting exhaustive tests across cores
e99b26fcd5 Give exhaustive_tests count and seed cmdline inputs
49e6630bca refactor: move RNG seeding to testrand
b110c106fa Change exhaustive test groups so they have a point with X=1
cec7b18a34 Select exhaustive lambda in function of order
78f6cdfaae Make the curve B constant a secp256k1_fe
d7f39ae4b6 Delete gej_is_valid_var: unused outside tests
8bcd78cd79 Make secp256k1_scalar_b32 detect overflow in scalar_low
c498366e5b Move exhaustive tests for recovery to module
be31791543 Make group order purely compile-time in exhaustive tests
e73ff30922 Enable -Wundef warning
c0041b5cfc Add static assertion that uint32_t is unsigned int or wider
4ad408faf3 Merge #782: Check if variable=yes instead of if var is set in travis.sh
412bf874d0 configure: Allow specifying --with[out]-valgrind explicitly
34debf7a6d Modify .travis.yml to explictly pass no in env vars instead of setting to nothing
a0e99fc121 Merge #814: tests: Initialize random group elements fully
5738e8622d tests: Initialize random group elements fully
c9939ba55d Merge #812: travis: run bench_schnorrsig
a51f2af62b travis: run bench_schnorrsig
ef37761fee Change travis.sh to check if variables are equal to yes instead of not-empty. Before this, setting `VALGRIND=wat` was considered as true, and to make it evaluate as false you had to unset the variable `VALGRIND=` but not it checks if `VALGRIND=yes` and if it's not `yes` then it's evaluated to false

git-subtree-dir: src/secp256k1
git-subtree-split: c6b6b8f1bb044d7d1aa065ebb674adde98a36a8e
2020-10-14 11:41:15 -07:00
Pieter Wuille
9e5626d2a8 Update libsecp256k1 subtree to latest master 2020-10-14 11:41:15 -07:00
Wladimir J. van der Laan
c2c4dbaebd
Merge #19988: Overhaul transaction request logic
fd9a0060f0 Report and verify expirations (Pieter Wuille)
86f50ed10f Delete limitedmap as it is unused now (Pieter Wuille)
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents (Pieter Wuille)
173a1d2d3f Expedite removal of tx requests that are no longer needed (Pieter Wuille)
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers (Pieter Wuille)
242d16477d Change transaction request logic to use txrequest (Pieter Wuille)
5b03121d60 Add txrequest fuzz tests (Pieter Wuille)
3c7fe0e5a0 Add txrequest unit tests (Pieter Wuille)
da3b8fde03 Add txrequest module (Pieter Wuille)

Pull request description:

  This replaces the transaction request logic with an encapsulated class that maintains all the state surrounding it. By keeping it stand alone, it can be easily tested (using included unit tests and fuzz tests).

  The major changes are:

  * Announcements from outbound (and whitelisted) peers are now always preferred over those from inbound peers. This used to be the case for the first request (by delaying the first request from inbound peers), and a bias afters. The 2s delay for requests from inbound peers still exists, but after that, if viable outbound peers remain for any given transaction, they will always be tried first.
  * No more hard cap of 100 in flight transactions per peer, as there is less need for it (memory usage is linear in the number of announcements, but independent from the number in flight, and CPU usage isn't affected by it). Furthermore, if only one peer announces a transaction, and it has over 100 in flight already, we still want to request it from them. The cap is replaced with a rule that announcements from such overloaded peers get an additional 2s delay (possibly combined with the existing 2s delays for inbound connections, and for txid peers when wtxid peers are available).
  * The limit of 100000 tracked announcements is reduced to 5000; this was excessive. This can be bypassed using the PF_RELAY permission (to accommodate locally dumping a batch of many transactions).

  This replaces #19184, rebased on #18044 and with many small changes.

ACKs for top commit:
  ariard:
    Code Review ACK fd9a006. I've reviewed the new TxRequestTracker, its integration in net_processing, unit/functional/fuzzing test coverage. I looked more for soundness of new specification rather than functional consistency with old transaction request logic.
  MarcoFalke:
    Approach ACK fd9a0060f0 🏹
  naumenkogs:
    Code Review ACK fd9a006. I've reviewed everything, mostly to see how this stuff works at the lower level (less documentation-wise, more implementation-wise), and to try breaking it with unexpected sequences of events.
  jnewbery:
    utACK fd9a0060f0
  jonatack:
    WIP light ACK fd9a0060f0 have read the code, verified that each commit is hygienic, e.g. debug build clean and tests green, and have been running a node on and off with this branch and grepping the net debug log. Am still unpacking the discussion hidden by GitHub by fetching it via the API and connecting the dots, storing notes and suggestions in a local branch; at this point none are blockers.
  ryanofsky:
    Light code review ACK fd9a0060f0, looking at txrequest implementation, unit test implementation, and net_processing integration, just trying to understand how it works and looking for anything potentially confusing in the implementation. Didn't look at functional tests or catch up on review discussion. Just a sanity check review focused on:

Tree-SHA512: ea7b52710371498b59d9c9cfb5230dd544fe9c6cb699e69178dea641646104f38a0b5ec7f5f0dbf1eb579b7ec25a31ea420593eff3b7556433daf92d4b0f0dd7
2020-10-14 18:36:59 +02:00
Andrew Chow
f023b7cac0 wallet: Enforce sqlite serialized threading mode 2020-10-14 11:28:18 -04:00
Andrew Chow
6173269866 Set and check the sqlite user version 2020-10-14 11:28:18 -04:00
Andrew Chow
9d3d2d263c Use network magic as sqlite wallet application ID 2020-10-14 11:28:18 -04:00
Andrew Chow
9af5de3798 Use SQLite for descriptor wallets
MakeWalletDatabase no longer has a default DatabaseFormat. Instead
callers, like CWallet::Create, need to specify the database type to
create if the file does not exist. If it exists and NONE is given, then
CreateWalletDatabase will try to autodetect the type.
2020-10-14 11:28:18 -04:00
Andrew Chow
9b78f3ce8e walletutil: Wallets can also be sqlite 2020-10-14 11:28:18 -04:00
Andrew Chow
ac38a87225 Determine wallet file type based on file magic 2020-10-14 11:28:18 -04:00
Andrew Chow
6045f77003 Implement SQLiteDatabase::MakeBatch 2020-10-14 11:28:18 -04:00
Andrew Chow
727e6b2a4e Implement SQLiteDatabase::Verify 2020-10-14 11:28:18 -04:00
Andrew Chow
b4df8fdb19 Implement SQLiteDatabase::Rewrite
Rewrite uses the VACUUM command which does exactly what we want. A
specific advertised use case is to compact a database and ensure that
any deleted data is actually deleted.
2020-10-14 11:28:18 -04:00
Andrew Chow
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort 2020-10-14 11:28:18 -04:00
Andrew Chow
ac5c1617e7 Implement SQLiteDatabase::Backup 2020-10-14 11:28:18 -04:00
Andrew Chow
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor 2020-10-14 11:28:18 -04:00
Andrew Chow
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey 2020-10-14 11:28:18 -04:00
Andrew Chow
7aa45620e2 Add SetupSQLStatements 2020-10-14 11:28:18 -04:00
Andrew Chow
6636a2608a Implement SQLiteBatch::Close 2020-10-14 11:28:18 -04:00
Andrew Chow
93825352a3 Implement SQLiteDatabase::Close 2020-10-14 11:28:18 -04:00
Andrew Chow
a0de83372b Implement SQLiteDatabase::Open 2020-10-14 11:28:18 -04:00
Andrew Chow
3bfa0fe125 Initialize and Shutdown sqlite3 globals
sqlite3 recommends that sqlite3_initialize be called when the
application starts, and sqlite3_shutdown when it stops. Since we don't
always use sqlite3, we initialize it when a SQLiteDatabse is constructed
(calling sqlite3_initialize after initialized is a no-op). We call
sqlite3_shutdown when we see that there are no databases opened. The
number of open databases is tracked by an atomic g_dbs_open.
2020-10-14 11:28:18 -04:00
Andrew Chow
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch 2020-10-14 11:28:17 -04:00
Andrew Chow
ca8b7e04ab Implement SQLiteDatabaseVersion 2020-10-14 11:27:40 -04:00
Andrew Chow
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes 2020-10-14 11:27:37 -04:00
Andrew Chow
54729f3f4e Add libsqlite3 2020-10-14 11:18:12 -04:00
Jon Atack
6272604bef
refactor: enable -netinfo to add future networks (i2p, cjdns)
After this commit, a new network may be added by changing 4 lines:
- increment the value of `m_networks_size`
- add the network name to `m_networks`
- add the network name to this line: `result += "        ipv4    ipv6   onion   total  block-relay\n";`
- add "counts.at(i).at(<m_networks pos>)" to this line: `result += strprintf("%-5s  %5i   %5i   %5i   %5i   %5i\n...`
2020-10-14 16:29:37 +02:00
Luke Dashjr
5b57dc5458 RPC: getpeerinfo: Wrap long help line for bytesrecv_per_msg 2020-10-14 14:16:43 +00:00
Luke Dashjr
d681a28219 RPC: getpeerinfo: Deprecate "whitelisted" field (replaced by "permissions") 2020-10-14 14:16:42 +00:00
Neha Narula
da0988daf1 scripted-diff: rename vRecvGetData
-BEGIN VERIFY SCRIPT-
sed -i 's/vRecvGetData/m_getdata_requests/g' src/net_processing.cpp
-END VERIFY SCRIPT-
2020-10-14 10:08:44 -04:00
Neha Narula
ba951812ec Guard vRecvGetData (now in net processing) with its own mutex
This requires slightly reorganizing the logic in GETBLOCKTXN to
maintain locking order.
2020-10-14 10:08:44 -04:00
Neha Narula
2d9f2fca43 Move vRecvGetData to net processing 2020-10-14 10:08:44 -04:00
Neha Narula
673247b58c Lock before checking if orphan_work_set is empty; indicate it is guarded 2020-10-14 10:08:44 -04:00
Neha Narula
8803aee668 Move m_orphan_work_set to net_processing 2020-10-14 10:08:37 -04:00
Wladimir J. van der Laan
99a1d572ea
Merge #18750: build: optionally skip external warnings
ba8950ee01 build: optionally skip external warnings (Vasil Dimov)

Pull request description:

  Add an option to `./configure` to suppress compilation warnings from
  external headers. The option is off by default (no change in behavior,
  show warnings from external headers).

  This option is useful if e.g. Boost or Qt is installed outside of
  `/usr/include` (warnings from headers in `/usr/include` are already
  suppressed by default) and those warnings stand in the way of compiling
  Bitcoin Core with `-Werror[=...]` or they just clutter the build output
  too much and make our own warnings hard to spot.

  `-isystem /usr/include` bricks GCC's `#include_next`, so we use
  `-idirafter` instead. This way we don't have to treat `/usr/include`
  specially.

ACKs for top commit:
  practicalswift:
    ACK ba8950ee01: diff looks correct!
  hebasto:
    ACK ba8950ee01, tested on Linux Mint 20 (x86_64).
  luke-jr:
    utACK ba8950ee01

Tree-SHA512: 9b54fae8590be6c79f2688a5aca09e0a9067f481dabecdd49bb278c08a62ac2b0cc704c894fbd53240e77ac84da0c7a237845df0a696cfbdb0359e1c8e2e10c9
2020-10-14 14:57:15 +02:00
Jon Atack
82fd40216c
refactor: promote some -netinfo localvars to class members 2020-10-14 14:56:07 +02:00
Jon Atack
5133fab37e
cli: simplify -netinfo using getpeerinfo network field 2020-10-14 14:56:05 +02:00
Jon Atack
4938a109ad
rpc, test: expose CNodeStats network in RPC getpeerinfo 2020-10-14 14:56:03 +02:00
Jon Atack
6df7882029
net: add peer network to CNodeStats 2020-10-14 14:55:55 +02:00
Wladimir J. van der Laan
9efa55c715
Merge #20130: Wallet: remove db mode string
135afa749c wallet: remove db mode string (Ivan Metlushko)

Pull request description:

  This is a [follow-up](https://github.com/bitcoin/bitcoin/pull/19077#discussion_r500261927) for #19077

  This PR simplifies DB interface by removing mode string from `WalletDatabase` and `WalletBatch`.

  The mode string was used to determine two flags for the instantiation of db connection:
  1) read-only flag. Never used on connection level. And on batch level Is only used within `BerkeleyDatabase::Rewrite` where it's replaced with bool flag.
  2) create flag. Is not required as we always check `require_existing` & `require_create` flags in `MakeDatabase()` before creating actual database instance. So we can safely default to always creating database if it doesn't exist yet.

ACKs for top commit:
  achow101:
    ACK 135afa749c
  laanwj:
    Code review ACK 135afa749c

Tree-SHA512: f49c07c7387c02e517a58199620a678a918f8dfc20d1347d29fd6adea0bc89698c26cb8eef42b0977961c11c207c4bbe109bc31059f47c126cc600b01fd987eb
2020-10-14 14:50:22 +02:00
MarcoFalke
3750f664b3
Merge #20129: tests: don't export in6addr_loopback
8e4d62280e tests: don't export in6addr_loopback (Vasil Dimov)

Pull request description:

  Don't export `in6addr_loopback` because that upsets
  `contrib/devtools/symbol-check.py`

  Fixes https://github.com/bitcoin/bitcoin/issues/20127

ACKs for top commit:
  sipa:
    utACK 8e4d62280e
  hebasto:
    ACK 8e4d62280e

Tree-SHA512: 216ffb53df55d2888317a81d18745308aaf93a3f3b45aa778166f7c91edb9741c28424d6333b35cefb5ece42b74e20ea21c761d93d8432798e7ec12097c2758f
2020-10-14 10:33:55 +02:00
MarcoFalke
fa1f6f237d
net: Send post-verack handshake messages at most once 2020-10-14 10:09:50 +02:00
fanquake
e21b824386
Merge #20109: Release notes and followups from 19339
88197b0769 [doc] release notes for max fee checking (gzhao408)
c201d73df3 style and nits for fee-checking in BroadcastTransaction (gzhao408)

Pull request description:

  Pretty trivial... addresses some tiny comments from #19339. Also fixes a docs typo from #19940 and adds a release note about the error message change for testmempoolaccept.

ACKs for top commit:
  achow101:
    ACK 88197b0769
  MarcoFalke:
    cr re-ACK 88197b0769

Tree-SHA512: fff16d731426b9b4db5222df02633983402f4c7241551eec98bb1554145dbdc132f40ed8ca4abd5edcebe1f4d1e879fb6d11bd91730604f6552c10cdf65706a1
2020-10-14 12:19:42 +08:00
Neha Narula
9c47cb29f9 [Rename only] Rename orphan_work_set to m_orphan_work_set.
This helps distinguish the member from any local variables.
2020-10-13 17:38:38 -04:00
MarcoFalke
3333077823
rpc: Adjust witness-tx deserialize error message 2020-10-13 14:57:52 +02:00
MarcoFalke
ec0453cd57
Merge bitcoin-core/gui#102: Fix SplashScreen crash when run with -disablewallet
c056064a4a gui: Fix SplashScreen crash when run with -disablewallet (Hennadii Stepanov)

Pull request description:

  This PR fixes the bug introduced in https://github.com/bitcoin/bitcoin/pull/19099:

  ```
  $ src/qt/bitcoin-qt -disablewallet
  bitcoin-qt: interfaces/node.cpp:236: auto interfaces::(anonymous namespace)::NodeImpl::walletClient()::(anonymous class)::operator()() const: Assertion `"m_context->wallet_client" && check' failed.
  Aborted (core dumped)
  ```

ACKs for top commit:
  Sjors:
    tACK c056064
  promag:
    ACK c056064a4a.

Tree-SHA512: 263d9efd5899cc6e447dfc5142bf911ca627149fac0a1c5e5b58dd196aa5e0d12fe13e3f750fb5f3c4338222f7959935d2f77391263f967dbca2e0e79a416a29
2020-10-13 14:42:12 +02:00
fanquake
ad90dd9f31
Merge #20141: Avoid the use of abs64 in timedata
d1292f25f2 Avoid the use of abs64 in timedata (Pieter Wuille)

Pull request description:

  Fixes #20135.

ACKs for top commit:
  kallewoof:
    ACK d1292f25f2
  jonatack:
    ACK d1292f25f2 code/logic review, verified there are no remaining callers of `abs64()`, verified no warnings in a debug build
  practicalswift:
    ACK d1292f25f2
  MarcoFalke:
    ACK d1292f25f2 🎹

Tree-SHA512: d17e95c668eb5e02ea546433b3d1b5a0ccbfb2c9cec62fa67dad1844d7e278a2576fbc0b75bddbf4db9af7331e978148c7bef7fce7e6a07e0eb917ef1392f302
2020-10-13 20:00:39 +08:00
Ivan Metlushko
135afa749c wallet: remove db mode string
We never need to open database in read-only mode as it's controlled
separately for every batch.

Also we can safely create database if it doesn't exist already
because require_existing option is verified in MakeDatabase
before creating a new WalletDatabase instance.
2020-10-13 18:42:59 +07:00
Pieter Wuille
d1292f25f2 Avoid the use of abs64 in timedata 2020-10-12 19:50:16 -07:00
Pieter Wuille
0e2a5e448f tests: dumping and minimizing of script assets data
This adds a --dumptests flag to the feature_taproot.py test, to dump all its
generated test cases to files, in a format compatible with the
script_assets_test unit test. A fuzzer for said format is added as well, whose
primary purpose is coverage-based minimization of those dumps.
2020-10-12 17:18:47 -07:00
Pieter Wuille
4567ba034c tests: add generic qa-asset-based script verification unit test
This adds a unit test that does generic script verification tests,
with positive/negative witnesses/scriptsigs, under various flags.
The test data is large (several MB) so it's stored in the qa-assets
repo.
2020-10-12 17:18:47 -07:00
Pieter Wuille
d7ff237f29 Activate Taproot/Tapscript on regtest (BIP 341, BIP 342)
Define a versionbits-based activation for the new consensus rules on regtest.
No activation or activation mechanism is defined for testnet or mainnet.
2020-10-12 17:18:47 -07:00
Pieter Wuille
e9a021d7e6 Make Taproot spends standard + policy limits
This adds a `TxoutType::WITNESS_V1_TAPROOT` for P2TR outputs, and permits spending
them in standardness rules. No corresponding `CTxDestination` is added for it,
as that isn't needed until we want wallet integration. The taproot validation flags
are also enabled for mempool transactions, and standardness rules are added
(stack item size limit, no annexes).
2020-10-12 17:18:47 -07:00
Johnson Lau
72422ce396 Implement Tapscript script validation rules (BIP 342)
This adds a new `SigVersion::TAPSCRIPT`, makes the necessary interpreter
changes to make it implement BIP342, and uses them for leaf version 0xc0
in Taproot script path spends.
2020-10-12 17:18:24 -07:00
Pieter Wuille
330de894a9 Use ScriptExecutionData to pass through annex hash
Instead of recomputing the annex hash every time a signature is verified, compute it
once and cache it in a new ScriptExecutionData structure.
2020-10-12 17:18:15 -07:00
Pieter Wuille
8bbed4b7ac Implement Taproot validation (BIP 341)
This includes key path spending and script path spending, but not the
Tapscript execution implementation (leaf 0xc0 remains unemcumbered in
this commit).

Includes constants for various aspects of the consensus rules suggested
by Jeremy Rubin.
2020-10-12 17:17:56 -07:00
Pieter Wuille
0664f5fe1f Support for Schnorr signatures and integration in SignatureCheckers (BIP 340)
This enables the schnorrsig module in libsecp256k1, adds the relevant types
and functions to src/pubkey, as well as in higher-level `SignatureChecker`
classes. The (verification side of the) BIP340 test vectors is also added.
2020-10-12 17:15:40 -07:00
Johnson Lau
5de246ca81 Implement Taproot signature hashing (BIP 341)
This implements the new sighashing scheme from BIP341, with all relevant
whole-transaction values precomputed once and cached.

Includes changes to PrecomputedTransactionData by Pieter Wuille.
2020-10-12 17:06:38 -07:00
Pieter Wuille
886be97af5 Ignore incorrectly-serialized banlist.dat entries 2020-10-12 15:35:08 -07:00
Pieter Wuille
883cea7dea Restore compatibility with old CSubNet serialization 2020-10-12 15:15:28 -07:00
Pieter Wuille
fd9a0060f0 Report and verify expirations 2020-10-12 12:14:53 -07:00
Pieter Wuille
86f50ed10f Delete limitedmap as it is unused now 2020-10-12 12:14:53 -07:00
Pieter Wuille
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents 2020-10-12 12:14:53 -07:00
Pieter Wuille
173a1d2d3f Expedite removal of tx requests that are no longer needed
Whenever a transaction is added to the mempool or orphan pool, both
its txid and wtxid are considered AlreadyHave, and thus will eventually
be removed from m_txrequest.

The same is true for hashes added to the reject filter, but note that sometimes
only the wtxid is added (in which case only the wtxid can be removed from
m_txrequest).
2020-10-12 12:14:53 -07:00
Pieter Wuille
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers
Maintaining up to 100000 INVs per peer is excessive, as that is far more
than fits in a typical mempool.

Also disable the "overload" penalty for PF_RELAY peers.
2020-10-12 12:14:53 -07:00
Pieter Wuille
242d16477d Change transaction request logic to use txrequest
This removes most transaction request logic from net_processing, and
replaces it with calls to a global TxRequestTracker object.

The major changes are:

* Announcements from outbound (and whitelisted) peers are now always
  preferred over those from inbound peers. This used to be the case for the
  first request (by delaying the first request from inbound peers), and
  a bias afters. The 2s delay for requests from inbound peers still exists,
  but after that, if viable outbound peers remain for any given transaction,
  they will always be tried first.

* No more hard cap of 100 in flight transactions per peer, as there is less
  need for it (memory usage is linear in the number of announcements, but
  independent from the number in flight, and CPU usage isn't affected by it).
  Furthermore, if only one peer announces a transaction, and it has over 100
  in flight and requestable already, we still want to request it from them.
  The cap is replaced with an additional 2s delay (possibly combined with the
  existing 2s delays for inbound connections, and for txid peers when wtxid
  peers are available).

Includes functional tests written by Marco Falke and Antoine Riard.
2020-10-12 12:14:11 -07:00
Pieter Wuille
5b03121d60 Add txrequest fuzz tests
This adds a fuzz test that reimplements a naive reimplementation of
TxRequestTracker (with up to 16 fixed peers and 16 fixed txhashes),
and compares the real implementation against it.
2020-10-12 12:08:47 -07:00
Pieter Wuille
3c7fe0e5a0 Add txrequest unit tests
Add unit tests for TxRequestTracker. Several scenarios are tested,
randomly interleaved with eachother.

Includes a test by Antoine Riard (ariard).
2020-10-12 12:08:43 -07:00
Pieter Wuille
da3b8fde03 Add txrequest module
This adds a new module (unused for now) which defines TxRequestTracker, a data
structure that maintains all information about transaction requests, and coordinates
requests.
2020-10-12 11:01:16 -07:00
Wladimir J. van der Laan
f79a4a8952
Merge #19998: net: Add CNode::ConnectedThroughNetwork member function
3984b78cd7 test: Add tests for CNode::ConnectedThroughNetwork (Hennadii Stepanov)
49fba9c1aa net: Add CNode::ConnectedThroughNetwork member function (Hennadii Stepanov)
d4dde24034 net: Add CNode::m_inbound_onion data member (Hennadii Stepanov)

Pull request description:

  This PR:
  - adds `CNode::ConnectedThroughNetwork` member function
  - is based on #19991, and only last two commits belong to it
  - is required for https://github.com/bitcoin-core/gui/pull/86 and #20002

ACKs for top commit:
  jonatack:
    re-ACK 3984b78cd7 per `git diff 3989fcf 3984b78c`
  laanwj:
    Code review ACK 3984b78cd7

Tree-SHA512: 23a9c8bca8dca75113b5505fe443b294f2d42d03c98c7e34919da12d8396beb8d0ada3a58ae16e3da04b7044395f72cf9c216625afc078256cd6c897ac42bf3d
2020-10-12 18:40:45 +02:00
Vasil Dimov
ba8950ee01
build: optionally skip external warnings
Add an option to `./configure` to suppress compilation warnings from
external headers. The option is off by default (no change in behavior,
show warnings from external headers).

This option is useful if e.g. Boost or Qt is installed outside of
`/usr/include` (warnings from headers in `/usr/include` are already
suppressed by default) and those warnings stand in the way of compiling
Bitcoin Core with `-Werror[=...]` or they just clutter the build output
too much and make our own warnings hard to spot.
2020-10-12 18:18:24 +02:00
MarcoFalke
faad92fe1c
test: Remove unused nVersion=1 in p2p tests
After commit ddefb5c0b7 nVersion is no
longer used in p2p logic when sending messages. Only when receiving
messages, but in this test no messages are received.
2020-10-12 13:11:26 +02:00
Pieter Wuille
9eb590894f Add TaggedHash function (BIP 340)
This adds the TaggedHash function as defined by BIP340 to the hash module, which
is used in BIP340 and BIP341 to produce domain-separated hashes.
2020-10-12 02:06:32 -07:00
Pieter Wuille
5d62e3a68b refactor: keep spent outputs in PrecomputedTransactionData
A BIP-341 signature message may commit to the scriptPubKeys and amounts
of all spent outputs (including other ones than the input being signed
for spends), so keep them available to signature hashing code.
2020-10-12 02:06:32 -07:00
Vasil Dimov
8e4d62280e
tests: don't export in6addr_loopback
Don't export `in6addr_loopback` because that upsets
`contrib/devtools/symbol-check.py`

Fixes https://github.com/bitcoin/bitcoin/issues/20127
2020-10-12 10:25:08 +02:00
Pieter Wuille
56f9dba015 Only relay IPv4, IPv6, Tor addresses 2020-10-11 11:29:11 -07:00
fanquake
0b2abaa666
Merge #19954: Complete the BIP155 implementation and upgrade to TORv3
dcf0cb4776 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d9 net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fd Support bypassing range check in ReadCompactSize (Pieter Wuille)

Pull request description:

  This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:

  `net: CAddress & CAddrMan: (un)serialize as ADDRv2`
  `net: advertise support for ADDRv2 via new message`

  plus one more commit:

  `tor: make a TORv3 hidden service instead of TORv2`

ACKs for top commit:
  jonatack:
    re-ACK dcf0cb4776 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
  sipa:
    ACK dcf0cb4776
  hebasto:
    re-ACK dcf0cb4776, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
  laanwj:
    Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb4776 merged on master (12a1c3ad1a).
  ariard:
    Code Review ACK dcf0cb4

Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
2020-10-11 08:51:57 +08:00
Amiti Uttarwar
41dca087b7 [trivial] Extract connection type doc into file where it is used.
This slightly reduces the size of the binary.
2020-10-09 16:09:51 -07:00
Amiti Uttarwar
3069b56a45 [doc] Improve help for getpeerinfo connection_type field. 2020-10-09 16:08:02 -07:00
Hennadii Stepanov
c056064a4a
gui: Fix SplashScreen crash when run with -disablewallet 2020-10-09 21:14:47 +03:00
Vasil Dimov
dcf0cb4776
tor: make a TORv3 hidden service instead of TORv2
TORv2 is deprecated [1], thus whenever we create the hidden service
ourselves create a TORv3 one instead.

[1] https://blog.torproject.org/v2-deprecation-timeline
2020-10-09 16:42:50 +02:00
Vasil Dimov
353a3fdaad
net: advertise support for ADDRv2 via new message
Introduce a new message `sendaddrv2` to signal support for ADDRv2.
Send the new message immediately after sending the `VERACK` message.

Add support for receiving and parsing ADDRv2 messages.

Send ADDRv2 messages (instead of ADDR) to a peer if he has
advertised support for it.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:50 +02:00
Vasil Dimov
201a4596d9
net: CAddress & CAddrMan: (un)serialize as ADDRv2
Change the serialization of `CAddrMan` to serialize its addresses
in ADDRv2/BIP155 format by default. Introduce a new `CAddrMan` format
version (3).

Add support for ADDRv2 format in `CAddress` (un)serialization.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:49 +02:00
Hennadii Stepanov
0a85e5a7bc
p2p: Try to connect to anchors once 2020-10-09 14:29:05 +03:00
Hennadii Stepanov
5543c7ab28
p2p: Fix off-by-one error in fetching address loop
This is a move-only commit.
2020-10-09 14:29:05 +03:00
Hennadii Stepanov
4170b46544
p2p: Integrate DumpAnchors() and ReadAnchors() into CConnman 2020-10-09 14:29:05 +03:00
Hennadii Stepanov
bad16aff49
p2p: Add CConnman::GetCurrentBlockRelayOnlyConns() 2020-10-09 14:29:04 +03:00
Hennadii Stepanov
c29272a157
p2p: Add ReadAnchors() 2020-10-09 14:29:04 +03:00
Hennadii Stepanov
567008d2a0
p2p: Add DumpAnchors() 2020-10-09 14:28:58 +03:00
Pieter Wuille
1d3ec2a1fd
Support bypassing range check in ReadCompactSize
This is needed when we want to encode an arbitrary number as CompactSize
like node service flags, which is a bitmask and could be bigger than the
usual size of an object.
2020-10-09 10:32:19 +02:00
gzhao408
c201d73df3 style and nits for fee-checking in BroadcastTransaction 2020-10-08 14:11:16 -07:00
Wladimir J. van der Laan
d9de00b3e0
Merge #20105: [net] Remove CombinerAll
1afcd41a90 [net] Remove CombinerAll (John Newbery)

Pull request description:

  This was introduced in 9519a9a4 for use with boost signals. Boost signals
  have not been used in net since 8ad663c1, so this code is unused.

ACKs for top commit:
  MarcoFalke:
    review ACK 1afcd41a90
  laanwj:
    code review ACK 1afcd41a90

Tree-SHA512: a4313142afb88bf12f15abc4e717b3b0d0b40d2d5db2638494af3181e1cd680d7b036087050fc0e0dfe606228849a2e20ae85135908a9ebe8ff2130f163920e1
2020-10-08 20:42:19 +02:00
MarcoFalke
392c6f4fb2
Merge #20101: rpc: change no wallet loaded message to be clearer
907f142fc7 rpc: change no wallet loaded message to be clearer (Andrew Chow)

Pull request description:

  Changes the no wallet is loaded rpc error message to be clearer that no wallet is loaded and how the user can load or create a wallet. Also changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as that makes more sense.

ACKs for top commit:
  MarcoFalke:
    review ACK 907f142fc7
  kristapsk:
    ACK 907f142fc7. In addition to standard tests, just in case tested that this doesn't break anything with JoinMarket.
  meshcollider:
    utACK 907f142fc7

Tree-SHA512: 4b413e6ab5430ec75a79de9db6583f2f3f38ccdf71aa373d8386a56e64f07f92200c8107c8c82c92c7c431d739615977c208b771a24c5960fa8676789b5497a2
2020-10-08 15:07:23 +02:00
John Newbery
1afcd41a90 [net] Remove CombinerAll
This was introduced in 9519a9a4 for use with boost signals. Boost signals
have not been used in net since 8ad663c1, so this code is unused
2020-10-08 12:28:38 +01:00
MarcoFalke
b337bd7bc0
Merge #20065: fuzz: Configure check for main function
fae7a1c188 fuzz: Configure check for main function (MarcoFalke)

Pull request description:

  Instead of the PP jungle, use a proper configure check

  Fixes https://github.com/google/honggfuzz/issues/336#issuecomment-702972138

ACKs for top commit:
  practicalswift:
    ACK fae7a1c188

Tree-SHA512: 2e55457d01f9ac598bb1e119d8b49dca55a28f88ec164cee6b5f071c29e9791f5a46cc8ee2b801b3a3faf906348da964ce32e7254da981c1104b9210a3508100
2020-10-08 13:17:45 +02:00
Pieter Wuille
b6834e312a Avoid 'timing mishap' warnings when mocking 2020-10-08 01:04:29 -07:00
Pieter Wuille
ec3916f40a Use mockable time everywhere in net_processing 2020-10-08 01:04:29 -07:00
Andrew Chow
907f142fc7 rpc: change no wallet loaded message to be clearer
Changes the no wallet is loaded rpc error message to be clearer that no
wallet is loaded and how the user can load or create a wallet. Also
changes the error code from METHOD_NOT_FOUND to RPC_WALLET_NOT_FOUND as
that makes more sense.
2020-10-07 21:36:44 -04:00
Ethan Heilman
bd5215103e random: fixes read buffer resizing in RandAddSeedPerfmon
+ Replaces std::max with std::min to resize buffer in RandAddSeedPerfmon
+ Documents behavior of RandAddSeedPerfmon
2020-10-07 20:46:49 -04:00
MarcoFalke
283a73d7ea
Merge #20092: util: Do not use gArgs global in ArgsManager member functions
d103484fe8 util: Do not use gArgs global in ArgsManager member functions (Hennadii Stepanov)

Pull request description:

ACKs for top commit:
  practicalswift:
    ACK d103484fe8: patch looks correct

Tree-SHA512: dda7a5062363170c6995f2fd8fda48c0a919e5ca67be9faa8f0fa66f9d3b535f134eb6f4860a0859bc5457c02230b34a8d1264045f22bed8d30668158ac2271f
2020-10-07 09:31:12 +02:00
fanquake
db88db4727
Merge #19339: validation: re-delegate absurd fee checking from mempool to clients
b048b275d9 [validation] Remove absurdfee from accepttomempool (John Newbery)
932564b9cf scripted-diff: update max-fee-exceeded error message to include RPC (gzhao408)
8f1290c601 [rpc/node] check for high fee before ATMP in clients (gzhao408)

Pull request description:

  Picked up from #15810. Add separate fee-checking logic for clients that need to enforce max fee rates, then remove the `absurdFee` logic from ATMP.

  ATMP's `nAbsurdFee` argument is used to enforce user-specific behavior (it is not policy since it isn't applied consistently: it is only ever used in RPC and wallet, and set to 0 everywhere else internally). It should be removed from `AcceptToMemoryPool` because (1) validation results/mempool behavior should not be user-specific and (2) enforcing a max fee rate should be the responsibility of the client instead of the mempool.

  Note: this PR does not intend to _remove_ protection from high fees, just re-delegate the responsibility to clients.

ACKs for top commit:
  jnewbery:
    utACK b048b275d9
  LarryRuane:
    re-ACK b048b275d9
  MarcoFalke:
    re-ACK b048b275d9 , only change is squashing one commit 🏦
  instagibbs:
    utACK b048b275d9

Tree-SHA512: 57c17ba16d230a4cae2896dd6a64c924f307757824e35784bf96da7b10aff2d8ea910710edf35e981035623a155f8766209a92a0fdb856549fde78bc3eaae4d2
2020-10-07 10:58:30 +08:00
Pieter Wuille
8bd2b4e784 refactor: rename scriptPubKey in VerifyWitnessProgram to exec_script
The old name is confusing, as it doesn't store a scriptPubKey, but the
actually executed script.
2020-10-06 14:51:30 -07:00
Pieter Wuille
107b57df9f scripted-diff: put ECDSA in name of signature functions
In preparation for adding Schnorr versions of `CheckSig`, `VerifySignature`, and
`ComputeEntry`, give them an ECDSA specific name.

-BEGIN VERIFY SCRIPT-
sed -i 's/CheckSig(/CheckECDSASignature(/g' $(git grep -l CheckSig ./src)
sed -i 's/VerifySignature(/VerifyECDSASignature(/g' $(git grep -l VerifySignature ./src)
sed -i 's/ComputeEntry(/ComputeEntryECDSA(/g' $(git grep -l ComputeEntry ./src)
-END VERIFY SCRIPT-
2020-10-06 14:51:30 -07:00
Hennadii Stepanov
d103484fe8
util: Do not use gArgs global in ArgsManager member functions 2020-10-06 14:57:06 +03:00
John Newbery
b048b275d9 [validation] Remove absurdfee from accepttomempool
Mempool behavior should not be user-specific.
Checking that txfee is acceptable should be
the responsibility of the wallet or client, not
the mempool.
2020-10-05 04:55:01 -07:00
gzhao408
932564b9cf scripted-diff: update max-fee-exceeded error message to include RPC
-BEGIN VERIFY SCRIPT-
sed -i 's/Fee exceeds maximum configured by \-\maxtxfee/Fee exceeds maximum configured by user (e.g. -maxtxfee, maxfeerate)/g' src/util/error.cpp test/functional/rpc_fundrawtransaction.py test/functional/rpc_psbt.py test/functional/rpc_rawtransaction.py test/functional/wallet_bumpfee.py test/functional/wallet_create_tx.py
-END VERIFY SCRIPT-
2020-10-05 04:55:01 -07:00
gzhao408
8f1290c601 [rpc/node] check for high fee before ATMP in clients
Check absurd fee in BroadcastTransaction and RPC,
return TransactionError::MAX_FEE_EXCEEDED instead
of TxValidationResult::TX_NOT_STANDARD because this
is client preference, not a node-wide policy.
2020-10-05 04:54:05 -07:00
fanquake
dde104963b
Merge #20064: RPC: remove duplicate line in getblock help
1885ad3546 RPC: remove duplicate line in getblock help (Fabian Jahr)

Pull request description:

  Line simply seems duplicated in error.

  Testing instructions:
  Run `src/bitcoin-cli help getblock` on master branch to reproduce. Then build this PR and compare its results.

ACKs for top commit:
  dhruv:
    tACK `1885ad3`
  kristapsk:
    ACK 1885ad3546
  Emzy:
    tACK 1885ad3546

Tree-SHA512: 870c035cb553b0e1d5ef72e64231ef277e0392efe94bc6ecf47129023bd94a6d5a276f46529807f68a1db55c7baa94d9119c7264d9947bc4e5dd9dcefd1b13e7
2020-10-05 09:40:17 +08:00
MarcoFalke
fae7a1c188
fuzz: Configure check for main function 2020-10-04 17:49:07 +02:00
MarcoFalke
cce1513179
Merge #19723: Ignore unknown messages before VERACK
675e55e013 Ignore unknown messages before VERACK (Suhas Daftuar)

Pull request description:

  This allows for feature negotiation to take place with messages between VERSION and VERACK in the future, without requiring additional software changes to specifically ignore messages for features that are unimplemented by our software.

ACKs for top commit:
  sipa:
    utACK 675e55e013
  practicalswift:
    ACK 675e55e013: patch looks correct
  MarcoFalke:
    ACK 675e55e013
  hebasto:
    ACK 675e55e013, the offender peer will be eventually disconnected due to the timeout.

Tree-SHA512: 8d2b1d8b9843f2ee26b2c30f7c5ff0bfcfbe3f46b32cd0369c48ece26624151091237e83ce3f18c6da004099026602cfab1642ac916db777f047d170b365c007
2020-10-04 15:39:26 +02:00
Hennadii Stepanov
3984b78cd7
test: Add tests for CNode::ConnectedThroughNetwork 2020-10-03 15:38:49 +03:00
Hennadii Stepanov
49fba9c1aa
net: Add CNode::ConnectedThroughNetwork member function 2020-10-03 15:38:19 +03:00
Hennadii Stepanov
d4dde24034
net: Add CNode::m_inbound_onion data member 2020-10-03 13:56:19 +03:00
fanquake
54fc96ffa7
Merge #19956: rpc: Improve invalid vout value rpc error message
f471a3be00 scripted diff: Improve invalid vout value rpc error message (Nima Yazdanmehr)

Pull request description:

  Since the `vout` value can start at `0`, the error message for *negative* values can be improved to something like: `vout cannot be negative`.

ACKs for top commit:
  fanquake:
    ACK f471a3be00
  promag:
    Code review ACK f471a3be00.

Tree-SHA512: fbdee3d0ddd5b58eb93934a1217b44e125a9ad39e672b1f35c7609c6c5fcf45ae1b731d3d6135b7225d98792dbfc34a50907b8c41274a5b029d7b5c59f886560
2020-10-03 11:13:21 +08:00
Fabian Jahr
1885ad3546
RPC: remove duplicate line in getblock help 2020-10-03 00:44:59 +02:00
Wladimir J. van der Laan
d9935222d5
Merge #19951: net, test: CNetAddr scoped ipv6 test coverage, rename scopeId to m_scope_id
f36887fa47 net: rename CNetAddr scopeId to m_scope_id, improve code doc (Jon Atack)
5cb5fd3005 test: add test coverage for CNetAddr ipv6 scoped addresses (Jon Atack)

Pull request description:

  Add some test coverage for the IPv6 scoped address feature in `netaddress`/`netbase` per https://tools.ietf.org/html/rfc4007, update the member name `scopeId` to `m_scope_id` per `doc/developer-notes.md`, and improve the code doc.

  ----

  Reviewers can manually verify the test with these steps:

  - [pull down the changes locally and check out this branch](https://jonatack.github.io/articles/how-to-review-pull-requests-in-bitcoin-core#pull-down-the-code-locally)

  - [build Bitcoin Core](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests)

  - run the test: `cd src ; test/test_bitcoin -t net_tests/cnetaddr_basic -l all`

  <details><summary><em>you should see passing test output like this</em></summary><p>

  ```
  Running 1 test case...
  Entering test module "Bitcoin Core Test Suite"
  test/net_tests.cpp(85): Entering test suite "net_tests"
  test/net_tests.cpp(204): Entering test case "cnetaddr_basic"
  test/net_tests.cpp(210): info: check LookupHost("0.0.0.0", addr, false) has passed
  test/net_tests.cpp(211): info: check !addr.IsValid() has passed
  test/net_tests.cpp(212): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(214): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(215): info: check addr.ToString() == "0.0.0.0" has passed
  test/net_tests.cpp(218): info: check LookupHost("255.255.255.255", addr, false) has passed
  test/net_tests.cpp(219): info: check !addr.IsValid() has passed
  test/net_tests.cpp(220): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(222): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(223): info: check addr.ToString() == "255.255.255.255" has passed
  test/net_tests.cpp(226): info: check LookupHost("12.34.56.78", addr, false) has passed
  test/net_tests.cpp(227): info: check addr.IsValid() has passed
  test/net_tests.cpp(228): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(230): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(231): info: check addr.ToString() == "12.34.56.78" has passed
  test/net_tests.cpp(234): info: check LookupHost("::", addr, false) has passed
  test/net_tests.cpp(235): info: check !addr.IsValid() has passed
  test/net_tests.cpp(236): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(238): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(239): info: check addr.ToString() == "::" has passed
  test/net_tests.cpp(242): info: check LookupHost(ipv6_addr, addr, false) has passed
  test/net_tests.cpp(243): info: check addr.IsValid() has passed
  test/net_tests.cpp(244): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(246): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(247): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): info: check addr.ToString() == scoped_addr has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(266): info: check LookupHost(ipv6_addr + "%0", addr, false) has passed
  test/net_tests.cpp(267): info: check addr.IsValid() has passed
  test/net_tests.cpp(268): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(269): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(270): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(274): info: check addr.IsValid() has passed
  test/net_tests.cpp(275): info: check addr.IsTor() has passed
  test/net_tests.cpp(277): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(278): info: check addr.ToString() == "6hzph5hv6337r6p2.onion" has passed
  test/net_tests.cpp(282): info: check !addr.IsValid() has passed
  test/net_tests.cpp(283): info: check addr.IsInternal() has passed
  test/net_tests.cpp(285): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(286): info: check addr.ToString() == "esffpvrt3wpeaygy.internal" has passed
  test/net_tests.cpp(204): Leaving test case "cnetaddr_basic"; testing time: 30933us
  test/net_tests.cpp(85): Leaving test suite "net_tests"; testing time: 30975us
  Leaving test module "Bitcoin Core Test Suite"; testing time: 31169us

  *** No errors detected
  ```
  </p>/</details>

  - change this line in the code to break the feature:
  ```diff
  --- a/src/netaddress.cpp
  +++ b/src/netaddress.cpp
  @@ -716,7 +716,7 @@ bool CService::GetSockAddr(struct sockaddr* paddr, socklen_t *addrlen) const
           memset(paddrin6, 0, *addrlen);
           if (!GetIn6Addr(&paddrin6->sin6_addr))
               return false;
  -        paddrin6->sin6_scope_id = m_scope_id;
  +        // paddrin6->sin6_scope_id = m_scope_id;
  ```

  - rebuild, e.g. `make`

  - run the test: `test/test_bitcoin -t net_tests/cnetaddr_basic -l all`, verify that the added tests break

  <details><summary><em>you should see test output with a few failed tests like this</em></summary><p>

  ```
  Running 1 test case...
  Entering test module "Bitcoin Core Test Suite"
  test/net_tests.cpp(85): Entering test suite "net_tests"
  test/net_tests.cpp(204): Entering test case "cnetaddr_basic"
  test/net_tests.cpp(210): info: check LookupHost("0.0.0.0", addr, false) has passed
  test/net_tests.cpp(211): info: check !addr.IsValid() has passed
  test/net_tests.cpp(212): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(214): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(215): info: check addr.ToString() == "0.0.0.0" has passed
  test/net_tests.cpp(218): info: check LookupHost("255.255.255.255", addr, false) has passed
  test/net_tests.cpp(219): info: check !addr.IsValid() has passed
  test/net_tests.cpp(220): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(222): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(223): info: check addr.ToString() == "255.255.255.255" has passed
  test/net_tests.cpp(226): info: check LookupHost("12.34.56.78", addr, false) has passed
  test/net_tests.cpp(227): info: check addr.IsValid() has passed
  test/net_tests.cpp(228): info: check addr.IsIPv4() has passed
  test/net_tests.cpp(230): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(231): info: check addr.ToString() == "12.34.56.78" has passed
  test/net_tests.cpp(234): info: check LookupHost("::", addr, false) has passed
  test/net_tests.cpp(235): info: check !addr.IsValid() has passed
  test/net_tests.cpp(236): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(238): info: check addr.IsBindAny() has passed
  test/net_tests.cpp(239): info: check addr.ToString() == "::" has passed
  test/net_tests.cpp(242): info: check LookupHost(ipv6_addr, addr, false) has passed
  test/net_tests.cpp(243): info: check addr.IsValid() has passed
  test/net_tests.cpp(244): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(246): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(247): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%1]
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%21]
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%19]
  test/net_tests.cpp(254): info: check LookupHost(scoped_addr, addr, false) has passed
  test/net_tests.cpp(255): info: check addr.IsValid() has passed
  test/net_tests.cpp(256): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(257): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(258): error: in "net_tests/cnetaddr_basic": check addr.ToString() == scoped_addr has failed [1122:3344:5566:7788:9900:aabb:ccdd:eeff != 1122:3344:5566:7788:9900:aabb:ccdd:eeff%3]
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(263): info: check !LookupHost(ipv6_addr + "%" + zone_id, addr, false) has passed
  test/net_tests.cpp(266): info: check LookupHost(ipv6_addr + "%0", addr, false) has passed
  test/net_tests.cpp(267): info: check addr.IsValid() has passed
  test/net_tests.cpp(268): info: check addr.IsIPv6() has passed
  test/net_tests.cpp(269): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(270): info: check addr.ToString() == ipv6_addr has passed
  test/net_tests.cpp(274): info: check addr.IsValid() has passed
  test/net_tests.cpp(275): info: check addr.IsTor() has passed
  test/net_tests.cpp(277): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(278): info: check addr.ToString() == "6hzph5hv6337r6p2.onion" has passed
  test/net_tests.cpp(282): info: check !addr.IsValid() has passed
  test/net_tests.cpp(283): info: check addr.IsInternal() has passed
  test/net_tests.cpp(285): info: check !addr.IsBindAny() has passed
  test/net_tests.cpp(286): info: check addr.ToString() == "esffpvrt3wpeaygy.internal" has passed
  test/net_tests.cpp(204): Leaving test case "cnetaddr_basic"; testing time: 32316us
  test/net_tests.cpp(85): Leaving test suite "net_tests"; testing time: 32354us
  Leaving test module "Bitcoin Core Test Suite"; testing time: 32522us

  *** 4 failures are detected in the test module "Bitcoin Core Test Suite"
  ```
  </p></details>

  - leave your review here

ACKs for top commit:
  laanwj:
    ACK f36887fa47

Tree-SHA512: 8e77e512db130642be7d3a910735ca803a2afdb5a704f713f163f8b5a80be3077a2be6f0a3ca43d299731dcd2545ac35571f8c74e5250a72a48233c26f9a3ab5
2020-10-02 17:06:59 +02:00
Wladimir J. van der Laan
597488d37c
Merge #19871: doc: Clarify scope of eviction protection of outbound block-relay peers
d76925478e [doc] Clarify semantic of peer's m_protect w.r.t to outbound eviction logics (Antoine Riard)
ac71fe936d [doc] Clarify scope of eviction protection of outbound block-relay peers (Antoine Riard)

Pull request description:

  Block-relay-only peers were introduced by #15759. According to its
  author, it was intented to make them only immune to outbound peer
  rotation-based eviction and not from all eviction as modified comment
  leans to think of.

  Clearly indicate that outbound block-relay peers aren't protected
  from eviction by the bad/lagging chain logic.

  Fix #19863

ACKs for top commit:
  naumenkogs:
    ACK d76925478e
  jonatack:
    ACK d76925478e

Tree-SHA512: 597fbd62838a6e39276024165b11514cad20a2e9d33cf9202d261cbadcb62b2df427c858e0cb57e585840d4c1d4600104aa53916bb868541f2580e4eed9b4b52
2020-10-02 16:42:38 +02:00
Jon Atack
f36887fa47
net: rename CNetAddr scopeId to m_scope_id, improve code doc 2020-10-02 16:42:02 +02:00
Jon Atack
5cb5fd3005
test: add test coverage for CNetAddr ipv6 scoped addresses 2020-10-02 16:40:15 +02:00
Wladimir J. van der Laan
df2129a234
Merge #19991: net: Use alternative port for incoming Tor connections
96571b3d4c doc: Update onion service target port numbers in tor.md (Hennadii Stepanov)
bb145c9050 net: Extend -bind config option with optional network type (Hennadii Stepanov)
92bd3c1da4 net, refactor: Move AddLocal call one level up (Hennadii Stepanov)
57f17e57c8 net: Pass onion service target to Tor controller (Hennadii Stepanov)
e3f07851f0 refactor: Rename TorController::target to m_tor_control_center (Hennadii Stepanov)
fdd3ae4d26 net, refactor: Refactor CBaseChainParams::RPCPort function (Hennadii Stepanov)
a5266d4546 net: Add alternative port for onion service (Hennadii Stepanov)
b3273cf403 net: Use network byte order for in_addr.s_addr (Hennadii Stepanov)

Pull request description:

  This PR adds ability to label incoming Tor connections as different from normal localhost connections.

  Closes #8973.
  Closes #16693.

  Default onion service target ports are:
  - 8334 on mainnnet
  - 18334 on testnet
  - 38334 on signet
  - 18445 on regtest

  To set the onion service target socket manually the extended `-bind` config option could be used:

  ```
  $ src/bitcoind -help | grep -A 6 -e '-bind'
    -bind=<addr>[:<port>][=onion]
         Bind to given address and always listen on it (default: 0.0.0.0). Use
         [host]:port notation for IPv6. Append =onion to tag any incoming
         connections to that address and port as incoming Tor connections
         (default: 127.0.0.1:8334=onion, testnet: 127.0.0.1:18334=onion,
         signet: 127.0.0.1:38334=onion, regtest: 127.0.0.1:18445=onion)

  ```

  Since [pr19991.02 update](https://github.com/bitcoin/bitcoin/pull/19991#issuecomment-698882284) this PR is an alternative to #19043.

ACKs for top commit:
  Sjors:
    re-utACK 96571b3d4c
  vasild:
    ACK 96571b3d4
  laanwj:
    Re-ACK 96571b3d4c

Tree-SHA512: cb0eade80f4b3395f405f775e1b89c086a1f09d5a4464df6cb4faf808d9c2245474e1720b2b538f203f6c1996507f69b09f5a6e35ea42633c10e22bd733d4438
2020-10-02 13:37:23 +02:00
fanquake
60ec57b3d7
Merge #20055: rpc: Set HTTP Content-Type in bitcoin-cli
7eab781a14 rpc: Set HTTP Content-Type in bitcoin-cli (Wladimir J. van der Laan)

Pull request description:

  We don't set any `Content-Type` in the client. It is more consistent with our other JSON-RPC use to set it to `application/json`.

  Note that our server doesn't enforce content types, so it doesn't make a difference in practice. But it is fairly strange HTTP behavior to not set it at all for a POST request.

  This came up in #18950.

ACKs for top commit:
  promag:
    ACK 7eab781a14.
  jonatack:
    Tested ACK 7eab781a14
  practicalswift:
    ACK 7eab781a14: patch looks correct
  fanquake:
    ACK 7eab781a14 - Looks fine to me.

Tree-SHA512: a9fa155324d0f7bff955585a336ead6bb60b721039f424521a435e4bb0fad3f4532e5cc7b7a9acc4e93585e8d3db3082c010138810f22c0e92b8f749b86ef653
2020-10-02 15:25:20 +08:00
fanquake
82b70f15c7
refactor: fix -Wbraced-scalar-init warning in validation tests
Introduced in #20004 (fa29b5ae66).

```bash
test/validation_tests.cpp:68:88: warning: braces around scalar initializer [-Wbraced-scalar-init]
    BOOST_CHECK(signet_params->GetConsensus().signet_challenge == std::vector<uint8_t>{{OP_TRUE}});
                                                                                       ^~~~~~~~~
/usr/local/include/boost/test/tools/old/interface.hpp:83:6: note: expanded from macro 'BOOST_CHECK'
    (P), BOOST_TEST_STRINGIZE( P ), CHECK, CHECK_PRED, _ )
     ^
/usr/local/include/boost/test/tools/old/interface.hpp:68:61: note: expanded from macro 'BOOST_TEST_TOOL_IMPL'
        BOOST_JOIN( BOOST_TEST_TOOL_PASS_PRED, frwd_type )( P, ARGS ),          \
                                                            ^
/usr/local/include/boost/test/tools/old/interface.hpp:51:47: note: expanded from macro 'BOOST_TEST_TOOL_PASS_PRED2'
                                              ^
1 warning generated.
```
2020-10-02 12:30:50 +08:00
Wladimir J. van der Laan
7532c9a221 qt: Periodic translations update 2020-10-01 22:24:08 +02:00
Hennadii Stepanov
bb145c9050
net: Extend -bind config option with optional network type 2020-10-01 19:19:20 +03:00