Commit graph

1942 commits

Author SHA1 Message Date
Sawyer Billings
e8640849c7
doc: Use https URLs where possible 2021-01-04 12:23:16 +08:00
Jon Atack
ee701a9204
doc: update developer notes for signet 2021-01-02 17:31:30 +01:00
Prayank
7117d7503f Update 'Secure string handling'
Add information about possible path traversal attack with example
2020-12-29 01:49:30 +05:30
Amiti Uttarwar
454a4088a8 [doc] Add release notes for removed getpeerinfo fields. 2020-12-26 13:30:54 -08:00
MarcoFalke
9b28bd73a3
Merge #20691: ci, doc: Travis CI features and mentions cleanup
95487b0553 doc: Drop mentions of Travis CI as it is no longer used (Hennadii Stepanov)
09d105ef0f ci: Drop travis_fold feature as Travis CI is no longer used (Hennadii Stepanov)

Pull request description:

  As Travis CI is no longer used, this PR:
  - drops `travis_fold` feature
  - drops mentions of Travis CI in docs

ACKs for top commit:
  MarcoFalke:
    ACK 95487b0553

Tree-SHA512: 2e259bb8b1e37bcefc1251737bb2716f06ddb57c490010b373825c4e70f42ca38efae69a2f63f21f577d7cee3725b94097bdddbd313f8ebf499281cf97c53cef
2020-12-18 07:32:28 +01:00
Hennadii Stepanov
95487b0553
doc: Drop mentions of Travis CI as it is no longer used 2020-12-18 01:15:53 +02:00
Wladimir J. van der Laan
7ef6b1c51d
Merge #19961: doc: tor.md updates
a34eceb4cc doc: update -externalip documentation in tor.md (Jon Atack)
dc8a591222 doc: add tor.md section on how to get tor info via bitcoind (Jon Atack)
e1765d8b04 doc: update tor.md address examples from onion v2 to v3 (Jon Atack)

Pull request description:

  It looks like `doc/tor.md` could use some updates and improvements, not only for Tor v3, but also for setting multiple addresses with `-externalip` (see the conversation from http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-16.html#l-39), how to see information about your Tor config via Bitcoin Core, and other improvements.

  Closes #19924.

ACKs for top commit:
  laanwj:
    ACK a34eceb4cc

Tree-SHA512: 3197cdca1188dbd645c8f9e6ed7c023da5ad9bcf246a6bcbfbe6078f40c01c563032b4906736cde253a2daf71aaed28a659121628891a5d0bf6e89f821a17a28
2020-12-17 11:40:00 +01:00
MarcoFalke
8452f922d2
Merge #19050: doc: Add warning for rest interface limitation
5c3eaf9983 doc: Add warnings for http interfaces limitations (Fabian Jahr)

Pull request description:

  `libevent`, which is used for our rest interface, can use up all of the available file descriptors in a system if too many connections are opened at once. If a new block is connected at the same time and can not be written to disk because there are no file descriptors available, the node crashes. Based on my investigation so far the issue is best solved upstream which means we have to wait for the next release (2.2). In the meantime it would be good if we would warn users of this limitation.

  See #11368 for more background.

ACKs for top commit:
  MarcoFalke:
    ACK 5c3eaf9983

Tree-SHA512: 73914538588477ead19068f5832fdcc8e0eb736e51f73b3aca501c93165e5ad634c2511a3fcffff251adcd3efda23a742b48211ad9d3b2a29cdeac17201d06a1
2020-12-17 10:00:38 +01:00
Jon Atack
a34eceb4cc
doc: update -externalip documentation in tor.md 2020-12-16 22:58:14 +01:00
Jon Atack
dc8a591222
doc: add tor.md section on how to get tor info via bitcoind 2020-12-16 22:58:12 +01:00
Jon Atack
e1765d8b04
doc: update tor.md address examples from onion v2 to v3 2020-12-16 22:58:10 +01:00
Wladimir J. van der Laan
2d4eeaf9c5
Merge #20601: doc: Update for FreeBSD 12.2, add GUI Build Instructions
c175690561 doc: Update for FreeBSD 12.2, add GUI Build Instructions (Jarol Rodriguez)

Pull request description:

  The current FreeBSD Build documentation is a little outdated and underwhelming. This PR intends to keep the build-freebsd.md doc up to date. Here are the main improvements:
  - Introduce dependency information
  - New instructions for building the GUI
  - Instructions for supporting descriptor wallets
  - Various notes on the build and compile process

  **Before/Master:** [render](https://github.com/bitcoin/bitcoin/blob/master/doc/build-freebsd.md)

  **After/PR:** [render](2e8b9a5aac/doc/build-freebsd.md)

ACKs for top commit:
  laanwj:
    utACK c175690561

Tree-SHA512: 64aeb743c7f4ed167451454564b53d13c5c30d82bd2423799655c7b5b465a75733072fb0c574927c2588a46e89a68c6c57b008220acfd25336d445c9dc309f3b
2020-12-16 15:39:28 +01:00
MarcoFalke
8bb40d5f56
Merge #20560: fuzz: Link all targets once
fa13e1b0c5 build: Add option --enable-danger-fuzz-link-all (MarcoFalke)
44444ba759 fuzz: Link all targets once (MarcoFalke)

Pull request description:

  Currently the linker is invoked more than 150 times when compiling with `--enable-fuzz`. This is problematic for several reasons:

  * It wastes disk space north of 20 GB, as all libraries and sanitizers are linked more than 150 times
  * It wastes CPU time, as the link step can practically not be cached (similar to ccache for object files)
  * It makes it a blocker to compile the fuzz tests by default for non-fuzz builds #19388, for the aforementioned reasons
  * The build file is several thousand lines of code, without doing anything meaningful except listing each fuzz target in a highly verbose manner
  * It makes writing new fuzz tests unnecessarily hard, as build system knowledge is required; Compare that to boost unit tests, which can be added by simply editing an existing cpp file
  * It encourages fuzz tests that re-use the `buffer` or assume the `buffer` to be concatenations of seeds, which increases complexity of seeds and complexity for the fuzz engine to explore; Thus reducing the effectiveness of the affected fuzz targets

  Fixes #20088

ACKs for top commit:
  practicalswift:
    Tested ACK fa13e1b0c5
  sipa:
    ACK fa13e1b0c5. Reviewed the code changes, and tested the 3 different test_runner.py modes (run once, merge, generate). I also tested building with the new --enable-danger-fuzz-link-all

Tree-SHA512: 962ab33269ebd51810924c51266ecc62edd6ddf2fcd9a8c359ed906766f58c3f73c223f8d3cc49f2c60f0053f65e8bdd86ce9c19e673f8c2b3cd676e913f2642
2020-12-15 19:00:36 +01:00
Jarol Rodriguez
c175690561
doc: Update for FreeBSD 12.2, add GUI Build Instructions
The current FreeBSD Build documentation is a little outdated and underwhelming. This PR updates the doc to
be more informative. It also adds new instructions for building the GUI and adding support for  descriptor
wallets.

on vasild's recommendation: it is ok to point the user to download db5 which has an active port package
instead of building db4 from the provided script.

Co-authored-by: Vasil Dimov <vd@freebsd.org>
2020-12-10 15:38:20 -05:00
Wladimir J. van der Laan
b76abae387
Merge #20587: [doc] Tidy up Tor doc (more stringent)
32045bbfd5 [doc] Tidy up Tor doc (more stringent) (wodry)

Pull request description:

  This is a follow up to https://github.com/bitcoin/bitcoin/pull/19638 that left two deprecated "hidden service/server" naming occurences.

  It also shall make the chapter titles regarding creation of onion services stringent and easy to read and distinguish.

  It removes the one and only reference to the testnet (here the testnet onion service port), as it is not explained that it references to the testnet and I do not know why it is mentioned there. It is only confusing. Also, as said, the testnet is not referenced at any other place in this document.

ACKs for top commit:
  practicalswift:
    ACK 32045bbfd5
  laanwj:
    Review ACK 32045bbfd5
  RiccardoMasutti:
    ACK 32045bb

Tree-SHA512: c623387b76d68845c0fa47f67a6f8ef70c9c560e3f8f8754e45a4f51e43198c2092be789588acd4ada607f42fbe62d51a4b1888d81b225df19b6557a081835c0
2020-12-10 14:44:41 +01:00
Jonas Schnelli
86f2007193
Merge #20527: build: Do not ignore Homebrew's SQLite on macOS
c932e0d67e doc: Update wallet database installation guide for macOS (Hennadii Stepanov)
ee7b84e63c build: Use Homebrew's sqlite package if it is available (Hennadii Stepanov)
c96d1f65a5 build, refactor: Check that Homebrew's qt5 package is actually installed (Hennadii Stepanov)

Pull request description:

  On master (7ae86b3c68) installed Homebrew `sqlite` package is ignored during build on macOS.

  This PR fixes this issue and update macOS build docs.

  Closes #20498.

ACKs for top commit:
  willcl-ark:
    > > That said, another tACK of [c932e0d](c932e0d67e)
  hebasto:
    > That said, another tACK of [c932e0d](c932e0d67e)
  laanwj:
    Code review ACK c932e0d67e
  jonasschnelli:
    code review re-ACK c932e0d67e

Tree-SHA512: 2563f25534d065556b17ee8c0fca957aea61b5ae288a2aa72743e77607843a45c39f209321e0f05b34283a74d2edcf961cf1dc54a35ed0cc21182304bb961505
2020-12-10 13:18:30 +01:00
MarcoFalke
22f13c1e08
Merge #19776: net, rpc: expose high bandwidth mode state via getpeerinfo
343dc4760f test: add test for high-bandwidth mode states in getpeerinfo (Sebastian Falbesoner)
dab6583307 doc: release note for new getpeerinfo fields "bip152_hb_{from,to}" (Sebastian Falbesoner)
a7ed00f8bb rpc: expose high-bandwidth mode states via getpeerinfo (Sebastian Falbesoner)
30bc8fab68 net: save high-bandwidth mode states in CNodeStats (Sebastian Falbesoner)

Pull request description:

  Fixes #19676, "_For every peer expose through getpeerinfo RPC whether or not we selected them as HB peers, and whether or not they selected us as HB peers._" See [BIP152](https://github.com/bitcoin/bips/blob/master/bip-0152.mediawiki), in particular the [protocol flow diagram](https://github.com/bitcoin/bips/raw/master/bip-0152/protocol-flow.png).  The newly introduced states are changed on the following places in the code:
  * on reception of a `SENDCMPCT` message with valid version, the field `m_highbandwidth_from` is changed depending on the first integer parameter in the message (1=high bandwidth, 0=low bandwidth), i.e. it just mirrors the field `CNodeState.fPreferHeaderAndIDs`.
  * after adding a `SENDCMPCT` message to the send queue, the field `m_highbandwidth_to` is changed depending on how the first integer parameter is set (same as above)

  Note that after receiving `VERACK`, the node also sends `SENDCMPCT`, but that is only to announce the preferred version and never selects high-bandwidth mode, hence there is no need to change the state variables there, which are initialized to `false` anyways.

ACKs for top commit:
  naumenkogs:
    reACK 343dc4760f
  jonatack:
    re-ACK 343dc4760f per `git range-diff 7ea6499 4df1d12 343dc47`

Tree-SHA512: f4999e6a935266812c2259a9b5dc459710037d3c9e938006d282557cc225e56128f72965faffb207fc60c6531fab1206db976dd8729a69e8ca29d4835317b99f
2020-12-10 08:21:36 +01:00
MarcoFalke
44444ba759
fuzz: Link all targets once 2020-12-10 07:15:42 +01:00
Hennadii Stepanov
c932e0d67e
doc: Update wallet database installation guide for macOS 2020-12-07 13:25:06 +02:00
wodry
32045bbfd5
[doc] Tidy up Tor doc (more stringent)
This is a follow up to https://github.com/bitcoin/bitcoin/pull/19638 that left some deprectaed "hidden service/server" naming occurences.

It also shall make the chapter titles regarding creation of onion services stringent and easy to read and distinguish.

It removes the one and only reference to the testnet (here the testnet onion service port), as it is not explained that it references to the testnet and I do not know why it is mentioned there. It is only confusing. Also, as said, the testnet is not referenced at any other place in this document.
2020-12-07 05:17:26 +01:00
Sebastian Falbesoner
cb0b7125c1 doc: libbitcoinconsensus: add missing error code description, fix NBitcoin link 2020-12-05 13:37:00 +01:00
MarcoFalke
dca80ffb45
Merge #20255: util: Add Assume() identity function
faa05854f8 util: Remove probably misleading TODO (MarcoFalke)
fac5efe730 util: Add Assume() identity function (MarcoFalke)
fa861569dc util: Allow Assert(...) to be used in all contexts (practicalswift)

Pull request description:

  This is needed for #20138. Please refer to the added documentation for motivation.

ACKs for top commit:
  practicalswift:
    cr ACK faa05854f8
  jnewbery:
    utACK faa05854f8
  hebasto:
    ACK faa05854f8, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: 72165fbd898b92ab9a79b070993fa1faa86c2e3545b6645e72c652bda295d5107bc298d0482bf3aaf0926fc0c3e6418a445c0e073b08568c44231f547f76a688
2020-12-04 11:07:28 +01:00
MarcoFalke
283f22cabb
Merge #20461: rpc: Validate -rpcauth arguments
053b4fbad8 doc: Release note regarding -rpcauth validation (João Barbosa)
46001323b1 rpc: Validate -rpcauth arguments (João Barbosa)
d37c813a43 rpc: Refactor to process -rpcauth once (João Barbosa)

Pull request description:

  Invalid `-rpcauth` arguments are currently silently ignored. This make server initialization fail if any `-rpcauth` is invalid.

ACKs for top commit:
  MarcoFalke:
    review ACK 053b4fbad8
  jonatack:
    ACK 053b4fbad8
  ryanofsky:
    Code review ACK 053b4fbad8. Only changes since last review are moving a variable declaration and adding a comment, release notes, and a `const`.

Tree-SHA512: c99923d4a121f0c9f882b07f5402ea53e9b2d9455ad34468a094ffab1d64df26c82e1279734c0d42bc2e113eae7b581fbc3be52f3ed4a2d7450d11793afcf406
2020-12-02 09:37:37 +01:00
Emil Engler
1d578c078f
doc: Add bash as an OpenBSD dependency
If we require Python for the test framework, we should also require
bash. It is required for the linters and other scripts and does not
comes in a default OpenBSD installation.
2020-11-26 21:34:25 +01:00
Wladimir J. van der Laan
0918eb49d5 doc: Document current boost dependency as 1.71.0
This was forgotten in #19764.
2020-11-24 10:36:07 +01:00
MarcoFalke
fac5efe730
util: Add Assume() identity function 2020-11-24 09:47:29 +01:00
João Barbosa
053b4fbad8 doc: Release note regarding -rpcauth validation 2020-11-23 21:02:54 +00:00
Wladimir J. van der Laan
555b5d1bf9
Merge #20419: build: set minimum supported macOS to 10.14
a52ecc936a build: set minimum supported macOS to 10.14 (fanquake)

Pull request description:

  This is a requirement for C++17 support. See my comments [here](https://github.com/bitcoin/bitcoin/issues/16684#issuecomment-643722538):

  > You cannot use std::get with std::variant on macOS < 10.14, because Apples libc++ doesn't support the std::bad_variant_access exception. [Relevant comment](https://github.com/bitcoin/bitcoin/pull/19183#discussion_r439794318) in #19183.

  > While we could work around this in our own code, using std::get_if, this would still be a problem for 3rd-party dependencies.

  > I've been testing Qt 5.15LTS (we'll have to enable C++17 in qt, and may upgrade to a newer version at the same time), and you can't enable -std c++17, while targeting a macOS deployment version < 10.14, configuring will fail. They are making use of std::get with std::variant throughout their cocoa code.

  We would have to had to have bumped to at least 10.13 in any case, as Qt 5.15 (#19716) [requires 10.13+](https://doc.qt.io/qt-5/supported-platforms.html).

ACKs for top commit:
  hebasto:
    ACK a52ecc936a, I have reviewed the code and it looks OK, I agree it can be merged.

Tree-SHA512: f669b2fc777aeea1e9afdbbc7bd9afe3997418211db6ba53c934cae0e62a9b999603da539518c229f34961d275c9e2f315c7b022cf5fb97bd201a69c85d470cc
2020-11-23 14:24:58 +01:00
Wladimir J. van der Laan
86bf3ae3b5
Merge #20202: wallet: Make BDB support optional
d52f502b1e Fix mock SQLiteDatabases (Andrew Chow)
99309ab3e9 Allow disabling BDB in configure with --without-bdb (Andrew Chow)
ee47f11f73 GUI: Force descriptor wallets when BDB is not compiled (Andrew Chow)
71e40b33bd RPC: Require descriptors=True for createwallet when BDB is not compiled (Andrew Chow)
6ebc41bf9c Enforce salvage is only for BDB wallets (Andrew Chow)
a58b719cf7 Do not compile BDB things when USE_BDB is defined (Andrew Chow)
b33af48210 Include wallet/bdb.h where it is actually being used (Andrew Chow)

Pull request description:

  Adds a `--without-bdb` option to `configure` which disables the compilation of the BDB stuff. Legacy wallets will not be created when BDB is not compiled. A legacy-sqlite wallet can be loaded, but we will not create them.

  Based on #20156 to resolve the situation where both `--without-sqlite` and `--without-bdb` are provided. In that case, the wallet is disabled and `--disable-wallet` is effectively set.

ACKs for top commit:
  laanwj:
    Code review ACK d52f502b1e

Tree-SHA512: 5a92ba7a542acc2e27003e9d4e5940e0d02d5c1f110db06cdcab831372bfd83e8d89c269caff31dd5bff062c1cf5f04683becff12bd23a33be731676f346553d
2020-11-23 10:30:01 +01:00
MarcoFalke
d4159984c3
Merge #20223: build: Drop the leading 0 from the version number
8f7b930475 Drop the leading 0 from the version number (Andrew Chow)

Pull request description:

  Removes the leading 0 from the version number. The minor version, which we had been using as the major version, is now the major version. The revision, which we had been using as the minor version, is now the minor version. The revision number is dropped. The build number is promoted to being part of the version number. This also avoids issues where it was accidentally not included in the version number.

  The CLIENT_VERSION remains the same format as previous as previously, as the Major version was 0 so it never actually got included in it.

  The user agent string formatter is updated to follow this new versioning.

  ***

  Honestly I'm just tired of all of the people asking for "1.0" that maybe this'll shut them up. Skip the whole 1.0 thing and go straight to version 22.0!

  Also, this means that the terminology we commonly use lines up with how the variables are named. So major versions are actually bumping the major version number, etc.

ACKs for top commit:
  jnewbery:
    Code review ACK 8f7b930475
  MarcoFalke:
    review ACK 8f7b930475 🎻

Tree-SHA512: b5c3fae14d4c0a9c0ab3b1db7c949ecc0ac3537646306b13d98dd0efc17c489cdd16d43f0a24aaa28e9c4a92ea360500e05480a335b03f9fb308010cdd93a436
2020-11-20 15:42:07 +01:00
MarcoFalke
db58b857f7
Merge #20329: docs/descriptors.md: Remove hardened marker in the path after xpub
dc80a7d0b0 docs/descriptors.md: Remove hardened marker in the path after xpub (Dmitry Petukhov)

Pull request description:

  As described in "Key origin identification" section, a descriptor
  that has hardened derivation after xpub does not let you compute scripts
  without access to the corresponding private keys. Such a descriptor is
  practically useless.

  The text after the descriptor said "with child key *1'/2* of the
  specified xpub", and clearly an xpub cannot have "child key" with
  hardened derivation. Therefore it makes sense to fix this inconsistency
  to not confuse the reader of the doc

Top commit has no ACKs.

Tree-SHA512: 9f35951d90868585303681c27ea2ef9d36d4036181e337cfbd48730de0c346320b08dd089350b116d4c8542f3b506868cf318796bb713e5f80600ccbd96f9970
2020-11-19 17:59:20 +01:00
fanquake
ea7926527c
Merge #20413: build: Require C++17 compiler
fac7198728 Use std::make_unique (MarcoFalke)
faaee810e6 build: Require C++17 compiler (MarcoFalke)

Pull request description:

  Developers have been compiling with C++17 for a few months now (fuzz tests and the msvc build have it even enabled by default). According to #16684, the 22.0 release shall be compiled with C++17 enabled.

  This only sets the build flag, any other changes need more discussion and can be done later.

ACKs for top commit:
  elichai:
    utACK fac7198728
  hebasto:
    ACK fac7198728, I've locally compiled on ARM 32bit SBC without GUI.
  fanquake:
    ACK fac7198728

Tree-SHA512: 53eb40ba5d496376a2d2cf16e2000bef36616cc2a3696c3ec59a5366e41fa8b872817a7ca21751f030f9c1efb339dfa63cc655eaa5199b9a3d2e52c2de0ccb29
2020-11-19 10:29:05 +08:00
Andrew Chow
8f7b930475 Drop the leading 0 from the version number
Removes the leading 0 from the version number. The minor version, which
we had been using as the major version, is now the major version. The
revision, which we had been using as the minor version, is now the minor
version. The revision number is dropped. The build number is promoted to
being part of the version number. This also avoids issues where it was
accidentally not included in the version number.

The CLIENT_VERSION remains the same format as previous as previously,
the Major version was 0 so that was never a factor in CLIENT_VERSION.
2020-11-18 12:00:57 -05:00
Andrew Chow
99309ab3e9 Allow disabling BDB in configure with --without-bdb 2020-11-18 11:56:12 -05:00
MarcoFalke
faaee810e6
build: Require C++17 compiler 2020-11-18 15:15:04 +01:00
fanquake
a52ecc936a
build: set minimum supported macOS to 10.14 2020-11-18 21:46:09 +08:00
MarcoFalke
50e019a97a
Merge #20414: doc: Remove generated manual pages from master branch
daf1ebf0b1 doc: Remove generated manual pages from master branch (Wladimir J. van der Laan)

Pull request description:

  Replace the generated manual pages on the master branch with placeholders with instructions. The master branch is too much in flux for anything generated from command output to be kept up to date, and having pages from versions ago looks silly.

  We can't remove them completely because `make dist` relies on the files being present.

  Resolves #20062.

ACKs for top commit:
  MarcoFalke:
    review ACK daf1ebf0b1
  RiccardoMasutti:
    ACK daf1ebf

Tree-SHA512: fdd0bb777ddf66c663e4f6b00d206b3f6283ae89028e00b38d16e68a224a781e9245a25edf6f3326f9252ae2ef7c2950c591667c56d4e9d10a16fc50a6c358f4
2020-11-18 14:42:37 +01:00
fanquake
dc5a35a507
doc: clean out release notes post branch-off 2020-11-18 20:52:38 +08:00
Wladimir J. van der Laan
daf1ebf0b1 doc: Remove generated manual pages from master branch
Replace the generated manual pages on the master branch with
placeholders with instructions. The master branch is too much in flux
for anything generated from command output to be kept up to date,
and having pages from versions ago looks silly.

We can't remove them completely because `make dist` relies
on the files being present.

Resolves #20031.
2020-11-18 11:12:46 +01:00
MarcoFalke
4b24c3962f
Merge #19504: Bump minimum python version to 3.6
97c738ff1b [tests] Recommend f-strings for formatting, update feature_block to use them (Anthony Towns)
8ae9d314e9 Bump minimum python version to 3.6 (Anthony Towns)

Pull request description:

  Python 3.5 has reached [end-of-life](https://devguide.python.org/#status-of-python-branches) as of September 2020, and 3.6 has some moderately nice [features](https://docs.python.org/3/whatsnew/3.6.html):

  - `f'x = {x}'` as an alternative to `'x = {}'.format(x)` format strings (cf https://github.com/bitcoin/bitcoin/pull/13718#issuecomment-406591027)
  - underscore separators for large numbers, like `1_234_567`
  - improvements to async
  - improvements to typing module

  Note that 3.6 is not available in xenial (16.04), but is available in bionic (18.04), while focal (20.04) has 3.8. CentOS 7 and 8 have 3.6.8, Debian stable has 3.7.3, and [gentoo and arch already had 3.6 and 3.7 in 2018](https://github.com/bitcoin/bitcoin/pull/14954#issuecomment-447118707).

ACKs for top commit:
  MarcoFalke:
    re-ACK 97c738ff1b

Tree-SHA512: ec7fce68845edde4d61a42de12c065fd49e5217311a6fda1323206f091a0afd50f293645dffc27d420127e4e5deb864e953f1b67eff735a0dfbbedd7899a9d60
2020-11-18 10:24:22 +01:00
practicalswift
fd0be92cff doc: Add instructions on how to fuzz the P2P layer using Honggfuzz NetDriver 2020-11-12 20:20:29 +00:00
Wladimir J. van der Laan
0bd4929cd0
Merge #20284: addrman: ensure old versions don't parse peers.dat
38ada892ed addrman: ensure old versions don't parse peers.dat (Vasil Dimov)

Pull request description:

  Even though the format of `peers.dat` was changed in a backwards
  incompatible way, it is not guaranteed that old versions will fail to
  parse it. There is a chance that old versions parse its contents as
  garbage and use it.

  Old versions expect the "key size" field to be 32 and fail the parsing
  if it is not. Thus, we put something other than 32 in it. This will make
  versions between 0.11.0 and 0.20.1 deterministically fail on the new
  format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
  will still parse it as garbage.

  Also, introduce a way to increment the `peers.dat` format in a way that
  does not necessary make older versions refuse to read it.

ACKs for top commit:
  jnewbery:
    ACK 38ada892ed
  laanwj:
    Code review ACK 38ada892ed
  MarcoFalke:
    re-ACK 38ada892ed 🥐

Tree-SHA512: 550bd660c5019dba0f9c334aca8a11c4a0463cfddf11efe7a4a5585ffb05549c82b95066fba5d073ae37893e0eccc158a7ffea9b33ea031d9be4a39e44f6face
2020-11-12 17:05:51 +01:00
Vasil Dimov
38ada892ed
addrman: ensure old versions don't parse peers.dat
Even though the format of `peers.dat` was changed in an incompatible
way (old software versions <0.21 cannot understand the new file format),
it is not guaranteed that old versions will fail to parse it. There is a
chance that old versions parse its contents as garbage and use it.

Old versions expect the "key size" field to be 32 and fail the parsing
if it is not. Thus, we put something other than 32 in it. This will make
versions between 0.11.0 and 0.20.1 deterministically fail on the new
format. Versions prior to https://github.com/bitcoin/bitcoin/pull/5941
(<0.11.0) will still parse it as garbage.

Also, introduce a way to increment the `peers.dat` format in a way that
does not necessary make older versions refuse to read it.
2020-11-11 16:05:15 +01:00
Anthony Towns
8ae9d314e9 Bump minimum python version to 3.6 2020-11-09 17:53:47 +10:00
Dmitry Petukhov
dc80a7d0b0
docs/descriptors.md: Remove hardened marker in the path after xpub
As described in "Key origin identification" section, a descriptor
that has hardened derivation after xpub does not let you compute scripts
without access to the corresponding private keys. Such a descriptor is
practically useless.

The text after the descriptor said "with child key *1'/2* of the
specified xpub", and clearly an xpub cannot have "child key" with
hardened derivation. Therefore it makes sense to fix this inconsistency
to not confuse the reader of the doc
2020-11-06 14:27:47 +05:00
Jon Atack
e5f3e95a8e
doc: fix getchaintxstats fields in release-process.md
also:

- use the getblockheader (and getblockhash) RPCs instead of getblockchaininfo
  for updating the nMinimumChainWork and defaultAssumeValid consensus params

- use "RPC" consistently

- update the example PR from 17002 to 20263

- improve a link with a named anchor tag
2020-11-01 22:19:24 +01:00
MarcoFalke
42b66a6b81
Merge #20186: wallet: Make -wallet setting not create wallets
01476a88a6 wallet: Make -wallet setting not create wallets (Russell Yanofsky)

Pull request description:

  This changes `-wallet` setting to only load existing wallets, not create new ones.

  - Fixes settings.json corner cases reported by sjors & promag: https://github.com/bitcoin-core/gui/issues/95, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578, https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

  - Prevents accidental creation of wallets reported most recently by jb55 http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

  - Simplifies behavior after #15454. #15454 took the big step of disabling creation of the default wallet. This PR extends it to avoid creating other wallets as well. With this change, new wallets just aren't created on startup, instead of sometimes being created, sometimes not. #15454 release notes are updated here and are simpler.

  This change should be targeted for 0.21.0. It's a bug fix and simplifies behavior of the #15937 / #19754 / #15454 features added in 0.21.0.

  ---

  This PR is implementing the simplest, most basic alternative listed in https://github.com/bitcoin-core/gui/issues/95#issuecomment-694236940. Other improvements mentioned there can build on top of this.

ACKs for top commit:
  achow101:
    ACK 01476a88a6
  hebasto:
    re-ACK 01476a88a6
  MarcoFalke:
    review ACK 01476a88a6 🏂

Tree-SHA512: 0d50f4e5dfbd04a2efd9fd66c02085a0ed705807bdec1cf5770d0ae8cb6af07080fb81306349937bf66acdb713d03fb35636f6442b650d0820e66cbae09c2f87
2020-10-29 15:01:39 +01:00
Wladimir J. van der Laan
f3727fd735
Merge #20156: build: Make sqlite support optional (compile-time)
bbb42a6896 RPC: createwallet: Nicer error message if descriptor wallet requested and sqlite support not compiled in (Luke Dashjr)
6608fec332 GUI: Create Wallet: Nicely disable descriptor wallet checkbox if sqlite support not compiled in (Luke Dashjr)
7b54d768e1 Make sqlite support optional (compile-time) (Luke Dashjr)

Pull request description:

  As a new requirement, sqlite support should be optional. This PR aims to be only minimum/blocker changes for 0.21.

  Potential follow-up PRs after this:
  * Make BDB support optional
  * Nicer error messages when user tries to load an unsupported wallet
  * Don't compile descriptor wallet code if sqlite disabled

ACKs for top commit:
  jonasschnelli:
    Tested ACK bbb42a6896
  achow101:
    ACK bbb42a6896
  Sjors:
    re-utACK bbb42a6896
  hebasto:
    ACK bbb42a6896, tested on Linux Mint 20 (x86_64, Qt 5.12.8).

Tree-SHA512: 500209dd1971310fab8ae51543343ce0ba91f088ccccff6109b4cc27547cd5532289dca6cb7dac2a7d7c59cdf3c8f5aacc31e9b0f912e38cea52ec26b97100bd
2020-10-29 12:03:36 +01:00
fanquake
67d4643a1a
Merge #20152: doc: Update wallet files in files.md
defe48a51f doc: Update wallet files in files.md (Hennadii Stepanov)

Pull request description:

  This PR is a #19077 follow up, and it addresses the [comment](https://github.com/bitcoin/bitcoin/pull/19077#discussion_r504805234):

  > If need to update, there are two corrections that could be made:
  >
  >  * Line 69 "Wallets are Berkeley DB (BDB) databases" is no longer true
  >
  >  * Line 76 "Wallet lock file" should say "BDB wallet lock file"

ACKs for top commit:
  RiccardoMasutti:
    ACK defe48a
  meshcollider:
    ACK defe48a51f

Tree-SHA512: 39939f86a9c7842bf06913998305dcbd6209585f1da0fe9c274bac0572eb8464e59176884dd9e2b91312f34efad40cdeb4085ec72c2a2c1b33d16b6ab505140c
2020-10-27 15:43:53 +08:00
Russell Yanofsky
01476a88a6 wallet: Make -wallet setting not create wallets
This changes -wallet setting to only load existing wallets, not create new ones.

- Fixes settings.json corner cases reported by sjors & promag:
  https://github.com/bitcoin-core/gui/issues/95,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578,
  https://github.com/bitcoin/bitcoin/pull/19754#issuecomment-685858578

- Prevents accidental creation of wallets reported most recently by jb55
  http://www.erisian.com.au/bitcoin-core-dev/log-2020-09-14.html#l-355

- Simplifies behavior after #15454. #15454 took the big step of disabling
  creation of the default wallet. This PR extends it to avoid creating other
  wallets as well. With this change, new wallets just aren't created on
  startup, instead of sometimes being created, sometimes not. #15454 release
  notes are updated here and are simpler.

This change should be targeted for 0.21.0. It's a bug fix and simplifies
behavior of the #15937 / #19754 / #15454 features added in 0.21.0.
2020-10-21 08:48:43 -04:00
Luke Dashjr
7b54d768e1 Make sqlite support optional (compile-time) 2020-10-20 13:44:43 +00:00
MarcoFalke
fa38093bee
doc: Merge release notes 2020-10-19 12:00:16 +02:00
Pieter Wuille
5669642a0b docs: mention BIPs 340-342 in doc/bips.md 2020-10-15 14:20:20 -07:00
Wladimir J. van der Laan
9855422e65
Merge #17428: p2p: Try to preserve outbound block-relay-only connections during restart
a490d074b3 doc: Add anchors.dat to files.md (Hennadii Stepanov)
0a85e5a7bc p2p: Try to connect to anchors once (Hennadii Stepanov)
5543c7ab28 p2p: Fix off-by-one error in fetching address loop (Hennadii Stepanov)
4170b46544 p2p: Integrate DumpAnchors() and ReadAnchors() into CConnman (Hennadii Stepanov)
bad16aff49 p2p: Add CConnman::GetCurrentBlockRelayOnlyConns() (Hennadii Stepanov)
c29272a157 p2p: Add ReadAnchors() (Hennadii Stepanov)
567008d2a0 p2p: Add DumpAnchors() (Hennadii Stepanov)

Pull request description:

  This is an implementation of #17326:
  - all (currently 2) outbound block-relay-only connections (#15759) are dumped to `anchors.dat` file
  - on restart a node tries to connect to the addresses from `anchors.dat`

  This PR prevents a type of eclipse attack when an attacker exploits a victim node restart to force it to connect to new, probably adversarial, peers.

ACKs for top commit:
  jnewbery:
    code review ACK a490d074b3
  laanwj:
    Code review ACK a490d074b3

Tree-SHA512: 0f5098a3882f2814be1aa21de308cd09e6654f4e7054b79f3cfeaf26bc02b814ca271497ed00018d199ee596a8cb9b126acee8b666a29e225b08eb2a49b02ddd
2020-10-15 20:19:55 +02:00
Hennadii Stepanov
defe48a51f
doc: Update wallet files in files.md 2020-10-15 11:32:26 +03:00
Samuel Dobson
8ed37f6c84
Merge #19077: wallet: Add sqlite as an alternative wallet database and use it for new descriptor wallets
c4a29d0a90 Update wallet_multiwallet.py for descriptor and sqlite wallets (Russell Yanofsky)
310b0fde04 Run dumpwallet for legacy wallets only in  wallet_backup.py (Andrew Chow)
6c6639ac9f Include sqlite3 in documentation (Andrew Chow)
f023b7cac0 wallet: Enforce sqlite serialized threading mode (Andrew Chow)
6173269866 Set and check the sqlite user version (Andrew Chow)
9d3d2d263c Use network magic as sqlite wallet application ID (Andrew Chow)
9af5de3798 Use SQLite for descriptor wallets (Andrew Chow)
9b78f3ce8e walletutil: Wallets can also be sqlite (Andrew Chow)
ac38a87225 Determine wallet file type based on file magic (Andrew Chow)
6045f77003 Implement SQLiteDatabase::MakeBatch (Andrew Chow)
727e6b2a4e Implement SQLiteDatabase::Verify (Andrew Chow)
b4df8fdb19 Implement SQLiteDatabase::Rewrite (Andrew Chow)
010e365906 Implement SQLiteDatabase::TxnBegin, TxnCommit, and TxnAbort (Andrew Chow)
ac5c1617e7 Implement SQLiteDatabase::Backup (Andrew Chow)
f6f9cd6a64 Implement SQLiteBatch::StartCursor, ReadAtCursor, and CloseCursor (Andrew Chow)
bf90e033f4 Implement SQLiteBatch::ReadKey, WriteKey, EraseKey, and HasKey (Andrew Chow)
7aa45620e2 Add SetupSQLStatements (Andrew Chow)
6636a2608a Implement SQLiteBatch::Close (Andrew Chow)
93825352a3 Implement SQLiteDatabase::Close (Andrew Chow)
a0de83372b Implement SQLiteDatabase::Open (Andrew Chow)
3bfa0fe125 Initialize and Shutdown sqlite3 globals (Andrew Chow)
5a488b3d77 Constructors, destructors, and relevant private fields for SQLiteDatabase/Batch (Andrew Chow)
ca8b7e04ab Implement SQLiteDatabaseVersion (Andrew Chow)
7577b6e1c8 Add SQLiteDatabase and SQLiteBatch dummy classes (Andrew Chow)
e87df82580 Add sqlite to travis and depends (Andrew Chow)
54729f3f4e Add libsqlite3 (Andrew Chow)

Pull request description:

  This PR adds a new class `SQLiteDatabase` which is a subclass of `WalletDatabase`. This provides access to a SQLite database that is used to store the wallet records. To keep compatibility with BDB and to complexity of the change down, we don't make use of many SQLite's features. We use it strictly as a key-value store. We create a table `main` which has two columns, `key` and `value` both with the type `blob`.

  For new descriptor wallets, we will create a `SQLiteDatabase` instead of a `BerkeleyDatabase`. There is no requirement that all SQLite wallets are descriptor wallets, nor is there a requirement that all descriptor wallets be SQLite wallets. This allows for existing descriptor wallets to work as well as keeping open the option to migrate existing wallets to SQLite.

  We keep the name `wallet.dat` for SQLite wallets. We are able to determine which database type to use by searching for specific magic bytes in the `wallet.dat` file. SQLite begins it's files with a null terminated string `SQLite format 3`. BDB has `0x00053162` at byte 12 (note that the byte order of this integer depends on the system endianness). So when we see that there is a `wallet.dat` file that we want to open, we check for the magic bytes to determine which database system to use.

  I decided to keep the `wallet.dat` naming to keep things like backup script to continue to function as they won't need to be modified to look for a different file name. It also simplifies a couple of things in the implementation and the tests as `wallet.dat` is something that is specifically being looked for. If we don't want this behavior, then I do have another branch which creates `wallet.sqlite` files instead, but I find that this direction is easier.

ACKs for top commit:
  Sjors:
    re-utACK c4a29d0a90
  promag:
    Tested ACK c4a29d0a90.
  fjahr:
    reACK c4a29d0a90
  S3RK:
    Re-review ACK c4a29d0a90
  meshcollider:
    re-utACK c4a29d0a90
  hebasto:
    re-ACK c4a29d0a90, only rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/19077#pullrequestreview-507743699) review, verified with `git range-diff master d18892dcc c4a29d0a9`.
  ryanofsky:
    Code review ACK c4a29d0a90. I am honestly confused about reasons for locking into `wallet.dat` again when it's so easy now to use a clean format. I assume I'm just very dense, or there's some unstated reason, because the only thing that's been brought up are unrealistic compatibility scenarios (all require actively creating a wallet with non-default descriptor+sqlite option, then trying to using the descriptor+sqlite wallets with old software or scripts and ignoring the results) that we didn't pay attention to with previous PRs like #11687, which did not require any active interfaction.
  jonatack:
    ACK c4a29d0a90, debug builds and test runs after rebase to latest master @ c2c4dbaebd, some manual testing creating, using, unloading and reloading a few different new sqlite descriptor wallets over several node restarts/shutdowns.

Tree-SHA512: 19145732e5001484947352d3175a660b5102bc6e833f227a55bd41b9b2f4d92737bbed7cead64b75b509decf9e1408cd81c185ab1fb4b90561aee427c4f9751c
2020-10-15 20:12:29 +13:00
Wladimir J. van der Laan
c2c4dbaebd
Merge #19988: Overhaul transaction request logic
fd9a0060f0 Report and verify expirations (Pieter Wuille)
86f50ed10f Delete limitedmap as it is unused now (Pieter Wuille)
cc16fff3e4 Make txid delay penalty also apply to fetches of orphan's parents (Pieter Wuille)
173a1d2d3f Expedite removal of tx requests that are no longer needed (Pieter Wuille)
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers (Pieter Wuille)
242d16477d Change transaction request logic to use txrequest (Pieter Wuille)
5b03121d60 Add txrequest fuzz tests (Pieter Wuille)
3c7fe0e5a0 Add txrequest unit tests (Pieter Wuille)
da3b8fde03 Add txrequest module (Pieter Wuille)

Pull request description:

  This replaces the transaction request logic with an encapsulated class that maintains all the state surrounding it. By keeping it stand alone, it can be easily tested (using included unit tests and fuzz tests).

  The major changes are:

  * Announcements from outbound (and whitelisted) peers are now always preferred over those from inbound peers. This used to be the case for the first request (by delaying the first request from inbound peers), and a bias afters. The 2s delay for requests from inbound peers still exists, but after that, if viable outbound peers remain for any given transaction, they will always be tried first.
  * No more hard cap of 100 in flight transactions per peer, as there is less need for it (memory usage is linear in the number of announcements, but independent from the number in flight, and CPU usage isn't affected by it). Furthermore, if only one peer announces a transaction, and it has over 100 in flight already, we still want to request it from them. The cap is replaced with a rule that announcements from such overloaded peers get an additional 2s delay (possibly combined with the existing 2s delays for inbound connections, and for txid peers when wtxid peers are available).
  * The limit of 100000 tracked announcements is reduced to 5000; this was excessive. This can be bypassed using the PF_RELAY permission (to accommodate locally dumping a batch of many transactions).

  This replaces #19184, rebased on #18044 and with many small changes.

ACKs for top commit:
  ariard:
    Code Review ACK fd9a006. I've reviewed the new TxRequestTracker, its integration in net_processing, unit/functional/fuzzing test coverage. I looked more for soundness of new specification rather than functional consistency with old transaction request logic.
  MarcoFalke:
    Approach ACK fd9a0060f0 🏹
  naumenkogs:
    Code Review ACK fd9a006. I've reviewed everything, mostly to see how this stuff works at the lower level (less documentation-wise, more implementation-wise), and to try breaking it with unexpected sequences of events.
  jnewbery:
    utACK fd9a0060f0
  jonatack:
    WIP light ACK fd9a0060f0 have read the code, verified that each commit is hygienic, e.g. debug build clean and tests green, and have been running a node on and off with this branch and grepping the net debug log. Am still unpacking the discussion hidden by GitHub by fetching it via the API and connecting the dots, storing notes and suggestions in a local branch; at this point none are blockers.
  ryanofsky:
    Light code review ACK fd9a0060f0, looking at txrequest implementation, unit test implementation, and net_processing integration, just trying to understand how it works and looking for anything potentially confusing in the implementation. Didn't look at functional tests or catch up on review discussion. Just a sanity check review focused on:

Tree-SHA512: ea7b52710371498b59d9c9cfb5230dd544fe9c6cb699e69178dea641646104f38a0b5ec7f5f0dbf1eb579b7ec25a31ea420593eff3b7556433daf92d4b0f0dd7
2020-10-14 18:36:59 +02:00
Andrew Chow
6c6639ac9f Include sqlite3 in documentation 2020-10-14 11:28:18 -04:00
fanquake
e21b824386
Merge #20109: Release notes and followups from 19339
88197b0769 [doc] release notes for max fee checking (gzhao408)
c201d73df3 style and nits for fee-checking in BroadcastTransaction (gzhao408)

Pull request description:

  Pretty trivial... addresses some tiny comments from #19339. Also fixes a docs typo from #19940 and adds a release note about the error message change for testmempoolaccept.

ACKs for top commit:
  achow101:
    ACK 88197b0769
  MarcoFalke:
    cr re-ACK 88197b0769

Tree-SHA512: fff16d731426b9b4db5222df02633983402f4c7241551eec98bb1554145dbdc132f40ed8ca4abd5edcebe1f4d1e879fb6d11bd91730604f6552c10cdf65706a1
2020-10-14 12:19:42 +08:00
Pieter Wuille
de11b0a4ef Reduce MAX_PEER_TX_ANNOUNCEMENTS for non-PF_RELAY peers
Maintaining up to 100000 INVs per peer is excessive, as that is far more
than fits in a typical mempool.

Also disable the "overload" penalty for PF_RELAY peers.
2020-10-12 12:14:53 -07:00
Pieter Wuille
79f3d9b932 Mention BIP155 in doc/bips.md 2020-10-11 11:29:07 -07:00
fanquake
0b2abaa666
Merge #19954: Complete the BIP155 implementation and upgrade to TORv3
dcf0cb4776 tor: make a TORv3 hidden service instead of TORv2 (Vasil Dimov)
353a3fdaad net: advertise support for ADDRv2 via new message (Vasil Dimov)
201a4596d9 net: CAddress & CAddrMan: (un)serialize as ADDRv2 (Vasil Dimov)
1d3ec2a1fd Support bypassing range check in ReadCompactSize (Pieter Wuille)

Pull request description:

  This PR contains the two remaining commits from #19031 to complete the [BIP155](https://github.com/bitcoin/bips/blob/master/bip-0155.mediawiki) implementation:

  `net: CAddress & CAddrMan: (un)serialize as ADDRv2`
  `net: advertise support for ADDRv2 via new message`

  plus one more commit:

  `tor: make a TORv3 hidden service instead of TORv2`

ACKs for top commit:
  jonatack:
    re-ACK dcf0cb4776 per `git diff 9b56a68 dcf0cb4` only change since last review is an update to the release notes which partially picked up the suggested text. Running a node on this branch and addnode-ing to 6 other Tor v3 nodes, I see "addrv2" and "sendaddrv2" messages in getpeerinfo in both the "bytesrecv_per_msg" and "bytessent_per_msg" JSON objects.
  sipa:
    ACK dcf0cb4776
  hebasto:
    re-ACK dcf0cb4776, the node works flawlessly in all of the modes: Tor-only, clearnet-only, mixed.
  laanwj:
    Edit: I have to retract this ACK for now, I'm having some problems with this PR on a FreeBSD node. It drops all outgoing connections with this dcf0cb4776 merged on master (12a1c3ad1a).
  ariard:
    Code Review ACK dcf0cb4

Tree-SHA512: 28d4d0d817b8664d2f4b18c0e0f31579b2f0f2d23310ed213f1f436a4242afea14dfbf99e07e15889bc5c5c71ad50056797e9307ff8a90e96704f588a6171308
2020-10-11 08:51:57 +08:00
gzhao408
88197b0769 [doc] release notes for max fee checking 2020-10-09 08:58:47 -07:00
Vasil Dimov
dcf0cb4776
tor: make a TORv3 hidden service instead of TORv2
TORv2 is deprecated [1], thus whenever we create the hidden service
ourselves create a TORv3 one instead.

[1] https://blog.torproject.org/v2-deprecation-timeline
2020-10-09 16:42:50 +02:00
Vasil Dimov
201a4596d9
net: CAddress & CAddrMan: (un)serialize as ADDRv2
Change the serialization of `CAddrMan` to serialize its addresses
in ADDRv2/BIP155 format by default. Introduce a new `CAddrMan` format
version (3).

Add support for ADDRv2 format in `CAddress` (un)serialization.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-10-09 16:42:49 +02:00
Hennadii Stepanov
a490d074b3
doc: Add anchors.dat to files.md 2020-10-09 14:30:09 +03:00
fanquake
12a1c3ad1a
Merge #20107: doc: Collect release-notes snippets
faa0847dec doc: Add release notes for #20101 (MarcoFalke)
99992e7832 doc: Collect release-notes snippets (MarcoFalke)

Pull request description:

  Also add a note for #20101

ACKs for top commit:
  fanquake:
    ACK faa0847dec - no need to bike-shed here as these will all get massaged to death in the wiki anyways.

Tree-SHA512: 63d3597e2bbd422ec182e76112110477d22e3afedb479114ddec958405bcdd63492df9477267aac65605612af49c0aff6246b1bc3d41dd606d6d61c30117c109
2020-10-09 18:08:14 +08:00
MarcoFalke
faa0847dec
doc: Add release notes for #20101 2020-10-08 15:11:25 +02:00
Hennadii Stepanov
2dc79c4264
doc: Update and improve files.md
Added the `signet` subdirectory and the `ip_asn.map` file.
2020-10-06 11:05:03 +03:00
Wladimir J. van der Laan
df2129a234
Merge #19991: net: Use alternative port for incoming Tor connections
96571b3d4c doc: Update onion service target port numbers in tor.md (Hennadii Stepanov)
bb145c9050 net: Extend -bind config option with optional network type (Hennadii Stepanov)
92bd3c1da4 net, refactor: Move AddLocal call one level up (Hennadii Stepanov)
57f17e57c8 net: Pass onion service target to Tor controller (Hennadii Stepanov)
e3f07851f0 refactor: Rename TorController::target to m_tor_control_center (Hennadii Stepanov)
fdd3ae4d26 net, refactor: Refactor CBaseChainParams::RPCPort function (Hennadii Stepanov)
a5266d4546 net: Add alternative port for onion service (Hennadii Stepanov)
b3273cf403 net: Use network byte order for in_addr.s_addr (Hennadii Stepanov)

Pull request description:

  This PR adds ability to label incoming Tor connections as different from normal localhost connections.

  Closes #8973.
  Closes #16693.

  Default onion service target ports are:
  - 8334 on mainnnet
  - 18334 on testnet
  - 38334 on signet
  - 18445 on regtest

  To set the onion service target socket manually the extended `-bind` config option could be used:

  ```
  $ src/bitcoind -help | grep -A 6 -e '-bind'
    -bind=<addr>[:<port>][=onion]
         Bind to given address and always listen on it (default: 0.0.0.0). Use
         [host]:port notation for IPv6. Append =onion to tag any incoming
         connections to that address and port as incoming Tor connections
         (default: 127.0.0.1:8334=onion, testnet: 127.0.0.1:18334=onion,
         signet: 127.0.0.1:38334=onion, regtest: 127.0.0.1:18445=onion)

  ```

  Since [pr19991.02 update](https://github.com/bitcoin/bitcoin/pull/19991#issuecomment-698882284) this PR is an alternative to #19043.

ACKs for top commit:
  Sjors:
    re-utACK 96571b3d4c
  vasild:
    ACK 96571b3d4
  laanwj:
    Re-ACK 96571b3d4c

Tree-SHA512: cb0eade80f4b3395f405f775e1b89c086a1f09d5a4464df6cb4faf808d9c2245474e1720b2b538f203f6c1996507f69b09f5a6e35ea42633c10e22bd733d4438
2020-10-02 13:37:23 +02:00
MarcoFalke
99992e7832
doc: Collect release-notes snippets 2020-10-02 11:24:46 +02:00
Hennadii Stepanov
96571b3d4c
doc: Update onion service target port numbers in tor.md 2020-10-01 19:19:43 +03:00
Wladimir J. van der Laan
a0185d90a7
Merge #18309: zmq: Add support to listen on multiple interfaces
e66870c5a4 zmq: Append address to notify log output (nthumann)
241803da21 test: Add zmq test to support multiple interfaces (nthumann)
a0b2e5cb6a doc: Add release notes to support multiple interfaces (nthumann)
b1c3f180ec doc: Adjust ZMQ usage to support multiple interfaces (nthumann)
347c94f551 zmq: Add support to listen on multiple interfaces (Nicolas Thumann)

Pull request description:

  This PR adds support for ZeroMQ to listen on multiple interfaces, just like the RPC server.
  Currently, if you specify more than one e.g. `zmqpubhashblock` paramter, only the first one will be used. Therefore a user may be forced to listen on all interfaces (e.g. `zmqpubhashblock=0.0.0.0:28332`), which can result in an increased attack surface.
  With this PR a user can specify multiple interfaces to listen on, e.g.
  `-zmqpubhashblock=tcp://127.0.0.1:28332 -zmqpubhashblock=tcp://192.168.1.123:28332`.

ACKs for top commit:
  laanwj:
    Code review ACK e66870c5a4
  instagibbs:
    reACK e66870c5a4

Tree-SHA512: f38ab4a6ff00dc821e5f4842508cefadb701e70bb3893992c1b32049be20247c8aa9476a1f886050c5f17fe7f2ce99ee30193ce2c81a7482a5a51f8fc22300c7
2020-10-01 17:43:34 +02:00
Wladimir J. van der Laan
72affcb16c
Merge #19802: doc: elaborate on release notes wrt netmasks
8de51d1513 doc: elaborate on release notes wrt netmasks (Vasil Dimov)

Pull request description:

  doc: elaborate on release notes wrt netmasks

  A minor followup from https://github.com/bitcoin/bitcoin/pull/19628#issuecomment-679958713.

ACKs for top commit:
  laanwj:
    ACK 8de51d1513
  practicalswift:
    ACK 8de51d1513
  theStack:
    ACK 8de51d1513

Tree-SHA512: ccece7c3057e476d59e9996582e8594b3db9eaed397139217335a41307e542194c8d13ad72458eaa8580a5c469fef9cc9e3f11c1c609617757579530b465e5b0
2020-09-30 16:03:02 +02:00
MarcoFalke
fa710a6d67
doc: Add 19501 release notes 2020-09-30 09:28:50 +02:00
fanquake
e36aa351a3
Merge #19969: Send RPC bug fix and touch-ups
f7b331ea85 rpc: add brackets to ConstructTransaction (Sjors Provoost)
d813d26f06 [rpc] send: various touch-ups (Sjors Provoost)
0fc1c685e1 [rpc] send: fix parsing replaceable option (Sjors Provoost)
efc9b85e6f Mark send RPC experimental (Sjors Provoost)

Pull request description:

  Followup based on #16378 nits. It also fixes an argument parsing error (uncaught because the test wasn't sufficiently thorough).

  I marked the RPC as experimental so we can tweak it a bit over the next release cycle.

ACKs for top commit:
  meshcollider:
    utACK f7b331ea85
  fjahr:
    utACK f7b331ea85
  kallewoof:
    ACK f7b331ea85

Tree-SHA512: 82dd8ac76a6558872db3f5249d4d6440469400aaa339153bc627d1ee673a91ecfadecb486bc1939ba87ebbd80e26ff29698e93e358599f3d26fde0e526892afe
2020-09-29 15:14:08 +08:00
Sebastian Falbesoner
dab6583307 doc: release note for new getpeerinfo fields "bip152_hb_{from,to}"
Co-authored-by: Jon Atack <jon@atack.com>
2020-09-29 00:42:06 +02:00
MarcoFalke
2552702000
Merge #15367: feature: Added ability for users to add a startup command
090530cc24 feature: Added ability for users to add a startup command (Ben Carman)

Pull request description:

  Thoughts for adding the feature is for users to be able to add things like electrum-personal-server or lnd to run whenever Bitcoin Core is running.  Open to feedback about the feature.

ACKs for top commit:
  MarcoFalke:
    re-ACK 090530cc24
  dongcarl:
    tACK 090530c

Tree-SHA512: ba514d2fc8b4fb12b781c1a9c89845a25fce0b80ba7c907761cde4abb81edd03fa643682edc895986dc20b273ac3b95769508806db7fbd99ec28623f85c41e67
2020-09-28 20:44:33 +02:00
Ben Carman
090530cc24
feature: Added ability for users to add a startup command 2020-09-28 10:38:36 -05:00
MarcoFalke
4f45ea1f73
Merge #19725: [RPC] Add connection type to getpeerinfo, improve logs
a512925e19 [doc] Release notes (Amiti Uttarwar)
50f94b34a3 [rpc] Deprecate getpeerinfo addnode field (Amiti Uttarwar)
df091b9b50 [refactor] Rename test file to allow any getpeerinfo deprecations. (Amiti Uttarwar)
395acfa83a [rpc] Add connection type to getpeerinfo RPC, update tests (Amiti Uttarwar)
49c10a9ca4 [log] Add connection type to log statement (Amiti Uttarwar)

Pull request description:

  After #19316, we can more directly expose information about the connection type on the `getpeerinfo` RPC. Doing so also makes the existing addnode field redundant, so this PR begins the process of deprecating this field.

  This PR also includes one commit that improves a log message, as both use a shared function to return the connection type as a string.

  Suggested by sdaftuar- https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468001604 & https://github.com/bitcoin/bitcoin/pull/19316#discussion_r468018093

ACKs for top commit:
  jnewbery:
    Code review ACK a512925e19.
  sipa:
    utACK a512925e19
  guggero:
    Tested and code review ACK a512925e.
  MarcoFalke:
    cr ACK a512925e19 🌇
  promag:
    Code review ACK a512925e19.

Tree-SHA512: 601a7a38aee235ee59aca690784f886dc2ae4e418b2e6422c4b58cd597376c00f74910f66920b08a08a0bec28bf8022e71a1435785ff6ba8a188954261aba78e
2020-09-26 17:24:54 +02:00
nthumann
a0b2e5cb6a
doc: Add release notes to support multiple interfaces 2020-09-23 23:14:29 +02:00
nthumann
b1c3f180ec
doc: Adjust ZMQ usage to support multiple interfaces 2020-09-23 23:14:28 +02:00
MarcoFalke
8235dca621
Merge #19979: Replace LockAssertion with AssertLockHeld, remove LockAssertion
0bd1184adf Remove unused LockAssertion struct (Hennadii Stepanov)
ab2a44297f Replace LockAssertion with a proper thread safety annotations (Hennadii Stepanov)
73f71e1996 refactor: Use explicit function type instead of template (Hennadii Stepanov)

Pull request description:

  This PR replaces `LockAssertion` with `AssertLockHeld`, and removes `LockAssertion`.

  This PR is compared with alternatives in https://github.com/bitcoin-core/bitcoin-devwiki/wiki/AssertLockHeld-PRs

ACKs for top commit:
  MarcoFalke:
    ACK 0bd1184adf
  ajtowns:
    ACK 0bd1184adf
  vasild:
    ACK 0bd1184ad

Tree-SHA512: ef7780dd689faf0bb479fdb97c49bc652e2dd10c148234bb95502dfbb676442d8565ee37864d923ca21a25f9dc2a335bf46ee82c095e387b59a664ab05c0ae41
2020-09-23 16:37:07 +02:00
Wladimir J. van der Laan
9e217f5a6f
Merge #19572: ZMQ: Create "sequence" notifier, enabling client-side mempool tracking
759d94e70f Update zmq notification documentation and sample consumer (Gregory Sanders)
68c3c7e1bd Add functional tests for zmq sequence topic and mempool sequence logic (Gregory Sanders)
e76fc2b84d Add 'sequence' zmq publisher to track all block (dis)connects, mempool deltas (Gregory Sanders)
1b615e61bf zmq test: Actually make reorg occur (Gregory Sanders)

Pull request description:

  This PR creates a new ZMQ notifier that gives a "total hash history" of block (dis)connection, mempool addition/substraction, all in one pipeline. It also exposes a "mempool sequence number" to both this notifier and `getrawmempool` results, which allows the consumer to use the results together without confusion about ordering of results and without excessive `getrawmempool` polling.

  See the functional test `interfaces_zmq.py::test_mempool_sync` which shows the proposed user flow for the client-side tracking of mempool contents and confirmations.

  Inspired by https://github.com/bitcoin/bitcoin/pull/19462#issuecomment-656140421
  Alternative to https://github.com/bitcoin/bitcoin/pull/19462 due to noted deficiencies in current zmq notification streams.

  Also fixes a legacy zmq test that didn't actually trigger a reorg because of identical blocks being generated on each side of the split(oops)

ACKs for top commit:
  laanwj:
    Code review ACK 759d94e70f

Tree-SHA512: 9daf0d7d996190f3a68ff40340a687519323d7a6c51dcb26be457fbc013217ea7b62fbd0700b74b654433d2e370704feb61e5584399290692464fcfcb72ce3b7
2020-09-23 13:55:24 +02:00
MarcoFalke
facaf9e61f
doc: Document signet BIP 2020-09-22 22:33:09 +02:00
Gregory Sanders
759d94e70f Update zmq notification documentation and sample consumer 2020-09-22 11:34:30 -04:00
Amiti Uttarwar
a512925e19 [doc] Release notes 2020-09-21 19:03:40 -07:00
Hennadii Stepanov
0bd1184adf
Remove unused LockAssertion struct 2020-09-19 18:02:42 +03:00
fanquake
c30f79d418
Merge #19940: rpc: Return fee and vsize from testmempoolaccept
23c35bf005 [test] add get_vsize util for more programmatic testing (gzhao408)
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept (codeShark149)

Pull request description:

  From #19093 and resolves #19057.

  Difference from #19093: return `vsize` and `fees` object (similar to `getmempoolentry`) when the test accept is successful. Updates release-notes.md.

ACKs for top commit:
  jnewbery:
    utACK 23c35bf005
  fjahr:
    utACK 23c35bf
  instagibbs:
    reACK 23c35bf005

Tree-SHA512: dcb81b7b817a4684e9076bc5d427a6f2d549d2edc66544e718260c4b5f8f1d5ae1d47b754175e9f0c8a3bd8371ce116c2dca0583588d513a7d733d5d614f2b04
2020-09-19 15:04:03 +08:00
Samuel Dobson
652c45fdbb
Merge #15454: Remove the automatic creation and loading of the default wallet
d26f0648f1 Tell users how to load or create a wallet when no wallet is loaded (Andrew Chow)
1bee1e6269 Do not create default wallet (Andrew Chow)

Pull request description:

  Instead of automatically creating and loading a default wallet, users should instead explicitly create their wallet or load it on start.

  Builds on #19754 which provides the `load_on_startup` behavior for the GUI.

ACKs for top commit:
  jnewbery:
    Manual test and very light code review ACK d26f0648f1
  ryanofsky:
    Code review ACK d26f0648f1. Just suggested changes to first commit (reusing MakeWalletDatabase and adding release notes), no changes to second commit
  jonatack:
    ACK d26f0648f1 light code review, debug build, ran tests, did manual testing with testnet, rebased on master, on linux debian.

Tree-SHA512: 091d785aef64736f7df661c576e815a87f3d029cfa32f3a75ba86fc25795f10b022ab3ae15c5b61a10b8cee16f5650f15cd79cbd6127e5e3ccbef631966d3c30
2020-09-18 12:03:55 +12:00
Sjors Provoost
efc9b85e6f
Mark send RPC experimental 2020-09-17 15:29:52 +02:00
codeShark149
2233a93a10 [rpc] Return fee and vsize from testmempoolaccept
Return fee and vsize if tx would pass ATMP.
2020-09-15 18:01:32 -07:00
fanquake
5d5e335f96
Merge #19903: Update build-openbsd.md with GUI support
d11020019a Add OpenBSD instructions for building the Qt GUI (grubles)

Pull request description:

  Using OpenBSD as a desktop OS is prevalent enough IMO to warrant updating the documentation for building the GUI.

ACKs for top commit:
  fanquake:
    ACK d11020019a - looks fine. Have not tested.

Tree-SHA512: a8078334fdd35438bcf87c3f5eae851c2a1ce961eb48ae50770bf2c556489da86b6ee198fe9fb732dcaddb2e0f2f4f55a3126971aae8f7d4e2e320dbb024e204
2020-09-15 12:05:21 +08:00
Samuel Dobson
ffaac6e614
Merge #16378: The ultimate send RPC
92326d8976 [rpc] add send method (Sjors Provoost)
2c2a1445dc [rpc] add snake case aliases for transaction methods (Sjors Provoost)
1bc8d0fd59 [rpc] walletcreatefundedpsbt: allow inputs to be null (Sjors Provoost)

Pull request description:

  `walletcreatefundedpsbt` has some interesting features that `sendtoaddress` and `sendmany` don't have:
  * manual coin selection
  * outputting a PSBT (it was controversial to add this, see #18201)
  * create a transaction without adding to wallet (which leads to broadcasting, unless `-walletbroadcast=0`)

  At the same time `walletcreatefundedpsbt` can't broadcast a transaction, which is inconvenient for simple use cases.

  This PR introduces a new `send` RPC method which creates a PSBT, signs it if possible and adds it to the wallet by default. If it can't sign all inputs, it outputs a PSBT. If `add_to_wallet` is set to `false` it will return the transaction in both PSBT and hex format.

  Because it uses a PSBT internally, it will much easier to add hardware wallet support to this method (see #16546).

  For `bitcoin-cli` users, it tries to keep the simplest use case easy to use:

  ```sh
  bitcoin-cli -regtest send '{"ADDRESS": 0.1}' 1 sat/b
  ```

  This paves the way for deprecating `sendtoaddress` and `sendmany` though there's no rush. The only missing feature compared to these older methods is adding labels to a destination address.

  Depends on:
  - [x] #16377 (`[rpc] don't automatically append inputs in walletcreatefundedpsbt`)
  - [x] #11413 (`[wallet] [rpc] sendtoaddress/sendmany: Add explicit feerate option`)
  - [x] #18244 (`[rpc] have lockUnspents also lock manually selected coins`)

ACKs for top commit:
  meshcollider:
    Light re-utACK 92326d8976
  achow101:
    ACK 92326d8976 Reviewed code and test, ran tests.
  kallewoof:
    utACK 92326d8976

Tree-SHA512: 7552ef1b193d4c06e381c44932fdb0d54f64383e4c7d6b988f49d059c7d4bba45ce6aa7813e03df86360ad9dad6f3010eb76ee7da480551742d5fd98c2251c0f
2020-09-15 14:49:08 +12:00
fanquake
dffefda21d
Merge #19870: doc: update PyZMQ install instructions, fix zmq_sub.py file permissions
062e6699c4 script: fix zmq_sub.py file permissions (Jon Atack)
36f8e0cce7 doc: update PyZMQ installation instructions, ZeroMQ link (Jon Atack)

Pull request description:

  Seen while reviewing #19572.

ACKs for top commit:
  theStack:
    ACK 062e6699c4 🧷
  fanquake:
    ACK 062e6699c4

Tree-SHA512: 2210d92385377d066984d0a83882c3ece9f0f41c901b7eb375af9cdb57296f50f227c68193ccf35926073c2b788d58976442791a9fce2fc0f76452804d5cee6a
2020-09-11 13:12:54 +08:00
grubles
d11020019a Add OpenBSD instructions for building the Qt GUI 2020-09-10 16:02:10 +02:00
Sjors Provoost
92326d8976
[rpc] add send method 2020-09-10 13:44:53 +02:00
Andrew Chow
1bee1e6269 Do not create default wallet
No longer create a default wallet. The default wallet will still be
loaded if it exists and not other wallets were specified (anywhere,
including settings.json, bitcoin.conf, and command line).

Tests are updated to be started with -wallet= if they need the default
wallet.

Added test to wallet_startup.py testing that no default wallet is
created and that it is loaded if it exists and no other wallets were
specified.
2020-09-08 21:02:53 -04:00
Jon Atack
36f8e0cce7
doc: update PyZMQ installation instructions, ZeroMQ link 2020-09-04 17:48:02 +02:00
Wladimir J. van der Laan
23d3ae7acc
Merge #19405: rpc, cli: add network in/out connections to getnetworkinfo and -getinfo
581b343d5b Add in/out connections to cli -getinfo (Jon Atack)
d9cc13e88d UNIX_EPOCH_TIME fixup in rpc getnettotals (Jon Atack)
1ab49b81cf Add in/out connections to rpc getnetworkinfo (Jon Atack)

Pull request description:

  This is basic info that is present in the GUI that I've been wishing to have exposed via the RPC and CLI without needing a bash workaround or script. For human users it would also be useful to have it in `-getinfo`.

  `bitcoin-cli getnetworkinfo`
  ```
    "connections": 15,
    "connections_in": 6,
    "connections_out": 9,
  ```

  `bitcoin-cli -getinfo`
  ```
    "connections": {
      "in": 6,
      "out": 9,
      "total": 15
    },
  ```

  Update the tests, RPC help, and release notes for the changes. Also fixup the `getnettotals` timemillis help while touching `rpc/net.cpp`.

  -----

  Reviewers can manually test this PR by [building from source](https://jonatack.github.io/articles/how-to-compile-bitcoin-core-and-run-the-tests), launching bitcoind, and then running `bitcoin-cli -getinfo`, `bitcoin-cli getnetworkinfo`, `bitcoin-cli help getnetworkinfo`, and `bitcoin-cli help getnettotals` (for the UNIX epoch time change).

ACKs for top commit:
  eriknylund:
    > tACK [581b343](581b343d5b) on master at [a0a422c](a0a422c34c), ran unit & functional tests and and confirmed changes on an existing datadir ✌️
  benthecarman:
    tACK `581b343`
  willcl-ark:
    tACK for 581b343d5b, this time rebased onto master at 862fde88be.
  shesek:
    tACK `581b343`. This provides what I needed, thanks!
  n-thumann:
    tACK 581b343 on master at a0a422c, ran unit & functional tests and and confirmed changes on an existing datadir ✌️

Tree-SHA512: 08dd3ac8fefae401bd8253ff3ac027603c528eeccba53cedcb127771316173a7052fce44af8fa33ac98ebc4cf2a2b11cdefd949995d55e9b9a5942b876d00dc5
2020-09-04 15:09:37 +02:00