Commit graph

25600 commits

Author SHA1 Message Date
fanquake
92735e45ba
Merge #19775: test: Activate segwit in TestChain100Setup
fad84b7e14 test: Activate segwit in TestChain100Setup (MarcoFalke)
fa11ff2980 test: Pass empty tx pool to block assembler (MarcoFalke)
fa96574b0d test: Move doxygen comment to header (MarcoFalke)

Pull request description:

  This fixes not only a TODO in the code, but also prevents a never ending source of uninitialized reads. E.g.

  * #18376
  * https://github.com/bitcoin/bitcoin/pull/19704#issuecomment-678259092
  * ...

ACKs for top commit:
  jnewbery:
    utACK fad84b7e14

Tree-SHA512: 64cf16a59656d49e022b603f3b06441ceae35a33a4253b4382bc8a89a56e08ad5412c8fa734d0fc7b58586f40ea6d57b348a3b4838bc6890a41ae2ec3902e378
2020-08-26 13:17:35 +08:00
fanquake
3de365e4f1
build: replace wingenminiupnpcstrings sed with a patch in miniupnpc package
We should be able to drop this once we are using 2.1 or later. See
upstream commit: 9663c55c61408fdcc39a82987d2243f816b22932.
2020-08-26 11:27:31 +08:00
fanquake
bbc01a753d
build: replace qtranslations lrelease sed with a patch in qt package 2020-08-26 11:27:12 +08:00
Andrew Chow
0bbe26a1af wallet: filter for keys only before record deser in salvage
When salvaging a wallet, avoid deserializing any records that we don't
care about, i.e. filter for keys only before the deserialization.
2020-08-25 13:23:40 -04:00
Andrew Chow
544e12a4e8 walletdb: Add KeyFilterFn to ReadKeyValue
Add a KeyFilterFn callback to ReadKeyValue which allows the caller to
specify which types to actually deserialize. A KeyFilterFn takes the
type as the parameter and returns a bool indicating whether
deserialization should continue.
2020-08-25 13:23:40 -04:00
Sebastian Falbesoner
e2aa1a585a util: make EncodeBase64 consume Spans 2020-08-25 18:52:57 +02:00
Sebastian Falbesoner
2bc207190e util: make EncodeBase32 consume Spans 2020-08-25 18:52:51 +02:00
Luke Dashjr
c4b85ba704 Bugfix: Define and use HAVE_FDATASYNC correctly outside LevelDB 2020-08-25 16:46:00 +00:00
MarcoFalke
8d6224fefe
Merge #19628: net: change CNetAddr::ip to have flexible size
102867c587 net: change CNetAddr::ip to have flexible size (Vasil Dimov)
1ea57ad674 net: don't accept non-left-contiguous netmasks (Vasil Dimov)

Pull request description:

  (chopped off from #19031 to ease review)

  Before this change `CNetAddr::ip` was a fixed-size array of 16 bytes,
  not being able to store larger addresses (e.g. TORv3) and encoded
  smaller ones as 16-byte IPv6 addresses.

  Change its type to `prevector`, so that it can hold larger addresses and
  do not disguise non-IPv6 addresses as IPv6. So the IPv4 address
  `1.2.3.4` is now encoded as `01020304` instead of
  `00000000000000000000FFFF01020304`.

  Rename `CNetAddr::ip` to `CNetAddr::m_addr` because it is not an "IP" or
  "IP address" (TOR addresses are not IP addresses).

  In order to preserve backward compatibility with serialization (where
  e.g. `1.2.3.4` is serialized as `00000000000000000000FFFF01020304`)
  introduce `CNetAddr` dedicated legacy serialize/unserialize methods.

  Adjust `CSubNet` accordingly. Still use `CSubNet::netmask[]` of fixed 16
  bytes, but use the first 4 for IPv4 (not the last 4). Do not accept
  invalid netmasks that have 0-bits followed by 1-bits and only allow
  subnetting for IPv4 and IPv6.

  Co-authored-by: Carl Dong <contact@carldong.me>

ACKs for top commit:
  sipa:
    utACK 102867c587
  MarcoFalke:
    Concept ACK 102867c587
  ryanofsky:
    Code review ACK 102867c587. Just many suggested updates since last review. Thanks for following up on everything!
  jonatack:
    re-ACK 102867c587 diff review, code review, build/tests/running bitcoind with ipv4/ipv6/onion peers
  kallewoof:
    ACK 102867c587

Tree-SHA512: d60bf716cecf8d3e8146d2f90f897ebe956befb16f711a24cfe680024c5afc758fb9e4a0a22066b42f7630d52cf916318bedbcbc069ae07092d5250a11e8f762
2020-08-25 18:10:25 +02:00
fanquake
f8462a6d27
Merge #19601: Refactoring CHashWriter & Get{Prevouts,Sequence,Outputs}Hash to SHA256 (Alternative to #18071)
9ab4cafabd Refactor Get{Prevout,Sequence,Outputs}Hash to Get{Prevouts,Sequences,Outputs}SHA256. (Jeremy Rubin)
6510d0ff41 Add SHA256Uint256 helper functions (Jeremy Rubin)
b475d7d0fa Add single sha256 call to CHashWriter (Jeremy Rubin)

Pull request description:

  Opened as an alternative to #18071 to be more similar to #17977.

  I'm fine with either, deferring to others.

  cc jnewbery Sjors

ACKs for top commit:
  jnewbery:
    Code review ACK 9ab4cafabd
  jonatack:
    Tested ACK 9ab4caf
  fjahr:
    tested ACK 9ab4cafabd
  instagibbs:
    reACK 9ab4cafabd

Tree-SHA512: 93a7a47697f1657f027b18407bdcce16963f6b23d12372e7ac8fd4ee96769b3e2639369f9956fee669cc881b6338641cddfeeef1516c7104cb50ef4b880bb0a7
2020-08-25 20:18:40 +08:00
John Newbery
d5800da519 [test] Remove final references to mininode 2020-08-25 10:04:25 +01:00
fanquake
c723e4176e
build: replace FreeType back-compat sed with a patch in qt package 2020-08-25 14:37:45 +08:00
fanquake
3aaa39d436
build: replace pwd sed in qt package with a patch 2020-08-25 14:37:45 +08:00
fanquake
9d440f4e11
build: remove no-longer needed qt workaround 2020-08-25 14:37:45 +08:00
fanquake
bf85eace1a
build: remove no-longer needed qt configure workaround
This was fixed upstream in c45595d64831990311f92fcebc4e34e2797f5352.
2020-08-25 14:37:44 +08:00
fanquake
4af59a407a
build: use patch rather than sed in zeromq package 2020-08-25 14:37:44 +08:00
fanquake
cc107a3af1
build: use patch rather than sed in native_cctools package 2020-08-25 14:37:44 +08:00
fanquake
865cb23a48
build: use patch rather than sed in fontconfig package 2020-08-25 14:37:35 +08:00
fanquake
335bd7f8bc
build: use patch rather than sed in Boost package
The depends comment is actually incorrect, and this can be dropped once
we move to 1.71.0 or later.
2020-08-25 13:19:56 +08:00
fanquake
f36140d00c
build: use patch rather than sed in bdb package 2020-08-25 13:19:55 +08:00
fanquake
8e0f341779
Merge #15704: Move Win32 defines to configure.ac to ensure they are globally defined
1ccb9f30c0 Move Win32 defines to configure.ac to ensure they are globally defined (Luke Dashjr)

Pull request description:

  #9245 no longer needs this, since the main `_WIN32_WINNT` got bumped by something else.

  So rather than just lose it, might as well get it merged in independently.

  I'm not aware of any practical effects, but it seems safer to use the same API versions everywhere.

ACKs for top commit:
  fanquake:
    ACK 1ccb9f30c0 - checked that the binaries produced are the same.

Tree-SHA512: 273e9186579197be01b443b6968e26b9a8031d356fabc5b73aa967fcdb837df195b7ce0fc4e4529c85d9b86da6f2d7ff1bf56a3ff0cbbcd8cee8a9c2bf70a244
2020-08-25 11:52:52 +08:00
fanquake
c6b730dbfc
Merge #18405: build: Drop all of the ZeroMQ patches
f642b49af7 build: Drop ZeroMQ patch for glibc < 2.12 (Hennadii Stepanov)
079df9609e build: Drop ZeroMQ patch for Mingw-w64 < 4.0 (Hennadii Stepanov)

Pull request description:

  This PR gets rid of the all patches for ZeroMQ:
  - the [Mingw-w64 5.0 (Ubuntu 18.04 bionic)](https://packages.ubuntu.com/bionic/mingw-w64) is used to build the [Windows](https://github.com/bitcoin/bitcoin/blob/master/doc/build-windows.md) binaries
  - it is safe to use `pthread_setname_np` since #17538 when the minimal `glibc` version is set to 2.17; see: https://github.com/bitcoin/bitcoin/pull/11986#issuecomment-366105050

ACKs for top commit:
  fanquake:
    ACK f642b49af7.

Tree-SHA512: a2c97cdb682cd7d96a666ad099f20725a32bf8fda0842fc5534ca08a1634c90ba80afde92f9054595ed2501fbc5c02abbe3da765934ecff12d836ff25e277fc5
2020-08-25 10:26:16 +08:00
Vasil Dimov
102867c587
net: change CNetAddr::ip to have flexible size
Before this change `CNetAddr::ip` was a fixed-size array of 16 bytes,
not being able to store larger addresses (e.g. TORv3) and encoded
smaller ones as 16-byte IPv6 addresses.

Change its type to `prevector`, so that it can hold larger addresses and
do not disguise non-IPv6 addresses as IPv6. So the IPv4 address
`1.2.3.4` is now encoded as `01020304` instead of
`00000000000000000000FFFF01020304`.

Rename `CNetAddr::ip` to `CNetAddr::m_addr` because it is not an "IP" or
"IP address" (TOR addresses are not IP addresses).

In order to preserve backward compatibility with serialization (where
e.g. `1.2.3.4` is serialized as `00000000000000000000FFFF01020304`)
introduce `CNetAddr` dedicated legacy serialize/unserialize methods.

Adjust `CSubNet` accordingly. Still use `CSubNet::netmask[]` of fixed 16
bytes, but use the first 4 for IPv4 (not the last 4). Only allow
subnetting for IPv4 and IPv6.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-08-24 21:50:59 +02:00
Vasil Dimov
1ea57ad674
net: don't accept non-left-contiguous netmasks
A netmask that contains 1-bits after 0-bits (the 1-bits are not
contiguous on the left side) is invalid [1] [2].

The code before this PR used to parse and accept such
non-left-contiguous netmasks. However, a coming change that will alter
`CNetAddr::ip` to have flexible size would make juggling with such
netmasks more difficult, thus drop support for those.

[1] https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#Subnet_masks
[2] https://tools.ietf.org/html/rfc4632#section-5.1
2020-08-24 21:50:59 +02:00
Wladimir J. van der Laan
7f609f68d8
Merge #19731: net, rpc: expose nLastBlockTime/nLastTXTime as last block/last_transaction in getpeerinfo
5da96210fc doc: release note for getpeerinfo last_block/last_transaction (Jon Atack)
cfef5a2c98 test: rpc_net.py logging and test naming improvements (Jon Atack)
21c57bacda test: getpeerinfo last_block and last_transaction tests (Jon Atack)
8a560a7d57 rpc: expose nLastBlockTime/TXTime as getpeerinfo last_block/transaction (Jon Atack)
02fbe3ae0b net: add nLastBlockTime/TXTime to CNodeStats, CNode::copyStats (Jon Atack)

Pull request description:

  This PR adds inbound peer eviction criteria `nLastBlockTime` and `nLastTXTime` to `CNodeStats` and `CNode::copyStats`, which then allows exposing them in the next commit as `last_transaction` and `last_block` Unix Epoch Time fields in RPC `getpeerinfo`.

  This may be useful for writing missing eviction tests. I'd also like to add `lasttx` and `lastblk` columns to the `-netinfo` dashboard as described in https://github.com/bitcoin/bitcoin/pull/19643#issuecomment-671093420.

  Relevant discussion at the p2p irc meeting http://www.erisian.com.au/bitcoin-core-dev/log-2020-08-11.html#l-549:
  ```text
  <jonatack> i was specifically trying to observe and figure out how to test https://github.com/bitcoin/bitcoin/issues/19500
  <jonatack> which made me realise that i didn't know what was going on with my peer conns in enough detail
  <jonatack> i'm running bitcoin locally with nLastBlockTime and nLastTXTime added to getpeerinfo for my peer connections dashboard
  <jonatack> sipa: is there a good reason why that (eviction criteria) data is not exposed through getpeerinfo currently?
  <sipa> jonatack: nope; i suspect just nobody ever added it
  <jonatack> sipa: thanks. will propose.
  ```

  The last commit is optional, but I think it would be good to have logging in `rpc_net.py`.

ACKs for top commit:
  jnewbery:
    Code review ACK 5da96210fc
  theStack:
    Code Review ACK 5da96210fc
  darosior:
    ACK 5da96210fc

Tree-SHA512: 2db164bc979c014837a676e890869a128beb7cf40114853239e7280f57e768bcb43bff6c1ea76a61556212135281863b5290b50ff9d24fce16c5b89b55d4cd70
2020-08-24 17:03:07 +02:00
fanquake
4fefd80f08
Merge #19704: Net processing: move ProcessMessage() to PeerLogicValidation
daed542a12 [net_processing] Move ProcessMessage to PeerLogicValidation (John Newbery)
c556770b5e [net_processing] Change PeerLogicValidation to hold a connman reference (John Newbery)

Pull request description:

  Rather than ProcessMessage() being a static function in net_processing.cpp, make it a private member function of PeerLogicValidation. This is the start of moving static functions and global variables into PeerLogicValidation to make it better encapsulated.

ACKs for top commit:
  jonatack:
    ACK daed542a12 code review and debug tested
  promag:
    Code review ACK daed542a12.
  MarcoFalke:
    re-ACK daed542a12, only change is removing second commit 🎴
  theStack:
    Code Review ACK daed542a12

Tree-SHA512: ddebf410d114d9ad5a9e536950018ff333a347c035d74fcc101fb4a3f20a281782c7eac2b7d1bd1c8f6bc7e59f5b5630fb52c2e1b4c32df454fa584673bd021e
2020-08-24 21:50:37 +08:00
fanquake
7ce7f2b251
Merge #17396: build: modest Android improvements
366913e307 build: AX_BOOST_THREAD serial 33 (Igor Cota)
cf0681133a build: disable D-Bus on Android by default (Igor Cota)

Pull request description:

  I've been trying to build for Android on different OSes/Gitian with varying success. Build system is quite the beast and sometimes it doesn't get it right. To make sure it does these three little tweaks make the Android build more robust:

  - disable D-Bus (Android doesn't support it and has its own way to trigger notifications)
  - don't flag `-lpthread` when linking Boost, [Bionic has built-in support](https://stackoverflow.com/questions/30801752/android-ndk-and-pthread)
  - ~~add `-static-libstdc++` to linker flags. This avoids having to bundle `libc++_shared` with CLI apps, still necessary with `bitcoin-qt` though (thanks Sjors)~~

  I think these are small and fairly straightforward so I put them all into this one PR.

ACKs for top commit:
  fanquake:
    ACK 366913e307

Tree-SHA512: 31465fd228a5877c20aa2a05f98242d4eeb328b9b35bd1a7a3dcfb1ef51379d84053a81ade5a65436ffc1bc8ccd21f11ed0539eb10e827d182c0c04394629af0
2020-08-24 21:27:29 +08:00
MarcoFalke
fafc9d5af4
test: Fix intermittent issue in wallet_bumpfee 2020-08-24 13:49:28 +02:00
MarcoFalke
fa347b2f25
test: Select at least the fee in wallet_bumpfee to avoid negative amounts 2020-08-24 13:44:07 +02:00
fanquake
8e94275f86
Merge #19786: doc: Remove label from good first issue template
fa30d5282c doc: Remove label from good first issue template (MarcoFalke)

Pull request description:

  Good first issues aren't that frequent that manually assigning the label is a problem, but this fixes the spam problem (e.g. https://twitter.com/GoodFirstIssues/status/1295455089491161088 )

ACKs for top commit:
  jnewbery:
    ACK fa30d5282c

Tree-SHA512: 59e7c707637cc328e2443c2b7e5d2c82ef151739ad5afb6cea1a60501318dc8c4c81c95591eed8172581ac99d43cf826dcdd547e096eff1038137853af67a975
2020-08-24 19:32:44 +08:00
MarcoFalke
fa30d5282c
doc: Remove label from good first issue template 2020-08-24 09:31:24 +02:00
MarcoFalke
1d53d72948
Merge #19659: Add a seed corpus generation option to the fuzzing test_runner
15ae4a17c4 test/fuzz: add a seed corpus generation option to the test_runner (Antoine Poinsot)

Pull request description:

  This adds a startup option to test/fuzz/test_runner.py which allows to generate seed corpus to the passed `seed_dir` instead of using them.

ACKs for top commit:
  MarcoFalke:
    ACK 15ae4a17c4

Tree-SHA512: f80ad58e48cc45272eace33dbf377848f31cbd6a25433786d50e9700f70185dff6513f71d885d0727ed57a2aa49163bfbdbc51a8091e99b4b1bae71e1504e6a5
2020-08-24 08:02:27 +02:00
MarcoFalke
fa9d5902f7
scripted-diff: gArgs -> args
-BEGIN VERIFY SCRIPT-
 # Replace gArgs with args
 sed -i 's/\<gArgs\>/args/g' src/init.cpp src/bitcoind.cpp
 sed -i 's/&args;/\&gArgs;/g' src/init.cpp

 # Format changed lines
 git diff -U0 | clang-format-diff -p1 -i -v
-END VERIFY SCRIPT-
2020-08-24 07:52:17 +02:00
MarcoFalke
fa33bc2dab
init: Capture copy of blocknotify setting for BlockNotifyCallback
Can be reviewed with --color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2020-08-24 07:51:48 +02:00
MarcoFalke
fa40017706
init: Pass reference to ArgsManager around instead of relying on global 2020-08-24 07:45:17 +02:00
fanquake
38c13a4a60
Merge #19689: build: Add Qt version checking
4af4672525 build, qt: Add Qt version checking (Hennadii Stepanov)
30e336f785 build: Drop unused bitcoin_cv_qt58 (Hennadii Stepanov)

Pull request description:

  Now `configure` script checks that Qt version is not less then minimum required (currently [5.5.1](https://github.com/bitcoin/bitcoin/pull/15393)).

  This PR is an alternative to #15706 (see https://github.com/bitcoin/bitcoin/pull/15706#issuecomment-629076962).

  Closes #15688.

  The first commit removes dead code (see https://github.com/bitcoin/bitcoin/pull/18297#issuecomment-603252662).

ACKs for top commit:
  fanquake:
    ACK 4af4672525 - this looks ok. I've tested this with Qt 5.15.0 and Qt 5.7.1 system libs, as well as 5.9.8 from depends.

Tree-SHA512: 8e3b82fa3a98926814923331038185633fabad962c271f31bd158e1ab293dcde52ab1dbf997745540a9ed27e16835cf5b5f3701d405876d877fa561eb03cc619
2020-08-24 12:14:18 +08:00
Antoine Poinsot
15ae4a17c4
test/fuzz: add a seed corpus generation option to the test_runner
Signed-off-by: Antoine Poinsot <darosior@protonmail.com>
2020-08-23 23:09:26 +02:00
MarcoFalke
197450f808
Merge #19777: docs: Correct description for getblockstats's txs field
4148f55dd0 docs: Correct description for getblockstats's txs field (Nadav Ivgi)

Pull request description:

  It does count the coinbase transaction.

  Refs #19766

ACKs for top commit:
  MarcoFalke:
    ACK 4148f55dd0
  theStack:
    ACK 4148f55dd0

Tree-SHA512: ccd420f19242efbbbecfe822c825363bc89e26618834de0d805f5cdb07461c8bdc6e077c61ea8cd0d40564a96c67d8a71c68175c8543bb849909d7ae375b2a92
2020-08-22 20:50:14 +02:00
MarcoFalke
fad84b7e14
test: Activate segwit in TestChain100Setup 2020-08-21 18:44:52 +02:00
MarcoFalke
fa11ff2980
test: Pass empty tx pool to block assembler 2020-08-21 18:44:50 +02:00
MarcoFalke
fa96574b0d
test: Move doxygen comment to header
Also, unrelated formatting fixups.

Can be reviewed with --word-diff-regex=.
2020-08-21 18:44:27 +02:00
John Newbery
5e8df3312e test: resort imports 2020-08-21 15:53:59 +01:00
John Newbery
85165d4332 scripted-diff: Rename mininode to p2p
-BEGIN VERIFY SCRIPT-
sed -i 's/\.mininode/\.p2p/g' $(git grep -l "mininode")
git mv test/functional/test_framework/mininode.py test/functional/test_framework/p2p.py
-END VERIFY SCRIPT-
2020-08-21 15:52:20 +01:00
John Newbery
9e2897d020 scripted-diff: Rename mininode_lock to p2p_lock
-BEGIN VERIFY SCRIPT-
sed -i 's/mininode_lock/p2p_lock/g' $(git grep -l "mininode_lock")
-END VERIFY SCRIPT-
2020-08-21 15:52:13 +01:00
Nadav Ivgi
4148f55dd0
docs: Correct description for getblockstats's txs field
It does count the coinbase transaction.

Refs #19766
2020-08-21 17:41:02 +03:00
MarcoFalke
d254e6e795
Merge #19722: test: Add test for getblockheader verboseness
5067c5acc3 [test] Add test for getblockheader verboseness (Torhte Butler)

Pull request description:

  Improve test coverage by adding a test for getblockheader with verbose argument set to false.

ACKs for top commit:
  theStack:
    ACK 5067c5acc3

Tree-SHA512: e55593f1026a89dc7b796fa985b4cbcdb596e91d80d42dfb0660bda1692aaa35749ec29f9cd7032803f6225afb323f085df1ef6a9982de87be8e098f7253cdd5
2020-08-21 14:34:02 +02:00
John Newbery
daed542a12 [net_processing] Move ProcessMessage to PeerLogicValidation 2020-08-21 13:10:41 +01:00
fanquake
0d9e14a646
Merge #19733: Move comment about BaseIndex::DB from TxIndex::DB
8ed2f1ed78 Remove unused includes (Marcin Jachymiak)
cf095a53fc Move comment about BaseIndex::DB from TxIndex::DB (Marcin Jachymiak)

Pull request description:

  Moves a comment about the `BaseIndex::DB` from the `TxIndex::DB` into the correct place. Originally part of https://github.com/bitcoin/bitcoin/pull/14053.

ACKs for top commit:
  fanquake:
    ACK 8ed2f1ed78

Tree-SHA512: cb4e2b916c7ab996961cc2e1d910bc4b8a1700eb32b70fc1657ca720117a7a84f7337fe5e4fb30e047aa92c31eaa976eaaa5cb8f861877f2ff6f4a59bb94f4e9
2020-08-21 12:48:46 +08:00
João Barbosa
b8405b833a wallet: IsChange requires cs_wallet lock 2020-08-21 00:28:10 +01:00
Luke Dashjr
1ccb9f30c0 Move Win32 defines to configure.ac to ensure they are globally defined
common.vcxproj used for MSVC builds
2020-08-20 17:55:06 +00:00