Commit graph

56 commits

Author SHA1 Message Date
Vasil Dimov
1ea57ad674
net: don't accept non-left-contiguous netmasks
A netmask that contains 1-bits after 0-bits (the 1-bits are not
contiguous on the left side) is invalid [1] [2].

The code before this PR used to parse and accept such
non-left-contiguous netmasks. However, a coming change that will alter
`CNetAddr::ip` to have flexible size would make juggling with such
netmasks more difficult, thus drop support for those.

[1] https://en.wikipedia.org/wiki/Classless_Inter-Domain_Routing#Subnet_masks
[2] https://tools.ietf.org/html/rfc4632#section-5.1
2020-08-24 21:50:59 +02:00
Wladimir J. van der Laan
14ceddd290
Merge #18991: Cache responses to GETADDR to prevent topology leaks
3bd67ba5a4 Test addr response caching (Gleb Naumenko)
cf1569e074 Add addr permission flag enabling non-cached addr sharing (Gleb Naumenko)
acd6135b43 Cache responses to addr requests (Gleb Naumenko)
7cc0e8101f Remove useless 2500 limit on AddrMan queries (Gleb Naumenko)
ded742bc5b Move filtering banned addrs inside GetAddresses() (Gleb Naumenko)

Pull request description:

  This is a very simple code change with a big p2p privacy benefit.

  It’s currently trivial to scrape any reachable node’s AddrMan (a database of all nodes known to them along with the timestamps).
  We do have a limit of one GETADDR per connection, but a spy can disconnect and reconnect even from the same IP, and send GETADDR again and again.

  Since we respond with 1,000 random records at most, depending on the AddrMan size it takes probably up to 100 requests for an spy to make sure they scraped (almost) everything.
  I even have a script for that. It is totally doable within couple minutes.

  Then, with some extra protocol knowledge a spy can infer the direct peers of the victim, and other topological stuff.

  I suggest to cache responses to GETADDR on a daily basis, so that an attacker gets at most 1,000 records per day, and can’t track the changes in real time. I will be following up with more improvements to addr relay privacy, but this one alone is a very effective. And simple!

  I doubt any of the real software does *reconnect to get new addrs from a given peer*, so we shouldn’t be cutting anyone.
  I also believe it doesn’t have any negative implications on the overall topology quality. And the records being “outdated” for at most a day doesn’t break any honest assumptions either.

ACKs for top commit:
  jnewbery:
    reACK 3bd67ba5a4
  promag:
    Code review ACK 3bd67ba5a4.
  ariard:
    Code Review ACK 3bd67ba

Tree-SHA512: dfa5d03205c2424e40a3f8a41af9306227e1ca18beead3b3dda44aa2a082175bb1c6d929dbc7ea8e48e01aed0d50f0d54491caa1147471a2b72a46c3ca06b66f
2020-08-03 14:48:52 +02:00
Gleb Naumenko
cf1569e074 Add addr permission flag enabling non-cached addr sharing 2020-07-30 14:38:50 +03:00
Vasil Dimov
bcfebb6d55
net: save the network type explicitly in CNetAddr
Before this change, we would analyze the contents of `CNetAddr::ip[16]`
in order to tell which type is an address. Change this by introducing a
new member `CNetAddr::m_net` that explicitly tells the type of the
address.

This is necessary because in BIP155 we will not be able to tell the
address type by just looking at its raw representation (e.g. both TORv3
and I2P are "seemingly random" 32 bytes).

As a side effect of this change we no longer need to store IPv4
addresses encoded as IPv6 addresses - we can store them in proper 4
bytes (will be done in a separate commit). Also the code gets
somewhat simplified - instead of
`memcmp(ip, pchIPv4, sizeof(pchIPv4)) == 0` we can use
`m_net == NET_IPV4`.

Co-authored-by: Carl Dong <contact@carldong.me>
2020-07-27 15:13:24 +02:00
MarcoFalke
fab5586122
doc: Use precise permission flags where possible 2020-07-10 15:37:42 +02:00
MarcoFalke
fa0540cd46
net: Extract download permission from noban 2020-07-09 12:48:05 +02:00
Vasil Dimov
ccef5d7bf0
test: add two edge case tests for CSubNet 2020-06-22 15:30:31 +02:00
Wladimir J. van der Laan
77b79fa6ef refactor: Error message bilingual_str consistency
- Move the decision whether to translate an error message to where it is
  defined. This simplifies call sites: no more `InitError(Untranslated(...))`.

- Make all functions in `util/error.h` consistently return a
  `bilingual_str`. We've decided to use this as error message type so
  let's roll with it.

This has no functional changes: no messages are changed, no new
translation messages are defined.
2020-06-09 15:39:44 +02:00
Ben Woosley
df37377e30
test: Fix outstanding -Wsign-compare errors 2020-05-08 11:18:43 -07:00
MarcoFalke
fa488f131f
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-
2020-04-16 13:33:09 -04:00
MarcoFalke
fac5c37300
scripted-diff: Sort test includes
-BEGIN VERIFY SCRIPT-
 # Mark all lines with #includes
 sed -i --regexp-extended -e 's/(#include <.*>)/\1 /g' $(git grep -l '#include' ./src/bench/ ./src/test ./src/wallet/test/)
 # Sort all marked lines
 git diff -U0 | ./contrib/devtools/clang-format-diff.py -p1 -i -v
-END VERIFY SCRIPT-
2020-04-16 13:32:36 -04:00
Wladimir J. van der Laan
01fc5891fb
Merge #16702: p2p: supplying and using asmap to improve IP bucketing in addrman
3c1bc40205 Add extra logging of asmap use and bucketing (Gleb Naumenko)
e4658aa8ea Return mapped AS in RPC call getpeerinfo (Gleb Naumenko)
ec45646de9 Integrate ASN bucketing in Addrman and add tests (Gleb Naumenko)
8feb4e4b66  Add asmap utility which queries a mapping (Gleb Naumenko)

Pull request description:

  This PR attempts to solve the problem explained in #16599.
  A particular attack which encouraged us to work on this issue is explained here  [[Erebus Attack against Bitcoin Peer-to-Peer Network](https://erebus-attack.comp.nus.edu.sg/)] (by @muoitranduc)

  Instead of relying on /16 prefix to diversify the connections every node creates, we would instead rely on the (ip -> ASN) mapping, if this mapping is provided.

  A .map file can be created by every user independently based on a router dump, or provided along with the Bitcoin release. Currently we use the python scripts written by @sipa to create a .map file, which is no larger than 2MB (awesome!).

  Here I suggest adding a field to peers.dat which would represent a hash of asmap file used while serializing addrman (or 0 for /16 prefix legacy approach).
  In this case, every time the file is updated (or grouping method changed), all buckets will be re-computed.
  I believe that alternative selective re-bucketing for only updated ranges would require substantial changes.

  TODO:
  - ~~more unit tests~~
  - ~~find a way to test the code without including >1 MB mapping file in the repo.~~
  - find a way to check that mapping file is not corrupted (checksum?)
  - comments and separate tests for asmap.cpp
  - make python code for .map generation public
  - figure out asmap distribution (?)

  ~Interesting corner case: I’m using std::hash to compute a fingerprint of asmap, and std::hash returns size_t. I guess  if a user updates the OS to 64-bit, then the hash of asap will change? Does it even matter?~

ACKs for top commit:
  laanwj:
    re-ACK 3c1bc40205
  jamesob:
    ACK 3c1bc40205 ([`jamesob/ackr/16702.3.naumenkogs.p2p_supplying_and_using`](https://github.com/jamesob/bitcoin/tree/ackr/16702.3.naumenkogs.p2p_supplying_and_using))
  jonatack:
    ACK 3c1bc40205

Tree-SHA512: e2dc6171188d5cdc2ab2c022fa49ed73a14a0acb8ae4c5ffa970172a0365942a249ad3d57e5fb134bc156a3492662c983f74bd21e78d316629dcadf71576800c
2020-01-29 13:55:43 +01:00
practicalswift
7a046cdc14 tests: Avoid using C-style NUL-terminated strings as arguments 2020-01-08 12:35:59 +00:00
practicalswift
fefb9165f2 tests: Add tests to make sure lookup methods fail on std::string parameters with embedded NUL characters 2020-01-08 12:35:59 +00:00
practicalswift
9574de86ad net: Avoid using C-style NUL-terminated strings as arguments in the netbase interface 2020-01-08 12:35:59 +00:00
Gleb Naumenko
ec45646de9 Integrate ASN bucketing in Addrman and add tests
Instead of using /16 netgroups to bucket nodes in Addrman for connection
diversification, ASN, which better represents an actor in terms
of network-layer infrastructure, is used.
For testing, asmap.raw is used. It represents a minimal
asmap needed for testing purposes.
2019-12-25 08:59:08 -05:00
Mark Tyneway
529d332fbf
test: add IsRFC2544 tests 2019-12-16 19:43:15 -08:00
MarcoFalke
faec28252c
scripted-diff: test: Move setup_common to test library
-BEGIN VERIFY SCRIPT-
 # Move files
 for f in $(git ls-files src/test/lib/); do git mv $f src/test/util/; done
 git mv src/test/setup_common.cpp                     src/test/util/
 git mv src/test/setup_common.h                       src/test/util/
 # Replace Windows paths
 sed -i -e 's|\\setup_common|\\util\\setup_common|g' $(git grep -l '\\setup_common')
 sed -i -e 's|src\\test\\lib\\|src\\test\\util\\|g'  build_msvc/test_bitcoin/test_bitcoin.vcxproj
 # Everything else
 sed -i -e 's|/setup_common|/util/setup_common|g'    $(git grep -l 'setup_common')
 sed -i -e 's|test/lib/|test/util/|g'                $(git grep -l 'test/lib/')
 # Fix include guard
 sed -i -e 's|BITCOIN_TEST_SETUP_COMMON_H|BITCOIN_TEST_UTIL_SETUP_COMMON_H|g' ./src/test/util/setup_common.h
 sed -i -e 's|BITCOIN_TEST_LIB_|BITCOIN_TEST_UTIL_|g'                     $(git grep -l 'BITCOIN_TEST_LIB_')
-END VERIFY SCRIPT-
2019-11-06 11:56:41 -05:00
nicolas.dorier
e5b26deaaa
Make whitebind/whitelist permissions more flexible 2019-08-11 11:33:27 +09:00
Carl Dong
8be3f30633 netaddress: Update CNetAddr for ORCHIDv2
The original ORCHID prefix was deprecated as of 2014-03, the new
ORCHIDv2 prefix was allocated by RFC7343 as of 2014-07. We did not
consider the original ORCHID prefix routable, and I don't see any reason
to consider the new one to be either.
2019-05-15 14:21:48 -04:00
MarcoFalke
faf400077d
scripted-diff: Bump copyright headers in test, bench
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./src/bench/
./contrib/devtools/copyright_header.py update ./src/test/
-END VERIFY SCRIPT-
2019-04-11 13:34:43 -04:00
MarcoFalke
fa821904bf
scripted-diff: Rename test_bitcoin to test/setup_common
-BEGIN VERIFY SCRIPT-
sed -i --regexp-extended -e 's/test_bitcoin\.(h|cpp)/setup_common.\1/g' $(git grep -l test_bitcoin)
git mv ./src/test/test_bitcoin.h   ./src/test/setup_common.h
git mv ./src/test/test_bitcoin.cpp ./src/test/setup_common.cpp
sed -i -e 's/BITCOIN_TEST_TEST_BITCOIN_H/BITCOIN_TEST_SETUP_COMMON_H/g' ./src/test/setup_common.h
-END VERIFY SCRIPT-
2019-04-11 10:12:36 -04:00
Jim Posen
2068f089c8 scripted-diff: Move util files to separate directory.
-BEGIN VERIFY SCRIPT-
mkdir -p src/util
git mv src/util.h src/util/system.h
git mv src/util.cpp src/util/system.cpp
git mv src/utilmemory.h src/util/memory.h
git mv src/utilmoneystr.h src/util/moneystr.h
git mv src/utilmoneystr.cpp src/util/moneystr.cpp
git mv src/utilstrencodings.h src/util/strencodings.h
git mv src/utilstrencodings.cpp src/util/strencodings.cpp
git mv src/utiltime.h src/util/time.h
git mv src/utiltime.cpp src/util/time.cpp

sed -i 's/<util\.h>/<util\/system\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utilmemory\.h>/<util\/memory\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utilmoneystr\.h>/<util\/moneystr\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utilstrencodings\.h>/<util\/strencodings\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')
sed -i 's/<utiltime\.h>/<util\/time\.h>/g' $(git ls-files 'src/*.h' 'src/*.cpp')

sed -i 's/BITCOIN_UTIL_H/BITCOIN_UTIL_SYSTEM_H/g' src/util/system.h
sed -i 's/BITCOIN_UTILMEMORY_H/BITCOIN_UTIL_MEMORY_H/g' src/util/memory.h
sed -i 's/BITCOIN_UTILMONEYSTR_H/BITCOIN_UTIL_MONEYSTR_H/g' src/util/moneystr.h
sed -i 's/BITCOIN_UTILSTRENCODINGS_H/BITCOIN_UTIL_STRENCODINGS_H/g' src/util/strencodings.h
sed -i 's/BITCOIN_UTILTIME_H/BITCOIN_UTIL_TIME_H/g' src/util/time.h

sed -i 's/ util\.\(h\|cpp\)/ util\/system\.\1/g' src/Makefile.am
sed -i 's/utilmemory\.\(h\|cpp\)/util\/memory\.\1/g' src/Makefile.am
sed -i 's/utilmoneystr\.\(h\|cpp\)/util\/moneystr\.\1/g' src/Makefile.am
sed -i 's/utilstrencodings\.\(h\|cpp\)/util\/strencodings\.\1/g' src/Makefile.am
sed -i 's/utiltime\.\(h\|cpp\)/util\/time\.\1/g' src/Makefile.am

sed -i 's/-> util ->/-> util\/system ->/' test/lint/lint-circular-dependencies.sh
sed -i 's/src\/util\.cpp/src\/util\/system\.cpp/g' test/lint/lint-format-strings.py test/lint/lint-locale-dependence.sh
sed -i 's/src\/utilmoneystr\.cpp/src\/util\/moneystr\.cpp/g' test/lint/lint-locale-dependence.sh
sed -i 's/src\/utilstrencodings\.\(h\|cpp\)/src\/util\/strencodings\.\1/g' test/lint/lint-locale-dependence.sh
sed -i 's/src\\utilstrencodings\.cpp/src\\util\\strencodings\.cpp/' build_msvc/libbitcoinconsensus/libbitcoinconsensus.vcxproj
-END VERIFY SCRIPT-
2018-11-04 22:46:07 -08:00
251
e2ba043b8d Implements ParseNetwork unit test.
This commit implements a unit test that validates the `ParseNetwork(std::string)` implementation in `netbase.cpp`.
2018-08-28 18:37:34 +02:00
DrahtBot
eb7daf4d60 Update copyright headers to 2018 2018-07-27 07:15:02 -04:00
MarcoFalke
fa451511a1
doc: Adjust bitcoincore.org links 2018-07-22 10:32:38 -04:00
wodry
07c493f2d1 scripted-diff: Replace NET_TOR with NET_ONION
-BEGIN VERIFY SCRIPT-

sed --in-place'' --expression='s/NET_TOR/NET_ONION/g' $(git grep -I --files-with-matches 'NET_TOR')

-END VERIFY SCRIPT-

The --in-place'' hack is required for sed on macOS to edit files in-place without passing a backup extension.
2018-07-02 16:41:13 +02:00
Akira Takizawa
595a7bab23 Increment MIT Licence copyright header year on files modified in 2017 2018-01-03 02:26:56 +09:00
MeshCollider
1a445343f6 scripted-diff: Replace #include "" with #include <> (ryanofsky)
-BEGIN VERIFY SCRIPT-
for f in \
  src/*.cpp \
  src/*.h \
  src/bench/*.cpp \
  src/bench/*.h \
  src/compat/*.cpp \
  src/compat/*.h \
  src/consensus/*.cpp \
  src/consensus/*.h \
  src/crypto/*.cpp \
  src/crypto/*.h \
  src/crypto/ctaes/*.h \
  src/policy/*.cpp \
  src/policy/*.h \
  src/primitives/*.cpp \
  src/primitives/*.h \
  src/qt/*.cpp \
  src/qt/*.h \
  src/qt/test/*.cpp \
  src/qt/test/*.h \
  src/rpc/*.cpp \
  src/rpc/*.h \
  src/script/*.cpp \
  src/script/*.h \
  src/support/*.cpp \
  src/support/*.h \
  src/support/allocators/*.h \
  src/test/*.cpp \
  src/test/*.h \
  src/wallet/*.cpp \
  src/wallet/*.h \
  src/wallet/test/*.cpp \
  src/wallet/test/*.h \
  src/zmq/*.cpp \
  src/zmq/*.h
do
  base=${f%/*}/ relbase=${base#src/} sed -i "s:#include \"\(.*\)\"\(.*\):if test -e \$base'\\1'; then echo \"#include <\"\$relbase\"\\1>\\2\"; else echo \"#include <\\1>\\2\"; fi:e" $f
done
-END VERIFY SCRIPT-
2017-11-16 08:23:01 +13:00
John Newbery
fe4fabaf12 [refactor] move SplitHostPort() into utilstrencodings
This moves SplitHostPort from libbitcoin_common to libbitcoin_util so it
is available to bitcoin-cli.
2017-07-12 17:06:15 -04:00
Cory Fields
6d0bd5b73d net: do not allow resolving to an internal address
In order to prevent mixups, our internal range is never allowed as a resolve
result. This means that no user-provided string will ever be confused with an
internal address.
2017-06-14 18:05:01 -04:00
Cory Fields
7f31762cb6 net: add an internal subnet for representing unresolved hostnames
We currently do two resolves for dns seeds: one for the results, and one to
serve in addrman as the source for those addresses.

There's no requirement that the source hostname resolves to the stored
identifier, only that the mapping is unique. So rather than incurring the
second lookup, combine a private subnet with a hash of the hostname.

The resulting v6 ip is guaranteed not to be publicy routable, and has only a
negligible chance of colliding with a user's internal network (which would be
of no consequence anyway).
2017-06-14 18:05:00 -04:00
practicalswift
3fb81a8480 Use list initialization (C++11) for maps/vectors instead of boost::assign::map_list_of/list_of 2017-06-06 21:29:16 +02:00
MarcoFalke
4cfd57d2e3
Merge #9281: Refactor: Remove using namespace <xxx> from bench/ & test/ sources
73f4119 Refactoring: Removed using namespace <xxx> from bench/ and test/ source files. (Karl-Johan Alm)
2017-01-05 11:32:05 +01:00
Karl-Johan Alm
73f41190b9 Refactoring: Removed using namespace <xxx> from bench/ and test/ source files. 2017-01-02 20:35:23 +09:00
isle2983
27765b6403 Increment MIT Licence copyright header year on files modified in 2016
Edited via:

$ contrib/devtools/copyright_header.py update .
2016-12-31 11:01:21 -07:00
Cory Fields
8945384bca net: Have LookupNumeric return a CService directly
Also fix up a few small issues:
- Lookup with "badip:port" now sets the port to 0
- Don't allow assert to have side-effects
2016-08-04 16:41:39 -04:00
Cory Fields
1017b8a960 net: Add direct tests for new CSubNet constructors 2016-07-31 14:01:43 -04:00
Cory Fields
b6c3ff3dae net: Split resolving out of CSubNet 2016-07-31 14:01:43 -04:00
Cory Fields
31d6b1d5f0 net: Split resolving out of CNetAddr 2016-07-31 13:24:07 -04:00
MarcoFalke
fa24439ff3 Bump copyright headers to 2015 2015-12-13 18:08:39 +01:00
Jonas Schnelli
07f70b2dde [QA] fix netbase tests because of new CSubNet::ToString() output 2015-09-16 16:50:20 +02:00
Wladimir J. van der Laan
e2b8028e4c net: Fix CIDR notation in ToString()
Only use CIDR notation if the netmask can be represented as such.
2015-09-16 16:50:19 +02:00
Jonas Schnelli
607809f037 net: use CIDR notation in CSubNet::ToString() 2015-09-16 16:50:19 +02:00
Alex Morcos
1123cdbf4d add unit test for CNetAddr::GetGroup. 2015-08-19 17:06:42 -04:00
Jonas Schnelli
177a0e4914 Adding CSubNet constructor over a single CNetAddr 2015-07-02 20:44:27 +02:00
Wladimir J. van der Laan
b45c50ce51 Fix two problems in CSubNet parsing
Fix two CSubNet constructor problems:

- The use of `/x` where 8 does not divide x was broken, due to a
  bit-order issue

- The use of e.g. `1.2.3.4/24` where the netmasked bits in the network
  are not 0 was broken. Fix this by explicitly normalizing the netwok
  according to the bitmask.

Also add tests for these cases.

Fixes #6179. Thanks to @jonasschnelli for reporting and initial fix.
2015-05-26 08:59:59 +02:00
Wladimir J. van der Laan
92fd887fd4 tests: add a BasicTestingSetup and apply to all tests
Make sure that chainparams and logging is properly initialized. Doing
this for every test may be overkill, but this initialization is so
simple that that does not matter.

This should fix the travis issues.
2015-03-12 09:45:22 +01:00
Michael Ford
78253fcbad Remove references to X11 licence 2014-12-16 15:56:50 +08:00
Wladimir J. van der Laan
e16be73753 net: Add CSubNet class for subnet matching 2014-05-09 16:45:57 +02:00