8ea45e626e build: use macOS 14 SDK (Xcode 15.0) (fanquake)
51c97ffb69 build: patch boost process for macOS 14 SDK (fanquake)
423949a13b depends: add -platform_version to macOS build flags (fanquake)
Pull request description:
This fixes: https://github.com/bitcoin/bitcoin/pull/28349#issuecomment-1748515277 (cross-compiling with C++20 for macOS). See https://developer.apple.com/xcode/cpp/#c++20 for C++20 support in Apples libc++, some features landed with Xcode 14.3, although many more landed with Xcode 15.0.
ACKs for top commit:
hebasto:
ACK 8ea45e626e.
TheCharlatan:
ACK 8ea45e626e
Tree-SHA512: 274ce2c9b9f8e4d755c07b8d0d4897a7f92708ac64e6afb7a3f75bdb485e863fc7f40badf3a88b129ce36f6cca72f768dc2ed7fba2bdf0bb6da2bf0c8fedee10
Currently, p2p_filter.py::test_msg_mempool is not testing much.
This extends the tests so the interaction between sending MEMPOOL messages with
a filter that does not include all transactions in the mempool reacts, plus how
it interacts with INV messages
fad1903b8a fuzz: Avoid timeout in bitdeque (MarcoFalke)
Pull request description:
Avoid timeouts such as https://github.com/bitcoin/bitcoin/issues/28812#issuecomment-1842914664
This is done by:
* Limiting the maximum number of iterations if the maximum size of the container is "large" (see the magic numbers in the code).
* Check the equality only once. This should be fine, because if a crash were to happen in the equality check, but the crash doesn't happen if further iterations were run, the fuzz engine should eventually find the crash by truncating the fuzz input.
ACKs for top commit:
sipa:
utACK fad1903b8a
dergoegge:
utACK fad1903b8a
brunoerg:
crACK fad1903b8a
Tree-SHA512: d3d83acb3e736b8fcaf5d17ce225ac82a9f9a2efea048512d2fed594ba6c76c25bae72eb0fab3276d4db37baec0752e5367cecfb18161301b921fed09693045e
3ea54e5db7 net: Add continuous ASMap health check logging (Fabian Jahr)
28d7e55dff test: Add tests for unfiltered GetAddr usage (Fabian Jahr)
b8843d37ae fuzz: Let fuzzers use filter options in GetAddr/GetAddresses (Fabian Jahr)
e16f420547 net: Optionally include terrible addresses in GetAddr results (Fabian Jahr)
Pull request description:
There are certain statistics we can collect by running all our known clearnet addresses against the ASMap file. This could show issues with a maliciously manipulated file or with an old file that has decayed with time.
This is just a proof of concept for now. My idea currently is to run the analysis once per day and print the results to logs if an ASMap file is used.
ACKs for top commit:
achow101:
ACK 3ea54e5db7
mzumsande:
ACK 3ea54e5db7
brunoerg:
crACK 3ea54e5db7
Tree-SHA512: 777acbfac43cc43ce4a0a3612434e4ddbc65f59ae8ffc9e24f21de09011bccb297f0599cbaa82bcf40ef68e5af582c4e98556379db7ceff7d9f97574a1cf8e09
ca09415e63 rpc, doc: encryptwallet, mention HD seed rotation and new backup (furszy)
Pull request description:
Small and simple PR, updating the `encryptwallet` help message.
Better to notify users about the HD seed rotation and the new
backup requirement before executing the encryption process.
Ensuring they are prepared to update previous backups and
securely safeguard the updated wallet file.
ACKs for top commit:
S3RK:
ACK ca09415e63
achow101:
ACK ca09415e63
Tree-SHA512: f0ee65f5cea66450566e3a85e066d4c06b3293dd0e0b2ed5fafdb7fb11da0a2cd94407299a3c57a0706c2ed782f8eabb73443e85d8099a62a3fb10a02636ab46
55e3dc3e03 test: Fix test by checking the actual exception instance (Hennadii Stepanov)
Pull request description:
The `system_tests/run_command` test is broken because it passes even with the diff as follows:
```diff
--- a/src/test/system_tests.cpp
+++ b/src/test/system_tests.cpp
@@ -90,7 +90,7 @@ BOOST_AUTO_TEST_CASE(run_command)
});
}
{
- BOOST_REQUIRE_THROW(RunCommandParseJSON("echo \"{\""), std::runtime_error); // Unable to parse JSON
+ BOOST_REQUIRE_THROW(RunCommandParseJSON("invalid_command \"{\""), std::runtime_error); // Unable to parse JSON
}
// Test std::in, except for Windows
#ifndef WIN32
```
The reason of such fragility is that the [`BOOST_REQUIRE_THROW`](https://www.boost.org/doc/libs/1_83_0/libs/test/doc/html/boost_test/utf_reference/testing_tool_ref/assertion_boost_level_throw.html) macro passes even if the command raises an exception in the underlying subprocess implementation, which might have a type derived from `std::runtime_error`.
ACKs for top commit:
maflcko:
lgtm ACK 55e3dc3e03
achow101:
ACK 55e3dc3e03
furszy:
Non-Windows code ACK 55e3dc3e
pablomartin4btc:
ACK 55e3dc3e03
Tree-SHA512: 32f49421bdcc94744c81e82dc10cfa02e3f8ed111974edf1c2a47bdaeb56d7baec1bede67301cc89464fba613029ecb131dedc6bc5948777ab52f0f12df8bfe9
9075a44646 test: add regression test for the getrawtransaction segfault (Martin Zumsande)
494a926d05 rpc: fix getrawtransaction segfault (Martin Zumsande)
Pull request description:
The crash, reported in #28986, happens when calling `getrawtransaction` for any mempool transaction with `verbosity=2`, while pruning, because the rpc calls `IsBlockPruned(const CBlockIndex* pblockindex)`, which dereferences `pblockindex` without a check.
For ease of backporting this PR fixes it just locally in `rpc/rawtransaction.cpp` by moving the check for`!blockindex` up so that `IsBlockPruned()` will not be called with a `nullptr`. We might also want to change `IsBlockPruned()` so it doesn't crash when called with a `nullptr`, but I didn't do that here.
Fixes#28986
ACKs for top commit:
maflcko:
lgtm test-was-added ACK 9075a44646
theStack:
Tested ACK 9075a44646
Tree-SHA512: 0f7ed52579487196c206e16b45582b64e4b02ecf2a2eb0a31d2f3b52415bc9c64278cb94259314ef14ab7fb393c6195f79b3027d6de471d67614e51474498b11
fad2392c58 ci: Use Ubuntu 24.04 Noble for asan (MarcoFalke)
fa83b65ef8 ci: Use Ubuntu 24.04 Noble for tsan,tidy,fuzz (MarcoFalke)
Pull request description:
23.10 will be EOL mid next year, so a bump is needed before then for the `master` branch (and possibly the `26.x` branch).
Doing the bump now is fine, because the clang version is pinned to 17 inside the CI tasks. So a default clang version change in the system image should not affect the tasks. Once clang-18 is available and the default in April next year (https://discourse.ubuntu.com/t/noble-numbat-release-schedule/35649#planned-and-potentially-disruptive-archive-wide-activities-2), the pinned version could be bumped (for CI tasks that require a pin, like tidy), or the pin can be removed (for CI tasks that usually do not require a pin, like fuzz or the sanitizers).
ACKs for top commit:
fanquake:
ACK fad2392c58
Tree-SHA512: c40aede4e2281a5d539d5f65d2c08a57bf92e4a00b4f45a4260b57b7443a63d1a0603115da4a3bbd100ac5f6ade3f2eda0916e4b565573741162a76294ec0ac5
Only relying on the number of peers for detecting a new connection
suffers from race conditions, as unrelated previous peers could
disconnect at anytime in-between. Use the more robust approach of
watching for an increased highest peer id instead (again using the
`getpeerinfo` RPC call), with a newly introduced context manager
method `TestNode.wait_for_new_peer()`.
Fixes#29009.
Verifying the wallet updates the birth time accordingly when it
detects a transaction with a time older than the oldest descriptor
timestamp.
This could happen when the user blindly imports a descriptor with
'timestamp=now'.
To avoid scanning blocks, as assumed by a wallet with no
generated keys or imported scripts, the default value for
the birth time needs to be set to the maximum int64_t value.
Once the first key is generated or the first script is imported,
the legacy SPKM will update the birth time automatically.
Better to notify users about the HD seed rotation and the new
backup requirement before executing the encryption process.
Ensuring they are prepared to update previous backups and
securely safeguard the updated wallet file.
Co-authored-by: jonatack <jon@atack.com>
11d797e3a0 depends: Build `native_capnp` package with CMake (Hennadii Stepanov)
90389c95e9 depends: Build `capnp` package with CMake (Hennadii Stepanov)
Pull request description:
The first commit fixes two bugs when cross-compiling the `capnp` package on the master branch @ 160d23677a:
- for `x86_64-w64-mingw32` (see https://github.com/bitcoin/bitcoin/pull/28735#issuecomment-1790406668):
```
libtool: link: x86_64-w64-mingw32-g++-posix -shared -nostdlib /usr/lib/gcc/x86_64-w64-mingw32/12-posix/../../../../x86_64-w64-mingw32/lib/dllcrt2.o /usr/lib/gcc/x86_64-w64-mingw32/12-posix/crtbegin.o src/kj/.libs/cidr.o src/kj/.libs/common.o src/kj/.libs/units.o src/kj/.libs/memory.o src/kj/.libs/refcount.o src/kj/.libs/array.o src/kj/.libs/list.o src/kj/.libs/string.o src/kj/.libs/string-tree.o src/kj/.libs/source-location.o src/kj/.libs/hash.o src/kj/.libs/table.o src/kj/.libs/encoding.o src/kj/.libs/exception.o src/kj/.libs/debug.o src/kj/.libs/arena.o src/kj/.libs/io.o src/kj/.libs/mutex.o src/kj/.libs/thread.o src/kj/.libs/time.o src/kj/.libs/filesystem.o src/kj/.libs/filesystem-disk-unix.o src/kj/.libs/filesystem-disk-win32.o src/kj/.libs/test-helpers.o src/kj/.libs/main.o src/kj/parse/.libs/char.o -L/home/hebasto/git/bitcoin/depends/x86_64-w64-mingw32/lib -L/usr/lib/gcc/x86_64-w64-mingw32/12-posix -L/usr/lib/gcc/x86_64-w64-mingw32/12-posix/../../../../x86_64-w64-mingw32/lib -lstdc++ -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 -lpthread -ladvapi32 -lshell32 -luser32 -lkernel32 -lmingw32 -lgcc_s -lgcc -lmoldname -lmingwex -lmsvcrt -lkernel32 /usr/lib/gcc/x86_64-w64-mingw32/12-posix/crtend.o -mthreads -O2 -mthreads -mthreads -o .libs/libkj-1-0-1.dll -Wl,--enable-auto-image-base -Xlinker --out-implib -Xlinker .libs/libkj.dll.a
/usr/bin/x86_64-w64-mingw32-ld: src/kj/.libs/cidr.o:cidr.c++:(.text+0x1dc): undefined reference to `__imp_inet_ntop'
/usr/bin/x86_64-w64-mingw32-ld: src/kj/.libs/cidr.o:cidr.c++:(.text+0x44b): undefined reference to `__imp_inet_pton'
collect2: error: ld returned 1 exit status
```
- for `arm64-apple-darwin`:
```
checking build system type... x86_64-pc-linux-gnu
checking host system type... Invalid configuration `arm64-apple-darwin': machine `arm64-apple' not recognized
configure: error: /bin/bash build-aux/config.sub arm64-apple-darwin failed
```
The second commit applies the same changes for the `native_capnp` package for [consistency](https://github.com/bitcoin/bitcoin/pull/28856#issuecomment-1807936546).
ACKs for top commit:
ryanofsky:
Code review ACK 11d797e3a0. Since last review arm64-apple-darwin platform is now mentioned in the commit message, and the change to `depends/packages/libmultiprocess.mk` in d1604d4b1d1ee8df279a1776303e167cc3d06193 which was unrelated (but probably still a good optimization) was reverted.
Tree-SHA512: c636e53073ce6fcda9724723bc59f3990fa9629a3b2f73d93dbc102a5a1badfbe8f4c5fef841f03588ebcad5cd4883f3ce32b128afcd75f6bc21eb801796a586
38816ff64e fuzz: txorphan check wtxids using GenTxid::Wtxid not GenTxid::Txid (Greg Sanders)
Pull request description:
Fixes the bugs in the fuzz test with no more changes as an alternative to https://github.com/bitcoin/bitcoin/pull/28658
ACKs for top commit:
naumenkogs:
ACK 38816ff64e
dergoegge:
ACK 38816ff64e
Tree-SHA512: 5e46a83f2b2a2ac0672a63eb6200b019e01089ab1aa80c4ab869b6fcf27ccf2e84a064e96397f1a1869ccfa43b0c9638cbae681a27c4ca3c96ac71f41262601e
```bash
-platform_version platform min_version sdk_version
This is set to indicate the platform, oldest supported version
of that platform that output is to be used on, and the SDK that
the output was built against.
```
11b7269d83 script: Enhance validations in utxo_snapshot.sh (pablomartin4btc)
Pull request description:
This PR resolves#27841 and some more:
- Ensure that the snapshot height is higher than the pruned block height when the node is pruned (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/28553#issuecomment-1804941396)).
- Validate the correctness of the file path and check if the file already exists (@hazeycode's [#27845](https://github.com/bitcoin/bitcoin/pull/27845)).
- Make network activity disablement optional for the user (Suggested by @Sjors [here](https://github.com/bitcoin/bitcoin/pull/16899#discussion_r342735815) and [here](https://github.com/bitcoin/bitcoin/pull/16899#issuecomment-536520911)).
- Ensure the `reconsiderblock` command is triggered on exit (@hazeycode's same PR as above), even in the case of user interruption (Ctrl-C).
In order to perform some testing please follow the instructions in the description of previous @hazeycode's PR #27845.
ACKs for top commit:
Sjors:
tACK 11b7269d83
ryanofsky:
Code review ACK 11b7269d83
Tree-SHA512: 2b699894c6f732ad5104f5a2bcf5dc86ed31edcc9d664690cab55b94a8ab00e2ca5bde901ee1d63acddca7ea80ad1734d8cfe78f9c02f8470f264fe93a2af759
This change is mostly a refectoring that removes some code and gets rid of an
unnecessary layer of indirection after #27861
But it is not a pure refactoring since StartShutdown, AbortShutdown, and
WaitForShutdown functions used to abort on failure, and the replacement code
logs or returns errors instead.
8f6ab31863 init: don't delete PID file if it was not generated (willcl-ark)
Pull request description:
In a similar vein to #28784, if a second `bitcoind` is started using the same datadir it will fail to start up, but during shutdown remove the PID file from the first `bitcoind` instance.
ACKs for top commit:
achow101:
ACK 8f6ab31863
andrewtoth:
ACK 8f6ab31863
romanz:
ACK 8f6ab31863
Tree-SHA512: c9af703cbfa179d33ef9580a51e86c1b0acbd28daa18c8d2e5e5ff796ab4d3e2009a962a47e6046a0e5ece936f8a06ee8af5fdf8ff4ae1e52cbcdbec4b942271
Having InitContext() avoids the need to add duplicate code to src/init/*.cpp
files in the next commit. It also lets these files avoid referencing global
variables like gArgs.
There is no change in behavior in this commit.
Use SignalInterrupt object instead. There is a slight change in behavior here
because the previous StartShutdown code used to abort on failure and the
new code logs errors instead.
Use SignalInterrupt object instead. There is a slight change in behavior here
because the previous StartShutdown code used to abort on failure and the
new code returns an RPC error instead.
Replace exceptions thrown by signal and wait methods with [[nodiscard]] return
values.
This is mostly a refactoring, but there is a slight change of behavior if
AbortShutdown function fails. The original behavior which was unintentionally
changed in #27861 is restored, so it now triggers an assert failure again
instead of throwing an exception. (The AbortShutdown function is only ever
called in the the GUI version of Bitcoin Core when corruption is detected on
loading and the user tries to reindex.)
Problems with using exceptions were pointed out by MarcoFalke in
https://github.com/bitcoin/bitcoin/pull/27861#discussion_r1255496707.
Pass HTTP server an interrupt object instead of having it depend on shutdown.h
and global shutdown state.
There is no change in behavior in this commit.
Add NodeContext::shutdown variable and start using it to replace the
kernel::Context::interrupt variable. The latter can't easily be removed right
away but will be removed later in this PR.
Moving the interrupt object from the kernel context to the node context
increases flexibility of the kernel API so it is possible to use multiple
interrupt objects, or avoid creating one if one is not needed. It will also
allow getting rid of the kernel::g_context global later in this PR, replacing
it with a private SignalInterrupt instance in init.cpp
There is no change in behavior in this commit outside of unit tests. In unit
tests there should be no visible change either, but internally now each test
has its own interrupt variable so the variable will be automatically reset
between tests.
Use interfaces::Node object instead.
There is a minor change in behavior in this commit, because the new code calls
InterruptRPC() and StopRPC() when previous code did not do this. But this
should be a good thing since it makes sense to interrupt RPC when the system is
shutting down, and it is better for the GUI shut down in a consistent way
regardless of how the shutdown is triggered.
Previously, starting a second bitcoind using the same datadir would
correctly fail to init and shutdown. However during shutdown the PID
file belonging to the first instance would be erroneously removed by
the second process shutting down.
Fix this to only delete the PID file if we created it.
