...instead pass in a std::function<int64_t()>
Note that the static_cast is needed (apparently) for the compiler to
know which overloaded GetTime to choose.
in src/node/miner to:
- BlockAssembler::addPackageTxs()
- BlockAssembler::SkipMapTxEntry()
- BlockAssembler::UpdatePackagesForAdded()
These functions have thread safety lock annotations in
their declarations but are missing the corresponding
run-time lock assertions in their definitions.
Per doc/developer-notes.md: "Combine annotations in function
declarations with run-time asserts in function definitions."
fa37e798b2 wallet: Replace confusing getAdjustedTime() with GetTime() (MarcoFalke)
Pull request description:
Setting `nTimeReceived` to the adjusted time has several issues:
* `m_best_block_time` is set to the "unadjusted" time, thus a comparison of the two times is like comparing apples to oranges. In the worst case this opens up an attack vector where remote peers can force a premature re-broadcast of wallet txs.
* The RPC documentation for `"timereceived"` doesn't mention that the network adjusted time is used, possibly confusing users when the time reported by RPC is off by a few seconds compared to their local timestamp.
Fix all issues by replacing the call with `GetTime()`. Also a style fix: Use non-narrowing integer conversion in the RPC method.
ACKs for top commit:
theStack:
Code-review ACK fa37e798b2
shaavan:
crACK fa37e798b2
Tree-SHA512: 8d020ba400521246b7aed4b6c41319fc70552e8c69e929a5994500375466a9edac02a0ae64b803dbc6695df22276489561a23bd6e030c44c97d288f7b9b2b3fa
...by moving the try/catch out of LoadChainstate
I strongly recommend reviewing with the following git-diff flags:
--color-moved=dimmed_zebra --color-moved-ws=allow-indentation-change
...instead allow the caller to optionally pass in callbacks which are
triggered for certain events.
Behaviour change: The string "Verifying blocks..." was previously
printed for each chainstate in chainman which did not have an
effectively empty coinsview, now it will be printed once unconditionally
before we call VerifyLoadedChain.
This allows us to separate the initialization code from translations and
error reporting.
This change changes the caller semantics of LoadChainstate quite
drastically.
To see that this change doesn't change behaviour, observe that:
1. Prior to this change, LoadChainstate returned false only in the "bad
genesis block" failure case (by returning InitError()), indicating
that the caller should immediately bail. After this change, the
corresponding ERROR_BAD_GENESIS_BLOCK handler in src/init.cpp
maintains behavioue by also bailing immediately.
2. The failed_* temporary booleans were only used to break out of the
outer do/while(false) loop. They can therefore be safely removed.
I strongly recommend reviewing with the following git-diff flags:
--color-moved=dimmed_zebra --color-moved-ws=allow-indentation-change
[META] This commit is intended to be as close to a move-only commit as
possible, and lingering ugliness will be resolved in subsequent
commits.
A few variables that are passed in by value instead of by reference
deserve explanation:
- fReset and fReindexChainstate are both local variables in AppInitMain
and are not modified in the sequence
- fPruneMode, despite being a global, is only modified in
AppInitParameterInteraction, long before LoadChainstate is called
----
[META] This semantic will change in a future commit named
"node/chainstate: Decouple from stringy errors"
fa4e09924b refactor: Replace validation.h include with forward-decl in miner.h (MarcoFalke)
fa0739a7d3 style: Sort file list after rename (MarcoFalke)
fa53e3a58c scripted-diff: Move miner to src/node (MarcoFalke)
Pull request description:
It is impossible to run the miner without a node (validation, chainstate, mempool, rpc, ...). Also, the module is in the node library. Thus, it should be moved to `src/node`.
Also, replace the `validation.h` include in the header with a forward-declaration.
ACKs for top commit:
theStack:
Code-review ACK fa4e09924b
Tree-SHA512: 791e6caa5839d8dc83b0f58f3f49bc0a7e3c1710822e8a44dede254c87b6f7531a0586fb95e8a067c181457a3895ad6041718aa2a2fac64cfc136bf04bb851d5
fa3e0da06b policy: Treat taproot as always active (MarcoFalke)
Pull request description:
Now that taproot is active, it can be treated as if it was always active for policy for the next major release. This simplifies the code and changes two things:
* Importing `tr` descriptors can be done before the chain is fully synced. This is fine, because the wallet will already generate `tr` descriptors by default (regardless of the taproot status) after commit 47fe7445e7.
* Valid taproot spends won't be rejected from the mempool before taproot is active. This is strictly speaking a bugfix after commit 47fe7445e7, since the wallet may generate taproot spends before the chain is fully synced. For example, a slow node or a purposefully offline node. Currently, the wallet needs the mempool to account for change. See https://github.com/bitcoin/bitcoin/issues/11887.
A similar change was done for segwit v0 in https://github.com/bitcoin/bitcoin/pull/13120 .
This effectively reverts commit c5ec0367d7.
ACKs for top commit:
mjdietzx:
Code Review ACK fa3e0da06b
achow101:
ACK fa3e0da06b
sipa:
utACK fa3e0da06b
gruve-p:
ACK fa3e0da06b
gunar:
Code Review + tACK fa3e0da06
rajarshimaitra:
code review + tACK fa3e0da06b
Tree-SHA512: c6dc7a4e6c345bdec33f256847dc63906ab1696aa683ab9b32a79e715613950884ac3a1a7a44e95f31bb28e58dd64679a616175f7e152b21f5550f3337c8e622
This fixes -Wbitwise-instead-of-logical compiler warnings:
node/interfaces.cpp:544:16: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical]
return FillBlock(ancestor, ancestor_out, lock, active) & FillBlock(block1, block1_out, lock, active) & FillBlock(block2, block2_out, lock, active);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
&&
node/interfaces.cpp:544:16: note: cast one or both operands to int to silence this warning
node/interfaces.cpp:544:16: warning: use of bitwise '&' with boolean operands [-Wbitwise-instead-of-logical]
return FillBlock(ancestor, ancestor_out, lock, active) & FillBlock(block1, block1_out, lock, active) & FillBlock(block2, block2_out, lock, active);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
&&
node/interfaces.cpp:544:16: note: cast one or both operands to int to silence this warning
2 warnings generated.
A similar change was recently made to libsecp in commit 16d13221
for the same reason.
faba1abe46 Sort file list after rename (MarcoFalke)
fa8f60e311 scripted-diff: Move minisketchwrapper to src/node (MarcoFalke)
Pull request description:
The newly added wrapper is currently in the node library, but not placed in the node directory. While it is possible to use the wrapper outside of a node context (for example in a utility), it seems unlikely. Either way, I think the wrapper should either be moved to the util lib+dir or the node lib+dir, not something in-between.
Also, fix incorrect comment `BITCOIN_DBWRAPPER_H`.
ACKs for top commit:
fanquake:
ACK faba1abe46. I saw the comment in #21515, however given there hasn't been any new activity there, I'm going to merge this now.
Tree-SHA512: fccc0cfd1fee661152a1378587b96795ffb7a7eceb6d2c27ea5401993fd8b9c0a92579fdba61203917ae6565269cb28d0973464fb6201dabf72a5143495d3e77
0e0f4fdd89 multiprocess: Add interfaces::Node::broadCastTransaction method (Russell Yanofsky)
Pull request description:
This fixes a null pointer crash in the bitcoin-gui PSBT dialog. The bitcoin-gui interfaces::Node object has a null NodeContext pointer, and can't broadcast transactions directly. It needs to broadcast transactions through the bitcoin-node process instead.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
ACKs for top commit:
lsilva01:
Code Review ACK 0e0f4fd
Tree-SHA512: cd2c1fe8dc15e7cecf01a21d64319d6add1124995305a9ef9cb72f8492dc692c62d4f846182567d47a5048a533178a925419250941a47cb39932467c36bea3e1
ad085f9ba1 multiprocess: Delay wallet client construction (Russell Yanofsky)
Pull request description:
Delay wallet client construction until after logging, thread and other init for two reasons:
- More responsive multiprocess GUI startup. When bitcoin-gui is started this moves the call from bitcoin-gui to bitcoin-node that spawns bitcoin-wallet off of the GUI event thread and onto the background GUI init executor thread.
- Avoids feature_logging.py test failures with bitcoin-node by making bitcoin-wallet logging start after bitcoin-node logging starts,
because the tests are not written to handle the bitcoin-wallet logging init code running first.
This partially reverts commit b266b3e0bf, moving wallet client creation back to the place it was located before.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
ACKs for top commit:
laanwj:
code review ACK ad085f9ba1
hebasto:
ACK ad085f9ba1, I have reviewed the code and it looks OK.
Tree-SHA512: 74d957ce2ee096db745c517124f60800185814b06c20db676090e10dce1b90311adbab02865a69731f8c39b9365f9ee14be0830ca1368cac9b474801ea92bad5
a032fa30d2 multiprocess: add interfaces::ExternalSigner class (Russell Yanofsky)
Pull request description:
Add `interfaces::ExternalSigner` class to let signer objects be passed between processes and let signer code run in the original process where the object was created.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
ACKs for top commit:
laanwj:
Concept and code review ACK a032fa30d2
hebasto:
re-ACK a032fa30d2
Tree-SHA512: 99a729fb3a64d010e142cc778a9f1f358e58345b77faaf2664de7d2277715d59df3352326e8f0f2a6628038670eaa4556310a549079fb28af6d2eeb05aea1460
This fixes a null pointer crash in the bitcoin-gui PSBT dialog. The
bitcoin-gui interfaces::Node object has a null NodeContext pointer, and
can't broadcast transactions directly. It needs to broadcast
transactions through the bitcoin-node process instead.
6544ea5035 refactor: Block unsafe fs::path std::string conversion calls (Russell Yanofsky)
b39a477ec6 refactor: Add fs::PathToString, fs::PathFromString, u8string, u8path functions (Russell Yanofsky)
Pull request description:
The `fs::path` class has a `std::string` constructor which will implicitly convert from strings. Implicit conversions like this are not great in general because they can hide complexity and inefficiencies in the code, but this case is especially bad, because after the transition from `boost::filesystem` to `std::filesystem` in #20744 the behavior of this constructor on windows will be more complicated and can mangle path strings. The `fs::path` class also has a `.string()` method which is inverse of the constructor and has the same problems.
Fix this by replacing the unsafe method calls with `PathToString` and `PathFromString` function calls, and by forbidding unsafe method calls in the future.
ACKs for top commit:
kiminuo:
ACK 6544ea5035
laanwj:
Code review ACK 6544ea5035
hebasto:
re-ACK 6544ea5035, only added `fsbridge_stem` test case, updated comment, and rebased since my [previous](https://github.com/bitcoin/bitcoin/pull/22937#pullrequestreview-765503126) review. Verified with the following command:
Tree-SHA512: c36324740eb4ee55151146626166c00d5ccc4b6f3df777e75c112bcb4d1db436c1d9cc8c29a1e7fb96051457d317961ab42e6c380c3be2771d135771b2b49fa0
b5950dd59c validation: put coins cache write log into bench debug log (Anthony Towns)
31b2b802b5 blockstorage: use debug log category (Anthony Towns)
da94ebc2fa validation: move header validation error logging to VALIDATION debug category (Anthony Towns)
1d7d835ec3 validation: include block hash when reporting prev block not found errors (Anthony Towns)
Pull request description:
Moves the following log messages into debug log categories:
* "AcceptBlockHeader: ..." to validation
* "Prune: deleted blk/rev" to new blockstorage log category
* "Leaving block file" moves from validation to blockstorage
* "write coins cache to disk" to bench
Also adds the hash of the block to the log message when AcceptBlockHeader is rejecting because of problems with the prev block.
ACKs for top commit:
practicalswift:
cr ACK b5950dd59c
Empact:
Code review ACK b5950dd59c
laanwj:
Code review ACK b5950dd59c
promag:
Code review ACK b5950dd59c.
meshcollider:
Code review ACK b5950dd59c
Tree-SHA512: a73fdbfe8d36da48a3e89c2d5e0b6a3c5045d280c1a57f61c38d0d21f4f198aece4bd85155be3439e179d5dabdb523bf15fa0395e0e3ceff19c878ba3112c840
7e88f61b28 multiprocess: Make interfaces::Chain::isTaprootActive non-const (Russell Yanofsky)
Pull request description:
`interfaces::Chain` is an abstract class, so declaring the method const would be exposing internal implementation details of subclasses to interface callers. And specifically this doesn't work because the multiprocess implementation of the `interfaces::Chain::isTaprootActive` method can't be const because IPC connection state and request state is not constant during the call.
---
This PR is part of the [process separation project](https://github.com/bitcoin/bitcoin/projects/10).
ACKs for top commit:
jamesob:
ACK 7e88f61b28
Tree-SHA512: 1c5ed89870aeb7170b9048c41299ab650dfa3d0978088e08c4c866fa0babb292722710b16f25540f26667220cb4747b1c256c4bd42893c552291eccc155346a3
Add interfaces::ExternalSigner to let signer objects be passed between
processes and signer code to run in the original process, without
multiple processes linking and running signer code.
There is no change in behavior. This just helps prepare for the
transition from boost::filesystem to std::filesystem by avoiding calls
to methods which will be unsafe after the transaction to std::filesystem
to due lack of a boost::filesystem::path::imbue equivalent and inability
to set a predictable locale.
Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
Co-authored-by: Kiminuo <kiminuo@protonmail.com>
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
021f86953e [style] Run changed files through clang formatter. (Amiti Uttarwar)
375750387e scripted-diff: Rename CAddrInfo to AddrInfo (Amiti Uttarwar)
dd8f7f2500 scripted-diff: Rename CAddrMan to AddrMan (Amiti Uttarwar)
3c263d3f63 [includes] Fix up included files (Amiti Uttarwar)
29727c2aa1 [doc] Update comments (Amiti Uttarwar)
14f9e000d0 [refactor] Update GetAddr_() function signature (Amiti Uttarwar)
40acd6fc9a [move-only] Move constants to test-only header (Amiti Uttarwar)
7cf41bbb38 [addrman] Change CAddrInfo access (Amiti Uttarwar)
e3f1ea659c [move-only] Move CAddrInfo to test-only header file (Amiti Uttarwar)
7cba9d5618 [net, addrman] Remove external dependencies on CAddrInfo objects (Amiti Uttarwar)
8af5b54f97 [addrman] Introduce CAddrMan::Impl to encapsulate addrman implementation. (Amiti Uttarwar)
f2e5f38f09 [move-only] Match ordering of CAddrMan declarations and definitions (Amiti Uttarwar)
5faa7dd6d8 [move-only] Move CAddrMan function definitions to cpp (Amiti Uttarwar)
Pull request description:
Introduce the pimpl pattern for AddrMan to separate the implementation details from the externally used object representation. This reduces compile-time dependencies and conceptually clarifies AddrMan's interface from the implementation specifics.
Since the unit & fuzz tests currently rely on accessing AddrMan internals, this PR introduces addrman_impl.h, which is exclusively imported by addrman.cpp and test files.
ACKs for top commit:
jnewbery:
ACK 021f86953e
GeneFerneau:
utACK [021f869](021f86953e)
mzumsande:
ACK 021f86953e
rajarshimaitra:
Concept + Code Review ACK 021f86953e
theuni:
ACK 021f86953e
Tree-SHA512: aa70cb77927a35c85230163c0cf6d3872382d79048b0fb79341493caa46f8e91498cb787d8b06aba4da17b2f921f2230e73f3d66385519794fff86a831b3a71d
9d0379cea6 consensus: use <cstdint> over <stdint.h> in amount.h (fanquake)
863e52fe63 consensus: make COIN & MAX_MONEY constexpr (fanquake)
d09071da5b [MOVEONLY] consensus: move amount.h into consensus (fanquake)
Pull request description:
A first step (of a few) towards some source code reorganization, as well as making libbitcoinconsensus slightly more self contained.
Related to #15732.
ACKs for top commit:
MarcoFalke:
concept ACK 9d0379cea6 🏝
Tree-SHA512: 97fc79262dcb8c00996852a288fee69ddf8398ae2c95700bba5b326f1f38ffcfaf8fa66e29d0cb446d9b3f4e608a96525fae0c2ad9cd531ad98ad2a4a687cd6a
4747da3a5b Add syscall sandboxing (seccomp-bpf) (practicalswift)
Pull request description:
Add experimental syscall sandboxing using seccomp-bpf (Linux secure computing mode).
Enable filtering of system calls using seccomp-bpf: allow only explicitly allowlisted (expected) syscalls to be called.
The syscall sandboxing implemented in this PR is an experimental feature currently available only under Linux x86-64.
To enable the experimental syscall sandbox the `-sandbox=<mode>` option must be passed to `bitcoind`:
```
-sandbox=<mode>
Use the experimental syscall sandbox in the specified mode
(-sandbox=log-and-abort or -sandbox=abort). Allow only expected
syscalls to be used by bitcoind. Note that this is an
experimental new feature that may cause bitcoind to exit or crash
unexpectedly: use with caution. In the "log-and-abort" mode the
invocation of an unexpected syscall results in a debug handler
being invoked which will log the incident and terminate the
program (without executing the unexpected syscall). In the
"abort" mode the invocation of an unexpected syscall results in
the entire process being killed immediately by the kernel without
executing the unexpected syscall.
```
The allowed syscalls are defined on a per thread basis.
I've used this feature since summer 2020 and I find it to be a helpful testing/debugging addition which makes it much easier to reason about the actual capabilities required of each type of thread in Bitcoin Core.
---
Quick start guide:
```
$ ./configure
$ src/bitcoind -regtest -debug=util -sandbox=log-and-abort
…
2021-06-09T12:34:56Z Experimental syscall sandbox enabled (-sandbox=log-and-abort): bitcoind will terminate if an unexpected (not allowlisted) syscall is invoked.
…
2021-06-09T12:34:56Z Syscall filter installed for thread "addcon"
2021-06-09T12:34:56Z Syscall filter installed for thread "dnsseed"
2021-06-09T12:34:56Z Syscall filter installed for thread "net"
2021-06-09T12:34:56Z Syscall filter installed for thread "msghand"
2021-06-09T12:34:56Z Syscall filter installed for thread "opencon"
2021-06-09T12:34:56Z Syscall filter installed for thread "init"
…
# A simulated execve call to show the sandbox in action:
2021-06-09T12:34:56Z ERROR: The syscall "execve" (syscall number 59) is not allowed by the syscall sandbox in thread "msghand". Please report.
…
Aborted (core dumped)
$
```
---
[About seccomp and seccomp-bpf](https://en.wikipedia.org/wiki/Seccomp):
> In computer security, seccomp (short for secure computing mode) is a facility in the Linux kernel. seccomp allows a process to make a one-way transition into a "secure" state where it cannot make any system calls except exit(), sigreturn(), and read() and write() to already-open file descriptors. Should it attempt any other system calls, the kernel will terminate the process with SIGKILL or SIGSYS. In this sense, it does not virtualize the system's resources but isolates the process from them entirely.
>
> […]
>
> seccomp-bpf is an extension to seccomp that allows filtering of system calls using a configurable policy implemented using Berkeley Packet Filter rules. It is used by OpenSSH and vsftpd as well as the Google Chrome/Chromium web browsers on Chrome OS and Linux. (In this regard seccomp-bpf achieves similar functionality, but with more flexibility and higher performance, to the older systrace—which seems to be no longer supported for Linux.)
ACKs for top commit:
laanwj:
Code review and lightly tested ACK 4747da3a5b
Tree-SHA512: e1c28e323eb4409a46157b7cc0fc29a057ba58d1ee2de268962e2ade28ebd4421b5c2536c64a3af6e9bd3f54016600fec88d016adb49864b63edea51ad838e17
Delay wallet client construction until after logging, thread and other
init for two reasons:
- More responsive multiprocess GUI startup. When bitcoin-gui is started
this moves the call from bitcoin-gui to bitcoin-node that spawns
bitcoin-wallet off of the GUI event thread and onto the background GUI
init executor thread.
- Avoids feature_logging.py test failures with bitcoin-node by making
bitcoin-wallet logging start after bitcoin-node logging starts,
because the tests are not written to handle the bitcoin-wallet logging
init code running first.
This partially reverts commit b266b3e0bf,
moving wallet client creation back to the place it was located before.
interfaces::Chain is an abstract class, so declaring the method const
would be exposing internal implementation details of subclasses to
interface callers. And specifically this doesn't work because the
multiprocess implementation of the interfaces::Chain::isTaprootActive
method can't be const because IPC connection state and request state is
not constant during the call.
