bitcoin/src
MarcoFalke 76d44e832f
Merge bitcoin/bitcoin#24469: test: Correctly decode UTF-8 literal string paths
2f5fd3cf92 test: Correctly decode UTF-8 literal string paths (Ryan Ofsky)

Pull request description:

  Call `fs::u8path()` to convert some UTF-8 string literals to paths, instead of relying on the implicit conversion. Fake Macro pointed out in https://github.com/bitcoin/bitcoin/pull/24306#discussion_r818566106 that `fs_tests` are incorrectly decoding some literal UTF-8 paths using the current windows codepage, instead of treating them as UTF-8. This could cause test failures depending what environment windows tests are run under.

  The `fs::path` class exists to avoid problems like this, but because it is lenient with `const char*` conversions, under assumption that they are ["safe as long as the literals are ASCII"](727b0cb592/src/fs.h (L39)), bugs like this are still possible.

  If we think this is a concern, followup options to try to prevent this bug in the future are:

  0. Do nothing
  1. Improve the "safe as long as the literals are ASCII" comment. Make it clear that non-ASCII strings are invalid.
  2. Drop the implicit `const char*` conversion functions. This would be nice because it would simplifify the `fs::path` class a little, while making it safer. Drawback is that it would require some more verbosity from callers. For example, instead of `GetDataDirNet() / "mempool.dat"` they would have to write `GetDataDirNet() / fs::u8path("mempool.dat")`
  3. Keep the implicit `const char*` conversion functions, but make them call `fs::u8path()` internally. Change the "safe as long as the literals are *ASCII*" comment to "safe as long as the literals are *UTF-8*".

  I'd be happy with 0, 1, or 2. I'd be a little resistant to 3 even though it was would add more safety, because it would slightly increase complexity, and because I think it would encourage representing paths as strings, when I think there are so many footguns associated with paths as strings, that it's best to convert strings to paths at the earliest point possible, and convert paths to strings at the latest point possible.

ACKs for top commit:
  laanwj:
    Code review ACK 2f5fd3cf92
  w0xlt:
    crACK 2f5fd3c

Tree-SHA512: 9c56714744592094d873b79843b526d20f31ed05eff957d698368d66025764eae8bfd5305d5f7b6cc38803f0d85fa5552003e5c6cacf1e076ea6d313bcbc960c
2022-03-10 12:49:50 +01:00
..
bench Use GetPathArg where possible 2022-03-02 12:09:27 +01:00
common refactor: Fixup uint64_t-cast style in touched line 2022-02-01 11:19:18 +01:00
compat scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
config
consensus scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
crc32c Update crc32c subtree 2021-09-29 14:10:29 +02:00
crypto doc: Fix typos 2022-02-17 03:42:08 +09:00
index index: Commit DB_MUHASH and DB_BEST_BLOCK to disk together 2022-02-21 11:37:19 -05:00
init Use GetPathArg where possible 2022-03-02 12:09:27 +01:00
interfaces Merge bitcoin/bitcoin#22834: net: respect -onlynet= when making outbound connections 2022-03-01 18:32:01 +01:00
ipc refactor: Block unsafe fs::path std::string conversion calls 2021-10-05 11:10:47 -04:00
leveldb build: Minor leveldb subtree update 2022-03-02 15:25:48 +01:00
logging scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
minisketch Update minisketch subtree to latest upstream 2022-02-04 22:47:49 +08:00
node Merge bitcoin/bitcoin#24050: validation: Give m_block_index ownership of CBlockIndexs 2022-03-07 13:15:27 +01:00
policy Remove outdated comment on CFeeRate 2022-02-14 16:01:26 -05:00
primitives Merge bitcoin/bitcoin#24350: Primitives: Correct CTransaction deserialization docstring 2022-02-19 09:35:10 +01:00
qt qt: Avoid crash on startup if int specified in settings.json 2022-03-07 13:29:46 -05:00
rpc Merge bitcoin/bitcoin#24050: validation: Give m_block_index ownership of CBlockIndexs 2022-03-07 13:15:27 +01:00
script Merge bitcoin/bitcoin#24043: Add (sorted)multi_a descriptor for k-of-n multisig inside tr 2022-03-04 07:28:23 -05:00
secp256k1 Update secp256k1 subtree to latest upstream master 2021-12-15 09:19:50 -05:00
support Use spans of std::byte in serialize 2022-01-02 11:40:31 +01:00
test Merge bitcoin/bitcoin#24469: test: Correctly decode UTF-8 literal string paths 2022-03-10 12:49:50 +01:00
univalue Update univalue subtree to latest upstream 2021-10-11 20:45:56 +08:00
util Merge bitcoin/bitcoin#24371: util: Fix ReadBinaryFile reading beyond maxsize 2022-03-10 10:24:05 +01:00
wallet Merge bitcoin/bitcoin#24198: wallet, rpc: add wtxid in WalletTxToJSON 2022-03-08 14:32:10 -05:00
zmq zmq: Fix implicit-integer-sign-change 2022-01-31 16:53:12 +01:00
.clang-format Use c++17 in clang-format 2021-11-12 11:46:34 +01:00
.clang-tidy Enable clang-tidy bugprone-argument-comment and fix violations 2021-09-07 09:11:10 +02:00
addrdb.cpp Merge bitcoin/bitcoin#24201: p2p: Avoid InitError when downgrading peers.dat 2022-02-25 08:45:11 +01:00
addrdb.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
addrman.cpp addrman: Log too low compat value 2022-02-25 14:16:32 +01:00
addrman.h p2p: Avoid InitError when downgrading peers.dat 2022-02-25 09:53:10 +09:00
addrman_impl.h test: Remove unused AddrManTest class 2021-12-28 21:54:51 +01:00
arith_uint256.cpp Fix implicit-integer-sign-change in arith_uint256 2022-01-13 15:56:59 +01:00
arith_uint256.h
attributes.h
banman.cpp Fix race condition for SetBannedSetDirty() calls 2022-01-28 19:27:25 +00:00
banman.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
base58.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
base58.h
bech32.cpp Avoid implicit-integer-sign-change in bech32.cpp 2022-02-25 09:43:54 +01:00
bech32.h Make Bech32 LocateErrors return error list rather than using out-arg 2021-12-06 14:17:41 +13:00
bitcoin-chainstate.cpp bitcoin-chainstate: Lock cs_main to UnloadBlockIndex 2022-03-08 16:12:03 -05:00
bitcoin-cli-res.rc windres: use PACKAGE_VERSION rather than building more version numbers 2021-08-17 16:54:47 +08:00
bitcoin-cli.cpp Output license info when binaries are passed -version 2022-02-22 15:36:19 +00:00
bitcoin-tx-res.rc windres: use PACKAGE_VERSION rather than building more version numbers 2021-08-17 16:54:47 +08:00
bitcoin-tx.cpp Output license info when binaries are passed -version 2022-02-22 15:36:19 +00:00
bitcoin-util-res.rc windres: use PACKAGE_VERSION rather than building more version numbers 2021-08-17 16:54:47 +08:00
bitcoin-util.cpp Output license info when binaries are passed -version 2022-02-22 15:36:19 +00:00
bitcoin-wallet-res.rc windres: use PACKAGE_VERSION rather than building more version numbers 2021-08-17 16:54:47 +08:00
bitcoin-wallet.cpp Output license info when binaries are passed -version 2022-02-22 15:36:19 +00:00
bitcoind-res.rc windres: use PACKAGE_VERSION rather than building more version numbers 2021-08-17 16:54:47 +08:00
bitcoind.cpp Output license info when binaries are passed -version 2022-02-22 15:36:19 +00:00
blockencodings.cpp
blockencodings.h
blockfilter.cpp scripted-diff: rename MapIntoRange to FastRange64 2022-01-06 11:29:55 -05:00
blockfilter.h
chain.cpp Avoid integer sanitizer warnings in chain.o 2022-01-25 10:49:46 +01:00
chain.h Merge bitcoin/bitcoin#24146: Avoid integer sanitizer warnings in chain.o 2022-01-31 09:23:54 +01:00
chainparams.cpp Update nMinimumChainWork, defaultAssumeValid for 23.x 2022-02-24 16:19:09 +01:00
chainparams.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
chainparamsbase.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
chainparamsbase.h
chainparamsseeds.h net: Update hardcoded seeds for 23.x 2022-02-22 15:15:27 +01:00
checkqueue.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
clientversion.cpp refactor: shift CopyrightHolders() and LicenseInfo() to clientversion.cpp 2022-02-22 15:36:19 +00:00
clientversion.h refactor: shift CopyrightHolders() and LicenseInfo() to clientversion.cpp 2022-02-22 15:36:19 +00:00
coins.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
coins.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
compat.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
compressor.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
compressor.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
core_io.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
core_memusage.h
core_read.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
core_write.cpp refactor: Avoid unsigned integer overflow in core_write 2022-02-11 17:21:44 +01:00
cuckoocache.h Add FastRange32 function and use it throughout the codebase 2022-01-07 13:37:47 -05:00
dbwrapper.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
dbwrapper.h Use spans of std::byte in serialize 2022-01-02 11:40:31 +01:00
deploymentinfo.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
deploymentinfo.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
deploymentstatus.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
deploymentstatus.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
dummywallet.cpp Add src/wallet/* code to wallet:: namespace 2022-01-06 22:14:16 -05:00
external_signer.cpp external_signer: improve fingerprint matching logic (stop on first match) 2021-08-24 11:30:09 +02:00
external_signer.h
flatfile.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
flatfile.h
fs.cpp refactor: replace boost::filesystem with std::filesystem 2022-02-03 18:35:52 +08:00
fs.h util: Work around libstdc++ create_directories issue 2022-02-17 12:30:11 +01:00
hash.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
hash.h Use spans of std::byte in serialize 2022-01-02 11:40:31 +01:00
httprpc.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
httprpc.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
httpserver.cpp Merge bitcoin/bitcoin#23607: rpc: Pass const char* to evhttp_connection_get_peer for new libevent 2022-01-13 18:35:25 +01:00
httpserver.h
i2p.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
i2p.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
indirectmap.h
init.cpp Merge bitcoin/bitcoin#24468: init, doc: improve -onlynet help and related tor/i2p documentation 2022-03-07 11:42:36 +01:00
init.h refactor: shift CopyrightHolders() and LicenseInfo() to clientversion.cpp 2022-02-22 15:36:19 +00:00
key.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
key.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
key_io.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
key_io.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
logging.cpp refactor: replace boost::filesystem with std::filesystem 2022-02-03 18:35:52 +08:00
logging.h refactor: replace boost::filesystem with std::filesystem 2022-02-03 18:35:52 +08:00
Makefile.am Merge bitcoin/bitcoin#24304: [kernel 0/n] Introduce bitcoin-chainstate 2022-03-03 19:31:36 +00:00
Makefile.bench.include build: header-only Boost 2022-02-13 20:59:07 +00:00
Makefile.crc32c.include build: remove support for weak linking getauxval() 2021-09-24 15:40:04 +08:00
Makefile.leveldb.include build: remove build stubs for external leveldb 2021-10-15 01:02:45 +00:00
Makefile.minisketch.include build: add minisketch build file and include it 2021-10-21 09:37:30 +08:00
Makefile.qt.include build: header-only Boost 2022-02-13 20:59:07 +00:00
Makefile.qt_locale.include qt: Pre-branch translation updates for 23.x 2022-02-28 16:59:56 +01:00
Makefile.qttest.include test: Add tests for GetArg methods / settings.json type coercion 2022-03-07 13:29:46 -05:00
Makefile.test.include fuzz: Split script formatting from script fuzz target 2022-02-15 12:19:34 +01:00
Makefile.test_fuzz.include scripted-diff: Rename libbitcoin_server.a to libbitcoin_node.a 2021-12-20 10:53:01 -05:00
Makefile.test_util.include scripted-diff: Rename libbitcoin_server.a to libbitcoin_node.a 2021-12-20 10:53:01 -05:00
Makefile.univalue.include Integrate univalue into our buildsystem 2021-10-11 20:46:25 +08:00
mapport.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
mapport.h
memusage.h
merkleblock.cpp
merkleblock.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
net.cpp net: fix GetListenPort() to derive the proper port 2022-03-02 15:42:37 +01:00
net.h net: only assume all local addresses if listening on any 2022-03-02 15:42:40 +01:00
net_permissions.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
net_permissions.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
net_processing.cpp Merge bitcoin/bitcoin#24427: refactor: Release cs_main before MaybeSendFeefilter 2022-03-07 08:47:05 +01:00
net_processing.h Merge bitcoin/bitcoin#23706: rpc: getblockfrompeer followups 2022-01-25 18:48:41 +01:00
net_types.cpp net: Drop only invalid entries when reading banlist.json 2021-12-14 18:58:45 +01:00
net_types.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
netaddress.cpp net: remove unused CNetAddr::GetHash() 2022-02-11 15:21:52 +01:00
netaddress.h net: remove unused CNetAddr::GetHash() 2022-02-11 15:21:52 +01:00
netbase.cpp Merge bitcoin/bitcoin#23542: net: open p2p connections to nodes that listen on non-default ports 2022-03-02 09:33:03 +01:00
netbase.h Merge bitcoin/bitcoin#23542: net: open p2p connections to nodes that listen on non-default ports 2022-03-02 09:33:03 +01:00
netmessagemaker.h
noui.cpp
noui.h
outputtype.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
outputtype.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
pow.cpp
pow.h
prevector.h
protocol.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
protocol.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
psbt.cpp Use spans of std::byte in serialize 2022-01-02 11:40:31 +01:00
psbt.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
pubkey.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
pubkey.h Use spans of std::byte in serialize 2022-01-02 11:40:31 +01:00
random.cpp build: remove unneeded getentropy detection (HAVE_GETENTROPY) 2022-02-02 17:22:42 +01:00
random.h doc: Fix typos pointed out by lint-spelling 2022-01-30 08:59:10 -03:00
randomenv.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
randomenv.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
rest.cpp Add src/node/* code to node:: namespace 2022-01-06 22:14:16 -05:00
reverse_iterator.h
scheduler.cpp scripted-diff: rename m_cs_callbacks_pending -> m_callbacks_mutex 2022-01-14 13:25:23 +01:00
scheduler.h refactor: replace RecursiveMutex m_callbacks_mutex with Mutex 2022-01-14 13:27:41 +01:00
serialize.h Remove unused char serialize 2022-01-02 11:52:11 +01:00
shutdown.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
shutdown.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
signet.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
signet.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
span.h span: Add BytePtr helper 2022-01-02 11:13:40 +01:00
streams.h scripted-diff: Rename nReadPos to m_read_pos in streams.h 2022-02-09 17:21:04 +01:00
sync.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
sync.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
threadinterrupt.cpp
threadinterrupt.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
threadsafety.h
timedata.cpp timedata: rename variables to match the coding style 2022-03-02 15:40:35 +01:00
timedata.h timedata: make it possible to reset the state 2022-03-02 15:40:30 +01:00
tinyformat.h
torcontrol.cpp Merge bitcoin/bitcoin#22834: net: respect -onlynet= when making outbound connections 2022-03-01 18:32:01 +01:00
torcontrol.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
txdb.cpp Replace "can not" with "cannot" in docs, user messages, and tests 2022-02-21 19:07:29 +01:00
txdb.h Replace lock with thread safety annotation in CBlockTreeDB::LoadBlockIndexGuts() 2022-01-28 20:52:53 +01:00
txmempool.cpp Merge bitcoin/bitcoin#21464: Mempool Update Cut-Through Optimization 2022-01-25 11:20:18 +08:00
txmempool.h Merge bitcoin/bitcoin#21464: Mempool Update Cut-Through Optimization 2022-01-25 11:20:18 +08:00
txorphanage.cpp
txorphanage.h [net processing] Add Orphanage empty consistency check 2021-07-20 13:12:42 +01:00
txrequest.cpp scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00
txrequest.h
uint256.cpp
uint256.h Use spans of std::byte in serialize 2022-01-02 11:40:31 +01:00
undo.h
validation.cpp Merge bitcoin/bitcoin#24050: validation: Give m_block_index ownership of CBlockIndexs 2022-03-07 13:15:27 +01:00
validation.h Merge bitcoin/bitcoin#24299: validation, refactor: UnloadBlockIndex and ChainstateManager::Reset thread safety cleanups 2022-03-07 12:13:32 +01:00
validationinterface.cpp
validationinterface.h Add coinstatsindex_unclean_shutdown test 2022-02-25 16:06:27 -05:00
version.h
versionbits.cpp trivial: comment tweaks 2022-01-28 18:07:08 +10:00
versionbits.h Merge bitcoin/bitcoin#23508: Add getdeploymentinfo RPC 2022-01-28 08:46:03 +01:00
walletinitinterface.h Add src/node/* code to node:: namespace 2022-01-06 22:14:16 -05:00
warnings.cpp
warnings.h scripted-diff: Bump copyright headers 2021-12-30 19:36:57 +02:00