Commit graph

183 commits

Author SHA1 Message Date
MarcoFalke
fab54db9f1
rest: Reject negative outpoint index in getutxos parsing 2024-07-12 17:48:26 +02:00
Roman Zeyde
1556d21599
rest: don't copy data when sending binary response
Also, change `HTTPRequest::WriteReply` to accept `std::span`.
2024-06-26 06:47:30 +03:00
Ava Chow
011a895a82
Merge bitcoin/bitcoin#29015: kernel: Streamline util library
c7376babd1 doc: Clarify distinction between util and common libraries in libraries.md (Ryan Ofsky)
4f74c59334 util: Move util/string.h functions to util namespace (Ryan Ofsky)
4d05d3f3b4 util: add TransactionError includes and namespace declarations (Ryan Ofsky)
680eafdc74 util: move fees.h and error.h to common/messages.h (Ryan Ofsky)
02e62c6c9a common: Add PSBTError enum (Ryan Ofsky)
0d44c44ae3 util: move error.h TransactionError enum to node/types.h (Ryan Ofsky)
9bcce2608d util: move spanparsing.h to script/parsing.h (Ryan Ofsky)
6dd2ad4792 util: move spanparsing.h Split functions to string.h (Ryan Ofsky)
23cc8ddff4 util: move HexStr and HexDigit from util to crypto (TheCharlatan)
6861f954f8 util: move util/message to common/signmessage (Ryan Ofsky)
cc5f29fbea build: move memory_cleanse from util to crypto (Ryan Ofsky)
5b9309420c build: move chainparamsbase from util to common (Ryan Ofsky)
ffa27af24d test: Add check-deps.sh script to check for unexpected library dependencies (Ryan Ofsky)

Pull request description:

  Remove `fees.h`, `errors.h`, and `spanparsing.h` from the util library. Specifically:

  - Move `Split` functions from `util/spanparsing.h` to `util/string.h`, using `util` namespace for clarity.
  - Move remaining spanparsing functions to `script/parsing.h` since they are used for descriptor and miniscript parsing.
  - Combine `util/fees.h` and `util/errors.h` into `common/messages.h` so there is a place for simple functions that generate user messages to live, and these functions are not part of the util library.

  Motivation for this change is that the util library is a dependency of the kernel, and we should remove functionality from util that shouldn't be called by kernel code or kernel applications. These changes should also improve code organization and make functions easier to discover. Some of these same moves are (or were) part of #28690, but did not help with code organization, or made it worse, so it is better to move them and clean them up in the same PR so code only has to change one time.

ACKs for top commit:
  achow101:
    ACK c7376babd1
  TheCharlatan:
    Re-ACK c7376babd1
  hebasto:
    re-ACK c7376babd1.

Tree-SHA512: 5bcef16c1255463b1b69270548711e7ff78ca0dd34e300b95e3ca1ce52ceb34f83d9ddb2839e83800ba36b200de30396e504bbb04fa02c6d0c24a16d06ae523d
2024-06-12 17:12:54 -04:00
Cory Fields
d7707d9843 rpc: avoid copying into UniValue
These are simple (and hopefully obviously correct) copies that can be moves
instead.
2024-05-20 16:48:19 +00:00
Ryan Ofsky
4f74c59334 util: Move util/string.h functions to util namespace
There are no changes to behavior. Changes in this commit are all additions, and
are easiest to review using "git diff -U0 --word-diff-regex=." options.

Motivation for this change is to keep util functions with really generic names
like "Split" and "Join" out of the global namespace so it is easier to see
where these functions are defined, and so they don't interfere with function
overloading, especially since the util library is a dependency of the kernel
library and intended to be used with external code.
2024-05-16 10:16:08 -05:00
MarcoFalke
dddd40ba82
scripted-diff: Add IWYU pragma keep to bitcoin-config.h includes
-BEGIN VERIFY SCRIPT-
 perl -0777 -pi -e 's/#if defined\(HAVE_CONFIG_H\)\n#include <config\/bitcoin-config.h>.*\n#endif.*\n/#include <config\/bitcoin-config.h> \/\/ IWYU pragma: keep\n/g' $( git grep -l '#include <config/bitcoin-config.h>' )
-END VERIFY SCRIPT-
2024-05-01 08:33:04 +02:00
Andrew Toth
e710cefd57
rest: read raw block in rest_block and deserialize for json
Note that for speed this commit also removes the proof of work and
signet signature checks before returning the block in getblock.
It is assumed if a block is stored it will be valid.
2024-03-12 12:48:04 -04:00
TheCharlatan
9d1dbbd4ce scripted-diff: Fix bitcoin_config_h includes
-BEGIN VERIFY SCRIPT-

regex_string='^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|CHAR_EQUALS_INT8|CLIENT_VERSION_BUILD|CLIENT_VERSION_IS_RELEASE|CLIENT_VERSION_MAJOR|CLIENT_VERSION_MINOR|COPYRIGHT_HOLDERS|COPYRIGHT_HOLDERS_FINAL|COPYRIGHT_HOLDERS_SUBSTITUTION|COPYRIGHT_YEAR|ENABLE_ARM_SHANI|ENABLE_AVX2|ENABLE_EXTERNAL_SIGNER|ENABLE_SSE41|ENABLE_TRACING|ENABLE_WALLET|ENABLE_X86_SHANI|ENABLE_ZMQ|HAVE_BOOST|HAVE_BUILTIN_CLZL|HAVE_BUILTIN_CLZLL|HAVE_BYTESWAP_H|HAVE_CLMUL|HAVE_CONSENSUS_LIB|HAVE_CXX20|HAVE_DECL_BE16TOH|HAVE_DECL_BE32TOH|HAVE_DECL_BE64TOH|HAVE_DECL_BSWAP_16|HAVE_DECL_BSWAP_32|HAVE_DECL_BSWAP_64|HAVE_DECL_FORK|HAVE_DECL_FREEIFADDRS|HAVE_DECL_GETIFADDRS|HAVE_DECL_HTOBE16|HAVE_DECL_HTOBE32|HAVE_DECL_HTOBE64|HAVE_DECL_HTOLE16|HAVE_DECL_HTOLE32|HAVE_DECL_HTOLE64|HAVE_DECL_LE16TOH|HAVE_DECL_LE32TOH|HAVE_DECL_LE64TOH|HAVE_DECL_PIPE2|HAVE_DECL_SETSID|HAVE_DECL_STRERROR_R|HAVE_DEFAULT_VISIBILITY_ATTRIBUTE|HAVE_DLFCN_H|HAVE_DLLEXPORT_ATTRIBUTE|HAVE_ENDIAN_H|HAVE_EVHTTP_CONNECTION_GET_PEER_CONST_CHAR|HAVE_FDATASYNC|HAVE_GETENTROPY_RAND|HAVE_GETRANDOM|HAVE_GMTIME_R|HAVE_INTTYPES_H|HAVE_LIBADVAPI32|HAVE_LIBCOMCTL32|HAVE_LIBCOMDLG32|HAVE_LIBGDI32|HAVE_LIBIPHLPAPI|HAVE_LIBKERNEL32|HAVE_LIBOLE32|HAVE_LIBOLEAUT32|HAVE_LIBSHELL32|HAVE_LIBSHLWAPI|HAVE_LIBUSER32|HAVE_LIBUUID|HAVE_LIBWINMM|HAVE_LIBWS2_32|HAVE_MALLOC_INFO|HAVE_MALLOPT_ARENA_MAX|HAVE_MINIUPNPC_MINIUPNPC_H|HAVE_MINIUPNPC_UPNPCOMMANDS_H|HAVE_MINIUPNPC_UPNPERRORS_H|HAVE_NATPMP_H|HAVE_O_CLOEXEC|HAVE_POSIX_FALLOCATE|HAVE_PTHREAD|HAVE_PTHREAD_PRIO_INHERIT|HAVE_STDINT_H|HAVE_STDIO_H|HAVE_STDLIB_H|HAVE_STRERROR_R|HAVE_STRINGS_H|HAVE_STRING_H|HAVE_STRONG_GETAUXVAL|HAVE_SYSCTL|HAVE_SYSCTL_ARND|HAVE_SYSTEM|HAVE_SYS_ENDIAN_H|HAVE_SYS_PRCTL_H|HAVE_SYS_RESOURCES_H|HAVE_SYS_SELECT_H|HAVE_SYS_STAT_H|HAVE_SYS_SYSCTL_H|HAVE_SYS_TYPES_H|HAVE_SYS_VMMETER_H|HAVE_THREAD_LOCAL|HAVE_TIMINGSAFE_BCMP|HAVE_UNISTD_H|HAVE_VM_VM_PARAM_H|LT_OBJDIR|PACKAGE_BUGREPORT|PACKAGE_NAME|PACKAGE_STRING|PACKAGE_TARNAME|PACKAGE_URL|PACKAGE_VERSION|PTHREAD_CREATE_JOINABLE|QT_QPA_PLATFORM_ANDROID|QT_QPA_PLATFORM_COCOA|QT_QPA_PLATFORM_MINIMAL|QT_QPA_PLATFORM_WINDOWS|QT_QPA_PLATFORM_XCB|QT_STATICPLUGIN|STDC_HEADERS|STRERROR_R_CHAR_P|USE_ASM|USE_BDB|USE_DBUS|USE_NATPMP|USE_QRCODE|USE_SQLITE|USE_UPNP|_FILE_OFFSET_BITS|_LARGE_FILES)'

exclusion_files=":(exclude)src/minisketch :(exclude)src/crc32c :(exclude)src/secp256k1 :(exclude)src/crypto/sha256_arm_shani.cpp :(exclude)src/crypto/sha256_avx2.cpp :(exclude)src/crypto/sha256_sse41.cpp :(exclude)src/crypto/sha256_x86_shani.cpp"

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.cpp' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do line_number=$(awk -v my_file="$file" '/\/\/ file COPYING or https?:\/\/www.opensource.org\/licenses\/mit-license.php\./ {line = NR} /^\/\// && NR == line + 1 {while(getline && /^\/\//) line = NR} END {print line+1}' "$file"); sed -i "${line_number}i\\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

git grep --perl-regexp --files-with-matches "$regex_string" -- '*.h' $exclusion_files | xargs git grep -L "bitcoin-config.h" | while read -r file; do sed -i "/#define.*_H/a \\\\n\#if defined(HAVE_CONFIG_H)\\n#include <config/bitcoin-config.h>\\n\#endif" "$file"; done;

for file in $(git grep --files-with-matches 'bitcoin-config.h' -- '*.cpp' '*.h' $exclusion_files); do if ! grep -q --perl-regexp "$regex_string" $file; then sed -i '/HAVE_CONFIG_H/{N;N;N;d;}' $file; fi; done;

-END VERIFY SCRIPT-

The first command creates a regular expression for matching all bitcoin-config.h symbols in the following form: ^(?!//).*(AC_APPLE_UNIVERSAL_BUILD|BOOST_PROCESS_USE_STD_FS|...|_LARGE_FILES). It was generated with:
./autogen.sh && printf '^(?!//).*(%s)' $(awk '/^#undef/ {print $2}' src/config/bitcoin-config.h.in | paste -sd "|" -)

The second command holds a list of files and directories that should not be processed. These include subtree directories as well as some crypto files that already get their symbols through the makefile.

The third command checks for missing bitcoin-config headers in .cpp files and adds the header if it is missing.

The fourth command checks for missing bitcoin-config headers in .h files and adds the header if it is missing.

The fifth command checks for unneeded bitcoin-config headers in sources files and removes the header if it is unneeded.
2024-02-13 20:10:44 +00:00
fanquake
143ace65db
Merge bitcoin/bitcoin#28890: rpc: Remove deprecated -rpcserialversion
fa46cc22bc Remove deprecated -rpcserialversion (MarcoFalke)

Pull request description:

  The flag is problematic for many reasons:

  * It is deprecated
  * It is a global flag, requiring a restart to change, as opposed to a flag that can be set on each RPC invocation
  * It may be hidden in config files by accident, hard to debug, causing LND crashes and bugs, see https://github.com/bitcoin/bitcoin/issues/28730#issuecomment-1780940868
  * It makes performance improvements harder to implement: https://github.com/bitcoin/bitcoin/pull/17529#issuecomment-556082818

  Fix all issues by removing it.

  If there is a use-case, likely a per-RPC flag can be added, if needed.

ACKs for top commit:
  ajtowns:
    crACK fa46cc22bc
  TheCharlatan:
    lgtm ACK fa46cc22bc

Tree-SHA512: 96ba1c60356ce93954fe5c2a59045771c6d1516ad0d9dc436ef1800a1f1b0153f0d5fb78ca99d53ad54ba25fbce36962bdf1d4325aceedfc8154a61347a6a915
2024-01-05 10:42:10 +00:00
MarcoFalke
fa46cc22bc
Remove deprecated -rpcserialversion 2023-12-11 18:22:13 +01:00
MarcoFalke
fa5989d514
refactor: rpc: Pass CBlockIndex by reference instead of pointer
All functions assume that the pointer is never null, so pass by
reference, to avoid accidental segfaults at runtime, or at least make
them more obvious.

Also, remove unused c-style casts in touched lines.

Also, add CHECK_NONFATAL checks, to turn segfault crashes into an
recoverable runtime error with debug information.
2023-12-07 12:05:21 +01:00
MarcoFalke
fa604eb6cf
refactor: Use reference instead of pointer in IsBlockPruned
This makes it harder to pass nullptr and cause issues such as
dde7ac5c70
2023-12-07 12:02:54 +01:00
dergoegge
9e58c5bcd9 Use Txid in COutpoint 2023-11-21 13:15:44 +00:00
Anthony Towns
83986f464c Include version.h in fewer places 2023-11-16 11:36:22 +10:00
Anthony Towns
6e9e4e6130 Use ParamsWrapper for witness serialization 2023-11-14 08:45:30 +10:00
Andrew Chow
e77339632e
Merge bitcoin/bitcoin#28136: refactor: move GetServicesNames from rpc/util.{h,cpp} to rpc/net.cpp
bbb68ffdbd refactor: drop protocol.h include header in rpc/util.h (Jon Atack)
1dd62c5295 refactor: move GetServicesNames from rpc/util.{h,cpp} to rpc/net.cpp (Jon Atack)

Pull request description:

  Move `GetServicesNames()` from `rpc/util` to `rpc/net.cpp`, as it is only called from that compilation unit and there is no reason for other ones to need it.

  Remove the `protocol.h` include in `rpc/util.h`, as it was only needed for `GetServicesNames()`, drop an unneeded forward declaration (the other IWYU suggestions would require more extensive changes in other files), and add 3 already-missing include headers in other translation units that are needed to compile without `protocol.h` in `rpc/util.h`, as `protocol.h` includes `netaddress.h`, which in turn includes `util/strencodings.h`.

ACKs for top commit:
  kevkevinpal:
    lgtm ACK [bbb68ff](bbb68ffdbd)
  ns-xvrn:
    ACK bbb68ff
  achow101:
    ACK bbb68ffdbd

Tree-SHA512: fcbe195874dd4aa9e86548685b6b28595a2c46f9869b79b6e2b3835f76b49cab4bef6a59c8ad6428063a41b7bb6f687229b06ea614fbd103e0531104af7de55d
2023-11-07 14:19:09 -05:00
MarcoFalke
fa05a726c2
tidy: modernize-use-emplace 2023-10-12 11:27:19 +02:00
Jon Atack
bbb68ffdbd refactor: drop protocol.h include header in rpc/util.h
as it was only needed for GetServicesNames(). This potentially avoids needlessly
compiling the 500 lines of protocol.h in the 35 files other than rpc/net.cpp
that include rpc/util.h.

Drop an unneeded CPubKey forward declaration. The other IWYU suggestions would
require more extensive changes in other files.

Add 3 already-missing include headers in other translation units that are needed
to compile without protocol.h in rpc/util.h, as it includes netaddress.h, which
in turn includes util/strencodings.h.
2023-09-19 15:54:17 -06:00
brunoerg
ce887eaf49 rest: bugfix, fix crash error when calling /deploymentinfo 2023-06-12 10:24:14 -03:00
TheCharlatan
7eee356c0a
refactor: Split util::AnyPtr into its own file 2023-05-20 12:03:33 +02:00
TheCharlatan
f0bb1021f0
refactor: Move functions to BlockManager methods
This is a commit in preparation for the next few commits. The functions
are moved to methods to avoid their re-declaration for the purpose of
passing in BlockManager options.

The functions that were now moved into the BlockManager should no longer
use the params as an argument, but instead use the member variable.

In the moved ReadBlockFromDisk and UndoReadFromDisk, change
the function signature to accept a reference to a CBlockIndex instead of
a raw pointer. The pointer is expected to be non-null, so reflect that
in the type.

To allow for the move of functions to BlockManager methods all call
sites require an instantiated BlockManager, or a callback to one.
2023-05-10 19:06:53 +02:00
pablomartin4btc
11422cc572 bugfix: rest: avoid segfault for invalid URI
`evhttp_uri_parse` can return a nullptr, for example when the URI
contains invalid characters (e.g. "%").
`GetQueryParameterFromUri` passes the output of `evhttp_uri_parse`
straight into `evhttp_uri_get_query`, which means that anyone calling
a REST endpoint in which query parameters are used (e.g. `rest_headers`)
can cause a segfault.

This bugfix is designed to be minimal and without additional behaviour change.
Follow-up work should be done to resolve this in a more general and robust way,
so not every endpoint has to handle it individually.
2023-04-17 10:13:34 -03:00
Andrew Chow
ebb15ea75a
Merge bitcoin/bitcoin#26207: rest: add verbose and mempool_sequence query params for mempool/contents
1ff5d61dfd doc: add mempool/contents rest verbose and mempool_sequence args (Andrew Toth)
52a31dccc9 tests: mempool/contents verbose and mempool_sequence query params tests (Andrew Toth)
a518fff0f2 rest: add verbose and mempool_sequence query params for mempool/contents (Andrew Toth)

Pull request description:

  The verbose mempool json response can get very large. This adds an option to return the non-verbose response of just the txids. It is identical to the rpc response so the diff here is minimal. This also adds the mempool_sequence parameter for rpc consistency. Verbose defaults to true to remain backwards compatible.

  It uses query parameters to be compatible with the efforts in https://github.com/bitcoin/bitcoin/issues/25752.

ACKs for top commit:
  achow101:
    ACK 1ff5d61dfd
  stickies-v:
    re-ACK [1ff5d61](1ff5d61dfd)
  pablomartin4btc:
    tested ACK 1ff5d61dfd.

Tree-SHA512: 1bf08a7ffde2e7db14dc746e421feedf17d84c4b3f1141e79e36feb6014811dfde80e1d8dbc476c15ff705de2d3c967b3081dcd80536d76b7edf888f1a92e9d1
2023-03-15 19:39:30 -04:00
MarcoFalke
fa29e73cda
Use DataStream where possible 2023-01-26 10:44:05 +01:00
Hennadii Stepanov
306ccd4927
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
- 2021: f47dda2c58
- 2020: fa0074e2d8
- 2019: aaaaad6ac9
2022-12-24 23:49:50 +00:00
Andrew Toth
c75e3d2772 rest: reduce LOCK(cs_main) scope in rest_block 2022-12-06 15:07:04 -05:00
Andrew Chow
92be831847
Merge bitcoin/bitcoin#25412: rest: add /deploymentinfo endpoint
a8250e30f1 doc: add release note about `/rest/deploymentinfo` (brunoerg)
5c96020024 doc: add `/deploymentinfo` in REST-interface (brunoerg)
3e44bee08e test: add coverage for `/rest/deploymentinfo` (brunoerg)
91497031cb rest: add `/deploymentinfo` (brunoerg)

Pull request description:

  #23508 added a new RPC named `getdeploymentinfo`, it moved the softfork section from `getblockchaininfo` into this new one. In the REST interface, we have an endpoint named`/rest/chaininfo.json` (which refers to `getblockchaininfo`), so, this PR adds a new REST endpoint named `/deploymentinfo` which refers to `getdeploymentinfo`.

  You can use it by passing a block hash, e.g: '/rest/deploymentinfo/<BLOCKHASH>.json' or you can use it without passing a block hash to get the 'deploymentinfo' for the last block.

ACKs for top commit:
  jonatack:
    re-ACK a8250e30f1 rebase-only since my last review at c65f82bb
  achow101:
    ACK a8250e30f1
  stickies-v:
    re-ACK a8250e30f1

Tree-SHA512: 0735183b6828d51a72ed0e2be5a09b314ac4693f548982c6e9adaa0ef07a55aa428d3b2d1b1de70b83169811a663a8624b686166e5797f624dcc00178b9796e6
2022-10-13 13:30:55 -04:00
Andrew Toth
a518fff0f2
rest: add verbose and mempool_sequence query params for mempool/contents 2022-10-05 09:25:07 -04:00
brunoerg
91497031cb rest: add /deploymentinfo 2022-08-16 19:21:46 -03:00
MacroFake
fac15ff673
Fix logical race in rest_getutxos
Calling ActiveHeight() and ActiveTip() subsequently without holding the
::cs_main lock over both calls may result in a height that does not
correspond to the tip due to a race.

Fix this by holding the lock.
2022-08-16 17:26:28 +02:00
brunoerg
acbea66589 rest: clean-up for mempool endpoints 2022-08-05 10:28:11 -03:00
Carl Dong
319f0ceeeb rest/getutxos: Don't construct empty mempool
...just don't try to consult it at all when fCheckMemPool is false
2022-06-15 17:28:55 -04:00
MacroFake
fa1b76aeb0
Do not call global Params() when chainman is in scope 2022-05-18 18:46:48 +02:00
fanquake
f436bfd126
Merge bitcoin/bitcoin#22953: refactor: introduce single-separator split helper (boost::split replacement)
a62e84438d fuzz: add `SplitString` fuzz target (MarcoFalke)
4fad7e46d9 test: add unit tests for `SplitString` helper (Kiminuo)
9cc8e876e4 refactor: introduce single-separator split helper `SplitString` (Sebastian Falbesoner)

Pull request description:

  This PR adds a simple string split helper `SplitString` that takes use of the spanparsing `Split` function that was first introduced in #13697 (commit fe8a7dcd78). This enables to replace most calls to `boost::split`, in the cases where only a single separator character is used. Note that while previous attempts to replace `boost::split` were controversial (e.g. #13751), this one has a trivial implementation: it merely uses an internal helper (that is unit tested and in regular use with output descriptiors) and converts its result from spans to strings. As a drawback though, not all `boost::split` instances can be tackled.

  As a possible optimization, one could return a vector of `std::string_view`s (available since C++17) instead of strings, to avoid copies. This would need more carefulness on the caller sites though, to avoid potential lifetime issues, and it's probably not worth it, considering that none of the places where strings are split are really performance-critical.

ACKs for top commit:
  martinus:
    Code review ACK a62e84438d. Ran all tests. I also like that with `boost::split` it was not obvious that the resulting container was cleared, and with `SplitString` API that's obvious.

Tree-SHA512: 10cb22619ebe46831b1f8e83584a89381a036b54c88701484ac00743e2a62cfe52c9f3ecdbb2d0815e536c99034558277cc263600ec3f3588b291c07eef8ed24
2022-04-26 09:54:49 +01:00
Carl Dong
3308ecd3fc move-mostly: Make fHavePruned a BlockMan member
[META] In the next commit, we move the clearing of fHavePruned to
       BlockManager::Unload()
2022-04-19 14:34:56 -04:00
Sebastian Falbesoner
9cc8e876e4 refactor: introduce single-separator split helper SplitString
This helper uses spanparsing::Split internally and enables to replace
all calls to boost::split where only a single separator is passed.

Co-authored-by: Martin Ankerl <Martin.Ankerl@gmail.com>
Co-authored-by: MarcoFalke <falke.marco@gmail.com>
2022-04-11 22:19:46 +02:00
MarcoFalke
ffffb7a25a
doc: Convert remaining comments to clang-tidy format 2022-04-06 15:37:07 +02:00
MarcoFalke
27cfaeed1e
Merge bitcoin/bitcoin#24098: rest: Use query parameters to control resource loading
54b39cfb34 Add release notes (stickies-v)
f959fc0397 Update /<count>/ endpoints to use a '?count=' query parameter instead (stickies-v)
a09497614e Add GetQueryParameter helper function (stickies-v)
fff771ee86 Handle query string when parsing data format (stickies-v)
c1aad1b3b9 scripted-diff: rename RetFormat to RESTResponseFormat (stickies-v)
9f1c54787c Refactoring: move declarations to rest.h (stickies-v)

Pull request description:

  In RESTful APIs, [typically](https://rapidapi.com/blog/api-glossary/parameters/query/) path parameters  (e.g. `/some/unique/resource/`) are used to represent resources, and query parameters (e.g. `?sort=asc`) are used to control how these resources are being loaded through e.g. sorting, pagination, filtering, ...

  As first [discussed in #17631](https://github.com/bitcoin/bitcoin/pull/17631#discussion_r733031180), the [current REST api](https://github.com/bitcoin/bitcoin/blob/master/doc/REST-interface.md) contains two endpoints `/headers/` and `/blockfilterheaders/` that rather unexpectedly use path parameters to control how many (filter) headers are returned in the response. While this is no critical issue, it is unintuitive and we are still early enough to easily phase this behaviour out and ensure new endpoints (if any) do not have to stick to non-standard behaviour just for internal consistency.

  In this PR, a new `HTTPRequest::GetQueryParameter` method is introduced to easily parse query parameters, as well as two new `/headers/` and `/blockfilterheaders/` endpoints that use a count query parameter are introduced. The old path parameter-based endpoints are kept without too much overhead, but the documentation now points to the new query parameter-based endpoints as the default interface to encourage standardness.

  ## Behaviour change
  ### New endpoints and default values
  `/headers/` and `/blockfilterheaders/` now have 2 new endpoints that contain query parameters (`?count=<count>`) instead of path parameters (`/<count>/`), as described in REST-interface.md. Since query parameters can easily have default values, I have set this at 5 for both endpoints.

  **headers**
  `GET /rest/headers/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/headers/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  **blockfilterheaders**
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<BLOCK-HASH>.<bin|hex|json>?count=<COUNT=5>`
  should now be used instead of
  `GET /rest/blockfilterheaders/<FILTERTYPE>/<COUNT>/<BLOCK-HASH>.<bin|hex|json>`

  ### Some previously invalid API calls are now valid
  API calls that contained query strings in the URI could not be parsed prior to this PR. This PR changes behaviour in that previously invalid calls (e.g. `GET /rest/headers/5/somehash.json?someunusedparam=foo`) would now become valid, as the query parameters are properly parsed, and discarded if unused.
  For example, prior to this PR, adding an irrelevant `someparam` parameter would be illegal:
  ```
  GET /rest/headers/5/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ->
  Invalid hash: 0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?someparam=true
  ```
  **This behaviour change affects all rest endpoints, not just the 2 new ones introduced here.**

  *(Note: I'd be open to implementing additional logic to refuse requests containing unrecognized query parameters to minimize behaviour change, but for the endpoints that we currently have I don't really see the point for that added complexity. E.g. I don't see any scenarios where misspelling a parameter could lead to harmful outcomes)*

  ## Using the REST API

  To run the API HTTP server, start a bitcoind instance with the `-rest` flag enabled. To use the
  `blockfilterheaders` endpoint, you'll also need to set `-blockfilterindex=1`:
  ```
  ./bitcoind -signet -rest -blockfilterindex=1
  ```

  As soon as bitcoind is fully up and running, you should be able to query the API, for example by
  using curl on the command line: ```curl "127.0.0.1:38332/rest/chaininfo.json"```.
  To more easily parse the JSON output, you can also use tools like 'jq' or `json_pp`, e.g.:
  ```
  curl -s "localhost:38332/rest/blockfilterheaders/basic/0000004c6aad0c89c1c060e8e116dcd849e0554935cd78ff9c6a398abeac6eda.json?count=2" | json_pp .
  ```

  ## To do
  - [x] update `doc/release-notes`

  ## Feedback
  This is my first PR (hooray!). Please don't hold back on any feedback/comments/nits/... you may have, big or small, whether they are code, process, language, ... related. I welcome private messages too if there's anything you don't want to clutter the PR with. I'm here to learn and am grateful for everyone's input.

ACKs for top commit:
  stickies-v:
    I've had to push a tiny doc update to `REST-interface.md` (`git range-diff 219d728 9aac438 54b39cf`) since this was not merged for v23, but since there are no significant changes beyond theStack and jnewbery's ACKs I think this PR is now ready to be considered for merging? @MarcoFalke
  jnewbery:
    ACK 54b39cfb34
  theStack:
    re-ACK 54b39cfb34

Tree-SHA512: 3b393ffde34f25605ca12c0b1300799a19684b816a1d03aed38b0f5439df47bfe6a589ffbcd7b83fd2def6c9d00a1bae5e45b1d18df4ae998c617c709990f83f
2022-04-06 09:25:56 +02:00
stickies-v
f959fc0397
Update /<count>/ endpoints to use a '?count=' query parameter instead
In most RESTful APIs, path parameters are used to represent resources, and
query parameters are used to control how these resources are being filtered/sorted/...

The old /<count>/ functionality is kept alive to maintain backwards compatibility,
but new paths with query parameters are introduced and documented as the default
interface so future API methods don't break consistency by using query parameters.
2022-04-05 13:19:37 -04:00
Michael Dietz
8b9efebb0a
refactor: use named args when ScriptToUniv or TxToUniv are invoked 2022-03-30 20:00:27 +01:00
Michael Dietz
828a094ecf
refactor: merge ScriptPubKeyToUniv & ScriptToUniv into one function 2022-03-30 20:00:23 +01:00
MarcoFalke
601bfc417d
Merge bitcoin/bitcoin#24515: Only load BlockMan in BlockMan member functions
f865cf8ded Add and use BlockManager::GetAllBlockIndices (Carl Dong)
28ba0313ea Add and use CBlockIndexHeightOnlyComparator (Carl Dong)
12eb05df63 move-only: Move CBlockIndexWorkComparator to blockstorage (Carl Dong)
c600ee3816 Only load BlockMan in BlockMan member functions (Carl Dong)
42e56d9b18 style-only: No need for std::pair for vSortedByHeight (Carl Dong)
3bbb6fea05 style-only: Various blockstorage.cpp cleanups (Carl Dong)
5be9ee3c54 refactor: more const annotations for uses of CBlockIndex* (Anthony Towns)

Pull request description:

  The only important commit is "Only load BlockMan in BlockMan member functions", everything else is all just small style changes.

  Here's the commit message, reproduced:
  ```
  This commit effectively splits the "load block index itself" logic from
  "derive Chainstate variables from loaded block index" logic.

  This means that BlockManager::LoadBlockIndex{,DB} will only load what's
  relevant to the BlockManager.
  ```

ACKs for top commit:
  ajtowns:
    ACK f865cf8ded ; code review only
  MarcoFalke:
    review ACK f865cf8ded 🗂

Tree-SHA512: 7b204d782834e06fd7329d022e2ae860181b4e8105c33bfb928539a4ec24161dc7438a9c4d4ee279dcad77de310c160b997bb8aa18923243d0fd55ccf4ad7c3a
2022-03-17 07:23:43 +01:00
MarcoFalke
fa2a5f301a
rpc: Move mempool RPCs to new file
Can be reviewed with:
--color-moved=dimmed-zebra --color-moved-ws=ignore-all-space
2022-03-11 17:46:58 +01:00
stickies-v
fff771ee86
Handle query string when parsing data format
URLs may contain a query string (prefixed with '?') and this should be ignored when parsing
the data format.

To facilitate testing this functionality, ParseDataFormat has been made non-static.
2022-03-10 12:01:53 +01:00
stickies-v
c1aad1b3b9
scripted-diff: rename RetFormat to RESTResponseFormat
As RetFormat is now exposed in a header, it is renamed to the more
understandable RESTResponseFormat
-BEGIN VERIFY SCRIPT-
s() { sed -i 's/RetFormat/RESTResponseFormat/g' $1; }
s src/rest.cpp
s src/rest.h
-END VERIFY SCRIPT-
2022-03-10 11:29:09 +01:00
stickies-v
9f1c54787c
Refactoring: move declarations to rest.h
This facilitates unit testing
2022-03-10 11:24:15 +01:00
Anthony Towns
5be9ee3c54 refactor: more const annotations for uses of CBlockIndex* 2022-03-09 14:32:47 -05:00
Russell Yanofsky
90fc8b089d Add src/node/* code to node:: namespace 2022-01-06 22:14:16 -05:00
fanquake
9d099b02d8
Merge bitcoin/bitcoin#23836: rest: Expose block filters follow-ups
4523d28b6b [test] compare filter and header with the result of the getblockfilter RPC (Niklas Gögge)
3a2464f216 [rest] drop superfluous rpc serializations flags for block filters (Niklas Gögge)
064abd14a5 [rest] add a more verbose error message for invalid header counts (Niklas Gögge)
83b8f3a896 [refactor] various style fix-ups (Niklas Gögge)

Pull request description:

  This PR addresses unresolved review comments from [#17631](https://github.com/bitcoin/bitcoin/pull/17631).
  This includes:
  * various style fix-ups
  * returning a more verbose error message for invalid header counts
  * removing superfluous rpc serializations flags for block filters
  * improving the test to include comparing the block filters returned from the rest  with the ones returned from the `getblockfilter` RPC.

ACKs for top commit:
  jnewbery:
    ACK 4523d28b6b
  brunoerg:
    tACK 4523d28b6b

Tree-SHA512: 634e6b2ae3e1d5f31675a50cfe11a5e74bf5a51b9e7b512d9e18879bf2ed424fc0ac6ec633023f192e3ad12cf0c73b0b51de79dd7ec00844dba3e1493d823b8c
2022-01-02 08:37:12 +08:00
Hennadii Stepanov
f47dda2c58
scripted-diff: Bump copyright headers
-BEGIN VERIFY SCRIPT-
./contrib/devtools/copyright_header.py update ./
-END VERIFY SCRIPT-

Commits of previous years:
* 2020: fa0074e2d8
* 2019: aaaaad6ac9
2021-12-30 19:36:57 +02:00