bitcoin/doc
Andrew Chow e7b0004b37
Merge bitcoin/bitcoin#27596: assumeutxo (2)
edbed31066 chainparams: add signet assumeutxo param at height 160_000 (Sjors Provoost)
b8cafe3871 chainparams: add testnet assumeutxo param at height 2_500_000 (Sjors Provoost)
99839bbfa7 doc: add note about confusing HaveTxsDownloaded name (James O'Beirne)
7ee46a755f contrib: add script to demo/test assumeutxo (James O'Beirne)
42cae39356 test: add feature_assumeutxo functional test (James O'Beirne)
0f64bac603 rpc: add getchainstates (James O'Beirne)
bb05857794 refuse to activate a UTXO snapshot if mempool not empty (James O'Beirne)
ce585a9a15 rpc: add loadtxoutset (James O'Beirne)
62ac519e71 validation: do not activate snapshot if behind active chain (James O'Beirne)
9511fb3616 validation: assumeutxo: swap m_mempool on snapshot activation (James O'Beirne)
7fcd21544a blockstorage: segment normal/assumedvalid blockfiles (James O'Beirne)
4c3b8ca35c validation: populate nChainTx value for assumedvalid chainstates (James O'Beirne)
49ef778158 test: adjust chainstate tests to use recognized snapshot base (James O'Beirne)
1019c39982 validation: pruning for multiple chainstates (James O'Beirne)
373cf91531 validation: indexing changes for assumeutxo (James O'Beirne)
1fffdd76a1 net_processing: validationinterface: ignore some events for bg chain (James O'Beirne)
fbe0a7d7ca wallet: validationinterface: only handle active chain notifications (James O'Beirne)
f073917a9e validationinterface: only send zmq notifications for active (James O'Beirne)
4d8f4dcb45 validation: pass ChainstateRole for validationinterface calls (James O'Beirne)
1e59acdf17 validation: only call UpdatedBlockTip for active chainstate (James O'Beirne)
c6af23c517 validation: add ChainstateRole (James O'Beirne)
9f2318c76c validation: MaybeRebalanceCaches when chain leaves IBD (James O'Beirne)
434495a8c1 chainparams: add blockhash to AssumeutxoData (James O'Beirne)
c711ca186f assumeutxo: remove snapshot during -reindex{-chainstate} (James O'Beirne)
c93ef43e4f bugfix: correct is_snapshot_cs in VerifyDB (James O'Beirne)
b73d3bbd23 net_processing: Request assumeutxo background chain blocks (Suhas Daftuar)

Pull request description:

  - Background and FAQ: https://github.com/jamesob/assumeutxo-docs/tree/2019-04-proposal/proposal
  - Prior progress/project: https://github.com/bitcoin/bitcoin/projects/11
  - Replaces https://github.com/bitcoin/bitcoin/pull/15606, which was closed due to Github slowness. Original description and commentary can be found there.

  ---

  This changeset finishes the first phase of the assumeutxo project. It makes UTXO snapshots loadable via RPC (`loadtxoutset`) and adds `assumeutxo` parameters to chainparams. It contains all the remaining changes necessary to both use an assumedvalid snapshot chainstate and do a full validation sync in the background.

  This may look like a lot to review, but note that
  - ~200 lines are a (non-essential) demo shell script
  - Many lines are functional test, documentation, and relatively dilute RPC code.

  So it shouldn't be as burdensome to review as the linecount might suggest.

  - **P2P**: minor changes are made to `init.cpp` and `net_processing.cpp` to make simultaneous IBD across multiple chainstates work.
  - **Pruning**: implement correct pruning behavior when using a background chainstate
  - **Blockfile separation**: to prevent "fragmentation" in blockfile storage, have background chainstates use separate blockfiles from active snapshot chainstates to avoid interleaving heights and impairing pruning.
  - **Indexing**: some `CValidationInterface` events are given with an additional parameter, ChainstateRole, and all indexers ignore events from ChainstateRole::ASSUMEDVALID so that indexation only happens sequentially.
  - Have `-reindex` properly wipe snapshot chainstates.
  - **RPC**: introduce RPC commands `loadtxoutset` and (hidden) `getchainstates`.
  - **Release docs & first assumeutxo commitment**: add notes and a particular assumeutxo hash value for first AU-enabled release.
    - This will complete the project and allow use of UTXO snapshots for faster node bootstrap.

  The next phase, if it were to be pursued, would be coming up with a way to distribute the UTXO snapshots over the P2P network.

  ---

  ### UTXO snapshots

  Create your own with `./contrib/devtools/utxo_snapshot.sh`, e.g.
  ```shell
  ./contrib/devtools/utxo_snapshot.sh 788000 utxo.dat ./src/bitcoin-cli -datadir=$(pwd)/testdata`)
  ```
  or use the pre-generated ones listed below.

  - Testnet: **2'500'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:511e09f4bf853aefab00de5c070b1e031f0ecbe9&dn=utxo-testnet-2500000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `79db4b025448cc0ac388d8589a28eab02de53055d181e34eb47391717aa16388`
  - Signet: **160'000** (Sjors):
    - torrent: `magnet:?xt=urn:btih:9da986cb27b3980ea7fd06b21e199b148d486880&dn=utxo-signet-160000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`
    - sha256: `eeeca845385ba91e84ef58c09d38f98f246a24feadaad57fe1e5874f3f92ef8c`
  - Mainnet: **800'000** (Sjors):
    - Note: this needs the following commit cherry-picked in: 24deb2022b
    - torrent: `magnet:?xt=urn:btih:50ee955bef37f5ec3e5b0df4cf0288af3d715a2e&dn=utxo-800000.dat&tr=udp%3A%2F%2Ftracker.bitcoin.sprovoost.nl%3A6969`

  ### Testing

  #### For fun (~5min)

  If you want to do a quick test, you can run `./contrib/devtools/test_utxo_snapshots.sh` and follow the instructions. This is mostly obviated by the functional tests, though.

  #### For real (longer)

  If you'd like to experience a real usage of assumeutxo, you can do that too.
  I've cut a new snapshot at height 788'000 (http://img.jameso.be/utxo-788000.dat - but you can do it yourself with `./contrib/devtools/utxo_snapshot.sh` if you want). Download that, and then create a datadir for testing:
  ```sh
  $ cd ~/src/bitcoin  # or whatever

  # get the snapshot
  $ curl http://img.jameso.be/utxo-788000.dat > utxo-788000.dat

  # you'll want to do this if you like copy/pasting
  $ export AU_DATADIR=/home/${USER}/au-test # or wherever

  $ mkdir ${AU_DATADIR}
  $ vim ${AU_DATADIR}/bitcoin.conf

  dbcache=8000  # or, you know, something high
  blockfilterindex=1
  coinstatsindex=1
  prune=3000
  logthreadnames=1
  ```
  Obtain this branch, build it, and then start bitcoind:
  ```sh
  $ git remote add jamesob https://github.com/jamesob/bitcoin
  $ git fetch jamesob assumeutxo
  $ git checkout jamesob/assumeutxo

  $ ./configure $conf_args && make  # (whatever you like to do here)

  # start 'er up and watch the logs
  $ ./src/bitcoind -datadir=${AU_DATADIR}
  ```
  Then, in some other window, load the snapshot
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} loadtxoutset $(pwd)/utxo-788000.dat
  ```

  You'll see some log messages about headers retrieval and waiting to see the snapshot in the headers chain. Once you get the full headers chain, you'll spend a decent amount of time (~10min) loading the snapshot, checking it, and flushing it to disk. After all that happens, you should be syncing to tip in pretty short order, and you'll see the occasional `[background validation]` log message go by.

  In yet another window, you can check out chainstate status with
  ```sh
  $ ./src/bitcoin-cli -datadir=${AU_DATADIR} getchainstates
  ```
  as well as usual favorites like `getblockchaininfo`.

ACKs for top commit:
  achow101:
    ACK edbed31066

Tree-SHA512: 6086fb9a38dc7df85fedc76b30084dd8154617a2a91e89a84fb41326d34ef8e7d7ea593107afba01369093bf8cc91770621d98f0ea42a5b3b99db868d2f14dc2
2023-10-02 17:09:44 -04:00
..
design Merge bitcoin/bitcoin#27596: assumeutxo (2) 2023-10-02 17:09:44 -04:00
man scripted-diff: Fix typo in stub manual pages 2021-01-12 16:46:55 +01:00
policy Replace MAX_PACKAGE_SIZE with MAX_PACKAGE_WEIGHT to avoid vbyte confusion 2023-09-20 08:10:30 -04:00
release-notes doc: Fix broken link in release notes 2023-05-26 09:47:15 +02:00
.gitignore Ignore Doxyfile generated from Doxyfile.in template. 2017-04-07 16:28:12 +02:00
assets-attribution.md [doc] Merge doc/assets-attribution.md into contrib/debian/copyright 2015-09-18 18:14:42 +02:00
benchmarking.md doc: update doc/benchmarking.md 2021-06-24 11:15:29 +02:00
bips.md doc: remove version number from bips.md 2023-05-11 17:49:16 +01:00
bitcoin-conf.md Security config warning 2022-08-19 23:18:13 -04:00
bitcoin_logo_doxygen.png
build-android.md build, qt: Use Android NDK r23 LTS 2021-12-05 03:00:02 +02:00
build-freebsd.md Merge bitcoin/bitcoin#26773: doc: FreeBSD build doc updates to reflect removal of install_db4.sh 2023-02-16 14:50:00 +00:00
build-netbsd.md doc: update for NetBSD 9.2, add GUI Build Instructions 2022-07-06 21:27:05 -04:00
build-openbsd.md doc: update OpenBSD build docs for 7.3 (external signer support available) 2023-04-11 22:59:08 +02:00
build-osx.md build: remove dmg dependencies 2023-09-15 13:47:50 +01:00
build-unix.md doc: remove Fedora libdb4-*-devel install docs 2023-08-07 14:48:35 +02:00
build-windows.md doc: Correct linked Microsoft URLs 2022-12-31 16:54:13 +01:00
cjdns.md doc: update getnodeaddresses for CJDNS, I2P and Tor and rm link 2023-06-01 15:52:51 +02:00
dependencies.md random: switch to using getrandom() directly 2023-05-20 17:20:01 +01:00
descriptors.md doc: correct deriveaddresses RPC name 2022-10-30 18:11:45 +00:00
developer-notes.md doc: describe 'init load' thread actions 2023-07-07 19:31:27 -03:00
dnsseed-policy.md Correct spelling mistakes in doc folder 2015-10-18 06:25:43 +10:00
Doxyfile.in Generate doxygen documentation for test sources 2021-05-19 22:08:18 -07:00
external-signer.md Move external signer out of wallet module 2021-04-08 17:56:00 +02:00
files.md Ignore banlist.dat 2021-07-30 11:21:51 +02:00
fuzzing.md docs: fixup honggfuzz patch 2023-07-03 11:00:57 +01:00
guix.md docs: Point to contrib/guix/README.md in doc/guix.md 2021-01-08 11:40:01 -05:00
i2p.md doc: simplify the router options in doc/i2p.md 2023-06-27 10:26:57 -06:00
init.md Remove -sysperms option 2023-02-05 08:09:16 +00:00
JSON-RPC-interface.md doc: document json rpc endpoints 2023-03-09 08:41:59 +00:00
managing-wallets.md Change dots to an ellipsis and fix capitalization 2023-01-12 11:15:02 -05:00
multisig-tutorial.md doc: update multisig-tutorial.md to default wallet type 2022-03-10 12:50:34 +01:00
p2p-bad-ports.md init, doc: improve -onlynet help and tor/i2p documentation 2022-03-03 16:14:01 +01:00
productivity.md doc: add distcc to productivity notes 2022-06-13 16:13:11 +02:00
psbt.md doc: M-of-N multisig using descriptor wallets and PSBTs, as well as a signing flow 2021-08-16 10:43:07 +05:00
README.md doc: consolidate & remove release-note fragments 2022-09-15 16:02:17 +01:00
README_doxygen.md doc: Improve doxygen readme navigation section 2019-09-23 19:22:06 -04:00
README_windows.txt doc: Remove version numbers from READMEs 2017-04-05 09:40:48 +02:00
reduce-memory.md doc: Be vague instead of wrong about MALLOC_ARENA_MAX 2023-09-20 17:12:24 +00:00
reduce-traffic.md doc: Use precise permission flags where possible 2020-07-10 15:37:42 +02:00
release-notes-24914.md doc: Add release note for wallet loading changes 2023-06-27 11:08:06 -04:00
release-notes-25158.md doc: add release notes for abandoned field in gettransaction and listtransactions 2023-01-13 10:29:05 -03:00
release-notes-26076.md doc: clarify PR 26076 release note 2023-05-08 16:07:15 +02:00
release-notes-26094.md rpc: return block hash & height in getbalances, gettransaction & getwalletinfo JSONs 2023-04-26 16:07:47 +02:00
release-notes-26485.md RPC: Allow RPC methods accepting options to take named parameters 2023-05-03 11:27:51 -05:00
release-notes-27213.md doc: Add release note 2023-08-03 13:16:38 -06:00
release-notes-27302.md init: Error if ignored bitcoin.conf file is found 2023-04-21 06:53:23 -04:00
release-notes-27501.md [rpc] add getprioritisedtransactions 2023-05-10 21:10:44 +01:00
release-notes-27596.md rpc: add getchainstates 2023-09-30 06:41:23 -04:00
release-notes-27632.md doc: release note re raising on invalid -debug/debugexclude/loglevel 2023-06-15 10:27:56 -06:00
release-notes-27757.md doc: add release note about removal of deprecatedrpc=walletwarningfield flag 2023-06-04 23:53:26 +02:00
release-notes-28113.md doc: Correct release-notes for sighashtype exceptions 2023-07-27 09:16:11 +02:00
release-notes-28354.md doc: Release notes for testnet defaulting to -acceptnonstdtxn=0 2023-08-28 22:09:39 +10:00
release-notes-28414.md doc: add release note 2023-09-15 16:53:59 +01:00
release-notes-28448.md rpc: Deprecate rpcserialversion=0 2023-09-11 17:21:53 +10:00
release-notes-empty-template.md depends: Bump MacOS minimum runtime requirement to 11.0 2023-06-22 15:28:47 +00:00
release-process.md build: remove dmg dependencies 2023-09-15 13:47:50 +01:00
REST-interface.md Merge bitcoin/bitcoin#26207: rest: add verbose and mempool_sequence query params for mempool/contents 2023-03-15 19:39:30 -04:00
shared-libraries.md doc: libbitcoinconsensus: add missing error code description, fix NBitcoin link 2020-12-05 13:37:00 +01:00
tor.md Merge bitcoin/bitcoin#27719: doc: remove Tor link & generalize onion getnodeaddresses RPC 2023-06-01 15:24:23 +01:00
tracing.md Use int32_t type for most transaction size/weight values 2023-06-12 19:47:19 +01:00
translation_process.md doc: Remove unnecessary steps from translations update process 2021-07-03 21:31:29 +02:00
translation_strings_policy.md doc: Do not translate technical or extremely rare errors 2020-05-05 04:46:08 +03:00
zmq.md validationinterface: only send zmq notifications for active 2023-09-30 06:38:47 -04:00

Bitcoin Core

Setup

Bitcoin Core is the original Bitcoin client and it builds the backbone of the network. It downloads and, by default, stores the entire history of Bitcoin transactions, which requires a few hundred gigabytes of disk space. Depending on the speed of your computer and network connection, the synchronization process can take anywhere from a few hours to a day or more.

To download Bitcoin Core, visit bitcoincore.org.

Running

The following are some helpful notes on how to run Bitcoin Core on your native platform.

Unix

Unpack the files into a directory and run:

  • bin/bitcoin-qt (GUI) or
  • bin/bitcoind (headless)

Windows

Unpack the files into a directory, and then run bitcoin-qt.exe.

macOS

Drag Bitcoin Core to your applications folder, and then run Bitcoin Core.

Need Help?

Building

The following are developer notes on how to build Bitcoin Core on your native platform. They are not complete guides, but include notes on the necessary libraries, compile flags, etc.

Development

The Bitcoin repo's root README contains relevant information on the development process and automated testing.

Resources

Miscellaneous

License

Distributed under the MIT software license.