项目作者: dan-da

项目描述 :
A command-line tool that performs Bitcoin wallet address discovery.
高级语言: PHP
项目地址: git://github.com/dan-da/hd-wallet-addrs.git
创建时间: 2015-12-31T02:29:19Z
项目社区:https://github.com/dan-da/hd-wallet-addrs

开源协议:

下载


hd-wallet-addrs

A command-line tool for finding bitcoin hd-wallet addresses that have received funds.

This tool does two primary things:

  1. derive hd-wallet addresses (both change and receive) according to bip32 rules.
  2. examines the blockchain to find the addresses that have actually been used. (received funds at least once)

A web frontend for this tool is available at:
https://mybitprices.info/hd-wallet-addrs.html

Both regular HD wallets (single address) and multi-sig wallets (eg Copay) are
supported.

Segwit addresses are generated if a ypub or zpub key is provided.
(ypub: segwit-p2sh, zpub: bech32)

Reports are available in json, plaintext, and html. Columns can be
changed or re-ordered via command-line.

hd-wallet-addrs is general purpose for anyone needing to discover which addresses
are actually used in their wallet, including change addresses.

The motivation for building this tool was to simplify extracting used wallet
addresses for accounting purposes. In particular for use with:

  • bitprices - a command line utility for wallet pricing history and cost-based accounting.
  • mybitprices.info - an easy-to-use web frontend to bitprices.

See also: hd-wallet-derive — a tool that derives bip32 addresses and private keys.

Let’s see some examples.

  1. $ ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR --logfile=/tmp/log.txt
  2. --- Wallet Discovery Report ---
  3. Found 3 Receive addresses and 2 Change addresses.
  4. Receive -- Used: 3 Unused: 0
  5. Change -- Used: 2 Unused: 0
  6. +------------------------------------+---------+----------------+------------+------------+---------+
  7. | addr | type | total_received | total_sent | balance | relpath |
  8. +------------------------------------+---------+----------------+------------+------------+---------+
  9. | 1Ge6rDuyCdYVGhXZjcK4251q67GXMKx6xK | Receive | 0.00120000 | 0.00100000 | 0.00020000 | 0/0 |
  10. | 1NVsB73WmDGXSxv77sh9PZENH2x3RRnkDY | Receive | 0.00130000 | 0.00100000 | 0.00030000 | 0/1 |
  11. | 1BkgqiHcvfnQ2wrPN5D2ycrvZas3nibMjC | Receive | 0.00040000 | 0.00000000 | 0.00040000 | 0/2 |
  12. | 12SisoiXLUEbkytL5Pzia1jBY8gJP5XN8D | Change | 0.00184874 | 0.00000000 | 0.00184874 | 1/0 |
  13. | 1CkvACVpFwkPnMG13w9kXXE9YcsiyL4pcY | Change | 0.00194876 | 0.00000000 | 0.00194876 | 1/1 |
  14. +------------------------------------+---------+----------------+------------+------------+---------+

We can change up the fields and specify to use bip44 derivation to generate an absolute path.

Tip: The abspath column is empty when —derivation=relative, which is the default.

  1. $ ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR --cols=type,abspath,relpath,addr --derivation=bip44 --logfile=/tmp/log.txt
  2. --- Wallet Discovery Report ---
  3. Found 3 Receive addresses and 2 Change addresses.
  4. Receive -- Used: 3 Unused: 0
  5. Change -- Used: 2 Unused: 0
  6. +---------+--------------+---------+------------------------------------+
  7. | type | abspath | relpath | addr |
  8. +---------+--------------+---------+------------------------------------+
  9. | Receive | m/44/0/0/0/0 | 0/0 | 1Ge6rDuyCdYVGhXZjcK4251q67GXMKx6xK |
  10. | Receive | m/44/0/0/0/1 | 0/1 | 1NVsB73WmDGXSxv77sh9PZENH2x3RRnkDY |
  11. | Receive | m/44/0/0/0/2 | 0/2 | 1BkgqiHcvfnQ2wrPN5D2ycrvZas3nibMjC |
  12. | Change | m/44/0/0/1/0 | 1/0 | 12SisoiXLUEbkytL5Pzia1jBY8gJP5XN8D |
  13. | Change | m/44/0/0/1/1 | 1/1 | 1CkvACVpFwkPnMG13w9kXXE9YcsiyL4pcY |
  14. +---------+--------------+---------+------------------------------------+

Or get a list for easy copy/paste.

  1. $ ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR --format=addrlist --logfile=/tmp/log.txt
  2. --- Wallet Discovery Report ---
  3. Found 3 Receive addresses and 2 Change addresses.
  4. Receive -- Used: 3 Unused: 0
  5. Change -- Used: 2 Unused: 0
  6. 1Ge6rDuyCdYVGhXZjcK4251q67GXMKx6xK
  7. 1NVsB73WmDGXSxv77sh9PZENH2x3RRnkDY
  8. 1BkgqiHcvfnQ2wrPN5D2ycrvZas3nibMjC
  9. 12SisoiXLUEbkytL5Pzia1jBY8gJP5XN8D
  10. 1CkvACVpFwkPnMG13w9kXXE9YcsiyL4pcY

Or JSON

  1. ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR --cols=type,abspath,relpath,addr --format=jsonpretty --derivation=bip44 --logfile=/tmp/log.txt
  2. [
  3. {
  4. "type": "Receive",
  5. "abspath": "m\/44\/0\/0\/0\/0",
  6. "relpath": "0\/0",
  7. "addr": "1Ge6rDuyCdYVGhXZjcK4251q67GXMKx6xK"
  8. },
  9. ...
  10. ]

Or CSV

  1. ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR --cols=type,abspath,relpath,addr --format=csv --derivation=bip44 --logfile=/tmp/log.txt
  2. type,abspath,relpath,addr
  3. Receive,m/44/0/0/0/0,0/0,1Ge6rDuyCdYVGhXZjcK4251q67GXMKx6xK
  4. Receive,m/44/0/0/0/1,0/1,1NVsB73WmDGXSxv77sh9PZENH2x3RRnkDY
  5. Receive,m/44/0/0/0/2,0/2,1BkgqiHcvfnQ2wrPN5D2ycrvZas3nibMjC
  6. Change,m/44/0/0/1/0,1/0,12SisoiXLUEbkytL5Pzia1jBY8gJP5XN8D
  7. Change,m/44/0/0/1/1,1/1,1CkvACVpFwkPnMG13w9kXXE9YcsiyL4pcY

multi-sig examples.

So far multi-sig has been tested with copay (bip44 and bip45) only.
Older versions of Copay using bip45 require the —derivation=copaylegacy flag.

multisig requires multiple xpub keys and use of the —numsig flag to indicate
the required number of signers. (m of n)

discovering an empty Copay 1.6.3+ (bip44) 2 of 3 wallet.

This test wallet has no funds, so we use —include-unused to obtain the initial addresses up to the
gap limit. The gap limit default is 20, but we use 2 here for brevity.

  1. $ ./hd-wallet-addrs.php -g --numsig=2 --gap-limit=2 --xpub=xpub6CZte6DfeMoVwxv3ShiMwQjET47nRENqrkZaSXTcP7Yaja6sxyRbiyqPD7kfy4W2dTTuTdV4jHMmSe1k1qteTMN7qDLndt1RfQ8RLM3pjzb,xpub6DUGj5hRwp7t3DoH554Ce7p3KLepccYfG5BVbvyPSArTepacc3aPRDTMz3GSdoX1HgVYKBSaR6fFDm1daEtSQFBSNTq4X93pd8dBFyPW2gz,xpub6DRFPDtHueJ5sfqzcLSyoKL6TQZMofvjpLzsVXsWqjgYuAtUtdU8YjWFvpa2xegWLFeLQ38KLJzWdKQ3CsAQQLoMYnBsQy3FCeTDuxgcsfK --include-unused --logfile=/tmp/out.txt
  2. --- Wallet Discovery Report ---
  3. Found 2 Receive addresses and 2 Change addresses.
  4. Receive -- Used: 0 Unused: 2
  5. Change -- Used: 0 Unused: 2
  6. +------------------------------------+---------+----------------+------------+------------+---------+
  7. | addr | type | total_received | total_sent | balance | relpath |
  8. +------------------------------------+---------+----------------+------------+------------+---------+
  9. | 339H3pYP9AKiEo74D1BWiSK8jhWXsrJ3yk | Receive | 0.00000000 | 0.00000000 | 0.00000000 | 0/0 |
  10. | 3NcBBWtDscKchgkUCY3eEQZgYh8STtcona | Receive | 0.00000000 | 0.00000000 | 0.00000000 | 0/1 |
  11. | 3QtjkbY8Km4v5KCgTZxD7VW2vPCsBqkV3V | Change | 0.00000000 | 0.00000000 | 0.00000000 | 1/0 |
  12. | 3B7xNx7dCT6ydcVF1xQpEtG8UFeeh2PyAk | Change | 0.00000000 | 0.00000000 | 0.00000000 | 1/1 |
  13. +------------------------------------+---------+----------------+------------+------------+---------+

discovering an empty Copay 1.1.x (bip45) 1 of 1 wallet.

Legacy versions of Copay used bip45 in a special way that the tool cannot detect without help.

Note the use of —derivation=copaylegacy

(Copay 1.6.3+ 1 of 1 wallets use bip44 derivation and do not require any special arguments.)

  1. $ ./hd-wallet-addrs.php -g --derivation=copaylegacy --gap-limit=2 --xpub=xpub697odnriKgTgWE4my6au8nd8haUfAMzLGFpDemAkRbCMgGVxANuj9DffNLgDjPA1dnxzi8oFmM79ZPgKVfCV7Saj8sQUL7tJfeZDuyQNGDm --include-unused --logfile=/tmp/out.txt
  2. --- Wallet Discovery Report ---
  3. Found 2 Receive addresses and 2 Change addresses.
  4. Receive -- Used: 0 Unused: 2
  5. Change -- Used: 0 Unused: 2
  6. +------------------------------------+---------+----------------+------------+------------+----------------+
  7. | addr | type | total_received | total_sent | balance | relpath |
  8. +------------------------------------+---------+----------------+------------+------------+----------------+
  9. | 3LHgjejeCnQEhLGpmc1q4RmPXypKhjbgpY | Receive | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/0/0 |
  10. | 3Jdd25xHSCDFrMeCoW62963vf22UoKBmtP | Receive | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/0/1 |
  11. | 3JZ3YR6sgyqq6xcGtpcAvYBCX7gM9cPU3c | Change | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/1/0 |
  12. | 32KNwkcQzBHYejvnJpWDwUWMbHGZd4Q6fH | Change | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/1/1 |
  13. +------------------------------------+---------+----------------+------------+------------+----------------+

Warning for users of Copay 1.6.2 and below

Older Copay versions made it possible to generate gaps larger than 20. This is
because it would generate a new address each time the receive screen was viewed
and did not respect the standard gap-limit of 20.

Checking only 20 addresses could possibly leave you without discovering funds.
If you suspect this may be happening, a workaround is to specify a larger gap
limit such as 100 via the gap-limit argument.

discovering an empty Copay 1.1.x (bip45) 2 of 2 wallet.

Again we must use —derivation=copaylegacy

  1. $ ./hd-wallet-addrs.php --derivation=copaylegacy -g --gap-limit=2 --xpub=xpub68bjYyPhqAwK4T8WtXuGvruSQoJu1vdLD7DYc591MkFCR7wD9gyzteFYmzRyytWJ2SzTqZNTgggvPEyqEy9oArjLF7xhte5js1Lp1EPipwJ,xpub68ufoGjY41tQqP4LpeyYornuNxm8DNy2Rn7KAPUTAwFouj821eqcVpWw1jonrm2Xg5jnnSrd1QPQzGve3f66ZLf6Ni9VY6aN3AjYa4e7XTE --numsig=2 --include-unused --logfile=/tmp/out.txt
  2. --- Wallet Discovery Report ---
  3. Found 2 Receive addresses and 2 Change addresses.
  4. Receive -- Used: 0 Unused: 2
  5. Change -- Used: 0 Unused: 2
  6. +------------------------------------+---------+----------------+------------+------------+----------------+
  7. | addr | type | total_received | total_sent | balance | relpath |
  8. +------------------------------------+---------+----------------+------------+------------+----------------+
  9. | 35uhrWpDTj3Y7EwR9AWjACGfT47txtpH1v | Receive | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/0/0 |
  10. | 3BnXxkW9CVCLn1EboGDJ8434eKFWZGHsjn | Receive | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/0/1 |
  11. | 38dzdCQXatNdT9nWG7thpGC9KjBVLphZRP | Change | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/1/0 |
  12. | 3CfbgQ5BxWRFBYXJxEVAmVCZsatdJfc2rS | Change | 0.00000000 | 0.00000000 | 0.00000000 | 2147483647/1/1 |
  13. +------------------------------------+---------+----------------+------------+------------+----------------+

How discovery works

In plain english, discovery works by mathematically deriving the addresses
for your wallet in order and checking if each one has been used or not.

A slightly more technical description of the process:

  • starting from the extended public key (xpub)
  • for receive addresses, then change addresses
    • derive batches of xpub child addresses (bip32: 0/*)
    • for each batch
      • check if each address has received funds (API call to oracle/server)
      • until 20 (default) unused addresses in a row are found.

Privacy implications

An important thing to recognize is that unless you are running a toshi or
insight server locally, the discovery process will send your public addresses
to a third party. ie: BlockChain.info, BitPay (insight), or CoinBase (toshi)

The third party will have no way to spend your funds.

The third party could track your requests and guess/assume that your addresses
are associated with your IP, or are associated with eachother.

If that is something you care about, then you should investigate how to run
toshi or insight locally and use the —toshi or —insight flags to specify
the local server URL.

There is now a feature that helps to improve privacy when using third-party
API servers. The —api=roundrobin flag will cycle through the available
blockchain providers and send individual addresses to each. In this way, no
single provider will have access to all the queried wallet addresses.

Querying for individual addresses is slow. The —batch-size flag may be used
to increase the number of addresses sent to each provider.

Use at your own risk.

The author makes no claims or guarantees of correctness.

Output formats

The report may be printed in the following formats:

  • plain - an ascii formatted table, as above. intended for humans.
  • csv - CSV format. For spreadsheet programs.
  • json - raw json format. for programs to read easily.
  • jsonpretty - pretty json format. for programs or humans.
  • addrlist - single column address list. for easy cut/paste.

Additionally, the report may contain incoming transactions only, outgoing
transactions only, or both types.

Usage

  1. $ ./hd-wallet-addrs.php
  2. hd-wallet-addrs.php
  3. This script discovers bitcoin HD wallet addresses that have been used.
  4. Options:
  5. -g go! ( required )
  6. --xpub=<csv> comma separated list of xpub keys
  7. --xpubfile=<path> file containing xpub keys, one per line.
  8. note: multiple keys implies multisig m of n.
  9. --derivation=<type> bip32|bip44|bip45|copaylegacy|relative.
  10. default=relative
  11. --numsig=<int> number of required signers for m-of-n multisig wallet.
  12. (required for multisig)
  13. --gap-limit=<int> bip32 unused addr gap limit. default=20
  14. --include=<type> include which addresses. one of [used, unused, both]
  15. note that unused addresses are subject to --gap-limit
  16. --include-unused equivalent to --include=both
  17. --gen-only=<n> will generate n receive addresses and n change addresses
  18. but will not query the blockchain to determine if they
  19. have been used.
  20. --type=<type> receive|change|both. default=both
  21. --api=<api> toshi|insight|blockchaindotinfo|btcd|roundrobin
  22. default = blockchaindotinfo (fastest)
  23. roundrobin will use a different API for each batch
  24. to improve privacy. It also sets --batch-size to
  25. 1 if set to auto.
  26. --batch-size=<n> integer|auto default=auto.
  27. The number of addresses to lookup in each batch.
  28. --cols=<cols> a csv list of columns, or "all"
  29. all:
  30. (addr,type,total_received,total_sent,balance,relpath,abspath,xpub)
  31. default:
  32. (addr,type,total_received,total_sent,balance,relpath)
  33. --outfile=<path> specify output file path.
  34. --format=<format> txt|csv|json|jsonpretty|html|addrlist|all default=txt
  35. if all is specified then a file will be created
  36. for each format with appropriate extension.
  37. only works when outfile is specified.
  38. --toshi=<url> toshi server. defaults to https://bitcoin.toshi.io
  39. --insight=<url> insight server. defaults to https://insight.bitpay.com/api
  40. --blockchaindotinfo=<url>
  41. blockchain.info server. defaults to https://blockchain.info
  42. --btcd=<url> btcd rpc server. specify as http://user:pass@host:port. https ok also
  43. btcd does not return balance or total sent/received.
  44. --oracle-raw=<p> path to save raw server response, optional.
  45. --oracle-json=<p> path to save formatted server response, optional.
  46. --logfile=<file> path to logfile. if not present logs to stdout.
  47. --loglevel=<level> debug,info,specialinfo,warning,exception,fatalerror
  48. default = info

Installation and Running.

PHP’s gmp extension is required. Here’s how to install on ubuntu.

  1. sudo apt-get install php-cli php-gmp composer

Basics

  1. git clone https://github.com/dan-da/hd-wallet-addrs
  2. cd hd-wallet-addrs
  3. composer install

Try an example

  1. ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR

Or to hide log messages

  1. ./hd-wallet-addrs.php -g --xpub=xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR --logfile=/tmp/log.txt

Run Test cases

  1. cd tests
  2. ./test_runner.php

It is really slow to generate keys in PHP. For a huge speedup, you can install
the secp256k1 extension from:

https://github.com/Bit-Wasp/secp256k1-php

Blockchain API provider notes.

tip! use the —api flag to switch between blockchain API providers.

Each API has strengths and weaknesses. Some are faster than others,
or easier/harder to run locally. The blockchain.info service is recommended
because it presently has the fastest API, and it is the default.

For best privacy, one should query an oracle that is running locally.
Esplora, Insight, toshi, and btcd can be operated this way.

blockchain.info

as of 2024-06-03: tested WORKING.

as of 2015-12-30:

  • supports multi address lookup in a single call.
  • max addrs per call: unknown.
  • returns extra un-needed info such as last 50 tx.
  • returns addresses in different order than requested.

esplora (by blockstream.info)

as of 2024-6-03: tested WORKING.

as of 2019-08-06:

  • does NOT support multi address lookup in a single call. feature requested.
  • open source, can be run locally.

btc.com

as of 2024-6-03: tested NOT WORKING.

as of 2019-08-06

  • API is returning 403 forbidden error except when requested via web browser. ymmv.

as of 2018-07-23:

  • supports multi address lookup in a single call.
  • max addrs per call: unknown.
  • returns an index with NULL value for any addresses without received funds.

bitcoin-core

as of 2019-08-06:

  • still does not support address index or API.
  • There is a chance this pull request might get merged.

as of 2015-12-30:

  • does not provide a suitable API for querying address total_received
  • does not have a public address index. Implementing an API would be difficult.
  • some 3rd party party patches or external solutions exist for creating an address index.
  • not supported by hd-wallet-addrs (sadly).

blockcypher.com

as of 2024-6-03: tested NOT WORKING.

as of 2018-07-23:

  • does support multi address lookup in a single call via batching.
  • max addrs per batched call is 100. however:
  • each address is counted as a request internally, and more than
    3 triggers the rate limiting, so the request fails. Thus, 100
    can only be achieved with an API key, and the limit for free
    usage is effectively 3.
  • See https://github.com/blockcypher/explorer/issues/245

Insight

as of 2024-06-03: tested NOT WORKING.

as of 2019-08-06

as of 2015-12-30:

  • does NOT support multi address lookup in a single call.
  • each candidate address must be queried separately.

blockr.io

as of 2017-09-04:

  • Dead. Killed by Coinbase.com.
  • Read the obituary.
  • R.I.P. blockr

as of 2016-02-16:

  • supports multi address lookup in a single call.
  • limits number of addresses per call to 20.
  • does not return un-needed tx data.

btcd

as of 2017-05-21:

  • btcd can now be queried from hd-wallet-addrs to find used wallet addresses, but values for balance/sent/received are empty.
  • does not support multi address lookup, so is not that fast.
  • is probably the simplest way to run a local oracle.
  • See this issue.

as of 2015-12-30:

  • does not provide a suitable API for querying address total_received or balance.
  • does have a public address index that should make such an API possible, if not performant.

Toshi

as of 2019-08-06:

as of 2017-05-21:

  • toshi.io no longer exists since Dec 31, 2016.
  • toshi can still be run locally by installing from github.
  • See the Coinbase announcement here.

as of 2015-12-30:

  • does NOT support multi address lookup in a single call.
  • each candidate address must be queried separately.

Embed in your own PHP project

Here’s a quick example how you one can access the API directly without invoking
the CLI program.

in your project’s composer.json:

  1. {
  2. "require": {
  3. "dan-da/hd-wallet-addrs": "0.2.0"
  4. }
  5. }

yourproject.php

  1. require_once 'vendor/autoload.php';
  2. // normally a single xpub is used. multiple can be provided for multisig key generation.
  3. $xpub_list = ['xpub6BfKpqjTwvH21wJGWEfxLppb8sU7C6FJge2kWb9315oP4ZVqCXG29cdUtkyu7YQhHyfA5nt63nzcNZHYmqXYHDxYo8mm1Xq1dAC7YtodwUR'];
  4. $params = walletaddrs::default_params();
  5. // modify params here if you need to. see default_params() for keys.
  6. $wa = new walletaddrs($params);
  7. $data = $wa->discover_wallet_addrs( $xpub_list );
  8. echo json_encode($data, JSON_PRETTY_PRINT);
  9. // or for fancier printing, you could use:
  10. // walletaddrsreport::print_results($wa->get_params(), $data);

Thanks

A big thank-you to the author of bitwasp/bitcoin-php. This library does the
heavy lifting of dealing with deterministic keys and multisig, amongst other
things.

Todos

  • add option to return only Receive or Change instead of both.
  • test with additional wallet software.
  • Add bip39 support to obtain xpub from secret words. maybe?
  • Add suitable API to btcd.