Starting a test chain from state taken from mainnet or testnet

Purpose

For testing purposes, it is often desirable to start a test chain with a starting state that looks like mainnet or testnet. This is usually done for the purpose of testing changes to neard itself, but it's also possible to do this if you're a contract developer and want to see what a change to your contract would look like on top of the current mainnet state. At the end of the process described here, you'll have a set of genesis records that can be used to start your own test chain, that'll be like any other test chain like the ones generated by the neard localnet command, except with account balances and data taken from mainnet

How-to

The first step is to obtain an RPC node home directory for the chain you'd like to spoon. So if you want to use mainnet state, you can follow the instructions here to obtain a recent snapshot of a mainnet node's home directory. Once you have your node's home directory set up, run the following state-viewer command to generate a dump of the chain's state:

$ neard --home $NEAR_HOME_DIRECTORY view-state dump-state --stream

This command will take a while (possibly many hours) to run. But at the end you should have genesis.json and records.json files in $NEAR_HOME_DIRECTORY/output. This records file represents all of the chain's current state, and is what we'll use to start our chain.

From here, we need to make some changes to the genesis.json that was generated in $NEAR_HOME_DIRECTORY/output. To see why, note that the validators field of this genesis file lists all the current mainnet validators and their keys. So that means if we were to try and start a test chain from the generated genesis and records files as-is, it would work, but our node would expect the current mainnet validators to be producing blocks and chunks (which they definitely won't be! Because we're the only ones who know or care about this new test chain).

So we need to select a new list of validators to start off our chain. Suppose that we want our chain to have two validators, validator0.near and validator1.near. Let's make a new directory where we'll be storing intermediate files during this process:

$ mkdir ~/test-chain-scratch

then using your favorite editor, lay out the validators you want in the test chain as a JSON list in the same format as the validators field in genesis.json, maybe in the file ~/test-chain-scratch/validators.json

[
  {
    "account_id": "validator0.near",
    "public_key": "ed25519:GRAFkrqEkJAbdbWUgc6fDnNpCTE83C3pzdJpjAHkMEhq",
    "amount": "100000000000000000000000000000000"
  },
  {
    "account_id": "validator1.near",
    "public_key": "ed25519:5FxQQTC9mk5kLAhTF9ffDMTXiyYrDXyGYskgz46kHMdd",
    "amount": "100000000000000000000000000000000"
  }
]

These validator keys should be keys you've already generated. So for the rest of this document, we'll assume you've run:

$ neard --home ~/near-test-chain/validator0 init --account-id validator0.near
$ neard --home ~/near-test-chain/validator1 init --account-id validator1.near

This is also a good time to think about what extra accounts you might want in your test chain. Since all accounts in the test chain will have the same keys as they do on mainnet, you'll only have access to the accounts that you have access to on mainnet. If you want to add an account with a large balance to properly test things out, you can write them out in a file as a JSON list of state records (in the same format as they appear in records.json). For example, you could put the following in ~/test-chain-scratch/extra-records.json:

[
  {
    "Account": {
      "account_id": "my-test-account.near",
      "account": {
        "amount": "10000000000000000000000000000000000",
        "locked": "0",
        "code_hash": "11111111111111111111111111111111",
        "storage_usage": 182,
        "version": "V1"
      }
    }
  },
  {
    "AccessKey": {
      "account_id": "my-test-account.near",
      "public_key": "ed25519:Eo9W44tRMwcYcoua11yM7Xfr1DjgR4EWQFM3RU27MEX8",
      "access_key": {
        "nonce": 0,
        "permission": "FullAccess"
      }
    }
  }
]

You'll want to include an access key here, otherwise you won't be able to do anything with the account. Note that here you can also add access keys for any mainnet account you want, so you'll be able to control it in the test chain.

Now to make these changes to the genesis and records files, you can use the neard amend-genesis command like so:

# mkdir ~/near-test-chain/
$ neard amend-genesis --genesis-file-in $NEAR_HOME_DIRECTORY/output/genesis.json --records-file-in $NEAR_HOME_DIRECTORY/output/records.json --validators ~/test-chain-scratch/validators.json --extra-records ~/test-chain-scratch/extra-records.json --chain-id $TEST_CHAIN_ID --records-file-out ~/near-test-chain/records.json --genesis-file-out ~/near-test-chain/genesis.json

Starting the network

After running the previous steps you should have the files genesis.json and records.json in ~/near-test-chain/. Assuming you've started it with the two validators validator0.near and validator1.near as described above, you'll want to run at least two nodes, one for each of these validator accounts. If you're working with multiple computers or VMs that can connect to each other over the internet, you'll be able to run your test network over the internet as is done with the "real" networks (mainnet, testnet, etc.). But for now let's assume that you want to run this on only one machine.

So assuming you've initialized home directories for each of the validators with the init command described above, you'll want to copy the records and genesis files generated in the previous step to each of these:

$ cp ~/near-test-chain/records.json ~/near-test-chain/validator0
$ cp ~/near-test-chain/genesis.json ~/near-test-chain/validator0
$ cp ~/near-test-chain/records.json ~/near-test-chain/validator1
$ cp ~/near-test-chain/genesis.json ~/near-test-chain/validator1

Now we'll need to make a few config changes to each of ~/near-test-chain/validator0/config.json and ~/near-test-chain/validator1/config.json:

changes to ~/near-test-chain/validator0/config.json:

{
  "genesis_records_file": "records.json",
  "rpc": {
    "addr": "0.0.0.0:3030"
  },
  "network": {
    "addr": "0.0.0.0:24567",
    "boot_nodes": "ed25519:Dk4A7NPBYFPwKWouiSUoyZ15igbLSrcPEJqUqDX4grb7@127.0.0.1:24568",
    "skip_sync_wait": false,
  },
  "consensus": {
    "min_num_peers": 1
  },
  "tracked_shards": [0],
}

changes to ~/near-test-chain/validator1/config.json:

{
  "genesis_records_file": "records.json",
  "rpc": {
    "addr": "0.0.0.0:3031"
  },
  "network": {
    "addr": "0.0.0.0:24568",
    "boot_nodes": "ed25519:6aR4xVQedQ7Z9URrASgwBY8bedpaYzgH8u5NqEHp2hBv@127.0.0.1:24567",
    "skip_sync_wait": false,
  },
  "consensus": {
    "min_num_peers": 1
  },
  "tracked_shards": [0],
}

Here we make sure to have each node listen on different ports, while telling each about the other via network.boot_nodes. In this boot_nodes string, we set the public key not to the validator key, but to whatever key is present in the node_key.json file you got when you initialized the home directory. So for validator0's config, we set its boot node to validator1's node key, followed by the address of the socket it should be listening on. We also want to drop the minimum required number of peers, since we're just running a small test network locally. We set skip_sync_wait to false, because otherwise we get strange behavior that will often make your network stall.

After making these changes, you can try running one neard process for each of your validators:

$ neard --home ~/near-test-chain/validator0 run
$ neard --home ~/near-test-chain/validator1 run

Now these nodes will begin by taking the records laid out in records.json and turning them into a genesis state. At the time of this writing, using the latest nearcore version from the master branch, this will take a couple hours. But your validators should begin producing blocks after that's done.