how to solve: docker mainnet.sh snapshot import corrupted store, genesis block not available

Question

Been out of the Tezos for a bit and coming back.

I've got a Raspi 4 with external USB 3 SSD booting Ubuntu Server and docker/docker-compose installed. The SSD is 220G at 23% usage.

Trying to get a docker node running for baking/endorsing/etc with an attached ledger.

On the Raspi 4 I ssh into Ubuntu and work with an up-to-date mainnet.sh script.

I run the following:

1. wget https://snapshots-tezos.giganode.io/snapshots/mainnet_20220109-000440_BLvFmR1mDkcHbJbqyAiarXt6xe47RC2zUcdmGcLAdTa1CpSsYE1.full

$ mainnet.sh snapshot import /home/tezos/mainnet_20220109-000440_BLvFmR1mDkcHbJbqyAiarXt6xe47RC2zUcdmGcLAdTa1CpSsYE1.full
v11.0: Pulling from tezos/tezos
Digest: sha256:55dabff7e5a98f05d27d3562377d47b2b71a811ead7bc9bf8eaddc45261bfe7d
Status: Image is up to date for tezos/tezos:v11.0
docker.io/tezos/tezos:v11.0
Creating volume "mainnet_node_data" with default driver
Creating mainnet_importer_1 ... done
Attaching to mainnet_importer_1
importer_1                    | Current public chain: 2018-06-30T16:07:32Z.
importer_1                    | Removing outdated chain data...
importer_1                    | Jan  9 12:22:27.150 - node.snapshots: importing data from snapshot
importer_1                    | Jan  9 12:22:27.150 - node.snapshots:   /snapshot: chain TEZOS_MAINNET, block hash BLvFmR1mDkcHbJbqyAiarXt6xe47RC2zUcdmGcLAdTa1CpSsYE1 at level 2014943 in full (snapshot version 2)
importer_1                    | Jan  9 12:22:27.150 - node.snapshots: you may consider using the --block <block_hash> argument to ensure that the
importer_1                    | Jan  9 12:22:27.150 - node.snapshots:   block imported is the one you expected
importer_1                    | Jan  9 12:22:27.150 - node.snapshots: retrieving and validating data. This can take a while, please bear with us
$

No complaints during the import.

Then I try to bring up the tezos system via docker-compose:

$ mainnet.sh start
v11.0: Pulling from tezos/tezos
Digest: sha256:55dabff7e5a98f05d27d3562377d47b2b71a811ead7bc9bf8eaddc45261bfe7d
Status: Image is up to date for tezos/tezos:v11.0
docker.io/tezos/tezos:v11.0
Removing orphan container "mainnet_importer_1"
Creating mainnet_node_1     ... done
Creating mainnet_upgrader_1 ... done
Creating mainnet_endorser-010-PtGRANAD-test_1 ... done
Creating mainnet_endorser-alpha-test_1        ... done
Creating mainnet_accuser-alpha_1              ... done
Creating mainnet_baker-010-PtGRANAD_1         ... done
Creating mainnet_baker-alpha_1                ... done
Creating mainnet_baker-011-PtHangz2-test_1    ... done
Creating mainnet_endorser-alpha_1             ... done
Creating mainnet_accuser-011-PtHangz2_1       ... done
Creating mainnet_accuser-011-PtHangz2-test_1  ... done
Creating mainnet_baker-011-PtHangz2_1         ... done
Creating mainnet_baker-010-PtGRANAD-test_1    ... done
Creating mainnet_endorser-011-PtHangz2-test_1 ... done
Creating mainnet_endorser-010-PtGRANAD_1      ... done
Creating mainnet_accuser-alpha-test_1         ... done
Creating mainnet_endorser-011-PtHangz2_1      ... done
Creating mainnet_accuser-010-PtGRANAD_1       ... done
Creating mainnet_baker-alpha-test_1           ... done
Creating mainnet_accuser-010-PtGRANAD-test_1  ... done

So far so good. Then I attach to mainnet_node_1 to watch its progress. First thing it does is generate a node identity for about 10 minutes or so and ends up failing on the snapshot.

Stored the new identity (idxxx...) into '/var/run/tezos/node/data/identity.json'.
Jan  9 13:10:25.030 - node.config.validation: the node configuration has been successfully validated.
Jan  9 13:10:25.035 - node.main: read identity file (peer_id = idxxx...)
Jan  9 13:10:25.036 - node.main: starting the Tezos node (chain = TEZOS_MAINNET)
Jan  9 13:10:25.037 - node.main: disabled local peer discovery
Jan  9 13:10:25.054 - node: shell-node initialization: bootstrapping
Jan  9 13:10:25.403 - node: shell-node initialization: p2p_maintain_started
Jan  9 13:10:25.404 - external_block_validator: initialized
Jan  9 13:10:33.903 - external_block_validator: block validator process started with pid 47
Jan  9 13:13:40.077 - node.store: the store is in an inconsistent state:
Jan  9 13:13:40.077 - node.store:   Error:
Jan  9 13:13:40.077 - node.store:     Failed to load on-disk data: no corresponding data found in file /var/run/tezos/node/data/store/chain_NetXdQprcVkpa/genesis.
Jan  9 13:13:40.078 - node.store: 
Jan  9 13:13:40.078 - node.store: attempting to restore the store's consistency...
tezos-node: Error:
              The store is corrupted irremediably: The genesis block is not available in the store..,
              trace:
              Failed to load on-disk data: no corresponding data found in file /var/run/tezos/node/data/store/chain_NetXdQprcVkpa/genesis.
              The store is corrupted irremediably: The genesis block is not available in the store

I've deleted the docker volume node_data a few times as things fail when the directory is not empty in the beginning and restart the process described above.

Any ideas on what's going wrong here?

I'm stumped and not sure where else to look.

Answer 1

Since I'm on a RaspPi 4 with 8G of RAM I thought perhaps I was running out of memory and the import was silently failing.

I temporarily added an additional 4G of swap space and the import succeeded.