6

So, I've set up a Tezos node on an AWS EC2 t2.medium instance. I followed the instructions here, but for alphanet instead of mainnet.

I got as far as: ./tezos-node run --rpc-addr :8732

It's running, and syncing, but VERY slowly. In a couple of hours of running, the data returned from 'client get timestamp' has moved forward only a day or two. I'm also frequently getting messages like p2p.maintenance: Too few connections (5)

I have tried opening ALL ports into and out of the instance just to make sure no traffic was being blocked. It makes no difference. What's wrong? What do I need to change?

6

You could try adding more peers if you are seeing the too few peers message. This script should work for the alphanet (you need jq installed)

for j in 0 1; do
  for i in `curl -s "http://api.alphanet.tzscan.io/v3/network?state=running&p=$j&number=50" \
    | jq -r '.[] | .point_id' | xargs`; do

    # handle ipv4 or ipv6
    numparts=$(echo $i | awk -F: '{print NF}')
    basenum=$((numparts-1))
    port=$(echo $i | cut -d: -f$numparts)
    base=$(echo $i | cut -d: -f1-$basenum)
    formatted="[$base]:$port"

    echo "Connecting $formatted..."
    ~/tezos/tezos-admin-client connect address $formatted
  done
done

Credit goes to creator of this script.

| improve this answer | |
5

T2 instances are general purpose compute instances which have low to moderate network performance and are not optimised for Input/Output operations (IOPS). Tezos is very IO intensive by all accounts.

Try spinning up a Storage Optimised instance (H1/I3/D2) and I suspect it will provide better performance.

See - https://aws.amazon.com/ec2/instance-types/

| improve this answer | |
  • Whilst this is indeed faster, it's still pretty slow, and frequently stops syncing entirely when the 'too few connections' message pops up. A t2 medium is perfectly capable of syncing the entire ethereum blockchain in reasonable time, so I have a hard time believing Tezos is so much more demanding that it should be orders of magnitude slower on just the testnet. There must be another factor at play here. – AndyK Mar 7 '19 at 8:42
  • I agree it shouldn't be the case it is that intensive however it seems to be. The community would benefit from some benchmarks being run to determine actually what sort of IOPS does a full sync need. – xtzbaker Mar 7 '19 at 14:30
  • I was told by a baker that AWS i3.larges are the most effective instances for Tezos nodes, especially for baking due to the great performance of NVMe SSDs. I also found that disk performance usually is the bottleneck on a VPS. – cryptodad Mar 21 '19 at 9:54
0

solution : instance type changing

while syncing the node for Alphanet I have tried multiple instances : from T2.micro to T3.xlarge.

At some point I thought that RAM size or Network performance may play a role here. But Even T3.xlarge instance did not bring the whole node synced fast.

What really helped is changing the instance types periodically. You may have noticed that node is syncing much faster in very beginning, when it just started. Then, after some time, it became very slow again.

I've made an observation that even bigger AWS instance type won't allow you to finish this operation fast in one take .

The plan may be:

  1. Stop the baker, endorser, accuser and then stop the node itself
  2. Stop the t2.medium instance
  3. Change the type of your instance to t2.small
  4. Start the t2.small instance
  5. Start the node and then baker, endorser, accuser. While starting these processes do not forget to redirect the output to log files respectively: tezos.log, baker.log, endorser.log and accuser.log
  6. Start watching how fast new blocks are getting synced on a new instance. Use tail -f tezos.log . You have to remember the block that new instance has started from
  7. Leave it alone for sometime. You may want to come later and see how many blocks have been synced since you started the node. If I recall correctly, it may sync very fast up to 10 000 blocks or so, though it depends on the instance you have chosen. Instead of t2.small, you may have selected t2.large for instance.
  8. When sync process will eventually slow down, repeat operation again. This time migrate from t2.small to t2.medium. It will give you another 10k blocks synced fast.

This approach worked, though it required some manual interventions.

PS: for better results you may use t2.large + t2.medium as a changing pair, not t2.small + t2.medium as described above. But the difference won't be significant.

| improve this answer | |
  • I suspect the reason you see initial good performance on the TX instances that slowly degrades is because these instance types are designed so they can temporarily increase their IO and CPU to deal with small periods of increased activity. This is called bursting and only lasts for a short time before reverting to a more mediocre performance. For sustained IO or CPU load a different instance type is required. This would certainly be the case for a node sync. – xtzbaker Mar 7 '19 at 7:59
  • Receiving the following error using the Alphanet DB above: Mar 7 09:25:47 - node.main: Starting the Tezos node... Mar 7 09:25:47 - node.main: No local peer discovery. Mar 7 09:25:47 - node.main: Peer's global id: idrJtoLevBnyf6ZzUqcmyGBFKUssa7 Mar 7 09:25:47 - node.worker: bootstrapping chain... tezos-node: Error: Missing key in store: chain/8eceda2f/genesis/hash – AndyK Mar 7 '19 at 9:27

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.