Error fetching branch from peer

Question

I'm currently working on a Proof of work consensus algorithm/protocol for Tezos as a project.

So my project already has Transactions between accounts/contracts and already has block mining.

Besides a few details, it's already comparable to a simpler version of Bitcoin (without Bitcoin Script and other stuff).

There's a baker/miner, that pulls the transactions from the mempool of a node, and finds a Nonce that, when hashing the whole header of the block, the hash is smaller than the current target. Also people can send Tez to each other using the client. (There's only Transactions as operations, no Smart Contracts).

Everything was going well until I had 2+ Nodes in my private network.

One was receiving the Block candidates from the miner and the other one was the one I was connecting with a client. In the Mining one (the one that gets injected with blocks) is working fine, but when I mine more than 2 blocks, I get this error on the other one

node.protocol: custom-demo: finalize: Message lvl 2, 0 ops, Fitness/Level 1,,,,, Level 2
block.validation: applied block BKiHMg6hjWZxrgTmqbJFVTnPpxEcaUivYU6qzYgvKWcfQ2pRSgH with an
block.validation:   inconsistent cache: reloading cache
node.protocol: custom-demo: begin_application: level 2, timestamp 2023-01-30T23:13:45Z
node.protocol: custom-demo: prepare: 2 2023-01-30T23:13:30Z
node.protocol: custom-demo: calculate_current_target: Level: 2 | Epoch_size: 10 | Reminder 2
node.protocol: custom-demo: finalize_block
node.protocol: custom-demo: finalize: Message lvl 2, 0 ops, Fitness/Level 1,,,,, Level 2
node.validator.bootstrap_pipeline: fetching branch of about 2 blocks from peer idsX53i8xfhkhpjE9ohRM36j7ZUG1w in
node.validator.bootstrap_pipeline:   1 steps
node.validator.bootstrap_pipeline: unexpected error while fetching headers:
node.validator.bootstrap_pipeline:   Inconsistent hash:
node.validator.bootstrap_pipeline:     got: CoV8XcsoZjsfBhxv5bqjqyT6mfT2s75AHig2t2EbXe53BN3UEYAj
node.validator.bootstrap_pipeline:     expected: CoWCzVmiki948VnPdSLYMXqhjon9Y5STB7rjHaouHi2F8pLqtiay
validator.peer: request
validator.peer:   New branch BKiHMg6hjWZxrgTmqbJFVTnPpxEcaUivYU6qzYgvKWcfQ2pRSgH, locator length 3
validator.peer:   failed
validator.peer:   (Request pushed on 2023-01-30T23:13:51.786-00:00, treated in 12.28us, completed in 640us):
validator.peer:   Error:
validator.peer:     Inconsistent hash:
validator.peer:       got: CoV8XcsoZjsfBhxv5bqjqyT6mfT2s75AHig2t2EbXe53BN3UEYAj
validator.peer:       expected: CoWCzVmiki948VnPdSLYMXqhjon9Y5STB7rjHaouHi2F8pLqtiay
validator.peer:
validator.peer: worker crashed [validator-peer]:
validator.peer:   Error:
validator.peer:     Inconsistent hash:
validator.peer:       got: CoV8XcsoZjsfBhxv5bqjqyT6mfT2s75AHig2t2EbXe53BN3UEYAj
validator.peer:       expected: CoWCzVmiki948VnPdSLYMXqhjon9Y5STB7rjHaouHi2F8pLqtiay
validator.peer:
p2p.maintenance: too few connections (0)
validator.chain: disconnection of idsX53i8xfhkhpjE9ohRM36j7ZUG1w
p2p.maintenance: too few connections (0)

And I got this on my baker's log (Notice that there's no CoV8XcsoZjsfBhxv5bqjqyT6mfT2s75AHig2t2EbXe53BN3UEYAj context).

Starting Mining worker loop
Got target as bytes
Current Header { "level": 2, "proto": 1,
  .....
  "context": "CoWCzVmiki948VnPdSLYMXqhjon9Y5STB7rjHaouHi2F8pLqtiay",
 ....

Also, neither the level 0 and level 1 Contexts were the one the error was refering to (CoV8XcsoZjsfBhxv5bqjqyT6mfT2s75AHig2t2EbXe53BN3UEYAj).

Where in the protocol can I define the rules for synchronization?