Consensus mechanisms explained

Consensus mechanisms are vital for the existence of the cryptocurrency markets and other blockchain applications. A (functional) decentralized blockchain could not exist without a consensus mechanism. To illustrate why it’s best to look at an example of a centralized setting first. [2]

Centralized setting

The central government of a country keeps track of who owns which house. If a house is sold, the central database is updated by the designated government body. Any changes (properties being bought and sold) are communicated with this central body. Your notary sends the required forms and deeds to the government and the government processes the forms and deeds and updates the database to reflect the new ownership. Consensus about who owns which house is reached because there is only one database and we trust the authority of the central government to keep fair and meticulous track of the property database.

Decentralized setting

Now consider the same situation, but without a central government. How do we know who owns the house on the corner? In other words, if there is no central entity that we can trust to update the database every time a property is sold, how can we reach a consensus on home ownership? Can we trust that everybody is honest about the property they own without a system to verify it? Simply put, no.

A decentralized blockchain does not have a central entity nor does it have a single database. A blockchain consists of multiple decentralized computer systems (nodes) that all have their own version of the database (ledger). However, these nodes don’t know each other or can trust each other, but they still have to reach a consensus about how to update the ledger. [3] A consensus mechanism is necessary to guarantee that every time data need to be added to the blockchain (for example, new transactions) all nodes know the new data is valid and can update their copy of the ledger accordingly.

Furthermore, the nodes have to update the data in such a way that the copies remain practically the same. [4] This becomes possible with the use of a well-designed consensus mechanism. It essentially allows for a system to function without a middleman. Or, as blockchain evangelists like to put it, it removes the central authority and gives power to the members of the network. [5]

Consensus mechanisms in action

So, how does this all work? We will explore how a consensus mechanism works in practice by looking at the best-known blockchain in the crypto industry: Bitcoin.

Consensus on the Bitcoin blockchain in a nutshell

The Bitcoin blockchain uses a consensus mechanism based on the principle of Proof-of-Work (PoW). The Bitcoin blockchain requires that “work” is performed by so-called mining nodes before these nodes can add valid blocks containing new transactions to the blockchain. The work consists of solving a computational puzzle which requires spendings vast amounts of computing power. In other words, spending vast amounts of money on hardware and electricity. The mining nodes each compete against each other to mine a new block (solve the puzzle) because only the first one to finish the work is allowed to propagate a new block to the network in exchange for a reward: newly minted bitcoin. When a block is mined, a new race to mine the next block begins and so on. [6]

However, before the winning mining node gets the bitcoin, its newly mined block is validated by the other nodes on the network in a fully automated process. The reward is only granted if the other nodes individually agree that the block is valid. if the block is valid, the node will add the block to its copy of the ledger, extending the blockchain. But, if the block is invalid, for example, the miner tried to add bogus transactions to the block, the block is rejected. The block will continue to travel through the network until it has reached each node or is rejected. A rejected block would result in having done all work (ie. spend money on hardware and electricity) for nothing. [7]

In this context, a valid block means a block that passed a 25-point checklist, including the validity of transactions. If you look closely at the way the Bitcoin blockchain implemented the PoW mechanism you notice that nodes don’t have to talk to each other. They only have to look at the block that is propagating through the network. If it’s valid, the block is added to the node’s copy of the ledger. If it’s invalid, the block is rejected. Because each node follows the same validation process and there is only one block each time, they all end up with the same copy of the ledger (or so it appears …)

Proof-of-Work is often referred to as a consensus mechanism. This is not correct. PoW is a principle that forms part of a consensus mechanism, but not the mechanism itself. A consensus mechanism is all rules, principles and processes that, together, guarantee consensus. [8]

Consensus on the Bitcoin blockchain continued

The explanation of PoW above explains consensus on the Bitcoin blockchain in a nutshell. But there is a lot more to it when you want to design a fully decentralized blockchain.

Validating transactions transactions

On the Bitcoin blockchain transactions are validated a total of three times. When you send bitcoin from one address to another, your transaction is initially validated by every full bitcoin node the transaction reaches (check 1). If the full node sees that you indeed have the required amount of bitcoin in your wallet and that the transaction is properly signed, it adds the transaction to the memory pool (mempool).

Mining nodes take the transactions from the mempool, validate them (check 2) and bundle them into blocks. As explained above, the miner that wins the race gets to propagate its block to the network. When this new block propagates through the network it is validated by each node that it reaches (check 3) to see if there are no bogus transactions included in the block (and 24 other checks). This is how the Bitcoin blockchain makes sure that there can be no discussion (consensus) about the validity of transactions. [9]

Standard situation: single block

This brings us to the situation that we have an undeniably valid and trustworthy block with trustworthy transactions. So, how do we add this block to the blockchain? In other words, how does each node know how to add the newly minded block to its copy and how do we make sure that all individual nodes follow the same logic? Well, a miner adds a reference to the new block that indicates who the parent of that block is. Block 100 (child) will have a reference to Block 99 (parent). This is how nodes know how to add the block to their copy of the blockchain. Each node does this on an individual basis, but because they all follow the same logic, the blockchain copies will remain the same and consensus is reached. [10]

Anomalies: two blocks at the same time.

To further complicate things, it can happen that two blocks are mined at the same time because not all nodes were aware a new race had started already. This occurs when two mining nodes both mine a new valid block mere seconds from each other. We’re now faced with a situation where two new and valid blocks are propagated through the network. Meaning that a node will receive two blocks with the same parent, creating a problem since they can only add one. Each node will add the block that it receives first and disregard the other. However, some nodes will receive Block 100a first and some nodes will receive Block 100b first with the result that different nodes now have different copies of the blockchain. So, how does the Bitcoin blockchain reach consensus?

A mining node also has its own copy of the blockchain. So, in the event of two blocks, it happens that one miner has a copy of the blockchain ending with Block 100a, the other in Block 100b. This is where it gets tricky. Now a new mining race starts. Each miner uses its own copy of the blockchain to add the parent reference to the new block that it’s mining. Hence creating a problem because some mining nodes will have Block 100a at the end of their chain and some 100b. In other words, some miners will add parent 100a and others will add parent 100b to their newly mined block. [11] This results in a conundrum for nodes because now the logic explained above, just adding it on top of the parent, is not so simple anymore. Let’s illustrate this with an example.

Let’s say Mining Node 12 has Block 100a as the last block of its copy of the blockchain. Let’s also say that Mining Node 12 finishes the work first and gets to propagate a new block (Block 101) to the network. Since Mining Node 12 had Block 100a as the last block, the newly-minded block will reference Block 100a as the parent block. There is no problem when Block 101 arrives at Full Node 123 which also has Block 100a as the last block of its copy. Full Node 123 will see that Block 101 has Block 100a as the parent and simply add Block 101 on top of its chain.

But what happens if Block 101 reaches Full Node 789 which received Block 100b first and thinks this block is the last one? Full Node 789 is now faced with a situation where it receives a valid block, but one that doesn’t fit on top of its chain. It’s now confronted with a so-called blockchain fork (a split in the blockchain) and has to choose between two different chains: [0-…-99-100a-101] or [0-…99-100b]. [12]

This is where PoW on the Bitcoin blockchain gets really clever. What happens is that the nodes faced with the dilemma will check both options they have to extend the blockchain to see which one has the most cumulative “work”. The chain referenced by the newly-mined Block 101 [0-…-99-100a-101] has a whole extra block attached as compared to copy the node has itself [0-…99-100b]. The version of the chain of the newly-mined block [0-…-99-100a-101] thus contains the most cumulative work. [13] The nodes that received Block 100b first will now accept chain [0-…-99-100a-101] as the correct chain and the transactions in Block 100b are added back to the mempool so that they can be mined again.

It can happen that transactions (the one in Block 100b of the example) are initially added to the blockchain, but later discarded and added back to the mempool. That’s why it’s always best to wait for several confirmations (blocks being added on top of the block with your transactions) before counting yourself rich. The time it takes for a transaction to become irreversible is known as finality. [14]

The above is an example of the Bitcoin blockchain’s consensus mechanism based on the PoW principle. However, there are many other consensus principles, each having a different approach to achieving consensus. Examples are Proof-of-Authority, Proof-of-Capacity and Proof-of-Activity and Proof-of-Burn.

Consensus attacks

Are there still ways that bad actors can try to cheat the system? In theory, yes. These so-called consensus attacks occur when participants of the network try to cheat the system to, for example, wrongfully enrich themselves (double-spending) or try to stop a blockchain from working (denial of service attack). One of the most notorious consensus attacks is a 51%-attack where a powerful network participant uses his/her might to overpower the network for illicit benefits. [15]

A common misconception is that a consensus attack means that miners and validators can create transactions out of thin air. A block containing bogus transactions will always be spotted by other nodes and simply rejected. However, attackers can try to fork the blockchain in an attempt to double-spend their cryptocurrencies.

SOURCES

  1. TechTarget.com: Consensus Algorithm | Author: Rahul Awati
  2. Crypto.com University:  Consensus Mechanisms in Blockchain: A Beginner’s Guide
  3. Coindesk: What is a Consensus Mechanism | Author: Benedict George
  4. Mastering Bitcoin: Chapter 8. The Blockchain | Author: Andreas M. Antonopoulos
  5. Medium MtheoryV: Blockchain Consensus Protocols & Game Theory | Author: MtheoryV
  6. Mastering Bitcoin: Chapter 8. The Blockchain | Author: Andreas M. Antonopoulos
  7. Coinbase Learn: What is “proof of work” or “proof of stake”?
  8. Ethereum.org: Consensus Mechanisms
  9. Mastering Bitcoin: Chapter 8. The Blockchain | Author: Andreas M. Antonopoulos
  10. Idem.
  11. Idem.
  12. Idem.
  13. Idem.
  14. Ethereum.org: Proof-of-Work (PoW)
  15. Coindesk: What is a 51% attack? | Author: Griffin Mcshane