Friday, October 7, 2022
HomeGolangBlockchain In Go: Half III: Redundant Storage And Consensus

Blockchain In Go: Half III: Redundant Storage And Consensus


Within the first two posts, I defined there have been 4 features of a blockchain that this collection would discover with a backing implementation supplied by the Ardan blockchain mission.

The primary put up targeted on how the Ardan blockchain offers help for digital accounts, signatures, and verification. The second put up targeted on transaction distribution and synchronization between completely different computer systems. On this third put up, I’ll concentrate on how the Ardan blockchain handles consensus between completely different computer systems which leads to the redundant storage of the blockchain database.

Supply Code

The supply code for the Ardan blockchain mission might be discovered on the hyperlink under.

It’s essential to grasp this code remains to be a work-in-progress and that issues are being refactored as extra is discovered.


Consensus in pc science is the method of getting completely different computer systems in a distributed and decentralized atmosphere to all agree about one thing. With a blockchain, that one thing is a proposed block of transactions to be included within the blockchain database for each node working on a pc.

There are two consensus protocols extensively used immediately by blockchains. One is proof of labor (POW) and the opposite is proof of stake (POS). Bitcoin and Ethereum at the moment use POW for his or her consensus protocol. Nonetheless, Ethereum shall be switching to POS in some unspecified time in the future in 2022. Take into consideration POW as consensus by competitors the place POS is consensus by committee.

Word: There are different consensus protocols which can be additionally utilized by blockchains equivalent to proof of house and proof of elapsed time to call a number of.

When implementing a consensus protocol you additionally want to decide on between security or liveness. If a blockchain had been to decide on security, then nodes would wish to attend for all different nodes within the community to vote and agree a couple of proposed block earlier than one other block might be proposed. If liveness is chosen, then nodes don’t want to attend to maintain proposing blocks. One good thing about liveness is that it permits a blockchain to take care of excessive availability, since nodes can hold proposing blocks with out ready.

Bitcoin and Ethereum each use liveness. When a node is profitable in mining a block, the node will write the block instantly into their blockchain database after which suggest the block for inclusion by different nodes. Because of this at occasions (as a rule) there are forks, completely different variations of the blockchain database on completely different nodes. When a fork happens, the consensus protocol will ultimately decide which fork is appropriate and people nodes with the fallacious fork will reset. Ultimately, there actually isn’t any true consensus till roughly 10 blocks have been mined and people blocks might be present in the entire nodes actively collaborating within the community.

It’s essential that Blockchain methods select liveness over security since particular person mining nodes are usually not dependable. At any time a rustic like China might shut down 1000’s of nodes, which might deliver a complete community to a halt when you wanted to attend for nodes that now not exist.

The Ardan blockchain makes use of a POW and liveness consensus protocol identical to Bitcoin and Ethereum. The implementation for the Ardan blockchain is pushed by two completely different mining workflows relying if the node is profitable in mining the block or not.


The opposite huge a part of a blockchain consensus protocol is how a block is validated. Over time these guidelines can change as a blockchain goes by upgrades. It’s at all times fascinating when nodes are within the means of upgrading since blocks might be invalidated by nodes working older or newer variations of the software program. There are at all times forks that happen and ultimately get resolved as soon as all of the nodes on the community are working the identical model.

Listed here are the present validation checks (with a code instance) which can be being carried out by the Ardan blockchain for all newly proposed blocks.

Itemizing 1: Block Problem

01     if proposedBlock.Header.Problem < latestBlock.Header.Problem {
02         return fmt.Errorf("message")
02     }

Itemizing 1 reveals the block issue test. If the proposed block’s issue will not be the identical or better than the mother or father block’s issue, it could possibly’t be accepted. Failing this test might determine a node attempting to suggest a block with out performing the right quantity of labor.

Itemizing 2: Block Hash

01     hash := proposedBlock.Hash()
02     if !isHashSolved(proposedBlock.Header.Problem, hash) {
03         return fmt.Errorf("message")
04     }

01 func isHashSolved(issue int, hash string) bool {
02     const match = "00000000000000000"
03     if len(hash) != 64 {
04         return false
05     }
06     return hash[:difficulty] == match[:difficulty]
07 }

Itemizing 2 reveals the block hash test. A proposed block has to show the right quantity of labor has been carried out by discovering a nonce that produces a correct hash based mostly on the rule set by the protocol. Within the Ardan blockchain, that could be a hash with a variety of main zeros that match the issue quantity. Right here the proposed block is hashed as soon as after which checked for the right variety of zeros. Failing this test might determine a node working beneath a special model of code that has modified the hash requirement or simply deciding to try to sneak a block with much less issue into the chain.

Itemizing 3: Block Quantity

01     if proposedBlock.Header.Quantity != latestBlock.Header.Quantity + 1 {
02         return fmt.Errorf("message")
03     }

Itemizing 3 reveals the block quantity test. A proposed block should be the subsequent block within the sequence of blocks based mostly on the node’s newest block. If the most recent block is block quantity 4, the proposed block should be set as block quantity 5. Failing this test might determine a node that’s working a forked chain or will not be correctly in sync.

Itemizing 4: Block Validation – Chain Forked

01     nextNumber := latestBlock.Header.Quantity + 1
02     if proposedBlock.Header.Quantity >= (nextNumber + 2) {
03         return ErrChainForked
04     }

Itemizing 4 reveals the chain forked test. This test identifies if the native node’s blockchain database will not be legitimate. The thought is that if the subsequent proposed block is 2 or extra block numbers forward of this node’s newest block, this node’s blockchain database has forked from the present chain that has been extensively accepted by different nodes. At this level, the node must take away their present newest block and start to resync the native blockchain database from the earlier block on.

Itemizing 5: Block Dad or mum Hash

01     if proposedBlock.Header.ParentHash != latestBlock.Hash() {
02         return fmt.Errorf("message")
03     }

Itemizing 5 reveals the mother or father hash test. A proposed block has a mother or father hash worth and that hash ought to match the hash of the present newest block. Failing this test might determine a node that’s working a forked chain because it has a special mother or father block.

Itemizing 6: Block Timestamp

01     parentTime := time.Unix(int64(latestBlock.Header.TimeStamp), 0)
02     blockTime := time.Unix(int64(proposedBlock.Header.TimeStamp), 0)
03     if !blockTime.After(parentTime) {
04         return fmt.Errorf("message")
05     }

Itemizing 6 reveals the block timestamp test. A proposed block will need to have a timestamp that’s better than the most recent block. Failing this test might determine a node that has an issue with its clock or is attempting to play video games on the community.

Itemizing 7: Block Time Distinction

01     parentTime := time.Unix(int64(latestBlock.Header.TimeStamp), 0)
02     blockTime := time.Unix(int64(proposedBlock.Header.TimeStamp), 0)
03     dur := blockTime.Sub(parentTime)
04     if dur.Seconds() > time.Period(15*time.Second).Seconds() {
05         return fmt.Errorf("message")
06     }

Itemizing 7 reveals the block time distinction test. A proposed block can’t be older than quarter-hour from its mother or father block. Failing this test might determine a node that has an issue with its clock or is attempting to play video games on the community. It is a test that’s commented out for the Ardan blockchain since growth and working the community is sporadic.

Mining Code

Now that you’ve seen the code used to carry out block validation, subsequent is to indicate you the code that handles the mining workflows. The mining workflows implement the POW and liveness elements of the consensus protocol.

Determine 1: Mined Block

Determine 1 reveals a workflow diagram of what occurs when a node is profitable in mining a block earlier than every other node.

You possibly can see how three goroutines (G) are concerned within the means of mining. The controller goroutine manages the workflow for the whole mining course of. The mining goroutine performs the precise work of mining and the cancel goroutine is answerable for canceling mining when signaled to take action. Nonetheless, on this situation the mining goroutine indicators the cancel goroutine to point that mining is full.

Now check out the workflow for when a node will not be profitable in mining a brand new block earlier than every other node.

Determine 2: Obtained Proposed Block

Determine 2 reveals a workflow diagram of what occurs when a node receives a proposed block on a fourth goroutine dealing with the request whereas within the means of mining a brand new block. This requires that the mining operation is canceled and the proposed block is verified and written to disk. The cancellation of the mining operation and the synchronization between all of the goroutines concerned is what’s tough on this workflow.

To start out, I’ll stroll by the code that units up the controller, cancel, and mining goroutines.

Employee and State Packages

Within the blockchain layer of the Ardan blockchain mission, there’s a package deal named employee with a supply code file named employee.go. That is the place the code for the completely different workflows described above have been applied.

Itemizing 8: Getting Issues Began

01 package deal employee
03 sort Employee struct {
04     state        *state.State
05     wg           sync.WaitGroup
06     ticker       time.Ticker
07     shut         chan struct{}
08     startMining  chan bool
09     cancelMining chan chan struct{}
10     txSharing    chan storage.BlockTx
11     evHandler    state.EventHandler
12     baseURL      string
13 }
15 func Run(state *State, evHandler EventHandler) {
16     w := &employee{
17         state:        state,
18         ticker:       *time.NewTicker(peerUpdateInterval),
19         shut:         make(chan struct{}),
20         startMining:  make(chan bool, 1),
21         cancelMining: make(chan chan struct{}, 1),
22         txSharing:    make(chan storage.BlockTx, maxTxShareRequests),
23         evHandler:    evHandler,
24         baseURL:      "http://%s/v1/node",
25     }
. . .

Itemizing 8 reveals the Employee sort and Run perform. The Run perform begins the goroutines that implement the completely different workflows to maintain the node in sync and wholesome. Since this put up is specializing in mining, take a look at the startMining and cancelMining channels on traces 08-09. These channels are linked to the mining workflow. Discover how they’re constructed as buffered channels of 1 on traces 20-21.

Word: Whenever you see a channel constructed with a bool, this is a sign that there shall be signaling with information, however the information is bigoted. Whenever you see a channel constructed with the empty struct (struct{}), this is a sign that solely a detailed operation shall be performed.

The State worth that’s handed into the Run perform wants entry to the Employee worth so State APIs can sync, shutdown, and sign the beginning and cancellation of workflows. The Employee worth wants entry to the State worth since that gives the core enterprise logic. So there’s a downside. Go doesn’t enable the criss-crossing of imports between packages. To unravel this downside, the employee package deal has an import to the state package deal, and the state package deal declares an interface known as Employee.

Itemizing 9: Employee Interface

01 package deal state
03 sort Employee interface {
04     Shutdown()
05     Sync()
06     SignalStartMining()
07     SignalCancelMining() (carried out func())
08     SignalShareTx(blockTx storage.BlockTx)
09 }
11 sort State struct {
. . .
27     Employee Employee
28 }

Itemizing 9 reveals the declaration of the Employee interface from the state package deal. This interface permits for the decoupling of a employee implementation. These 5 strategies are required to permit the state package deal to execute or sign the completely different occasions managed by the employee package deal. On line 27, you see how the Employee area is said utilizing the Employee interface.

Itemizing 10: Employee to State to Employee Assignments

03 import (
04     ""
05 )
. . .
15 func Run(state *State, evHandler EventHandler) {
16     w := &employee{
17         state:        state,
18         ticker:       *time.NewTicker(peerUpdateInterval),
19         shut:         make(chan struct{}),
20         startMining:  make(chan bool, 1),
21         cancelMining: make(chan chan struct{}, 1),
22         txSharing:    make(chan storage.BlockTx, maxTxShareRequests),
23         evHandler:    evHandler,
24         baseURL:      "http://%s/v1/node",
25     }
27     state.Employee = &w
. . .

Itemizing 10 reveals how the employee package deal imports the state package deal on line 04, which implies the state package deal can now not import employee. Because of the Employee interface declared by the state package deal, the project of the employee worth to state.Employee on line 27 can happen.

The opposite good factor about this interface is it lets you change the mission’s employee package deal with your individual implementation for studying functions.

Itemizing 11: Node Sync

15 func Run(state *State, evHandler EventHandler) {
. . .
29     w.sync()
. . .

In itemizing 11, the decision to sync on line 29 of the Run perform makes positive the blockchain database and state for the node is in sync with the remainder of the community earlier than collaborating in any blockchain actions.

Itemizing 12: Beginning Goroutines

15 func Run(state *State, evHandler EventHandler) {
. . .
31     operations := []func(){
32         w.peerOperations,
33         w.miningOperations,
34         w.shareTxOperations,
35     }
37     g := len(operations)
38     w.wg.Add(g)
40     hasStarted := make(chan bool)
42     for _, op := vary operations {
43         go func(op func()) {
44             defer w.wg.Executed()
45             hasStarted <- true
46             op()
47         }(op)
48     }
50     for i := 0; i < g; i++ {
51         <-hasStarted
52     }
53 }

Itemizing 12 reveals the remainder of the Run perform. As soon as the blockchain node is in sync with the community, a set of operation features are executed as completely different goroutines. These operation features handle the completely different workflows which can be wanted to maintain the blockchain node wholesome and in sync with the remainder of the community.

There are three operation features that exist immediately:
peerOperations: Manages the discovering of latest nodes on the community.
miningOperations: Manages the mining of latest blocks.
shareTxOperations: Manages the sharing of latest transactions throughout the community.

In itemizing 12 on traces 31-35, an array of operation features is constructed and initialized with the three features outlined above. Then the employee’s WaitGroup area is up to date on line 38 to mirror the three goroutines which can be going to be created for every operation. On line 40, a channel is constructed so the Run perform can’t return till it’s recognized that every one three of the goroutines which can be about to be created are up and working.

Line 42-48 creates the three goroutines utilizing a literal perform. The literal perform units up the decision to Executed on return, indicators that the goroutine is working, after which calls the operation perform which blocks till shutdown.

Lastly on traces 50-52, the Run perform waits to obtain the indicators that every one the goroutines are up and working. At this level, the node is prepared and may take part in blockchain actions.

Itemizing 13: Operational Operate: Controller G: Mining Operation

01 func (w *employee) miningOperations() {
02     for {
03         choose {
04         case <-w.startMining:
05             if !w.isShutdown() {
06                 w.runMiningOperation()
07             }
08         case <-w.shut:
09             w.evHandler("employee: miningOperations: acquired shut sign")
10             return
11         }
12     }
13 }

Itemizing 13 reveals the highest degree code for the operation perform that runs mining. The code is an limitless for loop that’s blocked on two indicators. The primary sign is used to begin a mining operation which it does by calling the runMiningOperation technique. The opposite sign is used to power the perform to return which can shutdown the goroutine.

Itemizing 14: Sign Mining

01 func (w *employee) SignalStartMining() {
02     choose {
03     case w.startMining <- true:
04     default:
05     }
06 }

Itemizing 14 reveals the SignalStartMining technique which implements one of many behaviors outlined by the state.Employee interface. This perform is named anytime a state package deal API needs to begin a mining operation. If this buffered channel of 1 already has a pending sign, the default case will drop any new sign so the caller doesn’t block. Just one pending sign is required to begin a brand new mining operation.

Itemizing 15: runMiningOperation

01 func (w *employee) runMiningOperation() {
02     size := w.state.QueryMempoolLength()
03     if size == 0 {
04         return
05     }
07     defer func() {
08         size := w.state.QueryMempoolLength()
09         if size > 0 {
10             w.signalStartMining()
11         }
12     }()
. . .

Itemizing 15 reveals the start code for the runMiningOperation technique that is named in itemizing 13 when a mining sign is acquired. The logic on this technique is on the core of what the controller goroutine does within the workflow diagrams proven initially of this put up.

On line 02, a test is made to confirm there are transactions within the mempool. Then on traces 07-12, a defer perform is said to be known as as soon as the mining operation is full. This permits the strategy to sign a brand new mining operation to begin once more as soon as this perform returns if there are new transactions within the mempool.

Itemizing 16: runMiningOperation continued

01 func (w *employee) runMiningOperation() {
. . .
14     var wait chan struct{}
15     defer func() {
16         if wait != nil {
17             <-wait
18         }
19     }()
. . .

Itemizing 16 reveals the subsequent part of code in runMiningOperation which is a essential piece of the logic. This code blocks the strategy from fully returning till the wait channel is closed. Discover how the channel is being declared with the empty struct on line 14. If this channel stays set to its zero worth building throughout mining, the defer perform doesn’t want to attend earlier than returning.

This logic is required for when the node will not be profitable in mining the block first and it has to attend on a request goroutine to course of a brand new proposed block. I’ll clarify extra about that situation later.

Itemizing 17: runMiningOperation continued

01 func (w *employee) runMiningOperation() {
. . .
21     choose {
22     case <-w.cancelMining:
23     default:
24     }
26     ctx, cancel := context.WithCancel(context.Background())
27     defer cancel()
29     var wg sync.WaitGroup
30     wg.Add(2)
32     go func() {
           // Cancel Goroutine
42     }()
44     go func() {
           // Mining Goroutine
66     }()
68     wg.Wait()
69 }

Itemizing 17 reveals the remainder of the management logic for beginning and managing the mining operation. Strains 21-24 are used to empty the cancel mining channel so it’s empty earlier than the mining begins. You don’t need the mining to finish earlier than it really begins. I don’t anticipate there to be a pending sign, however simply in case I believe it’s finest to clear the channel.

Strains 26-27 constructs a context for the aim of canceling the mining operation if a brand new proposed block is acquired by one other node. The cancel perform will present this facility.

Strains 29-30 constructs a WaitGroup to trace the cancel and mining goroutines which can be created subsequent. Following the creation of these two goroutines is the decision to Wait on line 68. This name forces the runMiningOperation technique to attend for the cancel and mining goroutines to finish earlier than returning. As soon as the runMiningOperation technique returns, the 2 defer features proven in listings 8 and 9 will execute.
Itemizing 18: Cancel Goroutine

01 func (w *employee) runMiningOperation() {
. . .
32     go func() {
33         defer func() {
34             cancel()
35             wg.Executed()
36         }()
38         choose {
39         case wait = <-w.cancelMining:
40         case <-ctx.Executed():
41         }
42     }()
. . .

Itemizing 18 reveals the goroutine for canceling the mining operation. On traces 38-41, the goroutine blocks ready for one among two completely different indicators to be acquired. If the cancel mining sign is acquired on line 39, then a brand new proposed block has been acquired and the mining operation must be canceled. Discover how the wait channel declared on line 14 in itemizing 16 is assigned when receiving this sign.

If the context canceled sign is acquired on line 40, then this node was profitable in mining the brand new block. The mining goroutine will name that context worth’s cancel perform when it efficiently mines a block to sign this occasion.

As soon as any sign is acquired, this goroutine can terminate and the literal defer perform on traces 33-36 shall be executed. The cancel perform is named first, however on this situation the context was already canceled and for the reason that cancel perform might be known as a number of occasions, no additional logic is required. Then the WaitGroup worth is decremented by 1 with the decision to Executed.

Itemizing 19: Mining Goroutine

01 func (w *employee) runMiningOperation() {
. . .
44     go func() {
45         defer func() {
46             cancel()
47             wg.Executed()
48         }()
50         block, period, err := w.state.MineNewBlock(ctx)
51         if err != nil {
52             swap {
53             case errors.Is(err, ErrNotEnoughTransactions):
54                 // Log: Not sufficient transactions in mempool
55             case ctx.Err() != nil:
56                 // Log: Mining Canceled
57             default:
58                // Log: One thing failed
59             }
60             return
61         }
63         if err := w.proposeBlockToPeers(block); err != nil {
64             // logging
65         }
66     }()
. . .

Itemizing 19 reveals the goroutine for performing the mining operation. On line 50, the goroutine calls the MineNewBlock technique of the state worth to carry out the mining. Discover how the context created on line 26 in itemizing 17 is handed into that technique. That offers the cancel goroutine the power to ship a sign to this goroutine to cease mining by calling that context worth’s cancel perform.

Within the case the place the MineNewBlock technique returns an error, then a brand new proposed block was acquired and the runMiningOperation technique can return. If the MineNewBlock technique is profitable, then a brand new block is mined by this node and results in executing the code on line 63, which proposes this new block to the opposite recognized blockchain nodes.

In both case, as soon as the mining goroutine is full, its defer perform on traces 45-48 will execute. This can have the cancel goroutine shutdown if it’s nonetheless working and decrement the WaitGroup by 1.

New Proposed Block Obtained

Within the workflow diagram for when a brand new proposed block is acquired from a special blockchain node, there’s a goroutine known as the New Block G. This goroutine is a request handler applied to obtain this new proposed block and carry out the work of canceling mining, validating, and writing the brand new block to disk.

Itemizing 20: New Proposed Block Request Goroutine

01 func (h Handlers) ProposeBlock(
       ctx context.Context, w http.ResponseWriter, r *http.Request) error {
03     var blockFS storage.BlockFS
04     if err := internet.Decode(r, &blockFS); err != nil {
05         return fmt.Errorf("unable to decode payload: %w", err)
06     }
08     block, err := storage.ToBlock(blockFS)
09     if err != nil {
10         return fmt.Errorf("unable to decode block: %w", err)
11     }
13     if err := h.State.ValidateProposedBlock(block); err != nil {
14         if errors.Is(err, state.ErrChainForked) {
15             h.State.Truncate()
16             return internet.NewShutdownError(err.Error())
17         }
19         return v1.NewRequestError(err, http.StatusNotAcceptable)
20     }
22     resp := struct {
23         Standing string `json:"standing"`
24     }{
25         Standing: "accepted",
26     }
28     return internet.Reply(ctx, w, resp, http.StatusOK)
29 }

Itemizing 20 reveals the handler perform for the brand new proposed block goroutine outlined in determine 2. On traces 03-11, the brand new proposed block acquired by a special node is decoded after which on line 13, the ValidateProposedBlock technique off the state worth is named to carry out the work of dealing with this new proposed block.

Itemizing 21: Validate Proposed Block

01 func (s *State) ValidateProposedBlock(block storage.Block) error {
02     carried out := s.Employee.SignalCancelMining()
03     defer carried out()
. . .

Itemizing 21 reveals the primary piece of logic for the ValidateProposedBlock technique. The very very first thing the strategy does is cancel the mining operation on line 02 by calling the employee’s SignalCancelMining technique. This technique returns a perform that must be known as after the ValidateProposedBlock technique is completed with all its work. That is setup with a defer on line 03. If the carried out perform will not be known as, it would block the goroutine working the runMiningOperation technique eternally.

Itemizing 22: Sign Cancel Mining

01 func (w *employee) SignalCancelMining() (carried out func()) {
02     wait := make(chan struct{})
04     choose {
05     case w.cancelMining <- wait:
06     default:
07     }
09     return func() { shut(wait) }
10 }

Itemizing 22 reveals the implementation of the SignalCancelMining technique from the employee package deal that was known as in itemizing 21. Right here is the place the wait channel is constructed, which was talked about in listings 9 and 11.
On traces 04-07, the wait channel is signaled to the cancel goroutine to provoke the canceling of the mining operation in progress. Then on line 09, a literal perform is said and returned that closes this channel, which when known as will sign again to the goroutine executing the runMiningOperation technique that it could possibly full its return, as seen in itemizing 16.

Itemizing 23: Validate Proposed Block

01 func (s *State) ValidateProposedBlock(block storage.Block) error {
02     carried out := s.Employee.SignalCancelMining()
03     defer carried out()
05     return s.validateUpdateDatabase(block)
06 }

Itemizing 23 continues with the logic for the ValidateProposedBlock technique. After the sign to cancel mining is full, the subsequent step is to validate the brand new proposed block and if the block handed validation, write the block to disk and replace the state of the node. That is carried out with the decision to validateUpdateDatabase on line 05.

The logic for validating the block was introduced at first of this put up.


This third put up targeted on how the Ardan blockchain handles redundant storage of a single ledger and consensus between completely different computer systems. This represents the third of the 4 features of a blockchain that this collection will discover with a backing implementation supplied by the Ardan blockchain.

Within the subsequent put up, I’ll share how the Ardan blockchain offers detection of any forgery to previous transactions or the blockchain database.



Please enter your comment!
Please enter your name here

Most Popular

Recent Comments