The era of Web3 is here! Big changes in application architecture


The era of Web3 is here! Big changes in application architecture

Introduction | What makes Web 3.0 so different from Web 2.0? This article uses the Ethereum ecosystem as an example, starting from Web 2.0, and introducing the application architecture of Web 3.0 step by step.

The era of Web3 is here! Big changes in application architectureWeb 3.0 vs Web 2.0

(1) The architecture of Web 3.0 applications (ie DApps) is completely different from that of Web 2.0

Take, for example, a simple blogging site, Medium, where users can publish their own content and interact with other users’ content.

As a Web 2.0 application, it may sound simple, but there are still so many features that make up Medium’s architecture and make everything possible:

  • first, there must be a place to store important data, such as user information, posts, tags, comments, likes, etc., which requires a constantly updated database.
  • secondly, the backend code (written in languages ​​such as Node.js, Java, or Python) must define Medium’s business logic. Like what happens when a new user signs up, publishes a new blog post, or comments on someone else’s blog.
  • third, the front-end code (usually written in JavaScript, HTML, and CSS) must define Medium’s UI logic. For example, what the web page looks like and what happens when the user interacts with each element on the page.

Putting this all together, when you write a blog post on Medium, you interact with its front end, the front end communicates with the back end, and the back end talks to the database. All of these codes are hosted on a centralized server and sent to users through a web browser . This is a good top-level abstraction of how most Web 2.0 applications work today.

The era of Web3 is here! Big changes in application architecture

But all of this is changing.

Blockchain technology unlocks an exciting new direction for Web 3.0 applications. In this article, we will focus on what the Ethereum blockchain brings us.

(2) What makes Web 3.0 so different?

Unlike Web 2.0 applications like Medium, Web 3.0 eliminates the middleman, there is no centralized database to store application state, and there is no centralized Web server to host back-end logic.

Instead, you can leverage the blockchain to build applications on a decentralized state machine maintained by anonymous nodes on the internet.

The “state machine” here refers to a machine that maintains a certain state, including a given program state and future allowed states on the machine. A blockchain is a state machine that is instantiated by a “creation” state and has very strict state transition rules (that is, a consensus mechanism).

Even better, there is no single entity that controls this decentralized state machine, as it is jointly maintained by everyone in the network.

What about the backend server? Unlike Medium’s backend control approach, in Web 3.0 you can write smart contracts that define your application logic and deploy them in a decentralized state machine. This means that everyone who wants to build a blockchain application deploys their code on this shared state machine.

And what about the front end? remains almost the same, except for a few special cases that we’ll cover later.

Now the structure looks like this:

The era of Web3 is here! Big changes in application architecture

The era of Web3 is here! Big changes in application architecturecheck it out

Now, let’s take a deeper dive into what makes this possible.

(1) Blockchain

The Ethereum blockchain has been dubbed the “world computer”. This is because it is a globally accessible, deterministic state machine maintained by a peer-to-peer network of nodes. The state change of the state machine is constrained by the consensus rules followed by the nodes in the network. So, in other words, it is literally a state machine designed to be accessible and writable by anyone in the world, literally. This makes the machine not exclusively owned by any single entity, but shared by everyone in the network.

One more thing to know: data can only be written to the Ethereum blockchain – you can never update existing data.

(2) Smart contract

A smart contract is a program running on the Ethereum blockchain that defines the logic behind state changes that occur on the blockchain. Smart contracts are written in a high-level language such as Solidity or Vyper.

The era of Web3 is here! Big changes in application architecture

Because the code of smart contracts is stored on the Ethereum blockchain, everyone can check the application logic of all smart contracts on the network.

(3) Ethereum Virtual Machine (Ethereum Virtual Machine, EVM)

Further down, there is the Ethereum Virtual Machine, which is used to execute the logic defined in the smart contract and handle state changes that occur on this globally accessible state machine.

EVM does not understand high-level languages ​​like Solidity and Vyper that are used to write smart contracts. Instead, you must compile the high-level language to bytecode that the EVM can execute.

(4) Front end

Finally, there is the front end. As mentioned earlier, the front end defines the UI logic, but it also communicates with the application logic defined in the smart contract. The communication between the front end and the smart contract is a bit more complicated than shown in the above diagram, let’s take a closer look at this part next.

The era of Web3 is here! Big changes in application architectureCommunication between the front end and the smart contract

We want the front end to be able to communicate with the smart contract so that functions can be called, but recall that Ethereum is a decentralized network. Every node in the Ethereum network keeps a copy of all states on the Ethereum state machine, including the code and data associated with each smart contract. When we want to interact with data and code on the blockchain, we need to interact with one of the nodes. This is because any node can broadcast a transaction request to be executed in the EVM, and miners will then execute the transaction and propagate the state change to other nodes in the network.

There are two methods to broadcast a new transaction:

  • Build your own node running the Ethereum blockchain software
  • Use nodes provided by third-party services such as Infura, Alchemy, and Quicknode

If you use a third-party service, you don’t have to worry about running a full node yourself. After all, it can take days to build a new Ethereum node on your own server. (There’s a lot of data to sync – this can even take up more bandwidth and storage than a typical laptop can normally handle)

Also, as your Dapp grows in size, the cost of storing the full Ethereum blockchain increases, and you need to add more nodes to scale your infrastructure. That’s why, as your infrastructure becomes more complex, you need full-time DevOps who will help you maintain the infrastructure to ensure reliable uptime and fast response times.

Arguably, avoiding these headaches is why many DApps choose to use services like Infura or Alchemy to manage their node infrastructure. Of course, this is a trade-off, as this creates a centralization point, but let’s not discuss this difficult problem for now. 😉

Moving on, let’s talk about Providers. When you need to interact with the blockchain, the nodes you connect to (whether you build your own or use an off-the-shelf one provided by a third-party service) are often called “providers”.

The era of Web3 is here! Big changes in application architecture

Every Ethereum client (aka provider) implements a JSON-RPC specification. This ensures that when front-end applications want to interact with the blockchain, there is a unified approach. If you need an introduction to JSON-RPC, in simple terms, it is a stateless, lightweight Remote Procedure Call (Remote Procedure Call, RPC) protocol, which defines some data structures and their processing rules. It is related to transmission Protocol-independent, so this concept can be used in in-process communication, socket communication, HTTP communication or various messaging environments. It uses JSON (RFC 4627) as the data format.

Once you are connected to the blockchain through a provider, you can read the state stored on the chain. But if you want to write state, there is one more thing you need to do before committing the transaction to the blockchain — sign the transaction with your private key.

For example, imagine we have a DApp where users can read blogs or post blogs to the blockchain. There may be a button on the front end that allows anyone to query blogs written by a specific user. (Recall that reading data from the blockchain does not require the user to sign the transaction.) However, when the user wants to publish a new post to the chain, the DApp will ask the user to “sign” the transaction with their private key— — Only then will the Dapp forward the transaction to the blockchain. Otherwise, the node will not accept the transaction.

When it comes to signing, this is where Metamask comes in.

The era of Web3 is here! Big changes in application architecture

Metamask is a tool that allows applications to easily handle key management and transaction signing. It’s very simple: Metamask stores the user’s private key in the browser, and whenever the front end needs the user to sign a transaction, it calls Metamask.

Metamask also provides a connection to the blockchain (as a provider) because it already has a connection to a node provided by Infura (because Infura is required to sign transactions Matamask may need to read information such as estimated Gas from Infura before signing. In addition, if the transaction is sent through Metamask, Metamask will naturally need to connect to the node provided by Infura)). In this way, Metamask is both a provider and a signer (signer).

The era of Web3 is here! Big changes in application architectureChain Decentralized Storage

Of course, if you build an application, all the smart contracts and data of this application are completely carried on the Ethereum blockchain, which is also feasible. However, anyone who has ever built an application on Ethereum knows that storing everything on the blockchain is really expensive and (even if) agile.

Remember, with Ethereum, users pay every time new data is added to the blockchain. This is because adding states to a decentralized state machine increases the cost of nodes maintaining that state machine. With your Dapp, asking users to pay extra every time their transaction needs to add a new state is not the best user experience. One way to mitigate this is to use a decentralized off-chain storage solution such as IPFS or Swarm.

IPFS is a distributed file system for storing and accessing data. Therefore, the IPFS system does not store data in a centralized database, but distributes and stores data in a peer-to-peer website, which allows you to easily obtain data when needed.

IPFS also has an incentive layer called “Filecoin”. The incentive layer incentivizes nodes around the world to store and retrieve this data. You can use Infura (gives you an IPFS node) or Pinata (gives you an easy-to-use service where you can “pin” your files to IPFS, get the IPFS hash and store it on the blockchain ) such as providers.

Swarm is similar in that it is a decentralized storage network, but with one notable difference. Filecoin is a self-contained system, while Swarm’s incentive system is built into the Ethereum blockchain and enforced through smart contracts to store and retrieve data.

So until now, with IPFS or Swarm, our application architecture looks like this:

The era of Web3 is here! Big changes in application architecture

Astute readers may have also noticed in the diagram below that the front-end code is not stored on the blockchain. We could host this code on AWS like we did in Web 2.0, but that creates a centralization point for your DApp. What if AWS goes down? What if it reviews your app?

This is why, if you want to build a truly decentralized application, you should choose to host your front end on a decentralized storage solution, such as IPFS or Swarm.

So now the application architecture looks more like this:

The era of Web3 is here! Big changes in application architecture

The era of Web3 is here! Big changes in application architectureBlockchain data query

So far we discussed how to write data to the blockchain by signing and sending transactions to the blockchain. But how to read data from the blockchain smart contract? There are two main methods:

(1) Smart contract events

You can use the Web.js library to query and listen to smart contract events. You can listen for specific events and specify a callback every time the event is fired. For example, if you have a smart contract that sends a continuous flow of payments from A to B (at each block), then you can publish an event every time a new payment to B occurs. Your front-end code can listen to the events triggered by the smart contract and perform specific actions based on this.

(2) The Graph

The above approach works, but it has some limitations. For example, what if you deploy a smart contract only to realize that you need to publish an event that was not originally included? Unfortunately, you’d have to redeploy a new smart contract containing that event and data. Also, using callbacks to handle various UI logic can quickly become quite complicated.

This is where “The Graph” comes into play.

The Graph is an off-chain indexing solution that makes it easier to query data on the Ethereum blockchain. The Graph allows you to define which smart contracts to index, which events and method calls to listen to, and how to convert incoming events into entities that your front-end logic (or any object using the API) can consume. It uses GraphQL as a query language that many front-end engineers prefer because it is more expressive than traditional REST APIs.

By indexing blockchain data, The Graph allows us to query on-chain data with low latency in application logic.

Now, the application architecture looks like this:

The era of Web3 is here! Big changes in application architecture

We’re almost done, but there’s one major topic left: scaling.

The era of Web3 is here! Big changes in application architectureDApp expansion

As you may have heard, Ethereum doesn’t scale — at least not yet.

The era of Web3 is here! Big changes in application architecture

Ethereum’s average gas price

The era of Web3 is here! Big changes in application architecture

Average transaction fee

The era of Web3 is here! Big changes in application architecture

average block size

Obviously, there’s a problem with this. Building a DApp on Ethereum with high gas fees and full blocks leads to a very poor user experience. Thankfully, there are solutions in development.

A popular scaling solution is Polygon, a L2 (layer 2, two-layer chain) scaling solution. Instead of executing transactions on the main chain, Polygon uses “sidechains” to process and execute transactions. Sidechains are secondary blockchains linked to the main chain. Every once in a while, the sidechain submits an aggregation of its most recent blocks to the mainchain.

The era of Web3 is here! Big changes in application architecture

Some other examples of L2 solutions are Optimistic Rollups and zkRollups. The idea here is similar. A “rollup” smart contract is used off-chain to package transactions in batches, and then submit these transactions to the main chain periodically.

The idea worth absorbing is: the L2 solution executes the transaction off-chain (i.e. the slower part), and only saves the transaction data on the chain. This allows us to scale the blockchain because we don’t have to execute every transaction on-chain. This also makes transactions faster and cheaper – and transactions can still communicate with the Ethereum main chain if necessary.

The era of Web3 is here! Big changes in application architecture

The era of Web3 is here! Big changes in application architectureDevelopment Framework

If all of this has you feeling dizzy, you’re not alone. Piecing all these tools together is complex and makes for a very painful development experience. But rest assured – we’re starting to see new developer frameworks that really improve the developer experience.

For example, Hardhat is a developer framework that makes it easier for Ethereum developers to build, deploy, and test their smart contracts. Hardhat provides the “Hardhat Network”, which allows developers to deploy smart contracts on a local network – without having to deal with real network environments. Even better, it also provides a very good plug-in ecology to make it easier for developers. Hardhat also provides a javascript-like console.log() method for debugging.

Of course, this is just the beginning. Hopefully we will continue to see better developer tools in the future.

The era of Web3 is here! Big changes in application architectureepilogue

Most people spend months figuring out how the toolchain works, so if you are a new DApp developer, hopefully this article has saved you some time.

The era of Web3 is here! Big changes in application architectureTranslator’s summary

The core of Web 3.0 is decentralization, which is also the most essential difference from Web 2.0. Decentralization requires a distributed network to maintain the system state, and the distributed network needs to be composed of nodes maintained by different stakeholders.

In addition, a general-purpose virtual machine that can execute custom logic is also very important, which makes it possible for various applications to flourish in the Web 3.0 ecosystem. We call the code that can be executed on this virtual machine a smart contract.

At the application level, when the front-end calls a smart contract, it needs to communicate with one of the nodes in the network. The most suitable way for decentralization is to deploy a blockchain node by the application itself and become a member of the decentralized network. This undoubtedly increases the workload of developers. Another more lightweight way is to choose a node provided by a third-party supplier, such as Infura. But the latter has the risk of centralization, so it needs to be weighed according to the actual situation. In either case, the connected nodes are called Providers, and each Provider needs to implement a set of standard interfaces (Ethereum JSON-RPC Specification) for reading and writing blockchain data.

For the user management of the application, it is slightly different from the centralized storage of user information and keys in Web 2.0. Each user on the blockchain is uniquely identified by an address, and each address corresponds to a private key. Whenever you want to use an address as the sender of a transaction, you need to use the corresponding private key to sign it. So as long as you have the private key of a certain address, you can use this address to send transactions on different applications. But if we have 10 applications, do we need to enter our private key when using each application? Leaving aside the discussion of security, user convenience has become a problem. After all, each application may use the same private key, and repeated input will only consume the user’s patience. So is there a unified method to manage private keys? This is what wallets like Metamask do. Users only need to enter the private key once in the wallet (the actual operation may be to import mnemonics), and each application will call up the wallet when it needs to use the private key to sign. In addition, the wallet itself also has some simple functions, such as displaying user balances, transferring funds, etc.

In terms of data storage, since the data stored on the blockchain needs to be jointly maintained by the entire network, it is very expensive, so you can choose to store some data in IPFS/SWARM and other off-chain centralized storage. These storages use incentives and other methods to ensure data security. safety.

In terms of data query, you can directly call Provider’s read interface to directly read blocks, transactions, contract data and events, etc., but this has a very big disadvantage compared to traditional centralized databases, that is, lack of indexes and cannot be quickly read to the desired data. Therefore, off-chain indexing services such as The Graph have emerged as the times require, and applications can use it to index the desired data more conveniently and quickly.

Blockchain is often criticized for performance issues. Some Layer 2 solutions have been proposed one after another and some have been put into practice. By executing transactions on the second-layer chain, the main chain only records compressed transaction data. Can enjoy the security guarantee brought by the main chain. In addition, Ethereum plans to upgrade the consensus algorithm from POW to POS in 2022, which will also be a very important milestone for performance optimization.

Finally, for developers, tools such as Hardhat, Truffle, and Remix can be used to improve the efficiency of DApp application development.

If you have any questions or find any mistakes in the text, welcome to comment and exchange!

Record a few interesting phrases in the original text:

  • rabbit hole: Used to denote a bizarre, puzzling, or absurd situation or circumstance from which it is often difficult to escape.
  • take-home: take sth. home can mean absorbing/remembering ideas, ideas or concepts from a meeting. The use of “-” in the text to connect two words makes it an adjective, which can be understood as “worth absorbing”.

This article is translated from: The Architecture of a Web 3.0 application

recommended reading

Good article: From stackless coroutine to C++ asynchronous framework!

4D Pit Avoidance Guide! Defects and Thinking of C++ (Part 1)

After reading this article, become a Grafana master!

Top 10 performance traps! What every C++ engineer should know