Class starts. Practice of server-side technology upgrade based on Dubbo go

Time:2022-5-7

Zeng fanwei, senior development engineer of Hangzhou kaikela Education Technology Co., Ltd

Technical selection background

2020 is a year for our company’s development and growth. The whole team of the company has expanded from hundreds of people to thousands of people now. When it is used intensively, there will basically be thousands of people operating in the operation background at the same time. The original internal background operation system of the company is built with PHP, and the performance and business can no longer meet the company’s demand planning. In addition, at present, the Development Department of our company has made a micro service split, The main external service is the Dubbo cluster of Java language. The background system needs to seamlessly connect the Dubbo service of Java, so PHP can not meet the needs of our company.

At that time, I also investigated the Dubbo project of PHP. Because the project was basically not updated and maintained, I lost pass. Later, I was interested in the concise and high-performance go language, and then I paid attention to the Dubbo go project. After a period of research, I found that Dubbo go met our business needs and the community was very active. Later, I decided to choose Dubbo go as the background PC business framework.

Some students may also ask why we don’t use grpc with better cross language support, because the RPC service clusters of many companies are built based on Dubbo ecology at the beginning. If the cost of changing the framework is too high, we won’t consider it. Although grpc has better cross language support, many things need to be built by ourselves, such as service registration, service discovery, log monitoring, etc.

At that time, when deciding to choose Dubbo go, there were also some opposition voices within the development department. Why not directly switch to Java? If you switch to Java, there will be no problem of cross language communication. The problem of switching to Java lies in the high entry cost. Moreover, for the technology stack of the whole company, maintaining the diversity of languages can more calmly deal with future business changes. Go itself is a high-performance language not weaker than Java, It is very suitable for micro service architecture.

Challenges faced

After determining the framework selection, the first task I received is to build a set of scaffolds that can quickly create business projects, develop RPC proxy services based on HTTP protocol, and deploy the container deployment platform that needs to be connected to the company. Everything starts from scratch, and I can’t find reference materials on the online base.
The first is to plan the architecture of Dubbo go project and determine the project directory structure. After referring to Dubbo go demo and other go projects, the project directory structure is finally determined. The following directory structure can be used as a reference.

Class starts. Practice of server-side technology upgrade based on Dubbo go

In order to be consistent with the Java service registry, Dubbo go selects the following components in project selection:

  • Use zookeeper as the registry
  • Nacos as configuration center
  • Gorm is adopted for ORM database
  • Message queuing uses rocketmq

In order to increase the efficiency of development, we can simplify the configuration before the initialization of the provider service, and only keep the most basic configuration. For the coding of the provider service, refer to the Dubbo go demo

Class starts. Practice of server-side technology upgrade based on Dubbo go

The following is the main method code of service startup

Class starts. Practice of server-side technology upgrade based on Dubbo go

Design of Dubbo go RPC service gateway

Generally, when Dubbo is used, the provider side needs to expose interfaces and methods. The consumer side needs to be very clear about the interface definition and method definition used by the service, as well as the input and return parameter types and other information. It also needs to be based on the API provided by the provider side, so that the two ends can communicate and call normally.

However, the use scenario of the gateway does not care about the detailed definition of the interface to be called. The gateway only cares about the method to be called, the parameters passed, and the ability to receive the returned results. The implementation of the gateway agent is based on the generalized call feature of Dubbo / Dubbo go.

The following is the official demo given by Dubbo go. After the generic service is loaded, it needs to wait for 3 seconds to complete the call. However, when it is actually used, it must not be able to load the service in real time to wait for 3 seconds. Therefore, it is necessary to load and cache the services that need to be generalized when the gateway application is started.

Class starts. Practice of server-side technology upgrade based on Dubbo go

After studying the Dubbo go generalized call demo, it is found that it is feasible to design the Dubbo go gateway with this feature. The difficulty is that we need to obtain and cache the parameters of each gateway proxy RPC service method and the path of the service, so as to initialize the generalized call service before the call. The configuration of a service is as follows.

Class starts. Practice of server-side technology upgrade based on Dubbo go

It is obviously difficult to obtain the service through the java package through the java package, and it is difficult to obtain the service through the java package through the jar. If there is an error, it is obvious that the proxy can not be used to obtain the service through the java package. After a period of understanding, Java services can obtain the configuration through annotations. When starting the service after adding annotations to the method, the Java side will send the configuration information to MQ through messages, and the gateway will consume these messages to obtain the configuration of Java RPC services

Dubbo go’s RPC service doesn’t support annotations because the go language doesn’t support annotations, so after thinking about it, I wrote a small tool for scanning code, added corresponding annotations in front of each RPC service method, scanned the annotations to obtain the RPC service configuration, generated the RPC service configuration in the project directory after obtaining the configuration, read the configuration when starting the application and sent it to MQ.

After the implementation of the gateway agent, more functions can be realized on the basis of the gateway, such as token verification, white list, current limiting, fusing and log monitoring. The implementation effects of the gateway agent request are as follows:
Class starts. Practice of server-side technology upgrade based on Dubbo go

Container deployment

The company’s internal container deployment environment is Alibaba cloud’s k8s. Only image files need to be provided for deployment to the k8s platform. Because Dubbo go is a binary file after compilation, it does not need any additional third-party libraries and can operate stably in the docker environment. There are docker image files, as shown in the figure below. Any Linux distribution such as CentOS can be used as the base image.

LABEL maintainer="<[email protected]>"
LABEL version="1.0"
LABEL description="KKL-GO-NKO-BASE"`

ARG envType=stable
#Setting environment variables
ENV envType ${envType}
#Compile the packed compressed package
ADD ./target/nko-base-${envType}.tar.gz /app/

WORKDIR /app
EXPOSE 20000

After the image is written, it is provided to the publishing platform. The publishing platform machine starts the image, decompresses the package file, and executes the Dubbo Go program.

Container entrypoint set to [bash, -c, tar -zxf nko-base-stable.tar.gz && SERVER_ENV=kubernetes && sh ./nko-base/bin/load.sh start -group=stable]

Since there are usually multiple deployment environments from development test to production, we need to change the compilation script in the Dubbo go samples demo [see related link 1] to support multi environment packaging.

In addition, the default registered IP of Dubbo go is the virtual IP of k8s pod, and the networks between different k8s clusters cannot be interconnected. Therefore, if you need to call across clusters, you need to modify the default registered IP. Modify the default registered pod IP + port to the IP of kubernetes entity plus the corresponding port, and kubernetes will write the IP of entity plus the corresponding port environment variable in pod, The application program can obtain the IP and port of the physical machine by reading the environment variable. If this function needs to be realized, the registration logic of Dubbo go needs to be modified. For example, taking the zookeeper registry as an example, we can expand registry / zookeeper / registry Go’s registertempzookeepernode method is used to modify the registered IP and port. The code is shown in the figure below. Dubbo go official will support the function of custom registered IP and port in the form of configuration in a later version.

func (r *zkRegistry) registerTempZookeeperNode(root string, node string) error {
  ...
  regIp = os. Getenv (constant2. Registryenvip) // the IP address of the physical machine
  regPort = os. Getenv (constant2. Registryenvport) // the port of the physical machine
  urlNode, _ := common.NewURL(node)
  role, _ := strconv.Atoi(urlNode.GetParam(constant.ROLE_KEY, ""))
  if role == common.PROVIDER && regIp != "" && regPort != "" {
    urlNode.Ip = regIp
    urlNode.Port = regPort
    node = url.QueryEscape(urlNode.String())
  }

  zkPath, err = r.client.RegisterTemp(root, node)
  ...
}

Last words

If you encounter some problems when using Dubbo go, you can submit the issue or enter the community nail group for communication.

Personally, if you are ready to enter Dubbo go, you’d better solve these problems yourself and give the official feedback pr. if you encounter problems and solve them, you can grow and gain something, right?

Related links

Author introduction

Zeng fanwei (GitHub @ jack15083), a front-line programmer with 9 years of service side business development experience, once worked as a back-end development engineer in Tencent Yuewen and other companies. At present, he works in Hangzhou kaikela Education Technology Co., Ltd. and is engaged in go language service infrastructure, middleware and some business development.

Students who are interested in Apache / Dubbo go are welcome to join the exchange group through nail scanning code [or search nail group No. 31363295]:
Class starts. Practice of server-side technology upgrade based on Dubbo go