The course starts. The actual combat of server technology upgrade based on Dubbo go

Time:2021-6-10

Zeng fanwei, senior development engineer of Hangzhou kaikela Education Technology Co., Ltd

Technology selection background

2020 is a year for our company to develop and grow. The whole team of the company has expanded from hundreds to thousands. When it is used intensively, thousands of people will operate in the operation background at the same time. The original internal background operation system of the company is built with PHP, and its performance and business can no longer meet the needs of the company, In addition, the Development Department of our company has split the micro service, the main external service is the Dubbo cluster of Java language, and the background system needs to seamlessly connect with the Dubbo service of Java, so PHP has gradually failed to meet the needs of our company.

At that time, I also researched the Dubbo project of PHP. Because the project was basically not updated and maintained, I passed it. Later, I was interested in the simple and high-performance go language, and then I focused on the Dubbo go project. After a period of research, I found that Dubbo go met our business needs, and the community was very active, Later, I decided to choose Dubbo go as the PC business framework in the background.

Some students may also ask why we don’t use grpc with better cross language support, because the initial RPC service clusters of many companies are built based on the Dubbo ecosystem. If the cost of changing the framework is too high, we will not consider it. Although grpc has better cross language support, many things need to build their own wheels, such as service registration and service discovery, Log monitoring, etc.

At that time, when we decided to choose Dubbo go, there were some voices of opposition in the development department. Why not transfer Java directly? If we transfer Java, there would be no problem of cross language communication. The problem of transferring Java is that the entry cost is high, and for the technology stack of the whole company, only by maintaining the diversity of languages can we cope with the future business changes more calmly, Go itself is a high performance language not inferior to Java, which is very suitable for microservice architecture.

Challenges

After determining the framework selection, the first task I received was to build a set of scaffolds that can quickly create business projects, develop RPC proxy services based on HTTP protocol, and access the company’s containerized deployment platform for deployment. Everything starts from scratch, and I can’t find any reference materials on the Internet.
The first is to plan the architecture of the Dubbo go project and determine the directory structure of the project. After referring to the Dubbo go demo and other go projects, the directory structure of the project is finally determined. The following directory structure can be used as a reference.

The course starts. The actual combat of server technology upgrade based on Dubbo go

In order to be consistent with the Java service registry, Dubbo go selects the following components for project selection:

  • Using zookeeper as the registry
  • Nacos as configuration center
  • Gorm is used in database orm
  • Message queuing using rocketmq

In order to increase the efficiency of development, we can simplify the configuration before the initialization of the provider service, and only keep the most basic configuration. It can be similar to the following. For the coding of the provider service, please refer to Dubbo go demo

The course starts. The actual combat of server technology upgrade based on Dubbo go

Here is the main method code for service startup

The course starts. The actual combat of server technology upgrade based on Dubbo go

Design of Dubbo go RPC service gateway

Generally, with Dubbo, the provider side needs to expose the interface and method, the consumer side needs to be very clear about the interface definition and method definition used by the service, as well as the input parameter and return parameter type and other information, and it also needs to be based on the API provided by the provider side, so that the two sides can communicate and call normally.

However, the use scenario of gateway does not care about the detailed definition of the interface to be called. The gateway only cares about the method to be called, the parameters to be passed, and the ability to receive the returned results. The implementation of gateway proxy is based on the generalized calling feature of Dubbo / Dubbo go.

The following is the demo officially given by Dubbo go. After the generic service is loaded, it needs to wait 3 seconds to complete the call. However, in actual use, it is certain that the service cannot be loaded in real time to wait 3 seconds. Therefore, when the gateway application is started, it is necessary to load and cache the services that need to be generalized.

The course starts. The actual combat of server technology upgrade based on Dubbo go

After the research of Dubbo go generalization call demo, it is found that it is feasible to design Dubbo go gateway with this feature. The difficulty lies in that we need to get and cache the configuration of each RPC service method parameter and service path that need gateway proxy, so as to initialize the generalization call service before calling. The configuration of a service is as follows.

The course starts. The actual combat of server technology upgrade based on Dubbo go

As the gateway agent is made in go language, Java RPC service configuration can not be obtained through Java jar package. If it is maintained manually, the workload is too heavy and error prone, which is obviously unacceptable. After a period of understanding, Java services can achieve configuration acquisition through annotations. When the Java side starts the service after adding annotations to the method, it will send configuration information to MQ through messages, and the gateway will consume these messages to achieve configuration acquisition of Java RPC services

Dubbo go’s RPC service does not support annotations, so after thinking, I wrote a small tool to scan the code. I added the corresponding annotations before each RPC service method, and obtained the RPC service configuration by scanning the annotations. After obtaining the configuration, I generated the RPC service configuration in the project directory, read the configuration and sent it to MQ when starting the application.

After the gateway proxy is implemented, more functions can be implemented on the basis of the gateway, such as token verification, white list, current limiting, fusing, and log monitoring
The course starts. The actual combat of server technology upgrade based on Dubbo go

Containerized deployment

The internal container deployment environment of the company is k8s of Alibaba cloud. To deploy to k8s platform, only image files are required. Since Dubbo go is compiled as a binary file, it does not need any additional third-party libraries and can run stably in docker environment. There are docker image files, as shown in the figure below. You can use any Linux distribution such as CentOS as the base image.

LABEL maintainer="<[email protected]>"
LABEL version="1.0"
LABEL description="KKL-GO-NKO-BASE"`

ARG envType=stable
#Setting environment variables
ENV envType ${envType}
#Compile the packed package
ADD ./target/nko-base-${envType}.tar.gz /app/

WORKDIR /app
EXPOSE 20000

After the image is written, it is provided to the publishing platform. The publishing platform machine starts the image, decompresses the package file, and executes the Dubbo Go program.

Container entrypoint set to [bash, -c, tar -zxf nko-base-stable.tar.gz && SERVER_ENV=kubernetes && sh ./nko-base/bin/load.sh start -group=stable]

Since there are usually multiple deployment environments from development testing to production, we need to change the compilation script in Dubbo go samples demo [see related link 1] to support multi environment packaging.

In addition, the default registered IP of Dubbo go is the virtual IP of k8s pod, and the networks between different k8s clusters cannot be interconnected. Therefore, if you need to call across clusters, you need to modify the default registered IP, change the default registered pod IP + port to the IP of kubernetes physical machine plus the corresponding port, and kubernetes will write the IP of the physical machine plus the corresponding port environment variable in the pod, The application program can obtain the IP plus port of the physical machine by reading the environment variables. If you need to implement this function, you need to modify the Dubbo go registration logic. For example, take the zookeeper registry as an example, we can modify the registered IP and port by extending the registertempzookeepernode method of registry / zookeeper / registry.go. The code is shown in the figure below. Dubbo go will officially support the function of custom registered IP and port in the form of configuration in later versions.

func (r *zkRegistry) registerTempZookeeperNode(root string, node string) error {
  ...
  Regip = OS. Getenv (constant2. Registryenvip) // IP of physical machine
  Regport = OS. Getenv (constant2. Registryenvport) // port of physical machine
  urlNode, _ := common.NewURL(node)
  role, _ := strconv.Atoi(urlNode.GetParam(constant.ROLE_KEY, ""))
  if role == common.PROVIDER && regIp != "" && regPort != "" {
    urlNode.Ip = regIp
    urlNode.Port = regPort
    node = url.QueryEscape(urlNode.String())
  }

  zkPath, err = r.client.RegisterTemp(root, node)
  ...
}

Last words

If you encounter some problems in the process of using Dubbo go, you can submit an issue or enter the community nail group for communication.

I suggest that if you are ready to go to Dubbo go, you’d better solve these problems yourself and give the official feedback. Only when you solve problems can you grow and gain something, right?

Related links

Introduction to the author

Zeng fanwei (GitHub @ jack15083), a front-line programmer with 9 years of service business development experience, has worked as a back-end development engineer in many companies such as Tencent Yuewen. Now he works in Hangzhou kaikela Education Technology Co., Ltd., and is engaged in the development of go language service infrastructure, middleware and some business.

Students interested in Apache / Dubbo go project are welcome to join the communication group [or search the nail group No. 31363295] through pin scanning code
The course starts. The actual combat of server technology upgrade based on Dubbo go