Containerization | implementation scheme of new generation MySQL high availability architecture based on kubernetes


Author: Gao riyao, senior MySQL kernel R & D

This article comes from the author’s speech “implementation scheme of new generation MySQL high availability architecture based on kubernetes” at kubesphere & Friends 2021 Hangzhou station.

This is the third article in the MySQL container series, which mainly introduces the design idea of MySQL container helm version [1].

Dockerfile introduction

First, the container role in a pod of radondb MySQL generally includes three containers: MySQL, xenon and slowlog.
Alt text

The directory structures of MySQL and xenon dockerfile are as follows:

Alt text

MySQL dockerfile parsing

Before starting the MySQL main process, you need to prepare database configuration and initialization, which should be solved before the final MySQL operation. When creating an image, configure MySQL dockerfileENTRYPOINTandCMDParameter to prepare database configuration, initialization and other processes in advance.

Docker is layered. Each command will create a mirror layer. Too many layers will lead to rapid expansion. When making images, it is not recommended to layer too much.

MySQL DockerfileExamples of commands in the file are as follows:
Alt text

MySQL mysql-entry.shThe file contains the startup command, and its main execution process is as follows:

Alt text

Xenon dockerfile parsing

Xenon dockerfile is relatively simple, similar to the MySQL dockerfile process.

Xenon DockerfileExamples of commands are as follows:

Alt text

xenon-entry.shMain functions:

  1. Generate xenon configuration file and call it when xenon starts
  2. ping host

Name generation and environment variables

Name generation

First, let’s look at the list of function files in chart directory. The files contained in chart are shown in the following figure:

Alt text

Among them,_helpers.tplFrequently citedChart.yamlandvalues.yamlVariables defined in, and then realize the following capabilities:

  • Generated app Name: helm install release < version name, such as emo > < project name, such as radondb MySQL >
  • generateserviceAccountNamename.
  • Generate chart name and version.

By commandhelm get all demo, you can view all the information in the demo. See the example of service name as follows:

104 # Source: radondb-mysql/templates/serviceaccount.yaml
105 apiVersion: v1
106 kind: ServiceAccount
107 metadata:
108   name: demo-radondb-mysql
109   labels:
110     app: demo-radondb-mysql
111     chart: radondb-mysql-1.0.0
112     release: "demo"
113     heritage: "Helm"

environment variable

environment variableThe main purpose is to save passwords and decouple configuration parameters.

  • Secrets.yaml function

    • Opaque
    • Base64 transcoding
    • Encryption plug-in
  • Configmap.yaml function

    • Decouple configuration parameters from docker images
    • Save some configuration parameters and modify the script of lable

Alt text

How to identify the roles of leader and follower in the cluster?

To identify the cluster node role, you need to create a service account and grant corresponding permissions. By executing inconfigThe script saved in the file calls the corresponding API to modify the label of the corresponding role.

Alt text

After the node role is modified, the corresponding label effect is as follows. At this time, the nodes of leader and follower roles can be easily identified through the service tag suffix.
Alt text

On the kubesphere [2] management console, you can view the modified role. The example is as follows:

Alt text

How to separate reading and writing?

Radondb MySQL read-write separation is realized through headless service for stable DNS [3].

Alt text

Assign a virtual IP (VIP) that can be accessed inside the cluster. The VIP is fixed, while the IP of the pod bound by the node can be changed. Each node is assigned a port as an external access entry. With this feature, one read IP and one write IP can be set to achieve the purpose of read-write separation without worrying about the IP changes caused by the new binding pod.

Take the leader node as an example. The clusterip shown in the following figure corresponds to the write IP(, which is bound to the current pod (primary) IP172.17.0.10

Alt text

Meet kubeshpere

Radondb MySQL has logged into the kubesphere App Store (version 3.1.0). You can view and manage clusters through kubesphere.

1. View the gtid and corresponding status of the cluster when it is normal through the terminal

Alt text

2. Simulate the scenario where the follower node hangs up

Kill the follower named demo-radondb-mysql-2:

Alt text

Log in to the terminal from another node and check the cluster status again. The MySQL and IO / SQL status of the follower node are abnormal.

Alt text

View from kubesphere interface:

Alt text

The suspended nodes are pulled up again, and the cluster starts to restart the election:

Alt text

3. Scenario where the leader (demo-radondb-mysql-0) node hangs up

The leader is demoted to follower, and the other two nodes enter the candidate state:

Alt text

From the kubesphere interface, you can’t find the leader Pod:

Alt text

After waiting for a period of time, the cluster selects a new master (demo-radondb-mysql-2):

Alt text

From kubesphere, you can see that the original master (demo-radondb-mysql-0) has changed into a follower:

Alt text

4. Network isolation

Isolate the new master (demo-radondb-mysql-2).

Alt text

After waiting for a while, you can see that the new master (demo-radondb-mysql-0) is re selected:

Alt text

[1]. RadonDB MySQL Kubernetes:

[2]. KubeSphere:

[3]. Headless service:😕/

About radondb

Radondb open source community is a cloud oriented native and containerized database open source community. It provides database technology lovers with a technology sharing platform around mainstream open source databases (mysql, PostgreSQL, redis, mongodb, Clickhouse, etc.), and provides enterprise radondb open source products and services.

At present, radondb open source database series products have beenEverbright Bank, Pudong Development Silicon Valley Bank, Hami bank, Taikang Insurance, Taiping Insurance, AXA, sunshine insurance, Centennial life insurance, Anji logistics, Anchang logistics, blue moon, Tiancai Shanglong, rockjiahua, Shengzhe technology, Wuxi huirun sports, Beijing Telecom, Jiangsu transportation holding, Sichuan Airlines, Kunming Airlines and state-controlled biologyAnd thousands of enterprises and community users.

Radondb can be delivered based on cloud platform and kubernetes container platform. It not only provides database product solutions covering multiple scenarios, but also provides professional cluster management and automatic operation and maintenance capabilities. The main functional features include:High availability master-slave switching, strong data consistency, read-write separation, one click installation and deployment, multi-dimensional indicator monitoring & alarm, elastic capacity expansion & capacity reduction, horizontal free expansion, automatic backup & recovery, multi activity in the same city and remote disaster recoveryWait. Radondb only needs enterprise and community users to focus on business layer logic development, without paying attention to complex issues such as cluster high availability selection, management and operation and maintenance, so as to help enterprise and community users greatly improve the efficiency of business development and value innovation!


Wechat group:Please search add group assistant micro signal radondb

Recommended Today

Programming Xiaobai must understand the network principle

How is the network composed? Why can we easily surf the Internet now?Whether you are a computer major or not, you may always have such questions in your heart!And today we will solve this matter and tell you the real answer! Basic composition of network First, let’s look at this sentence Connect all computers together […]