Springboot + netty + redis build a long connection cluster

Time:2021-11-24

1. Background

The company is working on a social project. Audio and video technology is the third-party technology used. The broadcasting in the live room, in and out of the room, and push related functions need to be developed by ourselves, so we need to build our own long connection server. Therefore, in terms of technical selection, in order to ensure the high concurrency performance and long connection performance of services, springboot + netty is used to realize the construction of long connection services in the microservice architecture itself. This paper is not the focus on the relationship between springboot and netty framework, but the focus of this paper is on the construction of netty cluster to realize the message forwarding function.

2. Technical Options & Implementation

2.1 about netty long connection

Netty is an excellent NiO asynchronous event driven framework. On the basis of JDK NiO, it encapsulates and expands the packaging, making NiO API simple and easy to use. Many frameworks are also applied at the bottom (such as Dubbo). Official website entry:Netty, netty can be used as a long connection service. Using WS protocol, a high-performance long connection server can be built. It is said that a single machine can support about 10000 connections.

Those familiar with netty or NiO know that the underlying data transmission is connected to the channel through a network. For single machine netty, the channels are connected to the same server, and the communication between channels can be obtained and forwarded directly according to the bound user information. However, for the netty cluster, the connection of each client may be on different servers. Therefore, when the channels communicate with each other, it is necessary to judge whether they are at the current node and realize message forwarding.

2.2 technical options

There are many technical frameworks to realize message forwarding, such as ZK, MQ, redis, etc. because the current project has built its own MQ and redis services, considering the cost, it is only selected from MQ and redis at present.

As mentioned above, after the netty cluster is set up, after the channel receives the message, it needs to judge whether the channel of the target user is in this node. If not, it needs to forward the message to the target node and write the message to the channel. Therefore, the broadcast mechanism of MQ and the publish subscription of redis can realize the function.

The comparison is as follows:

MQ: it is specialized in message queuing, with high reliability, but it is heavyweight and asynchronous, and can not guarantee real-time.

Redis: lightweight, low latency, high concurrency and low reliability.

Both are in use in the project. Considering that the live message can ensure timeliness, redis publish and subscribe function is selected, which is relatively simple to integrate.

2.3 implementation architecture diagram

Springboot + netty + redis build a long connection cluster

① When the service is started, register your own node information with redis and specify the listening channel.

② After the client successfully connects to the netty server, it binds the node joined by the user in the redis cache for subsequent information forwarding and forwarding the information to the specified channel.

③ During channel forwarding, it is judged that if the current user is in this node, it will be forwarded directly. If not, it will obtain the registered node information from redis and publish the message to the specified channel node.

3. Code implementation

Springboot + netty + redis build a long connection cluster

Bind the current node, start the service and listen to the specified channel, and specify the listener.

Springboot + netty + redis build a long connection cluster

Subscription node generator native IP + service port

Springboot + netty + redis build a long connection cluster

Listener implementation: the monitored channel receives the message, processes it and sends it to the corresponding channel

Springboot + netty + redis build a long connection cluster

Message forwarding sender

4. Summary

① Through redis publishing and subscription, netty server has multiple clusters. When adding multiple nodes, the new node will automatically register the channel to listen to, and the message will be automatically delivered to the newly registered node channel when forwarding. It has strong scalability.

② At present, there are many new advanced features in redis 5. The author is also slowly learning and can be applied to project business scenarios.

As time goes by, there will be a vast sky ahead!

Recommended Today

Seven solutions for distributed transactions

1、 What is distributed transaction Distributed transaction means that transaction participants, transaction supporting servers, resource servers and transaction managers are located on different nodes of different distributed systems. A large operation is completed by more than n small operations. These small operations are distributed on different services. For these operations, either all of them are […]