New features and future of superedge

author

Wang Dong, senior R & D Engineer of Tencent cloud, focuses on cloud native fields such as kubernetes and containers. He is a core developer of superedge. Now he is responsible for the privatization of Tencent cloud edge container tke edge.

background

On September 27, 2021, Wang Dong, a senior engineer from Tencent cloud, shared the new features of superedge and the way to the future at the edge computing special session of 2021 intelligent cloud edge open source summit held by VMware in conjunction with Intel, pingcap and other partners.

SuperEdgeIt was jointly launched by Tencent in 2020 with Intel, VMware, Huya live broadcast, Cambrian, capital online and meituanEdge computing distributed container management systemIt aims to seamlessly expand kubernetes’ centralized resource management capability to edge computing and distributed resource management scenarios, manage edge devices and applications, and enable many IOT devices.

The following is the full text of the sharing.

Four features of superedge

Superedge source

Superedge is an open source product in the TKE edge product family of Tencent cloud edge container management system, and its corresponding commercial products are tke edge public cloud service and tke edge privatization service. Its public cloud service will be incubated internally in 2018, officially tested and provided with services externally in 2019. At present, it is completely free externally; Its privatized products are currently provided with overall delivery and maintenance by lingqueyun.

Relationship between superedge and tke edge

Superedge is an open source edge capability component of tke edge, which does not include the creation of its commercial product cluster. Its edge capability components are completely open source, and the edge capabilities of open source products and commercial products are completely consistent. Even the functions of its open source product superedge will be updated earlier than its commercial products. Because Tencent currently maintains only one warehouse inside and outside, namely GitHub’s superedge.

The superedge core has four edge capabilities:

L3 level edge autonomy

This ability is mainly composed of light redlite-apiserverProvide. Why do you need competence? There are two reasons:

The first reason is that the cloud edge is generally a weak network or may be disconnected. In the case of weak network and disconnection, it is necessary to ensure the stability of edge services. Lite apiserver directly requests data from Kube apiserver in the cloud when the cloud side network is normal, but when the cloud request cannot be received, it will take out relevant components from the local cache and control the cache to return to the requester to ensure the stability of edge services.
Second, the edge node or edge site may be powered off and restarted, especially when the cloud edge is disconnected, the business container on the edge node will not be able to be pulled up. With the local cache of Lite apiserver, this problem can be avoided and the business container will be loaded from the local storage.

In addition, Lite apiserver provides some other capabilities. For example:

Access Kube apiserver as an incluster
Supports caching of all types of resources, including CRD
Edge node security: Lite apiserver uses the authority of the proxy component to request Kube apiserver, not super authority
Support a variety of cache storage: light edge can be stored in local files, and heavy edge can be stored in kV such as SQLite;

Cloud edge collaboration capability

This ability is mainly composed of light greentunnel-cloudandtunnel-edgeTwo components are provided. These two components are the cloud side tunnel completely developed by Tencent cloud edge computing team. At present, they can proxy four protocol requests: TCP, HTTP, HTTPS and SSH. Why do we need cloud side tunnel capability?

The first is that edge nodes generally do not have public IP. Edge nodes can actively access Kube apiserver in the cloud, but the cloud can not directly access edge nodes, so they need to open the cloud side reverse tunnel.
The second is to prepare for cloud side data transmission. The edge data should be sent back to the cloud for analysis and processing. An efficient and safe encrypted tunnel is a necessary condition.

However, superedge, a cloud side tunnel, is not exclusive to superedge. Any place that needs a tunnel can be configured and used directly according to its own scene.

Massive site management capability

This ability is mainly composed of light purpleapplication-grid-conterlloerandapplication-grid-wrapperTwo components are provided. Why do you need these two components?

First, there are generally many similar sites on the edge, which need to deploy the same set of applications. We can’t deploy them one by one. Direct circular deployment will be different for some sites,
application-grid-conterlloerIt was born to solve this problem. A user’s application can be deployed to multiple sites on the edge at the same time after one submission in the cloud, and allows site gray-scale capabilities and differences in site configuration.
The second is to prevent cross site access of edge applications. Because each site basically provides the same edge service, the service may be accessed across sites. Cross site access will cause two problems. Site a may write the data of site B disorderly, and the delay of cross site access is uncontrollable.

This isapplication-grid-wrapperTo solve the problem, he can lock the traffic of a site within a site and intelligently configure the back-end trafficendponit, lock the service in the range that users want.

The figure above is a typical use example of these two components. Site nodeunit-1 and site nodeunit-2 can deploy the same set of service servicegroup-1 at the same time. Site nodeunit-3 needs to deploy service servicegroup-2, and the service access of each site is only carried out in each site. The division of sites is also logical. A small computer room can be divided into one or more sites, and the nodes in the small computer room can also belong to multiple sites. Different sites can deploy different services to make full use of the resources in the small computer room.

Distributed health check

This ability is mainly composed of light yellowedge-health-admissionandedge-healthTwo components are provided. Why do you need these two components?

The first is to feed back the health of edge nodes as much as possible when the cloud edge is disconnected. For example, the cloud side of an available area in an edge small computer room is disconnected. For the time being, the cloud cannot know whether it is just the cloud side or the available area is down. There is no way to know the health status on the cloud, but other available areas of this small computer room can feed back each other’s health status through regular check.edge-healthIt plays this role.
The second is to maintain the stability of edge services and avoid repeated reconstruction. After pushing the original kubernetes to the edge, the expulsion ability of the original kubernetes does not fully accord with the edge. When the edge network is weak or disconnected, the state of nodes may change repeatedlyNotReadyHowever, the edge service is normal and is not affected by the weak cloud edge network. But kubernetes on the cloud don’t think so, but the nodes don’tNotReadyIt can lead to the expulsion of edge services, the repeated migration and reconstruction of edge services, and the instability of edge services. andedge-health-admissionTo solve this problem, he putedge-healthThe real health status of the feedback edge node is fed back to Kube apiserver to prevent the edge service from being expelled by mistake.

New features and principles of superedge

Since the open source in December last year, superedge has released five versions and brought many new features. Here are four typical ones. Others can pay attention to the superedge community.

New feature 1: ease of use, one key creation and one key integration

From open source to now, superedge has been deeply rooted in simplicity and ease of use.

Create edge k8s cluster with one click

When users do not have k8s clusters, you can useedgeadm initCreate an edge k8s cluster with one click:
```
##Create edge cluster with one click
./edgeadm init --apiserver-cert-extra-sans=…

##One click join edge node
./edgeadm join  kube-api-addr   --token xxxx…
```
With only one master node and one node, 2c2g’s resources can easily play around the edge and manage the edge nodes and edge devices scattered by users anywhere.

It is a transformation in realizationKubeadm, inKubeadm initAdded beforeInit nodeandInstall container runtimeAfter that, addon CNI network plug-in and the edge capability component mentioned above.

Usage andKubeadmExactly the same, only better thanKubeadmTwo more parameters. For details:Install edge k8s clusters and native k8s clusters with one click of edgeadm
One click integration edge capability

Users already have a native k8s cluster, which can be accessed throughAddon SuperEdgeOne click integrated edge capability.
```
##One click Addon superedge integrated edge capability
./edgeadm addon edge-apps --master-addr …

##One click join edge node at any position
./edgeadm  join kube-api-addr  --token=…
```
After integrating the edge capability, the native k8s cluster will have the ability to manage not only the central node and central application, but also the edge node and edge application, so as to realize the mixed management, mixed part and mutual elasticity of the center and edge. An edge node that can join at any position does not require SSH to the edge node. As long as the edge node can access the Kube apiserver in the center, it can be joined. In addition to this, superedge also has all the edge capabilities.

The principle of implementation is shown in the figure below:

A native k8s cluster built by users in any way,edgeadm addon SuperEdgeWill configure him to be standardKubeadm KubernetesCluster. If it is kubedm kubernetes, you can skip this step directly. After preparationAddon SuperEdgeandJoin edge nodePreconditions. The challenge here is preparationJoin edge nodeAny k8s cluster can join edge nodes at any position with one click. More detailed principles can be found:Addon superedge enables the native k8s cluster to manage edge applications and nodes

New feature 2: edge managed cluster + edge independent cluster + edge linked cluster

The following figure is the first step of superedge towards edge distributed multi cluster。

Superedge can now use the new open source distributed multi cluster project of Tencent cloud edge computing teamclusternetTo achieve unified control over edge managed clusters, managed edge independent clusters, and even edge linked clusters in the center.

Among them, the edge independent cluster of nano tube is not limited to superedge k8s cluster, but also includes lightweight k3s cluster, microk8s cluster… And other native k8s clusters.

New feature 3: tunnel remote login intranet node and HPA

Tunnel cloud and tunnel edge are the two ends of the cloud side tunnel. Superedge does not retain the connection of all edge nodes on each tunnel cloud instance pod, but each tunnel cloud only undertakes the tunnel edge tunnel connection of some edge nodes, that is, each cloud side tunnel has only one long connection. While most other cloud side tunnel projects are in the cloud, each instance needs to maintain all long links with edge nodes:

Number of long links = number of tunnel cloud instances * number of edge nodes

The main purpose of this is to support the automatic expansion and contraction of tunnel cloud

As the number of edge nodes in an edge cluster continues to break the upper limit of superedge, tunnel cloud can no longer maintain a fixed number of instances statically, but needs to dynamically expand the capacity of tunnel cloud instances to access more long connections and manage more edge nodes. This is the source of demand for automatic HPA of tunnel cloud.

Finally, with the help of tunnel tunnel, superedge supports remote secure SSH to edge nodes without public IP, which brings great convenience for users to remotely operate edge nodes without public IP.

New feature 4: remote batch adding edge nodes

The last new feature is the remote batch addition of edge nodes. This is a requirement for the mass production of superedge, and the relevant code has been open source to superedgepenetratormodular. Remote batch adding edge nodes can be divided into two cases:

Edge nodes that the cloud can SSH to

The cloud can SSH to the edge node. This operation is relatively routine. By issuing an SSH job, batch SSH Remote execution commands can be used to add edge nodes.

penetratorThe key is how to batch add edge nodes that cannot be SSH directly?
Edge nodes that the cloud cannot SSH to

As shown in the following figure, a node of the same subnet can be added to the edge k8s cluster through an agent or other methods. Take this edge node as a springboard, and then distribute the task job to this springboard node, and then you can batch execute and add edge nodes in the same intranet as this springboard node. This enables remote batch addition of edge intranet nodes that cannot SSH to.

Superedge’s future cloud side

Superedge on the future cloud

Tencent cloud edge team has just opened its second open source projectClusternet, this is not an open source project related to cluster network, but to realizeLike accessing the Internet network, users can access k8s clusters everywhereThe goal is to build a distributed multi cluster management open source project. Why do you need this project?

The first is to meet the management of massive edge nodes

The edge nodes managed by a k8s cluster are limited, and the upper limit of nodes given by the original k8s cluster is 5000. The more nodes the k8s cluster manages, the maintenance cost and technical difficulty will rise exponentially. Putting a large number of nodes in a cluster is a relatively high risk. Once there is a problem in the center, the application of nodes may be affected.
To manage tens of thousands of edge nodes, a single cluster is not elegant, but a small and beautiful multi cluster is safer and more stable. At present, clusternet can manage all kinds of k8s clusters, including public cloud, privatized and edge k8s clusters. It can manage and access all k8s clusters in a central control plane, and can access each other from the managed clusters.
The second is to meet the needs of site and application disaster recovery

Taking care of various k8s clusters is only the first step to realize distributed multi cluster management. Cluster disaster recovery and application disaster recovery are the purpose. Edge sites are more likely to be disconnected and more frequently than centers. After the site goes down, the corresponding site services need to continue to provide services on adjacent sites or backup sites. Cluster migration and dual activity in the same city are urgent needs. Edge applications will not only be deployed in one site, but also need to continue to provide services on other sites when one site crashes.

The figure above shows the architecture of clusternet, which currently consists of two componentsclusternet-agentandclusternet-hubform.clusternet-agentBe responsible for registering the k8s cluster with the parent cluster,clusternet-hubIt is responsible for registering, aggregating Kube apiserver of each sub k8s cluster, and deploying applications to multiple k8s clusters.

On the edge of superedge’s future

The following figure shows the current status of cloud side service exchange and side service exchange in edge k8s clusters.

Cloud side service mutual visitsMost of them are exposed through nodeport, and few edge projects are implemented like native k8s clusters,Seamlessly exchange visits within a cluster。

Exchange of visitsThe difficulty is even higher. If there is a one-way network between the sides, they can also visit each other through tunneling. If the physical network is completely blocked, they can only transit through the cloud. Even if the cloud side service mutual access and edge side service mutual access are realized, how to avoid performance loss and break through the instability of cloud side and edge physical network?

The solutions here can focus on the superedge community, and relevant solutions will be launched in the future.

The future of superedge

Superedge on the end has implemented the native edgex foundry of addon. You can selectively deploy the components of each layer of edgex foundry through the following commands:

attlee➜  ✗ ./edgeadm addon edgex -h
Addon edgex to Kubernetes cluster

Usage:
edgeadm addon edgex [flags]

Flags:
--core       Addon the edgex core-services to cluster.
--app        Addon the edgex application-services to cluster.
--device     Addon the edgex device-services to cluster.
--ui         Addon the edgex ui  web to your cluster.

Detailed operations can be viewedUse it on the superedge device.

This is only the first step for superedge to realize edge device management. Edgex foundry is only one of many device management platforms. In the future, superedge will abstract and integrate with more edge device platforms and launchSeamless connection of multi platform edge equipment platformSolutions. But no matter what kind of scheme, superedge will let users choose freely in the way of addon, and will never bind any edge device platform.

The last figure shows the current deployment mode of superedge and edgex foundry at the end and the access mode of equipment。 A site can manage the edge devices of the corresponding site only by deploying a set of edge services of superedge and edgex foundry.

In the future, superedge will also provide a series of support for edge sites, including site autonomy, site workload, site disaster recovery, etc., and uniformly manage users’ edge sites in the cloud.

Finally, I would like to give you a word:

Edge computing technology will become the key to the success of all things interconnection, and will serve 5g and digitization with low delay and low cost!

Original video of speech
https://attlee-1251707795.cos.ap-chengdu.myqcloud.com/superedge/v0.6.0/superedge_future.mp4

Focus on Tencent cloud primiband official account, background reply keyword [cloud edge open source summit] to get the speech PPT manuscript.

Superedge related articles:

Relevant materials of landing cases:

About us

More about cloud native cases and knowledge, you can focus on the same name [Tencent cloud primer] official account.

Welfare: the official account of background reply (manual) can get the “Tencent cloud native roadmap manual” & “cloud cloud native best practices”.

[Tencent cloud native] cloud says new products, Yunyan new technology, cloud tours, new cloud and cloud reward information, scanning code concern about the same official account number, and get more dry cargo in time!!

Recommended Today

While recruiting new people and cutting old people, are big Internet companies so two faced?