Election mechanism of pulsar functions worker

Time:2020-2-18

abstract

In pulsar, function, source and sink are all run on function worker. For function content, please refer to an article to understand pulsar functions. For use of source and sink, please refer to

  • Pular source beginner level chapter
  • A guide to getting started with pular sink.

This paper introduces the election mechanism of functions worker. Read this article for a deeper understanding of functions worker.

Function worker architecture

The following is the architecture diagram of pulsar functions worker
Election mechanism of pulsar functions worker

The components metadata manager, Scheduler Manager, runtime manager and membership manager in the schema are all running on the worker.

Worker’s election mechanism

Election mechanism of pulsar functions worker

  • Worker on blue background
  • The light green ellipse is a subscription and a consumer
  • Broker on a square green background
  • The square red background is topic

Let’s take a look at the worker election.

  1. In the figure, three workers are started, namely worker service1, worker service2 and worker service3. These workers can be on the same machine or on different machines.

2. When starting, a consumer will be started inside each worker, which is used for election. The ID, host name and port number of each worker will be bound to the consumer.
3. The consumer is started based on the failover mode (refer to [here] for the failover subscription mode of the consumer (http://pulsar.apache.org/docs…). Only one consumer connected to the same topic with the same subscription name is active at the same time.
4. The worker using the consumer is the leader, which is responsible for scheduling and processing some other operations.

Worker service1 and worker service2 are a cluster, connected to topic1 at the same time, with the same subscription name sub, and the leader will be generated in both of them.
Worker service3 is another cluster, connected to Topic2, and the subscription name is sub3. Because there is only one worker, it is a leader.

Test the election process

Let’s practice the election process.

Before building this test, the following dependencies need to be installed on the computer. This test is conducted on the MAC system.

  • Docker: you can install docker according to the instructions.

Start the stand-alone pulsar service

Pull the image of apachepulsar / pulsar: 2.3.1, and then usedocker runCommand to start,-dParameter to run the service in background mode,-itRun the container in interactive mode and assign a pseudo input terminal to the container.
--nameSpecify the container to usepulsar-standalone-function-leaderThe name.

docker run -d -it -p 6650:6650 -p 8080:8080 -v $PWD/data:/pulsar/data --name pulsar-standalone-function-leader apachepulsar/pulsar:2.3.1 bin/pulsar standalone
docker logs -f 
11:17:17.363 [pulsar-web-55-4] INFO  org.apache.pulsar.broker.web.PulsarWebResource - Successfully validated clusters on tenant [public]
11:17:17.369 [pulsar-web-55-4] INFO  org.apache.pulsar.broker.admin.impl.NamespacesBase - [null] Created namespace public/default
11:17:17.370 [pulsar-web-55-4] INFO  org.eclipse.jetty.server.RequestLog - 172.17.0.2 - - [18/May/2019:11:17:17 +0000] "PUT /admin/v2/namespaces/public/default HTTP/1.1" 204 0 "-" "Pulsar-Java-v2.3.1" 10
11:17:17.377 [pulsar-web-55-12] INFO  org.apache.pulsar.broker.web.PulsarWebResource - Successfully validated clusters on tenant [public]

After starting pulsar, the above log shows that the service started successfully.

Simulated election process

The worker’s election is based on the consumer’s failover mode, so in this test, start the consumer directly to simulate the election.
As shown in the above election figure, this test will start three consumers in failover mode, two of which are connected to topic1, and the subscription name is sub, which represents a worker cluster; the other consumer is connected to Topic2, and the subscription name sub3 represents another worker cluster.

For this test, four windows need to be opened, named windows1, windows2, Windows3 and windows4 respectively.
Windows 4 is used for verification.

window1

Start a consumer in failover mode, and use the subscription name sub to subscribe to public / default / topic1

docker exec -it pulsar-standalone-function-leader /bin/bash
./bin/pulsar-client consume persistent://public/default/topic1 --num-messages 0 --subscription-name sub -t Failover

window2
Start a consumer in failover mode, and use the subscription name sub to subscribe to public / default / topic1

docker exec -it pulsar-standalone-function-leader /bin/bash
./bin/pulsar-client consume persistent://public/default/topic1 --num-messages 0 --subscription-name sub -t Failover

window3
Start a consumer in failover mode, and use the subscription name sub3 to subscribe to public / default / Topic2

docker exec -it pulsar-standalone-function-leader /bin/bash
./bin/pulsar-client consume persistent://public/default/topic2 --num-messages 0 --subscription-name sub3 -t Failover

The operation of the three windows uses the same subscription mode failover, and windows 1 and 2 use the same subscription name to subscribe to the same topic.

window4
Get the statistics of topic1 through the command topic stats

./bin/pulsar-admin topics stats topic1
{
  "msgRateIn" : 0.0,
  "msgThroughputIn" : 0.0,
  "msgRateOut" : 0.0,
  "msgThroughputOut" : 0.0,
  "averageMsgSize" : 0.0,
  "storageSize" : 0,
  "publishers" : [ ],
  "subscriptions" : {
    "sub" : {
      "msgRateOut" : 0.0,
      "msgThroughputOut" : 0.0,
      "msgRateRedeliver" : 0.0,
      "msgBacklog" : 0,
      "blockedSubscriptionOnUnackedMsgs" : false,
      "unackedMessages" : 0,
      "type" : "Failover",
      "activeConsumerName" : "383dc",
      "msgRateExpired" : 0.0,
      "consumers" : [ {
        "msgRateOut" : 0.0,
        "msgThroughputOut" : 0.0,
        "msgRateRedeliver" : 0.0,
        "consumerName" : "383dc",
        "availablePermits" : 1000,
        "unackedMessages" : 0,
        "blockedConsumerOnUnackedMsgs" : false,
        "metadata" : { },
        "address" : "/127.0.0.1:50014",
        "connectedSince" : "2019-05-18T11:30:34.161Z",
        "clientVersion" : "2.3.1"
      }, {
        "msgRateOut" : 0.0,
        "msgThroughputOut" : 0.0,
        "msgRateRedeliver" : 0.0,
        "consumerName" : "51911",
        "availablePermits" : 1000,
        "unackedMessages" : 0,
        "blockedConsumerOnUnackedMsgs" : false,
        "metadata" : { },
        "address" : "/127.0.0.1:50018",
        "connectedSince" : "2019-05-18T11:30:42.742Z",
        "clientVersion" : "2.3.1"
      } ]
    }
  },
  "replication" : { },
  "deduplicationStatus" : "Disabled"
}

There is a subscription and two consumers under topic1. The current active consumername is383dc, which is the subscription in Windows 1. This indicates that the worker currently holding the subscription is a leader.
When you turn off the subscription of windows 1, take a look again. In Windows 1, use Ctrl + C to turn off the consumer and go back to windows 4 to view the subscription.

./bin/pulsar-admin topics stats topic1
{
  "msgRateIn" : 0.0,
  "msgThroughputIn" : 0.0,
  "msgRateOut" : 0.0,
  "msgThroughputOut" : 0.0,
  "averageMsgSize" : 0.0,
  "storageSize" : 0,
  "publishers" : [ ],
  "subscriptions" : {
    "sub" : {
      "msgRateOut" : 0.0,
      "msgThroughputOut" : 0.0,
      "msgRateRedeliver" : 0.0,
      "msgBacklog" : 0,
      "blockedSubscriptionOnUnackedMsgs" : false,
      "unackedMessages" : 0,
      "type" : "Failover",
      "activeConsumerName" : "51911",
      "msgRateExpired" : 0.0,
      "consumers" : [ {
        "msgRateOut" : 0.0,
        "msgThroughputOut" : 0.0,
        "msgRateRedeliver" : 0.0,
        "consumerName" : "51911",
        "availablePermits" : 1000,
        "unackedMessages" : 0,
        "blockedConsumerOnUnackedMsgs" : false,
        "metadata" : { },
        "connectedSince" : "2019-05-18T11:30:42.742Z",
        "clientVersion" : "2.3.1",
        "address" : "/127.0.0.1:50018"
      } ]
    }
  },
  "replication" : { },
  "deduplicationStatus" : "Disabled"
}

You can see that the subscription started in window 2 has been activated successfully. The current active consumer name is 51911. At this time, the worker holding the consumer becomes the leader.

summary

The above is the election mechanism for running the worker in the connector. You can see that it cleverly uses the failure mode of the consumer to realize the high availability mechanism of the worker.

Recommended Today

Python basics Chinese series tutorial · translation completed

Original: Python basics Python tutorial Protocol: CC by-nc-sa 4.0 Welcome anyone to participate and improve: a person can go very fast, but a group of people can go further. Online reading Apache CN learning resources catalog introduce Seven reasons to learn Python Why Python is great Learn Python introduction Executing Python scripts variable character string […]