Installation and use of librdkafka

Time:2020-3-28

Title: librdkafka installation and use
copyright: true
date: 2019-05-20 08:53:02
Categories: Message Queuing
tags:

  • kafka
  • Message queue
  • librdkafka
  • MQ

Install librdkafka

git clone https://github.com/edenhill/librdkafka.git ./librdkafka
cd ./librdkafka
./configure
# Or, to automatically install dependencies using the system's package manager:
# ./configure --install-deps
# Or, build dependencies from source:
# ./configure --install-deps --source-deps-only
make
sudo make install

Using librdkafka

Examples are included in librdkafka. Under the CPP directory, the C + + version includes two CPP files: consumer.cpp and producer.cpp, that is, producer and consumer. Modify itbrokersVariable sumtopic_strThe value of the variable.

Lines 58-63 of the consumer.cpp file are modified as follows (only lines 59 and 61 have been modified, and several more lines have been pasted to make it more intuitive):

//The broker list can be separated by commas and only one of them can be written. This is the offline test environment
    std::string brokers = "10.159.1.40:19092,10.159.1.41:19092,10.159.1.42:19092";
    std::string errstr;
    std::string topic_str="test_mq";
    std::vector<std::string> topics;
    topics.push_back(topic_str);

Lines 31-34 of the producer.cpp file are modified as follows (only 32 lines and 34 lines are modified, and several more lines are pasted to make it more intuitive):

//The broker list can be separated by commas and only one of them can be written. This is the offline test environment
    std::string brokers = "10.159.1.40:19092,10.159.1.41:19092,10.159.1.42:19092";
    std::string errstr;
    std::string topic_str="test_mq";

Compiling consumer.cpp and producer.cpp

$ cd ~/librdkafka/examples/cpp
$ g++ -o consumer consumer.cpp -lrdkafka++ -lrdkafka -lstdc++ -lbaas_c_style_interface -I~/librdkafka/include -L~/librdkafka/lib

$ g++ -o producer producer.cpp -lrdkafka++ -lrdkafka -lstdc++ -lbaas_c_style_interface -I~/librdkafka/include -L~/librdkafka/lib

Run producer and consumer (a console interactive program that requires two windows)

$ cd ~/librdkafka/examples/cpp

$ ./producer
% Created producer rdkafka#producer-1
hello world
% Produced message (11 bytes)

$ ./consumer
% Created consumer rdkafka#consumer-1
Read msg at offset 7
hello world

Installation and use of librdkafka