Elk multi tenant scheme


Elk multi tenant scheme

1、 Preface

Log analysis is one of the most important means of system debugging and problem checking. At present, there are many instances and machines in distributed system, so it is very necessary to build a unified log system. Elk provides a complete set of solutions, and they are all open source software, which can be used together perfectly and efficiently to meet the application of many occasions. Elk is the most important tool at present It’s one of the mainstream choices.

This paper mainly introduces how to realize a set ofElk log systemAt the same timeMultiple environmentsMultiple systemsCommon use / test, and realize mutual data and viewquarantineThey don’t affect each other.


2、 Isolation mode

commonELKThe architecture is shown in the figure below, which is composed ofElasticsearchLogstashKibanaAndFileBeatform.
Elk multi tenant scheme

Deploy one in each application serverFileBeatComponent as a log collector, through the input plug-in to get data from the file, and then transfer to theLogstashThe log data will be processed and structured through the filter plug-in and sent to theElasticsearchStorage, finally throughKibanaVisual display analysis.

PS: it is necessary toELKEach component of thequarantinehandle


2.1. Filebeat isolation

Because one is deployed on each machineBeatInstances are collected as logs, soFileBeatIt doesn’t need to do any isolation configuration, but as the data entry, it needs to put thetenantThe relevant information is transmitted to the downstream, as shown in the figure below

Elk multi tenant scheme

Through the project (project name) and env (environment) as thetenantIsolation sign


2.2. Logstash isolation

The main reason is that the log format of each project may be different, so there will be different personalized configuration files. ThisLog resolution profileIsolation rules need to be defined for separation;

Start with the following commandlogstashappointconfig/conf/Store the directory for the configuration and specify the hot load of the configuration file.

bin/logstash -f config/conf/ --config.reload.automatic


For the isolation method of log parsing configuration file, please refer to the following figure:

Elk multi tenant scheme


It is universalinputConfiguration, shared by each tenant, for receiving data from filebeat

input {
  beats {
    port => 5044



It is universaloutputConfiguration, shared by each tenant, used to store log data according to the definedIndex naming rulesCreate index and write to es

It needs to be added to the data sourceprojectenvanddocTypeThe three fields represent project name, environment and log type

output {
  elasticsearch {
    hosts => ["localhost"]
    user => "elastic"
    password => "changeme"
    index => "%{[fields][project]}-%{[fields][env]}-%{[fields][docType]}-%{+YYYY.MM.dd}"

IP, user name and password should be modified according to the actual situation



For personalizationLog analysisEach tenant creates a new configuration file to configure its ownfiltercontent

filter {
  if [fields][project] == "mp" and [fields][env] == "pre" and [fields][docType] == "syslog" {
    grok {

PS: must increaseifStatement to confirm whether it belongs to your own tenant’s log data!


2.3. Elasticsearch isolation

Through different index naming, create their own independent index to achieve physical isolationLogstashWhen the index is generated after structured data, it is automatically passedFilebeatThe specified index name is generated dynamically by the input parameter variable of.

The naming rule of index is ${project name} – ${environment} – ${log type} -%{+ YYYY.MM.dd }

For example: mp-pre-syslog-2020.12.01


2.4. Kibana isolation

Each tenant can create its own independent workspace to isolate its own index data, display views and other objects, andMutual invisibility

The configuration process of the workspace is as follows:

  1. Create workspace
  2. Create roles (configure permissions)
  3. Create user (associated role)

2.3.1 create workspace super administrator login

Use super administrator accountelasticLog in to kibana and selectDefault workspace
Elk multi tenant scheme enter the management page

Elk multi tenant scheme create workspace

Create a workspace and customize the function points to be displayed (all displayed by default)
Elk multi tenant scheme


2.3.2 create role binding workspace

Create a new role and assign the corresponding roleIndex permissionsAndWorkspace permissionsWait for permission to the role
Elk multi tenant scheme


2.3.3 creating users

Create users and bind yourselfworking spaceThe role of
Elk multi tenant scheme

PS: this user can only see his ownwork areaNextIndexesandinstrument panelAnd so on


3、 Summary

eachtenantNeed to be rightELKEach component of thequarantinehandle

  1. FilebeatResponsible for distinguishingtenantRelevant information is transmitted to the downstream
  2. Logstash: separate the personalization of each tenant independentlyFilterconfiguration file
  3. Elasticsearch: through standard index naming, each tenant creates index independently to realize physical isolation
  4. Kibana: the data and dashboard are not visible to each other through multi workspace isolation


PS: Although the isolation steps are a little cumbersome, you can develop a product log system by yourself in the later stage, and implement the above steps on the graphical interface.


Scan code, pay attention to surprise!

Elk multi tenant scheme