Elasticsearch optimization ideas notes


Summary from online content

Write part

  1. Batch writing is better than writing 1 piece of data each time.
  2. Set routing to a random value when writing each batch, and the batch data will be written to a shard. Reducing the number of shards written can reduce the number of nodes “harassed” each time and reduce the number of merges
  3. If the writing pressure is high, the segment merge can be executed regularly during the low peak period
  4. _ Source can be closed if there is no requirement to speed up writing
  5. When batch importing data, you can set the number of copies to 0, and then change it back after importing

Search section

  1. Too many segments will affect the search performance
  2. Through es/_ The cat/segments interface can obtain the segments information, timely count these information, and monitor the number of segments

Troubleshooting problems

  1. High CPU utilization, using ES/_ cat/hot_ The threads interface checks which threads occupy more CPU
  2. Open the GC log of ES. When the CPU or memory pressure is high, check whether GC is called a bottleneck from the GC log
  3. When writing is slow, check whether the iowait of the disk is high. The value of Wa of the top command

Today, I will write so much first. Later, I will test it in practice and continue to supplement it