Golang’s GC recovery mechanism

Time:2022-5-9

Golang’s GC recovery mechanism

Conditions triggered by GC

  • Threshold: double the default memory size and start GC

  • Periodic: default2minTrigger GC once,src/runtime/proc.go:forcegcperiod

  • Manual:runtime.gc()

v1. 3 version mark removal method

The first step is to find out the unreachable objects and mark them.

The second part is to recycle unmarked objects.

Disadvantages: STW (stop the world) will be performed during marking

Stop the world

  1. Set gcwaiting = 1. This state will be checked before each g task. If yes, the current M will sleep;

  2. If a G task is running for a long time in this m, what should I do? Will I wait for the G task to switch by itself? In this case, you have to wait 10ms. You can’t wait! Never wait! Therefore, it will actively issue the preemption flag (similar to the previous article) to interrupt the current g task, and then run the next g task, which will go to step 1

  3. Keep waiting for all m to enter sleep, and all business logic codes stop at this time

v1. Version 5 three color marking method

  1. Start STW

  2. The default color for newly created objects is white

  3. GC recycle traverses all objects from the root node at once, and puts the traversed objects from the white set into the gray set.

  4. Loop through the gray set, put the objects referenced by the gray object from the white set into the gray set, and then put the gray object into the black set until there are no objects in the gray

  5. Stop STW

  6. Reclaim all objects of the white label table That is, recycle objects that are not referenced (garbage).

Optimized points:There are only black and white objects left in the golang three color marking method. The black object is the object used after the program is restored. If you don’t touch the black object and only clear the white object, it will certainly not affect the program logic. So:Cleanup operations and user logic can be concurrent.

v1. Version 8 hybrid write barrier

Write barrier

Compared with the subsequent write operations, the previous write operations of the barrier are first perceived by other components of the system. In popular terms: in the process of GC running, you can monitor the memory modification of the object and re mark the object. (in fact, it is also a super short STW, and then mark the object)

Summary:Whether the object is created or the reference of the object changes, it will be grayed out first

step

  1. All memory objects in the scan stack are marked black, and newly added objects are marked black

  2. Scan all objects in the heap and put the traversed objects from the white set into the gray set.

  3. Loop through the gray set, put the objects referenced by the gray object into the gray set from the white set, and then put the gray object into the black set until there is no object in the gray, mark the re referenced object as gray, and mark the added object as gray.

  4. Reclaim all objects of the white label table That is, recycling garbage.

Optimized points:Tag operations and user logic are also concurrent. User logic will often generate objects or change references to objects. How can tag and user logic be concurrent? Because the user logic will create new objects and change object references, soWrite barrier mechanismWill turn new objects gray first.

Recommended Today

Hudi origin analysis — deepnova developer community

1. Overview The official introduction of Hudi (Hadoop update delete incremental) is a data Lake framework that provides transaction support, row level update / deletes and change stream on the data lake. It was first developed by Uber in 2016, entered the open source community in 2017 and became the top project of Apache in […]