Sentinel go source code series (III) engineering implementation of sliding time window algorithm

Time:2022-6-28

As for the most important ability of engineers, I think engineering ability should rank first.

Even now, big companies often need to use the hand tearing algorithm for interviews, which is more inclined to examine the ability of code engineering implementation. I thought it was outrageous to see such pictures in the group before.

image

Algorithm and engineering implementation

In sentinel go, a very core algorithm is the flow control (current limiting) algorithm.

Everyone may have heard of flow control, but it is still difficult to write one by hand. Why is flow control algorithm difficult to write? In my opinion, there are some differences between the algorithm and the engineering implementation. Although the algorithm is easy to understand, it can not be implemented according to it.

For example, the token bucket algorithm is very easy to understand. You only need to give a bucket, put tokens into the bucket at a constant rate, and discard them when they are full. Go to the bucket to get the token before executing the task. Only when you get the token can you execute it, otherwise it will be rejected.

If the token bucket is implemented, it is reasonable to use a separate thread (or process) to put tokens into the bucket, and the business thread to fetch them from the bucket. However, if this is really implemented, how can we ensure that the single thread can execute stably? Isn’t it very dangerous in case it hangs?

Therefore, there must be some differences between the project implementation and the original algorithm, which is one of the reasons why we need to go deep into the source code.

Evolution of sliding time window

Generally speaking, the measurement of flow control is the number of requests per second, that is, QPS

Qps:query per second refers to the number of queries per second. Of course, its meaning has been generalized. It does not specifically refer to queries, but to all requests. If it is necessary to distinguish, TPS refers to the number of transactions per second, that is, the number of writes, or RPS, the number of requests per second. The statistics are called QPS.

Of course, it is also measured by the number of concurrency. The flow control of concurrency is very simple

Concurrent data flow control

Concurrency is an instantaneous concept, which has nothing to do with time. Like the number of threads and coroutines in the process, you can only get an instant snapshot each time, but it may change soon.

How to define the concurrency number? It can be approximated that entering the business code is a concurrency, and the concurrency disappears after the execution.

image

In this way, the implementation is very simple. You only need to define a global variable. At the beginning of the responsibility chain, increase the atomic number of this variable by 1, and obtain a snapshot of the current concurrency number. Judge whether the concurrency number exceeds the limit. If it exceeds the limit, block it directly. Don’t forget to reduce the atomic number by 1 after the execution. Because it is too simple, you don’t need to release the code.

Fixed time window

Referring to concurrent data flow control, when QPS needs to be measured, can such an idea also be used?

Since QPS has time measurement, the first intuition is to get a variable like the concurrency number, and then start a separate thread to reset this variable every 1s.

However, individual threads are always uneasy and need to be changed slightly.

If the system has a start time, the current time is obtained at each request. The difference between the two can calculate the current time window. This time window can be counted separately.

image

If you think a little, you will find that the problem is not simple. As shown in the following figure, there are only 60 requests between 10t and 20t, and only 80 requests between 20t and 30t. However, there may be 110 requests between 16t and 26t, which is likely to break the system.

image

Sliding time window

In order to solve the above problems, the engineer came up with a good idea: do not fix the time window, and calculate the window forward with the current time

image

But the question arises again. How can this be achieved?

Implementation of sliding time window project

In engineering implementation, the time can be divided into small sampling windows and the sampling windows can be cached for a period of time. In this way, when a request comes, you only need to take the sampling window for a period of time, and then sum it to get the total number of requests.

image

Implementation of sentinel go sliding time window

Forward code high energy warning~

Sentinel go is based onLeapArrayThe data structure of the sliding window implemented is as follows

type LeapArray struct {
	Bucketlengthinms uint32 // bucket size
	Samplecount uint32 // number of buckets
	Intervalinms uint32 // total window size
	Array *atomicbucketwraparray // bucket array
	Updatelock mutex // update lock
}

type AtomicBucketWrapArray struct {
	base unsafe. Pointer // the starting address of the array
	Length int // length cannot be changed
	Data [] *bucketwrap // real bucket data
}

type BucketWrap struct {
	Bucket start Uint64 // bucket start time
	Value atomic. Value // bucket data structure, such as metricbucket
}

type MetricBucket struct {
	Counter [base.metriceventtotal]int64 // count array, which can contain different types
	Minrt Int64 // min RT
	Maxconcurrency int32 // maximum concurrency
}

Let’s look at how to write indicators, for example, when the process passes normally

// ①
sn.AddCount(base.MetricEventPass, int64(count))

// ②
func (bla *BucketLeapArray) AddCount(event base.MetricEvent, count int64) {
	bla.addCountWithTime(util.CurrentTimeMillis(), event, count)
}

// ③
func (bla *BucketLeapArray) addCountWithTime(now uint64, event base.MetricEvent, count int64) {
	b := bla.currentBucketWithTime(now)
	if b == nil {
		return
	}
	b.Add(event, count)
}

// ④
func (mb *MetricBucket) Add(event base.MetricEvent, count int64) {
	if event >= base.MetricEventTotal || event < 0 {
		logging.Error(errors.Errorf("Unknown metric event: %v", event), "")
		return
	}
	if event == base.MetricEventRt {
		mb.AddRt(count)
		return
	}
	mb.addCount(event, count)
}

// ⑤
func (mb *MetricBucket) addCount(event base.MetricEvent, count int64) {
	atomic.AddInt64(&mb.counter[event], count)
}

Get the corresponding bucket and write the count of the corresponding event. The RT will be specially processed because there is a minimum RT to be processed.

Focus on how to get the corresponding bucket:

func (bla *BucketLeapArray) currentBucketWithTime(now uint64) *MetricBucket {
	//① Fetch bucket based on current time
	curBucket, err := bla.data.currentBucketOfTime(now, bla)
	...
	b, ok := mb.(*MetricBucket)
	if !ok {
		...
		return nil
	}
	return b
}

func (la *LeapArray) currentBucketOfTime(now uint64, bg BucketGenerator) (*BucketWrap, error) {
	...
	//② Calculate index = (now / bucketlengthinms)% leaparray array. length
	idx := la.calculateTimeIdx(now)
	//③ Calculate bucket start time = now - (now% bucketlengthinms)
	bucketStart := calculateStartTime(now, la.bucketLengthInMs)

	for { 
		old := la.array.get(idx)
		If old = = nil {// ④ not used, return directly
			newWrap := &BucketWrap{
				BucketStart: bucketStart,
				Value:       atomic.Value{},
			}
			newWrap.Value.Store(bg.NewEmptyBucket())
			if la.array.compareAndSet(idx, nil, newWrap) {
				return newWrap, nil
			} else {
				runtime.Gosched()
			}
		} else if bucketStart == atomic. Loaduint64 (&old.bucketstart) {// ⑤ the current bucket is just retrieved and returned
			return old, nil
		} else if bucketStart > atomic. Loaduint64 (&old.bucketstart) {// ⑥ the old bucket is retrieved and reset for use
			if la.updateLock.TryLock() {
				old = bg.ResetBucketTo(old, bucketStart)
				la.updateLock.Unlock()
				return old, nil
			} else {
				runtime.Gosched()
			}
		} else if bucketStart < atomic. Loaduint64 (&old.bucketstart) {// ⑦ when a bucket that is newer than the current one is retrieved and there is only one bucket in total, this may occur in the concurrency situation. Otherwise, an error is reported directly
			if la.sampleCount == 1 {
				return old, nil
			}
			
			return nil, errors.New(fmt.Sprintf("Provided time timeMillis=%d is already behind old.BucketStart=%d.", bucketStart, old.BucketStart))
		}
	}
}

Take an intuitive example to see how to get a bucket:

image

  • Assuming that the B2 fetch is nil, a new bucket is written through the compareandset to ensure thread safety. If other threads write first, the execution will fail and the runtime Gosched(), give up the time slice and enter the next cycle
  • Assuming that the start time of B2 extraction is 3400, which is the same as the calculation, it is directly used
  • Assuming that the start time of the extracted B2 is less than 3400, it indicates that the bucket is too old and needs to be overwritten. Use the update lock to update to ensure thread safety. If the lock cannot be obtained, give up the time slice and enter the next cycle
  • If the start time of B2 fetching is greater than 3400, it indicates that other threads have been updated, and bucketlengthinms is usually much greater than the time of lock fetching. Therefore, only one bucket is considered here, and an error is reported in other cases

Return to QPS calculation:

qps := stat.InboundNode().GetQPS(base.MetricEventPass)

This method first calculates a starting time range

func (m *SlidingWindowMetric) getBucketStartRange(timeMs uint64) (start, end uint64) {
	curBucketStartTime := calculateStartTime(timeMs, m.real.BucketLengthInMs())
	end = curBucketStartTime
	start = end - uint64(m.intervalInMs) + uint64(m.real.BucketLengthInMs())
	return
}

For example, if the current time is 3500, the

  • end = 3400
  • start = 3400 – 1200 + 200 = 2400

image

Then traverse all buckets, take out all buckets within this range, and calculate QPS. You only need to add them.

last

This section evolves from the engineering implementation of sliding window flow control algorithm to the implementation of sliding window in sentinel go. From the implementation of sentinel go, we have to consider the use of memory, concurrency control, etc. if it is completely written, it is still very difficult.

Sentinel go source code series has written three articles, which only introduces two knowledge points: responsibility chain mode, sliding window flow restriction, and object pool. However, this is not very related to sentinel go. It will be written separately at that time and will not be included in this series.

This article is an end, not so much an end as a beginning.


Search and pay attention to the wechat official account “bug catching master”, back-end technology sharing, architecture design, performance optimization, source code reading, problem troubleshooting, and stepping on the pit practice.
image