• Tikv source code analysis series — placement driver


    This series of articles mainly for tikv community developers, focusing on tikv system architecture, source structure, process analysis. The purpose is to enable developers to have a preliminary understanding of the tikv project after reading, and better participate in the development of tikv. Tikv is a distributed kV system, which uses raft protocol to ensure […]

  • Flink — running architecture


    Task submission process Flink submits the task, and the client uploads Flink jar package and configuration to HDFS Then submit the task to the Yan ResourceManager The ResourceManager allocates the container resources and notifies the corresponding nodemanager to start the applicationmaster. After the applicationmaster starts, it loads Flink jar package and configuration to build the […]

  • [summary] analysis and summary


    Golang Analysis of map principle sync.Map Principle analysis sync.Pool Principle analysis sync.Mutex Principle analysis sync.WaitGroup Principle analysis Analysis of slice principle Interface analysis Analysis of channel principle Analysis of select principle Analysis of defer principle Analysis of panic principle Details of GMP scheduling Memory allocation details Detailed explanation of GC time.Timer Principle analysis time.Ticker Principle […]

  • SYS.BASE -Operating system notes (Theory)


    operating system: provides basic abstraction of resource set to application program, controls the allocation of processor, memory and other I / O interface devices (Resource Management: time reuse, space reuse) among competing programs. Computer hardware: CPU: In each CPU base cycle, instructions are fetched from memory decoded (to determine its type and operation) – executed, […]

  • EB level system air change engine: how to upgrade Ali dispatching execution framework?


    Introduction:As Alibaba’s core big data base, Fuxi scheduling and distributed execution system supports most of the big data computing needs of the big data platform within Alibaba group and on Alibaba cloud. Various computing engines such as maxcompute (ODPs) and Pai run on it to perform massive data operations for users every day. In order […]

  • Process management of a quick start operating system


    catalog 1. What is process 2. Process model 2.1 PCB 2.2 process status 2.2.1 five state model 2.2.2 seven state model 2.2.3 process switching 2.3 process organization 2.3.1 linear table 2.3.2 linked list 2.3.3 index 3. Thread 3.1 thread structure 3.2 thread status 4. Process scheduling 4.1 several dispatching modes 4.1.1 long distance dispatching 4.1.2 […]

  • Process Basics


    (1) Process and thread(1) A process is an execution process of a program in the operating system. An independent unit of the system for resource allocation and scheduling.(2) Thread is an executive entity of a process. It is the basic unit of CPU scheduling and allocation. It is smaller than the process and can run […]

  • (3) Easy to understand how MapReduce works


    I wrote an article earlier\(2) analysis of Hadoop’s MapReduce principleLater, it seems that I read it several times. Even I think it is too boring. Even the author himself has a hard time reading it, which shows that the writing is not good. =Therefore, we decided to simplify the work of MapReduce on the basis […]

  • Introduction to operating system (2)


    Scheduling of processes Considerations In order to build a scheduling strategy, we need to do someSimplify assumptionsThese assumptions and theProcess dependent, collectively referred to asWorkload(workload) (1) Every process (work)Run the same time (2) All workArrive at the same timeSometimes, when the time difference between the arrival time of multiple jobs is very small, it is […]

  • The principle of golang concurrency


    Mpg model There are four important structures in go scheduler: m, P, G, schedMrefer toMachine, aMIt is directly associated with a kernel thread. Managed by the operating system.P“Processor” means “processor”MThe required context is also the processor that handles user level code logic. It is responsible for bridging the scheduling context of M and G, and […]

  • Best practice of spring cloud application on kubernetes — high availability (elastic scaling)


    The author preface Elastic scaling is an important application management strategy to meet business requirements, ensure service quality and balance service cost. Elastic scaling enables the deployment scale of the application to be dynamically adjusted according to the real-time business volume, expanding the deployment scale in the business peak period to ensure that the service […]

  • Fuxi2.0 – Feitian big data platform scheduling system upgrade, debut in 2019 double 11


    Fuxi is one of the three major services (distributed storage Pangu, distributed computing ODPs, distributed scheduling Fuxi) when the flying platform was founded ten years ago. The original intention of the design at that time was to solve the scheduling problem of large-scale distributed resources (essentially, the optimal matching problem of multi-objective). With the continuous […]