• Data structure and underlying storage of index


    Index is a data structure to help database get data efficiently Data structure of index 1. Hash table a. Using hash storage, all data files need to be added to memory, which consumes memory spaceb. Hash table stores unordered data. When searching the range, it needs to traverse one by one, which is time-consuming. 2. […]

  • How to package a web project into an EXE file or install a package


    1. Download address of nw.js: https://nwjs.org.cn/     2. Create a new configuration file package.json and put it in the root directory {“Name”: “ddmsc”, / / the name attribute here is very important, similar to a unique identifier. No less!“Main”: “power Bi documents – power Bi Microsoft docs. HTM” / / the file to be […]

  • How is Kafka stored? Why so fast?


    Address:Java-Bang Focus on system architecture, high availability, high performance, high concurrency technology sharing Kafka relies on the file system (more fundamentally, disk) to store and cache messages. In our impression, the speed cognition of each storage medium is roughly the same as that shown in the figure below. The higher the level, the faster the […]

  • Raid Fundamentals


    Raid (redundant array of Independent Disks) is a common disk management technology for commercial servers. As a redundant array of disks, raid can provide an independent solution for large storage devices. Combining several physical disks into a larger virtual device can not only improve the capacity of the hard disk, but also fully improve the […]

  • 10 times performance improvement! Interpretation of DLA SQL data analysis acceleration function based on alluxio


    Introduction:In the scenario of separation of storage and computing, it is a costly operation to read data from remote storage through network, which often leads to performance loss. Taking OSS as an example, the data reading delay of OSS is usually much larger than that of local disk. At the same time, OSS limits the […]

  • The principle of transaction persistence in MySQL


    preface When it comes to database transactions, it’s easy for you to have a lot of knowledge about transactions in your mind, such as the acid characteristics of transactions, isolation levels, problems solved (dirty read, non repeatable read, magic read), etc. but few people really know how these characteristics of transactions are realized, and why […]

  • Spark performance tuning shuffle tuning and troubleshooting


    Shuffle tuning of Spark Tuning This section begins with an explanationShuffle core concepts; And then forHashShuffle、SortShuffleCarry out the optimization; Next, yesMap side、Reduce sideOptimization; And then for theData skewProblem analysis and optimization; Finally, the process of spark runningTroubleshooting。 This article starts with the official account.Learn big data in five minutes】, our company focuses on big data […]

  • Hello, Kafka


    Hello, I’m Kafka. Maybe many people have heard of me. I was born in LinkedIn in 2011. My functions have become more powerful since then. As an excellent and complete platform, you can store a huge amount of data redundantly on me. I have a message bus with high throughput (millions / s), on which […]

  • Analysis of MySQL redo and undo logs


    preface: The previous article described several common logs in MySQL system. In fact, there are transaction related logs, redo log and undo log. Compared with other logs, redo log and undo log are more mysterious and difficult to observe. This article will mainly introduce the functions and operation and maintenance methods of these two types […]

  • MySQL UPDATE statement execution process wal redolog binlog


    MySQL UPDATE statement execution process wal redolog binlog   WAL Full name: write ahead logging When a record needs to be updated, the InnoDB engine will first write the record to the redo log and update the memory. At this time, the update is complete. And update the operation record to disk at the appropriate […]

  • AIX performance management and monitoring suggestions (2)


    Transferred from the official account @twt community, author Chen Chihui 3 I / O monitoring 3.1 IO response time evaluation What kind of IO response time is reasonable? The following is a summary of some empirical rules: For the magnetic array with mechanical hard disk and without storage synchronous mirror, the empirical rule for evaluating […]

  • How to measure Linux performance and avoid the most typical error: CPU


    In this series, we will discuss Linux performance measurement and how to measure it correctly. Linux performance is a very broad topic, so we will focus on the four main resources that usually improve system performance — CPU, memory, disk storage and network. Now, when we talk about component related performance, many scenarios just need […]