Tag：partition
-
Time：2021-3-1
Hello, I’m Liang Xu. Familiar with the computer’s small partners know, we often say the host three pieces are the motherboard, CPU, memory, visible memory in the role of the computer host is not the same, like DIY host of small partners to understand these three pieces is better. Small partners who have written programs […]
-
Time：2021-3-1
1. Five basic properties A list of partitions A function for computing each split A list of dependencies on other RDDs Optionally, a Partitioner for key-value RDDs (e.g. to say that the RDD is hash-partitioned) Optionally, a list of preferred locations to compute each split on (e.g. block locations for an HDFS file) This is […]
-
Time：2021-2-27
1. Shuffle mechanism Shuffle refers to the data processing after the map() method and before reduce(). It is to distribute the result data output by maptask to reducetask according to the partition rules, and partition and sort the data in the process of distribution. 2. Partition In MapReduce calculation, sometimes the final output data needs […]
-
Time：2021-2-26
MySQL partition Advantages of partitioning Store more data To optimize the query, only scan one or more necessary partitions. For count () and sum (), only summarize the partition statistics For data that is out of date or does not need to be saved, partitioning is faster Data queries are distributed across multiple disks to […]
-
Time：2021-2-26
MySQL partition Advantages of partitioning Store more data To optimize the query, only scan one or more necessary partitions. For count () and sum (), only summarize the partition statistics For data that is out of date or does not need to be saved, partitioning is faster Data queries are distributed across multiple disks to […]
-
Time：2021-2-26
Apache pulsar is the next generation distributed messaging system of Yahoo open source. In September 2018, it graduated from Apache Software Foundation and became a top-level project. Pulsar’s unique layered and fragmented architecture not only ensures the performance and throughput of the big data message flow system, but also provides high availability, high scalability and […]
-
Time：2021-2-24
Messages in Kafka are classified by topic, and each topic is logically independent of each other. Each topic can be divided into one or more partitions. The number of partitions can be specified when the topic is created or modified later. Each message will be added to the specified partition according to the partition rules […]
-
Time：2021-2-22
0. Preparation before reassembly And remember to back up important files on your computer. 1、 Production of PE system 1.1 PE selection First Baidu download a PE, such as youqitong (of course, micro PE is also OK). Note that youqitong must go under the IT sky, there are many fake places elsewhere.Address link:https://www.itsk.com/thread-414955-1-1.html In a […]
-
Time：2021-2-21
First of all, I wish you a happy new year, good health and all the best.Generally speaking, the first bottleneck of a system is probably the database. For example, the concurrency of our production system is very high. After running for a period of time, the data volume of some tables in the database will […]
-
Time：2021-2-21
Corresponding to the producer is the consumer. The application can subscribe to the topic through Kafka consumer and pull messages from the subscribed topic. However, before using kafkaconsumer to consume messages, we need to understand the concepts of consumers and consumption groups, otherwise we cannot understand how to use kafkaconsumer. Today, we will first explain […]
-
Time：2021-2-20
When a table is misoperated, such as deleting data, drop table, or truncate table, we can restore it by using flashback table, flashback drop, or tspitr. However, in the following scenarios, the above recovery technology will be helpless: The table logic is damaged or the purge option is used to delete; When Undo is not […]
-
Time：2021-2-20
Before installation, prepare a USB flash disk of at least 8g. We use the U disk to install. Download CentOS installation package visit:CentOS websiteClick on the image aboveCentOS Linux After clicking to jump to the webpage, as shown in the figure below, because the listing time is not long and the information is less, I […]
