Tag：partition
-
Time：2020-4-6
Ckafka helps Tencent classroom to realize stable interaction of millions of messages Introduction: during the epidemic period, in order to ensure the normal learning progress of domestic students, Tencent classroom actively responded to the call of the state to “stop work and keep learning”, and launched a special “teacher’s fast version” during the epidemic period […]
-
Time：2020-4-5
1. Kafka profile After unpacking the Kafka installation package, in the conf directory, server.properties #The global unique number of the broker, which cannot be repeated in the Kafka cluster, is an integer number broker.id=0 #Enable the delete topic function delete.topic.enable=true #Number of threads processing network requests num.network.threads=3 #Off the shelf quantity used to process disk […]
-
Time：2020-4-5
Setting up a guest account is not enough if you want to prevent your relative’s bear children from messing with your computer. Although it can protect your system disk, it has no protection for other hard disk partitions. At this time, the partition encryption function can be used. It is very simple to turn on […]
-
Time：2020-4-1
1、 Storage file structure topic: can be understood as the name of a message queue partitionTo achieve scalability, a very large topic can be distributed to multiple brokers (i.e. servers), and a topic can be divided into multiple partitions, each of which is an orderly queue segment: partition is physically composed of multiple segments message: […]
-
Time：2020-3-31
This article is from oppo Internet technology team, the third in a series of articles on “analyzing spark data partition“. In this article, we will analyze the data partition in spark streaming, tispark. Series 1: analyzing Hadoop partition of spark data partition Series 2: analyzing spark RDD partition of spark data partition Series 3: analyzing […]
-
Time：2020-3-31
Kafka Series 1: overview of Kafka Kafka is one of the most popular message middleware in the current distributed system. With its high throughput design, Kafka is loved by developers in the application scenarios of log collection system and message system.This article talks about some knowledge about Kafka. It mainly includes the following contents: Kafka […]
-
Time：2020-3-30
After VMware sets the disk size of the virtual machine, it finds that the disk space is insufficient. How to expand the disk size at this time? First of all, when the virtual machine is shut down, right-click the settings, select the hard disk, and expand, so that the size of the disk can be […]
-
Time：2020-3-29
Computer operating system operating system Concurrency refers to the occurrence of multiple events in a period of time. Parallelism refers to the occurrence of multiple events at a certain time. Therefore, parallelism requires multiprocessors The introduction of virtual storage technology enables an application to run in memory space far smaller than it In order to […]
-
Time：2020-3-27
1. overview Apache Kafka is a fast, scalable, high throughput, fault-tolerant distributed publish subscribe message system. It has the characteristics of high throughput, built-in partition, supporting data copy and fault tolerance, and is suitable for large-scale message processing scenarios. The author previously worked in the Internet of things company, where Kafka is the factual standard […]
-
Time：2020-3-26
How to use combinebykey of RDD This is a very abstract method, at first, I was confused. But most aggregation functions are implemented based on this method, such as the commonly used reducebykey, so this method is very important. Method parameter def combineByKey[C]( //Called when the key first encountered in a given partition (in an […]
-
Time：2020-3-24
Controller is used to coordinate and manage Kafka clusters. In a Kafka cluster, only one broker can be selected as controller. When the broker where the controller is down, other brokers can compete to be selected as controller. State management Controller is used for state management, which is mainly used to manage the partition replica […]
-
Time：2020-3-23
Oracle partition 1、 Storage space occupied by query table Each table is stored as a “segment” and its corresponding information can be viewed through the user & segments view.Definition of segments: if a heap organization table is created, it is a segment. Select segment’name as tablename, bytes from user’segments where segment’name = ‘table name’; Interpretation:Segment […]
