• [big data practice] Kafka producer programming (5) — detailed explanation of producer config (2)


    preface In the last article [big data practice] Kafka producer programming (4) – producer config detailed explanation (Part I), the relevant configuration items of Kafka producer were introduced. In this article, we will continue to introduce the remaining configuration items. Producerconfig class buffer.memory Importance: highType: longDefault value: 33554432 bytes, i.e. 32m Producer can be used […]

  • Python 3 standard library: pickling object serialization


    1. Pickle object serialization The pickle module implements an algorithm to convert any Python object into a series of bytes. This process is also known as serializing objects. You can transfer or store a stream of bytes representing an object and then reconstruct it to create a new object with the same properties. 1.1 encoding […]

  • Rabbitmq series essays — Introduction and installation


    1、 Introduction to rabbitmq Rabbitmq is an open source implementation of AMQP (Advanced message queuing protocol) developed by Erlang. It is an advanced message queuing protocol, an open standard of application layer protocol, designed for message oriented intermediation. Message oriented middleware is mainly used to decouple components. The sender of message does not need to […]

  • SQL Server wait event pageiolatch_ EX


    What is?PAGEIOLATCH_EXWaiting for events? Now we’re going toPAGEIOLATCH_EXWait for the relevant information of the event to do a simple summary, collation. aboutPAGEIOLATCH_EXThe official documents are as follows:   PAGEIOLATCH_EX:   Occurs when a task is waiting on a latch for a buffer that is in an I/O request. The latch request is in Exclusive mode. […]

  • Summary of cout and CIN usage


    cout coutFor displaying messages on the screen, it should be short for console output. It isC++inostreamObject, which is encapsulated in the<iostream>Library, the name of the library definition is placed in the namespacestdMedium, socoutIs the full name ofstd::cout 。coutIs classified as a stream object, which means that the. To display a message on the screen, you […]

  • Redis replication process details


    The replication function of redis is divided into two steps: sync and command propagate Synchronization is used to update the database state of the slave server to the current database state of the master server. Command propagation is used to make the database of the master-slave server return to the consistent state when the database […]

  • Explore the zero copy technology in netty and Kafka!


    preface Literally, it means that the data does not need to be copied back and forth, which greatly improves the performance of the system. We often use this word in JavaNiO, netty, Kafka, rocketmq and other frameworks are often used as a highlight of their performance improvement. Next, we will start with several concepts of […]

  • Three cheap machines write 2 million per second! Why is Kafka so fast?


    Author: Bing Yue, profile: Ali senior engineer, focus on distributed system and high availability architecture, methodology and cognitive upgrading, practice and continuous learning. source: www.cnblogs.com/binyue/p/10308754.html A brief review of Kafka in 30 seconds Let’s take a quick look at Kafka and some details about how it works. Kafka is a distributed messaging system that was […]

  • Analysis of zero copy technology in Linux


    This paper discusses several main zero copy technologies in Linux and the applicable scenarios of zero copy technology. In order to quickly establish the concept of zero copy, we introduce a common scenario a citation When writing a server program (web server or file server), file download is a basic function. At this time, the […]

  • Shuffle of Hadoop


    Hadoop system performs partitioning, sorting, and passing map output as input to the reducershuffle。 Shuffle is the data processing process after the map method and before the reduce method. Map end Each maptask processes a slice, and the output is not simply written to disk. It will go through the following process, Each map task […]

  • Ultimate Guide: 12 tips for improving nginx server hardness


    Nginx is a lightweight web server / reverse proxy server and email (IMAP / POP3) proxy server, which is distributed under a BSD like protocol. Its characteristics are less memory and concurrent capability. In fact, nginx’s concurrent capability is better in the same type of web server. Mainland China uses nginx website users: Baidu, Jingdong, […]

  • IO and NiO in Java


    At the end of the paper, we will choose NiO and RPC related materials for further study, so this paper begins to learn NiO knowledge.  IOKnowledge review Before learning NiO, it is necessary to review some knowledge of Io. IOFlow in Java program completes input and output through stream. Flow is the abstraction of production […]