Tag:data

  • Redisobject of redis storage

    Time:2021-3-8

    redisObject Redis iskey-valueStorage system, wherekeyThe type is generallycharacter string, andvalueThe type is redis object(redisObject)。 Redis objects can bind various types of data, such as string, list and set. Therefore, it can well separate attributes from data. typedef struct redisObject { //Just right, 32 bits //Object type, string / list / set / hash table unsigned […]

  • Three step analysis of Axios source code

    Time:2021-3-8

    Pay attention to the official account of “kite holders” and reply to “Red Book”Get“JavaScript advanced program Fourth Edition (PDF)”And a large number of front-end learning materials. 1、 Comprehending thoughts Axios is an HTTP Library Based on promise. According to the official website, it has the following features: Xmlhttprequests will be created on the browser side […]

  • Kafka adventure – Introduction to architecture

    Time:2021-3-8

    For this Kafka project, I will focus on the overall architecture of the system, design and code implementation. With you bar source code, learn skills, knowledge. I hope you continue to pay attention and witness growth together!I believe: the road of technology, ten years as one day! Ten years to sharpen a sword! brief introduction […]

  • Spark optimization routine

    Time:2021-3-8

    The whole API of spark is relatively simple to use, and the expression of scala is relatively smooth. Java can understand several features of scala according to the official documents of spark. Optimization of spark offline processing Objective to reduce invalid operations, effectively utilize IO CPU, and directly check optimization skills, it is better to […]

  • Springboot integrates kafaka

    Time:2021-3-7

    package com.aicrs.engine.kafaka; import com.aicrs.engine.common.LogInfo; import com.aicrs.engine.common.RedisNameEnum; import com.aicrs.engine.component.DataSourceSwitchCompont; import com.aicrs.engine.constant.SwitchConstant; import com.aicrs.engine.entity.DetailLogEntity; import com.aicrs.engine.entity.QueryLogEntity; import com.aicrs.engine.kafaka.Singleton.DetailLogSingletonFactory; import com.aicrs.engine.kafaka.Singleton.QueryLogSingletonFactory; import com.aicrs.engine.mapper.DetailLogEntityMapper; import com.aicrs.engine.mapper.QueryLogEntityMapper; import com.aicrs.engine.utils.DateUtil; import org.apache.kafka.clients.consumer.ConsumerRecord; import org.apache.kafka.clients.consumer.ConsumerRecords; import org.apache.kafka.clients.consumer.KafkaConsumer; import org.slf4j.Logger; import org.slf4j.LoggerFactory; import org.springframework.beans.factory.annotation.Autowired; import org.springframework.data.redis.core.RedisTemplate; import org.springframework.scheduling.annotation.Async; import org.springframework.stereotype.Component; import org.springframework.transaction.annotation.Transactional; import org.springframework.util.CollectionUtils; import javax.annotation.PostConstruct; import java.util.*; import java.util.concurrent.BlockingQueue; @Component public class […]

  • With 80000 network nodes and 100 million online data, baidu super chain breaks the “black box” of offline advertising effect for the first time

    Time:2021-3-7

    The standardization and intelligent improvement of traditional offline scene marketing has become an important topic in the advertising industry. After the advertiser has launched the offline advertisement, how to determine whether the advertisement is played, and whether the frequency and position of the play are correct? The traditional solution is “relying on people”,That is to […]

  • Jpg learning notes 2 (with complete code)

    Time:2021-3-6

    We have got the data that need to compress RGB from BMP. We need to transform the original data from RGB domain to YCbCr domain, and then down sample the YCbCr data. For students who don’t need to read the article, the source code is given here directly.https://github.com/Cheemion/JPEG_COMPRESS Image reference “compressed image file formats, JPEG, […]

  • Flink learning — case practice of eventtime and watermarks

    Time:2021-3-6

    The article is reproduced from:https://blog.csdn.net/xu47043…Author: bigdata 1024Encroachment and deletion There are two ways to generate watermarks1: With periodic watermarks: periodically trigger the generation and sending of watermarks.2: With tapped watermarks: trigger the generation and sending of watermarks based on certain events. The first method is commonly used, so here we use the first method for […]

  • Summary after reading draveness’s introduction of Chan

    Time:2021-3-6

    readDraveness introduction to ChanSummary and questions after Chan realizes the effect of sharing memory through communication proposed by go Chan is essentially a locked queue for communication and synchronization When there is no buffer, it is synchronous Receiving and sending of Chan Recvq is a blocked receiver queue, that is, in x = < – […]

  • What if the master and slave databases of MySQL are not synchronized?

    Time:2021-3-6

    problem Today, we found that the master-slave database of MySQL is not synchronized Go to the master library first: mysql>show processlist; Check whether the next process has too many sleep. It’s normal. show master status; It’s normal, too. mysql> show master status; +——————-+———-+————–+——————————-+ | File | Position | Binlog_Do_DB | Binlog_Ignore_DB | +——————-+———-+————–+——————————-+ | mysqld-bin.000001 […]

  • First draft of data gate of the whole stack completed

    Time:2021-3-6

    The door of the whole stack data (temporary title) is the accumulation of working skills in the last 5 years. The first official account was written 8 months ago, and I didn’t know how many lonely nights I had spent. Writing an article is a hard work, and writing a book requires more. The original […]

  • High frequency test site for data warehouse interview

    Time:2021-3-6

    This article starts with the official account: five minutes big data. Causes of small files Small files in hive must be generated when importing data into hive table, so let’s first look at several ways to import data into hive Insert data directly into a table insert into table A values (1,’zhangsan’,88),(2,’lisi’,61); This method will […]