Tag:data stream

  • Official publicity | Apache Flink 1.14 0 post announcement

    Time:2021-12-27

    In the recent annual report released by the Apache Software Foundation, Apache Flink once again ranked among the top 5 most active projects! The latest release of the project is 1.14 Version 0 also reflects its extraordinary activity, including more than 1000 contributions from more than 200 contributors. We are proud that the whole community […]

  • [simplest implementation] Python implementation of median leetcode algorithm in computing data stream

    Time:2021-12-24

    1、 Problem Description: medianIs the number in the middle of a sequence table.If the list length is even, the median is the average of the middle two numbers. For example,The median of [2,3,4] is 3The median of [2,3] is (2 + 3) / 2 = 2.5 Design a data structure that supports the following two […]

  • Power Bi dataflow data flow (1)

    Time:2021-12-17

    About data flow What is it?: similar to online cloud power query, data is stored in azure data Lake Gen2(https://cloud.tencent.com/developer/news/341871Zorro’s introduction) 2. Benefits: 1) Reusable, using the same PQ table in multiple PBI reports 2) Prevent users from accessing the basic data source and reduce the load on the underlying system 3) Users create reports […]

  • How does Flink support batch streaming

    Time:2021-12-15

    There are many batch processing technologies, from SQL processing of various relational databases to MapReduce, hive, spark and so on in the field of big data. These are classic ways to deal with finite data streams. Flink focuses on infinite stream processing, so how does he do batch processing?Infinite stream processing: input data has no […]

  • Apache Flink fault tolerance mechanism

    Time:2021-12-9

    Original address:flink-release-1.2 Data Streaming Fault Tolerance Introduce Apache Flink provides a fault-tolerant mechanism that can recover the data flow and apply it to a consistent state. Ensure that in case of failure, each record of the program will only act on the state once (exactly once), of course, it can also be degraded to at […]

  • What should I pay attention to in the interview of big data development technology

    Time:2021-12-8

    Big data architecture and development As the name suggests, big data is an industry with data as the core. The big data industry can be divided into these parts from the transmission and evolution of data life cycle: data collection, data storage, data modeling, data analysis and data realization. Collect data through various channels, and […]

  • Getting started with Flink

    Time:2021-12-6

    About Flink Apache Flink is an open-source computing platform for distributed data stream processing and batch data processing. It provides functions to support two types of applications: stream processing and batch processing. Apache Flink, formerly a research project of Berlin Polytechnic University, was accepted by Apache incubator in 2014, and then quickly became one of […]

  • Learning Flink from 0 to 1 – Introduction to Apache Flink

    Time:2021-12-2

    preface Flink is a streaming computing framework. Why did I come into contact with Flink? At present, I am responsible for the alarm part of the monitoring platform. The collected monitoring data will be directly inserted into Kafka. Then, on the alarm side, I need to read the monitoring data in real time from Kafka […]

  • C # common method expansion and packaging records

    Time:2021-12-2

    1. Object to byte [] data /// ///Convert object to byte number /// ///Need to convert objects /// public static byte[] ConvertToBytes(this object obj) { using (var stream = new MemoryStream()) { var formatter=new BinaryFormatter(); formatter.Serialize(stream,obj); return stream.GetBuffer(); } } 2. Object to stream type /// ///Object to data flow /// ///Objects to convert /// […]

  • STM32 DMA details

    Time:2021-11-30

    This article is translated and sorted out according to the user manual of stm32f207 1. Overview DMA (direct memory access) direct memory access is used for high-speed data transmission between memory and memory or between memory and peripherals. Data transmission can move quickly without CPU intervention, which can maintain CPU resources to deal with other […]

  • Learning Flink from 0 to 1 – Introduction to data source

    Time:2021-11-28

    <!–more–> preface What is data sources? Literally, you can know: data source. As a streaming computing framework, Flink can be used for batch processing, that is, processing static data sets and historical data sets; It can also be used for stream processing, that is, to process some real-time data streams in real time and generate […]

  • Send binary data stream access request using HTTP request of idea

    Time:2021-11-6

    In the process of docking with hardware, we hope to simulate some binary data communicated with hardware through idea, such as: byte[] input = new byte[]{ //Type number 0x00, //Repeater number (byte) 0xff, (byte) 0xff, //Monitor category 0 0x12, //Monitor number (byte) 0xab, (byte) 0xff, (byte) 0xcb }; The HTTP request provided by idea can […]