• Accumulator of spark shared variables


    Let’s take a simple example. The requirement is: count the number of words. val data: RDD[String] = sc.makeRDD(Seq(“hadoop map reduce”, “spark mllib”)) // mode 1 val count1: Int = data.flatMap(line => line.split(” “)).map(word => 1).reduce(_ + _) println(count1) // mode 2 var acc = 0 data.flatMap(line => line.split(“”)).map(word => {acc += 1; word}) println(acc) Mode […]

  • C language review


    Usually when I start to learn C, the first sentence is “helloword” int main(){ printf(“hello word”); } In the following days, we need to pay attention to whether the input method is correct in both Chinese and English. This is the most basic, a byte is 8 bits Now, let’s take the keywords int, long, […]

  • Explain eslint rules in detail


    Explain eslint rules in detail I wanted to standardize my code style and reduce the probability of program error through tools a long time ago. If I read the self-cultivation of sublime Text3, a front-end program of mine, my friend of this blog would know that I used sublimelinter jshint at that time Plug in […]

  • Technical practice of hive migrating Presto


    This article is from oppo Internet technology team. If you need to reprint it, please indicate the source and author. Welcome to our official account: OPPO_tech At the beginning of hive design, it was positioned as an off-line warehouse product. Although hortonworks called out the “make Apache hive 100x fast” slogan, it also made a […]

  • RDD of distributed elastic data set helps spark improve operation efficiency


    As we have said in the introduction of spark, in order to speed up the processing of spark, one of the solutions is to introduce a distributed elastic data setRDD RDD concept What is RDD: RDD (resilient distributed Data set) elastic data set is the most basic data abstraction in spark. Although RDD is a […]

  • SQL starts from scratch — basic knowledge


    The full name of SQL is structured query language, / / the structured query language SQL is used to access and operate the database system. SQL statements are used to retrieve and update data in the database.SQL is not case sensitive. For specification, semicolons need to be added after each SQL statement.SQL can be divided […]

  • Challenge and practice of building CEP engine based on Flink


    How does qi’anxin group, as a network security company, build a CEP engine based on Flink to detect network attacks in real time? What are the challenges and valuable practical experiences? The main content of this paper is divided into the following four aspects: Background and current situation Technical framework Products and operation and maintenance […]

  • New features of es2020


    1、 Use the ා – sign to quickly create private variables of a class One of the main purposes of class is to reuse our code into more modules, but we don’t want to expose all the properties and methods. Now, with this new feature, we can easily implement this idea, just add a ා […]

  • Angular 2 AsyncPipe


    Update time – 2017-03-21 15:02 Today, let’s introduce asyncpipe in angular 2. With asyncpipe, we can use it directly in the templatePromiseandObservableObject instead of defining the member properties of a class to store the returned results. Asyncpipe subscribes to an observable or promise object and returns the latest value it issues. When a new value […]

  • Understanding and using of pointer in C language


    Author: heart leafTime: April 21, 2018 17:24 Definition A pointer is a variable whose value is the address of another variable. The address represents the location in memory. Remember that an array variable is itself a pointer. Type of address The address is of type, isn’t it strange? Isn’t the pointer representing an address? Is […]

  • Extended operators spread and rest parameters of “ES6 foot series”


    [ES6 foot series] ES6 reference link summary by Extended operators spread and rest parameters of “ES6 foot series” Deconstruction and assignment of [ES6 foot series] variables [ES6 foot series] module [ES6 foot series] arrow function [ES6 foot series] Set + weakset + map + weakmap [ES6 foot series] symbol [ES6 foot series] iterator [ES6 foot […]

  • CSAPP Experiment 1: bit operation


    CSAPP is known as the programmer’s Bible. Although it is translated into Chinese as “in-depth understanding of computer systems”, it is not so “deep”, but it has a wide coverage, and is generally used as a textbook for the introduction course of computer major. I’ve heard that the supporting experiments in the book are very […]