Learning Flink from 0 to 1 — an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

Time:2021-11-30

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

<!– more –>

preparation

1. Check the version number of Java after installation. Java 8 is recommended.

Install Flink

2. It is very convenient to install Flink on Mac OS X. It is recommended to install through homebrew.

brew install apache-flink

3. Check installation:

flink --version

result:

Version: 1.6.0, Commit ID: ff472b4

4. Start flick

[email protected]  /usr/local/Cellar/apache-flink/1.6.0/libexec/bin  ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host zhisheng.
Starting taskexecutor daemon on host zhisheng.

Then you can enter the web page(http://localhost: 8081 /) view

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

demo

1. Create a maven project

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

Create a sockettextstreamwordcount file and add the following code:

package com.zhisheng.flink;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.util.Collector;

/**
 * Created by zhisheng_tian on 2018/9/18
 */
public class SocketTextStreamWordCount {
    public static void main(String[] args) throws Exception {
        //Parameter check
        if (args.length != 2) {
            System.err.println("USAGE:\nSocketTextStreamWordCount <hostname> <port>");
            return;
        }

        String hostname = args[0];
        Integer port = Integer.parseInt(args[1]);


        // set up the streaming execution environment
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        //Get data
        DataStreamSource<String> stream = env.socketTextStream(hostname, port);

        //Count
        SingleOutputStreamOperator<Tuple2<String, Integer>> sum = stream.flatMap(new LineSplitter())
                .keyBy(0)
                .sum(1);

        sum.print();

        env.execute("Java WordCount from SocketTextStream Example");
    }

    public static final class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> {
        @Override
        public void flatMap(String s, Collector<Tuple2<String, Integer>> collector) {
            String[] tokens = s.toLowerCase().split("\\W+");

            for (String token: tokens) {
                if (token.length() > 0) {
                    collector.collect(new Tuple2<String, Integer>(token, 1));
                }
            }
        }
    }
}

Then enter the project directory and package with the following command.

mvn clean package -Dmaven.test.skip=true

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

Then we open the listening port:

nc -l 9000  

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

Finally, enter the Flink installation directory bin and execute the following command to run the program:

flink run -c com.zhisheng.flink.SocketTextStreamWordCount /Users/zhisheng/IdeaProjects/flink/word-count/target/original-word-count-1.0-SNAPSHOT.jar 127.0.0.1 9000

Pay attention to changing the path to your own project.

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

After executing the above command, we can see the running program in the webui:

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

We can enter text in the NC listening port, for example:

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

Then we use the tail command to look at the output log file to observe the statistical results. Enter the directory Apache Flex / 1.6.0/libexec/log and execute the following commands:

tail -f flink-zhisheng-taskexecutor-0-zhisheng.out

Note: switch to your own path and view your own directory.

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

summary

This article describes how to install Flink on a Mac and run it. Then it introduces how to build and run Flink program through a simple Flink program.

Pay attention to me

Please indicate the address for Reprint:http://www.54tianzhisheng.cn/2018/09/18/flink-install

In addition, I have compiled some Flink learning materials, and I have put all the official account of WeChat. You can add my wechat: Zhisheng_ Tian, and then reply to the keyword: Flink, you can get it unconditionally.

Learning Flink from 0 to 1 -- an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

Related articles

1、Learning Flink from 0 to 1 – Introduction to Apache Flink

2、Learning Flink from 0 to 1 — an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

3、Learn Flink from 0 to 1 – detailed explanation of Flink profile

4、Learning Flink from 0 to 1 – Introduction to data source

5、Learn Flink from 0 to 1 – how to customize the data source?

6、Learning Flink from 0 to 1 – Introduction to data sink

7、Learn Flink from 0 to 1 – how to customize data sink?