Learning Flink from 0 to 1 — an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

<!– more –>

preparation

1. Check the version number of Java after installation. Java 8 is recommended.

Install Flink

2. It is very convenient to install Flink on Mac OS X. It is recommended to install through homebrew.

brew install apache-flink

3. Check installation:

flink --version

result:

Version: 1.6.0, Commit ID: ff472b4

4. Start flick

[email protected]  /usr/local/Cellar/apache-flink/1.6.0/libexec/bin  ./start-cluster.sh
Starting cluster.
Starting standalonesession daemon on host zhisheng.
Starting taskexecutor daemon on host zhisheng.

Then you can enter the web page（http://localhost: 8081 /) view

demo

1. Create a maven project

Create a sockettextstreamwordcount file and add the following code:

package com.zhisheng.flink;

import org.apache.flink.api.common.functions.FlatMapFunction;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.util.Collector;

/**
 * Created by zhisheng_tian on 2018/9/18
 */
public class SocketTextStreamWordCount {
    public static void main(String[] args) throws Exception {
        //Parameter check
        if (args.length != 2) {
            System.err.println("USAGE:\nSocketTextStreamWordCount <hostname> <port>");
            return;
        }

        String hostname = args[0];
        Integer port = Integer.parseInt(args[1]);


        // set up the streaming execution environment
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        //Get data
        DataStreamSource<String> stream = env.socketTextStream(hostname, port);

        //Count
        SingleOutputStreamOperator<Tuple2<String, Integer>> sum = stream.flatMap(new LineSplitter())
                .keyBy(0)
                .sum(1);

        sum.print();

        env.execute("Java WordCount from SocketTextStream Example");
    }

    public static final class LineSplitter implements FlatMapFunction<String, Tuple2<String, Integer>> {
        @Override
        public void flatMap(String s, Collector<Tuple2<String, Integer>> collector) {
            String[] tokens = s.toLowerCase().split("\\W+");

            for (String token: tokens) {
                if (token.length() > 0) {
                    collector.collect(new Tuple2<String, Integer>(token, 1));
                }
            }
        }
    }
}

Then enter the project directory and package with the following command.

mvn clean package -Dmaven.test.skip=true

Then we open the listening port:

nc -l 9000  

Finally, enter the Flink installation directory bin and execute the following command to run the program:

flink run -c com.zhisheng.flink.SocketTextStreamWordCount /Users/zhisheng/IdeaProjects/flink/word-count/target/original-word-count-1.0-SNAPSHOT.jar 127.0.0.1 9000

Pay attention to changing the path to your own project.

After executing the above command, we can see the running program in the webui:

We can enter text in the NC listening port, for example:

Then we use the tail command to look at the output log file to observe the statistical results. Enter the directory Apache Flex / 1.6.0/libexec/log and execute the following commands:

tail -f flink-zhisheng-taskexecutor-0-zhisheng.out

Note: switch to your own path and view your own directory.

summary

This article describes how to install Flink on a Mac and run it. Then it introduces how to build and run Flink program through a simple Flink program.

Pay attention to me

Please indicate the address for Reprint:http://www.54tianzhisheng.cn/2018/09/18/flink-install

In addition, I have compiled some Flink learning materials, and I have put all the official account of WeChat. You can add my wechat: Zhisheng_ Tian, and then reply to the keyword: Flink, you can get it unconditionally.

Related articles

1、Learning Flink from 0 to 1 – Introduction to Apache Flink

2、Learning Flink from 0 to 1 — an introduction to building Flink 1.6.0 environment and building and running simple programs on MAC

3、Learn Flink from 0 to 1 – detailed explanation of Flink profile

4、Learning Flink from 0 to 1 – Introduction to data source

5、Learn Flink from 0 to 1 – how to customize the data source?

6、Learning Flink from 0 to 1 – Introduction to data sink

7、Learn Flink from 0 to 1 – how to customize data sink?