Spark java + Scala project package (jar)

Time:2020-9-29

1. Method 1: Maven packing
pom.xml file

<plugins>
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <configuration>
                    <appendAssemblyId>false</appendAssemblyId>
                    <descriptorRefs>
                        <descriptorRef>jar-with-dependencies</descriptorRef>
                    </descriptorRefs>
                    <archive>
                        <manifest>
                            <! -- the class of the main method entry is specified here ch.kmeans2 .SparkStreamingKMeansKafkaExample</mainClass>
                        </manifest>
                    </archive>
                </configuration>
                <executions>
                    <execution>
                        <id>make-assembly</id>
                        <phase>package</phase>
                        <goals>
                            <goal>assembly</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
            <plugin>
                <groupId>org.scala-tools</groupId>
                <artifactId>maven-scala-plugin</artifactId>
                <version>2.15.2</version>
                <executions>
                    <execution>
                        <id>scala-compile-first</id>
                        <goals>
                            <goal>compile</goal>
                        </goals>
                        <configuration>
                            <includes>
                                <include>**/*.scala</include>
                            </includes>
                        </configuration>
                    </execution>
                    <execution>
                        <id>scala-test-compile</id>
                        <goals>
                            <goal>testCompile</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>

If “thread clear” is encountered, run “mvmainpack” command java.lang.AbstractMethodError Consider whether the version of spark and the running version of the cluster in the project are consistent
2. Method 2: idea package
Project->Module Setting->Artifacts
Spark java + Scala project package (jar)
Spark java + Scala project package (jar)
Spark java + Scala project package (jar)
Then build artifact outputs the jar package

Recommended Today

Basic knowledge of react

React.FunctionComponent ReactProvides a component typeReact.FunctionComponent, can be abbreviatedReact.FC, Can receive a genericp, the default is{} children, return aReact.ReactNodeThis onechildrenIt’s anythingcomponentAll of them Static propertiesdefaultProps, the default property of the component, which can not be passed externally. interface IHelloProps { message?: string; } const Hello: React.FunctionComponent<IHelloProps> = (props) => { return <h2>{props.message}</h2>; }; Hello.defaultProps = { […]