Eclipse Maven - Scala Spark 프로그래밍 및 MongoDB, Kafka, Redis 연동

<Eclipse Maven - Scala Spark 프로그래밍>

https://www.cloudera.com/documentation/enterprise/5-5-x/topics/spark_develop_run.html

https://m.blog.naver.com/PostView.nhn?blogId=estern&logNo=220824080406&proxyReferer=https%3A%2F%2Fwww.google.co.kr%2F

http://scala-ide.org/docs/tutorials/m2eclipse/

http://jdm.kr/blog/176

http://www.gooper.com/ss/bigdata/271691

http://blog.naver.com/PostView.nhn?blogId=kimyoon21&logNo=220297814742&parentCategoryNo=&categoryNo=43&viewDate=&isShowPopularPosts=true&from=search

<Spark Streaming 과 MongoDB 연동>

https://docs.mongodb.com/spark-connector/master/scala/streaming/

https://github.com/mongodb/mongo-spark/blob/master/examples/src/test/scala/tour/TourHelper.scala

<Spark Streaming과 Redis 연동 - Maven에서 로컬 jar 사용>

https://gs.saro.me/#!m=elec&jn=745

https://stackoverflow.com/questions/4955635/how-to-add-local-jar-files-to-a-maven-project

http://roufid.com/3-ways-to-add-local-jar-to-maven-project/

http://kimtaewook.tistory.com/entry/%EB%B3%B8%EB%AC%B8%EC%8A%A4%ED%81%AC%EB%9E%A9-Maven%EB%A9%94%EC%9D%B4%EB%B8%90%EC%9D%98-%ED%99%9C%EC%9A%A9

https://stackoverflow.com/questions/2065928/maven-2-assembly-with-dependencies-jar-under-scope-system-not-included

<Spark Streaming에서 Kafka로 결과 출력>

https://github.com/BenFradet/spark-kafka-writer

          <dependencies>

<dependency>

    <groupId>org.mongodb</groupId>

    <artifactId>bson</artifactId>

    <version>3.4.1</version>

</dependency>

<dependency>

<groupId>org.apache.spark</groupId>

<artifactId>spark-core_2.11</artifactId>

<version>${spark.version}</version>

<scope>provided</scope>

</dependency>

<dependency>

<groupId>org.apache.spark</groupId>

<artifactId>spark-streaming_2.11</artifactId>

<version>${spark.version}</version>

<scope>provided</scope>

</dependency>

<dependency>

    <groupId>org.apache.spark</groupId>

    <artifactId>spark-sql_2.11</artifactId>

    <version>${spark.version}</version>

    <scope>provided</scope>

</dependency>

<dependency>

<groupId>org.apache.spark</groupId>

<artifactId>spark-streaming-kafka-0-10_2.11</artifactId>

<version>2.2.0</version>

<scope>provided</scope>

</dependency>

<dependency>

<groupId>com.github.benfradet</groupId>

<artifactId>spark-kafka-writer_2.11</artifactId>

<version>0.4.0</version>

</dependency>

<dependency>

<groupId>org.mongodb.spark</groupId>

<artifactId>mongo-spark-connector_2.11</artifactId>

<version>2.2.0</version>

</dependency>

<!--dependency>

    <groupId>RedisLabs</groupId>

    <artifactId>spark-redis</artifactId>

    <version>0.3.2</version>

</dependency-->

</dependencies>

  <build>

    <sourceDirectory>src</sourceDirectory>

    <plugins>

      <plugin>

        <artifactId>maven-compiler-plugin</artifactId>

        <version>3.5.1</version>

        <configuration>

          <source>1.8</source>

          <target>1.8</target>

        </configuration>

      </plugin>

      <plugin>

<artifactId>maven-assembly-plugin</artifactId>

<configuration>

<descriptorRefs>

<descriptorRef>jar-with-dependencies</descriptorRef>

</descriptorRefs>

<archive>

<manifest>

<mainClass>WordCount</mainClass>

</manifest>

</archive>

</configuration>

<executions>

<execution>

<phase>package</phase>

<goals>

<goal>single</goal>

</goals>

</execution>

</executions>

  </plugin> 

      <plugin>

  <groupId>org.scala-tools</groupId>

      <artifactId>maven-scala-plugin</artifactId>

      <version>2.15.2</version>

      <executions>

        <execution>

          <goals>

            <goal>compile</goal>

            <goal>testCompile</goal>

          </goals>

        </execution>

      </executions>

</plugin>

    </plugins>

  </build>