Voir la source de Spark on your own machine


== Installing and setting ==


'''Java'''

  $ sudo apt-add-repository ppa:webupd8team/java
  $ sudo apt-get update
  $ sudo apt-get install oracle-java7-installer

'''Scala'''

  $ wget http://www.scala-lang.org/files/archive/scala-2.10.4.tgz
  $ sudo mkdir /usr/local/src/scala
  $ sudo tar xvf scala-2.10.4.tgz -C /usr/local/src/scala/

'''Spark'''
  
  $ wget http://d3kbcqa49mib13.cloudfront.net/spark-1.5.2-bin-hadoop2.6.tgz
  $ sudo mkdir /usr/lib/spark
  $ sudo tar xvf spark-1.5.2-bin-hadoop2.6.tgz -C /usr/lib/spark

'''SBT'''
  $ wget https://dl.bintray.com/sbt/native-packages/sbt/0.13.9/sbt-0.13.9.tgz 
  $ sudo tar xvf sbt-0.13.9.tgz -C /usr/lib/

Add the following lines in the ''.bashrc'' file:
  export JAVA_HOME=/usr/lib/jvm/java-7-oracle
  export SCALA_HOME=/usr/local/src/scala/scala-2.10.4
  export SPARK_HOME=/usr/lib/spark/spark-1.5.2-bin-hadoop2.6
  export SBT_HOME=/usr/lib/sbt
  export PATH=$SCALA_HOME/bin:$JAVA_HOME/bin:$SPARK_HOME/bin:$SBT_HOME/bin:$PATH

$ source ~/.bashrc (to reload the file .bashrc)

'''Scala IDE for Eclipse'''

http://scala-ide.org/download/sdk.html

== Create a Spark-SBT project ==

- Write your application

  $ mkdir -p project1/src/main/scala 
  $ cd project1/
  $ echo " " > src/main/scala/SimpleApp.scala
  $ gedit src/main/scala/[[SimpleApp.scala]] &

- Create the build definition of the program (build.sbt) and add the following lines:

  name := "Simple Project"
  
  version := "1.0"
  
  scalaVersion := "2.10.4"
  
  libraryDependencies += "org.apache.spark" %% "spark-core" % "1.5.2"

- Create the ''JAR'':
  $ sbt package
- Run the project via spark-submit :
  $ spark-submit --class "SimpleApp" --master local[2] target/scala-2.10/simple-project_2.10-1.0.jar

== Use an Editor ==

- Add sbteclipse to your plugin definition file (or create one if doesn't exist). You can use either:
  - the global file (for version 0.13 and up) at ~/.sbt/0.13/plugins/plugins.sbt
  - the project-specific file at PROJECT_DIR/project/plugins.sbt
For the latest version:
  addSbtPlugin("com.typesafe.sbteclipse" % "sbteclipse-plugin" % "4.0.0")

- Run the command ''eclipse''
  $ sbt eclipse

- Import your project on Eclipse : import > General > Existing project into workspace > your-project