Apache Spark, Java, Linux, Scala

Install Apache Spark on Ubuntu 14.04

Download and install jdk

Download and install scala (Apache spark has a dependency on scala version. Apache Spark 1.2.1 and 1.3.0 need scala 2.10.x)

Set path environment variable

vim ~/.bashrc

export JAVA_HOME=/usr/local/java/jdk8

export PATH=$PATH:$JAVA_HOME/bin/

export SCALA_HOME=/usr/local/scala

export PATH=$PATH:$SCALA_HOME/bin/

Install Git

sudo apt-get install git

Download the Apache Spark

cd spark

sbt/sbt package

sbt/sbt assembly

above two command will take around 20-30 mins.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s