diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2013-12-06 17:29:03 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2013-12-06 17:29:03 -0800 |
| commit | 241336add5be07fca5ff6c17eed368df7d0c3e3c (patch) | |
| tree | 659edd04c75af22b14a9303d157c244177bf2b96 /docs/index.md | |
| parent | e0392343a026d632bac0df0ad4f399fce742c151 (diff) | |
| parent | e2c2914faaf3d8ab849ad0477152f64df7adf4c3 (diff) | |
| download | spark-241336add5be07fca5ff6c17eed368df7d0c3e3c.tar.gz spark-241336add5be07fca5ff6c17eed368df7d0c3e3c.tar.bz2 spark-241336add5be07fca5ff6c17eed368df7d0c3e3c.zip | |
Merge pull request #234 from alig/master
Updated documentation about the YARN v2.2 build process
Diffstat (limited to 'docs/index.md')
| -rw-r--r-- | docs/index.md | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/docs/index.md b/docs/index.md index bd386a8a8f..bbb27338bc 100644 --- a/docs/index.md +++ b/docs/index.md @@ -56,14 +56,16 @@ Hadoop, you must build Spark against the same version that your cluster uses. By default, Spark links to Hadoop 1.0.4. You can change this by setting the `SPARK_HADOOP_VERSION` variable when compiling: - SPARK_HADOOP_VERSION=1.2.1 sbt/sbt assembly + SPARK_HADOOP_VERSION=2.2.0 sbt/sbt assembly In addition, if you wish to run Spark on [YARN](running-on-yarn.md), set `SPARK_YARN` to `true`: SPARK_HADOOP_VERSION=2.0.5-alpha SPARK_YARN=true sbt/sbt assembly -(Note that on Windows, you need to set the environment variables on separate lines, e.g., `set SPARK_HADOOP_VERSION=1.2.1`.) +Note that on Windows, you need to set the environment variables on separate lines, e.g., `set SPARK_HADOOP_VERSION=1.2.1`. + +For this version of Spark (0.8.1) Hadoop 2.2.x (or newer) users will have to build Spark and publish it locally. See [Launching Spark on YARN](running-on-yarn.md). This is needed because Hadoop 2.2 has non backwards compatible API changes. # Where to Go from Here |