diff options
| author | Theodore Vasiloudis <tvas@sics.se> | 2015-03-17 11:25:01 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-03-17 11:26:11 +0000 |
| commit | e26db9be47b295c4ec9e651f4cce321ba1fddfcd (patch) | |
| tree | 5507d9b8a983f7464d79069976fb591be59a9d0d /docs | |
| parent | 005d1c5f290decc606a0be59fb191136dafc0c9d (diff) | |
| download | spark-e26db9be47b295c4ec9e651f4cce321ba1fddfcd.tar.gz spark-e26db9be47b295c4ec9e651f4cce321ba1fddfcd.tar.bz2 spark-e26db9be47b295c4ec9e651f4cce321ba1fddfcd.zip | |
[docs] [SPARK-4820] Spark build encounters "File name too long" on some encrypted filesystems
Added a note instructing users how to build Spark in an encrypted file system.
Author: Theodore Vasiloudis <tvas@sics.se>
Closes #5041 from thvasilo/patch-2 and squashes the following commits:
09d890b [Theodore Vasiloudis] Workaroung for buiding in an encrypted filesystem
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/building-spark.md | 12 |
1 files changed, 12 insertions, 0 deletions
diff --git a/docs/building-spark.md b/docs/building-spark.md index 57d0ca834f..ea79c5bc27 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -23,6 +23,18 @@ build/mvn -Pyarn -Phadoop-2.4 -Dhadoop.version=2.4.0 -DskipTests clean package Other build examples can be found below. +**Note:** When building on an encrypted filesystem (if your home directory is encrypted, for example), then the Spark build might fail with a "Filename too long" error. As a workaround, add the following in the configuration args of the `scala-maven-plugin` in the project `pom.xml`: + + <arg>-Xmax-classfile-name</arg> + <arg>128</arg> + +and in `project/SparkBuild.scala` add: + + scalacOptions in Compile ++= Seq("-Xmax-classfile-name", "128"), + +to the `sharedSettings` val. See also [this PR](https://github.com/apache/spark/pull/2883/files) if you are unsure of where to add these lines. + + # Setting up Maven's Memory Usage You'll need to configure Maven to use more memory than usual by setting `MAVEN_OPTS`. We recommend the following settings: |