diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-11-11 11:16:39 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-11-11 11:16:39 -0800 |
| commit | 529a1d3380c4c23fed068ad05a6376162c4b76d6 (patch) | |
| tree | 3b18b3ab42840c8893b4e5ce9c2fb8d99f2264cf /docs/building-spark.md | |
| parent | e71ba56586ba64da18f412bc0e0263777c46ac4a (diff) | |
| download | spark-529a1d3380c4c23fed068ad05a6376162c4b76d6.tar.gz spark-529a1d3380c4c23fed068ad05a6376162c4b76d6.tar.bz2 spark-529a1d3380c4c23fed068ad05a6376162c4b76d6.zip | |
[SPARK-6152] Use shaded ASM5 to support closure cleaning of Java 8 compiled classes
This patch modifies Spark's closure cleaner (and a few other places) to use ASM 5, which is necessary in order to support cleaning of closures that were compiled by Java 8.
In order to avoid ASM dependency conflicts, Spark excludes ASM from all of its dependencies and uses a shaded version of ASM 4 that comes from `reflectasm` (see [SPARK-782](https://issues.apache.org/jira/browse/SPARK-782) and #232). This patch updates Spark to use a shaded version of ASM 5.0.4 that was published by the Apache XBean project; the POM used to create the shaded artifact can be found at https://github.com/apache/geronimo-xbean/blob/xbean-4.4/xbean-asm5-shaded/pom.xml.
http://movingfulcrum.tumblr.com/post/80826553604/asm-framework-50-the-missing-migration-guide was a useful resource while upgrading the code to use the new ASM5 opcodes.
I also added a new regression tests in the `java8-tests` subproject; the existing tests were insufficient to catch this bug, which only affected Scala 2.11 user code which was compiled targeting Java 8.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #9512 from JoshRosen/SPARK-6152.
Diffstat (limited to 'docs/building-spark.md')
| -rw-r--r-- | docs/building-spark.md | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/docs/building-spark.md b/docs/building-spark.md index 4f73adb854..3d38edbdad 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -190,6 +190,10 @@ Running only Java 8 tests and nothing else. mvn install -DskipTests -Pjava8-tests +or + + sbt -Pjava8-tests java8-tests/test + Java 8 tests are run when `-Pjava8-tests` profile is enabled, they will run in spite of `-DskipTests`. For these tests to run your system must have a JDK 8 installation. If you have JDK 8 installed but it is not the system default, you can set JAVA_HOME to point to JDK 8 before running the tests. |