diff options
| author | GuoQiang Li <witgo@qq.com> | 2014-12-10 12:24:04 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-01-21 10:57:15 -0800 |
| commit | e90f6b5c6e27c60219aa0129b50985ebade048a0 (patch) | |
| tree | 685e0d90e2d676ff8547b7a86979ac3aa7deb39f | |
| parent | 1d730170cac5f30ce79bbbc3b6ae1dfff7b82f4c (diff) | |
| download | spark-e90f6b5c6e27c60219aa0129b50985ebade048a0.tar.gz spark-e90f6b5c6e27c60219aa0129b50985ebade048a0.tar.bz2 spark-e90f6b5c6e27c60219aa0129b50985ebade048a0.zip | |
[SPARK-4161]Spark shell class path is not correctly set if "spark.driver.extraClassPath" is set in defaults.conf
Author: GuoQiang Li <witgo@qq.com>
Closes #3050 from witgo/SPARK-4161 and squashes the following commits:
abb6fa4 [GuoQiang Li] move usejavacp opt to spark-shell
89e39e7 [GuoQiang Li] review commit
c2a6f04 [GuoQiang Li] Spark shell class path is not correctly set if "spark.driver.extraClassPath" is set in defaults.conf
| -rwxr-xr-x | bin/spark-shell | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/bin/spark-shell b/bin/spark-shell index 4a0670fc6c..cca5aa0676 100755 --- a/bin/spark-shell +++ b/bin/spark-shell @@ -45,6 +45,13 @@ source "$FWDIR"/bin/utils.sh SUBMIT_USAGE_FUNCTION=usage gatherSparkSubmitOpts "$@" +# SPARK-4161: scala does not assume use of the java classpath, +# so we need to add the "-Dscala.usejavacp=true" flag mnually. We +# do this specifically for the Spark shell because the scala REPL +# has its own class loader, and any additional classpath specified +# through spark.driver.extraClassPath is not automatically propagated. +SPARK_SUBMIT_OPTS="$SPARK_SUBMIT_OPTS -Dscala.usejavacp=true" + function main() { if $cygwin; then # Workaround for issue involving JLine and Cygwin |