diff options
author | GuoQiang Li <witgo@qq.com> | 2014-12-10 12:24:04 -0800 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2014-12-10 12:26:42 -0800 |
commit | 742e7093eca8865225c29bacf4344f2e89bfea41 (patch) | |
tree | 0c478f939ccdfdc325c3e4e0e4d80240ba9f81c9 /bin | |
parent | 94b377f94487109a1cc3e07dd230b1df7a96e28d (diff) | |
download | spark-742e7093eca8865225c29bacf4344f2e89bfea41.tar.gz spark-742e7093eca8865225c29bacf4344f2e89bfea41.tar.bz2 spark-742e7093eca8865225c29bacf4344f2e89bfea41.zip |
[SPARK-4161]Spark shell class path is not correctly set if "spark.driver.extraClassPath" is set in defaults.conf
Author: GuoQiang Li <witgo@qq.com>
Closes #3050 from witgo/SPARK-4161 and squashes the following commits:
abb6fa4 [GuoQiang Li] move usejavacp opt to spark-shell
89e39e7 [GuoQiang Li] review commit
c2a6f04 [GuoQiang Li] Spark shell class path is not correctly set if "spark.driver.extraClassPath" is set in defaults.conf
Diffstat (limited to 'bin')
-rwxr-xr-x | bin/spark-shell | 7 |
1 files changed, 7 insertions, 0 deletions
diff --git a/bin/spark-shell b/bin/spark-shell index 4a0670fc6c..cca5aa0676 100755 --- a/bin/spark-shell +++ b/bin/spark-shell @@ -45,6 +45,13 @@ source "$FWDIR"/bin/utils.sh SUBMIT_USAGE_FUNCTION=usage gatherSparkSubmitOpts "$@" +# SPARK-4161: scala does not assume use of the java classpath, +# so we need to add the "-Dscala.usejavacp=true" flag mnually. We +# do this specifically for the Spark shell because the scala REPL +# has its own class loader, and any additional classpath specified +# through spark.driver.extraClassPath is not automatically propagated. +SPARK_SUBMIT_OPTS="$SPARK_SUBMIT_OPTS -Dscala.usejavacp=true" + function main() { if $cygwin; then # Workaround for issue involving JLine and Cygwin |