diff options
| author | Sandy Ryza <sandy@cloudera.com> | 2014-04-29 23:24:34 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-29 23:24:34 -0700 |
| commit | ff5be9a41e52454e0f9cae83dd1fd50fbeaa684a (patch) | |
| tree | 5bd17eaa50b3120317370821104c9c6d9e238b56 /sbin/spark-config.sh | |
| parent | 7025dda8fa84b57d6f12bc770df2fa10eef21d88 (diff) | |
| download | spark-ff5be9a41e52454e0f9cae83dd1fd50fbeaa684a.tar.gz spark-ff5be9a41e52454e0f9cae83dd1fd50fbeaa684a.tar.bz2 spark-ff5be9a41e52454e0f9cae83dd1fd50fbeaa684a.zip | |
SPARK-1004. PySpark on YARN
This reopens https://github.com/apache/incubator-spark/pull/640 against the new repo
Author: Sandy Ryza <sandy@cloudera.com>
Closes #30 from sryza/sandy-spark-1004 and squashes the following commits:
89889d4 [Sandy Ryza] Move unzipping py4j to the generate-resources phase so that it gets included in the jar the first time
5165a02 [Sandy Ryza] Fix docs
fd0df79 [Sandy Ryza] PySpark on YARN
Diffstat (limited to 'sbin/spark-config.sh')
| -rwxr-xr-x | sbin/spark-config.sh | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/sbin/spark-config.sh b/sbin/spark-config.sh index cd2c7b7b0d..147b506dd5 100755 --- a/sbin/spark-config.sh +++ b/sbin/spark-config.sh @@ -34,3 +34,6 @@ this="$config_bin/$script" export SPARK_PREFIX=`dirname "$this"`/.. export SPARK_HOME=${SPARK_PREFIX} export SPARK_CONF_DIR="$SPARK_HOME/conf" +# Add the PySpark classes to the PYTHONPATH: +export PYTHONPATH=$SPARK_HOME/python:$PYTHONPATH +export PYTHONPATH=$SPARK_HOME/python/lib/py4j-0.8.1-src.zip:$PYTHONPATH |