diff options
| author | Yong Tang <yong.tang.github@outlook.com> | 2017-03-10 13:33:58 -0800 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2017-03-10 13:34:01 -0800 |
| commit | 8f0490e22b4c7f1fdf381c70c5894d46b7f7e6fb (patch) | |
| tree | 69fd6b9c92caa17b58e05b847dff8a26ae7052f2 /conf | |
| parent | dd9049e0492cc70b629518fee9b3d1632374c612 (diff) | |
| download | spark-8f0490e22b4c7f1fdf381c70c5894d46b7f7e6fb.tar.gz spark-8f0490e22b4c7f1fdf381c70c5894d46b7f7e6fb.tar.bz2 spark-8f0490e22b4c7f1fdf381c70c5894d46b7f7e6fb.zip | |
[SPARK-17979][SPARK-14453] Remove deprecated SPARK_YARN_USER_ENV and SPARK_JAVA_OPTS
This fix removes deprecated support for config `SPARK_YARN_USER_ENV`, as is mentioned in SPARK-17979.
This fix also removes deprecated support for the following:
```
SPARK_YARN_USER_ENV
SPARK_JAVA_OPTS
SPARK_CLASSPATH
SPARK_WORKER_INSTANCES
```
Related JIRA:
[SPARK-14453]: https://issues.apache.org/jira/browse/SPARK-14453
[SPARK-12344]: https://issues.apache.org/jira/browse/SPARK-12344
[SPARK-15781]: https://issues.apache.org/jira/browse/SPARK-15781
Existing tests should pass.
Author: Yong Tang <yong.tang.github@outlook.com>
Closes #17212 from yongtang/SPARK-17979.
Diffstat (limited to 'conf')
| -rwxr-xr-x | conf/spark-env.sh.template | 3 |
1 files changed, 0 insertions, 3 deletions
diff --git a/conf/spark-env.sh.template b/conf/spark-env.sh.template index 5c1e876ef9..94bd2c477a 100755 --- a/conf/spark-env.sh.template +++ b/conf/spark-env.sh.template @@ -25,12 +25,10 @@ # - HADOOP_CONF_DIR, to point Spark towards Hadoop configuration files # - SPARK_LOCAL_IP, to set the IP address Spark binds to on this node # - SPARK_PUBLIC_DNS, to set the public dns name of the driver program -# - SPARK_CLASSPATH, default classpath entries to append # Options read by executors and drivers running inside the cluster # - SPARK_LOCAL_IP, to set the IP address Spark binds to on this node # - SPARK_PUBLIC_DNS, to set the public DNS name of the driver program -# - SPARK_CLASSPATH, default classpath entries to append # - SPARK_LOCAL_DIRS, storage directories to use on this node for shuffle and RDD data # - MESOS_NATIVE_JAVA_LIBRARY, to point to your libmesos.so if you use Mesos @@ -48,7 +46,6 @@ # - SPARK_WORKER_CORES, to set the number of cores to use on this machine # - SPARK_WORKER_MEMORY, to set how much total memory workers have to give executors (e.g. 1000m, 2g) # - SPARK_WORKER_PORT / SPARK_WORKER_WEBUI_PORT, to use non-default ports for the worker -# - SPARK_WORKER_INSTANCES, to set the number of worker processes per node # - SPARK_WORKER_DIR, to set the working directory of worker processes # - SPARK_WORKER_OPTS, to set config properties only for the worker (e.g. "-Dx=y") # - SPARK_DAEMON_MEMORY, to allocate to the master, worker and history server themselves (default: 1g). |