diff options
| author | WangTaoTheTonic <wangtao111@huawei.com> | 2015-06-10 13:30:16 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-06-10 13:30:16 -0700 |
| commit | cb871c44c38a4c1575ed076389f14641afafad7d (patch) | |
| tree | 52e8e9f3569b020ec37c0de023f3e5e3e260976a /launcher | |
| parent | 80043e9e761c44ce2c3a432dcd1989be573f8bb4 (diff) | |
| download | spark-cb871c44c38a4c1575ed076389f14641afafad7d.tar.gz spark-cb871c44c38a4c1575ed076389f14641afafad7d.tar.bz2 spark-cb871c44c38a4c1575ed076389f14641afafad7d.zip | |
[SPARK-8290] spark class command builder need read SPARK_JAVA_OPTS and SPARK_DRIVER_MEMORY properly
SPARK_JAVA_OPTS was missed in reconstructing the launcher part, we should add it back so process launched by spark-class could read it properly. And so does `SPARK_DRIVER_MEMORY`.
The missing part is [here](https://github.com/apache/spark/blob/1c30afdf94b27e1ad65df0735575306e65d148a1/bin/spark-class#L97).
Author: WangTaoTheTonic <wangtao111@huawei.com>
Author: Tao Wang <wangtao111@huawei.com>
Closes #6741 from WangTaoTheTonic/SPARK-8290 and squashes the following commits:
bd89f0f [Tao Wang] make sure the memory setting is right too
e313520 [WangTaoTheTonic] spark class command builder need read SPARK_JAVA_OPTS
Diffstat (limited to 'launcher')
| -rw-r--r-- | launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java b/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java index d80abf2a86..de85720feb 100644 --- a/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java +++ b/launcher/src/main/java/org/apache/spark/launcher/SparkClassCommandBuilder.java @@ -93,6 +93,9 @@ class SparkClassCommandBuilder extends AbstractCommandBuilder { toolsDir.getAbsolutePath(), className); javaOptsKeys.add("SPARK_JAVA_OPTS"); + } else { + javaOptsKeys.add("SPARK_JAVA_OPTS"); + memKey = "SPARK_DRIVER_MEMORY"; } List<String> cmd = buildJavaCommand(extraClassPath); |