diff options
| author | xutingjun <xutingjun@huawei.com> | 2015-07-09 13:21:10 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-07-09 13:21:14 -0700 |
| commit | 930fe95350f8865e2af2d7afa5b717210933cd43 (patch) | |
| tree | dd574fdbcf4e75b07ea759a58821ec45a7aa68e0 /core | |
| parent | 7ce3b818fb1ba3f291eda58988e4808e999cae3a (diff) | |
| download | spark-930fe95350f8865e2af2d7afa5b717210933cd43.tar.gz spark-930fe95350f8865e2af2d7afa5b717210933cd43.tar.bz2 spark-930fe95350f8865e2af2d7afa5b717210933cd43.zip | |
[SPARK-8953] SPARK_EXECUTOR_CORES is not read in SparkSubmit
The configuration ```SPARK_EXECUTOR_CORES``` won't put into ```SparkConf```, so it has no effect to the dynamic executor allocation.
Author: xutingjun <xutingjun@huawei.com>
Closes #7322 from XuTingjun/SPARK_EXECUTOR_CORES and squashes the following commits:
2cafa89 [xutingjun] make SPARK_EXECUTOR_CORES has effect to dynamicAllocation
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala index 73ab18332f..6e3c0b21b3 100644 --- a/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala +++ b/core/src/main/scala/org/apache/spark/deploy/SparkSubmitArguments.scala @@ -162,6 +162,7 @@ private[deploy] class SparkSubmitArguments(args: Seq[String], env: Map[String, S .orNull executorCores = Option(executorCores) .orElse(sparkProperties.get("spark.executor.cores")) + .orElse(env.get("SPARK_EXECUTOR_CORES")) .orNull totalExecutorCores = Option(totalExecutorCores) .orElse(sparkProperties.get("spark.cores.max")) |