diff options
| author | Andrew Or <andrew@databricks.com> | 2014-10-30 15:31:23 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-10-30 15:31:23 -0700 |
| commit | 26f092d4e32cc1f7e279646075eaf1e495395923 (patch) | |
| tree | 302e0b43e8ef09135a13159ba0cac4dedae94328 /sql | |
| parent | 24c5129257ce6e3b734f168e860b714c2730b55f (diff) | |
| download | spark-26f092d4e32cc1f7e279646075eaf1e495395923.tar.gz spark-26f092d4e32cc1f7e279646075eaf1e495395923.tar.bz2 spark-26f092d4e32cc1f7e279646075eaf1e495395923.zip | |
[SPARK-4138][SPARK-4139] Improve dynamic allocation settings
This should be merged after #2746 (SPARK-3795).
**SPARK-4138**. If the user sets both the number of executors and `spark.dynamicAllocation.enabled`, we should throw an exception.
**SPARK-4139**. If the user sets `spark.dynamicAllocation.enabled`, we should use the max number of executors as the starting number of executors because the first job is likely to run immediately after application startup. If the latter is not set, throw an exception.
Author: Andrew Or <andrew@databricks.com>
Closes #3002 from andrewor14/yarn-set-executors and squashes the following commits:
c528fce [Andrew Or] Merge branch 'master' of github.com:apache/spark into yarn-set-executors
55d4699 [Andrew Or] Bug fix: `isDynamicAllocationEnabled` was always false
2b0ccec [Andrew Or] Start the number of executors at the max
022bfde [Andrew Or] Guard against incompatible settings of number of executors
Diffstat (limited to 'sql')
0 files changed, 0 insertions, 0 deletions