diff options
| author | jerryshao <sshao@hortonworks.com> | 2016-07-13 13:24:47 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-07-13 13:24:47 -0500 |
| commit | d8220c1e5e94abbdb9643672b918f0d748206db9 (patch) | |
| tree | 4322602ced176a94531da134a32868ebd4971c83 /python/pyspark/mllib/fpm.py | |
| parent | f376c37268848dbb4b2fb57677e22ef2bf207b49 (diff) | |
| download | spark-d8220c1e5e94abbdb9643672b918f0d748206db9.tar.gz spark-d8220c1e5e94abbdb9643672b918f0d748206db9.tar.bz2 spark-d8220c1e5e94abbdb9643672b918f0d748206db9.zip | |
[SPARK-16435][YARN][MINOR] Add warning log if initialExecutors is less than minExecutors
## What changes were proposed in this pull request?
Currently if `spark.dynamicAllocation.initialExecutors` is less than `spark.dynamicAllocation.minExecutors`, Spark will automatically pick the minExecutors without any warning. While in 1.6 Spark will throw exception if configured like this. So here propose to add warning log if these parameters are configured invalidly.
## How was this patch tested?
Unit test added to verify the scenario.
Author: jerryshao <sshao@hortonworks.com>
Closes #14149 from jerryshao/SPARK-16435.
Diffstat (limited to 'python/pyspark/mllib/fpm.py')
0 files changed, 0 insertions, 0 deletions