diff options
| author | Kishor Patil <kpatil@yahoo-inc.com> | 2016-11-03 16:10:26 -0500 |
|---|---|---|
| committer | Tom Graves <tgraves@yahoo-inc.com> | 2016-11-03 16:10:26 -0500 |
| commit | 098e4ca9c7af61e64839a50c65be449749af6482 (patch) | |
| tree | 49d79c9bf9f5c27b34760687a62fd99838b78ecf /external | |
| parent | 16293311cdb25a62733a9aae4355659b971a3ce1 (diff) | |
| download | spark-098e4ca9c7af61e64839a50c65be449749af6482.tar.gz spark-098e4ca9c7af61e64839a50c65be449749af6482.tar.bz2 spark-098e4ca9c7af61e64839a50c65be449749af6482.zip | |
[SPARK-18099][YARN] Fail if same files added to distributed cache for --files and --archives
## What changes were proposed in this pull request?
During spark-submit, if yarn dist cache is instructed to add same file under --files and --archives, This code change ensures the spark yarn distributed cache behaviour is retained i.e. to warn and fail if same files is mentioned in both --files and --archives.
## How was this patch tested?
Manually tested:
1. if same jar is mentioned in --jars and --files it will continue to submit the job.
- basically functionality [SPARK-14423] #12203 is unchanged
1. if same file is mentioned in --files and --archives it will fail to submit the job.
Please review https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark before opening a pull request.
… under archives and files
Author: Kishor Patil <kpatil@yahoo-inc.com>
Closes #15627 from kishorvpatil/spark18099.
Diffstat (limited to 'external')
0 files changed, 0 insertions, 0 deletions