diff options
| author | Takeshi YAMAMURO <linguin.m.s@gmail.com> | 2016-06-13 13:41:26 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2016-06-13 13:41:26 -0700 |
| commit | 5ad4e32d46599ae1b8626f08aa97345d078c28d7 (patch) | |
| tree | 604dd944bf2ed054dfef6f4fc7a727d95f75870b /dev | |
| parent | 3b7fb84cf88bcae56713fd56396db537fa18f2e5 (diff) | |
| download | spark-5ad4e32d46599ae1b8626f08aa97345d078c28d7.tar.gz spark-5ad4e32d46599ae1b8626f08aa97345d078c28d7.tar.bz2 spark-5ad4e32d46599ae1b8626f08aa97345d078c28d7.zip | |
[SPARK-15530][SQL] Set #parallelism for file listing in listLeafFilesInParallel
## What changes were proposed in this pull request?
This pr is to set the number of parallelism to prevent file listing in `listLeafFilesInParallel` from generating many tasks in case of large #defaultParallelism.
## How was this patch tested?
Manually checked
Author: Takeshi YAMAMURO <linguin.m.s@gmail.com>
Closes #13444 from maropu/SPARK-15530.
Diffstat (limited to 'dev')
0 files changed, 0 insertions, 0 deletions