diff options
| author | jerryshao <sshao@hortonworks.com> | 2016-01-13 16:34:23 -0800 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2016-01-13 16:34:23 -0800 |
| commit | cd81fc9e8652c07b84f0887a24d67381b4e605fa (patch) | |
| tree | 83f75886b61d38a0650101f8dfddcd418ad0b018 /examples | |
| parent | eabc7b8ee7e809bab05361ed154f87bff467bd88 (diff) | |
| download | spark-cd81fc9e8652c07b84f0887a24d67381b4e605fa.tar.gz spark-cd81fc9e8652c07b84f0887a24d67381b4e605fa.tar.bz2 spark-cd81fc9e8652c07b84f0887a24d67381b4e605fa.zip | |
[SPARK-12400][SHUFFLE] Avoid generating temp shuffle files for empty partitions
This problem lies in `BypassMergeSortShuffleWriter`, empty partition will also generate a temp shuffle file with several bytes. So here change to only create file when partition is not empty.
This problem only lies in here, no such issue in `HashShuffleWriter`.
Please help to review, thanks a lot.
Author: jerryshao <sshao@hortonworks.com>
Closes #10376 from jerryshao/SPARK-12400.
Diffstat (limited to 'examples')
0 files changed, 0 insertions, 0 deletions