diff options
| author | hyukjinkwon <gurwls223@gmail.com> | 2016-06-21 21:58:38 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-06-21 21:58:38 -0700 |
| commit | 7580f3041a1a3757a0b14b9d8afeb720f261fff6 (patch) | |
| tree | a4da4d014c9a95d4f57e82e9f8081e2a21eedd44 /mllib | |
| parent | d77c4e6e2eef24f4276c38b3add8c29bb885f4db (diff) | |
| download | spark-7580f3041a1a3757a0b14b9d8afeb720f261fff6.tar.gz spark-7580f3041a1a3757a0b14b9d8afeb720f261fff6.tar.bz2 spark-7580f3041a1a3757a0b14b9d8afeb720f261fff6.zip | |
[SPARK-16104] [SQL] Do not creaate CSV writer object for every flush when writing
## What changes were proposed in this pull request?
This PR let `CsvWriter` object is not created for each time but able to be reused. This way was taken after from JSON data source.
Original `CsvWriter` was being created for each row but it was enhanced in https://github.com/apache/spark/pull/13229. However, it still creates `CsvWriter` object for each `flush()` in `LineCsvWriter`. It seems it does not have to close the object and re-create this for every flush.
It follows the original logic as it is but `CsvWriter` is reused by reseting `CharArrayWriter`.
## How was this patch tested?
Existing tests should cover this.
Author: hyukjinkwon <gurwls223@gmail.com>
Closes #13809 from HyukjinKwon/write-perf.
Diffstat (limited to 'mllib')
0 files changed, 0 insertions, 0 deletions