diff options
| author | Jurriaan Pruis <email@jurriaanpruis.nl> | 2016-05-25 12:40:16 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-05-25 12:40:16 -0700 |
| commit | c875d81a3de3f209b9eb03adf96b7c740b2c7b52 (patch) | |
| tree | f09a2b335d592b1c40e42b2f557e8f643768dc3e /sql/core/pom.xml | |
| parent | 4b88067416ce922ae15a1445cf953fb9b5c43427 (diff) | |
| download | spark-c875d81a3de3f209b9eb03adf96b7c740b2c7b52.tar.gz spark-c875d81a3de3f209b9eb03adf96b7c740b2c7b52.tar.bz2 spark-c875d81a3de3f209b9eb03adf96b7c740b2c7b52.zip | |
[SPARK-15493][SQL] default QuoteEscapingEnabled flag to true when writing CSV
## What changes were proposed in this pull request?
Default QuoteEscapingEnabled flag to true when writing CSV and add an escapeQuotes option to be able to change this.
See https://github.com/uniVocity/univocity-parsers/blob/f3eb2af26374940e60d91d1703bde54619f50c51/src/main/java/com/univocity/parsers/csv/CsvWriterSettings.java#L231-L247
This change is needed to be able to write RFC 4180 compatible CSV files (https://tools.ietf.org/html/rfc4180#section-2)
https://issues.apache.org/jira/browse/SPARK-15493
## How was this patch tested?
Added a test that verifies the output is quoted correctly.
Author: Jurriaan Pruis <email@jurriaanpruis.nl>
Closes #13267 from jurriaan/quote-escaping.
Diffstat (limited to 'sql/core/pom.xml')
| -rw-r--r-- | sql/core/pom.xml | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/core/pom.xml b/sql/core/pom.xml index 2ea980bf20..b833b9369e 100644 --- a/sql/core/pom.xml +++ b/sql/core/pom.xml @@ -39,7 +39,7 @@ <dependency> <groupId>com.univocity</groupId> <artifactId>univocity-parsers</artifactId> - <version>2.1.0</version> + <version>2.1.1</version> <type>jar</type> </dependency> <dependency> |