diff options
| author | windpiger <songjun@outlook.com> | 2017-02-08 14:30:28 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2017-02-08 14:30:28 +0800 |
| commit | d60dde26f98164ae146da1b5f409f4eb7c3621aa (patch) | |
| tree | 477654049b435a3aefd5bd1e8e0a997de47b6c23 /pom.xml | |
| parent | 5a0569ce693c635c5fa12b2de33ed3643ce888e3 (diff) | |
| download | spark-d60dde26f98164ae146da1b5f409f4eb7c3621aa.tar.gz spark-d60dde26f98164ae146da1b5f409f4eb7c3621aa.tar.bz2 spark-d60dde26f98164ae146da1b5f409f4eb7c3621aa.zip | |
[SPARK-19488][SQL] fix csv infer schema when the field is Nan/Inf etc
## What changes were proposed in this pull request?
when csv infer schema, it does not use user defined csvoptions to parse the field, such as `inf`, `-inf` which are should be parsed to DoubleType
this pr add `options.nanValue`, `options.negativeInf`, `options.positiveIn` to check if the field is a DoubleType
## How was this patch tested?
unit test added
Author: windpiger <songjun@outlook.com>
Closes #16834 from windpiger/fixinferInfSchemaCsv.
Diffstat (limited to 'pom.xml')
0 files changed, 0 insertions, 0 deletions