diff options
| author | Brandon Bradley <bradleytastic@gmail.com> | 2016-01-28 15:25:57 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-01-28 15:25:57 -0800 |
| commit | 3a40c0e575fd4215302ea60c9821d31a5a138b8a (patch) | |
| tree | fc77ca55ccb10ed9a251ce386be7304608c1f81d /python | |
| parent | abae889f08eb412cb897e4e63614ec2c93885ffd (diff) | |
| download | spark-3a40c0e575fd4215302ea60c9821d31a5a138b8a.tar.gz spark-3a40c0e575fd4215302ea60c9821d31a5a138b8a.tar.bz2 spark-3a40c0e575fd4215302ea60c9821d31a5a138b8a.zip | |
[SPARK-12749][SQL] add json option to parse floating-point types as DecimalType
I tried to add this via `USE_BIG_DECIMAL_FOR_FLOATS` option from Jackson with no success.
Added test for non-complex types. Should I add a test for complex types?
Author: Brandon Bradley <bradleytastic@gmail.com>
Closes #10936 from blbradley/spark-12749.
Diffstat (limited to 'python')
| -rw-r--r-- | python/pyspark/sql/readwriter.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py index 0b20022b14..b1453c637f 100644 --- a/python/pyspark/sql/readwriter.py +++ b/python/pyspark/sql/readwriter.py @@ -152,6 +152,8 @@ class DataFrameReader(object): You can set the following JSON-specific options to deal with non-standard JSON files: * ``primitivesAsString`` (default ``false``): infers all primitive values as a string \ type + * `floatAsBigDecimal` (default `false`): infers all floating-point values as a decimal \ + type * ``allowComments`` (default ``false``): ignores Java/C++ style comment in JSON records * ``allowUnquotedFieldNames`` (default ``false``): allows unquoted JSON field names * ``allowSingleQuotes`` (default ``true``): allows single quotes in addition to double \ |