diff options
| author | Davies Liu <davies@databricks.com> | 2015-07-12 20:25:06 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2015-07-12 20:25:06 -0700 |
| commit | 20b474335c68c644150fdc8443a2d0d2dad5e27d (patch) | |
| tree | c367f6665eb399521df4216da567ce0599f9b06d /python/pyspark | |
| parent | 30090884f96efde72d9653f2253070b68f87782c (diff) | |
| download | spark-20b474335c68c644150fdc8443a2d0d2dad5e27d.tar.gz spark-20b474335c68c644150fdc8443a2d0d2dad5e27d.tar.bz2 spark-20b474335c68c644150fdc8443a2d0d2dad5e27d.zip | |
[SPARK-9006] [PYSPARK] fix microsecond loss in Python 3
It may loss a microsecond if using timestamp as float, should be `int` instead.
Author: Davies Liu <davies@databricks.com>
Closes #7363 from davies/fix_microsecond and squashes the following commits:
36f6007 [Davies Liu] fix microsecond loss in Python 3
Diffstat (limited to 'python/pyspark')
| -rw-r--r-- | python/pyspark/sql/types.py | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py index d638576916..f75791fad1 100644 --- a/python/pyspark/sql/types.py +++ b/python/pyspark/sql/types.py @@ -188,7 +188,8 @@ class TimestampType(AtomicType): def fromInternal(self, ts): if ts is not None: - return datetime.datetime.fromtimestamp(ts / 1e6) + # using int to avoid precision loss in float + return datetime.datetime.fromtimestamp(ts // 1000000).replace(microsecond=ts % 1000000) class DecimalType(FractionalType): |