diff options
| author | Yijie Shen <henry.yijieshen@gmail.com> | 2015-07-08 20:20:17 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-07-08 20:20:17 -0700 |
| commit | a290814877308c6fa9b0f78b1a81145db7651ca4 (patch) | |
| tree | 2ed969329abf881ed157734259cdbba93bbb3afb /python/pyspark | |
| parent | 28fa01e2ba146e823489f6d81c5eb3a76b20c71f (diff) | |
| download | spark-a290814877308c6fa9b0f78b1a81145db7651ca4.tar.gz spark-a290814877308c6fa9b0f78b1a81145db7651ca4.tar.bz2 spark-a290814877308c6fa9b0f78b1a81145db7651ca4.zip | |
[SPARK-8866][SQL] use 1us precision for timestamp type
JIRA: https://issues.apache.org/jira/browse/SPARK-8866
Author: Yijie Shen <henry.yijieshen@gmail.com>
Closes #7283 from yijieshen/micro_timestamp and squashes the following commits:
dc735df [Yijie Shen] update CastSuite to avoid round error
714eaea [Yijie Shen] add timestamp_udf into blacklist due to precision lose
c3ca2f4 [Yijie Shen] fix unhandled case in CurrentTimestamp
8d4aa6b [Yijie Shen] use 1us precision for timestamp type
Diffstat (limited to 'python/pyspark')
| -rw-r--r-- | python/pyspark/sql/types.py | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/python/pyspark/sql/types.py b/python/pyspark/sql/types.py index 7e64cb0b54..fecfe6d71e 100644 --- a/python/pyspark/sql/types.py +++ b/python/pyspark/sql/types.py @@ -775,7 +775,7 @@ def _python_to_sql_converter(dataType): if dt: seconds = (calendar.timegm(dt.utctimetuple()) if dt.tzinfo else time.mktime(dt.timetuple())) - return int(seconds * 1e7 + dt.microsecond * 10) + return int(seconds * 1e6 + dt.microsecond) return to_posix_timstamp else: |