diff options
author | anabranch <wac.chambers@gmail.com> | 2017-02-07 15:50:30 +0100 |
---|---|---|
committer | Herman van Hovell <hvanhovell@databricks.com> | 2017-02-07 15:50:30 +0100 |
commit | 7a7ce272fe9a703f58b0180a9d2001ecb5c4b8db (patch) | |
tree | 4a2533e6d7dcc2e38d62db64f87634a15f3e2439 /sql/hive/src/test/scala/org | |
parent | 6ed285c68fee451c45db7b01ca8ec1dea2efd479 (diff) | |
download | spark-7a7ce272fe9a703f58b0180a9d2001ecb5c4b8db.tar.gz spark-7a7ce272fe9a703f58b0180a9d2001ecb5c4b8db.tar.bz2 spark-7a7ce272fe9a703f58b0180a9d2001ecb5c4b8db.zip |
[SPARK-16609] Add to_date/to_timestamp with format functions
## What changes were proposed in this pull request?
This pull request adds two new user facing functions:
- `to_date` which accepts an expression and a format and returns a date.
- `to_timestamp` which accepts an expression and a format and returns a timestamp.
For example, Given a date in format: `2016-21-05`. (YYYY-dd-MM)
### Date Function
*Previously*
```
to_date(unix_timestamp(lit("2016-21-05"), "yyyy-dd-MM").cast("timestamp"))
```
*Current*
```
to_date(lit("2016-21-05"), "yyyy-dd-MM")
```
### Timestamp Function
*Previously*
```
unix_timestamp(lit("2016-21-05"), "yyyy-dd-MM").cast("timestamp")
```
*Current*
```
to_timestamp(lit("2016-21-05"), "yyyy-dd-MM")
```
### Tasks
- [X] Add `to_date` to Scala Functions
- [x] Add `to_date` to Python Functions
- [x] Add `to_date` to SQL Functions
- [X] Add `to_timestamp` to Scala Functions
- [x] Add `to_timestamp` to Python Functions
- [x] Add `to_timestamp` to SQL Functions
- [x] Add function to R
## How was this patch tested?
- [x] Add Functions to `DateFunctionsSuite`
- Test new `ParseToTimestamp` Expression (*not necessary*)
- Test new `ParseToDate` Expression (*not necessary*)
- [x] Add test for R
- [x] Add test for Python in test.py
Please review http://spark.apache.org/contributing.html before opening a pull request.
Author: anabranch <wac.chambers@gmail.com>
Author: Bill Chambers <bill@databricks.com>
Author: anabranch <bill@databricks.com>
Closes #16138 from anabranch/SPARK-16609.
Diffstat (limited to 'sql/hive/src/test/scala/org')
-rw-r--r-- | sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionToSQLSuite.scala | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionToSQLSuite.scala b/sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionToSQLSuite.scala index df9390aec7..1daa6f7822 100644 --- a/sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionToSQLSuite.scala +++ b/sql/hive/src/test/scala/org/apache/spark/sql/catalyst/ExpressionToSQLSuite.scala @@ -253,6 +253,7 @@ class ExpressionToSQLSuite extends SQLBuilderTest with SQLTestUtils { checkSqlGeneration("SELECT count(now())") checkSqlGeneration("SELECT quarter('2001-05-02')") checkSqlGeneration("SELECT second('11:35:55')") + checkSqlGeneration("SELECT to_timestamp('2001-10-30 10:30:00', 'yyyy-MM-dd HH:mm:ss')") checkSqlGeneration("SELECT to_date('2001-10-30 10:30:00')") checkSqlGeneration("SELECT to_unix_timestamp('2015-07-24 00:00:00', 'yyyy-MM-dd HH:mm:ss')") checkSqlGeneration("SELECT to_utc_timestamp('2015-07-24 00:00:00', 'PST')") |