diff options
author | gatorsmile <gatorsmile@gmail.com> | 2016-02-22 22:17:56 -0800 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-02-22 22:17:56 -0800 |
commit | 9dd5399d78d74a8ba2326db25704ba7cb7aa353d (patch) | |
tree | 5aa9be89454d316b9142f497b792e325300e0a70 /sql/core/src | |
parent | 5d80fac58f837933b5359a8057676f45539e53af (diff) | |
download | spark-9dd5399d78d74a8ba2326db25704ba7cb7aa353d.tar.gz spark-9dd5399d78d74a8ba2326db25704ba7cb7aa353d.tar.bz2 spark-9dd5399d78d74a8ba2326db25704ba7cb7aa353d.zip |
[SPARK-12723][SQL] Comprehensive Verification and Fixing of SQL Generation Support for Expressions
#### What changes were proposed in this pull request?
Ensure that all built-in expressions can be mapped to its SQL representation if there is one (e.g. ScalaUDF doesn't have a SQL representation). The function lists are from the expression list in `FunctionRegistry`.
window functions, grouping sets functions (`cube`, `rollup`, `grouping`, `grouping_id`), generator functions (`explode` and `json_tuple`) are covered by separate JIRA and PRs. Thus, this PR does not cover them. Except these functions, all the built-in expressions are covered. For details, see the list in `ExpressionToSQLSuite`.
Fixed a few issues. For example, the `prettyName` of `approx_count_distinct` is not right. The `sql` of `hash` function is not right, since the `hash` function does not accept `seed`.
Additionally, also correct the order of expressions in `FunctionRegistry` so that people are easier to find which functions are missing.
cc liancheng
#### How was the this patch tested?
Added two test cases in LogicalPlanToSQLSuite for covering `not like` and `not in`.
Added a new test suite `ExpressionToSQLSuite` to cover the functions:
1. misc non-aggregate functions + complex type creators + null expressions
2. math functions
3. aggregate functions
4. string functions
5. date time functions + calendar interval
6. collection functions
7. misc functions
Author: gatorsmile <gatorsmile@gmail.com>
Closes #11314 from gatorsmile/expressionToSQL.
Diffstat (limited to 'sql/core/src')
-rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/functions.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala index 510894afac..b9873d38a6 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala @@ -1972,7 +1972,7 @@ object functions extends LegacyFunctions { def crc32(e: Column): Column = withExpr { Crc32(e.expr) } /** - * Calculates the hash code of given columns, and returns the result as a int column. + * Calculates the hash code of given columns, and returns the result as an int column. * * @group misc_funcs * @since 2.0 |