diff options
author | Xiao Li <gatorsmile@gmail.com> | 2017-04-17 09:50:20 -0700 |
---|---|---|
committer | Xiao Li <gatorsmile@gmail.com> | 2017-04-17 09:50:20 -0700 |
commit | 01ff0350a85b179715946c3bd4f003db7c5e3641 (patch) | |
tree | 8c4a43ff6d217c1b4587c7b9d06afa0e7fa9b44a /sql/core/src/main/scala/org/apache/spark | |
parent | 24f09b39c7b947e52fda952676d5114c2540e732 (diff) | |
download | spark-01ff0350a85b179715946c3bd4f003db7c5e3641.tar.gz spark-01ff0350a85b179715946c3bd4f003db7c5e3641.tar.bz2 spark-01ff0350a85b179715946c3bd4f003db7c5e3641.zip |
[SPARK-20349][SQL] ListFunctions returns duplicate functions after using persistent functions
### What changes were proposed in this pull request?
The session catalog caches some persistent functions in the `FunctionRegistry`, so there can be duplicates. Our Catalog API `listFunctions` does not handle it.
It would be better if `SessionCatalog` API can de-duplciate the records, instead of doing it by each API caller. In `FunctionRegistry`, our functions are identified by the unquoted string. Thus, this PR is try to parse it using our parser interface and then de-duplicate the names.
### How was this patch tested?
Added test cases.
Author: Xiao Li <gatorsmile@gmail.com>
Closes #17646 from gatorsmile/showFunctions.
Diffstat (limited to 'sql/core/src/main/scala/org/apache/spark')
-rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala index e0d0029369..545082324f 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala @@ -207,8 +207,6 @@ case class ShowFunctionsCommand( case (f, "USER") if showUserFunctions => f.unquotedString case (f, "SYSTEM") if showSystemFunctions => f.unquotedString } - // The session catalog caches some persistent functions in the FunctionRegistry - // so there can be duplicates. - functionNames.distinct.sorted.map(Row(_)) + functionNames.sorted.map(Row(_)) } } |