diff options
| author | Reynold Xin <rxin@databricks.com> | 2016-01-02 22:31:39 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-01-02 22:31:39 -0800 |
| commit | 513e3b092c4f3d58058ff64c861ea35cfec04205 (patch) | |
| tree | 9261547956aaf05953296302da9e4fc561a7dc05 /sql/core | |
| parent | 15bd73627e04591fd13667b4838c9098342db965 (diff) | |
| download | spark-513e3b092c4f3d58058ff64c861ea35cfec04205.tar.gz spark-513e3b092c4f3d58058ff64c861ea35cfec04205.tar.bz2 spark-513e3b092c4f3d58058ff64c861ea35cfec04205.zip | |
[SPARK-12599][MLLIB][SQL] Remove the use of callUDF in MLlib
callUDF has been deprecated. However, we do not have an alternative for users to specify the output data type without type tags. This pull request introduced a new API for that, and replaces the invocation of the deprecated callUDF with that.
Author: Reynold Xin <rxin@databricks.com>
Closes #10547 from rxin/SPARK-12599.
Diffstat (limited to 'sql/core')
| -rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/functions.scala | 14 |
1 files changed, 14 insertions, 0 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala index 97c5aed6da..3572f3c3a1 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/functions.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/functions.scala @@ -2844,6 +2844,20 @@ object functions extends LegacyFunctions { // scalastyle:on line.size.limit /** + * Defines a user-defined function (UDF) using a Scala closure. For this variant, the caller must + * specifcy the output data type, and there is no automatic input type coercion. + * + * @param f A closure in Scala + * @param dataType The output data type of the UDF + * + * @group udf_funcs + * @since 2.0.0 + */ + def udf(f: AnyRef, dataType: DataType): UserDefinedFunction = { + UserDefinedFunction(f, dataType, None) + } + + /** * Call an user-defined function. * Example: * {{{ |