diff options
| author | Jeff Zhang <zjffdu@apache.org> | 2016-10-14 15:50:35 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2016-10-14 15:50:35 -0700 |
| commit | f00df40cfefef0f3fc73f16ada1006e4dcfa5a39 (patch) | |
| tree | 17197231fa9c82e3158b039d950c021383e94885 /sql/catalyst | |
| parent | 5aeb7384c7aa5f487f031f9ae07d3f1653399d14 (diff) | |
| download | spark-f00df40cfefef0f3fc73f16ada1006e4dcfa5a39.tar.gz spark-f00df40cfefef0f3fc73f16ada1006e4dcfa5a39.tar.bz2 spark-f00df40cfefef0f3fc73f16ada1006e4dcfa5a39.zip | |
[SPARK-11775][PYSPARK][SQL] Allow PySpark to register Java UDF
Currently pyspark can only call the builtin java UDF, but can not call custom java UDF. It would be better to allow that. 2 benefits:
* Leverage the power of rich third party java library
* Improve the performance. Because if we use python UDF, python daemons will be started on worker which will affect the performance.
Author: Jeff Zhang <zjffdu@apache.org>
Closes #9766 from zjffdu/SPARK-11775.
Diffstat (limited to 'sql/catalyst')
| -rw-r--r-- | sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala index e6f61b00eb..04f0cfce88 100644 --- a/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala +++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/JavaTypeInference.scala @@ -59,7 +59,7 @@ object JavaTypeInference { * @param typeToken Java type * @return (SQL data type, nullable) */ - private def inferDataType(typeToken: TypeToken[_]): (DataType, Boolean) = { + private[sql] def inferDataType(typeToken: TypeToken[_]): (DataType, Boolean) = { typeToken.getRawType match { case c: Class[_] if c.isAnnotationPresent(classOf[SQLUserDefinedType]) => (c.getAnnotation(classOf[SQLUserDefinedType]).udt().newInstance(), true) |