diff options
| author | jiangxingbo <jiangxb1987@gmail.com> | 2016-08-24 23:36:04 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-08-24 23:36:04 -0700 |
| commit | 5f02d2e5b4d37f554629cbd0e488e856fffd7b6b (patch) | |
| tree | 00d1c3d44c017f3960888b37c15b08ec2c6d603e /python/pyspark/sql/functions.py | |
| parent | 4d0706d616176dc29ff3562e40cb00dd4eb9c302 (diff) | |
| download | spark-5f02d2e5b4d37f554629cbd0e488e856fffd7b6b.tar.gz spark-5f02d2e5b4d37f554629cbd0e488e856fffd7b6b.tar.bz2 spark-5f02d2e5b4d37f554629cbd0e488e856fffd7b6b.zip | |
[SPARK-17215][SQL] Method `SQLContext.parseDataType(dataTypeString: String)` could be removed.
## What changes were proposed in this pull request?
Method `SQLContext.parseDataType(dataTypeString: String)` could be removed, we should use `SparkSession.parseDataType(dataTypeString: String)` instead.
This require updating PySpark.
## How was this patch tested?
Existing test cases.
Author: jiangxingbo <jiangxb1987@gmail.com>
Closes #14790 from jiangxb1987/parseDataType.
Diffstat (limited to 'python/pyspark/sql/functions.py')
| -rw-r--r-- | python/pyspark/sql/functions.py | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/python/pyspark/sql/functions.py b/python/pyspark/sql/functions.py index 4ea83e24bb..89b3c07c07 100644 --- a/python/pyspark/sql/functions.py +++ b/python/pyspark/sql/functions.py @@ -1760,11 +1760,11 @@ class UserDefinedFunction(object): self._judf = self._create_judf(name) def _create_judf(self, name): - from pyspark.sql import SQLContext + from pyspark.sql import SparkSession sc = SparkContext.getOrCreate() wrapped_func = _wrap_function(sc, self.func, self.returnType) - ctx = SQLContext.getOrCreate(sc) - jdt = ctx._ssql_ctx.parseDataType(self.returnType.json()) + spark = SparkSession.builder.getOrCreate() + jdt = spark._jsparkSession.parseDataType(self.returnType.json()) if name is None: f = self.func name = f.__name__ if hasattr(f, '__name__') else f.__class__.__name__ |