diff options
| author | Liang-Chi Hsieh <simonh@tw.ibm.com> | 2016-06-13 19:59:53 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2016-06-13 19:59:53 -0700 |
| commit | baa3e633e18c47b12e79fe3ddc01fc8ec010f096 (patch) | |
| tree | 83c91014b9d46fc9efc4bf1f5dcef5cee1fe184a /python/pyspark/java_gateway.py | |
| parent | 5827b65e28da168286c771c53a38620d79f5e74f (diff) | |
| download | spark-baa3e633e18c47b12e79fe3ddc01fc8ec010f096.tar.gz spark-baa3e633e18c47b12e79fe3ddc01fc8ec010f096.tar.bz2 spark-baa3e633e18c47b12e79fe3ddc01fc8ec010f096.zip | |
[SPARK-15364][ML][PYSPARK] Implement PySpark picklers for ml.Vector and ml.Matrix under spark.ml.python
## What changes were proposed in this pull request?
Now we have PySpark picklers for new and old vector/matrix, individually. However, they are all implemented under `PythonMLlibAPI`. To separate spark.mllib from spark.ml, we should implement the picklers of new vector/matrix under `spark.ml.python` instead.
## How was this patch tested?
Existing tests.
Author: Liang-Chi Hsieh <simonh@tw.ibm.com>
Closes #13219 from viirya/pyspark-pickler-ml.
Diffstat (limited to 'python/pyspark/java_gateway.py')
| -rw-r--r-- | python/pyspark/java_gateway.py | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/python/pyspark/java_gateway.py b/python/pyspark/java_gateway.py index cd4c55f79f..527ca82d31 100644 --- a/python/pyspark/java_gateway.py +++ b/python/pyspark/java_gateway.py @@ -116,6 +116,7 @@ def launch_gateway(): java_import(gateway.jvm, "org.apache.spark.SparkConf") java_import(gateway.jvm, "org.apache.spark.api.java.*") java_import(gateway.jvm, "org.apache.spark.api.python.*") + java_import(gateway.jvm, "org.apache.spark.ml.python.*") java_import(gateway.jvm, "org.apache.spark.mllib.api.python.*") # TODO(davies): move into sql java_import(gateway.jvm, "org.apache.spark.sql.*") |