diff options
| author | Xusen Yin <yinxusen@gmail.com> | 2016-03-16 13:49:40 -0700 |
|---|---|---|
| committer | Joseph K. Bradley <joseph@databricks.com> | 2016-03-16 13:49:40 -0700 |
| commit | ae6c677c8a03174787be99af6238a5e1fbe4e389 (patch) | |
| tree | 75943410b6cfbe50c66ff199ab6164d24edeef84 /python/pyspark/ml/util.py | |
| parent | c4bd57602c0b14188d364bb475631bf473d25082 (diff) | |
| download | spark-ae6c677c8a03174787be99af6238a5e1fbe4e389.tar.gz spark-ae6c677c8a03174787be99af6238a5e1fbe4e389.tar.bz2 spark-ae6c677c8a03174787be99af6238a5e1fbe4e389.zip | |
[SPARK-13038][PYSPARK] Add load/save to pipeline
## What changes were proposed in this pull request?
JIRA issue: https://issues.apache.org/jira/browse/SPARK-13038
1. Add load/save to PySpark Pipeline and PipelineModel
2. Add `_transfer_stage_to_java()` and `_transfer_stage_from_java()` for `JavaWrapper`.
## How was this patch tested?
Test with doctest.
Author: Xusen Yin <yinxusen@gmail.com>
Closes #11683 from yinxusen/SPARK-13038-only.
Diffstat (limited to 'python/pyspark/ml/util.py')
| -rw-r--r-- | python/pyspark/ml/util.py | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/python/pyspark/ml/util.py b/python/pyspark/ml/util.py index d7a813f56c..42801c91bb 100644 --- a/python/pyspark/ml/util.py +++ b/python/pyspark/ml/util.py @@ -161,6 +161,9 @@ class JavaMLReader(object): the Python full class name. """ java_package = clazz.__module__.replace("pyspark", "org.apache.spark") + if clazz.__name__ in ("Pipeline", "PipelineModel"): + # Remove the last package name "pipeline" for Pipeline and PipelineModel. + java_package = ".".join(java_package.split(".")[0:-1]) return ".".join([java_package, clazz.__name__]) @classmethod |