diff options
| author | Zheng RuiFeng <ruifengz@foxmail.com> | 2017-01-28 10:18:47 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2017-01-28 10:18:47 +0000 |
| commit | 42ad93b2c9047a68c14cbf681508157101f43c0e (patch) | |
| tree | 691ec59e88d2a35c91b0866b80333d4edc2f66d3 /mllib/src | |
| parent | 1b5ee2003c368d18a5f8c17c2a869ef5770c60a1 (diff) | |
| download | spark-42ad93b2c9047a68c14cbf681508157101f43c0e.tar.gz spark-42ad93b2c9047a68c14cbf681508157101f43c0e.tar.bz2 spark-42ad93b2c9047a68c14cbf681508157101f43c0e.zip | |
[SPARK-19384][ML] forget unpersist input dataset in IsotonicRegression
## What changes were proposed in this pull request?
unpersist the input dataset if `handlePersistence` = true
## How was this patch tested?
existing tests
Author: Zheng RuiFeng <ruifengz@foxmail.com>
Closes #16718 from zhengruifeng/isoReg_unpersisit.
Diffstat (limited to 'mllib/src')
| -rw-r--r-- | mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala b/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala index 90e77bc76e..a6c29433d7 100644 --- a/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala +++ b/mllib/src/main/scala/org/apache/spark/ml/regression/IsotonicRegression.scala @@ -175,6 +175,8 @@ class IsotonicRegression @Since("1.5.0") (@Since("1.5.0") override val uid: Stri val isotonicRegression = new MLlibIsotonicRegression().setIsotonic($(isotonic)) val oldModel = isotonicRegression.run(instances) + if (handlePersistence) instances.unpersist() + val model = copyValues(new IsotonicRegressionModel(uid, oldModel).setParent(this)) instr.logSuccess(model) model |