diff options
author | MechCoder <manojkumarsivaraj334@gmail.com> | 2015-07-21 10:31:31 -0700 |
---|---|---|
committer | Joseph K. Bradley <joseph@databricks.com> | 2015-07-21 10:31:31 -0700 |
commit | 89db3c0b6edcffed7e1e12c202e6827271ddba26 (patch) | |
tree | 4940bd42da6119c47fa1331a700565505bba51ea /docs | |
parent | 7f072c3d5ec50c65d76bd9f28fac124fce96a89e (diff) | |
download | spark-89db3c0b6edcffed7e1e12c202e6827271ddba26.tar.gz spark-89db3c0b6edcffed7e1e12c202e6827271ddba26.tar.bz2 spark-89db3c0b6edcffed7e1e12c202e6827271ddba26.zip |
[SPARK-5989] [MLLIB] Model save/load for LDA
Add support for saving and loading LDA both the local and distributed versions.
Author: MechCoder <manojkumarsivaraj334@gmail.com>
Closes #6948 from MechCoder/lda_save_load and squashes the following commits:
49bcdce [MechCoder] minor style fixes
cc14054 [MechCoder] minor
4587d1d [MechCoder] Minor changes
c753122 [MechCoder] Load and save the model in private methods
2782326 [MechCoder] [SPARK-5989] Model save/load for LDA
Diffstat (limited to 'docs')
-rw-r--r-- | docs/mllib-clustering.md | 10 |
1 files changed, 9 insertions, 1 deletions
diff --git a/docs/mllib-clustering.md b/docs/mllib-clustering.md index 0fc7036bff..bb875ae2ae 100644 --- a/docs/mllib-clustering.md +++ b/docs/mllib-clustering.md @@ -472,7 +472,7 @@ to the algorithm. We then output the topics, represented as probability distribu <div data-lang="scala" markdown="1"> {% highlight scala %} -import org.apache.spark.mllib.clustering.LDA +import org.apache.spark.mllib.clustering.{LDA, DistributedLDAModel} import org.apache.spark.mllib.linalg.Vectors // Load and parse the data @@ -492,6 +492,11 @@ for (topic <- Range(0, 3)) { for (word <- Range(0, ldaModel.vocabSize)) { print(" " + topics(word, topic)); } println() } + +// Save and load model. +ldaModel.save(sc, "myLDAModel") +val sameModel = DistributedLDAModel.load(sc, "myLDAModel") + {% endhighlight %} </div> @@ -551,6 +556,9 @@ public class JavaLDAExample { } System.out.println(); } + + ldaModel.save(sc.sc(), "myLDAModel"); + DistributedLDAModel sameModel = DistributedLDAModel.load(sc.sc(), "myLDAModel"); } } {% endhighlight %} |