diff options
author | krishnakalyan3 <krishnakalyan3@gmail.com> | 2016-07-27 15:37:38 +0200 |
---|---|---|
committer | Nick Pentreath <nickp@za.ibm.com> | 2016-07-27 15:37:38 +0200 |
commit | 7e8279fde176b08687adf2b410693b35cfbd4b46 (patch) | |
tree | 9342eb9868f5f5a1ed7358a8883605c31a15d24d /mllib/src/main | |
parent | 045fc3606698b017a4addf5277808883e6fe76b6 (diff) | |
download | spark-7e8279fde176b08687adf2b410693b35cfbd4b46.tar.gz spark-7e8279fde176b08687adf2b410693b35cfbd4b46.tar.bz2 spark-7e8279fde176b08687adf2b410693b35cfbd4b46.zip |
[SPARK-15254][DOC] Improve ML pipeline Cross Validation Scaladoc & PyDoc
## What changes were proposed in this pull request?
Updated ML pipeline Cross Validation Scaladoc & PyDoc.
## How was this patch tested?
Documentation update
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Author: krishnakalyan3 <krishnakalyan3@gmail.com>
Closes #13894 from krishnakalyan3/kfold-cv.
Diffstat (limited to 'mllib/src/main')
-rw-r--r-- | mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala | 10 |
1 files changed, 8 insertions, 2 deletions
diff --git a/mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala b/mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala index 520557849b..6ea52ef7f0 100644 --- a/mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala +++ b/mllib/src/main/scala/org/apache/spark/ml/tuning/CrossValidator.scala @@ -55,7 +55,11 @@ private[ml] trait CrossValidatorParams extends ValidatorParams { } /** - * K-fold cross validation. + * K-fold cross validation performs model selection by splitting the dataset into a set of + * non-overlapping randomly partitioned folds which are used as separate training and test datasets + * e.g., with k=3 folds, K-fold cross validation will generate 3 (training, test) dataset pairs, + * each of which uses 2/3 of the data for training and 1/3 for testing. Each fold is used as the + * test set exactly once. */ @Since("1.2.0") class CrossValidator @Since("1.2.0") (@Since("1.4.0") override val uid: String) @@ -188,7 +192,9 @@ object CrossValidator extends MLReadable[CrossValidator] { } /** - * Model from k-fold cross validation. + * CrossValidatorModel contains the model with the highest average cross-validation + * metric across folds and uses this model to transform input data. CrossValidatorModel + * also tracks the metrics for each param map evaluated. * * @param bestModel The best model selected from k-fold cross validation. * @param avgMetrics Average cross-validation metrics for each paramMap in |