diff options
| author | GayathriMurali <gayathri.m@intel.com> | 2016-06-24 13:25:40 +0200 |
|---|---|---|
| committer | Nick Pentreath <nickp@za.ibm.com> | 2016-06-24 13:25:40 +0200 |
| commit | be88383e15a86d094963de5f7e8792510bc990de (patch) | |
| tree | 3e4c7b8101ea1ea6a919f283e6d31b47fbee96c4 /examples/src/main/java | |
| parent | 158af162eac7348464c6751c8acd48fc6c117688 (diff) | |
| download | spark-be88383e15a86d094963de5f7e8792510bc990de.tar.gz spark-be88383e15a86d094963de5f7e8792510bc990de.tar.bz2 spark-be88383e15a86d094963de5f7e8792510bc990de.zip | |
[SPARK-15997][DOC][ML] Update user guide for HashingTF, QuantileVectorizer and CountVectorizer
## What changes were proposed in this pull request?
Made changes to HashingTF,QuantileVectorizer and CountVectorizer
Author: GayathriMurali <gayathri.m@intel.com>
Closes #13745 from GayathriMurali/SPARK-15997.
Diffstat (limited to 'examples/src/main/java')
| -rw-r--r-- | examples/src/main/java/org/apache/spark/examples/ml/JavaQuantileDiscretizerExample.java | 7 |
1 files changed, 6 insertions, 1 deletions
diff --git a/examples/src/main/java/org/apache/spark/examples/ml/JavaQuantileDiscretizerExample.java b/examples/src/main/java/org/apache/spark/examples/ml/JavaQuantileDiscretizerExample.java index 16f58a852d..dd20cac621 100644 --- a/examples/src/main/java/org/apache/spark/examples/ml/JavaQuantileDiscretizerExample.java +++ b/examples/src/main/java/org/apache/spark/examples/ml/JavaQuantileDiscretizerExample.java @@ -54,7 +54,12 @@ public class JavaQuantileDiscretizerExample { }); Dataset<Row> df = spark.createDataFrame(data, schema); - + // $example off$ + // Output of QuantileDiscretizer for such small datasets can depend on the number of + // partitions. Here we force a single partition to ensure consistent results. + // Note this is not necessary for normal use cases + df = df.repartition(1); + // $example on$ QuantileDiscretizer discretizer = new QuantileDiscretizer() .setInputCol("hour") .setOutputCol("result") |