diff options
| author | Xiangrui Meng <meng@databricks.com> | 2014-07-23 00:58:55 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-07-23 00:58:55 -0700 |
| commit | 4c7243e109c713bdfb87891748800109ffbaae07 (patch) | |
| tree | 4e280534354b05c0a313336e51b9555e25d8c6d1 /core/src/main/resources/org/apache | |
| parent | 6c2be93f081f33e9e97e1231b0084a6a0eb4fa22 (diff) | |
| download | spark-4c7243e109c713bdfb87891748800109ffbaae07.tar.gz spark-4c7243e109c713bdfb87891748800109ffbaae07.tar.bz2 spark-4c7243e109c713bdfb87891748800109ffbaae07.zip | |
[SPARK-2617] Correct doc and usages of preservesPartitioning
The name `preservesPartitioning` is ambiguous: 1) preserves the indices of partitions, 2) preserves the partitioner. The latter is correct and `preservesPartitioning` should really be called `preservesPartitioner` to avoid confusion. Unfortunately, this is already part of the API and we cannot change. We should be clear in the doc and fix wrong usages.
This PR
1. adds notes in `maPartitions*`,
2. makes `RDD.sample` preserve partitioner,
3. changes `preservesPartitioning` to false in `RDD.zip` because the keys of the first RDD are no longer the keys of the zipped RDD,
4. fixes some wrong usages in MLlib.
Author: Xiangrui Meng <meng@databricks.com>
Closes #1526 from mengxr/preserve-partitioner and squashes the following commits:
b361e65 [Xiangrui Meng] update doc based on pwendell's comments
3b1ba19 [Xiangrui Meng] update doc
357575c [Xiangrui Meng] fix unit test
20b4816 [Xiangrui Meng] Merge branch 'master' into preserve-partitioner
d1caa65 [Xiangrui Meng] add doc to explain preservesPartitioning fix wrong usage of preservesPartitioning make sample preserse partitioning
Diffstat (limited to 'core/src/main/resources/org/apache')
0 files changed, 0 insertions, 0 deletions