diff options
| author | Sean Owen <sowen@cloudera.com> | 2016-02-19 10:26:38 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-02-19 10:26:38 +0000 |
| commit | fb7e21797ed618d9754545a44f8f95f75b66757a (patch) | |
| tree | 42b592cf1f25aeaf067c35afd75f9a3403182b99 /R | |
| parent | c776fce99b496a789ffcf2cfab78cf51eeea032b (diff) | |
| download | spark-fb7e21797ed618d9754545a44f8f95f75b66757a.tar.gz spark-fb7e21797ed618d9754545a44f8f95f75b66757a.tar.bz2 spark-fb7e21797ed618d9754545a44f8f95f75b66757a.zip | |
[SPARK-13339][DOCS] Clarify commutative / associative operator requirements for reduce, fold
Clarify that reduce functions need to be commutative, and fold functions do not
See https://github.com/apache/spark/pull/11091
Author: Sean Owen <sowen@cloudera.com>
Closes #11217 from srowen/SPARK-13339.
Diffstat (limited to 'R')
| -rw-r--r-- | R/pkg/R/pairRDD.R | 10 |
1 files changed, 5 insertions, 5 deletions
diff --git a/R/pkg/R/pairRDD.R b/R/pkg/R/pairRDD.R index f7131140fe..4075ef4377 100644 --- a/R/pkg/R/pairRDD.R +++ b/R/pkg/R/pairRDD.R @@ -305,11 +305,11 @@ setMethod("groupByKey", #' Merge values by key #' #' This function operates on RDDs where every element is of the form list(K, V) or c(K, V). -#' and merges the values for each key using an associative reduce function. +#' and merges the values for each key using an associative and commutative reduce function. #' #' @param x The RDD to reduce by key. Should be an RDD where each element is #' list(K, V) or c(K, V). -#' @param combineFunc The associative reduce function to use. +#' @param combineFunc The associative and commutative reduce function to use. #' @param numPartitions Number of partitions to create. #' @return An RDD where each element is list(K, V') where V' is the merged #' value @@ -347,12 +347,12 @@ setMethod("reduceByKey", #' Merge values by key locally #' #' This function operates on RDDs where every element is of the form list(K, V) or c(K, V). -#' and merges the values for each key using an associative reduce function, but return the -#' results immediately to the driver as an R list. +#' and merges the values for each key using an associative and commutative reduce function, but +#' return the results immediately to the driver as an R list. #' #' @param x The RDD to reduce by key. Should be an RDD where each element is #' list(K, V) or c(K, V). -#' @param combineFunc The associative reduce function to use. +#' @param combineFunc The associative and commutative reduce function to use. #' @return A list of elements of type list(K, V') where V' is the merged value for each key #' @seealso reduceByKey #' @examples |