diff options
| author | Aaron Davidson <aaron@databricks.com> | 2014-02-20 16:46:13 -0800 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-02-20 16:46:13 -0800 |
| commit | 3fede4831eeb7d36d4f8fa4aaa02ad0cc8b4b09e (patch) | |
| tree | 5cfaa28d5e71ca38533979556e150a476ca885f4 /core | |
| parent | 9e63f80e75bb6d9bbe6df268908c3219de6852d9 (diff) | |
| download | spark-3fede4831eeb7d36d4f8fa4aaa02ad0cc8b4b09e.tar.gz spark-3fede4831eeb7d36d4f8fa4aaa02ad0cc8b4b09e.tar.bz2 spark-3fede4831eeb7d36d4f8fa4aaa02ad0cc8b4b09e.zip | |
Super minor: Add require for mergeCombiners in combineByKey
We changed the behavior in 0.9.0 from requiring that mergeCombiners be null when mapSideCombine was false to requiring that mergeCombiners *never* be null, for external sorting. This patch adds a require() to make this behavior change explicitly messaged rather than resulting in a NPE.
Author: Aaron Davidson <aaron@databricks.com>
Closes #623 from aarondav/master and squashes the following commits:
520b80c [Aaron Davidson] Super minor: Add require for mergeCombiners in combineByKey
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala b/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala index 15bec39659..39c3a4996c 100644 --- a/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala +++ b/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala @@ -77,6 +77,7 @@ class PairRDDFunctions[K: ClassTag, V: ClassTag](self: RDD[(K, V)]) partitioner: Partitioner, mapSideCombine: Boolean = true, serializerClass: String = null): RDD[(K, C)] = { + require(mergeCombiners != null, "mergeCombiners must be defined") // required as of Spark 0.9.0 if (getKeyClass().isArray) { if (mapSideCombine) { throw new SparkException("Cannot use map-side combining with array keys.") |