diff options
| author | Aaron Davidson <aaron@databricks.com> | 2014-05-09 14:51:34 -0700 |
|---|---|---|
| committer | Aaron Davidson <aaron@databricks.com> | 2014-05-09 14:51:34 -0700 |
| commit | 59577df14c06417676a9ffdd599f5713c448e299 (patch) | |
| tree | 178e51adb447421a30822fbd8fb1ebc87d6fa040 | |
| parent | bd67551ee724fd7cce4f2e2977a862216c992ef5 (diff) | |
| download | spark-59577df14c06417676a9ffdd599f5713c448e299.tar.gz spark-59577df14c06417676a9ffdd599f5713c448e299.tar.bz2 spark-59577df14c06417676a9ffdd599f5713c448e299.zip | |
SPARK-1770: Revert accidental(?) fix
Looks like this change was accidentally committed here: https://github.com/apache/spark/commit/06b15baab25951d124bbe6b64906f4139e037deb
but the change does not show up in the PR itself (#704).
Other than not intending to go in with that PR, this also broke the test JavaAPISuite.repartition.
Author: Aaron Davidson <aaron@databricks.com>
Closes #716 from aarondav/shufflerand and squashes the following commits:
b1cf70b [Aaron Davidson] SPARK-1770: Revert accidental(?) fix
| -rw-r--r-- | core/src/main/scala/org/apache/spark/rdd/RDD.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/rdd/RDD.scala b/core/src/main/scala/org/apache/spark/rdd/RDD.scala index 9d8d8044f0..a1ca612cc9 100644 --- a/core/src/main/scala/org/apache/spark/rdd/RDD.scala +++ b/core/src/main/scala/org/apache/spark/rdd/RDD.scala @@ -330,9 +330,9 @@ abstract class RDD[T: ClassTag]( if (shuffle) { // include a shuffle step so that our upstream tasks are still distributed new CoalescedRDD( - new ShuffledRDD[Int, T, (Int, T)](map(x => (Utils.random.nextInt(), x)), + new ShuffledRDD[T, Null, (T, Null)](map(x => (x, null)), new HashPartitioner(numPartitions)), - numPartitions).values + numPartitions).keys } else { new CoalescedRDD(this, numPartitions) } |