diff options
| author | Erik Erlandson <eerlands@redhat.com> | 2014-08-08 20:58:44 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-08-08 20:58:44 -0700 |
| commit | 43af2817007eaa2cce2567bd83f5cde1ee28d1f7 (patch) | |
| tree | cd8316e124465c09a72e44071dee977f2e6fd9ff | |
| parent | 1c84dba9881118204687c81003bded6d49e27255 (diff) | |
| download | spark-43af2817007eaa2cce2567bd83f5cde1ee28d1f7.tar.gz spark-43af2817007eaa2cce2567bd83f5cde1ee28d1f7.tar.bz2 spark-43af2817007eaa2cce2567bd83f5cde1ee28d1f7.zip | |
[SPARK-2911] apply parent[T](j) to clarify UnionRDD code
References to dependencies(j) for actually obtaining RDD parents are less common than I originally estimated. It does clarify UnionRDD (also will clarify some of my other PRs)
Use of firstParent[T] is ubiquitous, but not as sure that benefits from being replaced with parent(0)[T].
Author: Erik Erlandson <eerlands@redhat.com>
Closes #1858 from erikerlandson/spark-2911-pr2 and squashes the following commits:
7ffea74 [Erik Erlandson] [SPARK-2911] apply parent[T](j) to clarify UnionRDD code
| -rw-r--r-- | core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala | 3 |
1 files changed, 1 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala b/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala index 197167ecad..0c97eb0aaa 100644 --- a/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala +++ b/core/src/main/scala/org/apache/spark/rdd/UnionRDD.scala @@ -83,8 +83,7 @@ class UnionRDD[T: ClassTag]( override def compute(s: Partition, context: TaskContext): Iterator[T] = { val part = s.asInstanceOf[UnionPartition[T]] - val parentRdd = dependencies(part.parentRddIndex).rdd.asInstanceOf[RDD[T]] - parentRdd.iterator(part.parentPartition, context) + parent[T](part.parentRddIndex).iterator(part.parentPartition, context) } override def getPreferredLocations(s: Partition): Seq[String] = |