diff options
| author | Reynold Xin <rxin@apache.org> | 2013-12-17 18:44:39 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2013-12-17 18:44:39 -0800 |
| commit | 9a6864d016f0f923c885ca2a4977104cd19ded4f (patch) | |
| tree | a5ab5eae51c0f393efa3a415ee1a255bba9d6924 | |
| parent | 7a8169be9a0b6b3d0d53a98aa38940d47b201296 (diff) | |
| download | spark-9a6864d016f0f923c885ca2a4977104cd19ded4f.tar.gz spark-9a6864d016f0f923c885ca2a4977104cd19ded4f.tar.bz2 spark-9a6864d016f0f923c885ca2a4977104cd19ded4f.zip | |
Fixed a performance problem in RDD.top and BoundedPriorityQueue (size in BoundedPriority was actually traversing the entire queue to calculate the size, resulting in bad performance in insertion).
| -rw-r--r-- | core/src/main/scala/org/apache/spark/util/BoundedPriorityQueue.scala | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/util/BoundedPriorityQueue.scala b/core/src/main/scala/org/apache/spark/util/BoundedPriorityQueue.scala index 0b51c23f7b..a38329df03 100644 --- a/core/src/main/scala/org/apache/spark/util/BoundedPriorityQueue.scala +++ b/core/src/main/scala/org/apache/spark/util/BoundedPriorityQueue.scala @@ -34,6 +34,8 @@ class BoundedPriorityQueue[A](maxSize: Int)(implicit ord: Ordering[A]) override def iterator: Iterator[A] = underlying.iterator.asScala + override def size: Int = underlying.size + override def ++=(xs: TraversableOnce[A]): this.type = { xs.foreach { this += _ } this |