diff options
| author | proflin <proflin.me@gmail.com> | 2015-12-15 20:22:56 -0800 |
|---|---|---|
| committer | Shixiong Zhu <shixiong@databricks.com> | 2015-12-15 20:22:56 -0800 |
| commit | d52bf47e13e0186590437f71040100d2f6f11da9 (patch) | |
| tree | cdcbbe1c67e56e25279ecef985dbb7b92e074dc7 /streaming | |
| parent | ca0690b5ef10b14ce57a0c30d5308eb02f163f39 (diff) | |
| download | spark-d52bf47e13e0186590437f71040100d2f6f11da9.tar.gz spark-d52bf47e13e0186590437f71040100d2f6f11da9.tar.bz2 spark-d52bf47e13e0186590437f71040100d2f6f11da9.zip | |
[SPARK-12304][STREAMING] Make Spark Streaming web UI display more fri…
…endly Receiver graphs
Currently, the Spark Streaming web UI uses the same maxY when displays 'Input Rate Times& Histograms' and 'Per-Receiver Times& Histograms'.
This may lead to somewhat un-friendly graphs: once we have tens of Receivers or more, every 'Per-Receiver Times' line almost hits the ground.
This issue proposes to calculate a new maxY against the original one, which is shared among all the `Per-Receiver Times& Histograms' graphs.
Before:
After:
Author: proflin <proflin.me@gmail.com>
Closes #10318 from proflin/SPARK-12304.
Diffstat (limited to 'streaming')
| -rw-r--r-- | streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala | 8 |
1 files changed, 7 insertions, 1 deletions
diff --git a/streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala b/streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala index 88a4483e80..b3692c3ea3 100644 --- a/streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala +++ b/streaming/src/main/scala/org/apache/spark/streaming/ui/StreamingPage.scala @@ -392,9 +392,15 @@ private[ui] class StreamingPage(parent: StreamingTab) maxX: Long, minY: Double, maxY: Double): Seq[Node] = { + val maxYCalculated = listener.receivedEventRateWithBatchTime.values + .flatMap { case streamAndRates => streamAndRates.map { case (_, eventRate) => eventRate } } + .reduceOption[Double](math.max) + .map(_.ceil.toLong) + .getOrElse(0L) + val content = listener.receivedEventRateWithBatchTime.toList.sortBy(_._1).map { case (streamId, eventRates) => - generateInputDStreamRow(jsCollector, streamId, eventRates, minX, maxX, minY, maxY) + generateInputDStreamRow(jsCollector, streamId, eventRates, minX, maxX, minY, maxYCalculated) }.foldLeft[Seq[Node]](Nil)(_ ++ _) // scalastyle:off |