diff options
author | Reynold Xin <rxin@databricks.com> | 2017-03-29 00:02:15 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2017-03-29 00:02:15 -0700 |
commit | 9712bd3954c029de5c828f27b57d46e4a6325a38 (patch) | |
tree | fbd8403b5c40ca234009f03913837121cd6be4cf /sql/core/src/test | |
parent | a5c87707eaec5cacdfb703eb396dfc264bc54cda (diff) | |
download | spark-9712bd3954c029de5c828f27b57d46e4a6325a38.tar.gz spark-9712bd3954c029de5c828f27b57d46e4a6325a38.tar.bz2 spark-9712bd3954c029de5c828f27b57d46e4a6325a38.zip |
[SPARK-20134][SQL] SQLMetrics.postDriverMetricUpdates to simplify driver side metric updates
## What changes were proposed in this pull request?
It is not super intuitive how to update SQLMetric on the driver side. This patch introduces a new SQLMetrics.postDriverMetricUpdates function to do that, and adds documentation to make it more obvious.
## How was this patch tested?
Updated a test case to use this method.
Author: Reynold Xin <rxin@databricks.com>
Closes #17464 from rxin/SPARK-20134.
Diffstat (limited to 'sql/core/src/test')
-rw-r--r-- | sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala | 8 |
1 files changed, 5 insertions, 3 deletions
diff --git a/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala b/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala index e41c00ecec..e6cd41e4fa 100644 --- a/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala +++ b/sql/core/src/test/scala/org/apache/spark/sql/execution/ui/SQLListenerSuite.scala @@ -477,9 +477,11 @@ private case class MyPlan(sc: SparkContext, expectedValue: Long) extends LeafExe override def doExecute(): RDD[InternalRow] = { longMetric("dummy") += expectedValue - sc.listenerBus.post(SparkListenerDriverAccumUpdates( - sc.getLocalProperty(SQLExecution.EXECUTION_ID_KEY).toLong, - metrics.values.map(m => m.id -> m.value).toSeq)) + + SQLMetrics.postDriverMetricUpdates( + sc, + sc.getLocalProperty(SQLExecution.EXECUTION_ID_KEY), + metrics.values.toSeq) sc.emptyRDD } } |