aboutsummaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorReynold Xin <rxin@databricks.com>2016-12-06 11:48:11 -0800
committerHerman van Hovell <hvanhovell@databricks.com>2016-12-06 11:48:11 -0800
commitcb1f10b468e7771af75cb2288d375a87ab66d316 (patch)
tree2ab0ed2cb0157af151c8cdcda84b5f810585c784
parent05d416ffc616930d05b59bce0ca6cbd682f8b5bc (diff)
downloadspark-cb1f10b468e7771af75cb2288d375a87ab66d316.tar.gz
spark-cb1f10b468e7771af75cb2288d375a87ab66d316.tar.bz2
spark-cb1f10b468e7771af75cb2288d375a87ab66d316.zip
[SPARK-18714][SQL] Add a simple time function to SparkSession
## What changes were proposed in this pull request? Many Spark developers often want to test the runtime of some function in interactive debugging and testing. This patch adds a simple time function to SparkSession: ``` scala> spark.time { spark.range(1000).count() } Time taken: 77 ms res1: Long = 1000 ``` ## How was this patch tested? I tested this interactively in spark-shell. Author: Reynold Xin <rxin@databricks.com> Closes #16140 from rxin/SPARK-18714.
-rw-r--r--sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala16
1 files changed, 16 insertions, 0 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala b/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
index 08d74ac018..f3dde480ea 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala
@@ -618,6 +618,22 @@ class SparkSession private(
@InterfaceStability.Evolving
def readStream: DataStreamReader = new DataStreamReader(self)
+ /**
+ * Executes some code block and prints to stdout the time taken to execute the block. This is
+ * available in Scala only and is used primarily for interactive testing and debugging.
+ *
+ * @since 2.1.0
+ */
+ @InterfaceStability.Stable
+ def time[T](f: => T): T = {
+ val start = System.nanoTime()
+ val ret = f
+ val end = System.nanoTime()
+ // scalastyle:off println
+ println(s"Time taken: ${(end - start) / 1000 / 1000} ms")
+ // scalastyle:on println
+ ret
+ }
// scalastyle:off
// Disable style checker so "implicits" object can start with lowercase i