diff options
author | Reynold Xin <rxin@databricks.com> | 2016-12-06 11:48:11 -0800 |
---|---|---|
committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-12-06 11:48:11 -0800 |
commit | cb1f10b468e7771af75cb2288d375a87ab66d316 (patch) | |
tree | 2ab0ed2cb0157af151c8cdcda84b5f810585c784 | |
parent | 05d416ffc616930d05b59bce0ca6cbd682f8b5bc (diff) | |
download | spark-cb1f10b468e7771af75cb2288d375a87ab66d316.tar.gz spark-cb1f10b468e7771af75cb2288d375a87ab66d316.tar.bz2 spark-cb1f10b468e7771af75cb2288d375a87ab66d316.zip |
[SPARK-18714][SQL] Add a simple time function to SparkSession
## What changes were proposed in this pull request?
Many Spark developers often want to test the runtime of some function in interactive debugging and testing. This patch adds a simple time function to SparkSession:
```
scala> spark.time { spark.range(1000).count() }
Time taken: 77 ms
res1: Long = 1000
```
## How was this patch tested?
I tested this interactively in spark-shell.
Author: Reynold Xin <rxin@databricks.com>
Closes #16140 from rxin/SPARK-18714.
-rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala | 16 |
1 files changed, 16 insertions, 0 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala b/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala index 08d74ac018..f3dde480ea 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala @@ -618,6 +618,22 @@ class SparkSession private( @InterfaceStability.Evolving def readStream: DataStreamReader = new DataStreamReader(self) + /** + * Executes some code block and prints to stdout the time taken to execute the block. This is + * available in Scala only and is used primarily for interactive testing and debugging. + * + * @since 2.1.0 + */ + @InterfaceStability.Stable + def time[T](f: => T): T = { + val start = System.nanoTime() + val ret = f + val end = System.nanoTime() + // scalastyle:off println + println(s"Time taken: ${(end - start) / 1000 / 1000} ms") + // scalastyle:on println + ret + } // scalastyle:off // Disable style checker so "implicits" object can start with lowercase i |