diff options
| author | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2014-10-16 19:43:33 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@apache.org> | 2014-10-16 19:43:33 -0700 |
| commit | e678b9f02a2936b35c95e91a5f0ff388b5720261 (patch) | |
| tree | bb27d935071a0165505ce785dc8bdedda8c67c6a | |
| parent | dedace83f35cba0f833d962acbd75572318948c4 (diff) | |
| download | spark-e678b9f02a2936b35c95e91a5f0ff388b5720261.tar.gz spark-e678b9f02a2936b35c95e91a5f0ff388b5720261.tar.bz2 spark-e678b9f02a2936b35c95e91a5f0ff388b5720261.zip | |
[SPARK-3973] Print call site information for broadcasts
Its hard to debug which broadcast variables refer to what in a big codebase. Printing call site information helps in debugging.
Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
Closes #2829 from shivaram/spark-broadcast-print and squashes the following commits:
cd6dbdf [Shivaram Venkataraman] Print call site information for broadcasts
| -rw-r--r-- | core/src/main/scala/org/apache/spark/SparkContext.scala | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/SparkContext.scala b/core/src/main/scala/org/apache/spark/SparkContext.scala index 354116286c..dd3157990e 100644 --- a/core/src/main/scala/org/apache/spark/SparkContext.scala +++ b/core/src/main/scala/org/apache/spark/SparkContext.scala @@ -818,6 +818,8 @@ class SparkContext(config: SparkConf) extends Logging { */ def broadcast[T: ClassTag](value: T): Broadcast[T] = { val bc = env.broadcastManager.newBroadcast[T](value, isLocal) + val callSite = getCallSite + logInfo("Created broadcast " + bc.id + " from " + callSite.shortForm) cleaner.foreach(_.registerBroadcastForCleanup(bc)) bc } |