diff options
| author | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2014-10-28 10:14:16 -0700 |
|---|---|---|
| committer | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2014-10-28 10:14:16 -0700 |
| commit | 7768a800d4c4c19d76cef1ee40af6900bbac821c (patch) | |
| tree | af70427d0d20f1e33056d288bff271fa8219ee63 /sbin | |
| parent | 0ac52e30552530b247e37a470b8503346f19605c (diff) | |
| download | spark-7768a800d4c4c19d76cef1ee40af6900bbac821c.tar.gz spark-7768a800d4c4c19d76cef1ee40af6900bbac821c.tar.bz2 spark-7768a800d4c4c19d76cef1ee40af6900bbac821c.zip | |
[SPARK-4031] Make torrent broadcast read blocks on use.
This avoids reading torrent broadcast variables when they are referenced in the closure but not used in the closure. This is done by using a `lazy val` to read broadcast blocks
cc rxin JoshRosen for review
Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
Closes #2871 from shivaram/broadcast-read-value and squashes the following commits:
1456d65 [Shivaram Venkataraman] Use getUsedTimeMs and remove readObject
d6c5ee9 [Shivaram Venkataraman] Use laxy val to implement readBroadcastBlock
0b34df7 [Shivaram Venkataraman] Merge branch 'master' of https://github.com/apache/spark into broadcast-read-value
9cec507 [Shivaram Venkataraman] Test if broadcast variables are read lazily
768b40b [Shivaram Venkataraman] Merge branch 'master' of https://github.com/apache/spark into broadcast-read-value
8792ed8 [Shivaram Venkataraman] Make torrent broadcast read blocks on use. This avoids reading broadcast variables when they are referenced in the closure but not used by the code.
Diffstat (limited to 'sbin')
0 files changed, 0 insertions, 0 deletions