diff options
| author | Kay Ousterhout <kayousterhout@gmail.com> | 2014-11-05 15:30:31 -0800 |
|---|---|---|
| committer | Kay Ousterhout <kayousterhout@gmail.com> | 2014-11-05 15:30:31 -0800 |
| commit | a46497eecc50f854c5c5701dc2b8a2468b76c085 (patch) | |
| tree | 65e8b8040442c42ed751334076390ad7e3186c57 /pom.xml | |
| parent | 4c42986cc070d9c5c55c7bf8a2a67585967b1082 (diff) | |
| download | spark-a46497eecc50f854c5c5701dc2b8a2468b76c085.tar.gz spark-a46497eecc50f854c5c5701dc2b8a2468b76c085.tar.bz2 spark-a46497eecc50f854c5c5701dc2b8a2468b76c085.zip | |
[SPARK-3984] [SPARK-3983] Fix incorrect scheduler delay and display task deserialization time in UI
This commit fixes the scheduler delay in the UI (which previously
included things that are not scheduler delay, like time to
deserialize the task and serialize the result), and also
adds information about time to deserialize tasks to the optional
additional metrics. Time to deserialize the task can be large relative
to task time for short jobs, and understanding when it is high can help
developers realize that they should try to reduce closure size (e.g, by including
less data in the task description).
cc shivaram etrain
Author: Kay Ousterhout <kayousterhout@gmail.com>
Closes #2832 from kayousterhout/SPARK-3983 and squashes the following commits:
0c1398e [Kay Ousterhout] Fixed ordering
531575d [Kay Ousterhout] Removed executor launch time
1f13afe [Kay Ousterhout] Minor spacing fixes
335be4b [Kay Ousterhout] Made metrics hideable
5bc3cba [Kay Ousterhout] [SPARK-3984] [SPARK-3983] Improve UI task metrics.
Diffstat (limited to 'pom.xml')
0 files changed, 0 insertions, 0 deletions