aboutsummaryrefslogtreecommitdiff
path: root/external
diff options
context:
space:
mode:
authorKay Ousterhout <kayousterhout@gmail.com>2015-02-12 14:35:44 -0800
committerAndrew Or <andrew@databricks.com>2015-02-12 14:36:27 -0800
commit893d6fd7049daf3c4d01eb6a960801cd064d5f73 (patch)
treec7571d8be3859ed07458d25741d7e625cac8804a /external
parentaa4ca8b873fd83e64e5faea6f7febcc830e30b02 (diff)
downloadspark-893d6fd7049daf3c4d01eb6a960801cd064d5f73.tar.gz
spark-893d6fd7049daf3c4d01eb6a960801cd064d5f73.tar.bz2
spark-893d6fd7049daf3c4d01eb6a960801cd064d5f73.zip
[SPARK-5645] Added local read bytes/time to task metrics
ksakellis I stumbled on your JIRA for this yesterday; I know it's assigned to you but I'd already done this for my own uses a while ago so thought I could help save you the work of doing it! Hopefully this doesn't duplicate any work you've already done. Here's a screenshot of what the UI looks like: ![image](https://cloud.githubusercontent.com/assets/1108612/6135352/c03e7276-b11c-11e4-8f11-c6aefe1f35b9.png) Based on a discussion with pwendell, I put the data read remotely in as an additional metric rather than showing it in brackets as you'd suggested, Kostas. The assumption here is that the average user doesn't care about the differentiation between local / remote data, so it's better not to pollute the UI. I also added data about the local read time, which I've found very helpful for debugging, but I didn't put it in the UI because I think it's probably something not a ton of people will need to use. With this change, the total read time and total write time shown in the UI will be equal, fixing a long-term source of user confusion: ![image](https://cloud.githubusercontent.com/assets/1108612/6135399/25f14490-b11d-11e4-8086-20be5f4002e6.png) Author: Kay Ousterhout <kayousterhout@gmail.com> Closes #4510 from kayousterhout/SPARK-5645 and squashes the following commits: 4a0182c [Kay Ousterhout] oops 5f5da1b [Kay Ousterhout] Small style fix 5da04cf [Kay Ousterhout] Addressed more comments from Kostas ba05149 [Kay Ousterhout] Remove parens a9dc685 [Kay Ousterhout] Kostas comment, test fix 33d2e2d [Kay Ousterhout] Merge remote-tracking branch 'upstream/master' into SPARK-5645 347e2cd [Kay Ousterhout] [SPARK-5645] Added local read bytes/time to task metrics
Diffstat (limited to 'external')
0 files changed, 0 insertions, 0 deletions