diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-10-06 16:51:03 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-10-06 16:51:03 -0700 |
| commit | ffe6831e49e28eb855f857fdfa5dd99341e80c9d (patch) | |
| tree | ee64cd7d62505fbe44905d9b3e4791e845e50925 /docs/mllib-dimensionality-reduction.md | |
| parent | 5e035403d41527f092705c68f0dd1b4b946014d6 (diff) | |
| download | spark-ffe6831e49e28eb855f857fdfa5dd99341e80c9d.tar.gz spark-ffe6831e49e28eb855f857fdfa5dd99341e80c9d.tar.bz2 spark-ffe6831e49e28eb855f857fdfa5dd99341e80c9d.zip | |
[SPARK-10885] [STREAMING] Display the failed output op in Streaming UI
This PR implements the following features for both `master` and `branch-1.5`.
1. Display the failed output op count in the batch list
2. Display the failure reason of output op in the batch detail page
Screenshots:
<img width="1356" alt="1" src="https://cloud.githubusercontent.com/assets/1000778/10198387/5b2b97ec-67ce-11e5-81c2-f818b9d2f3ad.png">
<img width="1356" alt="2" src="https://cloud.githubusercontent.com/assets/1000778/10198388/5b76ac14-67ce-11e5-8c8b-de2683c5b485.png">
There are still two remaining problems in the UI.
1. If an output operation doesn't run any spark job, we cannot get the its duration since now it's the sum of all jobs' durations.
2. If an output operation doesn't run any spark job, we cannot get the description since it's the latest job's call site.
We need to add new `StreamingListenerEvent` about output operations to fix them. So I'd like to fix them only for `master` in another PR.
Author: zsxwing <zsxwing@gmail.com>
Closes #8950 from zsxwing/batch-failure.
Diffstat (limited to 'docs/mllib-dimensionality-reduction.md')
0 files changed, 0 insertions, 0 deletions