diff options
| author | Ryan Blue <blue@apache.org> | 2017-03-31 09:42:49 -0700 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2017-03-31 09:42:49 -0700 |
| commit | c4c03eed67c05a78dc8944f6119ea708d6b955be (patch) | |
| tree | 8f50bc293129fa85fc104e29dd260f045e9bf6f3 /sbin | |
| parent | 254877c2f04414c70d92fa0a00c0ecee1d73aba7 (diff) | |
| download | spark-c4c03eed67c05a78dc8944f6119ea708d6b955be.tar.gz spark-c4c03eed67c05a78dc8944f6119ea708d6b955be.tar.bz2 spark-c4c03eed67c05a78dc8944f6119ea708d6b955be.zip | |
[SPARK-20084][CORE] Remove internal.metrics.updatedBlockStatuses from history files.
## What changes were proposed in this pull request?
Remove accumulator updates for internal.metrics.updatedBlockStatuses from SparkListenerTaskEnd entries in the history file. These can cause history files to grow to hundreds of GB because the value of the accumulator contains all tracked blocks.
## How was this patch tested?
Current History UI tests cover use of the history file.
Author: Ryan Blue <blue@apache.org>
Closes #17412 from rdblue/SPARK-20084-remove-block-accumulator-info.
Diffstat (limited to 'sbin')
0 files changed, 0 insertions, 0 deletions