diff options
author | Andrew Or <andrew@databricks.com> | 2015-07-09 13:25:11 -0700 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-07-09 13:25:11 -0700 |
commit | ebdf58538058e57381c04b6725d4be0c37847ed3 (patch) | |
tree | a2fdad7fdc56cc74fa05d7e5dafb2896c3e733fa /external/kafka | |
parent | 88bf430331eef3c02438ca441616034486e15789 (diff) | |
download | spark-ebdf58538058e57381c04b6725d4be0c37847ed3.tar.gz spark-ebdf58538058e57381c04b6725d4be0c37847ed3.tar.bz2 spark-ebdf58538058e57381c04b6725d4be0c37847ed3.zip |
[SPARK-2017] [UI] Stage page hangs with many tasks
(This reopens a patch that was closed in the past: #6248)
When you view the stage page while running the following:
```
sc.parallelize(1 to X, 10000).count()
```
The page never loads, the job is stalled, and you end up running into an OOM:
```
HTTP ERROR 500
Problem accessing /stages/stage/. Reason:
Server Error
Caused by:
java.lang.OutOfMemoryError: Java heap space
at java.util.Arrays.copyOf(Arrays.java:2367)
at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:130)
```
This patch compresses Jetty responses in gzip. The correct long-term fix is to add pagination.
Author: Andrew Or <andrew@databricks.com>
Closes #7296 from andrewor14/gzip-jetty and squashes the following commits:
a051c64 [Andrew Or] Use GZIP to compress Jetty responses
Diffstat (limited to 'external/kafka')
0 files changed, 0 insertions, 0 deletions