diff options
| author | Davies Liu <davies@databricks.com> | 2015-08-31 15:55:22 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-08-31 15:55:22 -0700 |
| commit | 540bdee93103a73736d282b95db6a8cda8f6a2b1 (patch) | |
| tree | 4296a5e874aec24a2bad82cae5091a0ebf24ddb8 /sbin | |
| parent | 5b3245d6dff65972fc39c73f90d5cbdf84d19129 (diff) | |
| download | spark-540bdee93103a73736d282b95db6a8cda8f6a2b1.tar.gz spark-540bdee93103a73736d282b95db6a8cda8f6a2b1.tar.bz2 spark-540bdee93103a73736d282b95db6a8cda8f6a2b1.zip | |
[SPARK-10341] [SQL] fix memory starving in unsafe SMJ
In SMJ, the first ExternalSorter could consume all the memory before spilling, then the second can not even acquire the first page.
Before we have a better memory allocator, SMJ should call prepare() before call any compute() of it's children.
cc rxin JoshRosen
Author: Davies Liu <davies@databricks.com>
Closes #8511 from davies/smj_memory.
Diffstat (limited to 'sbin')
0 files changed, 0 insertions, 0 deletions