diff options
| author | petermaxlee <petermaxlee@gmail.com> | 2016-09-20 19:08:07 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-09-20 19:08:07 -0700 |
| commit | 976f3b1227c1a9e0b878e010531285fdba57b6a7 (patch) | |
| tree | 4071c38e8b08b7cb4b4ec35ff7af881184b9a45f /pom.xml | |
| parent | 7e418e99cff4cf512ab2a9fa74221c4655048c8d (diff) | |
| download | spark-976f3b1227c1a9e0b878e010531285fdba57b6a7.tar.gz spark-976f3b1227c1a9e0b878e010531285fdba57b6a7.tar.bz2 spark-976f3b1227c1a9e0b878e010531285fdba57b6a7.zip | |
[SPARK-17513][SQL] Make StreamExecution garbage-collect its metadata
## What changes were proposed in this pull request?
This PR modifies StreamExecution such that it discards metadata for batches that have already been fully processed. I used the purge method that was added as part of SPARK-17235.
This is a resubmission of 15126, which was based on work by frreiss in #15067, but fixed the test case along with some typos.
## How was this patch tested?
A new test case in StreamingQuerySuite. The test case would fail without the changes in this pull request.
Author: petermaxlee <petermaxlee@gmail.com>
Closes #15166 from petermaxlee/SPARK-17513-2.
Diffstat (limited to 'pom.xml')
0 files changed, 0 insertions, 0 deletions