diff options
| author | Burak Yavuz <brkyvz@gmail.com> | 2015-10-27 16:01:26 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-10-27 16:01:26 -0700 |
| commit | 4f030b9e82172659d250281782ac573cbd1438fc (patch) | |
| tree | 335d4f235faefc33acf42e6498ceb5be0b7a633d /data/mllib | |
| parent | 9dba5fb2b59174cefde5b62a5c892fe5925bea38 (diff) | |
| download | spark-4f030b9e82172659d250281782ac573cbd1438fc.tar.gz spark-4f030b9e82172659d250281782ac573cbd1438fc.tar.bz2 spark-4f030b9e82172659d250281782ac573cbd1438fc.zip | |
[SPARK-11324][STREAMING] Flag for closing Write Ahead Logs after a write
Currently the Write Ahead Log in Spark Streaming flushes data as writes need to be made. S3 does not support flushing of data, data is written once the stream is actually closed.
In case of failure, the data for the last minute (default rolling interval) will not be properly written. Therefore we need a flag to close the stream after the write, so that we achieve read after write consistency.
cc tdas zsxwing
Author: Burak Yavuz <brkyvz@gmail.com>
Closes #9285 from brkyvz/caw-wal.
Diffstat (limited to 'data/mllib')
0 files changed, 0 insertions, 0 deletions