diff options
| author | Xutingjun <1039320815@qq.com> | 2015-02-16 14:54:23 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-02-16 14:54:23 +0000 |
| commit | 1115e8e739ec4d60604cd79bd452770f041510d4 (patch) | |
| tree | 3fb9314aede5aaf4ded73d4e0013d298289a142e /streaming | |
| parent | 3ce58cf9c0ffe8b867ca79b404fe3fa291cf0e56 (diff) | |
| download | spark-1115e8e739ec4d60604cd79bd452770f041510d4.tar.gz spark-1115e8e739ec4d60604cd79bd452770f041510d4.tar.bz2 spark-1115e8e739ec4d60604cd79bd452770f041510d4.zip | |
[SPARK-5831][Streaming]When checkpoint file size is bigger than 10, then delete the old ones
Author: Xutingjun <1039320815@qq.com>
Closes #4621 from XuTingjun/checkpointFile and squashes the following commits:
b5f2816 [Xutingjun] fix bug
Diffstat (limited to 'streaming')
| -rw-r--r-- | streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala b/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala index b780282bda..f88a8a0151 100644 --- a/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala +++ b/streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala @@ -152,7 +152,7 @@ class CheckpointWriter( // Delete old checkpoint files val allCheckpointFiles = Checkpoint.getCheckpointFiles(checkpointDir, fs) - if (allCheckpointFiles.size > 4) { + if (allCheckpointFiles.size > 10) { allCheckpointFiles.take(allCheckpointFiles.size - 10).foreach(file => { logInfo("Deleting " + file) fs.delete(file, true) |