diff options
| author | Holden Karau <holden@us.ibm.com> | 2016-05-03 00:18:10 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2016-05-03 00:18:10 -0700 |
| commit | f10ae4b1e169495af11b8e8123c60dd96174477e (patch) | |
| tree | 323357bac8e3b933780625c92292a53c7043e17a /docs/quick-start.md | |
| parent | d8f528ceb61e3c2ac7ac97cd8147dafbb625932f (diff) | |
| download | spark-f10ae4b1e169495af11b8e8123c60dd96174477e.tar.gz spark-f10ae4b1e169495af11b8e8123c60dd96174477e.tar.bz2 spark-f10ae4b1e169495af11b8e8123c60dd96174477e.zip | |
[SPARK-6717][ML] Clear shuffle files after checkpointing in ALS
## What changes were proposed in this pull request?
When ALS is run with a checkpoint interval, during the checkpoint materialize the current state and cleanup the previous shuffles (non-blocking).
## How was this patch tested?
Existing ALS unit tests, new ALS checkpoint cleanup unit tests added & shuffle files checked after ALS w/checkpointing run.
Author: Holden Karau <holden@us.ibm.com>
Author: Holden Karau <holden@pigscanfly.ca>
Closes #11919 from holdenk/SPARK-6717-clear-shuffle-files-after-checkpointing-in-ALS.
Diffstat (limited to 'docs/quick-start.md')
0 files changed, 0 insertions, 0 deletions