diff options
author | ding <ding@localhost.localdomain> | 2016-10-04 00:00:10 -0700 |
---|---|---|
committer | Joseph K. Bradley <joseph@databricks.com> | 2016-10-04 00:00:10 -0700 |
commit | 126baa8d32bc0e7bf8b43f9efa84f2728f02347d (patch) | |
tree | 9aaa195bd73b5958ebf65395d0b25fe520e0d10f /yarn | |
parent | d2dc8c4a162834818190ffd82894522c524ca3e5 (diff) | |
download | spark-126baa8d32bc0e7bf8b43f9efa84f2728f02347d.tar.gz spark-126baa8d32bc0e7bf8b43f9efa84f2728f02347d.tar.bz2 spark-126baa8d32bc0e7bf8b43f9efa84f2728f02347d.zip |
[SPARK-17559][MLLIB] persist edges if their storage level is non in PeriodicGraphCheckpointer
## What changes were proposed in this pull request?
When use PeriodicGraphCheckpointer to persist graph, sometimes the edges isn't persisted. As currently only when vertices's storage level is none, graph is persisted. However there is a chance vertices's storage level is not none while edges's is none. Eg. graph created by a outerJoinVertices operation, vertices is automatically cached while edges is not. In this way, edges will not be persisted if we use PeriodicGraphCheckpointer do persist. We need separately check edges's storage level and persisted it if it's none.
## How was this patch tested?
manual tests
Author: ding <ding@localhost.localdomain>
Closes #15124 from dding3/spark-persisitEdge.
Diffstat (limited to 'yarn')
0 files changed, 0 insertions, 0 deletions