diff options
| author | dding3 <ding.ding@intel.com> | 2016-05-27 21:01:50 -0500 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-05-27 21:01:50 -0500 |
| commit | 88c9c467a31630c558719679ca0894873a268b27 (patch) | |
| tree | c9d0db10251a19814d9c2afcc7dad6a74f919e2e /core/src | |
| parent | 5d4dafe8fdea49dcbd6b0e4c23e3791fa30c8911 (diff) | |
| download | spark-88c9c467a31630c558719679ca0894873a268b27.tar.gz spark-88c9c467a31630c558719679ca0894873a268b27.tar.bz2 spark-88c9c467a31630c558719679ca0894873a268b27.zip | |
[SPARK-15562][ML] Delete temp directory after program exit in DataFrameExample
## What changes were proposed in this pull request?
Temp directory used to save records is not deleted after program exit in DataFrameExample. Although it called deleteOnExit, it doesn't work as the directory is not empty. Similar things happend in ContextCleanerSuite. Update the code to make sure temp directory is deleted after program exit.
## How was this patch tested?
unit tests and local build.
Author: dding3 <ding.ding@intel.com>
Closes #13328 from dding3/master.
Diffstat (limited to 'core/src')
| -rw-r--r-- | core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala b/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala index 69ff6c7c28..6724af9525 100644 --- a/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala +++ b/core/src/test/scala/org/apache/spark/ContextCleanerSuite.scala @@ -32,6 +32,7 @@ import org.apache.spark.internal.Logging import org.apache.spark.rdd.{RDD, ReliableRDDCheckpointData} import org.apache.spark.shuffle.sort.SortShuffleManager import org.apache.spark.storage._ +import org.apache.spark.util.Utils /** * An abstract base class for context cleaner tests, which sets up a context with a config @@ -206,8 +207,7 @@ class ContextCleanerSuite extends ContextCleanerSuiteBase { } test("automatically cleanup normal checkpoint") { - val checkpointDir = java.io.File.createTempFile("temp", "") - checkpointDir.deleteOnExit() + val checkpointDir = Utils.createTempDir() checkpointDir.delete() var rdd = newPairRDD() sc.setCheckpointDir(checkpointDir.toString) |