diff options
| author | Matei Zaharia <matei@eecs.berkeley.edu> | 2011-02-10 11:20:51 -0800 |
|---|---|---|
| committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2011-02-10 12:07:48 -0800 |
| commit | dc24aecd8fc8b5d9b19c10e8301dbe1107412c8a (patch) | |
| tree | da092278a5c480825f7376126027831466faa0e8 | |
| parent | 62f1c6f5a85e5104383b5bd3204ce4b3c29a2bd1 (diff) | |
| download | spark-dc24aecd8fc8b5d9b19c10e8301dbe1107412c8a.tar.gz spark-dc24aecd8fc8b5d9b19c10e8301dbe1107412c8a.tar.bz2 spark-dc24aecd8fc8b5d9b19c10e8301dbe1107412c8a.zip | |
Close record readers in HadoopFile after finishing a split
| -rw-r--r-- | core/src/main/scala/spark/HadoopFile.scala | 3 |
1 files changed, 3 insertions, 0 deletions
diff --git a/core/src/main/scala/spark/HadoopFile.scala b/core/src/main/scala/spark/HadoopFile.scala index a63c9d8a94..629dcc7da5 100644 --- a/core/src/main/scala/spark/HadoopFile.scala +++ b/core/src/main/scala/spark/HadoopFile.scala @@ -76,6 +76,9 @@ extends RDD[(K, V)](sc) { } gotNext = true } + if (finished) { + reader.close() + } !finished } |