diff options
| author | tedyu <yuzhihong@gmail.com> | 2015-02-26 23:26:07 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-02-26 23:27:09 +0000 |
| commit | e60ad2f4c47b011be7a3198689ac2b82ee317d96 (patch) | |
| tree | 91ef65994760b6417c20e9ebb78f8e78b799c465 /core | |
| parent | b38dec2ffdf724ff4e181cc8c7427d074b442670 (diff) | |
| download | spark-e60ad2f4c47b011be7a3198689ac2b82ee317d96.tar.gz spark-e60ad2f4c47b011be7a3198689ac2b82ee317d96.tar.bz2 spark-e60ad2f4c47b011be7a3198689ac2b82ee317d96.zip | |
SPARK-6045 RecordWriter should be checked against null in PairRDDFunctio...
...ns#saveAsNewAPIHadoopDataset
Author: tedyu <yuzhihong@gmail.com>
Closes #4794 from tedyu/master and squashes the following commits:
2632a57 [tedyu] SPARK-6045 RecordWriter should be checked against null in PairRDDFunctions#saveAsNewAPIHadoopDataset
2d8d4b1 [tedyu] SPARK-6045 RecordWriter should be checked against null in PairRDDFunctions#saveAsNewAPIHadoopDataset
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala b/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala index 955b42c3ba..6b4f097ea9 100644 --- a/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala +++ b/core/src/main/scala/org/apache/spark/rdd/PairRDDFunctions.scala @@ -993,6 +993,7 @@ class PairRDDFunctions[K, V](self: RDD[(K, V)]) val (outputMetrics, bytesWrittenCallback) = initHadoopOutputMetrics(context) val writer = format.getRecordWriter(hadoopContext).asInstanceOf[NewRecordWriter[K,V]] + require(writer != null, "Unable to obtain RecordWriter") var recordsWritten = 0L try { while (iter.hasNext) { |