diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-05-23 02:11:17 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-05-23 02:11:17 -0700 |
| commit | ad0badba1450295982738934da2cc121cde18213 (patch) | |
| tree | 58e2abb36dc3ba6cd2a4be855b3d9600d8267805 /python/pyspark/sql/dataframe.py | |
| parent | a40bca0111de45763c3ef4270afb2185c16b8f95 (diff) | |
| download | spark-ad0badba1450295982738934da2cc121cde18213.tar.gz spark-ad0badba1450295982738934da2cc121cde18213.tar.bz2 spark-ad0badba1450295982738934da2cc121cde18213.zip | |
[SPARK-7777][Streaming] Handle the case when there is no block in a batch
In the old implementation, if a batch has no block, `areWALRecordHandlesPresent` will be `true` and it will return `WriteAheadLogBackedBlockRDD`.
This PR handles this case by returning `WriteAheadLogBackedBlockRDD` or `BlockRDD` according to the configuration.
Author: zsxwing <zsxwing@gmail.com>
Closes #6372 from zsxwing/SPARK-7777 and squashes the following commits:
788f895 [zsxwing] Handle the case when there is no block in a batch
Diffstat (limited to 'python/pyspark/sql/dataframe.py')
0 files changed, 0 insertions, 0 deletions