diff options
author | Yadong Qi <qiyadong2010@gmail.com> | 2014-11-19 15:53:06 -0800 |
---|---|---|
committer | Tathagata Das <tathagata.das1565@gmail.com> | 2014-11-19 15:53:06 -0800 |
commit | c3002c4a61c4fc5b966aa384c41c3cba33de0aa6 (patch) | |
tree | 382c1ee65210dcbfd2f1929ac9af50dc082d06ba /graphx | |
parent | 73c8ea84a668f443eb18ce15ba97023da041d808 (diff) | |
download | spark-c3002c4a61c4fc5b966aa384c41c3cba33de0aa6.tar.gz spark-c3002c4a61c4fc5b966aa384c41c3cba33de0aa6.tar.bz2 spark-c3002c4a61c4fc5b966aa384c41c3cba33de0aa6.zip |
[SPARK-4294][Streaming] UnionDStream stream should express the requirements in the same way as TransformedDStream
In class TransformedDStream:
```scala
require(parents.length > 0, "List of DStreams to transform is empty")
require(parents.map(.ssc).distinct.size == 1, "Some of the DStreams have different contexts")
require(parents.map(.slideDuration).distinct.size == 1,
"Some of the DStreams have different slide durations")
```
In class UnionDStream:
```scala
if (parents.length == 0)
{ throw new IllegalArgumentException("Empty array of parents") }
if (parents.map(.ssc).distinct.size > 1)
{ throw new IllegalArgumentException("Array of parents have different StreamingContexts") }
if (parents.map(.slideDuration).distinct.size > 1)
{ throw new IllegalArgumentException("Array of parents have different slide times") }
```
The function is the same, but the realization is not. I think they shoule be the same.
Author: Yadong Qi <qiyadong2010@gmail.com>
Closes #3152 from watermen/bug-fix1 and squashes the following commits:
ed66db6 [Yadong Qi] Change transform to union
b6b3b8b [Yadong Qi] The same function should have the same realization.
Diffstat (limited to 'graphx')
0 files changed, 0 insertions, 0 deletions