aboutsummaryrefslogtreecommitdiff
path: root/scalastyle-config.xml
diff options
context:
space:
mode:
authorJacker Hu <gt.hu.chang@gmail.com>2015-10-10 11:36:18 +0100
committerSean Owen <sowen@cloudera.com>2015-10-10 11:36:18 +0100
commita16396df76cc27099011bfb96b28cbdd7f964ca8 (patch)
treefa24e961cd38f01993cee88208bf99acaf008444 /scalastyle-config.xml
parent864de3bf4041c829e95d278b9569e91448bab0cc (diff)
downloadspark-a16396df76cc27099011bfb96b28cbdd7f964ca8.tar.gz
spark-a16396df76cc27099011bfb96b28cbdd7f964ca8.tar.bz2
spark-a16396df76cc27099011bfb96b28cbdd7f964ca8.zip
[SPARK-10772] [STREAMING] [SCALA] NullPointerException when transform function in DStream returns NULL
Currently, the ```TransformedDStream``` will using ```Some(transformFunc(parentRDDs, validTime))``` as compute return value, when the ```transformFunc``` somehow returns null as return value, the followed operator will have NullPointerExeception. This fix uses the ```Option()``` instead of ```Some()``` to deal with the possible null value. When ```transformFunc``` returns ```null```, the option will transform null to ```None```, the downstream can handle ```None``` correctly. NOTE (2015-09-25): The latest fix will check the return value of transform function, if it is ```NULL```, a spark exception will be thrown out Author: Jacker Hu <gt.hu.chang@gmail.com> Author: jhu-chang <gt.hu.chang@gmail.com> Closes #8881 from jhu-chang/Fix_Transform.
Diffstat (limited to 'scalastyle-config.xml')
0 files changed, 0 insertions, 0 deletions