diff options
author | zsxwing <zsxwing@gmail.com> | 2014-12-25 19:46:05 -0800 |
---|---|---|
committer | Tathagata Das <tathagata.das1565@gmail.com> | 2014-12-25 19:46:05 -0800 |
commit | f9ed2b6641b9df39cee4b98a33cd5a3ddda2d146 (patch) | |
tree | 3f07c19a7e12db441ff1eba906ac5b671f644fe3 /examples/scala-2.10 | |
parent | f205fe477c33a541053c198cd43a5811d6cf9fe2 (diff) | |
download | spark-f9ed2b6641b9df39cee4b98a33cd5a3ddda2d146.tar.gz spark-f9ed2b6641b9df39cee4b98a33cd5a3ddda2d146.tar.bz2 spark-f9ed2b6641b9df39cee4b98a33cd5a3ddda2d146.zip |
[SPARK-4608][Streaming] Reorganize StreamingContext implicit to improve API convenience
There is only one implicit function `toPairDStreamFunctions` in `StreamingContext`. This PR did similar reorganization like [SPARK-4397](https://issues.apache.org/jira/browse/SPARK-4397).
Compiled the following codes with Spark Streaming 1.1.0 and ran it with this PR. Everything is fine.
```Scala
import org.apache.spark._
import org.apache.spark.streaming._
import org.apache.spark.streaming.StreamingContext._
object StreamingApp {
def main(args: Array[String]) {
val conf = new SparkConf().setMaster("local[2]").setAppName("FileWordCount")
val ssc = new StreamingContext(conf, Seconds(10))
val lines = ssc.textFileStream("/some/path")
val words = lines.flatMap(_.split(" "))
val pairs = words.map(word => (word, 1))
val wordCounts = pairs.reduceByKey(_ + _)
wordCounts.print()
ssc.start()
ssc.awaitTermination()
}
}
```
Author: zsxwing <zsxwing@gmail.com>
Closes #3464 from zsxwing/SPARK-4608 and squashes the following commits:
aa6d44a [zsxwing] Fix a copy-paste error
f74c190 [zsxwing] Merge branch 'master' into SPARK-4608
e6f9cc9 [zsxwing] Update the docs
27833bb [zsxwing] Remove `import StreamingContext._`
c15162c [zsxwing] Reorganize StreamingContext implicit to improve API convenience
Diffstat (limited to 'examples/scala-2.10')
-rw-r--r-- | examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala | 1 |
1 files changed, 0 insertions, 1 deletions
diff --git a/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala b/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala index c9e1511278..2adc63f7ff 100644 --- a/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala +++ b/examples/scala-2.10/src/main/scala/org/apache/spark/examples/streaming/KafkaWordCount.scala @@ -22,7 +22,6 @@ import java.util.Properties import kafka.producer._ import org.apache.spark.streaming._ -import org.apache.spark.streaming.StreamingContext._ import org.apache.spark.streaming.kafka._ import org.apache.spark.SparkConf |