diff options
Diffstat (limited to 'docs/streaming-programming-guide.md')
| -rw-r--r-- | docs/streaming-programming-guide.md | 12 |
1 files changed, 10 insertions, 2 deletions
diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md index bc2f4f884f..c7df172024 100644 --- a/docs/streaming-programming-guide.md +++ b/docs/streaming-programming-guide.md @@ -13,6 +13,14 @@ A Spark Streaming application is very similar to a Spark application; it consist This guide shows some how to start programming with DStreams. +# Linking with Spark Streaming + +Add the following SBT or Maven dependency to your project to use Spark Streaming: + + groupId = org.apache.spark + artifactId = spark-streaming_{{site.SCALA_VERSION}} + version = {{site.SPARK_VERSION}} + # Initializing Spark Streaming The first thing a Spark Streaming program must do is create a `StreamingContext` object, which tells Spark how to access a cluster. A `StreamingContext` can be created by using @@ -301,8 +309,8 @@ dstream.checkpoint(checkpointInterval) // checkpointInterval must be a multiple For DStreams that must be checkpointed (that is, DStreams created by `updateStateByKey` and `reduceByKeyAndWindow` with inverse function), the checkpoint interval of the DStream is by default set to a multiple of the DStream's sliding interval such that its at least 10 seconds. -## Customizing Receiver -Spark comes with a built in support for most common usage scenarios where input stream source can be either a network socket stream to support for a few message queues. Apart from that it is also possible to supply your own custom receiver via a convenient API. Find more details at [Custom Receiver Guide](streaming-custom-receivers.html) +## Custom Receivers +Spark comes with a built in support for most common usage scenarios where input stream source can be either a network socket stream to support for a few message queues. Apart from that it is also possible to supply your own custom receiver via a convenient API. Find more details at [Custom Receiver Guide](streaming-custom-receivers.html). # Performance Tuning Getting the best performance of a Spark Streaming application on a cluster requires a bit of tuning. This section explains a number of the parameters and configurations that can tuned to improve the performance of you application. At a high level, you need to consider two things: |