aboutsummaryrefslogtreecommitdiff
path: root/docs/streaming-programming-guide.md
diff options
context:
space:
mode:
authorShixiong Zhu <shixiong@databricks.com>2016-03-14 23:21:30 -0700
committerReynold Xin <rxin@databricks.com>2016-03-14 23:21:30 -0700
commit43304b1758dec141b7fe9ed33cac976d75efdf91 (patch)
treec05690bea980854517baa6b0bb7495f87d844d9d /docs/streaming-programming-guide.md
parente64958001cb95d53c441131f8c7a92556f49fd7d (diff)
downloadspark-43304b1758dec141b7fe9ed33cac976d75efdf91.tar.gz
spark-43304b1758dec141b7fe9ed33cac976d75efdf91.tar.bz2
spark-43304b1758dec141b7fe9ed33cac976d75efdf91.zip
[SPARK-13888][DOC] Remove Akka Receiver doc and refer to the DStream Akka project
## What changes were proposed in this pull request? I have copied the docs of Streaming Akka to https://github.com/spark-packages/dstream-akka/blob/master/README.md So we can remove them from Spark now. ## How was this patch tested? Only document changes. (If this patch involves UI changes, please attach a screenshot; otherwise, remove this) Author: Shixiong Zhu <shixiong@databricks.com> Closes #11711 from zsxwing/remove-akka-doc.
Diffstat (limited to 'docs/streaming-programming-guide.md')
-rw-r--r--docs/streaming-programming-guide.md24
1 files changed, 7 insertions, 17 deletions
diff --git a/docs/streaming-programming-guide.md b/docs/streaming-programming-guide.md
index 998644f2e2..6c36b41e78 100644
--- a/docs/streaming-programming-guide.md
+++ b/docs/streaming-programming-guide.md
@@ -594,7 +594,7 @@ data from a source and stores it in Spark's memory for processing.
Spark Streaming provides two categories of built-in streaming sources.
- *Basic sources*: Sources directly available in the StreamingContext API.
- Examples: file systems, socket connections, and Akka actors.
+ Examples: file systems, and socket connections.
- *Advanced sources*: Sources like Kafka, Flume, Kinesis, Twitter, etc. are available through
extra utility classes. These require linking against extra dependencies as discussed in the
[linking](#linking) section.
@@ -631,7 +631,7 @@ as well as to run the receiver(s).
We have already taken a look at the `ssc.socketTextStream(...)` in the [quick example](#a-quick-example)
which creates a DStream from text
data received over a TCP socket connection. Besides sockets, the StreamingContext API provides
-methods for creating DStreams from files and Akka actors as input sources.
+methods for creating DStreams from files as input sources.
- **File Streams:** For reading data from files on any file system compatible with the HDFS API (that is, HDFS, S3, NFS, etc.), a DStream can be created as:
@@ -658,17 +658,12 @@ methods for creating DStreams from files and Akka actors as input sources.
<span class="badge" style="background-color: grey">Python API</span> `fileStream` is not available in the Python API, only `textFileStream` is available.
-- **Streams based on Custom Actors:** DStreams can be created with data streams received through Akka
- actors by using `AkkaUtils.createStream(ssc, actorProps, actor-name)`. See the [Custom Receiver
- Guide](streaming-custom-receivers.html) for more details.
-
- <span class="badge" style="background-color: grey">Python API</span> Since actors are available only in the Java and Scala
- libraries, `AkkaUtils.createStream` is not available in the Python API.
+- **Streams based on Custom Receivers:** DStreams can be created with data streams received through custom receivers. See the [Custom Receiver
+ Guide](streaming-custom-receivers.html) and [DStream Akka](https://github.com/spark-packages/dstream-akka) for more details.
- **Queue of RDDs as a Stream:** For testing a Spark Streaming application with test data, one can also create a DStream based on a queue of RDDs, using `streamingContext.queueStream(queueOfRDDs)`. Each RDD pushed into the queue will be treated as a batch of data in the DStream, and processed like a stream.
-For more details on streams from sockets, files, and actors,
-see the API documentations of the relevant functions in
+For more details on streams from sockets and files, see the API documentations of the relevant functions in
[StreamingContext](api/scala/index.html#org.apache.spark.streaming.StreamingContext) for
Scala, [JavaStreamingContext](api/java/index.html?org/apache/spark/streaming/api/java/JavaStreamingContext.html)
for Java, and [StreamingContext](api/python/pyspark.streaming.html#pyspark.streaming.StreamingContext) for Python.
@@ -2439,13 +2434,8 @@ that can be called to store the data in Spark. So, to migrate your custom networ
BlockGenerator object (does not exist any more in Spark 1.0 anyway), and use `store(...)` methods on
received data.
-**Actor-based Receivers**: Data could have been received using any Akka Actors by extending the actor class with
-`org.apache.spark.streaming.receivers.Receiver` trait. This has been renamed to
-[`org.apache.spark.streaming.receiver.ActorHelper`](api/scala/index.html#org.apache.spark.streaming.receiver.ActorHelper)
-and the `pushBlock(...)` methods to store received data has been renamed to `store(...)`. Other helper classes in
-the `org.apache.spark.streaming.receivers` package were also moved
-to [`org.apache.spark.streaming.receiver`](api/scala/index.html#org.apache.spark.streaming.receiver.package)
-package and renamed for better clarity.
+**Actor-based Receivers**: The Actor-based Receiver APIs have been moved to [DStream Akka](https://github.com/spark-packages/dstream-akka).
+Please refer to the project for more details.
***************************************************************************************************
***************************************************************************************************