diff options
| author | Sean Owen <sowen@cloudera.com> | 2014-11-10 11:47:27 -0800 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2014-11-10 11:47:27 -0800 |
| commit | 3a02d416cd82a7a942fd6ff4a0e05ff070eb218a (patch) | |
| tree | 7fd5e50588b6033c3f599808c3f27bf34dcfaf0b /examples/src/main/scala | |
| parent | ed8bf1eac548577c4bbad7ce3f7f301a2f52ef17 (diff) | |
| download | spark-3a02d416cd82a7a942fd6ff4a0e05ff070eb218a.tar.gz spark-3a02d416cd82a7a942fd6ff4a0e05ff070eb218a.tar.bz2 spark-3a02d416cd82a7a942fd6ff4a0e05ff070eb218a.zip | |
SPARK-2548 [STREAMING] JavaRecoverableWordCount is missing
Here's my attempt to re-port `RecoverableNetworkWordCount` to Java, following the example of its Scala and Java siblings. I fixed a few minor doc/formatting issues along the way I believe.
Author: Sean Owen <sowen@cloudera.com>
Closes #2564 from srowen/SPARK-2548 and squashes the following commits:
0d0bf29 [Sean Owen] Update checkpoint call as in https://github.com/apache/spark/pull/2735
35f23e3 [Sean Owen] Remove old comment about running in standalone mode
179b3c2 [Sean Owen] Re-port RecoverableNetworkWordCount to Java example, and touch up doc / formatting in related examples
Diffstat (limited to 'examples/src/main/scala')
| -rw-r--r-- | examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala | 15 |
1 files changed, 1 insertions, 14 deletions
diff --git a/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala b/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala index 6af3a0f33e..eb48db85d3 100644 --- a/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala +++ b/examples/src/main/scala/org/apache/spark/examples/streaming/RecoverableNetworkWordCount.scala @@ -31,15 +31,13 @@ import org.apache.spark.util.IntParam /** * Counts words in text encoded with UTF8 received from the network every second. * - * Usage: NetworkWordCount <hostname> <port> <checkpoint-directory> <output-file> + * Usage: RecoverableNetworkWordCount <hostname> <port> <checkpoint-directory> <output-file> * <hostname> and <port> describe the TCP server that Spark Streaming would connect to receive * data. <checkpoint-directory> directory to HDFS-compatible file system which checkpoint data * <output-file> file to which the word counts will be appended * - * In local mode, <master> should be 'local[n]' with n > 1 * <checkpoint-directory> and <output-file> must be absolute paths * - * * To run this on your local machine, you need to first run a Netcat server * * `$ nc -lk 9999` @@ -54,19 +52,8 @@ import org.apache.spark.util.IntParam * checkpoint data exists in ~/checkpoint/, then it will create StreamingContext from * the checkpoint data. * - * To run this example in a local standalone cluster with automatic driver recovery, - * - * `$ bin/spark-class org.apache.spark.deploy.Client -s launch <cluster-url> \ - * <path-to-examples-jar> \ - * org.apache.spark.examples.streaming.RecoverableNetworkWordCount <cluster-url> \ - * localhost 9999 ~/checkpoint ~/out` - * - * <path-to-examples-jar> would typically be - * <spark-dir>/examples/target/scala-XX/spark-examples....jar - * * Refer to the online documentation for more details. */ - object RecoverableNetworkWordCount { def createContext(ip: String, port: Int, outputPath: String) = { |