diff options
| author | Tathagata Das <tathagata.das1565@gmail.com> | 2014-05-05 15:28:19 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2014-05-05 15:28:19 -0700 |
| commit | a975a19f21e71f448b3fdb2ed4461e28ef439900 (patch) | |
| tree | 3305b05ddec8ccebaa1f3a88dfef516afb6d0ad2 /docs/configuration.md | |
| parent | 3292e2a71bfb5df5ba156cf7557747d164d12291 (diff) | |
| download | spark-a975a19f21e71f448b3fdb2ed4461e28ef439900.tar.gz spark-a975a19f21e71f448b3fdb2ed4461e28ef439900.tar.bz2 spark-a975a19f21e71f448b3fdb2ed4461e28ef439900.zip | |
[SPARK-1504], [SPARK-1505], [SPARK-1558] Updated Spark Streaming guide
- SPARK-1558: Updated custom receiver guide to match it with the new API
- SPARK-1504: Added deployment and monitoring subsection to streaming
- SPARK-1505: Added migration guide for migrating from 0.9.x and below to Spark 1.0
- Updated various Java streaming examples to use JavaReceiverInputDStream to highlight the API change.
- Removed the requirement for cleaner ttl from streaming guide
Author: Tathagata Das <tathagata.das1565@gmail.com>
Closes #652 from tdas/doc-fix and squashes the following commits:
cb4f4b7 [Tathagata Das] Possible fix for flaky graceful shutdown test.
ab71f7f [Tathagata Das] Merge remote-tracking branch 'apache-github/master' into doc-fix
8d6ff9b [Tathagata Das] Addded migration guide to Spark Streaming.
7d171df [Tathagata Das] Added reference to JavaReceiverInputStream in examples and streaming guide.
49edd7c [Tathagata Das] Change java doc links to use Java docs.
11528d7 [Tathagata Das] Updated links on index page.
ff80970 [Tathagata Das] More updates to streaming guide.
4dc42e9 [Tathagata Das] Added monitoring and other documentation in the streaming guide.
14c6564 [Tathagata Das] Updated custom receiver guide.
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 6 |
1 files changed, 3 insertions, 3 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 81ad895d90..d6f316ba5f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -462,7 +462,7 @@ Apart from these, the following properties are also available, and may be useful <td>(infinite)</td> <td> Duration (seconds) of how long Spark will remember any metadata (stages generated, tasks generated, etc.). - Periodic cleanups will ensure that metadata older than this duration will be forgetten. This is + Periodic cleanups will ensure that metadata older than this duration will be forgotten. This is useful for running Spark for many hours / days (for example, running 24/7 in case of Spark Streaming applications). Note that any RDD that persists in memory for more than this duration will be cleared as well. </td> @@ -471,8 +471,8 @@ Apart from these, the following properties are also available, and may be useful <td>spark.streaming.blockInterval</td> <td>200</td> <td> - Duration (milliseconds) of how long to batch new objects coming from network receivers used - in Spark Streaming. + Interval (milliseconds) at which data received by Spark Streaming receivers is coalesced + into blocks of data before storing them in Spark. </td> </tr> <tr> |