diff options
| author | Shixiong Zhu <shixiong@databricks.com> | 2016-01-07 17:37:46 -0800 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2016-01-07 17:37:46 -0800 |
| commit | c94199e977279d9b4658297e8108b46bdf30157b (patch) | |
| tree | 0f33916993f15858184e52647ced22521ee165bb /docs/configuration.md | |
| parent | 5a4021998ab0f1c8bbb610eceecdf879d149a7b8 (diff) | |
| download | spark-c94199e977279d9b4658297e8108b46bdf30157b.tar.gz spark-c94199e977279d9b4658297e8108b46bdf30157b.tar.bz2 spark-c94199e977279d9b4658297e8108b46bdf30157b.zip | |
[SPARK-12507][STREAMING][DOCUMENT] Expose closeFileAfterWrite and allowBatching configurations for Streaming
/cc tdas brkyvz
Author: Shixiong Zhu <shixiong@databricks.com>
Closes #10453 from zsxwing/streaming-conf.
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 18 |
1 files changed, 18 insertions, 0 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 6bd0658b3e..08392c3918 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1574,6 +1574,24 @@ Apart from these, the following properties are also available, and may be useful How many batches the Spark Streaming UI and status APIs remember before garbage collecting. </td> </tr> +<tr> + <td><code>spark.streaming.driver.writeAheadLog.closeFileAfterWrite</code></td> + <td>false</td> + <td> + Whether to close the file after writing a write ahead log record on the driver. Set this to 'true' + when you want to use S3 (or any file system that does not support flushing) for the metadata WAL + on the driver. + </td> +</tr> +<tr> + <td><code>spark.streaming.receiver.writeAheadLog.closeFileAfterWrite</code></td> + <td>false</td> + <td> + Whether to close the file after writing a write ahead log record on the receivers. Set this to 'true' + when you want to use S3 (or any file system that does not support flushing) for the data WAL + on the receivers. + </td> +</tr> </table> #### SparkR |