From dca946ff6779cda9c50ae0069734b7802bb4d24a Mon Sep 17 00:00:00 2001 From: Prashant Sharma Date: Tue, 26 Nov 2013 20:44:47 +0530 Subject: Documenting the newly added spark properties. --- docs/configuration.md | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) (limited to 'docs/configuration.md') diff --git a/docs/configuration.md b/docs/configuration.md index 97183bafdb..25e7cecbfc 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -275,11 +275,32 @@ Apart from these, the following properties are also available, and may be useful spark.akka.timeout - 20 + 60 Communication timeout between Spark nodes, in seconds. + + spark.akka.pauses + 60 + + Acceptable heart beat pause in seconds for akka, tune this if you expect GC pauses or network delays (reconnections) etc. + + + + spark.akka.failure-detector.threshold + 12.0 + + Tuning this is not recommended unless you are sure what you are doing. This maps to akka's `akka.remote.transport-failure-detector.threshold`. Usually having a larger value of `spark.akka.pauses` should suffice. + + + + spark.akka.heartbeat.interval + 5 + + A larger interval value in seconds reduces network overhead and a smaller value might be more informative for akka's failure detector. Tune this in combination of `spark.akka.pauses` and `spark.akka.failure-detector.threshold` if you need to. + + spark.driver.host (local hostname) -- cgit v1.2.3