diff options
| author | Matei Zaharia <matei@eecs.berkeley.edu> | 2013-09-06 00:29:37 -0400 |
|---|---|---|
| committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2013-09-08 00:29:11 -0700 |
| commit | 98fb69822cf780160bca51abeaab7c82e49fab54 (patch) | |
| tree | 524a1e75519f6a5cc65d004501d6237228db97f2 /docs/configuration.md | |
| parent | 38488aca8a67d5d2749b82e3fd5f3dc50873d09a (diff) | |
| download | spark-98fb69822cf780160bca51abeaab7c82e49fab54.tar.gz spark-98fb69822cf780160bca51abeaab7c82e49fab54.tar.bz2 spark-98fb69822cf780160bca51abeaab7c82e49fab54.zip | |
Work in progress:
- Add job scheduling docs
- Rename some fair scheduler properties
- Organize intro page better
- Link to Apache wiki for "contributing to Spark"
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 32 |
1 files changed, 21 insertions, 11 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index 310e78a9eb..d4f85538b2 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -82,17 +82,6 @@ Apart from these, the following properties are also available, and may be useful <table class="table"> <tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr> <tr> - <td>spark.mesos.coarse</td> - <td>false</td> - <td> - If set to "true", runs over Mesos clusters in - <a href="running-on-mesos.html#mesos-run-modes">"coarse-grained" sharing mode</a>, - where Spark acquires one long-lived Mesos task on each machine instead of one Mesos task per Spark task. - This gives lower-latency scheduling for short queries, but leaves resources in use for the whole - duration of the Spark job. - </td> -</tr> -<tr> <td>spark.default.parallelism</td> <td>8</td> <td> @@ -110,6 +99,17 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> + <td>spark.mesos.coarse</td> + <td>false</td> + <td> + If set to "true", runs over Mesos clusters in + <a href="running-on-mesos.html#mesos-run-modes">"coarse-grained" sharing mode</a>, + where Spark acquires one long-lived Mesos task on each machine instead of one Mesos task per Spark task. + This gives lower-latency scheduling for short queries, but leaves resources in use for the whole + duration of the Spark job. + </td> +</tr> +<tr> <td>spark.ui.port</td> <td>3030</td> <td> @@ -161,6 +161,16 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> + <td>spark.scheduler.mode</td> + <td>FIFO</td> + <td> + The <a href="job-scheduling.html#scheduling-within-an-application">scheduling mode</a> between + jobs submitted to the same SparkContext. Can be set to <code>FAIR</code> + to use fair sharing instead of queueing jobs one after another. Useful for + multi-user services. + </td> +</tr> +<tr> <td>spark.reducer.maxMbInFlight</td> <td>48</td> <td> |