diff options
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configuration.md | 21 |
1 files changed, 19 insertions, 2 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index e33094b062..ae753189b5 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -427,6 +427,21 @@ Apart from these, the following properties are also available, and may be useful with <code>spark.jars.packages</code>. </td> </tr> +<tr> + <td><code>spark.pyspark.driver.python</code></td> + <td></td> + <td> + Python binary executable to use for PySpark in driver. + (default is <code>spark.pyspark.python</code>) + </td> +</tr> +<tr> + <td><code>spark.pyspark.python</code></td> + <td></td> + <td> + Python binary executable to use for PySpark in both driver and executors. + </td> +</tr> </table> #### Shuffle Behavior @@ -1786,11 +1801,13 @@ The following variables can be set in `spark-env.sh`: </tr> <tr> <td><code>PYSPARK_PYTHON</code></td> - <td>Python binary executable to use for PySpark in both driver and workers (default is <code>python2.7</code> if available, otherwise <code>python</code>).</td> + <td>Python binary executable to use for PySpark in both driver and workers (default is <code>python2.7</code> if available, otherwise <code>python</code>). + Property <code>spark.pyspark.python</code> take precedence if it is set</td> </tr> <tr> <td><code>PYSPARK_DRIVER_PYTHON</code></td> - <td>Python binary executable to use for PySpark in driver only (default is <code>PYSPARK_PYTHON</code>).</td> + <td>Python binary executable to use for PySpark in driver only (default is <code>PYSPARK_PYTHON</code>). + Property <code>spark.pyspark.driver.python</code> take precedence if it is set</td> </tr> <tr> <td><code>SPARKR_DRIVER_R</code></td> |