diff options
| author | lewuathe <lewuathe@me.com> | 2014-12-04 15:14:36 -0800 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2014-12-04 15:25:21 -0800 |
| commit | f5c5647b863b593eae88fe5b6ba580413584ed06 (patch) | |
| tree | 08a7e8b5888d3fc3c56dbe2f989a27d002ab0f4c /docs | |
| parent | 0d159de39a21ab82d4ecdfa2d88fa525339daee1 (diff) | |
| download | spark-f5c5647b863b593eae88fe5b6ba580413584ed06.tar.gz spark-f5c5647b863b593eae88fe5b6ba580413584ed06.tar.bz2 spark-f5c5647b863b593eae88fe5b6ba580413584ed06.zip | |
[SPARK-4652][DOCS] Add docs about spark-git-repo option
There might be some cases when WIPS spark version need to be run
on EC2 cluster. In order to setup this type of cluster more easily,
add --spark-git-repo option description to ec2 documentation.
Author: lewuathe <lewuathe@me.com>
Author: Josh Rosen <joshrosen@databricks.com>
Closes #3513 from Lewuathe/doc-for-development-spark-cluster and squashes the following commits:
6dae8ee [lewuathe] Wrap consistent with other descriptions
cfaf9be [lewuathe] Add docs about spark-git-repo option
(Editing / cleanup by Josh Rosen)
(cherry picked from commit ab8177da2defab1ecd8bc0cd5a21f07be5b8d2c5)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/ec2-scripts.md | 5 |
1 files changed, 5 insertions, 0 deletions
diff --git a/docs/ec2-scripts.md b/docs/ec2-scripts.md index 66bf5f1a85..ed51d0abb3 100644 --- a/docs/ec2-scripts.md +++ b/docs/ec2-scripts.md @@ -85,6 +85,11 @@ another. specified version of Spark. The `<version>` can be a version number (e.g. "0.7.3") or a specific git hash. By default, a recent version will be used. +- `--spark-git-repo=<repository url>` will let you run a custom version of + Spark that is built from the given git repository. By default, the + [Apache Github mirror](https://github.com/apache/spark) will be used. + When using a custom Spark version, `--spark-version` must be set to git + commit hash, such as 317e114, instead of a version number. - If one of your launches fails due to e.g. not having the right permissions on your private key file, you can run `launch` with the `--resume` option to restart the setup process on an existing cluster. |