diff options
| author | Xiangrui Meng <meng@databricks.com> | 2014-11-25 16:07:09 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2014-11-25 16:07:09 -0800 |
| commit | 7eba0fbe456c451122d7a2353ff0beca00f15223 (patch) | |
| tree | ca88df11d76b55975c5d5ab5e772bcf2ff3120bc /docs/ec2-scripts.md | |
| parent | 9bdf5da59036c0b052df756fc4a28d64677072e7 (diff) | |
| download | spark-7eba0fbe456c451122d7a2353ff0beca00f15223.tar.gz spark-7eba0fbe456c451122d7a2353ff0beca00f15223.tar.bz2 spark-7eba0fbe456c451122d7a2353ff0beca00f15223.zip | |
[Spark-4509] Revert EC2 tag-based cluster membership patch
This PR reverts changes related to tag-based cluster membership. As discussed in SPARK-3332, we didn't figure out a safe strategy to use tags to determine cluster membership, because tagging is not atomic. The following changes are reverted:
SPARK-2333: 94053a7b766788bb62e2dbbf352ccbcc75f71fc0
SPARK-3213: 7faf755ae4f0cf510048e432340260a6e609066d
SPARK-3608: 78d4220fa0bf2f9ee663e34bbf3544a5313b02f0.
I tested launch, login, and destroy. It is easy to check the diff by comparing it to Josh's patch for branch-1.1:
https://github.com/apache/spark/pull/2225/files
JoshRosen I sent the PR to master. It might be easier for us to keep master and branch-1.2 the same at this time. We can always re-apply the patch once we figure out a stable solution.
Author: Xiangrui Meng <meng@databricks.com>
Closes #3453 from mengxr/SPARK-4509 and squashes the following commits:
f0b708b [Xiangrui Meng] revert 94053a7b766788bb62e2dbbf352ccbcc75f71fc0
4298ea5 [Xiangrui Meng] revert 7faf755ae4f0cf510048e432340260a6e609066d
35963a1 [Xiangrui Meng] Revert "SPARK-3608 Break if the instance tag naming succeeds"
Diffstat (limited to 'docs/ec2-scripts.md')
| -rw-r--r-- | docs/ec2-scripts.md | 14 |
1 files changed, 6 insertions, 8 deletions
diff --git a/docs/ec2-scripts.md b/docs/ec2-scripts.md index 530798f2b8..66bf5f1a85 100644 --- a/docs/ec2-scripts.md +++ b/docs/ec2-scripts.md @@ -12,16 +12,14 @@ on the [Amazon Web Services site](http://aws.amazon.com/). `spark-ec2` is designed to manage multiple named clusters. You can launch a new cluster (telling the script its size and giving it a name), -shutdown an existing cluster, or log into a cluster. Each cluster -launches a set of instances, which are tagged with the cluster name, -and placed into EC2 security groups. If you don't specify a security -group, the `spark-ec2` script will create security groups based on the -cluster name you request. For example, a cluster named +shutdown an existing cluster, or log into a cluster. Each cluster is +identified by placing its machines into EC2 security groups whose names +are derived from the name of the cluster. For example, a cluster named `test` will contain a master node in a security group called `test-master`, and a number of slave nodes in a security group called -`test-slaves`. You can also specify a security group prefix to be used -in place of the cluster name. Machines in a cluster can be identified -by looking for the "Name" tag of the instance in the Amazon EC2 Console. +`test-slaves`. The `spark-ec2` script will create these security groups +for you based on the cluster name you request. You can also use them to +identify machines belonging to each cluster in the Amazon EC2 Console. # Before You Start |