diff options
| author | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2015-05-26 15:01:27 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-05-26 15:01:27 -0700 |
| commit | 2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a (patch) | |
| tree | 8aca19bcc06903a77d999dad4b4fdb1a5237439a /ec2 | |
| parent | 61664732b25b35f94be35a42cde651cbfd0e02b7 (diff) | |
| download | spark-2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a.tar.gz spark-2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a.tar.bz2 spark-2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a.zip | |
[SPARK-3674] YARN support in Spark EC2
This corresponds to https://github.com/mesos/spark-ec2/pull/116 in the spark-ec2 repo. The only changes required on the spark_ec2.py script is to open the RM port.
cc andrewor14
Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
Closes #6376 from shivaram/spark-ec2-yarn and squashes the following commits:
961504a [Shivaram Venkataraman] Merge branch 'master' of https://github.com/apache/spark into spark-ec2-yarn
152c94c [Shivaram Venkataraman] Open 8088 for YARN in EC2
Diffstat (limited to 'ec2')
| -rwxr-xr-x | ec2/spark_ec2.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py index 724811eaa1..ee0904c9e5 100755 --- a/ec2/spark_ec2.py +++ b/ec2/spark_ec2.py @@ -491,6 +491,8 @@ def launch_cluster(conn, opts, cluster_name): master_group.authorize('udp', 2049, 2049, authorized_address) master_group.authorize('tcp', 4242, 4242, authorized_address) master_group.authorize('udp', 4242, 4242, authorized_address) + # RM in YARN mode uses 8088 + master_group.authorize('tcp', 8088, 8088, authorized_address) if opts.ganglia: master_group.authorize('tcp', 5080, 5080, authorized_address) if slave_group.rules == []: # Group was just now created |