diff options
| author | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2015-05-26 15:01:27 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-05-26 15:01:44 -0700 |
| commit | d014a447a361f65db8986e839e2f9a7a7a356f57 (patch) | |
| tree | baeae3e335aa2dac1f7dabfaabb1da7b8f04361d /ec2/spark_ec2.py | |
| parent | 51d98b0e97c97a7eca2d4ff2fc14b9cfe9af9e2f (diff) | |
| download | spark-d014a447a361f65db8986e839e2f9a7a7a356f57.tar.gz spark-d014a447a361f65db8986e839e2f9a7a7a356f57.tar.bz2 spark-d014a447a361f65db8986e839e2f9a7a7a356f57.zip | |
[SPARK-3674] YARN support in Spark EC2
This corresponds to https://github.com/mesos/spark-ec2/pull/116 in the spark-ec2 repo. The only changes required on the spark_ec2.py script is to open the RM port.
cc andrewor14
Author: Shivaram Venkataraman <shivaram@cs.berkeley.edu>
Closes #6376 from shivaram/spark-ec2-yarn and squashes the following commits:
961504a [Shivaram Venkataraman] Merge branch 'master' of https://github.com/apache/spark into spark-ec2-yarn
152c94c [Shivaram Venkataraman] Open 8088 for YARN in EC2
(cherry picked from commit 2e9a5f229e1a2ccffa74fa59fa6a55b2704d9c1a)
Signed-off-by: Andrew Or <andrew@databricks.com>
Diffstat (limited to 'ec2/spark_ec2.py')
| -rwxr-xr-x | ec2/spark_ec2.py | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py index 2fab5c428d..2154046a3a 100755 --- a/ec2/spark_ec2.py +++ b/ec2/spark_ec2.py @@ -492,6 +492,8 @@ def launch_cluster(conn, opts, cluster_name): master_group.authorize('udp', 2049, 2049, authorized_address) master_group.authorize('tcp', 4242, 4242, authorized_address) master_group.authorize('udp', 4242, 4242, authorized_address) + # RM in YARN mode uses 8088 + master_group.authorize('tcp', 8088, 8088, authorized_address) if opts.ganglia: master_group.authorize('tcp', 5080, 5080, authorized_address) if slave_group.rules == []: # Group was just now created |