diff options
| author | Allan Douglas R. de Oliveira <allan@chaordicsystems.com> | 2014-05-03 16:52:19 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-05-03 16:52:19 -0700 |
| commit | 4669a84ab10296e01a5fbbae1de9574b793b7ed5 (patch) | |
| tree | b295f1f39cb84ebbb9ee3e1f758bef54a154abd5 /ec2/deploy.generic/root | |
| parent | 34719ba32ed421701eaa08bd47ce953cd9267ad7 (diff) | |
| download | spark-4669a84ab10296e01a5fbbae1de9574b793b7ed5.tar.gz spark-4669a84ab10296e01a5fbbae1de9574b793b7ed5.tar.bz2 spark-4669a84ab10296e01a5fbbae1de9574b793b7ed5.zip | |
EC2 configurable workers
Added option to configure number of worker instances and to set SPARK_MASTER_OPTS
Depends on: https://github.com/mesos/spark-ec2/pull/46
Author: Allan Douglas R. de Oliveira <allan@chaordicsystems.com>
Closes #612 from douglaz/ec2_configurable_workers and squashes the following commits:
d6c5d65 [Allan Douglas R. de Oliveira] Added master opts parameter
6c34671 [Allan Douglas R. de Oliveira] Use number of worker instances as string on template
ba528b9 [Allan Douglas R. de Oliveira] Added SPARK_WORKER_INSTANCES parameter
Diffstat (limited to 'ec2/deploy.generic/root')
| -rw-r--r-- | ec2/deploy.generic/root/spark-ec2/ec2-variables.sh | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/ec2/deploy.generic/root/spark-ec2/ec2-variables.sh b/ec2/deploy.generic/root/spark-ec2/ec2-variables.sh index 42e8faa26e..3570891be8 100644 --- a/ec2/deploy.generic/root/spark-ec2/ec2-variables.sh +++ b/ec2/deploy.generic/root/spark-ec2/ec2-variables.sh @@ -28,3 +28,5 @@ export SPARK_VERSION="{{spark_version}}" export SHARK_VERSION="{{shark_version}}" export HADOOP_MAJOR_VERSION="{{hadoop_major_version}}" export SWAP_MB="{{swap}}" +export SPARK_WORKER_INSTANCES="{{spark_worker_instances}}" +export SPARK_MASTER_OPTS="{{spark_master_opts}}" |