aboutsummaryrefslogtreecommitdiff
path: root/ec2/deploy.generic
Commit message (Collapse)AuthorAgeFilesLines
* [SPARK-787] Add S3 configuration parameters to the EC2 deploy scriptsDan Osipov2014-09-161-0/+2
| | | | | | | | | | | | | | | When deploying to AWS, there is additional configuration that is required to read S3 files. EMR creates it automatically, there is no reason that the Spark EC2 script shouldn't. This PR requires a corresponding PR to the mesos/spark-ec2 to be merged, as it gets cloned in the process of setting up machines: https://github.com/mesos/spark-ec2/pull/58 Author: Dan Osipov <daniil.osipov@shazam.com> Closes #1120 from danosipov/s3_credentials and squashes the following commits: 758da8b [Dan Osipov] Modify documentation to include the new parameter 71fab14 [Dan Osipov] Use a parameter --copy-aws-credentials to enable S3 credential deployment 7e0da26 [Dan Osipov] Get AWS credentials out of boto connection instance 39bdf30 [Dan Osipov] Add S3 configuration parameters to the EC2 deploy scripts
* EC2 configurable workersAllan Douglas R. de Oliveira2014-05-031-0/+2
| | | | | | | | | | | | | | Added option to configure number of worker instances and to set SPARK_MASTER_OPTS Depends on: https://github.com/mesos/spark-ec2/pull/46 Author: Allan Douglas R. de Oliveira <allan@chaordicsystems.com> Closes #612 from douglaz/ec2_configurable_workers and squashes the following commits: d6c5d65 [Allan Douglas R. de Oliveira] Added master opts parameter 6c34671 [Allan Douglas R. de Oliveira] Use number of worker instances as string on template ba528b9 [Allan Douglas R. de Oliveira] Added SPARK_WORKER_INSTANCES parameter
* Adding Apache license to two filesPatrick Wendell2013-09-071-0/+17
|
* Merge branch 'master' into ec2-updatesPatrick Wendell2013-07-311-1/+1
|\ | | | | | | | | Conflicts: ec2/deploy.generic/root/mesos-ec2/ec2-variables.sh
| * Consistently invoke bash with /usr/bin/env bash in scripts to make code more ↵Ubuntu2013-07-182-2/+2
| | | | | | | | portable (JIRA Ticket SPARK-817)
* | Removing mesos terminologyPatrick Wendell2013-07-111-7/+6
| |
* | Removing mesos supportPatrick Wendell2013-06-261-9/+0
| |
* | Support for multiple hadoop versionsPatrick Wendell2013-06-121-0/+1
| |
* | Re-working spark versions to use templatesPatrick Wendell2013-06-111-0/+2
|/
* Fix swap variable nameShivaram Venkataraman2013-01-271-1/+1
|
* Update spark_ec2.py to use new spark-ec2 scriptsShivaram Venkataraman2013-01-261-0/+11
|
* Update EC2 scripts for Spark 0.6Matei Zaharia2012-10-121-0/+1
|
* Added script for launching Spark on EC2 from Mesos, to make it easierMatei Zaharia2012-06-101-0/+8
for new users to get up and running on EC2.