Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | SPARK-1106: check key name and identity file before launch a cluster | Xiangrui Meng | 2014-02-18 | 1 | -0/+6 |
| | | | | | | | | | | | | I launched an EC2 cluster without providing a key name and an identity file. The error showed up after two minutes. It would be good to check those options before launch, given the fact that EC2 billing rounds up to hours. JIRA: https://spark-project.atlassian.net/browse/SPARK-1106 Author: Xiangrui Meng <meng@databricks.com> Closes #617 from mengxr/ec2 and squashes the following commits: 2dfb316 [Xiangrui Meng] check key name and identity file before launch a cluster | ||||
* | Merge pull request #598 from shivaram/master. | Shivaram Venkataraman | 2014-02-13 | 1 | -2/+2 |
| | | | | | | | | | | | | Update spark_ec2 to use 0.9.0 by default Backports change from branch-0.9 Author: Shivaram Venkataraman <shivaram@eecs.berkeley.edu> Closes #598 and squashes the following commits: f6d3ed0 [Shivaram Venkataraman] Update spark_ec2 to use 0.9.0 by default Backports change from branch-0.9 | ||||
* | Add c3 instance types to Spark EC2 | Christian Lundgren | 2014-02-13 | 1 | -2/+12 |
| | | | | | | | | | | | | | The number of disks for the c3 instance types taken from here: http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/InstanceStorage.html#StorageOnInstanceTypes Author: Christian Lundgren <christian.lundgren@gameanalytics.com> Closes #595 from chrisavl/branch-0.9 and squashes the following commits: c8af5f9 [Christian Lundgren] Add c3 instance types to Spark EC2 (cherry picked from commit 19b4bb2b444f1dbc4592bf3d58b17652e0ae6d6b) Signed-off-by: Patrick Wendell <pwendell@gmail.com> | ||||
* | Add i2 instance types to Spark EC2. | Shivaram Venkataraman | 2014-01-10 | 1 | -2/+10 |
| | |||||
* | a few left over document change | Prashant Sharma | 2014-01-02 | 1 | -1/+1 |
| | |||||
* | Force pseudo-tty allocation in spark-ec2 script. | Ewen Cheslack-Postava | 2013-12-16 | 1 | -2/+2 |
| | | | | | | ssh commands need the -t argument repeated twice if there is no local tty, e.g. if the process running spark-ec2 uses nohup and the parent process exits. | ||||
* | Enable stopping and starting a spot cluster | Ankur Dave | 2013-11-11 | 1 | -19/+32 |
| | |||||
* | expose UI port only | Haoyuan Li | 2013-11-10 | 1 | -1/+1 |
| | |||||
* | add tachyon module | Haoyuan Li | 2013-11-09 | 1 | -7/+8 |
| | |||||
* | update default github | Fabrizio (Misto) Milo | 2013-11-01 | 1 | -1/+1 |
| | |||||
* | Adding Shark 0.7.1 to EC2 scripts | Patrick Wendell | 2013-10-07 | 1 | -1/+1 |
| | |||||
* | Bumping EC2 default version in master to `0.8.0`. | Patrick Wendell | 2013-10-05 | 1 | -2/+2 |
| | |||||
* | Merge pull request #670 from jey/ec2-ssh-improvements | Reynold Xin | 2013-09-26 | 1 | -26/+80 |
|\ | | | | | EC2 SSH improvements | ||||
| * | Clarify error messages on SSH failure | Jey Kottalam | 2013-09-11 | 1 | -6/+21 |
| | | |||||
| * | Generate new SSH key for the cluster, make "--identity-file" optional | Jey Kottalam | 2013-09-06 | 1 | -21/+37 |
| | | |||||
| * | Construct shell commands as sequences for safety and composability | Jey Kottalam | 2013-09-06 | 1 | -11/+34 |
| | | |||||
* | | Fix issue with spark_ec2 seeing empty security groups | Aaron Davidson | 2013-09-19 | 1 | -4/+4 |
| | | | | | | | | | | | | | | | | | | | | Under unknown, but occasional, circumstances, reservation.groups is empty despite reservation.instances each having groups. This means that the spark_ec2 get_existing_clusters() method would fail to find any instances. To fix it, we simply use the instances' groups as the source of truth. Note that this is actually just a revival of PR #827, now that the issue has been reproduced. | ||||
* | | Change port from 3030 to 4040 | Patrick Wendell | 2013-09-11 | 1 | -2/+1 |
|/ | |||||
* | Make initial connection failure message less daunting. | Patrick Wendell | 2013-08-21 | 1 | -1/+1 |
| | | | | | Right now it seems like something has gone wrong when this message is printed out. Instead, this is a normal condition. So I changed the message a bit. | ||||
* | Merge branch 'master' into ec2-updates | Patrick Wendell | 2013-08-21 | 1 | -19/+17 |
|\ | | | | | | | | | Conflicts: ec2/spark_ec2.py | ||||
| * | Allow some wiggle room in UISuite port test and in EC2 ports | Matei Zaharia | 2013-08-19 | 1 | -1/+1 |
| | | |||||
| * | Small fixes to web UI: | Matei Zaharia | 2013-08-19 | 1 | -1/+1 |
| | | | | | | | | | | | | - Use SPARK_PUBLIC_DNS environment variable if set (for EC2) - Use a non-ephemeral port (3030 instead of 33000) by default - Updated test to use non-ephemeral port too | ||||
| * | Open up Job UI ports (33000-33010) on EC2 clusters | Matei Zaharia | 2013-07-29 | 1 | -19/+20 |
| | | |||||
* | | Bump spark version | Patrick Wendell | 2013-08-20 | 1 | -2/+2 |
| | | |||||
* | | Set default Hadoop version to 1 | Patrick Wendell | 2013-08-20 | 1 | -2/+2 |
| | | |||||
* | | Adding web UI port | Patrick Wendell | 2013-08-08 | 1 | -0/+1 |
| | | |||||
* | | Point to new repository/branch | Patrick Wendell | 2013-08-01 | 1 | -1/+1 |
| | | |||||
* | | Pointing to new EC2 repository | Patrick Wendell | 2013-08-01 | 1 | -3/+1 |
| | | |||||
* | | Merge branch 'master' into ec2-updates | Patrick Wendell | 2013-07-31 | 1 | -1/+3 |
|\| | | | | | | | | | Conflicts: ec2/deploy.generic/root/mesos-ec2/ec2-variables.sh | ||||
| * | Add Apache license headers and LICENSE and NOTICE files | Matei Zaharia | 2013-07-16 | 1 | -0/+2 |
| | | |||||
| * | Fix SPARK-670: EC2 start command should require -i option. | Josh Rosen | 2013-05-05 | 1 | -1/+1 |
| | | |||||
* | | Don't install MapReduce for Hadoop1 | Patrick Wendell | 2013-07-30 | 1 | -0/+3 |
| | | |||||
* | | Small improvement in python script | Patrick Wendell | 2013-07-17 | 1 | -1/+1 |
| | | |||||
* | | Bug fix | Patrick Wendell | 2013-07-11 | 1 | -1/+1 |
| | | |||||
* | | Removing support for custom Shark version | Patrick Wendell | 2013-07-11 | 1 | -11/+2 |
| | | |||||
* | | Small fix | Patrick Wendell | 2013-07-11 | 1 | -1/+1 |
| | | |||||
* | | Adding ability to custom deploy only Spark | Patrick Wendell | 2013-07-11 | 1 | -0/+7 |
| | | |||||
* | | Removing mesos support | Patrick Wendell | 2013-06-26 | 1 | -100/+30 |
| | | |||||
* | | Cleanup of AMI tag | Patrick Wendell | 2013-06-13 | 1 | -4/+3 |
| | | |||||
* | | Support for multiple hadoop versions | Patrick Wendell | 2013-06-12 | 1 | -1/+4 |
| | | |||||
* | | Re-working spark versions to use templates | Patrick Wendell | 2013-06-11 | 1 | -9/+23 |
| | | |||||
* | | Remving support for old scripts | Patrick Wendell | 2013-06-11 | 1 | -13/+6 |
| | | |||||
* | | Minor doc cleanup | Patrick Wendell | 2013-06-07 | 1 | -2/+5 |
| | | |||||
* | | Removing now defunct ami documentation | Patrick Wendell | 2013-06-07 | 1 | -2/+1 |
| | | |||||
* | | Fixing syntax error | Patrick Wendell | 2013-06-07 | 1 | -2/+2 |
| | | |||||
* | | Adding spark shark modules | Patrick Wendell | 2013-06-07 | 1 | -3/+4 |
| | | |||||
* | | Decoupling spark version from ami | Patrick Wendell | 2013-06-07 | 1 | -7/+16 |
| | | |||||
* | | Some changes in response to Josh | Patrick Wendell | 2013-05-18 | 1 | -5/+4 |
| | | |||||
* | | Adding support for tag | Patrick Wendell | 2013-05-10 | 1 | -4/+4 |
| | | |||||
* | | Removing unnecessary parsing | Patrick Wendell | 2013-05-09 | 1 | -3/+1 |
| | |