diff options
author | Anant <anant.asty@gmail.com> | 2014-06-16 23:42:27 -0700 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-06-16 23:45:31 -0700 |
commit | 6b5f64aafcfe023851039f8caf7ba71e7d56d26e (patch) | |
tree | a11c566bc5808044bde4e402d86f32e79e8b2777 /ec2 | |
parent | d7db2e63a474d1fae3864872269f248df64e6510 (diff) | |
download | spark-6b5f64aafcfe023851039f8caf7ba71e7d56d26e.tar.gz spark-6b5f64aafcfe023851039f8caf7ba71e7d56d26e.tar.bz2 spark-6b5f64aafcfe023851039f8caf7ba71e7d56d26e.zip |
SPARK-1990: added compatibility for python 2.6 for ssh_read command
https://issues.apache.org/jira/browse/SPARK-1990
There were some posts on the lists that spark-ec2 does not work with Python 2.6. In addition, we should check the Python version at the top of the script and exit if it's too old
Author: Anant <anant.asty@gmail.com>
Closes #941 from anantasty/SPARK-1990 and squashes the following commits:
4ca441d [Anant] Implmented check_optput withinthe module to work with python 2.6
c6ed85c [Anant] added compatibility for python 2.6 for ssh_read command
Conflicts:
ec2/spark_ec2.py
Diffstat (limited to 'ec2')
-rwxr-xr-x | ec2/spark_ec2.py | 16 |
1 files changed, 15 insertions, 1 deletions
diff --git a/ec2/spark_ec2.py b/ec2/spark_ec2.py index d2c29d0db0..89244642f9 100755 --- a/ec2/spark_ec2.py +++ b/ec2/spark_ec2.py @@ -642,9 +642,23 @@ def ssh(host, opts, command): time.sleep(30) tries = tries + 1 +# Backported from Python 2.7 for compatiblity with 2.6 (See SPARK-1990) +def _check_output(*popenargs, **kwargs): + if 'stdout' in kwargs: + raise ValueError('stdout argument not allowed, it will be overridden.') + process = subprocess.Popen(stdout=PIPE, *popenargs, **kwargs) + output, unused_err = process.communicate() + retcode = process.poll() + if retcode: + cmd = kwargs.get("args") + if cmd is None: + cmd = popenargs[0] + raise subprocess.CalledProcessError(retcode, cmd, output=output) + return output + def ssh_read(host, opts, command): - return subprocess.check_output( + return _check_output( ssh_command(opts) + ['%s@%s' % (opts.user, host), stringify_command(command)]) |