diff options
| author | Kay Ousterhout <kayousterhout@gmail.com> | 2014-06-18 13:16:26 -0700 |
|---|---|---|
| committer | Kay Ousterhout <kayousterhout@gmail.com> | 2014-06-18 13:16:26 -0700 |
| commit | 3870248740d83b0292ccca88a494ce19783847f0 (patch) | |
| tree | a619c9f31901a2f4f2f572ea9b5be75c23f537b1 /yarn | |
| parent | dd96fcda0145810785b67f847f98b04ff7f0d7c3 (diff) | |
| download | spark-3870248740d83b0292ccca88a494ce19783847f0.tar.gz spark-3870248740d83b0292ccca88a494ce19783847f0.tar.bz2 spark-3870248740d83b0292ccca88a494ce19783847f0.zip | |
[SPARK-1466] Raise exception if pyspark Gateway process doesn't start.
If the gateway process fails to start correctly (e.g., because JAVA_HOME isn't set correctly, there's no Spark jar, etc.), right now pyspark fails because of a very difficult-to-understand error, where we try to parse stdout to get the port where Spark started and there's nothing there. This commit properly catches the error and throws an exception that includes the stderr output for much easier debugging.
Thanks to @shivaram and @stogers for helping to fix this issue!
Author: Kay Ousterhout <kayousterhout@gmail.com>
Closes #383 from kayousterhout/pyspark and squashes the following commits:
36dd54b [Kay Ousterhout] [SPARK-1466] Raise exception if Gateway process doesn't start.
Diffstat (limited to 'yarn')
0 files changed, 0 insertions, 0 deletions