diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-06-27 14:40:45 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2015-06-27 14:41:03 -0700 |
| commit | e1bbf1a080296e4d3d692379f06b7db0d0629573 (patch) | |
| tree | b68e0a5e2813bd2a767c1a5f72716ee63a15a6a3 /python/pyspark | |
| parent | a2dbb4807136b3c66ffd353340a54ad704c6f99e (diff) | |
| download | spark-e1bbf1a080296e4d3d692379f06b7db0d0629573.tar.gz spark-e1bbf1a080296e4d3d692379f06b7db0d0629573.tar.bz2 spark-e1bbf1a080296e4d3d692379f06b7db0d0629573.zip | |
[SPARK-8606] Prevent exceptions in RDD.getPreferredLocations() from crashing DAGScheduler
If `RDD.getPreferredLocations()` throws an exception it may crash the DAGScheduler and SparkContext. This patch addresses this by adding a try-catch block.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #7023 from JoshRosen/SPARK-8606 and squashes the following commits:
770b169 [Josh Rosen] Fix getPreferredLocations() DAGScheduler crash with try block.
44a9b55 [Josh Rosen] Add test of a buggy getPartitions() method
19aa9f7 [Josh Rosen] Add (failing) regression test for getPreferredLocations() DAGScheduler crash
(cherry picked from commit 0b5abbf5f96a5f6bfd15a65e8788cf3fa96fe54c)
Signed-off-by: Josh Rosen <joshrosen@databricks.com>
Diffstat (limited to 'python/pyspark')
0 files changed, 0 insertions, 0 deletions