aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark/ml/regression.py
diff options
context:
space:
mode:
authorSameer Agarwal <sameer@databricks.com>2016-04-08 17:23:32 -0700
committerDavies Liu <davies.liu@gmail.com>2016-04-08 17:23:32 -0700
commit813e96e6faee44079eb52acbdc6c8aa58fb8d191 (patch)
treec9b8bfd92499f34c3a3b82a4ea5c16af59e2c7dd /python/pyspark/ml/regression.py
parent4d7c35926371f9e016577987c037abcf984443d9 (diff)
downloadspark-813e96e6faee44079eb52acbdc6c8aa58fb8d191.tar.gz
spark-813e96e6faee44079eb52acbdc6c8aa58fb8d191.tar.bz2
spark-813e96e6faee44079eb52acbdc6c8aa58fb8d191.zip
[SPARK-14454] Better exception handling while marking tasks as failed
## What changes were proposed in this pull request? This patch adds support for better handling of exceptions inside catch blocks if the code within the block throws an exception. For instance here is the code in a catch block before this change in `WriterContainer.scala`: ```scala logError("Aborting task.", cause) // call failure callbacks first, so we could have a chance to cleanup the writer. TaskContext.get().asInstanceOf[TaskContextImpl].markTaskFailed(cause) if (currentWriter != null) { currentWriter.close() } abortTask() throw new SparkException("Task failed while writing rows.", cause) ``` If `markTaskFailed` or `currentWriter.close` throws an exception, we currently lose the original cause. This PR fixes this problem by implementing a utility function `Utils.tryWithSafeCatch` that suppresses (`Throwable.addSuppressed`) the exception that are thrown within the catch block and rethrowing the original exception. ## How was this patch tested? No new functionality added Author: Sameer Agarwal <sameer@databricks.com> Closes #12234 from sameeragarwal/fix-exception.
Diffstat (limited to 'python/pyspark/ml/regression.py')
0 files changed, 0 insertions, 0 deletions