diff options
author | Reynold Xin <rxin@databricks.com> | 2016-03-25 01:17:23 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-03-25 01:17:23 -0700 |
commit | 70a6f0bb57ca2248444157e2707fbcc3cb04e3bc (patch) | |
tree | bf5d71237ea18b50668937647d678b378df3fb8a /sql/core | |
parent | 20ddf5fddf40b543edc61d6e4687988489dea64c (diff) | |
download | spark-70a6f0bb57ca2248444157e2707fbcc3cb04e3bc.tar.gz spark-70a6f0bb57ca2248444157e2707fbcc3cb04e3bc.tar.bz2 spark-70a6f0bb57ca2248444157e2707fbcc3cb04e3bc.zip |
[SPARK-14149] Log exceptions in tryOrIOException
## What changes were proposed in this pull request?
We ran into a problem today debugging some class loading problem during deserialization, and JVM was masking the underlying exception which made it very difficult to debug. We can however log the exceptions using try/catch ourselves in serialization/deserialization. The good thing is that all these methods are already using Utils.tryOrIOException, so we can just put the try catch and logging in a single place.
## How was this patch tested?
A logging change with a manual test.
Author: Reynold Xin <rxin@databricks.com>
Closes #11951 from rxin/SPARK-14149.
Diffstat (limited to 'sql/core')
-rw-r--r-- | sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala | 8 |
1 files changed, 3 insertions, 5 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala index a30b52080f..e31380e17d 100644 --- a/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala +++ b/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelation.scala @@ -88,11 +88,9 @@ private[sql] case class InsertIntoHadoopFsRelation( case (SaveMode.ErrorIfExists, true) => throw new AnalysisException(s"path $qualifiedOutputPath already exists.") case (SaveMode.Overwrite, true) => - Utils.tryOrIOException { - if (!fs.delete(qualifiedOutputPath, true /* recursively */)) { - throw new IOException(s"Unable to clear output " + - s"directory $qualifiedOutputPath prior to writing to it") - } + if (!fs.delete(qualifiedOutputPath, true /* recursively */)) { + throw new IOException(s"Unable to clear output " + + s"directory $qualifiedOutputPath prior to writing to it") } true case (SaveMode.Append, _) | (SaveMode.Overwrite, _) | (SaveMode.ErrorIfExists, false) => |