aboutsummaryrefslogtreecommitdiff
path: root/sql/core
diff options
context:
space:
mode:
authorYin Huai <yhuai@databricks.com>2015-02-13 13:51:06 -0800
committerMichael Armbrust <michael@databricks.com>2015-02-13 13:51:06 -0800
commit2e0c084528409e1c565e6945521a33c0835ebbee (patch)
tree4d96b325d0ed771c898936c8572d228389f75e99 /sql/core
parent2cbb3e433ae334d5c318f05b987af314c854fbcc (diff)
downloadspark-2e0c084528409e1c565e6945521a33c0835ebbee.tar.gz
spark-2e0c084528409e1c565e6945521a33c0835ebbee.tar.bz2
spark-2e0c084528409e1c565e6945521a33c0835ebbee.zip
[SPARK-5789][SQL]Throw a better error message if JsonRDD.parseJson encounters unrecoverable parsing errors.
Author: Yin Huai <yhuai@databricks.com> Closes #4582 from yhuai/jsonErrorMessage and squashes the following commits: 152dbd4 [Yin Huai] Update error message. 1466256 [Yin Huai] Throw a better error message when a JSON object in the input dataset span multiple records (lines for files or strings for an RDD of strings).
Diffstat (limited to 'sql/core')
-rw-r--r--sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala4
1 files changed, 4 insertions, 0 deletions
diff --git a/sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala b/sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala
index 7dfb304021..3b8dde1823 100644
--- a/sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala
+++ b/sql/core/src/main/scala/org/apache/spark/sql/json/JsonRDD.scala
@@ -306,6 +306,10 @@ private[sql] object JsonRDD extends Logging {
val parsed = mapper.readValue(record, classOf[Object]) match {
case map: java.util.Map[_, _] => scalafy(map).asInstanceOf[Map[String, Any]] :: Nil
case list: java.util.List[_] => scalafy(list).asInstanceOf[Seq[Map[String, Any]]]
+ case _ =>
+ sys.error(
+ s"Failed to parse record $record. Please make sure that each line of the file " +
+ "(or each string in the RDD) is a valid JSON object or an array of JSON objects.")
}
parsed