aboutsummaryrefslogtreecommitdiff
path: root/sql/catalyst/src/main
diff options
context:
space:
mode:
authorDaoyuan Wang <daoyuan.wang@intel.com>2016-07-07 11:08:06 -0700
committerReynold Xin <rxin@databricks.com>2016-07-07 11:08:06 -0700
commit28710b42b0d18a55bd64d597558649537259b127 (patch)
treefe0f88b0e8c3d656ff4fc842aff80500441da628 /sql/catalyst/src/main
parent0f7175def985a7f1e37198680f893e749612ab76 (diff)
downloadspark-28710b42b0d18a55bd64d597558649537259b127.tar.gz
spark-28710b42b0d18a55bd64d597558649537259b127.tar.bz2
spark-28710b42b0d18a55bd64d597558649537259b127.zip
[SPARK-16415][SQL] fix catalog string error
## What changes were proposed in this pull request? In #13537 we truncate `simpleString` if it is a long `StructType`. But sometimes we need `catalogString` to reconstruct `TypeInfo`, for example in description of [SPARK-16415 ](https://issues.apache.org/jira/browse/SPARK-16415). So we need to keep the implementation of `catalogString` not affected by our truncate. ## How was this patch tested? added a test case. Author: Daoyuan Wang <daoyuan.wang@intel.com> Closes #14089 from adrian-wang/catalogstring.
Diffstat (limited to 'sql/catalyst/src/main')
-rw-r--r--sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala6
1 files changed, 6 insertions, 0 deletions
diff --git a/sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala b/sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala
index 0c2ebb0e5b..dd4c88c4c4 100644
--- a/sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala
+++ b/sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala
@@ -333,6 +333,12 @@ case class StructType(fields: Array[StructField]) extends DataType with Seq[Stru
Utils.truncatedString(fieldTypes, "struct<", ",", ">")
}
+ override def catalogString: String = {
+ // in catalogString, we should not truncate
+ val fieldTypes = fields.map(field => s"${field.name}:${field.dataType.catalogString}")
+ s"struct<${fieldTypes.mkString(",")}>"
+ }
+
override def sql: String = {
val fieldTypes = fields.map(f => s"${quoteIdentifier(f.name)}: ${f.dataType.sql}")
s"STRUCT<${fieldTypes.mkString(", ")}>"