aboutsummaryrefslogtreecommitdiff
path: root/sql/hive/src/main/scala/org
diff options
context:
space:
mode:
authorgatorsmile <gatorsmile@gmail.com>2016-04-05 11:19:46 +0200
committerHerman van Hovell <hvanhovell@questtec.nl>2016-04-05 11:19:46 +0200
commit78071736799b6c86b5c01b27395f4ab87075342b (patch)
tree08a4b18ebd0563c84cc8540ac81f511838e3810d /sql/hive/src/main/scala/org
parent2715bc68bd1661d207b1af5f44ae8d02aec9d4ec (diff)
downloadspark-78071736799b6c86b5c01b27395f4ab87075342b.tar.gz
spark-78071736799b6c86b5c01b27395f4ab87075342b.tar.bz2
spark-78071736799b6c86b5c01b27395f4ab87075342b.zip
[SPARK-14349][SQL] Issue Error Messages for Unsupported Operators/DML/DDL in SQL Context.
#### What changes were proposed in this pull request? Currently, the weird error messages are issued if we use Hive Context-only operations in SQL Context. For example, - When calling `Drop Table` in SQL Context, we got the following message: ``` Expected exception org.apache.spark.sql.catalyst.parser.ParseException to be thrown, but java.lang.ClassCastException was thrown. ``` - When calling `Script Transform` in SQL Context, we got the message: ``` assertion failed: No plan for ScriptTransformation [key#9,value#10], cat, [tKey#155,tValue#156], null +- LogicalRDD [key#9,value#10], MapPartitionsRDD[3] at beforeAll at BeforeAndAfterAll.scala:187 ``` Updates: Based on the investigation from hvanhovell , the root cause is `visitChildren`, which is the default implementation. It always returns the result of the last defined context child. After merging the code changes from hvanhovell , it works! Thank you hvanhovell ! #### How was this patch tested? A few test cases are added. Not sure if the same issue exist for the other operators/DDL/DML. hvanhovell Author: gatorsmile <gatorsmile@gmail.com> Author: xiaoli <lixiao1983@gmail.com> Author: Herman van Hovell <hvanhovell@questtec.nl> Author: Xiao Li <xiaoli@Xiaos-MacBook-Pro.local> Closes #12134 from gatorsmile/hiveParserCommand.
Diffstat (limited to 'sql/hive/src/main/scala/org')
-rw-r--r--sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveSqlParser.scala16
1 files changed, 15 insertions, 1 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveSqlParser.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveSqlParser.scala
index 12e4f49756..55e69f99a4 100644
--- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveSqlParser.scala
+++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveSqlParser.scala
@@ -134,6 +134,18 @@ class HiveSqlAstBuilder extends SparkSqlAstBuilder {
}
/**
+ * Create a [[CatalogStorageFormat]]. This is part of the [[CreateTableAsSelect]] command.
+ */
+ override def visitCreateFileFormat(
+ ctx: CreateFileFormatContext): CatalogStorageFormat = withOrigin(ctx) {
+ if (ctx.storageHandler == null) {
+ typedVisit[CatalogStorageFormat](ctx.fileFormat)
+ } else {
+ visitStorageHandler(ctx.storageHandler)
+ }
+ }
+
+ /**
* Create a [[CreateTableAsSelect]] command.
*/
override def visitCreateTable(ctx: CreateTableContext): LogicalPlan = {
@@ -282,6 +294,7 @@ class HiveSqlAstBuilder extends SparkSqlAstBuilder {
* Create a [[HiveScriptIOSchema]].
*/
override protected def withScriptIOSchema(
+ ctx: QuerySpecificationContext,
inRowFormat: RowFormatContext,
recordWriter: Token,
outRowFormat: RowFormatContext,
@@ -391,7 +404,8 @@ class HiveSqlAstBuilder extends SparkSqlAstBuilder {
/**
* Storage Handlers are currently not supported in the statements we support (CTAS).
*/
- override def visitStorageHandler(ctx: StorageHandlerContext): AnyRef = withOrigin(ctx) {
+ override def visitStorageHandler(
+ ctx: StorageHandlerContext): CatalogStorageFormat = withOrigin(ctx) {
throw new ParseException("Storage Handlers are currently unsupported.", ctx)
}