diff options
| author | gatorsmile <gatorsmile@gmail.com> | 2016-05-23 18:03:45 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2016-05-23 18:03:45 -0700 |
| commit | 5afd927a47aa7ede3039234f2f7262e2247aa2ae (patch) | |
| tree | d5bd14f5548aca7666ca389af83a6be5701e70b5 /python/pyspark/sql/column.py | |
| parent | 01659bc50cd3d53815d205d005c3678e714c08e0 (diff) | |
| download | spark-5afd927a47aa7ede3039234f2f7262e2247aa2ae.tar.gz spark-5afd927a47aa7ede3039234f2f7262e2247aa2ae.tar.bz2 spark-5afd927a47aa7ede3039234f2f7262e2247aa2ae.zip | |
[SPARK-15311][SQL] Disallow DML on Regular Tables when Using In-Memory Catalog
#### What changes were proposed in this pull request?
So far, when using In-Memory Catalog, we allow DDL operations for the tables. However, the corresponding DML operations are not supported for the tables that are neither temporary nor data source tables. For example,
```SQL
CREATE TABLE tabName(i INT, j STRING)
SELECT * FROM tabName
INSERT OVERWRITE TABLE tabName SELECT 1, 'a'
```
In the above example, before this PR fix, we will get very confusing exception messages for either `SELECT` or `INSERT`
```
org.apache.spark.sql.AnalysisException: unresolved operator 'SimpleCatalogRelation default, CatalogTable(`default`.`tbl`,CatalogTableType(MANAGED),CatalogStorageFormat(None,Some(org.apache.hadoop.mapred.TextInputFormat),Some(org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat),None,false,Map()),List(CatalogColumn(i,int,true,None), CatalogColumn(j,string,true,None)),List(),List(),List(),-1,,1463928681802,-1,Map(),None,None,None,List()), None;
```
This PR is to issue appropriate exceptions in this case. The message will be like
```
org.apache.spark.sql.AnalysisException: Please enable Hive support when operating non-temporary tables: `tbl`;
```
#### How was this patch tested?
Added a test case in `DDLSuite`.
Author: gatorsmile <gatorsmile@gmail.com>
Author: xiaoli <lixiao1983@gmail.com>
Author: Xiao Li <xiaoli@Xiaos-MacBook-Pro.local>
Closes #13093 from gatorsmile/selectAfterCreate.
Diffstat (limited to 'python/pyspark/sql/column.py')
0 files changed, 0 insertions, 0 deletions