diff options
| author | Yin Huai <yhuai@databricks.com> | 2015-02-17 15:47:59 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-02-17 15:47:59 -0800 |
| commit | 117121a4ecaadda156a82255333670775e7727db (patch) | |
| tree | d20cb0ce4d7e58623a5526a6e5fd749cdb1ee95a /python | |
| parent | 4d4cc760fa9687ce563320094557ef9144488676 (diff) | |
| download | spark-117121a4ecaadda156a82255333670775e7727db.tar.gz spark-117121a4ecaadda156a82255333670775e7727db.tar.bz2 spark-117121a4ecaadda156a82255333670775e7727db.zip | |
[SPARK-5852][SQL]Fail to convert a newly created empty metastore parquet table to a data source parquet table.
The problem is that after we create an empty hive metastore parquet table (e.g. `CREATE TABLE test (a int) STORED AS PARQUET`), Hive will create an empty dir for us, which cause our data source `ParquetRelation2` fail to get the schema of the table. See JIRA for the case to reproduce the bug and the exception.
This PR is based on #4562 from chenghao-intel.
JIRA: https://issues.apache.org/jira/browse/SPARK-5852
Author: Yin Huai <yhuai@databricks.com>
Author: Cheng Hao <hao.cheng@intel.com>
Closes #4655 from yhuai/CTASParquet and squashes the following commits:
b8b3450 [Yin Huai] Update tests.
2ac94f7 [Yin Huai] Update tests.
3db3d20 [Yin Huai] Minor update.
d7e2308 [Yin Huai] Revert changes in HiveMetastoreCatalog.scala.
36978d1 [Cheng Hao] Update the code as feedback
a04930b [Cheng Hao] fix bug of scan an empty parquet based table
442ffe0 [Cheng Hao] passdown the schema for Parquet File in HiveContext
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions