diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-09-06 14:17:47 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-09-06 14:17:47 +0800 |
| commit | c0ae6bc6ea38909730fad36e653d3c7ab0a84b44 (patch) | |
| tree | 25c1e98400ccadb9db221cf02cda423fd0bd7eb4 /docs/spark-standalone.md | |
| parent | 64e826f91eabb1a22d3d163d71fbb7b6d2185f25 (diff) | |
| download | spark-c0ae6bc6ea38909730fad36e653d3c7ab0a84b44.tar.gz spark-c0ae6bc6ea38909730fad36e653d3c7ab0a84b44.tar.bz2 spark-c0ae6bc6ea38909730fad36e653d3c7ab0a84b44.zip | |
[SPARK-17361][SQL] file-based external table without path should not be created
## What changes were proposed in this pull request?
Using the public `Catalog` API, users can create a file-based data source table, without giving the path options. For this case, currently we can create the table successfully, but fail when we read it. Ideally we should fail during creation.
This is because when we create data source table, we resolve the data source relation without validating path: `resolveRelation(checkPathExist = false)`.
Looking back to why we add this trick(`checkPathExist`), it's because when we call `resolveRelation` for managed table, we add the path to data source options but the path is not created yet. So why we add this not-yet-created path to data source options? This PR fix the problem by adding path to options after we call `resolveRelation`. Then we can remove the `checkPathExist` parameter in `DataSource.resolveRelation` and do some related cleanups.
## How was this patch tested?
existing tests and new test in `CatalogSuite`
Author: Wenchen Fan <wenchen@databricks.com>
Closes #14921 from cloud-fan/check-path.
Diffstat (limited to 'docs/spark-standalone.md')
0 files changed, 0 insertions, 0 deletions