diff options
| author | gatorsmile <gatorsmile@gmail.com> | 2016-11-07 18:34:21 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-11-07 18:34:21 -0800 |
| commit | 1da64e1fa0970277d1fb47dec8adca47b068b1ec (patch) | |
| tree | a9eefedc47b08c544cc24863382cd8366df6bed0 /docs | |
| parent | c1a0c66bd2662bc40f312da474c3b95229fe92d0 (diff) | |
| download | spark-1da64e1fa0970277d1fb47dec8adca47b068b1ec.tar.gz spark-1da64e1fa0970277d1fb47dec8adca47b068b1ec.tar.bz2 spark-1da64e1fa0970277d1fb47dec8adca47b068b1ec.zip | |
[SPARK-18217][SQL] Disallow creating permanent views based on temporary views or UDFs
### What changes were proposed in this pull request?
Based on the discussion in [SPARK-18209](https://issues.apache.org/jira/browse/SPARK-18209). It doesn't really make sense to create permanent views based on temporary views or temporary UDFs.
To disallow the supports and issue the exceptions, this PR needs to detect whether a temporary view/UDF is being used when defining a permanent view. Basically, this PR can be split to two sub-tasks:
**Task 1:** detecting a temporary view from the query plan of view definition.
When finding an unresolved temporary view, Analyzer replaces it by a `SubqueryAlias` with the corresponding logical plan, which is stored in an in-memory HashMap. After replacement, it is impossible to detect whether the `SubqueryAlias` is added/generated from a temporary view. Thus, to detect the usage of a temporary view in view definition, this PR traverses the unresolved logical plan and uses the name of an `UnresolvedRelation` to detect whether it is a (global) temporary view.
**Task 2:** detecting a temporary UDF from the query plan of view definition.
Detecting usage of a temporary UDF in view definition is not straightfoward.
First, in the analyzed plan, we are having different forms to represent the functions. More importantly, some classes (e.g., `HiveGenericUDF`) are not accessible from `CreateViewCommand`, which is part of `sql/core`. Thus, we used the unanalyzed plan `child` of `CreateViewCommand` to detect the usage of a temporary UDF. Because the plan has already been successfully analyzed, we can assume the functions have been defined/registered.
Second, in Spark, the functions have four forms: Spark built-in functions, built-in hash functions, permanent UDFs and temporary UDFs. We do not have any direct way to determine whether a function is temporary or not. Thus, we introduced a function `isTemporaryFunction` in `SessionCatalog`. This function contains the detailed logics to determine whether a function is temporary or not.
### How was this patch tested?
Added test cases.
Author: gatorsmile <gatorsmile@gmail.com>
Closes #15764 from gatorsmile/blockTempFromPermViewCreation.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions