diff options
author | wangfei <wangfei1@huawei.com> | 2015-05-07 22:55:42 -0700 |
---|---|---|
committer | Yin Huai <yhuai@databricks.com> | 2015-05-07 22:57:15 -0700 |
commit | bb5872f2d099bd6d679c20c2a4ff28eb35143e6d (patch) | |
tree | 0924dcaa03cbf3e29fc7af08e24cdfb18c88c9a7 /yarn | |
parent | 1a3e9e9828e12836a2600a62a8afefb23f7386d4 (diff) | |
download | spark-bb5872f2d099bd6d679c20c2a4ff28eb35143e6d.tar.gz spark-bb5872f2d099bd6d679c20c2a4ff28eb35143e6d.tar.bz2 spark-bb5872f2d099bd6d679c20c2a4ff28eb35143e6d.zip |
[SPARK-7232] [SQL] Add a Substitution batch for spark sql analyzer
Added a new batch named `Substitution` before `Resolution` batch. The motivation for this is there are kind of cases we want to do some substitution on the parsed logical plan before resolve it.
Consider this two cases:
1 CTE, for cte we first build a row logical plan
```
'With Map(q1 -> 'Subquery q1
'Project ['key]
'UnresolvedRelation [src], None)
'Project [*]
'Filter ('key = 5)
'UnresolvedRelation [q1], None
```
In `With` logicalplan here is a map stored the (`q1-> subquery`), we want first take off the with command and substitute the `q1` of `UnresolvedRelation` by the `subquery`
2 Another example is Window function, in window function user may define some windows, we also need substitute the window name of child by the concrete window. this should also done in the Substitution batch.
Author: wangfei <wangfei1@huawei.com>
Closes #5776 from scwf/addbatch and squashes the following commits:
d4b962f [wangfei] added WindowsSubstitution
70f6932 [wangfei] Merge branch 'master' of https://github.com/apache/spark into addbatch
ecaeafb [wangfei] address yhuai's comments
553005a [wangfei] fix test case
0c54798 [wangfei] address comments
29aaaaf [wangfei] fix compile
1c9a092 [wangfei] added Substitution bastch
(cherry picked from commit f496bf3c539a873ffdf3aa803847ef7b50135bd7)
Signed-off-by: Yin Huai <yhuai@databricks.com>
Diffstat (limited to 'yarn')
0 files changed, 0 insertions, 0 deletions