diff options
author | linweizhong <linweizhong@huawei.com> | 2015-05-14 00:23:27 -0700 |
---|---|---|
committer | Michael Armbrust <michael@databricks.com> | 2015-05-14 00:23:41 -0700 |
commit | c80e0cff2596cf7e9d149b92980177a43bb501a3 (patch) | |
tree | 25cbf8424150208bba7175346d2223f2bbbc7992 /mllib/src/main | |
parent | 9ab4db29ffa9f0811a6ecc85638784fef7bf9c61 (diff) | |
download | spark-c80e0cff2596cf7e9d149b92980177a43bb501a3.tar.gz spark-c80e0cff2596cf7e9d149b92980177a43bb501a3.tar.bz2 spark-c80e0cff2596cf7e9d149b92980177a43bb501a3.zip |
[SPARK-7595] [SQL] Window will cause resolve failed with self join
for example:
table: src(key string, value string)
sql: with v1 as(select key, count(value) over (partition by key) cnt_val from src), v2 as(select v1.key, v1_lag.cnt_val from v1, v1 v1_lag where v1.key = v1_lag.key) select * from v2 limit 5;
then will analyze fail when resolving conflicting references in Join:
'Limit 5
'Project [*]
'Subquery v2
'Project ['v1.key,'v1_lag.cnt_val]
'Filter ('v1.key = 'v1_lag.key)
'Join Inner, None
Subquery v1
Project [key#95,cnt_val#94L]
Window [key#95,value#96], [HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCount(value#96) WindowSpecDefinition [key#95], [], ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING AS cnt_val#94L], WindowSpecDefinition [key#95], [], ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
Project [key#95,value#96]
MetastoreRelation default, src, None
Subquery v1_lag
Subquery v1
Project [key#97,cnt_val#94L]
Window [key#97,value#98], [HiveWindowFunction#org.apache.hadoop.hive.ql.udf.generic.GenericUDAFCount(value#98) WindowSpecDefinition [key#97], [], ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING AS cnt_val#94L], WindowSpecDefinition [key#97], [], ROWS BETWEEN UNBOUNDED PRECEDING AND UNBOUNDED FOLLOWING
Project [key#97,value#98]
MetastoreRelation default, src, None
Conflicting attributes: cnt_val#94L
Author: linweizhong <linweizhong@huawei.com>
Closes #6114 from Sephiroth-Lin/spark-7595 and squashes the following commits:
f8f2637 [linweizhong] Add unit test
dfe9169 [linweizhong] Handle windowExpression with self join
(cherry picked from commit 13e652b61a81b2d2e94088006fbd5fd4ed383e3d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
Diffstat (limited to 'mllib/src/main')
0 files changed, 0 insertions, 0 deletions