diff options
| author | eyal farago <eyal farago> | 2016-11-02 11:12:20 +0100 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-11-02 11:12:20 +0100 |
| commit | f151bd1af8a05d4b6c901ebe6ac0b51a4a1a20df (patch) | |
| tree | ca9c328efdb1cf9961223196b0396800714eb72b /sql/core/src/test/resources | |
| parent | 9c8deef64efee20a0ddc9b612f90e77c80aede60 (diff) | |
| download | spark-f151bd1af8a05d4b6c901ebe6ac0b51a4a1a20df.tar.gz spark-f151bd1af8a05d4b6c901ebe6ac0b51a4a1a20df.tar.bz2 spark-f151bd1af8a05d4b6c901ebe6ac0b51a4a1a20df.zip | |
[SPARK-16839][SQL] Simplify Struct creation code path
## What changes were proposed in this pull request?
Simplify struct creation, especially the aspect of `CleanupAliases` which missed some aliases when handling trees created by `CreateStruct`.
This PR includes:
1. A failing test (create struct with nested aliases, some of the aliases survive `CleanupAliases`).
2. A fix that transforms `CreateStruct` into a `CreateNamedStruct` constructor, effectively eliminating `CreateStruct` from all expression trees.
3. A `NamePlaceHolder` used by `CreateStruct` when column names cannot be extracted from unresolved `NamedExpression`.
4. A new Analyzer rule that resolves `NamePlaceHolder` into a string literal once the `NamedExpression` is resolved.
5. `CleanupAliases` code was simplified as it no longer has to deal with `CreateStruct`'s top level columns.
## How was this patch tested?
Running all tests-suits in package org.apache.spark.sql, especially including the analysis suite, making sure added test initially fails, after applying suggested fix rerun the entire analysis package successfully.
Modified few tests that expected `CreateStruct` which is now transformed into `CreateNamedStruct`.
Author: eyal farago <eyal farago>
Author: Herman van Hovell <hvanhovell@databricks.com>
Author: eyal farago <eyal.farago@gmail.com>
Author: Eyal Farago <eyal.farago@actimize.com>
Author: Hyukjin Kwon <gurwls223@gmail.com>
Author: eyalfa <eyal.farago@gmail.com>
Closes #15718 from hvanhovell/SPARK-16839-2.
Diffstat (limited to 'sql/core/src/test/resources')
| -rw-r--r-- | sql/core/src/test/resources/sql-tests/results/group-by.sql.out | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/core/src/test/resources/sql-tests/results/group-by.sql.out b/sql/core/src/test/resources/sql-tests/results/group-by.sql.out index a91f04e098..af6c930d64 100644 --- a/sql/core/src/test/resources/sql-tests/results/group-by.sql.out +++ b/sql/core/src/test/resources/sql-tests/results/group-by.sql.out @@ -87,7 +87,7 @@ struct<foo:string,approx_count_distinct(a):bigint> -- !query 9 SELECT 'foo', MAX(STRUCT(a)) FROM testData WHERE a = 0 GROUP BY 1 -- !query 9 schema -struct<foo:string,max(struct(a)):struct<a:int>> +struct<foo:string,max(named_struct(a, a)):struct<a:int>> -- !query 9 output |