diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2017-04-04 16:38:32 -0700 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2017-04-04 16:38:32 -0700 |
| commit | 295747e59739ee8a697ac3eba485d3439e4a04c3 (patch) | |
| tree | 7eaf925a2f58e8beccde78aa652befea5209e092 /mllib/src/test | |
| parent | 402bf2a50ddd4039ff9f376b641bd18fffa54171 (diff) | |
| download | spark-295747e59739ee8a697ac3eba485d3439e4a04c3.tar.gz spark-295747e59739ee8a697ac3eba485d3439e4a04c3.tar.bz2 spark-295747e59739ee8a697ac3eba485d3439e4a04c3.zip | |
[SPARK-19716][SQL] support by-name resolution for struct type elements in array
## What changes were proposed in this pull request?
Previously when we construct deserializer expression for array type, we will first cast the corresponding field to expected array type and then apply `MapObjects`.
However, by doing that, we lose the opportunity to do by-name resolution for struct type inside array type. In this PR, I introduce a `UnresolvedMapObjects` to hold the lambda function and the input array expression. Then during analysis, after the input array expression is resolved, we get the actual array element type and apply by-name resolution. Then we don't need to add `Cast` for array type when constructing the deserializer expression, as the element type is determined later at analyzer.
## How was this patch tested?
new regression test
Author: Wenchen Fan <wenchen@databricks.com>
Closes #17398 from cloud-fan/dataset.
Diffstat (limited to 'mllib/src/test')
0 files changed, 0 insertions, 0 deletions