diff options
| author | Wenchen Fan <cloud0fan@outlook.com> | 2015-02-09 16:39:34 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-02-09 16:39:34 -0800 |
| commit | 0ee53ebce9944722e76b2b28fae79d9956be9f17 (patch) | |
| tree | 2607124e553ce958e1b60b460e949727532104c4 /dev | |
| parent | 2a36292534a1e9f7a501e88f69bfc3a09fb62cb3 (diff) | |
| download | spark-0ee53ebce9944722e76b2b28fae79d9956be9f17.tar.gz spark-0ee53ebce9944722e76b2b28fae79d9956be9f17.tar.bz2 spark-0ee53ebce9944722e76b2b28fae79d9956be9f17.zip | |
[SPARK-2096][SQL] support dot notation on array of struct
~~The rule is simple: If you want `a.b` work, then `a` must be some level of nested array of struct(level 0 means just a StructType). And the result of `a.b` is same level of nested array of b-type.
An optimization is: the resolve chain looks like `Attribute -> GetItem -> GetField -> GetField ...`, so we could transmit the nested array information between `GetItem` and `GetField` to avoid repeated computation of `innerDataType` and `containsNullList` of that nested array.~~
marmbrus Could you take a look?
to evaluate `a.b`, if `a` is array of struct, then `a.b` means get field `b` on each element of `a`, and return a result of array.
Author: Wenchen Fan <cloud0fan@outlook.com>
Closes #2405 from cloud-fan/nested-array-dot and squashes the following commits:
08a228a [Wenchen Fan] support dot notation on array of struct
Diffstat (limited to 'dev')
0 files changed, 0 insertions, 0 deletions