diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-05-06 01:08:04 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2016-05-06 01:08:04 +0800 |
| commit | 55cc1c991a9e39efb14177a948b09b7909e53e25 (patch) | |
| tree | 4ba9dafd35df7e8374688169b72e37a5a51cb196 /R/pkg/.lintr | |
| parent | 77361a433adce109c2b752b11dda25b56eca0352 (diff) | |
| download | spark-55cc1c991a9e39efb14177a948b09b7909e53e25.tar.gz spark-55cc1c991a9e39efb14177a948b09b7909e53e25.tar.bz2 spark-55cc1c991a9e39efb14177a948b09b7909e53e25.zip | |
[SPARK-14139][SQL] RowEncoder should preserve schema nullability
## What changes were proposed in this pull request?
The problem is: In `RowEncoder`, we use `Invoke` to get the field of an external row, which lose the nullability information. This PR creates a `GetExternalRowField` expression, so that we can preserve the nullability info.
TODO: simplify the null handling logic in `RowEncoder`, to remove so many if branches, in follow-up PR.
## How was this patch tested?
new tests in `RowEncoderSuite`
Note that, This PR takes over https://github.com/apache/spark/pull/11980, with a little simplification, so all credits should go to koertkuipers
Author: Wenchen Fan <wenchen@databricks.com>
Author: Koert Kuipers <koert@tresata.com>
Closes #12364 from cloud-fan/nullable.
Diffstat (limited to 'R/pkg/.lintr')
0 files changed, 0 insertions, 0 deletions