diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2015-11-13 11:25:33 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-11-13 11:25:33 -0800 |
| commit | d7b2b97ad67f9700fb8c13422c399f2edb72f770 (patch) | |
| tree | 8f23b645a0e50ec19feeee1bf0bbfb56c83fc12d /mllib | |
| parent | 23b8188f75d945ef70fbb1c4dc9720c2c5f8cbc3 (diff) | |
| download | spark-d7b2b97ad67f9700fb8c13422c399f2edb72f770.tar.gz spark-d7b2b97ad67f9700fb8c13422c399f2edb72f770.tar.bz2 spark-d7b2b97ad67f9700fb8c13422c399f2edb72f770.zip | |
[SPARK-11727][SQL] Split ExpressionEncoder into FlatEncoder and ProductEncoder
also add more tests for encoders, and fix bugs that I found:
* when convert array to catalyst array, we can only skip element conversion for native types(e.g. int, long, boolean), not `AtomicType`(String is AtomicType but we need to convert it)
* we should also handle scala `BigDecimal` when convert from catalyst `Decimal`.
* complex map type should be supported
other issues that still in investigation:
* encode java `BigDecimal` and decode it back, seems we will loss precision info.
* when encode case class that defined inside a object, `ClassNotFound` exception will be thrown.
I'll remove unused code in a follow-up PR.
Author: Wenchen Fan <wenchen@databricks.com>
Closes #9693 from cloud-fan/split.
Diffstat (limited to 'mllib')
0 files changed, 0 insertions, 0 deletions