diff options
| author | Davies Liu <davies.liu@gmail.com> | 2014-09-15 18:57:25 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@apache.org> | 2014-09-15 18:57:25 -0700 |
| commit | da33acb8b681eca5e787d546fe922af76a151398 (patch) | |
| tree | 1160991b887d207efc3cf3fcad786de8811570e2 /ec2 | |
| parent | fdb302f49c021227026909bdcdade7496059013f (diff) | |
| download | spark-da33acb8b681eca5e787d546fe922af76a151398.tar.gz spark-da33acb8b681eca5e787d546fe922af76a151398.tar.bz2 spark-da33acb8b681eca5e787d546fe922af76a151398.zip | |
[SPARK-2951] [PySpark] support unpickle array.array for Python 2.6
Pyrolite can not unpickle array.array which pickled by Python 2.6, this patch fix it by extend Pyrolite.
There is a bug in Pyrolite when unpickle array of float/double, this patch workaround it by reverse the endianness for float/double. This workaround should be removed after Pyrolite have a new release to fix this issue.
I had send an PR to Pyrolite to fix it: https://github.com/irmen/Pyrolite/pull/11
Author: Davies Liu <davies.liu@gmail.com>
Closes #2365 from davies/pickle and squashes the following commits:
f44f771 [Davies Liu] enable tests about array
3908f5c [Davies Liu] Merge branch 'master' into pickle
c77c87b [Davies Liu] cleanup debugging code
60e4e2f [Davies Liu] support unpickle array.array for Python 2.6
Diffstat (limited to 'ec2')
0 files changed, 0 insertions, 0 deletions