diff options
author | Ward Viaene <ward.viaene@bigdatapartnership.com> | 2014-09-07 18:54:36 -0700 |
---|---|---|
committer | Josh Rosen <joshrosen@apache.org> | 2014-09-07 18:54:36 -0700 |
commit | ecfa76cdfe846c75e1b7ebc556167e46963289c5 (patch) | |
tree | 00813772c2aa51ae1edc2714aa408a39c10d3dd1 /python/pyspark/tests.py | |
parent | e2614038e78f4693fafedeee15b6fdf0ea1be473 (diff) | |
download | spark-ecfa76cdfe846c75e1b7ebc556167e46963289c5.tar.gz spark-ecfa76cdfe846c75e1b7ebc556167e46963289c5.tar.bz2 spark-ecfa76cdfe846c75e1b7ebc556167e46963289c5.zip |
[SPARK-3415] [PySpark] removes SerializingAdapter code
This code removes the SerializingAdapter code that was copied from PiCloud
Author: Ward Viaene <ward.viaene@bigdatapartnership.com>
Closes #2287 from wardviaene/feature/pythonsys and squashes the following commits:
5f0d426 [Ward Viaene] SPARK-3415: modified test class to do dump and load
5f5d559 [Ward Viaene] SPARK-3415: modified test class name and call cloudpickle.dumps instead using StringIO
afc4a9a [Ward Viaene] SPARK-3415: added newlines to pass lint
aaf10b7 [Ward Viaene] SPARK-3415: removed references to SerializingAdapter and rewrote test
65ffeff [Ward Viaene] removed duplicate test
a958866 [Ward Viaene] SPARK-3415: test script
e263bf5 [Ward Viaene] SPARK-3415: removes legacy SerializingAdapter code
Diffstat (limited to 'python/pyspark/tests.py')
-rw-r--r-- | python/pyspark/tests.py | 11 |
1 files changed, 11 insertions, 0 deletions
diff --git a/python/pyspark/tests.py b/python/pyspark/tests.py index 2ade15b35a..9fbeb36f4f 100644 --- a/python/pyspark/tests.py +++ b/python/pyspark/tests.py @@ -169,6 +169,17 @@ class SerializationTestCase(unittest.TestCase): self.assertEquals(p1, p2) +# Regression test for SPARK-3415 +class CloudPickleTest(unittest.TestCase): + def test_pickling_file_handles(self): + from pyspark.cloudpickle import dumps + from StringIO import StringIO + from pickle import load + out1 = sys.stderr + out2 = load(StringIO(dumps(out1))) + self.assertEquals(out1, out2) + + class PySparkTestCase(unittest.TestCase): def setUp(self): |