diff options
| author | Kevin Conor <kevin@discoverybayconsulting.com> | 2015-06-19 00:12:20 -0700 |
|---|---|---|
| committer | Davies Liu <davies@databricks.com> | 2015-06-19 00:12:20 -0700 |
| commit | fdf63f12490c674cc1877ddf7b70343c4fd6f4f1 (patch) | |
| tree | e251ad4b92e16e9743920d523c96f91e14514668 /python/pyspark/serializers.py | |
| parent | a2016b4bc4ef13339f168c3f4e135fa422046137 (diff) | |
| download | spark-fdf63f12490c674cc1877ddf7b70343c4fd6f4f1.tar.gz spark-fdf63f12490c674cc1877ddf7b70343c4fd6f4f1.tar.bz2 spark-fdf63f12490c674cc1877ddf7b70343c4fd6f4f1.zip | |
[SPARK-8339] [PYSPARK] integer division for python 3
Itertools islice requires an integer for the stop argument. Switching to integer division here prevents a ValueError when vs is evaluated above.
davies
This is my original work, and I license it to the project.
Author: Kevin Conor <kevin@discoverybayconsulting.com>
Closes #6794 from kconor/kconor-patch-1 and squashes the following commits:
da5e700 [Kevin Conor] Integer division for batch size
Diffstat (limited to 'python/pyspark/serializers.py')
| -rw-r--r-- | python/pyspark/serializers.py | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/python/pyspark/serializers.py b/python/pyspark/serializers.py index d8cdcda3a3..7f9d0a338d 100644 --- a/python/pyspark/serializers.py +++ b/python/pyspark/serializers.py @@ -272,7 +272,7 @@ class AutoBatchedSerializer(BatchedSerializer): if size < best: batch *= 2 elif size > best * 10 and batch > 1: - batch /= 2 + batch //= 2 def __repr__(self): return "AutoBatchedSerializer(%s)" % self.serializer |