diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-06-17 13:59:39 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-06-17 13:59:39 -0700 |
| commit | 0fc4b96f3e3bf81724ac133a6acc97c1b77271b4 (patch) | |
| tree | 7d5e783967ac602ce979fe0f15a3d3613f44a4f8 /python/pyspark/rdd.py | |
| parent | 2837e067099921dd4ab6639ac5f6e89f789d4ff4 (diff) | |
| download | spark-0fc4b96f3e3bf81724ac133a6acc97c1b77271b4.tar.gz spark-0fc4b96f3e3bf81724ac133a6acc97c1b77271b4.tar.bz2 spark-0fc4b96f3e3bf81724ac133a6acc97c1b77271b4.zip | |
[SPARK-8373] [PYSPARK] Add emptyRDD to pyspark and fix the issue when calling sum on an empty RDD
This PR fixes the sum issue and also adds `emptyRDD` so that it's easy to create a test case.
Author: zsxwing <zsxwing@gmail.com>
Closes #6826 from zsxwing/python-emptyRDD and squashes the following commits:
b36993f [zsxwing] Update the return type to JavaRDD[T]
71df047 [zsxwing] Add emptyRDD to pyspark and fix the issue when calling sum on an empty RDD
Diffstat (limited to 'python/pyspark/rdd.py')
| -rw-r--r-- | python/pyspark/rdd.py | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/python/pyspark/rdd.py b/python/pyspark/rdd.py index 98a8ff8606..20c0bc93f4 100644 --- a/python/pyspark/rdd.py +++ b/python/pyspark/rdd.py @@ -960,7 +960,7 @@ class RDD(object): >>> sc.parallelize([1.0, 2.0, 3.0]).sum() 6.0 """ - return self.mapPartitions(lambda x: [sum(x)]).reduce(operator.add) + return self.mapPartitions(lambda x: [sum(x)]).fold(0, operator.add) def count(self): """ |