diff options
author | Anant <anant.asty@gmail.com> | 2014-06-20 18:54:00 -0700 |
---|---|---|
committer | Patrick Wendell <pwendell@gmail.com> | 2014-06-20 18:57:24 -0700 |
commit | 010c460d627c1917dc47b09e59fd41172bbf90b3 (patch) | |
tree | 73fd45ff574c2a335198ca50d316588397eedd84 /python/pyspark/context.py | |
parent | a6786424954218fc31d8cc638a5c7d6567b55047 (diff) | |
download | spark-010c460d627c1917dc47b09e59fd41172bbf90b3.tar.gz spark-010c460d627c1917dc47b09e59fd41172bbf90b3.tar.bz2 spark-010c460d627c1917dc47b09e59fd41172bbf90b3.zip |
[SPARK-2061] Made splits deprecated in JavaRDDLike
The jira for the issue can be found at: https://issues.apache.org/jira/browse/SPARK-2061
Most of spark has used over to consistently using `partitions` instead of `splits`. We should do likewise and add a `partitions` method to JavaRDDLike and have `splits` just call that. We should also go through all cases where other API's (e.g. Python) call `splits` and we should change those to use the newer API.
Author: Anant <anant.asty@gmail.com>
Closes #1062 from anantasty/SPARK-2061 and squashes the following commits:
b83ce6b [Anant] Fixed syntax issue
21f9210 [Anant] Fixed version number in deprecation string
9315b76 [Anant] made related changes to use partitions in python api
8c62dd1 [Anant] Made splits deprecated in JavaRDDLike
Diffstat (limited to 'python/pyspark/context.py')
-rw-r--r-- | python/pyspark/context.py | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/python/pyspark/context.py b/python/pyspark/context.py index 062bec2381..95c54e7a5a 100644 --- a/python/pyspark/context.py +++ b/python/pyspark/context.py @@ -704,7 +704,7 @@ class SparkContext(object): [0, 1, 16, 25] """ if partitions == None: - partitions = range(rdd._jrdd.splits().size()) + partitions = range(rdd._jrdd.partitions().size()) javaPartitions = ListConverter().convert(partitions, self._gateway._gateway_client) # Implementation note: This is implemented as a mapPartitions followed |