diff options
| author | Davies Liu <davies.liu@gmail.com> | 2014-08-13 14:57:12 -0700 |
|---|---|---|
| committer | Matei Zaharia <matei@databricks.com> | 2014-08-13 14:57:12 -0700 |
| commit | 434bea1c002b597cff9db899da101490e1f1e9ed (patch) | |
| tree | 4ddf5a151343d3c4012ff248af6efe3ef295c04d /tox.ini | |
| parent | c974a716e17c9fe2628b1ba1d4309ead1bd855ad (diff) | |
| download | spark-434bea1c002b597cff9db899da101490e1f1e9ed.tar.gz spark-434bea1c002b597cff9db899da101490e1f1e9ed.tar.bz2 spark-434bea1c002b597cff9db899da101490e1f1e9ed.zip | |
[SPARK-2983] [PySpark] improve performance of sortByKey()
1. skip partitionBy() when numOfPartition is 1
2. use bisect_left (O(lg(N))) instread of loop (O(N)) in
rangePartitioner
Author: Davies Liu <davies.liu@gmail.com>
Closes #1898 from davies/sort and squashes the following commits:
0a9608b [Davies Liu] Merge branch 'master' into sort
1cf9565 [Davies Liu] improve performance of sortByKey()
Diffstat (limited to 'tox.ini')
0 files changed, 0 insertions, 0 deletions