diff options
| author | Kay Ousterhout <kayousterhout@gmail.com> | 2016-06-14 17:26:33 -0700 |
|---|---|---|
| committer | Kay Ousterhout <kayousterhout@gmail.com> | 2016-06-14 17:27:01 -0700 |
| commit | 5d50d4f0f9db3e6cc7c51e35cdb2d12daa4fd108 (patch) | |
| tree | 346f79d8713d1922a979e5d9decf857b94ffcf9f /python/pyspark/sql/context.py | |
| parent | dae4d5db21368faaa46fa8d1a256c27428694c2c (diff) | |
| download | spark-5d50d4f0f9db3e6cc7c51e35cdb2d12daa4fd108.tar.gz spark-5d50d4f0f9db3e6cc7c51e35cdb2d12daa4fd108.tar.bz2 spark-5d50d4f0f9db3e6cc7c51e35cdb2d12daa4fd108.zip | |
[SPARK-15927] Eliminate redundant DAGScheduler code.
To try to eliminate redundant code to traverse the RDD dependency graph,
this PR creates a new function getShuffleDependencies that returns
shuffle dependencies that are immediate parents of a given RDD. This
new function is used by getParentStages and
getAncestorShuffleDependencies.
Author: Kay Ousterhout <kayousterhout@gmail.com>
Closes #13646 from kayousterhout/SPARK-15927.
Diffstat (limited to 'python/pyspark/sql/context.py')
0 files changed, 0 insertions, 0 deletions