diff options
author | Josh Rosen <joshrosen@databricks.com> | 2014-11-14 23:46:25 -0800 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2014-11-14 23:46:25 -0800 |
commit | 40eb8b6ef3a67e36d0d9492c044981a1da76351d (patch) | |
tree | 9138527b5d90b13ad2195e7d34661db5f2c9d2d5 /pom.xml | |
parent | cbddac23696d89b672dce380cc7360a873e27b3b (diff) | |
download | spark-40eb8b6ef3a67e36d0d9492c044981a1da76351d.tar.gz spark-40eb8b6ef3a67e36d0d9492c044981a1da76351d.tar.bz2 spark-40eb8b6ef3a67e36d0d9492c044981a1da76351d.zip |
[SPARK-2321] Several progress API improvements / refactorings
This PR refactors / extends the status API introduced in #2696.
- Change StatusAPI from a mixin trait to a class. Before, the new status API methods were directly accessible through SparkContext, whereas now they're accessed through a `sc.statusAPI` field. As long as we were going to add these methods directly to SparkContext, the mixin trait seemed like a good idea, but this might be simpler to reason about and may avoid pitfalls that I've run into while attempting to refactor other parts of SparkContext to use mixins (see #3071, for example).
- Change the name from SparkStatusAPI to SparkStatusTracker.
- Make `getJobIdsForGroup(null)` return ids for jobs that aren't associated with any job group.
- Add `getActiveStageIds()` and `getActiveJobIds()` methods that return the ids of whatever's currently active in this SparkContext. This should simplify davies's progress bar code.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #3197 from JoshRosen/progress-api-improvements and squashes the following commits:
30b0afa [Josh Rosen] Rename SparkStatusAPI to SparkStatusTracker.
d1b08d8 [Josh Rosen] Add missing newlines
2cc7353 [Josh Rosen] Add missing file.
d5eab1f [Josh Rosen] Add getActive[Stage|Job]Ids() methods.
a227984 [Josh Rosen] getJobIdsForGroup(null) should return jobs for default group
c47e294 [Josh Rosen] Remove StatusAPI mixin trait.
Diffstat (limited to 'pom.xml')
0 files changed, 0 insertions, 0 deletions