diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-03-25 17:40:00 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-03-25 17:40:00 -0700 |
| commit | d44a3362ed8cf3068f8ff233e13851a39da42219 (patch) | |
| tree | 632629fcc93cdc599f4fbc302e0420b08d06146a /sql/hive | |
| parent | 4fc4d0369e8240defe0ee83252426402f1a28a36 (diff) | |
| download | spark-d44a3362ed8cf3068f8ff233e13851a39da42219.tar.gz spark-d44a3362ed8cf3068f8ff233e13851a39da42219.tar.bz2 spark-d44a3362ed8cf3068f8ff233e13851a39da42219.zip | |
[SPARK-6079] Use index to speed up StatusTracker.getJobIdsForGroup()
`StatusTracker.getJobIdsForGroup()` is implemented via a linear scan over a HashMap rather than using an index, which might be an expensive operation if there are many (e.g. thousands) of retained jobs.
This patch adds a new map to `JobProgressListener` in order to speed up these lookups.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #4830 from JoshRosen/statustracker-job-group-indexing and squashes the following commits:
e39c5c7 [Josh Rosen] Address review feedback
6709fb2 [Josh Rosen] Merge remote-tracking branch 'origin/master' into statustracker-job-group-indexing
2c49614 [Josh Rosen] getOrElse
97275a7 [Josh Rosen] Add jobGroup to jobId index to JobProgressListener
Diffstat (limited to 'sql/hive')
0 files changed, 0 insertions, 0 deletions