diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-04-29 13:31:52 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-04-29 13:31:52 -0700 |
| commit | 3a180c19a4ef165366186e23d8e8844c5baaecdd (patch) | |
| tree | 6c9b3750cd4369daea5ad5d30e75b5cdf1252d1a /NOTICE | |
| parent | a9c4e29950a14e32acaac547e9a0e8879fd37fc9 (diff) | |
| download | spark-3a180c19a4ef165366186e23d8e8844c5baaecdd.tar.gz spark-3a180c19a4ef165366186e23d8e8844c5baaecdd.tar.bz2 spark-3a180c19a4ef165366186e23d8e8844c5baaecdd.zip | |
[SPARK-6629] cancelJobGroup() may not work for jobs whose job groups are inherited from parent threads
When a job is submitted with a job group and that job group is inherited from a parent thread, there are multiple bugs that may prevent this job from being cancelable via `SparkContext.cancelJobGroup()`:
- When filtering jobs based on their job group properties, DAGScheduler calls `get()` instead of `getProperty()`, which does not respect inheritance, so it will skip over jobs whose job group properties were inherited.
- `Properties` objects are mutable, but we do not make defensive copies / snapshots, so modifications of the parent thread's job group will cause running jobs' groups to change; this also breaks cancelation.
Both of these issues are easy to fix: use `getProperty()` and perform defensive copying.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #5288 from JoshRosen/localProperties-mutability-race and squashes the following commits:
9e29654 [Josh Rosen] Fix style issue
5d90750 [Josh Rosen] Merge remote-tracking branch 'origin/master' into localProperties-mutability-race
3f7b9e8 [Josh Rosen] Add JIRA reference; move clone into DAGScheduler
707e417 [Josh Rosen] Clone local properties to prevent mutations from breaking job cancellation.
b376114 [Josh Rosen] Fix bug that prevented jobs with inherited job group properties from being cancelled.
Diffstat (limited to 'NOTICE')
0 files changed, 0 insertions, 0 deletions