diff options
| author | Ahir Reddy <ahirreddy@gmail.com> | 2014-04-24 20:21:10 -0700 |
|---|---|---|
| committer | Matei Zaharia <matei@databricks.com> | 2014-04-24 20:21:18 -0700 |
| commit | 7b6d774825578774794b508254c142f9248dd5dd (patch) | |
| tree | b9ca15f01f8c0ed3af53493b906d6a7692f2dffa /bin | |
| parent | 963046cf1bc093a377af39e07bc75afe1b64ade6 (diff) | |
| download | spark-7b6d774825578774794b508254c142f9248dd5dd.tar.gz spark-7b6d774825578774794b508254c142f9248dd5dd.tar.bz2 spark-7b6d774825578774794b508254c142f9248dd5dd.zip | |
[SPARK-986]: Job cancelation for PySpark
* Additions to the PySpark API to cancel jobs
* Monitor Thread in PythonRDD to kill Python workers if a task is interrupted
Author: Ahir Reddy <ahirreddy@gmail.com>
Closes #541 from ahirreddy/python-cancel and squashes the following commits:
dfdf447 [Ahir Reddy] Changed success -> completed and made logging message clearer
6c860ab [Ahir Reddy] PR Comments
4b4100a [Ahir Reddy] Success flag
adba6ed [Ahir Reddy] Destroy python workers
27a2f8f [Ahir Reddy] Start the writer thread...
d422f7b [Ahir Reddy] Remove unnecesssary vals
adda337 [Ahir Reddy] Busy wait on the ocntext.interrupted flag, and then kill the python worker
d9e472f [Ahir Reddy] Revert "removed unnecessary vals"
5b9cae5 [Ahir Reddy] removed unnecessary vals
07b54d9 [Ahir Reddy] Fix canceling unit test
8ae9681 [Ahir Reddy] Don't interrupt worker
7722342 [Ahir Reddy] Monitor Thread for python workers
db04e16 [Ahir Reddy] Added canceling api to PySpark
(cherry picked from commit e53eb4f0159ebd4d72c4bbc3586fdfc66ccacab7)
Signed-off-by: Matei Zaharia <matei@databricks.com>
Diffstat (limited to 'bin')
0 files changed, 0 insertions, 0 deletions