diff options
| author | Reynold Xin <rxin@apache.org> | 2014-09-12 21:55:39 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@apache.org> | 2014-09-12 21:55:39 -0700 |
| commit | 2584ea5b23b1c5a4df9549b94bfc9b8e0900532e (patch) | |
| tree | 915dda0c0c1376178113b614094d874c1749e400 /docs/sql-programming-guide.md | |
| parent | 6d887db7891be643f0131b136e82191b5f6eb407 (diff) | |
| download | spark-2584ea5b23b1c5a4df9549b94bfc9b8e0900532e.tar.gz spark-2584ea5b23b1c5a4df9549b94bfc9b8e0900532e.tar.bz2 spark-2584ea5b23b1c5a4df9549b94bfc9b8e0900532e.zip | |
[SPARK-3469] Make sure all TaskCompletionListener are called even with failures
This is necessary because we rely on this callback interface to clean resources up. The old behavior would lead to resource leaks.
Note that this also changes the fault semantics of TaskCompletionListener. Previously failures in TaskCompletionListeners would result in the task being reported immediately. With this change, we report the exception at the end, and the reported exception is a TaskCompletionListenerException that contains all the exception messages.
Author: Reynold Xin <rxin@apache.org>
Closes #2343 from rxin/taskcontext-callback and squashes the following commits:
a3845b2 [Reynold Xin] Mark TaskCompletionListenerException as private[spark].
ac5baea [Reynold Xin] Removed obsolete comment.
aa68ea4 [Reynold Xin] Throw an exception if task completion callback fails.
29b6162 [Reynold Xin] oops compilation failed.
1cb444d [Reynold Xin] [SPARK-3469] Call all TaskCompletionListeners even if some fail.
Diffstat (limited to 'docs/sql-programming-guide.md')
0 files changed, 0 insertions, 0 deletions