diff options
| author | Richard W. Eggert II <richard.eggert@gmail.com> | 2015-12-15 18:22:58 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-12-15 18:22:58 -0800 |
| commit | 765a488494dac0ed38d2b81742c06467b79d96b2 (patch) | |
| tree | afccd68b4da804f2a84a3995669370e083b772c2 /docs/configuration.md | |
| parent | a63d9edcfb8a714a17492517927aa114dea8fea0 (diff) | |
| download | spark-765a488494dac0ed38d2b81742c06467b79d96b2.tar.gz spark-765a488494dac0ed38d2b81742c06467b79d96b2.tar.bz2 spark-765a488494dac0ed38d2b81742c06467b79d96b2.zip | |
[SPARK-9026][SPARK-4514] Modifications to JobWaiter, FutureAction, and AsyncRDDActions to support non-blocking operation
These changes rework the implementations of `SimpleFutureAction`, `ComplexFutureAction`, `JobWaiter`, and `AsyncRDDActions` such that asynchronous callbacks on the generated `Futures` NEVER block waiting for a job to complete. A small amount of mutex synchronization is necessary to protect the internal fields that manage cancellation, but these locks are only held very briefly and in practice should almost never cause any blocking to occur. The existing blocking APIs of these classes are retained, but they simply delegate to the underlying non-blocking API and `Await` the results with indefinite timeouts.
Associated JIRA ticket: https://issues.apache.org/jira/browse/SPARK-9026
Also fixes: https://issues.apache.org/jira/browse/SPARK-4514
This pull request contains all my own original work, which I release to the Spark project under its open source license.
Author: Richard W. Eggert II <richard.eggert@gmail.com>
Closes #9264 from reggert/fix-futureaction.
Diffstat (limited to 'docs/configuration.md')
0 files changed, 0 insertions, 0 deletions