diff options
| author | Cheng Lian <lian@databricks.com> | 2015-08-19 14:15:28 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-08-19 14:26:11 -0700 |
| commit | b32a31d64588f2fd3926f09482e2bfe968370fc0 (patch) | |
| tree | e37401a69af259b17a0182bffb4a64cf4602ae90 /network | |
| parent | d9dfd43d463cd5e3c9e72197850382ad8897b8ac (diff) | |
| download | spark-b32a31d64588f2fd3926f09482e2bfe968370fc0.tar.gz spark-b32a31d64588f2fd3926f09482e2bfe968370fc0.tar.bz2 spark-b32a31d64588f2fd3926f09482e2bfe968370fc0.zip | |
[SPARK-9899] [SQL] Disables customized output committer when speculation is on
Speculation hates direct output committer, as there are multiple corner cases that may cause data corruption and/or data loss.
Please see this [PR comment] [1] for more details.
[1]: https://github.com/apache/spark/pull/8191#issuecomment-131598385
Author: Cheng Lian <lian@databricks.com>
Closes #8317 from liancheng/spark-9899/speculation-hates-direct-output-committer.
(cherry picked from commit f3ff4c41d2e32bd0f2419d1c9c68fcd0c2593e41)
Signed-off-by: Michael Armbrust <michael@databricks.com>
Conflicts:
sql/hive/src/test/scala/org/apache/spark/sql/sources/hadoopFsRelationSuites.scala
Diffstat (limited to 'network')
0 files changed, 0 insertions, 0 deletions