diff options
author | Josh Rosen <joshrosen@databricks.com> | 2016-04-18 19:36:40 -0700 |
---|---|---|
committer | Tathagata Das <tathagata.das1565@gmail.com> | 2016-04-18 19:36:40 -0700 |
commit | ed2de0299a5a54b566b91ae9f47b6626c484c1d3 (patch) | |
tree | 36858c6e740c9867259856cf8a90c6d381f4b88f /sbin/stop-master.sh | |
parent | 5e92583d38e11d39deb429a39725443111205a4a (diff) | |
download | spark-ed2de0299a5a54b566b91ae9f47b6626c484c1d3.tar.gz spark-ed2de0299a5a54b566b91ae9f47b6626c484c1d3.tar.bz2 spark-ed2de0299a5a54b566b91ae9f47b6626c484c1d3.zip |
[SPARK-14719] WriteAheadLogBasedBlockHandler should ignore BlockManager put errors
WriteAheadLogBasedBlockHandler will currently throw exceptions if its BlockManager `put()` calls fail, even though those calls are only performed as a performance optimization. Instead, it should log and ignore exceptions during that `put()`.
This is a longstanding issue that was masked by an incorrect test case. I think that we haven't noticed this in production because
1. most people probably use a `MEMORY_AND_DISK` storage level, and
2. typically, individual blocks may be small enough relative to the total storage memory such that they're able to evict blocks from previous batches, so `put()` failures here may be rare in practice.
This patch fixes the faulty test and fixes the bug.
/cc tdas
Author: Josh Rosen <joshrosen@databricks.com>
Closes #12484 from JoshRosen/received-block-hadndler-fix.
Diffstat (limited to 'sbin/stop-master.sh')
0 files changed, 0 insertions, 0 deletions