aboutsummaryrefslogtreecommitdiff
path: root/.gitignore
diff options
context:
space:
mode:
authorJosh Rosen <joshrosen@databricks.com>2016-04-18 19:36:40 -0700
committerTathagata Das <tathagata.das1565@gmail.com>2016-04-18 19:36:40 -0700
commited2de0299a5a54b566b91ae9f47b6626c484c1d3 (patch)
tree36858c6e740c9867259856cf8a90c6d381f4b88f /.gitignore
parent5e92583d38e11d39deb429a39725443111205a4a (diff)
downloadspark-ed2de0299a5a54b566b91ae9f47b6626c484c1d3.tar.gz
spark-ed2de0299a5a54b566b91ae9f47b6626c484c1d3.tar.bz2
spark-ed2de0299a5a54b566b91ae9f47b6626c484c1d3.zip
[SPARK-14719] WriteAheadLogBasedBlockHandler should ignore BlockManager put errors
WriteAheadLogBasedBlockHandler will currently throw exceptions if its BlockManager `put()` calls fail, even though those calls are only performed as a performance optimization. Instead, it should log and ignore exceptions during that `put()`. This is a longstanding issue that was masked by an incorrect test case. I think that we haven't noticed this in production because 1. most people probably use a `MEMORY_AND_DISK` storage level, and 2. typically, individual blocks may be small enough relative to the total storage memory such that they're able to evict blocks from previous batches, so `put()` failures here may be rare in practice. This patch fixes the faulty test and fixes the bug. /cc tdas Author: Josh Rosen <joshrosen@databricks.com> Closes #12484 from JoshRosen/received-block-hadndler-fix.
Diffstat (limited to '.gitignore')
0 files changed, 0 insertions, 0 deletions