diff options
author | Josh Rosen <joshrosen@databricks.com> | 2016-09-12 13:09:33 -0700 |
---|---|---|
committer | Josh Rosen <joshrosen@databricks.com> | 2016-09-12 13:09:33 -0700 |
commit | 3d40896f410590c0be044b3fa7e5d32115fac05e (patch) | |
tree | 726ea096c4af3311b7333647cabe04b52c91f975 /project/SparkBuild.scala | |
parent | 1742c3ab86d75ce3d352f7cddff65e62fb7c8dd4 (diff) | |
download | spark-3d40896f410590c0be044b3fa7e5d32115fac05e.tar.gz spark-3d40896f410590c0be044b3fa7e5d32115fac05e.tar.bz2 spark-3d40896f410590c0be044b3fa7e5d32115fac05e.zip |
[SPARK-17483] Refactoring in BlockManager status reporting and block removal
This patch makes three minor refactorings to the BlockManager:
- Move the `if (info.tellMaster)` check out of `reportBlockStatus`; this fixes an issue where a debug logging message would incorrectly claim to have reported a block status to the master even though no message had been sent (in case `info.tellMaster == false`). This also makes it easier to write code which unconditionally sends block statuses to the master (which is necessary in another patch of mine).
- Split `removeBlock()` into two methods, the existing method and an internal `removeBlockInternal()` method which is designed to be called by internal code that already holds a write lock on the block. This is also needed by a followup patch.
- Instead of calling `getCurrentBlockStatus()` in `removeBlock()`, just pass `BlockStatus.empty`; the block status should always be empty following complete removal of a block.
These changes were originally authored as part of a bug fix patch which is targeted at branch-2.0 and master; I've split them out here into their own separate PR in order to make them easier to review and so that the behavior-changing parts of my other patch can be isolated to their own PR.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #15036 from JoshRosen/cache-failure-race-conditions-refactorings-only.
Diffstat (limited to 'project/SparkBuild.scala')
0 files changed, 0 insertions, 0 deletions