Constructor and Description |
---|
BlockManagerMaster(akka.actor.ActorRef driverActor,
SparkConf conf,
boolean isDriver) |
Modifier and Type | Method and Description |
---|---|
boolean |
contains(BlockId blockId)
Check if block manager master has a block.
|
String |
DRIVER_AKKA_ACTOR_NAME() |
akka.actor.ActorRef |
driverActor() |
scala.Option<scala.Tuple2<String,Object>> |
getActorSystemHostPortForExecutor(String executorId) |
scala.collection.immutable.Map<BlockManagerId,BlockStatus> |
getBlockStatus(BlockId blockId,
boolean askSlaves)
Return the block's status on all block managers, if any.
|
scala.collection.Seq<BlockManagerId> |
getLocations(BlockId blockId)
Get locations of the blockId from the driver
|
scala.collection.Seq<scala.collection.Seq<BlockManagerId>> |
getLocations(BlockId[] blockIds)
Get locations of multiple blockIds from the driver
|
scala.collection.Seq<BlockId> |
getMatchingBlockIds(scala.Function1<BlockId,Object> filter,
boolean askSlaves)
Return a list of ids of existing blocks such that the ids match the given filter.
|
scala.collection.immutable.Map<BlockManagerId,scala.Tuple2<Object,Object>> |
getMemoryStatus()
Return the memory status for each block manager, in the form of a map from
the block manager's id to two long values.
|
scala.collection.Seq<BlockManagerId> |
getPeers(BlockManagerId blockManagerId)
Get ids of other nodes in the cluster from the driver
|
StorageStatus[] |
getStorageStatus() |
void |
registerBlockManager(BlockManagerId blockManagerId,
long maxMemSize,
akka.actor.ActorRef slaveActor)
Register the BlockManager's id with the driver.
|
void |
removeBlock(BlockId blockId)
Remove a block from the slaves that have it.
|
void |
removeBroadcast(long broadcastId,
boolean removeFromMaster,
boolean blocking)
Remove all blocks belonging to the given broadcast.
|
void |
removeExecutor(String execId)
Remove a dead executor from the driver actor.
|
void |
removeRdd(int rddId,
boolean blocking)
Remove all blocks belonging to the given RDD.
|
void |
removeShuffle(int shuffleId,
boolean blocking)
Remove all blocks belonging to the given shuffle.
|
void |
stop()
Stop the driver actor, called only on the Spark driver node
|
scala.concurrent.duration.FiniteDuration |
timeout() |
boolean |
updateBlockInfo(BlockManagerId blockManagerId,
BlockId blockId,
StorageLevel storageLevel,
long memSize,
long diskSize,
long tachyonSize) |
equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait
initializeIfNecessary, initializeLogging, isTraceEnabled, log_, log, logDebug, logDebug, logError, logError, logInfo, logInfo, logName, logTrace, logTrace, logWarning, logWarning
public BlockManagerMaster(akka.actor.ActorRef driverActor, SparkConf conf, boolean isDriver)
public akka.actor.ActorRef driverActor()
public String DRIVER_AKKA_ACTOR_NAME()
public scala.concurrent.duration.FiniteDuration timeout()
public void removeExecutor(String execId)
public void registerBlockManager(BlockManagerId blockManagerId, long maxMemSize, akka.actor.ActorRef slaveActor)
public boolean updateBlockInfo(BlockManagerId blockManagerId, BlockId blockId, StorageLevel storageLevel, long memSize, long diskSize, long tachyonSize)
public scala.collection.Seq<BlockManagerId> getLocations(BlockId blockId)
public scala.collection.Seq<scala.collection.Seq<BlockManagerId>> getLocations(BlockId[] blockIds)
public boolean contains(BlockId blockId)
public scala.collection.Seq<BlockManagerId> getPeers(BlockManagerId blockManagerId)
public scala.Option<scala.Tuple2<String,Object>> getActorSystemHostPortForExecutor(String executorId)
public void removeBlock(BlockId blockId)
public void removeRdd(int rddId, boolean blocking)
public void removeShuffle(int shuffleId, boolean blocking)
public void removeBroadcast(long broadcastId, boolean removeFromMaster, boolean blocking)
public scala.collection.immutable.Map<BlockManagerId,scala.Tuple2<Object,Object>> getMemoryStatus()
public StorageStatus[] getStorageStatus()
public scala.collection.immutable.Map<BlockManagerId,BlockStatus> getBlockStatus(BlockId blockId, boolean askSlaves)
If askSlaves is true, this invokes the master to query each block manager for the most updated block statuses. This is useful when the master is not informed of the given block by all block managers.
public scala.collection.Seq<BlockId> getMatchingBlockIds(scala.Function1<BlockId,Object> filter, boolean askSlaves)
If askSlaves is true, this invokes the master to query each block manager for the most updated block statuses. This is useful when the master is not informed of the given block by all block managers.
public void stop()