diff options
author | jerryshao <sshao@hortonworks.com> | 2015-11-25 11:42:53 -0800 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-11-25 11:42:53 -0800 |
commit | 88875d9413ec7d64a88d40857ffcf97b5853a7f2 (patch) | |
tree | 80b00ce434f823ae9209198e8bb8840a6110791e /python/pyspark/streaming | |
parent | 9f3e59a16822fb61d60cf103bd4f7823552939c6 (diff) | |
download | spark-88875d9413ec7d64a88d40857ffcf97b5853a7f2.tar.gz spark-88875d9413ec7d64a88d40857ffcf97b5853a7f2.tar.bz2 spark-88875d9413ec7d64a88d40857ffcf97b5853a7f2.zip |
[SPARK-10558][CORE] Fix wrong executor state in Master
`ExecutorAdded` can only be sent to `AppClient` when worker report back the executor state as `LOADING`, otherwise because of concurrency issue, `AppClient` will possibly receive `ExectuorAdded` at first, then `ExecutorStateUpdated` with `LOADING` state.
Also Master will change the executor state from `LAUNCHING` to `RUNNING` (`AppClient` report back the state as `RUNNING`), then to `LOADING` (worker report back to state as `LOADING`), it should be `LAUNCHING` -> `LOADING` -> `RUNNING`.
Also it is wrongly shown in master UI, the state of executor should be `RUNNING` rather than `LOADING`:
![screen shot 2015-09-11 at 2 30 28 pm](https://cloud.githubusercontent.com/assets/850797/9809254/3155d840-5899-11e5-8cdf-ad06fef75762.png)
Author: jerryshao <sshao@hortonworks.com>
Closes #8714 from jerryshao/SPARK-10558.
Diffstat (limited to 'python/pyspark/streaming')
0 files changed, 0 insertions, 0 deletions