diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-02-01 17:47:51 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-02-01 17:48:41 -0800 |
| commit | 883bc88d520b27bdeb74a1837b45ef0b59753568 (patch) | |
| tree | 76e1e33a5373af8b3636385e6f555fced822d2dc /repl | |
| parent | 4a171225ba628192a5ae43a99dc50508cf12491c (diff) | |
| download | spark-883bc88d520b27bdeb74a1837b45ef0b59753568.tar.gz spark-883bc88d520b27bdeb74a1837b45ef0b59753568.tar.bz2 spark-883bc88d520b27bdeb74a1837b45ef0b59753568.zip | |
[SPARK-4859][Core][Streaming] Refactor LiveListenerBus and StreamingListenerBus
This PR refactors LiveListenerBus and StreamingListenerBus and extracts the common codes to a parent class `ListenerBus`.
It also includes bug fixes in #3710:
1. Fix the race condition of queueFullErrorMessageLogged in LiveListenerBus and StreamingListenerBus to avoid outputing `queue-full-error` logs multiple times.
2. Make sure the SHUTDOWN message will be delivered to listenerThread, so that we can make sure listenerThread will always be able to exit.
3. Log the error from listener rather than crashing listenerThread in StreamingListenerBus.
During fixing the above bugs, we find it's better to make LiveListenerBus and StreamingListenerBus have the same bahaviors. Then there will be many duplicated codes in LiveListenerBus and StreamingListenerBus.
Therefore, I extracted their common codes to `ListenerBus` as a parent class: LiveListenerBus and StreamingListenerBus only need to extend `ListenerBus` and implement `onPostEvent` (how to process an event) and `onDropEvent` (do something when droppping an event).
Author: zsxwing <zsxwing@gmail.com>
Closes #4006 from zsxwing/SPARK-4859-refactor and squashes the following commits:
c8dade2 [zsxwing] Fix the code style after renaming
5715061 [zsxwing] Rename ListenerHelper to ListenerBus and the original ListenerBus to AsynchronousListenerBus
f0ef647 [zsxwing] Fix the code style
4e85ffc [zsxwing] Merge branch 'master' into SPARK-4859-refactor
d2ef990 [zsxwing] Add private[spark]
4539f91 [zsxwing] Remove final to pass MiMa tests
a9dccd3 [zsxwing] Remove SparkListenerShutdown
7cc04c3 [zsxwing] Refactor LiveListenerBus and StreamingListenerBus and make them share same code base
Diffstat (limited to 'repl')
0 files changed, 0 insertions, 0 deletions