From 640f942337e1ce87075195998bd051e19c4b50b9 Mon Sep 17 00:00:00 2001 From: José Hiram Soltren Date: Thu, 19 Jan 2017 09:08:18 -0600 Subject: [SPARK-16654][CORE] Add UI coverage for Application Level Blacklisting MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Builds on top of work in SPARK-8425 to update Application Level Blacklisting in the scheduler. ## What changes were proposed in this pull request? Adds a UI to these patches by: - defining new listener events for blacklisting and unblacklisting, nodes and executors; - sending said events at the relevant points in BlacklistTracker; - adding JSON (de)serialization code for these events; - augmenting the Executors UI page to show which, and how many, executors are blacklisted; - adding a unit test to make sure events are being fired; - adding HistoryServerSuite coverage to verify that the SHS reads these events correctly. - updates the Executor UI to show Blacklisted/Active/Dead as a tri-state in Executors Status Updates .rat-excludes to pass tests. username squito ## How was this patch tested? ./dev/run-tests testOnly org.apache.spark.util.JsonProtocolSuite testOnly org.apache.spark.scheduler.BlacklistTrackerSuite testOnly org.apache.spark.deploy.history.HistoryServerSuite https://github.com/jsoltren/jose-utils/blob/master/blacklist/test-blacklist.sh ![blacklist-20161219](https://cloud.githubusercontent.com/assets/1208477/21335321/9eda320a-c623-11e6-8b8c-9c912a73c276.jpg) Author: José Hiram Soltren Closes #16346 from jsoltren/SPARK-16654-submit. --- dev/.rat-excludes | 2 ++ 1 file changed, 2 insertions(+) (limited to 'dev/.rat-excludes') diff --git a/dev/.rat-excludes b/dev/.rat-excludes index 6be1c72bc6..6d24434ccc 100644 --- a/dev/.rat-excludes +++ b/dev/.rat-excludes @@ -73,6 +73,8 @@ logs .*dependency-reduced-pom.xml known_translations json_expectation +app-20161115172038-0000 +app-20161116163331-0000 local-1422981759269 local-1422981780767 local-1425081759269 -- cgit v1.2.3