diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-07-09 13:48:29 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-07-09 13:48:29 -0700 |
| commit | 1f6b0b1234cc03aa2e07aea7fec2de7563885238 (patch) | |
| tree | c1a8c2d2f750ca5d8db14f6bf1558d58649396e5 /project | |
| parent | c4830598b271cc6390d127bd4cf8ab02b28792e0 (diff) | |
| download | spark-1f6b0b1234cc03aa2e07aea7fec2de7563885238.tar.gz spark-1f6b0b1234cc03aa2e07aea7fec2de7563885238.tar.bz2 spark-1f6b0b1234cc03aa2e07aea7fec2de7563885238.zip | |
[SPARK-8701] [STREAMING] [WEBUI] Add input metadata in the batch page
This PR adds `metadata` to `InputInfo`. `InputDStream` can report its metadata for a batch and it will be shown in the batch page.
For example,
FileInputDStream will display the new files for a batch, and DirectKafkaInputDStream will display its offset ranges.
Author: zsxwing <zsxwing@gmail.com>
Closes #7081 from zsxwing/input-metadata and squashes the following commits:
f7abd9b [zsxwing] Revert the space changes in project/MimaExcludes.scala
d906209 [zsxwing] Merge branch 'master' into input-metadata
74762da [zsxwing] Fix MiMa tests
7903e33 [zsxwing] Merge branch 'master' into input-metadata
450a46c [zsxwing] Address comments
1d94582 [zsxwing] Raname InputInfo to StreamInputInfo and change "metadata" to Map[String, Any]
d496ae9 [zsxwing] Add input metadata in the batch page
Diffstat (limited to 'project')
| -rw-r--r-- | project/MimaExcludes.scala | 6 |
1 files changed, 6 insertions, 0 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 821aadd477..79089aae2a 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -77,6 +77,12 @@ object MimaExcludes { // SPARK-8914 Remove RDDApi ProblemFilters.exclude[MissingClassProblem]( "org.apache.spark.sql.RDDApi") + ) ++ Seq( + // SPARK-8701 Add input metadata in the batch page. + ProblemFilters.exclude[MissingClassProblem]( + "org.apache.spark.streaming.scheduler.InputInfo$"), + ProblemFilters.exclude[MissingClassProblem]( + "org.apache.spark.streaming.scheduler.InputInfo") ) case v if v.startsWith("1.4") => |