diff options
| author | zsxwing <zsxwing@gmail.com> | 2015-07-09 13:48:29 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2015-07-09 13:48:29 -0700 |
| commit | 1f6b0b1234cc03aa2e07aea7fec2de7563885238 (patch) | |
| tree | c1a8c2d2f750ca5d8db14f6bf1558d58649396e5 /python/pyspark/streaming | |
| parent | c4830598b271cc6390d127bd4cf8ab02b28792e0 (diff) | |
| download | spark-1f6b0b1234cc03aa2e07aea7fec2de7563885238.tar.gz spark-1f6b0b1234cc03aa2e07aea7fec2de7563885238.tar.bz2 spark-1f6b0b1234cc03aa2e07aea7fec2de7563885238.zip | |
[SPARK-8701] [STREAMING] [WEBUI] Add input metadata in the batch page
This PR adds `metadata` to `InputInfo`. `InputDStream` can report its metadata for a batch and it will be shown in the batch page.
For example,
FileInputDStream will display the new files for a batch, and DirectKafkaInputDStream will display its offset ranges.
Author: zsxwing <zsxwing@gmail.com>
Closes #7081 from zsxwing/input-metadata and squashes the following commits:
f7abd9b [zsxwing] Revert the space changes in project/MimaExcludes.scala
d906209 [zsxwing] Merge branch 'master' into input-metadata
74762da [zsxwing] Fix MiMa tests
7903e33 [zsxwing] Merge branch 'master' into input-metadata
450a46c [zsxwing] Address comments
1d94582 [zsxwing] Raname InputInfo to StreamInputInfo and change "metadata" to Map[String, Any]
d496ae9 [zsxwing] Add input metadata in the batch page
Diffstat (limited to 'python/pyspark/streaming')
0 files changed, 0 insertions, 0 deletions