diff options
| author | Liang-Chi Hsieh <viirya@gmail.com> | 2014-12-23 14:58:33 -0800 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2014-12-23 14:58:44 -0800 |
| commit | 96281cd0c3ffb4c0fbdb64b8f7b705f863d46b33 (patch) | |
| tree | fdbed618295222cb22891c42d895047247b9e881 /streaming/src | |
| parent | 27c5399f4dd542e36ea579956b8cb0613de25c6d (diff) | |
| download | spark-96281cd0c3ffb4c0fbdb64b8f7b705f863d46b33.tar.gz spark-96281cd0c3ffb4c0fbdb64b8f7b705f863d46b33.tar.bz2 spark-96281cd0c3ffb4c0fbdb64b8f7b705f863d46b33.zip | |
[SPARK-4913] Fix incorrect event log path
SPARK-2261 uses a single file to log events for an app. `eventLogDir` in `ApplicationDescription` is replaced with `eventLogFile`. However, `ApplicationDescription` in `SparkDeploySchedulerBackend` is initialized with `SparkContext`'s `eventLogDir`. It is just the log directory, not the actual log file path. `Master.rebuildSparkUI` can not correctly rebuild a new SparkUI for the app.
Because the `ApplicationDescription` is remotely registered with `Master` and the app's id is then generated in `Master`, we can not get the app id in advance before registration. So the received description needs to be modified with correct `eventLogFile` value.
Author: Liang-Chi Hsieh <viirya@gmail.com>
Closes #3755 from viirya/fix_app_logdir and squashes the following commits:
5e0ea35 [Liang-Chi Hsieh] Revision for comment.
b5730a1 [Liang-Chi Hsieh] Fix incorrect event log path.
Closes #3777 (a duplicate PR for the same JIRA)
Diffstat (limited to 'streaming/src')
0 files changed, 0 insertions, 0 deletions