diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-04-06 23:33:16 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2015-04-06 23:33:16 -0700 |
| commit | a0846c4b635eac8d8637c83d490177f881952d27 (patch) | |
| tree | a8290626a78702e93488fd1c94a22989054b18af /graphx/src | |
| parent | e40ea8742a8771ecd46b182f45b5fcd8bd6dd725 (diff) | |
| download | spark-a0846c4b635eac8d8637c83d490177f881952d27.tar.gz spark-a0846c4b635eac8d8637c83d490177f881952d27.tar.bz2 spark-a0846c4b635eac8d8637c83d490177f881952d27.zip | |
[SPARK-6716] Change SparkContext.DRIVER_IDENTIFIER from <driver> to driver
Currently, the driver's executorId is set to `<driver>`. This choice of ID was present in older Spark versions, but it has started to cause problems now that executorIds are used in more contexts, such as Ganglia metric names or driver thread-dump links the web UI. The angle brackets must be escaped when embedding this ID in XML or as part of URLs and this has led to multiple problems:
- https://issues.apache.org/jira/browse/SPARK-6484
- https://issues.apache.org/jira/browse/SPARK-4313
The simplest solution seems to be to change this id to something that does not contain any special characters, such as `driver`.
I'm not sure whether we can perform this change in a patch release, since this ID may be considered a stable API by metrics users, but it's probably okay to do this in a major release as long as we document it in the release notes.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #5372 from JoshRosen/driver-id-fix and squashes the following commits:
42d3c10 [Josh Rosen] Clarify comment
0c5d04b [Josh Rosen] Add backwards-compatibility in BlockManagerId.isDriver
7ff12e0 [Josh Rosen] Change SparkContext.DRIVER_IDENTIFIER from <driver> to driver
Diffstat (limited to 'graphx/src')
0 files changed, 0 insertions, 0 deletions