diff options
| author | Vladimir Vladimirov <vladimir.vladimirov@magnetic.com> | 2015-06-29 12:03:41 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2015-06-29 12:03:41 -0700 |
| commit | 492dca3a73e70705b5d5639e8fe4640b80e78d31 (patch) | |
| tree | 2bb564110a0188be74d200defcc69a1dfae8f74d /core | |
| parent | a5c2961caaafd751f11bdd406bb6885443d7572e (diff) | |
| download | spark-492dca3a73e70705b5d5639e8fe4640b80e78d31.tar.gz spark-492dca3a73e70705b5d5639e8fe4640b80e78d31.tar.bz2 spark-492dca3a73e70705b5d5639e8fe4640b80e78d31.zip | |
[SPARK-8528] Expose SparkContext.applicationId in PySpark
Use case - we want to log applicationId (YARN in hour case) to request help with troubleshooting from the DevOps
Author: Vladimir Vladimirov <vladimir.vladimirov@magnetic.com>
Closes #6936 from smartkiwi/master and squashes the following commits:
870338b [Vladimir Vladimirov] this would make doctest to run in python3
0eae619 [Vladimir Vladimirov] Scala doesn't use u'...' for unicode literals
14d77a8 [Vladimir Vladimirov] stop using ELLIPSIS
b4ebfc5 [Vladimir Vladimirov] addressed PR feedback - updated docstring
223a32f [Vladimir Vladimirov] fixed test - applicationId is property that returns the string
3221f5a [Vladimir Vladimirov] [SPARK-8528] added documentation for Scala
2cff090 [Vladimir Vladimirov] [SPARK-8528] add applicationId property for SparkContext object in pyspark
Diffstat (limited to 'core')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/SparkContext.scala | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/core/src/main/scala/org/apache/spark/SparkContext.scala b/core/src/main/scala/org/apache/spark/SparkContext.scala index c7a7436462..b3c3bf3746 100644 --- a/core/src/main/scala/org/apache/spark/SparkContext.scala +++ b/core/src/main/scala/org/apache/spark/SparkContext.scala @@ -315,6 +315,14 @@ class SparkContext(config: SparkConf) extends Logging with ExecutorAllocationCli _dagScheduler = ds } + /** + * A unique identifier for the Spark application. + * Its format depends on the scheduler implementation. + * (i.e. + * in case of local spark app something like 'local-1433865536131' + * in case of YARN something like 'application_1433865536131_34483' + * ) + */ def applicationId: String = _applicationId def applicationAttemptId: Option[String] = _applicationAttemptId |