From 6f27027d96ada29d8bb1d626f2cc7c856df3d597 Mon Sep 17 00:00:00 2001 From: Eric Liang Date: Mon, 11 Apr 2016 18:33:54 -0700 Subject: [SPARK-14475] Propagate user-defined context from driver to executors ## What changes were proposed in this pull request? This adds a new API call `TaskContext.getLocalProperty` for getting properties set in the driver from executors. These local properties are automatically propagated from the driver to executors. For streaming, the context for streaming tasks will be the initial driver context when ssc.start() is called. ## How was this patch tested? Unit tests. cc JoshRosen Author: Eric Liang Closes #12248 from ericl/sc-2813. --- project/MimaExcludes.scala | 3 +++ 1 file changed, 3 insertions(+) (limited to 'project') diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index 290de794dc..a30581eb48 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -625,6 +625,9 @@ object MimaExcludes { ) ++ Seq( // [SPARK-13048][ML][MLLIB] keepLastCheckpoint option for LDA EM optimizer ProblemFilters.exclude[DirectMissingMethodProblem]("org.apache.spark.mllib.clustering.DistributedLDAModel.this") + ) ++ Seq( + // [SPARK-14475] Propagate user-defined context from driver to executors + ProblemFilters.exclude[ReversedMissingMethodProblem]("org.apache.spark.TaskContext.getLocalProperty") ) case v if v.startsWith("1.6") => Seq( -- cgit v1.2.3