diff options
| author | Andrew Or <andrew@databricks.com> | 2015-03-02 16:36:42 -0800 |
|---|---|---|
| committer | Patrick Wendell <patrick@databricks.com> | 2015-03-02 16:36:42 -0800 |
| commit | 258d154c9f1afdd52dce19f03d81683ee34effac (patch) | |
| tree | c5a94b159950d22902d90d0c5adcc1a014ad73bf /docs | |
| parent | 6776cb33ea691f7843b956b3e80979282967e826 (diff) | |
| download | spark-258d154c9f1afdd52dce19f03d81683ee34effac.tar.gz spark-258d154c9f1afdd52dce19f03d81683ee34effac.tar.bz2 spark-258d154c9f1afdd52dce19f03d81683ee34effac.zip | |
[SPARK-6048] SparkConf should not translate deprecated configs on set
There are multiple issues with translating on set outlined in the JIRA.
This PR reverts the translation logic added to `SparkConf`. In the future, after the 1.3.0 release we will figure out a way to reorganize the internal structure more elegantly. For now, let's preserve the existing semantics of `SparkConf` since it's a public interface. Unfortunately this means duplicating some code for now, but this is all internal and we can always clean it up later.
Author: Andrew Or <andrew@databricks.com>
Closes #4799 from andrewor14/conf-set-translate and squashes the following commits:
11c525b [Andrew Or] Move warning to driver
10e77b5 [Andrew Or] Add documentation for deprecation precedence
a369cb1 [Andrew Or] Merge branch 'master' of github.com:apache/spark into conf-set-translate
c26a9e3 [Andrew Or] Revert all translate logic in SparkConf
fef6c9c [Andrew Or] Restore deprecation logic for spark.executor.userClassPathFirst
94b4dfa [Andrew Or] Translate on get, not set
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/configuration.md | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index c11787b17e..ae90fe1f8f 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -70,7 +70,9 @@ each line consists of a key and a value separated by whitespace. For example: Any values specified as flags or in the properties file will be passed on to the application and merged with those specified through SparkConf. Properties set directly on the SparkConf take highest precedence, then flags passed to `spark-submit` or `spark-shell`, then options -in the `spark-defaults.conf` file. +in the `spark-defaults.conf` file. A few configuration keys have been renamed since earlier +versions of Spark; in such cases, the older key names are still accepted, but take lower +precedence than any instance of the newer key. ## Viewing Spark Properties |