diff options
| author | Marcelo Vanzin <vanzin@cloudera.com> | 2016-12-12 16:27:04 -0800 |
|---|---|---|
| committer | Shixiong Zhu <shixiong@databricks.com> | 2016-12-12 16:27:04 -0800 |
| commit | bc59951babbe4d7d5265a5dbccd50ea84ad74592 (patch) | |
| tree | 83e9864e9349eca833b4afae780ad4c77d37f0f7 /docs/configuration.md | |
| parent | 8a51cfdcad5f8397558ed2e245eb03650f37ce66 (diff) | |
| download | spark-bc59951babbe4d7d5265a5dbccd50ea84ad74592.tar.gz spark-bc59951babbe4d7d5265a5dbccd50ea84ad74592.tar.bz2 spark-bc59951babbe4d7d5265a5dbccd50ea84ad74592.zip | |
[SPARK-18773][CORE] Make commons-crypto config translation consistent.
This change moves the logic that translates Spark configuration to
commons-crypto configuration to the network-common module. It also
extends TransportConf and ConfigProvider to provide the necessary
interfaces for the translation to work.
As part of the change, I removed SystemPropertyConfigProvider, which
was mostly used as an "empty config" in unit tests, and adjusted the
very few tests that required a specific config.
I also changed the config keys for AES encryption to live under the
"spark.network." namespace, which is more correct than their previous
names under "spark.authenticate.".
Tested via existing unit test.
Author: Marcelo Vanzin <vanzin@cloudera.com>
Closes #16200 from vanzin/SPARK-18773.
Diffstat (limited to 'docs/configuration.md')
| -rw-r--r-- | docs/configuration.md | 21 |
1 files changed, 10 insertions, 11 deletions
diff --git a/docs/configuration.md b/docs/configuration.md index b1e731182f..7e466d7dc1 100644 --- a/docs/configuration.md +++ b/docs/configuration.md @@ -1558,14 +1558,15 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> - <td><code>spark.authenticate.encryption.aes.enabled</code></td> + <td><code>spark.network.aes.enabled</code></td> <td>false</td> <td> - Enable AES for over-the-wire encryption + Enable AES for over-the-wire encryption. This is supported for RPC and the block transfer service. + This option has precedence over SASL-based encryption if both are enabled. </td> </tr> <tr> - <td><code>spark.authenticate.encryption.aes.cipher.keySize</code></td> + <td><code>spark.network.aes.keySize</code></td> <td>16</td> <td> The bytes of AES cipher key which is effective when AES cipher is enabled. AES @@ -1573,14 +1574,12 @@ Apart from these, the following properties are also available, and may be useful </td> </tr> <tr> - <td><code>spark.authenticate.encryption.aes.cipher.class</code></td> - <td>null</td> + <td><code>spark.network.aes.config.*</code></td> + <td>None</td> <td> - Specify the underlying implementation class of crypto cipher. Set null here to use default. - In order to use OpenSslCipher users should install openssl. Currently, there are two cipher - classes available in Commons Crypto library: - org.apache.commons.crypto.cipher.OpenSslCipher - org.apache.commons.crypto.cipher.JceCipher + Configuration values for the commons-crypto library, such as which cipher implementations to + use. The config name should be the name of commons-crypto configuration without the + "commons.crypto" prefix. </td> </tr> <tr> @@ -1658,7 +1657,7 @@ Apart from these, the following properties are also available, and may be useful </tr> </table> -#### Encryption +#### TLS / SSL <table class="table"> <tr><th>Property Name</th><th>Default</th><th>Meaning</th></tr> |