diff options
| author | Bryan Cutler <cutlerb@gmail.com> | 2017-01-11 11:57:38 -0800 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2017-01-11 11:57:38 -0800 |
| commit | 3bc2eff8880a3ba8d4318118715ea1a47048e3de (patch) | |
| tree | ed2191ec24974ab6866fd05fa5e62dcd7a94dfdd /docs/mllib-classification-regression.md | |
| parent | d749c06677c2fd383733337f1c00f542da122b8d (diff) | |
| download | spark-3bc2eff8880a3ba8d4318118715ea1a47048e3de.tar.gz spark-3bc2eff8880a3ba8d4318118715ea1a47048e3de.tar.bz2 spark-3bc2eff8880a3ba8d4318118715ea1a47048e3de.zip | |
[SPARK-17568][CORE][DEPLOY] Add spark-submit option to override ivy settings used to resolve packages/artifacts
## What changes were proposed in this pull request?
Adding option in spark-submit to allow overriding the default IvySettings used to resolve artifacts as part of the Spark Packages functionality. This will allow all artifact resolution to go through a central managed repository, such as Nexus or Artifactory, where site admins can better approve and control what is used with Spark apps.
This change restructures the creation of the IvySettings object in two distinct ways. First, if the `spark.ivy.settings` option is not defined then `buildIvySettings` will create a default settings instance, as before, with defined repositories (Maven Central) included. Second, if the option is defined, the ivy settings file will be loaded from the given path and only repositories defined within will be used for artifact resolution.
## How was this patch tested?
Existing tests for default behaviour, Manual tests that load a ivysettings.xml file with local and Nexus repositories defined. Added new test to load a simple Ivy settings file with a local filesystem resolver.
Author: Bryan Cutler <cutlerb@gmail.com>
Author: Ian Hummel <ian@themodernlife.net>
Closes #15119 from BryanCutler/spark-custom-IvySettings.
Diffstat (limited to 'docs/mllib-classification-regression.md')
0 files changed, 0 insertions, 0 deletions