diff options
author | Steve Loughran <stevel@apache.org> | 2016-11-02 11:52:29 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-11-02 11:52:29 -0700 |
commit | 37d95227a21de602b939dae84943ba007f434513 (patch) | |
tree | d55490b0d67566c7d178064ad386d74be8aa22be /pom.xml | |
parent | 3c24299b71e23e159edbb972347b13430f92a465 (diff) | |
download | spark-37d95227a21de602b939dae84943ba007f434513.tar.gz spark-37d95227a21de602b939dae84943ba007f434513.tar.bz2 spark-37d95227a21de602b939dae84943ba007f434513.zip |
[SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts
## What changes were proposed in this pull request?
Adds a `snapshots-and-staging profile` so that RCs of projects like Hadoop and HBase can be used in developer-only build and test runs. There's a comment above the profile telling people not to use this in production.
There's no attempt to do the same for SBT, as Ivy is different.
## How was this patch tested?
Tested by building against the Hadoop 2.7.3 RC 1 JARs
without the profile (and without any local copy of the 2.7.3 artifacts), the build failed
```
mvn install -DskipTests -Pyarn,hadoop-2.7,hive -Dhadoop.version=2.7.3
...
[INFO] ------------------------------------------------------------------------
[INFO] Building Spark Project Launcher 2.1.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.pom
[WARNING] The POM for org.apache.hadoop:hadoop-client:jar:2.7.3 is missing, no dependency information available
Downloading: https://repo1.maven.org/maven2/org/apache/hadoop/hadoop-client/2.7.3/hadoop-client-2.7.3.jar
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary:
[INFO]
[INFO] Spark Project Parent POM ........................... SUCCESS [ 4.482 s]
[INFO] Spark Project Tags ................................. SUCCESS [ 17.402 s]
[INFO] Spark Project Sketch ............................... SUCCESS [ 11.252 s]
[INFO] Spark Project Networking ........................... SUCCESS [ 13.458 s]
[INFO] Spark Project Shuffle Streaming Service ............ SUCCESS [ 9.043 s]
[INFO] Spark Project Unsafe ............................... SUCCESS [ 16.027 s]
[INFO] Spark Project Launcher ............................. FAILURE [ 1.653 s]
[INFO] Spark Project Core ................................. SKIPPED
...
```
With the profile, the build completed
```
mvn install -DskipTests -Pyarn,hadoop-2.7,hive,snapshots-and-staging -Dhadoop.version=2.7.3
```
Author: Steve Loughran <stevel@apache.org>
Closes #14646 from steveloughran/stevel/SPARK-17058-support-asf-snapshots.
Diffstat (limited to 'pom.xml')
-rw-r--r-- | pom.xml | 48 |
1 files changed, 48 insertions, 0 deletions
@@ -2694,6 +2694,54 @@ </profile> <!-- + This is a profile to enable the use of the ASF snapshot and staging repositories + during a build. It is useful when testing againt nightly or RC releases of dependencies. + It MUST NOT be used when building copies of Spark to use in production of for distribution, + --> + <profile> + <id>snapshots-and-staging</id> + <properties> + <!-- override point for ASF staging/snapshot repos --> + <asf.staging>https://repository.apache.org/content/groups/staging/</asf.staging> + <asf.snapshots>https://repository.apache.org/content/repositories/snapshots/</asf.snapshots> + </properties> + + <pluginRepositories> + <pluginRepository> + <id>ASF Staging</id> + <url>${asf.staging}</url> + </pluginRepository> + <pluginRepository> + <id>ASF Snapshots</id> + <url>${asf.snapshots}</url> + <snapshots> + <enabled>true</enabled> + </snapshots> + <releases> + <enabled>false</enabled> + </releases> + </pluginRepository> + + </pluginRepositories> + <repositories> + <repository> + <id>ASF Staging</id> + <url>${asf.staging}</url> + </repository> + <repository> + <id>ASF Snapshots</id> + <url>${asf.snapshots}</url> + <snapshots> + <enabled>true</enabled> + </snapshots> + <releases> + <enabled>false</enabled> + </releases> + </repository> + </repositories> + </profile> + + <!-- These empty profiles are available in some sub-modules. Declare them here so that maven does not complain when they're provided on the command line for a sub-module that does not have them. |