diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2016-01-03 22:05:02 -0800 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2016-01-03 22:05:02 -0800 |
| commit | 0d165ec2050aa06aa545d74c8d7c2ff197fa02de (patch) | |
| tree | 5819b076d2339b64d51ef44168f5c9b87f1da44b /dev/run-tests.py | |
| parent | 84f8492c1555bf8ab44c9818752278f61768eb16 (diff) | |
| download | spark-0d165ec2050aa06aa545d74c8d7c2ff197fa02de.tar.gz spark-0d165ec2050aa06aa545d74c8d7c2ff197fa02de.tar.bz2 spark-0d165ec2050aa06aa545d74c8d7c2ff197fa02de.zip | |
[SPARK-12612][PROJECT-INFRA] Add missing Hadoop profiles to dev/run-tests-*.py scripts and dev/deps
There are a couple of places in the `dev/run-tests-*.py` scripts which deal with Hadoop profiles, but the set of profiles that they handle does not include all Hadoop profiles defined in our POM. Similarly, the `hadoop-2.2` and `hadoop-2.6` profiles were missing from `dev/deps`.
This patch updates these scripts to include all four Hadoop profiles defined in our POM.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #10565 from JoshRosen/add-missing-hadoop-profiles-in-test-scripts.
Diffstat (limited to 'dev/run-tests.py')
| -rwxr-xr-x | dev/run-tests.py | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/dev/run-tests.py b/dev/run-tests.py index 23278d298c..9db728d799 100755 --- a/dev/run-tests.py +++ b/dev/run-tests.py @@ -295,13 +295,14 @@ def exec_sbt(sbt_args=()): def get_hadoop_profiles(hadoop_version): """ - For the given Hadoop version tag, return a list of SBT profile flags for + For the given Hadoop version tag, return a list of Maven/SBT profile flags for building and testing against that Hadoop version. """ sbt_maven_hadoop_profiles = { "hadoop2.2": ["-Pyarn", "-Phadoop-2.2"], - "hadoop2.3": ["-Pyarn", "-Phadoop-2.3", "-Dhadoop.version=2.3.0"], + "hadoop2.3": ["-Pyarn", "-Phadoop-2.3"], + "hadoop2.4": ["-Pyarn", "-Phadoop-2.4"], "hadoop2.6": ["-Pyarn", "-Phadoop-2.6"], } |