diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2016-01-11 12:56:43 -0800 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2016-01-11 12:56:43 -0800 |
| commit | a44991453a43615028083ba9546f5cd93112f6bd (patch) | |
| tree | 7892c2a789e9ec12bdba1732e597ecbfcfc31d21 /dev | |
| parent | b313badaa049f847f33663c61cd70ee2f2cbebac (diff) | |
| download | spark-a44991453a43615028083ba9546f5cd93112f6bd.tar.gz spark-a44991453a43615028083ba9546f5cd93112f6bd.tar.bz2 spark-a44991453a43615028083ba9546f5cd93112f6bd.zip | |
[SPARK-12734][HOTFIX] Build changes must trigger all tests; clean after install in dep tests
This patch fixes a build/test issue caused by the combination of #10672 and a latent issue in the original `dev/test-dependencies` script.
First, changes which _only_ touched build files were not triggering full Jenkins runs, making it possible for a build change to be merged even though it could cause failures in other tests. The `root` build module now depends on `build`, so all tests will now be run whenever a build-related file is changed.
I also added a `clean` step to the Maven install step in `dev/test-dependencies` in order to address an issue where the dummy JARs stuck around and caused "multiple assembly JARs found" errors in tests.
/cc zsxwing
Author: Josh Rosen <joshrosen@databricks.com>
Closes #10704 from JoshRosen/fix-build-test-problems.
Diffstat (limited to 'dev')
| -rw-r--r-- | dev/sparktestsupport/modules.py | 2 | ||||
| -rwxr-xr-x | dev/test-dependencies.sh | 2 |
2 files changed, 2 insertions, 2 deletions
diff --git a/dev/sparktestsupport/modules.py b/dev/sparktestsupport/modules.py index 1fc6596164..93a8c15e3e 100644 --- a/dev/sparktestsupport/modules.py +++ b/dev/sparktestsupport/modules.py @@ -426,7 +426,7 @@ yarn = Module( # No other modules should directly depend on this module. root = Module( name="root", - dependencies=[], + dependencies=[build], # Changes to build should trigger all tests. source_file_regexes=[], # In order to run all of the tests, enable every test profile: build_profile_flags=list(set( diff --git a/dev/test-dependencies.sh b/dev/test-dependencies.sh index def87aa408..3cb5d2be2a 100755 --- a/dev/test-dependencies.sh +++ b/dev/test-dependencies.sh @@ -70,7 +70,7 @@ $MVN -q versions:set -DnewVersion=$TEMP_VERSION -DgenerateBackupPoms=false > /de # Generate manifests for each Hadoop profile: for HADOOP_PROFILE in "${HADOOP_PROFILES[@]}"; do echo "Performing Maven install for $HADOOP_PROFILE" - $MVN $HADOOP2_MODULE_PROFILES -P$HADOOP_PROFILE jar:jar jar:test-jar install:install -q + $MVN $HADOOP2_MODULE_PROFILES -P$HADOOP_PROFILE jar:jar jar:test-jar install:install clean -q echo "Performing Maven validate for $HADOOP_PROFILE" $MVN $HADOOP2_MODULE_PROFILES -P$HADOOP_PROFILE validate -q |