diff options
| author | DB Tsai <dbt@netflix.com> | 2016-04-11 09:35:47 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2016-04-11 09:35:47 -0700 |
| commit | efaf7d18205f5ae3a1c767942ee7d7320f7410de (patch) | |
| tree | c775391456436534bd7f9a3679f2687c7eb6f114 /dev/sparktestsupport | |
| parent | 643b4e2257c56338b192f8554e2fe5523bea4bdf (diff) | |
| download | spark-efaf7d18205f5ae3a1c767942ee7d7320f7410de.tar.gz spark-efaf7d18205f5ae3a1c767942ee7d7320f7410de.tar.bz2 spark-efaf7d18205f5ae3a1c767942ee7d7320f7410de.zip | |
[SPARK-14462][ML][MLLIB] Add the mllib-local build to maven pom
## What changes were proposed in this pull request?
In order to separate the linear algebra, and vector matrix classes into a standalone jar, we need to setup the build first. This PR will create a new jar called mllib-local with minimal dependencies.
The previous PR was failing the build because of `spark-core:test` dependency, and that was reverted. In this PR, `FunSuite` with `// scalastyle:ignore funsuite` in mllib-local test was used, similar to sketch.
Thanks.
## How was this patch tested?
Unit tests
mengxr tedyu holdenk
Author: DB Tsai <dbt@netflix.com>
Closes #12298 from dbtsai/dbtsai-mllib-local-build-fix.
Diffstat (limited to 'dev/sparktestsupport')
| -rw-r--r-- | dev/sparktestsupport/modules.py | 14 |
1 files changed, 13 insertions, 1 deletions
diff --git a/dev/sparktestsupport/modules.py b/dev/sparktestsupport/modules.py index bb04ec6ee6..c844bcff7e 100644 --- a/dev/sparktestsupport/modules.py +++ b/dev/sparktestsupport/modules.py @@ -256,9 +256,21 @@ streaming_flume_assembly = Module( ) +mllib_local = Module( + name="mllib-local", + dependencies=[], + source_file_regexes=[ + "mllib-local", + ], + sbt_test_goals=[ + "mllib-local/test", + ] +) + + mllib = Module( name="mllib", - dependencies=[streaming, sql], + dependencies=[mllib_local, streaming, sql], source_file_regexes=[ "data/mllib/", "mllib/", |