diff options
| author | Szul, Piotr <Piotr.Szul@csiro.au> | 2014-06-25 21:55:49 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2014-06-25 21:55:49 -0700 |
| commit | fa167194ce1b5898e4d7232346c9f86b2897a722 (patch) | |
| tree | a2d35858922858fc17ae20ebe8f6609619a93d0b /mllib | |
| parent | 92b01250246ef1211d6ea15036ebc705ccabe7f8 (diff) | |
| download | spark-fa167194ce1b5898e4d7232346c9f86b2897a722.tar.gz spark-fa167194ce1b5898e4d7232346c9f86b2897a722.tar.bz2 spark-fa167194ce1b5898e4d7232346c9f86b2897a722.zip | |
[SPARK-2172] PySpark cannot import mllib modules in YARN-client mode
Include pyspark/mllib python sources as resources in the mllib.jar.
This way they will be included in the final assembly
Author: Szul, Piotr <Piotr.Szul@csiro.au>
Closes #1223 from piotrszul/branch-1.0 and squashes the following commits:
69d5174 [Szul, Piotr] Removed unsed resource directory src/main/resource from mllib pom
f8c52a0 [Szul, Piotr] [SPARK-2172] PySpark cannot import mllib modules in YARN-client mode Include pyspark/mllib python sources as resources in the jar
Diffstat (limited to 'mllib')
| -rw-r--r-- | mllib/pom.xml | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/mllib/pom.xml b/mllib/pom.xml index 4c67de8b24..c6bc8bf90f 100644 --- a/mllib/pom.xml +++ b/mllib/pom.xml @@ -76,5 +76,13 @@ <artifactId>scalatest-maven-plugin</artifactId> </plugin> </plugins> + <resources> + <resource> + <directory>../python</directory> + <includes> + <include>pyspark/mllib/*.py</include> + </includes> + </resource> + </resources> </build> </project> |