diff options
| author | Xiangrui Meng <meng@databricks.com> | 2014-08-26 15:12:27 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2014-08-26 15:12:27 -0700 |
| commit | adbd5c1636669fc474ab02b54cd1ced353f68712 (patch) | |
| tree | 1553d031ac637c9803325db9f5d73e73f34f12ef /docs/mllib-guide.md | |
| parent | 6b5584ef1c605cd30f25dbe7099ab32aea1746fb (diff) | |
| download | spark-adbd5c1636669fc474ab02b54cd1ced353f68712.tar.gz spark-adbd5c1636669fc474ab02b54cd1ced353f68712.tar.bz2 spark-adbd5c1636669fc474ab02b54cd1ced353f68712.zip | |
[SPARK-3226][MLLIB] doc update for native libraries
to mention `-Pnetlib-lgpl` option. atalwalkar
Author: Xiangrui Meng <meng@databricks.com>
Closes #2128 from mengxr/mllib-native and squashes the following commits:
4cbba57 [Xiangrui Meng] update mllib dependencies
Diffstat (limited to 'docs/mllib-guide.md')
| -rw-r--r-- | docs/mllib-guide.md | 25 |
1 files changed, 15 insertions, 10 deletions
diff --git a/docs/mllib-guide.md b/docs/mllib-guide.md index ca0a84a8c5..4d4198b9e0 100644 --- a/docs/mllib-guide.md +++ b/docs/mllib-guide.md @@ -35,18 +35,23 @@ and the migration guide below will explain all changes between releases. # Dependencies -MLlib uses the linear algebra package [Breeze](http://www.scalanlp.org/), which depends on -[netlib-java](https://github.com/fommil/netlib-java), and -[jblas](https://github.com/mikiobraun/jblas). +MLlib uses the linear algebra package [Breeze](http://www.scalanlp.org/), +which depends on [netlib-java](https://github.com/fommil/netlib-java), +and [jblas](https://github.com/mikiobraun/jblas). `netlib-java` and `jblas` depend on native Fortran routines. You need to install the -[gfortran runtime library](https://github.com/mikiobraun/jblas/wiki/Missing-Libraries) if it is not -already present on your nodes. MLlib will throw a linking error if it cannot detect these libraries -automatically. Due to license issues, we do not include `netlib-java`'s native libraries in MLlib's -dependency set. If no native library is available at runtime, you will see a warning message. To -use native libraries from `netlib-java`, please include artifact -`com.github.fommil.netlib:all:1.1.2` as a dependency of your project or build your own (see -[instructions](https://github.com/fommil/netlib-java/blob/master/README.md#machine-optimised-system-libraries)). +[gfortran runtime library](https://github.com/mikiobraun/jblas/wiki/Missing-Libraries) +if it is not already present on your nodes. +MLlib will throw a linking error if it cannot detect these libraries automatically. +Due to license issues, we do not include `netlib-java`'s native libraries in MLlib's +dependency set under default settings. +If no native library is available at runtime, you will see a warning message. +To use native libraries from `netlib-java`, please build Spark with `-Pnetlib-lgpl` or +include `com.github.fommil.netlib:all:1.1.2` as a dependency of your project. +If you want to use optimized BLAS/LAPACK libraries such as +[OpenBLAS](http://www.openblas.net/), please link its shared libraries to +`/usr/lib/libblas.so.3` and `/usr/lib/liblapack.so.3`, respectively. +BLAS/LAPACK libraries on worker nodes should be built without multithreading. To use MLlib in Python, you will need [NumPy](http://www.numpy.org) version 1.4 or newer. |