diff options
| author | Xiangrui Meng <meng@databricks.com> | 2015-05-07 15:45:37 -0700 |
|---|---|---|
| committer | Xiangrui Meng <meng@databricks.com> | 2015-05-07 15:45:47 -0700 |
| commit | 475143a56b224e7cea3b2dc4a4cfbf7bb75129b7 (patch) | |
| tree | fc07e1ac554459470f6b51e5ec1685df991b2992 /mllib/src/main | |
| parent | 76e58b5d8897fa62a21745d3cfb4a0efa52de1e8 (diff) | |
| download | spark-475143a56b224e7cea3b2dc4a4cfbf7bb75129b7.tar.gz spark-475143a56b224e7cea3b2dc4a4cfbf7bb75129b7.tar.bz2 spark-475143a56b224e7cea3b2dc4a4cfbf7bb75129b7.zip | |
[SPARK-6948] [MLLIB] compress vectors in VectorAssembler
The compression is based on storage. brkyvz
Author: Xiangrui Meng <meng@databricks.com>
Closes #5985 from mengxr/SPARK-6948 and squashes the following commits:
df56a00 [Xiangrui Meng] update python tests
6d90d45 [Xiangrui Meng] compress vectors in VectorAssembler
(cherry picked from commit e43803b8f477b2c8d28836ac163cb54328d13f1a)
Signed-off-by: Xiangrui Meng <meng@databricks.com>
Diffstat (limited to 'mllib/src/main')
| -rw-r--r-- | mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala b/mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala index b5a69cee6d..796758a70e 100644 --- a/mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala +++ b/mllib/src/main/scala/org/apache/spark/ml/feature/VectorAssembler.scala @@ -102,6 +102,6 @@ object VectorAssembler { case o => throw new SparkException(s"$o of type ${o.getClass.getName} is not supported.") } - Vectors.sparse(cur, indices.result(), values.result()) + Vectors.sparse(cur, indices.result(), values.result()).compressed } } |