diff options
| author | Reynold Xin <rxin@databricks.com> | 2015-02-10 19:50:44 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-02-10 19:50:44 -0800 |
| commit | 7e24249af1e2f896328ef0402fa47db78cb6f9ec (patch) | |
| tree | c78e19ed2273af7f53e2640a4884e669d3ea979a /mllib | |
| parent | 45df77b8418873a00d770e435358bf603765595f (diff) | |
| download | spark-7e24249af1e2f896328ef0402fa47db78cb6f9ec.tar.gz spark-7e24249af1e2f896328ef0402fa47db78cb6f9ec.tar.bz2 spark-7e24249af1e2f896328ef0402fa47db78cb6f9ec.zip | |
[SQL][DataFrame] Fix column computability bug.
Do not recursively strip out projects. Only strip the first level project.
```scala
df("colA") + df("colB").as("colC")
```
Previously, the above would construct an invalid plan.
Author: Reynold Xin <rxin@databricks.com>
Closes #4519 from rxin/computability and squashes the following commits:
87ff763 [Reynold Xin] Code review feedback.
015c4fc [Reynold Xin] [SQL][DataFrame] Fix column computability.
Diffstat (limited to 'mllib')
| -rw-r--r-- | mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala b/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala index 9ff06ac362..16979c9ed4 100644 --- a/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala +++ b/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala @@ -180,7 +180,7 @@ object MatrixFactorizationModel extends Loader[MatrixFactorizationModel] { def save(model: MatrixFactorizationModel, path: String): Unit = { val sc = model.userFeatures.sparkContext val sqlContext = new SQLContext(sc) - import sqlContext.implicits.createDataFrame + import sqlContext.implicits._ val metadata = (thisClassName, thisFormatVersion, model.rank) val metadataRDD = sc.parallelize(Seq(metadata), 1).toDataFrame("class", "version", "rank") metadataRDD.toJSON.saveAsTextFile(metadataPath(path)) |