diff options
| author | Reynold Xin <rxin@databricks.com> | 2015-02-10 19:50:44 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-02-10 19:50:56 -0800 |
| commit | e477e91e3b65a6feb5f8d5593a2e69f3c715497a (patch) | |
| tree | f56ebd5230ddff9db7bd33216ccad0aab41a0a1e /mllib | |
| parent | 7fa0d5f5c8f8f712d5ed787b5731d4ac57eea7a7 (diff) | |
| download | spark-e477e91e3b65a6feb5f8d5593a2e69f3c715497a.tar.gz spark-e477e91e3b65a6feb5f8d5593a2e69f3c715497a.tar.bz2 spark-e477e91e3b65a6feb5f8d5593a2e69f3c715497a.zip | |
[SQL][DataFrame] Fix column computability bug.
Do not recursively strip out projects. Only strip the first level project.
```scala
df("colA") + df("colB").as("colC")
```
Previously, the above would construct an invalid plan.
Author: Reynold Xin <rxin@databricks.com>
Closes #4519 from rxin/computability and squashes the following commits:
87ff763 [Reynold Xin] Code review feedback.
015c4fc [Reynold Xin] [SQL][DataFrame] Fix column computability.
(cherry picked from commit 7e24249af1e2f896328ef0402fa47db78cb6f9ec)
Signed-off-by: Michael Armbrust <michael@databricks.com>
Diffstat (limited to 'mllib')
| -rw-r--r-- | mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala b/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala index 9ff06ac362..16979c9ed4 100644 --- a/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala +++ b/mllib/src/main/scala/org/apache/spark/mllib/recommendation/MatrixFactorizationModel.scala @@ -180,7 +180,7 @@ object MatrixFactorizationModel extends Loader[MatrixFactorizationModel] { def save(model: MatrixFactorizationModel, path: String): Unit = { val sc = model.userFeatures.sparkContext val sqlContext = new SQLContext(sc) - import sqlContext.implicits.createDataFrame + import sqlContext.implicits._ val metadata = (thisClassName, thisFormatVersion, model.rank) val metadataRDD = sc.parallelize(Seq(metadata), 1).toDataFrame("class", "version", "rank") metadataRDD.toJSON.saveAsTextFile(metadataPath(path)) |