diff options
author | Sean Zhong <seanzhong@databricks.com> | 2016-08-04 13:43:25 +0800 |
---|---|---|
committer | Wenchen Fan <wenchen@databricks.com> | 2016-08-04 13:43:25 +0800 |
commit | 27e815c31de26636df089b0b8d9bd678b92d3588 (patch) | |
tree | 01e2e747fec4ea4f21bb8141b16305be2fc93f03 /launcher | |
parent | 780c7224a5b8dd3bf7838c6f280c61daeef1dcbc (diff) | |
download | spark-27e815c31de26636df089b0b8d9bd678b92d3588.tar.gz spark-27e815c31de26636df089b0b8d9bd678b92d3588.tar.bz2 spark-27e815c31de26636df089b0b8d9bd678b92d3588.zip |
[SPARK-16888][SQL] Implements eval method for expression AssertNotNull
## What changes were proposed in this pull request?
Implements `eval()` method for expression `AssertNotNull` so that we can convert local projection on LocalRelation to another LocalRelation.
### Before change:
```
scala> import org.apache.spark.sql.catalyst.dsl.expressions._
scala> import org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull
scala> import org.apache.spark.sql.Column
scala> case class A(a: Int)
scala> Seq((A(1),2)).toDS().select(new Column(AssertNotNull("_1".attr, Nil))).explain
java.lang.UnsupportedOperationException: Only code-generated evaluation is supported.
at org.apache.spark.sql.catalyst.expressions.objects.AssertNotNull.eval(objects.scala:850)
...
```
### After the change:
```
scala> Seq((A(1),2)).toDS().select(new Column(AssertNotNull("_1".attr, Nil))).explain(true)
== Parsed Logical Plan ==
'Project [assertnotnull('_1) AS assertnotnull(_1)#5]
+- LocalRelation [_1#2, _2#3]
== Analyzed Logical Plan ==
assertnotnull(_1): struct<a:int>
Project [assertnotnull(_1#2) AS assertnotnull(_1)#5]
+- LocalRelation [_1#2, _2#3]
== Optimized Logical Plan ==
LocalRelation [assertnotnull(_1)#5]
== Physical Plan ==
LocalTableScan [assertnotnull(_1)#5]
```
## How was this patch tested?
Unit test.
Author: Sean Zhong <seanzhong@databricks.com>
Closes #14486 from clockfly/assertnotnull_eval.
Diffstat (limited to 'launcher')
0 files changed, 0 insertions, 0 deletions