diff options
| author | Reynold Xin <rxin@databricks.com> | 2016-03-22 11:37:37 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-03-22 11:37:37 -0700 |
| commit | 297c20226d3330309c9165d789749458f8f4ab8e (patch) | |
| tree | 84d0bf8d211783732fcb5eb8455675b014d45e09 /bin | |
| parent | caea15214571d9b12dcf1553e5c1cc8b83a8ba5b (diff) | |
| download | spark-297c20226d3330309c9165d789749458f8f4ab8e.tar.gz spark-297c20226d3330309c9165d789749458f8f4ab8e.tar.bz2 spark-297c20226d3330309c9165d789749458f8f4ab8e.zip | |
[SPARK-14063][SQL] SQLContext.range should return Dataset[java.lang.Long]
## What changes were proposed in this pull request?
This patch changed the return type for SQLContext.range from `Dataset[Long]` (Scala primitive) to `Dataset[java.lang.Long]` (Java boxed long).
Previously, SPARK-13894 changed the return type of range from `Dataset[Row]` to `Dataset[Long]`. The problem is that due to https://issues.scala-lang.org/browse/SI-4388, Scala compiles primitive types in generics into just Object, i.e. range at bytecode level now just returns `Dataset[Object]`. This is really bad for Java users because they are losing type safety and also need to add a type cast every time they use range.
Talked to Jason Zaugg from Lightbend (Typesafe) who suggested the best approach is to return `Dataset[java.lang.Long]`. The downside is that when Scala users want to explicitly type a closure used on the dataset returned by range, they would need to use `java.lang.Long` instead of the Scala `Long`.
## How was this patch tested?
The signature change should be covered by existing unit tests and API tests. I also added a new test case in DatasetSuite for range.
Author: Reynold Xin <rxin@databricks.com>
Closes #11880 from rxin/SPARK-14063.
Diffstat (limited to 'bin')
0 files changed, 0 insertions, 0 deletions