aboutsummaryrefslogtreecommitdiff
path: root/sql/core/src/test
diff options
context:
space:
mode:
authorwangyang <wangyang@haizhi.com>2016-11-05 14:32:28 +0100
committerHerman van Hovell <hvanhovell@databricks.com>2016-11-05 14:32:28 +0100
commitfb0d60814a79747beb68da9613679141c44f2540 (patch)
tree8bf2e4e959b546a9a69bd2b3ab0841869ab9d9ce /sql/core/src/test
parenta87471c83006ec11c372b4f915e17a0501f1f536 (diff)
downloadspark-fb0d60814a79747beb68da9613679141c44f2540.tar.gz
spark-fb0d60814a79747beb68da9613679141c44f2540.tar.bz2
spark-fb0d60814a79747beb68da9613679141c44f2540.zip
[SPARK-17849][SQL] Fix NPE problem when using grouping sets
## What changes were proposed in this pull request? Prior this pr, the following code would cause an NPE: `case class point(a:String, b:String, c:String, d: Int)` `val data = Seq( point("1","2","3", 1), point("4","5","6", 1), point("7","8","9", 1) )` `sc.parallelize(data).toDF().registerTempTable("table")` `spark.sql("select a, b, c, count(d) from table group by a, b, c GROUPING SETS ((a)) ").show()` The reason is that when the grouping_id() behavior was changed in #10677, some code (which should be changed) was left out. Take the above code for example, prior #10677, the bit mask for set "(a)" was `001`, while after #10677 the bit mask was changed to `011`. However, the `nonNullBitmask` was not changed accordingly. This pr will fix this problem. ## How was this patch tested? add integration tests Author: wangyang <wangyang@haizhi.com> Closes #15416 from yangw1234/groupingid.
Diffstat (limited to 'sql/core/src/test')
-rw-r--r--sql/core/src/test/resources/sql-tests/inputs/grouping_set.sql17
-rw-r--r--sql/core/src/test/resources/sql-tests/results/grouping_set.sql.out42
2 files changed, 59 insertions, 0 deletions
diff --git a/sql/core/src/test/resources/sql-tests/inputs/grouping_set.sql b/sql/core/src/test/resources/sql-tests/inputs/grouping_set.sql
new file mode 100644
index 0000000000..3594283505
--- /dev/null
+++ b/sql/core/src/test/resources/sql-tests/inputs/grouping_set.sql
@@ -0,0 +1,17 @@
+CREATE TEMPORARY VIEW grouping AS SELECT * FROM VALUES
+ ("1", "2", "3", 1),
+ ("4", "5", "6", 1),
+ ("7", "8", "9", 1)
+ as grouping(a, b, c, d);
+
+-- SPARK-17849: grouping set throws NPE #1
+SELECT a, b, c, count(d) FROM grouping GROUP BY a, b, c GROUPING SETS (());
+
+-- SPARK-17849: grouping set throws NPE #2
+SELECT a, b, c, count(d) FROM grouping GROUP BY a, b, c GROUPING SETS ((a));
+
+-- SPARK-17849: grouping set throws NPE #3
+SELECT a, b, c, count(d) FROM grouping GROUP BY a, b, c GROUPING SETS ((c));
+
+
+
diff --git a/sql/core/src/test/resources/sql-tests/results/grouping_set.sql.out b/sql/core/src/test/resources/sql-tests/results/grouping_set.sql.out
new file mode 100644
index 0000000000..edb38a52b7
--- /dev/null
+++ b/sql/core/src/test/resources/sql-tests/results/grouping_set.sql.out
@@ -0,0 +1,42 @@
+-- Automatically generated by SQLQueryTestSuite
+-- Number of queries: 4
+
+
+-- !query 0
+CREATE TEMPORARY VIEW grouping AS SELECT * FROM VALUES
+ ("1", "2", "3", 1),
+ ("4", "5", "6", 1),
+ ("7", "8", "9", 1)
+ as grouping(a, b, c, d)
+-- !query 0 schema
+struct<>
+-- !query 0 output
+
+
+
+-- !query 1
+SELECT a, b, c, count(d) FROM grouping GROUP BY a, b, c GROUPING SETS (())
+-- !query 1 schema
+struct<a:string,b:string,c:string,count(d):bigint>
+-- !query 1 output
+NULL NULL NULL 3
+
+
+-- !query 2
+SELECT a, b, c, count(d) FROM grouping GROUP BY a, b, c GROUPING SETS ((a))
+-- !query 2 schema
+struct<a:string,b:string,c:string,count(d):bigint>
+-- !query 2 output
+1 NULL NULL 1
+4 NULL NULL 1
+7 NULL NULL 1
+
+
+-- !query 3
+SELECT a, b, c, count(d) FROM grouping GROUP BY a, b, c GROUPING SETS ((c))
+-- !query 3 schema
+struct<a:string,b:string,c:string,count(d):bigint>
+-- !query 3 output
+NULL NULL 3 1
+NULL NULL 6 1
+NULL NULL 9 1