diff options
author | gatorsmile <gatorsmile@gmail.com> | 2016-07-22 13:27:17 +0800 |
---|---|---|
committer | Wenchen Fan <wenchen@databricks.com> | 2016-07-22 13:27:17 +0800 |
commit | 94f14b52a6a99047c0e30015d435bddb7f2b95fe (patch) | |
tree | d62259a9b3583af09a091cacd9def543d9c7bae4 /bin/spark-class2.cmd | |
parent | df2c6d59d0e1a3db9942dbc5e4993cf3babc2d60 (diff) | |
download | spark-94f14b52a6a99047c0e30015d435bddb7f2b95fe.tar.gz spark-94f14b52a6a99047c0e30015d435bddb7f2b95fe.tar.bz2 spark-94f14b52a6a99047c0e30015d435bddb7f2b95fe.zip |
[SPARK-16556][SPARK-16559][SQL] Fix Two Bugs in Bucket Specification
### What changes were proposed in this pull request?
**Issue 1: Silent Ignorance of Bucket Specification When Creating Table Using Schema Inference**
When creating a data source table without explicit specification of schema or SELECT clause, we silently ignore the bucket specification (CLUSTERED BY... SORTED BY...) in [the code](https://github.com/apache/spark/blob/ce3b98bae28af72299722f56e4e4ef831f471ec0/sql/core/src/main/scala/org/apache/spark/sql/execution/command/createDataSourceTables.scala#L339-L354).
For example,
```SQL
CREATE TABLE jsonTable
USING org.apache.spark.sql.json
OPTIONS (
path '${tempDir.getCanonicalPath}'
)
CLUSTERED BY (inexistentColumnA) SORTED BY (inexistentColumnB) INTO 2 BUCKETS
```
This PR captures it and issues an error message.
**Issue 2: Got a run-time `java.lang.ArithmeticException` when num of buckets is set to zero.**
For example,
```SQL
CREATE TABLE t USING PARQUET
OPTIONS (PATH '${path.toString}')
CLUSTERED BY (a) SORTED BY (b) INTO 0 BUCKETS
AS SELECT 1 AS a, 2 AS b
```
The exception we got is
```
ERROR org.apache.spark.executor.Executor: Exception in task 0.0 in stage 1.0 (TID 2)
java.lang.ArithmeticException: / by zero
```
This PR captures the misuse and issues an appropriate error message.
### How was this patch tested?
Added a test case in DDLSuite
Author: gatorsmile <gatorsmile@gmail.com>
Closes #14210 from gatorsmile/createTableWithoutSchema.
Diffstat (limited to 'bin/spark-class2.cmd')
0 files changed, 0 insertions, 0 deletions