diff options
author | hyukjinkwon <gurwls223@gmail.com> | 2017-02-13 16:08:31 +0100 |
---|---|---|
committer | Herman van Hovell <hvanhovell@databricks.com> | 2017-02-13 16:08:31 +0100 |
commit | 4321ff9edda4961273ac4a5b02dc1aed03f05e47 (patch) | |
tree | a12f4daf2b7dd1e58b92e3ee94bbb7ff2d8c807c /python/pyspark | |
parent | 04ad822534e8ded96a9ba4b7d43320e53c6d2808 (diff) | |
download | spark-4321ff9edda4961273ac4a5b02dc1aed03f05e47.tar.gz spark-4321ff9edda4961273ac4a5b02dc1aed03f05e47.tar.bz2 spark-4321ff9edda4961273ac4a5b02dc1aed03f05e47.zip |
[SPARK-19544][SQL] Improve error message when some column types are compatible and others are not in set operations
## What changes were proposed in this pull request?
This PR proposes to fix the error message when some data types are compatible and others are not in set/union operation.
Currently, the code below:
```scala
Seq((1,("a", 1))).toDF.union(Seq((1L,("a", "b"))).toDF)
```
throws an exception saying `LongType` and `IntegerType` are incompatible types. It should say something about `StructType`s with more readable format as below:
**Before**
```
Union can only be performed on tables with the compatible column types.
LongType <> IntegerType at the first column of the second table;;
```
**After**
```
Union can only be performed on tables with the compatible column types.
struct<_1:string,_2:string> <> struct<_1:string,_2:int> at the second column of the second table;;
```
*I manually inserted a newline in the messages above for readability only in this PR description.
## How was this patch tested?
Unit tests in `AnalysisErrorSuite`, manual tests and build wth Scala 2.10.
Author: hyukjinkwon <gurwls223@gmail.com>
Closes #16882 from HyukjinKwon/SPARK-19544.
Diffstat (limited to 'python/pyspark')
0 files changed, 0 insertions, 0 deletions