aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark
diff options
context:
space:
mode:
authorhyukjinkwon <gurwls223@gmail.com>2017-02-13 16:08:31 +0100
committerHerman van Hovell <hvanhovell@databricks.com>2017-02-13 16:08:31 +0100
commit4321ff9edda4961273ac4a5b02dc1aed03f05e47 (patch)
treea12f4daf2b7dd1e58b92e3ee94bbb7ff2d8c807c /python/pyspark
parent04ad822534e8ded96a9ba4b7d43320e53c6d2808 (diff)
downloadspark-4321ff9edda4961273ac4a5b02dc1aed03f05e47.tar.gz
spark-4321ff9edda4961273ac4a5b02dc1aed03f05e47.tar.bz2
spark-4321ff9edda4961273ac4a5b02dc1aed03f05e47.zip
[SPARK-19544][SQL] Improve error message when some column types are compatible and others are not in set operations
## What changes were proposed in this pull request? This PR proposes to fix the error message when some data types are compatible and others are not in set/union operation. Currently, the code below: ```scala Seq((1,("a", 1))).toDF.union(Seq((1L,("a", "b"))).toDF) ``` throws an exception saying `LongType` and `IntegerType` are incompatible types. It should say something about `StructType`s with more readable format as below: **Before** ``` Union can only be performed on tables with the compatible column types. LongType <> IntegerType at the first column of the second table;; ``` **After** ``` Union can only be performed on tables with the compatible column types. struct<_1:string,_2:string> <> struct<_1:string,_2:int> at the second column of the second table;; ``` *I manually inserted a newline in the messages above for readability only in this PR description. ## How was this patch tested? Unit tests in `AnalysisErrorSuite`, manual tests and build wth Scala 2.10. Author: hyukjinkwon <gurwls223@gmail.com> Closes #16882 from HyukjinKwon/SPARK-19544.
Diffstat (limited to 'python/pyspark')
0 files changed, 0 insertions, 0 deletions