diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2016-08-25 23:22:40 +0200 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-08-25 23:22:40 +0200 |
| commit | a133057ce5817f834babe9f25023092aec3c321d (patch) | |
| tree | 02ac9d496d357a5ae7f450fd00b0e8c9cc53cd19 /sql/catalyst | |
| parent | 9958ac0ce2b9e451d400604767bef2fe12a3399d (diff) | |
| download | spark-a133057ce5817f834babe9f25023092aec3c321d.tar.gz spark-a133057ce5817f834babe9f25023092aec3c321d.tar.bz2 spark-a133057ce5817f834babe9f25023092aec3c321d.zip | |
[SPARK-17229][SQL] PostgresDialect shouldn't widen float and short types during reads
## What changes were proposed in this pull request?
When reading float4 and smallint columns from PostgreSQL, Spark's `PostgresDialect` widens these types to Decimal and Integer rather than using the narrower Float and Short types. According to https://www.postgresql.org/docs/7.1/static/datatype.html#DATATYPE-TABLE, Postgres maps the `smallint` type to a signed two-byte integer and the `real` / `float4` types to single precision floating point numbers.
This patch fixes this by adding more special-cases to `getCatalystType`, similar to what was done for the Derby JDBC dialect. I also fixed a similar problem in the write path which causes Spark to create integer columns in Postgres for what should have been ShortType columns.
## How was this patch tested?
New test cases in `PostgresIntegrationSuite` (which I ran manually because Jenkins can't run it right now).
Author: Josh Rosen <joshrosen@databricks.com>
Closes #14796 from JoshRosen/postgres-jdbc-type-fixes.
Diffstat (limited to 'sql/catalyst')
0 files changed, 0 insertions, 0 deletions