diff options
| author | hyukjinkwon <gurwls223@gmail.com> | 2016-09-28 00:39:47 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2016-09-28 00:39:47 +0800 |
| commit | 5de1737b02710e36f6804d2ae243d1aeb30a0b32 (patch) | |
| tree | f11d8c8057f66ccf35e1186e5cedeef18e390af5 /common | |
| parent | 6a68c5d7b4eb07e4ed6b702dd1536cd08d9bba7d (diff) | |
| download | spark-5de1737b02710e36f6804d2ae243d1aeb30a0b32.tar.gz spark-5de1737b02710e36f6804d2ae243d1aeb30a0b32.tar.bz2 spark-5de1737b02710e36f6804d2ae243d1aeb30a0b32.zip | |
[SPARK-16777][SQL] Do not use deprecated listType API in ParquetSchemaConverter
## What changes were proposed in this pull request?
This PR removes build waning as below.
```scala
[WARNING] .../spark/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala:448: method listType in object ConversionPatterns is deprecated: see corresponding Javadoc for more information.
[WARNING] ConversionPatterns.listType(
[WARNING] ^
[WARNING] .../spark/sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetSchemaConverter.scala:464: method listType in object ConversionPatterns is deprecated: see corresponding Javadoc for more information.
[WARNING] ConversionPatterns.listType(
[WARNING] ^
```
This should not use `listOfElements` (recommended to be replaced from `listType`) instead because the new method checks if the name of elements in Parquet's `LIST` is `element` in Parquet schema and throws an exception if not. However, It seems Spark prior to 1.4.x writes `ArrayType` with Parquet's `LIST` but with `array` as its element name.
Therefore, this PR avoids to use both `listOfElements` and `listType` but just use the existing schema builder to construct the same `GroupType`.
## How was this patch tested?
Existing tests should cover this.
Author: hyukjinkwon <gurwls223@gmail.com>
Closes #14399 from HyukjinKwon/SPARK-16777.
Diffstat (limited to 'common')
0 files changed, 0 insertions, 0 deletions