diff options
| author | Michael Armbrust <michael@databricks.com> | 2014-11-20 18:31:02 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-11-20 18:31:02 -0800 |
| commit | 02ec058efe24348cdd3691b55942e6f0ef138732 (patch) | |
| tree | aeb665a9e313f6dfe9de73329987c26762537c8f /docs/README.md | |
| parent | 84d79ee9ec47465269f7b0a7971176da93c96f3f (diff) | |
| download | spark-02ec058efe24348cdd3691b55942e6f0ef138732.tar.gz spark-02ec058efe24348cdd3691b55942e6f0ef138732.tar.bz2 spark-02ec058efe24348cdd3691b55942e6f0ef138732.zip | |
[SPARK-4413][SQL] Parquet support through datasource API
Goals:
- Support for accessing parquet using SQL but not requiring Hive (thus allowing support of parquet tables with decimal columns)
- Support for folder based partitioning with automatic discovery of available partitions
- Caching of file metadata
See scaladoc of `ParquetRelation2` for more details.
Author: Michael Armbrust <michael@databricks.com>
Closes #3269 from marmbrus/newParquet and squashes the following commits:
1dd75f1 [Michael Armbrust] Pass all paths for FileInputFormat at once.
645768b [Michael Armbrust] Review comments.
abd8e2f [Michael Armbrust] Alternative implementation of parquet based on the datasources API.
938019e [Michael Armbrust] Add an experimental interface to data sources that exposes catalyst expressions.
e9d2641 [Michael Armbrust] logging / formatting improvements.
Diffstat (limited to 'docs/README.md')
0 files changed, 0 insertions, 0 deletions