diff options
| author | Cheng Lian <lian@databricks.com> | 2015-07-23 17:49:33 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-07-23 17:49:33 -0700 |
| commit | bebe3f7b45f7b0a96f20d5af9b80633fd40cff06 (patch) | |
| tree | aa8529470a6a9625fb4883082cd590ae73c9e1c5 /docs | |
| parent | b2f3aca1e8c182b93e250f9d9c4aa69f97eaa11a (diff) | |
| download | spark-bebe3f7b45f7b0a96f20d5af9b80633fd40cff06.tar.gz spark-bebe3f7b45f7b0a96f20d5af9b80633fd40cff06.tar.bz2 spark-bebe3f7b45f7b0a96f20d5af9b80633fd40cff06.zip | |
[SPARK-9207] [SQL] Enables Parquet filter push-down by default
PARQUET-136 and PARQUET-173 have been fixed in parquet-mr 1.7.0. It's time to enable filter push-down by default now.
Author: Cheng Lian <lian@databricks.com>
Closes #7612 from liancheng/spark-9207 and squashes the following commits:
77e6b5e [Cheng Lian] Enables Parquet filter push-down by default
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/sql-programming-guide.md | 9 |
1 files changed, 2 insertions, 7 deletions
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 5838bc172f..95945eb7fc 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -1332,13 +1332,8 @@ Configuration of Parquet can be done using the `setConf` method on `SQLContext` </tr> <tr> <td><code>spark.sql.parquet.filterPushdown</code></td> - <td>false</td> - <td> - Turn on Parquet filter pushdown optimization. This feature is turned off by default because of a known - bug in Parquet 1.6.0rc3 (<a href="https://issues.apache.org/jira/browse/PARQUET-136">PARQUET-136</a>). - However, if your table doesn't contain any nullable string or binary columns, it's still safe to turn - this feature on. - </td> + <td>true</td> + <td>Enables Parquet filter push-down optimization when set to true.</td> </tr> <tr> <td><code>spark.sql.hive.convertMetastoreParquet</code></td> |