diff options
| author | hyukjinkwon <gurwls223@gmail.com> | 2015-08-20 08:13:25 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2015-08-20 08:13:25 +0800 |
| commit | ba5f7e1842f2c5852b5309910c0d39926643da69 (patch) | |
| tree | b88884aee0f09fe55a67508a60a875e8f62c9d3c /examples/src/main/python/transitive_closure.py | |
| parent | 8e0a072f78b4902d5f7ccc6b15232ed202a117f9 (diff) | |
| download | spark-ba5f7e1842f2c5852b5309910c0d39926643da69.tar.gz spark-ba5f7e1842f2c5852b5309910c0d39926643da69.tar.bz2 spark-ba5f7e1842f2c5852b5309910c0d39926643da69.zip | |
[SPARK-10035] [SQL] Parquet filters does not process EqualNullSafe filter.
As I talked with Lian,
1. I added EquelNullSafe to ParquetFilters
- It uses the same equality comparison filter with EqualTo since the Parquet filter performs actually null-safe equality comparison.
2. Updated the test code (ParquetFilterSuite)
- Convert catalyst.Expression to sources.Filter
- Removed Cast since only Literal is picked up as a proper Filter in DataSourceStrategy
- Added EquelNullSafe comparison
3. Removed deprecated createFilter for catalyst.Expression
Author: hyukjinkwon <gurwls223@gmail.com>
Author: 권혁진 <gurwls223@gmail.com>
Closes #8275 from HyukjinKwon/master.
Diffstat (limited to 'examples/src/main/python/transitive_closure.py')
0 files changed, 0 insertions, 0 deletions