diff options
| author | Davies Liu <davies@databricks.com> | 2016-08-22 16:16:03 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-08-22 16:16:03 +0800 |
| commit | 8d35a6f68d6d733212674491cbf31bed73fada0f (patch) | |
| tree | 4574a1aeef90dab75b1300abb599632cde3973e7 /docs | |
| parent | 4b6c2cbcb109c7cef6087bae32d87cc3ddb69cf9 (diff) | |
| download | spark-8d35a6f68d6d733212674491cbf31bed73fada0f.tar.gz spark-8d35a6f68d6d733212674491cbf31bed73fada0f.tar.bz2 spark-8d35a6f68d6d733212674491cbf31bed73fada0f.zip | |
[SPARK-17115][SQL] decrease the threshold when split expressions
## What changes were proposed in this pull request?
In 2.0, we change the threshold of splitting expressions from 16K to 64K, which cause very bad performance on wide table, because the generated method can't be JIT compiled by default (above the limit of 8K bytecode).
This PR will decrease it to 1K, based on the benchmark results for a wide table with 400 columns of LongType.
It also fix a bug around splitting expression in whole-stage codegen (it should not split them).
## How was this patch tested?
Added benchmark suite.
Author: Davies Liu <davies@databricks.com>
Closes #14692 from davies/split_exprs.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions