diff options
| author | Forest Fang <forest.fang@outlook.com> | 2015-07-18 21:05:44 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-07-18 21:05:44 -0700 |
| commit | 6cb6096c016178b9ce5c97592abe529ddb18cef2 (patch) | |
| tree | 20c8f6292c8dcf610abb4fe58fe479e6e2999abf /sql/hive/compatibility/src/test/scala | |
| parent | 45d798c323ffe32bc2eba4dbd271c4572f5a30cf (diff) | |
| download | spark-6cb6096c016178b9ce5c97592abe529ddb18cef2.tar.gz spark-6cb6096c016178b9ce5c97592abe529ddb18cef2.tar.bz2 spark-6cb6096c016178b9ce5c97592abe529ddb18cef2.zip | |
[SPARK-8443][SQL] Split GenerateMutableProjection Codegen due to JVM Code Size Limits
By grouping projection calls into multiple apply function, we are able to push the number of projections codegen can handle from ~1k to ~60k. I have set the unit test to test against 5k as 60k took 15s for the unit test to complete.
Author: Forest Fang <forest.fang@outlook.com>
Closes #7076 from saurfang/codegen_size_limit and squashes the following commits:
b7a7635 [Forest Fang] [SPARK-8443][SQL] Execute and verify split projections in test
adef95a [Forest Fang] [SPARK-8443][SQL] Use safer factor and rewrite splitting code
1b5aa7e [Forest Fang] [SPARK-8443][SQL] inline execution if one block only
9405680 [Forest Fang] [SPARK-8443][SQL] split projection code by size limit
Diffstat (limited to 'sql/hive/compatibility/src/test/scala')
0 files changed, 0 insertions, 0 deletions