diff options
| author | Davies Liu <davies@databricks.com> | 2016-06-10 21:12:06 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-06-10 21:12:06 -0700 |
| commit | 7504bc73f20fe0e6546a019ed91c3fd3804287ba (patch) | |
| tree | c6dd39e4a922bce5747d1d333add2e4be6e304e5 /docs/ml-classification-regression.md | |
| parent | 468da03e23a01e02718608f05d778386cbb8416b (diff) | |
| download | spark-7504bc73f20fe0e6546a019ed91c3fd3804287ba.tar.gz spark-7504bc73f20fe0e6546a019ed91c3fd3804287ba.tar.bz2 spark-7504bc73f20fe0e6546a019ed91c3fd3804287ba.zip | |
[SPARK-15759] [SQL] Fallback to non-codegen when fail to compile generated code
## What changes were proposed in this pull request?
In case of any bugs in whole-stage codegen, the generated code can't be compiled, we should fallback to non-codegen to make sure that query could run.
The batch mode of new parquet reader depends on codegen, can't be easily switched to non-batch mode, so we still use codegen for batched scan (for parquet). Because it only support primitive types and the number of columns is less than spark.sql.codegen.maxFields (100), it should not fail.
This could be configurable by `spark.sql.codegen.fallback`
## How was this patch tested?
Manual test it with buggy operator, it worked well.
Author: Davies Liu <davies@databricks.com>
Closes #13501 from davies/codegen_fallback.
Diffstat (limited to 'docs/ml-classification-regression.md')
0 files changed, 0 insertions, 0 deletions