diff options
| author | Davies Liu <davies@databricks.com> | 2016-04-22 12:29:53 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-04-22 12:29:53 -0700 |
| commit | 0419d63169274ecd60c05c1ef4ce2d4ed3a49605 (patch) | |
| tree | 40000dc4997db973193e110e16ab9c9993bb2fa2 /sql/core/pom.xml | |
| parent | c417cec067715455c1536d37e0dba97cc8657f7b (diff) | |
| download | spark-0419d63169274ecd60c05c1ef4ce2d4ed3a49605.tar.gz spark-0419d63169274ecd60c05c1ef4ce2d4ed3a49605.tar.bz2 spark-0419d63169274ecd60c05c1ef4ce2d4ed3a49605.zip | |
[SPARK-14791] [SQL] fix risk condition between broadcast and subquery
## What changes were proposed in this pull request?
SparkPlan.prepare() could be called in different threads (BroadcastExchange will call it in a thread pool), it only make sure that doPrepare() will only be called once, the second call to prepare() may return earlier before all the children had finished prepare(). Then some operator may call doProduce() before prepareSubqueries(), `null` will be used as the result of subquery, which is wrong. This cause TPCDS Q23B returns wrong answer sometimes.
This PR added synchronization for prepare(), make sure all the children had finished prepare() before return. Also call prepare() in produce() (similar to execute()).
Added checking for ScalarSubquery to make sure that the subquery has finished before using the result.
## How was this patch tested?
Manually tested with Q23B, no wrong answer anymore.
Author: Davies Liu <davies@databricks.com>
Closes #12600 from davies/fix_risk.
Diffstat (limited to 'sql/core/pom.xml')
0 files changed, 0 insertions, 0 deletions