diff options
author | Reynold Xin <rxin@databricks.com> | 2017-02-14 14:11:17 -0800 |
---|---|---|
committer | Wenchen Fan <wenchen@databricks.com> | 2017-02-14 14:11:17 -0800 |
commit | da7aef7a0ea921a2d8ee37b0e3939245e4168355 (patch) | |
tree | 52bac081267f2c474a9dc953546ab745e175238f /external/docker-integration-tests | |
parent | a3626ca333e6e1881e2f09ccae0fa8fa7243223e (diff) | |
download | spark-da7aef7a0ea921a2d8ee37b0e3939245e4168355.tar.gz spark-da7aef7a0ea921a2d8ee37b0e3939245e4168355.tar.bz2 spark-da7aef7a0ea921a2d8ee37b0e3939245e4168355.zip |
[SPARK-16475][SQL] Broadcast hint for SQL Queries
## What changes were proposed in this pull request?
This pull request introduces a simple hint infrastructure to SQL and implements broadcast join hint using the infrastructure.
The hint syntax looks like the following:
```
SELECT /*+ BROADCAST(t) */ * FROM t
```
For broadcast hint, we accept "BROADCAST", "BROADCASTJOIN", and "MAPJOIN", and a sequence of relation aliases can be specified in the hint. A broadcast hint plan node will be inserted on top of any relation (that is not aliased differently), subquery, or common table expression that match the specified name.
The hint resolution works by recursively traversing down the query plan to find a relation or subquery that matches one of the specified broadcast aliases. The traversal does not go past beyond any existing broadcast hints, subquery aliases. This rule happens before common table expressions.
Note that there was an earlier patch in https://github.com/apache/spark/pull/14426. This is a rewrite of that patch, with different semantics and simpler test cases.
## How was this patch tested?
Added a new unit test suite for the broadcast hint rule (SubstituteHintsSuite) and new test cases for parser change (in PlanParserSuite). Also added end-to-end test case in BroadcastSuite.
Author: Reynold Xin <rxin@databricks.com>
Author: Dongjoon Hyun <dongjoon@apache.org>
Closes #16925 from rxin/SPARK-16475-broadcast-hint.
Diffstat (limited to 'external/docker-integration-tests')
0 files changed, 0 insertions, 0 deletions