diff options
| author | Dongjoon Hyun <dongjoon@apache.org> | 2016-08-12 19:07:34 +0200 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2016-08-12 19:07:34 +0200 |
| commit | 2a105134e9a3efd46b761fab5e563ddebb26575d (patch) | |
| tree | 9fafaf2d7af7655569111b749222831326ef8a44 /docs/streaming-flume-integration.md | |
| parent | bbae20ade14e50541e4403ca7b45bf6c11695d15 (diff) | |
| download | spark-2a105134e9a3efd46b761fab5e563ddebb26575d.tar.gz spark-2a105134e9a3efd46b761fab5e563ddebb26575d.tar.bz2 spark-2a105134e9a3efd46b761fab5e563ddebb26575d.zip | |
[SPARK-16771][SQL] WITH clause should not fall into infinite loop.
## What changes were proposed in this pull request?
This PR changes the CTE resolving rule to use only **forward-declared** tables in order to prevent infinite loops. More specifically, new logic is like the following.
* Resolve CTEs in `WITH` clauses first before replacing the main SQL body.
* When resolving CTEs, only forward-declared CTEs or base tables are referenced.
- Self-referencing is not allowed any more.
- Cross-referencing is not allowed any more.
**Reported Error Scenarios**
```scala
scala> sql("WITH t AS (SELECT 1 FROM t) SELECT * FROM t")
java.lang.StackOverflowError
...
scala> sql("WITH t1 AS (SELECT * FROM t2), t2 AS (SELECT 2 FROM t1) SELECT * FROM t1, t2")
java.lang.StackOverflowError
...
```
Note that `t`, `t1`, and `t2` are not declared in database. Spark falls into infinite loops before resolving table names.
## How was this patch tested?
Pass the Jenkins tests with new two testcases.
Author: Dongjoon Hyun <dongjoon@apache.org>
Closes #14397 from dongjoon-hyun/SPARK-16771-TREENODE.
Diffstat (limited to 'docs/streaming-flume-integration.md')
0 files changed, 0 insertions, 0 deletions