diff options
| author | Herman van Hovell <hvanhovell@databricks.com> | 2017-03-26 22:47:31 +0200 |
|---|---|---|
| committer | Herman van Hovell <hvanhovell@databricks.com> | 2017-03-26 22:47:31 +0200 |
| commit | 617ab6445ea33d8297f0691723fd19bae19228dc (patch) | |
| tree | 54eff793a6e82c0e3a319f1b6647057b443c010d /docs/mllib-naive-bayes.md | |
| parent | 362ee93296a0de6342b4339e941e6a11f445c5b2 (diff) | |
| download | spark-617ab6445ea33d8297f0691723fd19bae19228dc.tar.gz spark-617ab6445ea33d8297f0691723fd19bae19228dc.tar.bz2 spark-617ab6445ea33d8297f0691723fd19bae19228dc.zip | |
[SPARK-20086][SQL] CollapseWindow should not collapse dependent adjacent windows
## What changes were proposed in this pull request?
The `CollapseWindow` is currently to aggressive when collapsing adjacent windows. It also collapses windows in the which the parent produces a column that is consumed by the child; this creates an invalid window which will fail at runtime.
This PR fixes this by adding a check for dependent adjacent windows to the `CollapseWindow` rule.
## How was this patch tested?
Added a new test case to `CollapseWindowSuite`
Author: Herman van Hovell <hvanhovell@databricks.com>
Closes #17432 from hvanhovell/SPARK-20086.
Diffstat (limited to 'docs/mllib-naive-bayes.md')
0 files changed, 0 insertions, 0 deletions