diff options
| author | Yin Huai <yhuai@databricks.com> | 2016-07-25 20:58:07 -0700 |
|---|---|---|
| committer | Yin Huai <yhuai@databricks.com> | 2016-07-25 20:58:07 -0700 |
| commit | 815f3eece5f095919a329af8cbd762b9ed71c7a8 (patch) | |
| tree | 8078da6a29d07ca784c07d54813b2a0c6d256361 /docs/ml-migration-guides.md | |
| parent | f99e34e8e58c97ff30c6e054875533350d99fe5b (diff) | |
| download | spark-815f3eece5f095919a329af8cbd762b9ed71c7a8.tar.gz spark-815f3eece5f095919a329af8cbd762b9ed71c7a8.tar.bz2 spark-815f3eece5f095919a329af8cbd762b9ed71c7a8.zip | |
[SPARK-16633][SPARK-16642][SPARK-16721][SQL] Fixes three issues related to lead and lag functions
## What changes were proposed in this pull request?
This PR contains three changes.
First, this PR changes the behavior of lead/lag back to Spark 1.6's behavior, which is described as below:
1. lead/lag respect null input values, which means that if the offset row exists and the input value is null, the result will be null instead of the default value.
2. If the offset row does not exist, the default value will be used.
3. OffsetWindowFunction's nullable setting also considers the nullability of its input (because of the first change).
Second, this PR fixes the evaluation of lead/lag when the input expression is a literal. This fix is a result of the first change. In current master, if a literal is used as the input expression of a lead or lag function, the result will be this literal even if the offset row does not exist.
Third, this PR makes ResolveWindowFrame not fire if a window function is not resolved.
## How was this patch tested?
New tests in SQLWindowFunctionSuite
Author: Yin Huai <yhuai@databricks.com>
Closes #14284 from yhuai/lead-lag.
Diffstat (limited to 'docs/ml-migration-guides.md')
0 files changed, 0 insertions, 0 deletions