diff options
| author | Dongjoon Hyun <dongjoon@apache.org> | 2016-06-02 11:12:17 -0500 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-06-02 11:12:17 -0500 |
| commit | b85d18f3bdedca7ae7f2c26ff64ce38c2796bd63 (patch) | |
| tree | ff6e1b4f5d898b8999023944fb301a18db57c862 /repl | |
| parent | 4fe7c7bd1eeb4ae954e750e521e6d72dc8ff5225 (diff) | |
| download | spark-b85d18f3bdedca7ae7f2c26ff64ce38c2796bd63.tar.gz spark-b85d18f3bdedca7ae7f2c26ff64ce38c2796bd63.tar.bz2 spark-b85d18f3bdedca7ae7f2c26ff64ce38c2796bd63.zip | |
[SPARK-15709][SQL] Prevent `freqItems` from raising `UnsupportedOperationException: empty.min`
## What changes were proposed in this pull request?
Currently, `freqItems` raises `UnsupportedOperationException` on `empty.min` usually when its `support` argument is high.
```scala
scala> spark.createDataset(Seq(1, 2, 2, 3, 3, 3)).stat.freqItems(Seq("value"), 2)
16/06/01 11:11:38 ERROR Executor: Exception in task 5.0 in stage 0.0 (TID 5)
java.lang.UnsupportedOperationException: empty.min
...
```
Also, the parameter checking message is wrong.
```
require(support >= 1e-4, s"support ($support) must be greater than 1e-4.")
```
This PR changes the logic to handle the `empty` case and also improves parameter checking.
## How was this patch tested?
Pass the Jenkins tests (with a new testcase).
Author: Dongjoon Hyun <dongjoon@apache.org>
Closes #13449 from dongjoon-hyun/SPARK-15709.
Diffstat (limited to 'repl')
0 files changed, 0 insertions, 0 deletions