diff options
| author | Huaxin Gao <huaxing@us.ibm.com> | 2016-12-10 22:41:40 +0800 |
|---|---|---|
| committer | Wenchen Fan <wenchen@databricks.com> | 2016-12-10 22:41:40 +0800 |
| commit | c5172568b59b4cf1d3dc7ed8c17a9bea2ea2ab79 (patch) | |
| tree | be3605d08b48437dc41b3524ccd9bc351d895c2f /README.md | |
| parent | 63c9159870ee274c68e24360594ca01d476b9ace (diff) | |
| download | spark-c5172568b59b4cf1d3dc7ed8c17a9bea2ea2ab79.tar.gz spark-c5172568b59b4cf1d3dc7ed8c17a9bea2ea2ab79.tar.bz2 spark-c5172568b59b4cf1d3dc7ed8c17a9bea2ea2ab79.zip | |
[SPARK-17460][SQL] Make sure sizeInBytes in Statistics will not overflow
## What changes were proposed in this pull request?
1. In SparkStrategies.canBroadcast, I will add the check plan.statistics.sizeInBytes >= 0
2. In LocalRelations.statistics, when calculate the statistics, I will change the size to BigInt so it won't overflow.
## How was this patch tested?
I will add a test case to make sure the statistics.sizeInBytes won't overflow.
Author: Huaxin Gao <huaxing@us.ibm.com>
Closes #16175 from huaxingao/spark-17460.
Diffstat (limited to 'README.md')
0 files changed, 0 insertions, 0 deletions