diff options
| author | Liang-Chi Hsieh <viirya@appier.com> | 2015-11-02 08:52:52 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-11-02 08:52:52 +0000 |
| commit | e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c (patch) | |
| tree | b71bcd0d0c04c71fff145b9a5bcce5963783322b /pom.xml | |
| parent | e963070c13f56fbc2dfaf9f5d4e69d34afd0957c (diff) | |
| download | spark-e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c.tar.gz spark-e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c.tar.bz2 spark-e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c.zip | |
[SPARK-11271][SPARK-11016][CORE] Use Spark BitSet instead of RoaringBitmap to reduce memory usage
JIRA: https://issues.apache.org/jira/browse/SPARK-11271
As reported in the JIRA ticket, when there are too many tasks, the memory usage of MapStatus will cause problem. Use BitSet instead of RoaringBitMap should be more efficient in memory usage.
Author: Liang-Chi Hsieh <viirya@appier.com>
Closes #9243 from viirya/mapstatus-bitset.
Diffstat (limited to 'pom.xml')
| -rw-r--r-- | pom.xml | 5 |
1 files changed, 0 insertions, 5 deletions
@@ -624,11 +624,6 @@ </exclusions> </dependency> <dependency> - <groupId>org.roaringbitmap</groupId> - <artifactId>RoaringBitmap</artifactId> - <version>0.4.5</version> - </dependency> - <dependency> <groupId>commons-net</groupId> <artifactId>commons-net</artifactId> <version>2.2</version> |