diff options
author | Liang-Chi Hsieh <viirya@appier.com> | 2015-11-02 08:52:52 +0000 |
---|---|---|
committer | Sean Owen <sowen@cloudera.com> | 2015-11-02 08:52:52 +0000 |
commit | e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c (patch) | |
tree | b71bcd0d0c04c71fff145b9a5bcce5963783322b /core/pom.xml | |
parent | e963070c13f56fbc2dfaf9f5d4e69d34afd0957c (diff) | |
download | spark-e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c.tar.gz spark-e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c.tar.bz2 spark-e209fa271ae57dc8849f8b1241bf1ea7d6d3d62c.zip |
[SPARK-11271][SPARK-11016][CORE] Use Spark BitSet instead of RoaringBitmap to reduce memory usage
JIRA: https://issues.apache.org/jira/browse/SPARK-11271
As reported in the JIRA ticket, when there are too many tasks, the memory usage of MapStatus will cause problem. Use BitSet instead of RoaringBitMap should be more efficient in memory usage.
Author: Liang-Chi Hsieh <viirya@appier.com>
Closes #9243 from viirya/mapstatus-bitset.
Diffstat (limited to 'core/pom.xml')
-rw-r--r-- | core/pom.xml | 4 |
1 files changed, 0 insertions, 4 deletions
diff --git a/core/pom.xml b/core/pom.xml index 319a50049a..1b6b13517b 100644 --- a/core/pom.xml +++ b/core/pom.xml @@ -174,10 +174,6 @@ <artifactId>lz4</artifactId> </dependency> <dependency> - <groupId>org.roaringbitmap</groupId> - <artifactId>RoaringBitmap</artifactId> - </dependency> - <dependency> <groupId>commons-net</groupId> <artifactId>commons-net</artifactId> </dependency> |