diff options
| author | huangzhaowei <carlmartinmax@gmail.com> | 2016-02-24 23:52:17 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-02-24 23:52:17 -0800 |
| commit | 264533b553be806b6c45457201952e83c028ec78 (patch) | |
| tree | 6d4c4bb595de5d698f08bfa1e4ce734e96eaf7a6 /network | |
| parent | 4d2864b2d7cac027cf6bb78fc3cac9bc37534b07 (diff) | |
| download | spark-264533b553be806b6c45457201952e83c028ec78.tar.gz spark-264533b553be806b6c45457201952e83c028ec78.tar.bz2 spark-264533b553be806b6c45457201952e83c028ec78.zip | |
[SPARK-13482][MINOR][CONFIGURATION] Make consistency of the configuraiton named in TransportConf.
`spark.storage.memoryMapThreshold` has two kind of the value, one is 2*1024*1024 as integer and the other one is '2m' as string.
"2m" is recommanded in document but it will go wrong if the code goes into `TransportConf#memoryMapBytes`.
[Jira](https://issues.apache.org/jira/browse/SPARK-13482)
Author: huangzhaowei <carlmartinmax@gmail.com>
Closes #11360 from SaintBacchus/SPARK-13482.
Diffstat (limited to 'network')
| -rw-r--r-- | network/common/src/main/java/org/apache/spark/network/util/TransportConf.java | 3 |
1 files changed, 2 insertions, 1 deletions
diff --git a/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java b/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java index 115135d44a..9f030da2b3 100644 --- a/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java +++ b/network/common/src/main/java/org/apache/spark/network/util/TransportConf.java @@ -132,7 +132,8 @@ public class TransportConf { * memory mapping has high overhead for blocks close to or below the page size of the OS. */ public int memoryMapBytes() { - return conf.getInt("spark.storage.memoryMapThreshold", 2 * 1024 * 1024); + return Ints.checkedCast(JavaUtils.byteStringAsBytes( + conf.get("spark.storage.memoryMapThreshold", "2m"))); } /** |