diff options
author | zsxwing <zsxwing@gmail.com> | 2014-12-01 00:32:54 -0800 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2014-12-01 00:32:54 -0800 |
commit | 1d238f221c3e13c525b3af0c78eda95059ce9fc6 (patch) | |
tree | 316905374e8dcd4819fa78c07c5a43346a2bbcc0 /core/src | |
parent | 6384f42ab2e5c2b3e767ab4a428cda20a8ddcbe1 (diff) | |
download | spark-1d238f221c3e13c525b3af0c78eda95059ce9fc6.tar.gz spark-1d238f221c3e13c525b3af0c78eda95059ce9fc6.tar.bz2 spark-1d238f221c3e13c525b3af0c78eda95059ce9fc6.zip |
[SPARK-4664][Core] Throw an exception when spark.akka.frameSize > 2047
If `spark.akka.frameSize` > 2047, it will overflow and become negative. Should have some assertion in `maxFrameSizeBytes` to warn people.
Author: zsxwing <zsxwing@gmail.com>
Closes #3527 from zsxwing/SPARK-4664 and squashes the following commits:
0089c7a [zsxwing] Throw an exception when spark.akka.frameSize > 2047
Diffstat (limited to 'core/src')
-rw-r--r-- | core/src/main/scala/org/apache/spark/util/AkkaUtils.scala | 9 |
1 files changed, 8 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/util/AkkaUtils.scala b/core/src/main/scala/org/apache/spark/util/AkkaUtils.scala index 10010bdfa1..8c2457f56b 100644 --- a/core/src/main/scala/org/apache/spark/util/AkkaUtils.scala +++ b/core/src/main/scala/org/apache/spark/util/AkkaUtils.scala @@ -134,9 +134,16 @@ private[spark] object AkkaUtils extends Logging { Duration.create(conf.getLong("spark.akka.lookupTimeout", 30), "seconds") } + private val AKKA_MAX_FRAME_SIZE_IN_MB = Int.MaxValue / 1024 / 1024 + /** Returns the configured max frame size for Akka messages in bytes. */ def maxFrameSizeBytes(conf: SparkConf): Int = { - conf.getInt("spark.akka.frameSize", 10) * 1024 * 1024 + val frameSizeInMB = conf.getInt("spark.akka.frameSize", 10) + if (frameSizeInMB > AKKA_MAX_FRAME_SIZE_IN_MB) { + throw new IllegalArgumentException("spark.akka.frameSize should not be greater than " + + AKKA_MAX_FRAME_SIZE_IN_MB + "MB") + } + frameSizeInMB * 1024 * 1024 } /** Space reserved for extra data in an Akka message besides serialized task or task result. */ |