From 4b3d1294aeecc0001a7fa48c92796e6075d34540 Mon Sep 17 00:00:00 2001 From: CodingCat Date: Mon, 18 Apr 2016 18:51:23 -0700 Subject: [SPARK-13227] Risky apply() in OpenHashMap https://issues.apache.org/jira/browse/SPARK-13227 It might confuse the future developers when they use OpenHashMap.apply() with a numeric value type. null.asInstance[Int], null.asInstance[Long], null.asInstace[Float] and null.asInstance[Double] will return 0/0.0/0L, which might confuse the developer if the value set contains 0/0.0/0L with an existing key The current patch only adds the comments describing the issue, with the respect to apply the minimum changes to the code base The more direct, yet more aggressive, approach is use Option as the return type andrewor14 JoshRosen any thoughts about how to avoid the potential issue? Author: CodingCat Closes #11107 from CodingCat/SPARK-13227. --- core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala | 3 +++ 1 file changed, 3 insertions(+) (limited to 'core/src/main/scala/org/apache') diff --git a/core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala b/core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala index 22d7a4988b..10ab0b3f89 100644 --- a/core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala +++ b/core/src/main/scala/org/apache/spark/util/collection/OpenHashMap.scala @@ -25,6 +25,9 @@ import scala.reflect.ClassTag * space overhead. * * Under the hood, it uses our OpenHashSet implementation. + * + * NOTE: when using numeric type as the value type, the user of this class should be careful to + * distinguish between the 0/0.0/0L and non-exist value */ private[spark] class OpenHashMap[K : ClassTag, @specialized(Long, Int, Double) V: ClassTag]( -- cgit v1.2.3