diff options
author | Josh Rosen <joshrosen@apache.org> | 2014-01-25 16:39:20 -0800 |
---|---|---|
committer | Josh Rosen <joshrosen@apache.org> | 2014-01-25 16:41:12 -0800 |
commit | 740e865f40704dc9158a6cf635990580fb6adcac (patch) | |
tree | e6343f303a9f79b1fca0b29900b233d9d5c61f07 /sbt | |
parent | 531d9d75762bf04ebb5580f42ce24b178d721686 (diff) | |
download | spark-740e865f40704dc9158a6cf635990580fb6adcac.tar.gz spark-740e865f40704dc9158a6cf635990580fb6adcac.tar.bz2 spark-740e865f40704dc9158a6cf635990580fb6adcac.zip |
Fix ClassCastException in JavaPairRDD.collectAsMap() (SPARK-1040)
This fixes an issue where collectAsMap() could
fail when called on a JavaPairRDD that was derived
by transforming a non-JavaPairRDD.
The root problem was that we were creating the
JavaPairRDD's ClassTag by casting a
ClassTag[AnyRef] to a ClassTag[Tuple2[K2, V2]].
To fix this, I cast a ClassTag[Tuple2[_, _]]
instead, since this actually produces a ClassTag
of the appropriate type because ClassTags don't
capture type parameters:
scala> implicitly[ClassTag[Tuple2[_, _]]] == implicitly[ClassTag[Tuple2[Int, Int]]]
res8: Boolean = true
scala> implicitly[ClassTag[AnyRef]].asInstanceOf[ClassTag[Tuple2[Int, Int]]] == implicitly[ClassTag[Tuple2[Int, Int]]]
res9: Boolean = false
Diffstat (limited to 'sbt')
0 files changed, 0 insertions, 0 deletions