aboutsummaryrefslogtreecommitdiff
path: root/python
diff options
context:
space:
mode:
authorJosh Rosen <joshrosen@apache.org>2014-01-25 16:39:20 -0800
committerJosh Rosen <joshrosen@apache.org>2014-01-25 16:41:12 -0800
commit740e865f40704dc9158a6cf635990580fb6adcac (patch)
treee6343f303a9f79b1fca0b29900b233d9d5c61f07 /python
parent531d9d75762bf04ebb5580f42ce24b178d721686 (diff)
downloadspark-740e865f40704dc9158a6cf635990580fb6adcac.tar.gz
spark-740e865f40704dc9158a6cf635990580fb6adcac.tar.bz2
spark-740e865f40704dc9158a6cf635990580fb6adcac.zip
Fix ClassCastException in JavaPairRDD.collectAsMap() (SPARK-1040)
This fixes an issue where collectAsMap() could fail when called on a JavaPairRDD that was derived by transforming a non-JavaPairRDD. The root problem was that we were creating the JavaPairRDD's ClassTag by casting a ClassTag[AnyRef] to a ClassTag[Tuple2[K2, V2]]. To fix this, I cast a ClassTag[Tuple2[_, _]] instead, since this actually produces a ClassTag of the appropriate type because ClassTags don't capture type parameters: scala> implicitly[ClassTag[Tuple2[_, _]]] == implicitly[ClassTag[Tuple2[Int, Int]]] res8: Boolean = true scala> implicitly[ClassTag[AnyRef]].asInstanceOf[ClassTag[Tuple2[Int, Int]]] == implicitly[ClassTag[Tuple2[Int, Int]]] res9: Boolean = false
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions