aboutsummaryrefslogtreecommitdiff
path: root/graphx
diff options
context:
space:
mode:
authorKevin (Sangwoo) Kim <sangwookim.me@gmail.com>2015-03-16 23:49:23 -0700
committerReynold Xin <rxin@databricks.com>2015-03-16 23:49:23 -0700
commitf0edeae7f9ab7eae02c227be9162ec69d22c92bd (patch)
tree74b077a59f8d088bf6dde6527349daf9bf5774ad /graphx
parent9667b9f9c3239f814a0b1120355d9e7bd7a89158 (diff)
downloadspark-f0edeae7f9ab7eae02c227be9162ec69d22c92bd.tar.gz
spark-f0edeae7f9ab7eae02c227be9162ec69d22c92bd.tar.bz2
spark-f0edeae7f9ab7eae02c227be9162ec69d22c92bd.zip
[SPARK-6299][CORE] ClassNotFoundException in standalone mode when running groupByKey with class defined in REPL
``` case class ClassA(value: String) val rdd = sc.parallelize(List(("k1", ClassA("v1")), ("k1", ClassA("v2")) )) rdd.groupByKey.collect ``` This code used to be throw exception in spark-shell, because while shuffling ```JavaSerializer```uses ```defaultClassLoader``` which was defined like ```env.serializer.setDefaultClassLoader(urlClassLoader)```. It should be ```env.serializer.setDefaultClassLoader(replClassLoader)```, like ``` override def run() { val deserializeStartTime = System.currentTimeMillis() Thread.currentThread.setContextClassLoader(replClassLoader) ``` in TaskRunner. When ```replClassLoader``` cannot be defined, it's identical with ```urlClassLoader``` Author: Kevin (Sangwoo) Kim <sangwookim.me@gmail.com> Closes #5046 from swkimme/master and squashes the following commits: fa2b9ee [Kevin (Sangwoo) Kim] stylish test codes ( collect -> collect() ) 6e9620b [Kevin (Sangwoo) Kim] stylish test codes ( collect -> collect() ) d23e4e2 [Kevin (Sangwoo) Kim] stylish test codes ( collect -> collect() ) a4a3c8a [Kevin (Sangwoo) Kim] add 'class defined in repl - shuffle' test to ReplSuite bd00da5 [Kevin (Sangwoo) Kim] add 'class defined in repl - shuffle' test to ReplSuite c1b1fc7 [Kevin (Sangwoo) Kim] use REPL class loader for executor's serializer
Diffstat (limited to 'graphx')
0 files changed, 0 insertions, 0 deletions