diff options
| author | Bharath Bhushan <manku.timma@outlook.com> | 2014-04-12 20:52:29 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-12 20:53:44 -0700 |
| commit | ca11919e6e97a62eb3e3ce882ffa29eae36f50f7 (patch) | |
| tree | ae0b14fbfe14ff6d676086014516c4911c7d6842 /repl | |
| parent | c2d160fbee2ef90a7683d9771f2f632b68d74aef (diff) | |
| download | spark-ca11919e6e97a62eb3e3ce882ffa29eae36f50f7.tar.gz spark-ca11919e6e97a62eb3e3ce882ffa29eae36f50f7.tar.bz2 spark-ca11919e6e97a62eb3e3ce882ffa29eae36f50f7.zip | |
[SPARK-1403] Move the class loader creation back to where it was in 0.9.0
[SPARK-1403] I investigated why spark 0.9.0 loads fine on mesos while spark 1.0.0 fails. What I found was that in SparkEnv.scala, while creating the SparkEnv object, the current thread's classloader is null. But in 0.9.0, at the same place, it is set to org.apache.spark.repl.ExecutorClassLoader . I saw that https://github.com/apache/spark/commit/7edbea41b43e0dc11a2de156be220db8b7952d01 moved it to it current place. I moved it back and saw that 1.0.0 started working fine on mesos.
I just created a minimal patch that allows me to run spark on mesos correctly. It seems like SecurityManager's creation needs to be taken into account for a correct fix. Also moving the creation of the serializer out of SparkEnv might be a part of the right solution. PTAL.
Author: Bharath Bhushan <manku.timma@outlook.com>
Closes #322 from manku-timma/spark-1403 and squashes the following commits:
606c2b9 [Bharath Bhushan] Merge remote-tracking branch 'upstream/master' into spark-1403
ec8f870 [Bharath Bhushan] revert the logger change for java 6 compatibility as PR 334 is doing it
728beca [Bharath Bhushan] Merge remote-tracking branch 'upstream/master' into spark-1403
044027d [Bharath Bhushan] fix compile error
6f260a4 [Bharath Bhushan] Merge remote-tracking branch 'upstream/master' into spark-1403
b3a053f [Bharath Bhushan] Merge remote-tracking branch 'upstream/master' into spark-1403
04b9662 [Bharath Bhushan] add missing line
4803c19 [Bharath Bhushan] Merge remote-tracking branch 'upstream/master' into spark-1403
f3c9a14 [Bharath Bhushan] Merge remote-tracking branch 'upstream/master' into spark-1403
42d3d6a [Bharath Bhushan] used code fragment from @ueshin to fix the problem in a better way
89109d7 [Bharath Bhushan] move the class loader creation back to where it was in 0.9.0
Diffstat (limited to 'repl')
0 files changed, 0 insertions, 0 deletions