diff options
author | Min Zhou <coderplay@gmail.com> | 2015-07-10 09:52:40 -0700 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-07-10 09:52:40 -0700 |
commit | c185f3a45ddbc073192f7da41303941ee4cebd4f (patch) | |
tree | 27e289012a1d800348f98c08e834cec6f249a78b /sql | |
parent | db6d57f87a9d657d764e01879014bf406971855a (diff) | |
download | spark-c185f3a45ddbc073192f7da41303941ee4cebd4f.tar.gz spark-c185f3a45ddbc073192f7da41303941ee4cebd4f.tar.bz2 spark-c185f3a45ddbc073192f7da41303941ee4cebd4f.zip |
[SPARK-8675] Executors created by LocalBackend won't get the same classpath as other executor backends
AFAIK, some spark application always use LocalBackend to do some local initiatives, spark sql is an example. Starting a LocalPoint won't add user classpath into executor.
```java
override def start() {
localEndpoint = SparkEnv.get.rpcEnv.setupEndpoint(
"LocalBackendEndpoint", new LocalEndpoint(SparkEnv.get.rpcEnv, scheduler, this, totalCores))
}
```
Thus will cause local executor fail with these scenarios, loading hadoop built-in native libraries, loading other user defined native libraries, loading user jars, reading s3 config from a site.xml file, etc
Author: Min Zhou <coderplay@gmail.com>
Closes #7091 from coderplay/master and squashes the following commits:
365838f [Min Zhou] Fixed java.net.MalformedURLException, add default scheme, support relative path
d215b7f [Min Zhou] Follows spark standard scala style, make the auto testing happy
84ad2cd [Min Zhou] Use system specific path separator instead of ','
01f5d1a [Min Zhou] Merge branch 'master' of https://github.com/apache/spark
e528be7 [Min Zhou] Merge branch 'master' of https://github.com/apache/spark
45bf62c [Min Zhou] SPARK-8675 Executors created by LocalBackend won't get the same classpath as other executor backends
Diffstat (limited to 'sql')
0 files changed, 0 insertions, 0 deletions