aboutsummaryrefslogtreecommitdiff
path: root/project
diff options
context:
space:
mode:
authorAndrew Or <andrewor14@gmail.com>2014-07-29 23:52:09 -0700
committerPatrick Wendell <pwendell@gmail.com>2014-07-29 23:52:09 -0700
commit4ce92ccaf761e48a10fc4fe4927dbfca858ca22b (patch)
treeaf298364e8b7cff17d1fa93c7f60b4883b0fa88c /project
parent077f633b4720422c5efbf0382e869ead3dc49612 (diff)
downloadspark-4ce92ccaf761e48a10fc4fe4927dbfca858ca22b.tar.gz
spark-4ce92ccaf761e48a10fc4fe4927dbfca858ca22b.tar.bz2
spark-4ce92ccaf761e48a10fc4fe4927dbfca858ca22b.zip
[SPARK-2260] Fix standalone-cluster mode, which was broken
The main thing was that spark configs were not propagated to the driver, and so applications that do not specify `master` or `appName` automatically failed. This PR fixes that and a couple of miscellaneous things that are related. One thing that may or may not be an issue is that the jars must be available on the driver node. In `standalone-cluster` mode, this effectively means these jars must be available on all the worker machines, since the driver is launched on one of them. The semantics here are not the same as `yarn-cluster` mode, where all the relevant jars are uploaded to a distributed cache automatically and shipped to the containers. This is probably not a concern, but still worth a mention. Author: Andrew Or <andrewor14@gmail.com> Closes #1538 from andrewor14/standalone-cluster and squashes the following commits: 8c11a0d [Andrew Or] Clean up imports / comments (minor) 2678d13 [Andrew Or] Handle extraJavaOpts properly 7660547 [Andrew Or] Merge branch 'master' of github.com:apache/spark into standalone-cluster 6f64a9b [Andrew Or] Revert changes in YARN 2f2908b [Andrew Or] Fix tests ed01491 [Andrew Or] Don't go overboard with escaping 8e105e1 [Andrew Or] Merge branch 'master' of github.com:apache/spark into standalone-cluster b890949 [Andrew Or] Abstract usages of converting spark opts to java opts 79f63a3 [Andrew Or] Move sparkProps into javaOpts 78752f8 [Andrew Or] Fix tests 5a9c6c7 [Andrew Or] Fix line too long c141a00 [Andrew Or] Don't display "unknown app" on driver log pages d7e2728 [Andrew Or] Avoid deprecation warning in standalone Client 6ceb14f [Andrew Or] Allow relevant configs to propagate to standalone Driver 7f854bc [Andrew Or] Fix test 855256e [Andrew Or] Fix standalone-cluster mode fd9da51 [Andrew Or] Formatting changes (minor)
Diffstat (limited to 'project')
0 files changed, 0 insertions, 0 deletions