diff options
| author | Michael Gummelt <mgummelt@mesosphere.io> | 2016-02-10 10:53:33 -0800 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2016-02-10 10:53:33 -0800 |
| commit | 80cb963ad963e26c3a7f8388bdd4ffd5e99aad1a (patch) | |
| tree | ebd19083ee53f66618a81280c6b6667db129b179 /sql/hive | |
| parent | c0b71e0b8f3c068f2f092bb118a16611b3d38d7a (diff) | |
| download | spark-80cb963ad963e26c3a7f8388bdd4ffd5e99aad1a.tar.gz spark-80cb963ad963e26c3a7f8388bdd4ffd5e99aad1a.tar.bz2 spark-80cb963ad963e26c3a7f8388bdd4ffd5e99aad1a.zip | |
[SPARK-5095][MESOS] Support launching multiple mesos executors in coarse grained mesos mode.
This is the next iteration of tnachen's previous PR: https://github.com/apache/spark/pull/4027
In that PR, we resolved with andrewor14 and pwendell to implement the Mesos scheduler's support of `spark.executor.cores` to be consistent with YARN and Standalone. This PR implements that resolution.
This PR implements two high-level features. These two features are co-dependent, so they're implemented both here:
- Mesos support for spark.executor.cores
- Multiple executors per slave
We at Mesosphere have been working with Typesafe on a Spark/Mesos integration test suite: https://github.com/typesafehub/mesos-spark-integration-tests, which passes for this PR.
The contribution is my original work and I license the work to the project under the project's open source license.
Author: Michael Gummelt <mgummelt@mesosphere.io>
Closes #10993 from mgummelt/executor_sizing.
Diffstat (limited to 'sql/hive')
0 files changed, 0 insertions, 0 deletions