diff options
author | Iulian Dragos <jaguarul@gmail.com> | 2015-04-28 12:08:18 -0700 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-04-28 12:08:18 -0700 |
commit | 8aab94d8984e9d12194dbda47b2e7d9dbc036889 (patch) | |
tree | 6ae73a4d711bb4a44db690390ab7626ad2c40f61 /sbin/stop-shuffle-service.sh | |
parent | 52ccf1d3739694826915cdf01642bab02958eb78 (diff) | |
download | spark-8aab94d8984e9d12194dbda47b2e7d9dbc036889.tar.gz spark-8aab94d8984e9d12194dbda47b2e7d9dbc036889.tar.bz2 spark-8aab94d8984e9d12194dbda47b2e7d9dbc036889.zip |
[SPARK-4286] Add an external shuffle service that can be run as a daemon.
This allows Mesos deployments to use the shuffle service (and implicitly dynamic allocation). It does so by adding a new "main" class and two corresponding scripts in `sbin`:
- `sbin/start-shuffle-service.sh`
- `sbin/stop-shuffle-service.sh`
Specific options can be passed in `SPARK_SHUFFLE_OPTS`.
This is picking up work from #3861 /cc tnachen
Author: Iulian Dragos <jaguarul@gmail.com>
Closes #4990 from dragos/feature/external-shuffle-service and squashes the following commits:
6c2b148 [Iulian Dragos] Import order and wrong name fixup.
07804ad [Iulian Dragos] Moved ExternalShuffleService to the `deploy` package + other minor tweaks.
4dc1f91 [Iulian Dragos] Reviewer’s comments:
8145429 [Iulian Dragos] Add an external shuffle service that can be run as a daemon.
Diffstat (limited to 'sbin/stop-shuffle-service.sh')
-rwxr-xr-x | sbin/stop-shuffle-service.sh | 25 |
1 files changed, 25 insertions, 0 deletions
diff --git a/sbin/stop-shuffle-service.sh b/sbin/stop-shuffle-service.sh new file mode 100755 index 0000000000..4cb6891ae2 --- /dev/null +++ b/sbin/stop-shuffle-service.sh @@ -0,0 +1,25 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# Stops the external shuffle service on the machine this script is executed on. + +sbin="`dirname "$0"`" +sbin="`cd "$sbin"; pwd`" + +"$sbin"/spark-daemon.sh stop org.apache.spark.deploy.ExternalShuffleService 1 |