diff options
| author | Matei Zaharia <matei@eecs.berkeley.edu> | 2012-09-26 19:17:58 -0700 |
|---|---|---|
| committer | Matei Zaharia <matei@eecs.berkeley.edu> | 2012-09-26 19:17:58 -0700 |
| commit | 874a9fd407943c7102395cfc64762dfd0ecf9b00 (patch) | |
| tree | 35ccd0bb96acfb17275b4a12c14ab611e91c1376 /docs/ec2-scripts.md | |
| parent | ee71fa49c1d958f1723fd3fba59b8e247cb4af76 (diff) | |
| download | spark-874a9fd407943c7102395cfc64762dfd0ecf9b00.tar.gz spark-874a9fd407943c7102395cfc64762dfd0ecf9b00.tar.bz2 spark-874a9fd407943c7102395cfc64762dfd0ecf9b00.zip | |
More updates to docs, including tuning guide
Diffstat (limited to 'docs/ec2-scripts.md')
| -rw-r--r-- | docs/ec2-scripts.md | 10 |
1 files changed, 10 insertions, 0 deletions
diff --git a/docs/ec2-scripts.md b/docs/ec2-scripts.md index a1cc7985b4..faa89b5d56 100644 --- a/docs/ec2-scripts.md +++ b/docs/ec2-scripts.md @@ -99,6 +99,16 @@ permissions on your private key file, you can run `launch` with the - Finally, if you get errors while running your jobs, look at the slave's logs for that job using the Mesos web UI (`http://<master-hostname>:8080`). +# Configuration + +You can edit `/root/spark/conf/spark-env.sh` on each machine to set Spark configuration options, such +as JVM options and, most crucially, the amount of memory to use per machine (`SPARK_MEM`). +This file needs to be copied to **every machine** to reflect the change. The easiest way to do this +is to use a script we provide called `copy-dir`. First edit your `spark-env.sh` file on the master, +then run `~/mesos-ec2/copy-dir /root/spark/conf` to RSYNC it to all the workers. + +The [configuration guide]({{HOME_PATH}}configuration.html) describes the available configuration options. + # Terminating a Cluster ***Note that there is no way to recover data on EC2 nodes after shutting |