diff options
author | Joseph K. Bradley <joseph@databricks.com> | 2014-12-04 00:59:32 -0800 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2014-12-04 00:59:44 -0800 |
commit | bf720ef98f49bcc49b9a3b1a281b2373bf8d739a (patch) | |
tree | 167885cbff8ae7c81855edb3f926e968bb1e7d3c /docs/tuning.md | |
parent | dec838bcbd6e3ba5844173036f5caae3e67eb490 (diff) | |
download | spark-bf720ef98f49bcc49b9a3b1a281b2373bf8d739a.tar.gz spark-bf720ef98f49bcc49b9a3b1a281b2373bf8d739a.tar.bz2 spark-bf720ef98f49bcc49b9a3b1a281b2373bf8d739a.zip |
[docs] Fix outdated comment in tuning guide
When you use the SPARK_JAVA_OPTS env variable, Spark complains:
```
SPARK_JAVA_OPTS was detected (set to ' -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps ').
This is deprecated in Spark 1.0+.
Please instead use:
- ./spark-submit with conf/spark-defaults.conf to set defaults for an application
- ./spark-submit with --driver-java-options to set -X options for a driver
- spark.executor.extraJavaOptions to set -X options for executors
- SPARK_DAEMON_JAVA_OPTS to set java options for standalone daemons (master or worker)
```
This updates the docs to redirect the user to the relevant part of the configuration docs.
CC: mengxr but please CC someone else as needed
Author: Joseph K. Bradley <joseph@databricks.com>
Closes #3592 from jkbradley/tuning-doc and squashes the following commits:
0760ce1 [Joseph K. Bradley] fixed outdated comment in tuning guide
(cherry picked from commit 529439bd506949f272a2b6f099ea549b097428f3)
Signed-off-by: Reynold Xin <rxin@databricks.com>
Diffstat (limited to 'docs/tuning.md')
-rw-r--r-- | docs/tuning.md | 3 |
1 files changed, 1 insertions, 2 deletions
diff --git a/docs/tuning.md b/docs/tuning.md index 9b5c9adac6..0e2447dd46 100644 --- a/docs/tuning.md +++ b/docs/tuning.md @@ -143,8 +143,7 @@ the space allocated to the RDD cache to mitigate this. **Measuring the Impact of GC** The first step in GC tuning is to collect statistics on how frequently garbage collection occurs and the amount of -time spent GC. This can be done by adding `-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps` to your -`SPARK_JAVA_OPTS` environment variable. Next time your Spark job is run, you will see messages printed in the worker's logs +time spent GC. This can be done by adding `-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps` to the Java options. (See the [configuration guide](configuration.html#Dynamically-Loading-Spark-Properties) for info on passing Java options to Spark jobs.) Next time your Spark job is run, you will see messages printed in the worker's logs each time a garbage collection occurs. Note these logs will be on your cluster's worker nodes (in the `stdout` files in their work directories), *not* on your driver program. |