diff options
| author | Joseph K. Bradley <joseph@databricks.com> | 2014-12-04 00:59:32 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2014-12-04 00:59:32 -0800 |
| commit | 529439bd506949f272a2b6f099ea549b097428f3 (patch) | |
| tree | 14561629a4afca0b4e6e17f41a28d7f4c547775d /docs/tuning.md | |
| parent | c6c7165e7ecf1690027d6bd4e0620012cd0d2310 (diff) | |
| download | spark-529439bd506949f272a2b6f099ea549b097428f3.tar.gz spark-529439bd506949f272a2b6f099ea549b097428f3.tar.bz2 spark-529439bd506949f272a2b6f099ea549b097428f3.zip | |
[docs] Fix outdated comment in tuning guide
When you use the SPARK_JAVA_OPTS env variable, Spark complains:
```
SPARK_JAVA_OPTS was detected (set to ' -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps ').
This is deprecated in Spark 1.0+.
Please instead use:
- ./spark-submit with conf/spark-defaults.conf to set defaults for an application
- ./spark-submit with --driver-java-options to set -X options for a driver
- spark.executor.extraJavaOptions to set -X options for executors
- SPARK_DAEMON_JAVA_OPTS to set java options for standalone daemons (master or worker)
```
This updates the docs to redirect the user to the relevant part of the configuration docs.
CC: mengxr but please CC someone else as needed
Author: Joseph K. Bradley <joseph@databricks.com>
Closes #3592 from jkbradley/tuning-doc and squashes the following commits:
0760ce1 [Joseph K. Bradley] fixed outdated comment in tuning guide
Diffstat (limited to 'docs/tuning.md')
| -rw-r--r-- | docs/tuning.md | 3 |
1 files changed, 1 insertions, 2 deletions
diff --git a/docs/tuning.md b/docs/tuning.md index 9b5c9adac6..0e2447dd46 100644 --- a/docs/tuning.md +++ b/docs/tuning.md @@ -143,8 +143,7 @@ the space allocated to the RDD cache to mitigate this. **Measuring the Impact of GC** The first step in GC tuning is to collect statistics on how frequently garbage collection occurs and the amount of -time spent GC. This can be done by adding `-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps` to your -`SPARK_JAVA_OPTS` environment variable. Next time your Spark job is run, you will see messages printed in the worker's logs +time spent GC. This can be done by adding `-verbose:gc -XX:+PrintGCDetails -XX:+PrintGCTimeStamps` to the Java options. (See the [configuration guide](configuration.html#Dynamically-Loading-Spark-Properties) for info on passing Java options to Spark jobs.) Next time your Spark job is run, you will see messages printed in the worker's logs each time a garbage collection occurs. Note these logs will be on your cluster's worker nodes (in the `stdout` files in their work directories), *not* on your driver program. |