diff options
| author | James Lohse <jimlohse@users.noreply.github.com> | 2016-01-28 10:50:50 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2016-01-28 10:50:50 +0000 |
| commit | c2204436a15838f2dce44e3cfb0fe58236ef6196 (patch) | |
| tree | e130663a48ccc9b0d2b77f3312b47607407fb532 /docs/submitting-applications.md | |
| parent | 4a091232122b51f10521a68de8b1d9eb853b563d (diff) | |
| download | spark-c2204436a15838f2dce44e3cfb0fe58236ef6196.tar.gz spark-c2204436a15838f2dce44e3cfb0fe58236ef6196.tar.bz2 spark-c2204436a15838f2dce44e3cfb0fe58236ef6196.zip | |
Provide same info as in spark-submit --help
this is stated for --packages and --repositories. Without stating it for --jars, people expect a standard java classpath to work, with expansion and using a different delimiter than a comma. Currently this is only state in the --help for spark-submit "Comma-separated list of local jars to include on the driver and executor classpaths."
Author: James Lohse <jimlohse@users.noreply.github.com>
Closes #10890 from jimlohse/patch-1.
Diffstat (limited to 'docs/submitting-applications.md')
| -rw-r--r-- | docs/submitting-applications.md | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/docs/submitting-applications.md b/docs/submitting-applications.md index acbb0f298f..413532f2f6 100644 --- a/docs/submitting-applications.md +++ b/docs/submitting-applications.md @@ -177,8 +177,9 @@ debugging information by running `spark-submit` with the `--verbose` option. # Advanced Dependency Management When using `spark-submit`, the application jar along with any jars included with the `--jars` option -will be automatically transferred to the cluster. Spark uses the following URL scheme to allow -different strategies for disseminating jars: +will be automatically transferred to the cluster. URLs supplied after `--jars` must be separated by commas. That list is included on the driver and executor classpaths. Directory expansion does not work with `--jars`. + +Spark uses the following URL scheme to allow different strategies for disseminating jars: - **file:** - Absolute paths and `file:/` URIs are served by the driver's HTTP file server, and every executor pulls the file from the driver HTTP server. |