diff options
| author | Mark Grover <mark@apache.org> | 2016-04-14 18:51:43 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-04-14 18:51:43 -0700 |
| commit | ff9ae61a3b7bbbfc2aac93a99c05a9e1ea9c08bc (patch) | |
| tree | b2ea17d703d24540a69f40c66f8bfc35c6a2cc37 /docs | |
| parent | c80586d9e820d19fc328b3e4c6f1c1439f5583a7 (diff) | |
| download | spark-ff9ae61a3b7bbbfc2aac93a99c05a9e1ea9c08bc.tar.gz spark-ff9ae61a3b7bbbfc2aac93a99c05a9e1ea9c08bc.tar.bz2 spark-ff9ae61a3b7bbbfc2aac93a99c05a9e1ea9c08bc.zip | |
[SPARK-14601][DOC] Minor doc/usage changes related to removal of Spark assembly
## What changes were proposed in this pull request?
Removing references to assembly jar in documentation.
Adding an additional (previously undocumented) usage of spark-submit to run examples.
## How was this patch tested?
Ran spark-submit usage to ensure formatting was fine. Ran examples using SparkSubmit.
Author: Mark Grover <mark@apache.org>
Closes #12365 from markgrover/spark-14601.
Diffstat (limited to 'docs')
| -rw-r--r-- | docs/building-spark.md | 2 | ||||
| -rw-r--r-- | docs/sql-programming-guide.md | 4 |
2 files changed, 3 insertions, 3 deletions
diff --git a/docs/building-spark.md b/docs/building-spark.md index 40661604af..fec442af95 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -192,7 +192,7 @@ If you have JDK 8 installed but it is not the system default, you can set JAVA_H # Packaging without Hadoop Dependencies for YARN -The assembly jar produced by `mvn package` will, by default, include all of Spark's dependencies, including Hadoop and some of its ecosystem projects. On YARN deployments, this causes multiple versions of these to appear on executor classpaths: the version packaged in the Spark assembly and the version on each node, included with `yarn.application.classpath`. The `hadoop-provided` profile builds the assembly without including Hadoop-ecosystem projects, like ZooKeeper and Hadoop itself. +The assembly directory produced by `mvn package` will, by default, include all of Spark's dependencies, including Hadoop and some of its ecosystem projects. On YARN deployments, this causes multiple versions of these to appear on executor classpaths: the version packaged in the Spark assembly and the version on each node, included with `yarn.application.classpath`. The `hadoop-provided` profile builds the assembly without including Hadoop-ecosystem projects, like ZooKeeper and Hadoop itself. # Building with SBT diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 2d9849d032..77887f4ca3 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -1651,7 +1651,7 @@ SELECT * FROM jsonTable Spark SQL also supports reading and writing data stored in [Apache Hive](http://hive.apache.org/). However, since Hive has a large number of dependencies, it is not included in the default Spark assembly. Hive support is enabled by adding the `-Phive` and `-Phive-thriftserver` flags to Spark's build. -This command builds a new assembly jar that includes Hive. Note that this Hive assembly jar must also be present +This command builds a new assembly directory that includes Hive. Note that this Hive assembly directory must also be present on all of the worker nodes, as they will need access to the Hive serialization and deserialization libraries (SerDes) in order to access data stored in Hive. @@ -1770,7 +1770,7 @@ The following options can be used to configure the version of Hive that is used property can be one of three options: <ol> <li><code>builtin</code></li> - Use Hive 1.2.1, which is bundled with the Spark assembly jar when <code>-Phive</code> is + Use Hive 1.2.1, which is bundled with the Spark assembly when <code>-Phive</code> is enabled. When this option is chosen, <code>spark.sql.hive.metastore.version</code> must be either <code>1.2.1</code> or not defined. <li><code>maven</code></li> |