diff options
| author | Sean R. Owen <srowen@apache.org> | 2015-05-08 13:59:51 +0000 |
|---|---|---|
| committer | Sean R. Owen <srowen@apache.org> | 2015-05-08 13:59:51 +0000 |
| commit | 090a6d74cea47ee2023e2f15aa207afe83194702 (patch) | |
| tree | d9a77f60ec22facf4ea1740a0a50aee5804169aa | |
| parent | 89f7bb0ead0b91237b156377f12b0216347d5b75 (diff) | |
| download | spark-website-090a6d74cea47ee2023e2f15aa207afe83194702.tar.gz spark-website-090a6d74cea47ee2023e2f15aa207afe83194702.tar.bz2 spark-website-090a6d74cea47ee2023e2f15aa207afe83194702.zip | |
Reapply my past changes, which I had only applied to .html, to .md too, and add the changes from the regenerated .html too
| -rw-r--r-- | community.md | 2 | ||||
| -rw-r--r-- | downloads.md | 6 | ||||
| -rw-r--r-- | examples.md | 24 | ||||
| -rw-r--r-- | faq.md | 4 | ||||
| -rw-r--r-- | index.md | 8 | ||||
| -rw-r--r-- | releases/_posts/2015-03-13-spark-release-1-3-0.md | 2 | ||||
| -rw-r--r-- | site/community.html | 2 | ||||
| -rw-r--r-- | site/downloads.html | 6 | ||||
| -rw-r--r-- | site/examples.html | 24 | ||||
| -rw-r--r-- | site/faq.html | 1 | ||||
| -rw-r--r-- | site/index.html | 8 | ||||
| -rw-r--r-- | site/releases/spark-release-1-3-0.html | 2 | ||||
| -rw-r--r-- | sql/index.md | 2 |
13 files changed, 48 insertions, 43 deletions
diff --git a/community.md b/community.md index f32c249c4..23046902d 100644 --- a/community.md +++ b/community.md @@ -28,6 +28,8 @@ navigation: </li> </ul> +<p>The StackOverflow tag <a href="http://stackoverflow.com/questions/tagged/apache-spark"><code>apache-spark</code></a> is an unofficial but active forum for Spark users' questions and answers.</p> + <a name="events"></a> <h3>Events and Meetups</h3> diff --git a/downloads.md b/downloads.md index 5587d5021..c96ad69f4 100644 --- a/downloads.md +++ b/downloads.md @@ -20,13 +20,13 @@ The latest release of Spark is Spark 1.3.1, released on April 17, 2015 <a href="{{site.url}}releases/spark-release-1-3-1.html">(release notes)</a> <a href="https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=3e8391327ba586eaf54447043bd526d919043a44">(git tag)</a><br/> -1. Chose a Spark release: +1. Choose a Spark release: <select id="sparkVersionSelect" onChange="javascript:onVersionSelect();"></select><br> -2. Chose a package type: +2. Choose a package type: <select id="sparkPackageSelect" onChange="javascript:onPackageSelect();"></select><br> -3. Chose a download type: +3. Choose a download type: <select id="sparkDownloadSelect" onChange="javascript:onDownloadSelect()"></select><br> 4. Download Spark: <span id="spanDownloadLink"></span> diff --git a/examples.md b/examples.md index 3658d1788..18d869353 100644 --- a/examples.md +++ b/examples.md @@ -26,8 +26,8 @@ In this example, we search through the error messages in a log file: <div class="tab-content"> <div class="tab-pane tab-pane-python active"> <div class="code code-tab"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br> - errors = file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + errors = text_file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br /> <span class="comment"># Count all the errors</span><br> errors.<span class="sparkop">count</span>()<br> <span class="comment"># Count errors mentioning MySQL</span><br> @@ -38,8 +38,8 @@ In this example, we search through the error messages in a log file: </div> <div class="tab-pane tab-pane-scala"> <div class="code code-tab"> - <span class="keyword">val</span> file = spark.textFile(<span class="string">"hdfs://..."</span>)<br> - <span class="keyword">val</span> errors = file.<span class="sparkop">filter</span>(<span class="closure">line => line.contains("ERROR")</span>)<br> + <span class="keyword">val</span> textFile = spark.textFile(<span class="string">"hdfs://..."</span>)<br> + <span class="keyword">val</span> errors = textFile.<span class="sparkop">filter</span>(<span class="closure">line => line.contains("ERROR")</span>)<br> <span class="comment">// Count all the errors</span><br> errors.<span class="sparkop">count</span>()<br> <span class="comment">// Count errors mentioning MySQL</span><br> @@ -50,8 +50,8 @@ In this example, we search through the error messages in a log file: </div> <div class="tab-pane tab-pane-java"> <div class="code code-tab"> - JavaRDD<String> file = spark.textFile(<span class="string">"hdfs://..."</span>);<br> - JavaRDD<String> errors = file.<span class="sparkop">filter</span>(<span class="closure">new Function<String, Boolean>() {<br> + JavaRDD<String> textFile = spark.textFile(<span class="string">"hdfs://..."</span>);<br> + JavaRDD<String> errors = textFile.<span class="sparkop">filter</span>(<span class="closure">new Function<String, Boolean>() {<br> public Boolean call(String s) { return s.contains("ERROR"); }<br> }</span>);<br> <span class="comment">// Count all the errors</span><br> @@ -112,8 +112,8 @@ In this example, we search through the error messages in a log file: <div class="tab-content"> <div class="tab-pane tab-pane-python active"> <div class="code code-tab"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br> - counts = file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split(" ")</span>) \<br> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br> + counts = text_file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split(" ")</span>) \<br> .<span class="sparkop">map</span>(<span class="closure">lambda word: (word, 1)</span>) \<br> .<span class="sparkop">reduceByKey</span>(<span class="closure">lambda a, b: a + b</span>)<br> counts.<span class="sparkop">saveAsTextFile</span>(<span class="string">"hdfs://..."</span>) @@ -121,8 +121,8 @@ In this example, we search through the error messages in a log file: </div> <div class="tab-pane tab-pane-scala"> <div class="code code-tab"> - <span class="keyword">val</span> file = spark.textFile(<span class="string">"hdfs://..."</span>)<br> - <span class="keyword">val</span> counts = file.<span class="sparkop">flatMap</span>(<span class="closure">line => line.split(" ")</span>)<br> + <span class="keyword">val</span> textFile = spark.textFile(<span class="string">"hdfs://..."</span>)<br> + <span class="keyword">val</span> counts = textFile.<span class="sparkop">flatMap</span>(<span class="closure">line => line.split(" ")</span>)<br> .<span class="sparkop">map</span>(<span class="closure">word => (word, 1)</span>)<br> .<span class="sparkop">reduceByKey</span>(<span class="closure">_ + _</span>)<br> counts.<span class="sparkop">saveAsTextFile</span>(<span class="string">"hdfs://..."</span>) @@ -130,8 +130,8 @@ In this example, we search through the error messages in a log file: </div> <div class="tab-pane tab-pane-java"> <div class="code code-tab"> - JavaRDD<String> file = spark.textFile(<span class="string">"hdfs://..."</span>);<br> - JavaRDD<String> words = file.<span class="sparkop">flatMap</span>(<span class="closure">new FlatMapFunction<String, String>() {<br> + JavaRDD<String> textFile = spark.textFile(<span class="string">"hdfs://..."</span>);<br> + JavaRDD<String> words = textFile.<span class="sparkop">flatMap</span>(<span class="closure">new FlatMapFunction<String, String>() {<br> public Iterable<String> call(String s) { return Arrays.asList(s.split(" ")); }<br> }</span>);<br> JavaPairRDD<String, Integer> pairs = words.<span class="sparkop">mapToPair</span>(<span class="closure">new PairFunction<String, String, Integer>() {<br> @@ -53,8 +53,8 @@ Spark is a fast and general processing engine compatible with Hadoop data. It ca <p class="answer">Starting in version 0.8, Spark is under the <a href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0 license</a>. Previous versions used the <a href="https://github.com/mesos/spark/blob/branch-0.7/LICENSE">BSD license</a>.</p> <p class="question">How can I contribute to Spark?</p> -<p class="answer">Contact the <a href="{{site.url}}community.html">mailing list</a> or send us a pull request on <a href="https://github.com/apache/spark">GitHub</a> (instructions <a href="https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark">here</a>). We're glad to hear about your experience using Spark and to accept patches.</p> -<p>If you would like to report an issue, post it to the <a href="https://issues.apache.org/jira/browse/SPARK">Spark issue tracker</a>.</p> + +<p class="answer">See the <a href="https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark">Contributing to Spark wiki</a> for more information.</p> <p class="question">Where can I get more help?</p> <p class="answer">Please post on the <a href="http://apache-spark-user-list.1001560.n3.nabble.com">Spark Users</a> mailing list. We'll be glad to help!</p> @@ -53,9 +53,9 @@ navigation: <div class="col-md-5 col-sm-5 col-padded-top col-center"> <div style="text-align: left; display: inline-block;"> <div class="code"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br/> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br/> <br/> - file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split()</span>)<br/> + text_file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split()</span>)<br/> .<span class="sparkop">map</span>(<span class="closure">lambda word: (word, 1)</span>)<br/> .<span class="sparkop">reduceByKey</span>(<span class="closure">lambda a, b: a+b</span>) </div> @@ -63,9 +63,9 @@ navigation: </div> <!-- <div class="code" style="margin-top: 20px; text-align: left; display: inline-block;"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br/> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br/> <br/> - file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br/> + text_file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br/> .<span class="sparkop">count</span>() </div> --> diff --git a/releases/_posts/2015-03-13-spark-release-1-3-0.md b/releases/_posts/2015-03-13-spark-release-1-3-0.md index a7c2a7924..bc9c4db84 100644 --- a/releases/_posts/2015-03-13-spark-release-1-3-0.md +++ b/releases/_posts/2015-03-13-spark-release-1-3-0.md @@ -36,7 +36,7 @@ GraphX adds a handful of utility functions in this release, including conversion ## Upgrading to Spark 1.3 Spark 1.3 is binary compatible with Spark 1.X releases, so no code changes are necessary. This excludes API’s marked explicitly as unstable. -As part of stabilizing the Spark SQL API, the `SchemaRDD` class has been extended renamed to `DataFrame`. Spark SQL's [migration guide](http://spark.apache.org/docs/1.3.0/sql-programming-guide.html#migration-guide) describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as "string" or "table") be escaped using backticks. +As part of stabilizing the Spark SQL API, the `SchemaRDD` class has been renamed to `DataFrame`. Spark SQL's [migration guide](http://spark.apache.org/docs/1.3.0/sql-programming-guide.html#migration-guide) describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as "string" or "table") be escaped using backticks. ### Known Issues This release has few known issues which will be addressed in Spark 1.3.1: diff --git a/site/community.html b/site/community.html index 46fe44f2f..42701b422 100644 --- a/site/community.html +++ b/site/community.html @@ -188,6 +188,8 @@ </li> </ul> +<p>The StackOverflow tag <a href="http://stackoverflow.com/questions/tagged/apache-spark"><code>apache-spark</code></a> is an unofficial but active forum for Spark users' questions and answers.</p> + <p><a name="events"></a></p> <h3>Events and Meetups</h3> diff --git a/site/downloads.html b/site/downloads.html index d9d4fa9f5..e8aec835f 100644 --- a/site/downloads.html +++ b/site/downloads.html @@ -182,15 +182,15 @@ $(document).ready(function() { <ol> <li> - <p>Chose a Spark release: + <p>Choose a Spark release: <select id="sparkVersionSelect" onchange="javascript:onVersionSelect();"></select><br /></p> </li> <li> - <p>Chose a package type: + <p>Choose a package type: <select id="sparkPackageSelect" onchange="javascript:onPackageSelect();"></select><br /></p> </li> <li> - <p>Chose a download type: + <p>Choose a download type: <select id="sparkDownloadSelect" onchange="javascript:onDownloadSelect()"></select><br /></p> </li> <li> diff --git a/site/examples.html b/site/examples.html index fca0b08aa..2176ba99a 100644 --- a/site/examples.html +++ b/site/examples.html @@ -187,8 +187,8 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste <div class="tab-content"> <div class="tab-pane tab-pane-python active"> <div class="code code-tab"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> - errors = file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br /> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + errors = text_file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br /> <span class="comment"># Count all the errors</span><br /> errors.<span class="sparkop">count</span>()<br /> <span class="comment"># Count errors mentioning MySQL</span><br /> @@ -199,8 +199,8 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste </div> <div class="tab-pane tab-pane-scala"> <div class="code code-tab"> - <span class="keyword">val</span> file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> - <span class="keyword">val</span> errors = file.<span class="sparkop">filter</span>(<span class="closure">line => line.contains("ERROR")</span>)<br /> + <span class="keyword">val</span> textFile = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + <span class="keyword">val</span> errors = textFile.<span class="sparkop">filter</span>(<span class="closure">line => line.contains("ERROR")</span>)<br /> <span class="comment">// Count all the errors</span><br /> errors.<span class="sparkop">count</span>()<br /> <span class="comment">// Count errors mentioning MySQL</span><br /> @@ -211,8 +211,8 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste </div> <div class="tab-pane tab-pane-java"> <div class="code code-tab"> - JavaRDD<String> file = spark.textFile(<span class="string">"hdfs://..."</span>);<br /> - JavaRDD<String> errors = file.<span class="sparkop">filter</span>(<span class="closure">new Function<String, Boolean>() {<br /> + JavaRDD<String> textFile = spark.textFile(<span class="string">"hdfs://..."</span>);<br /> + JavaRDD<String> errors = textFile.<span class="sparkop">filter</span>(<span class="closure">new Function<String, Boolean>() {<br /> public Boolean call(String s) { return s.contains("ERROR"); }<br /> }</span>);<br /> <span class="comment">// Count all the errors</span><br /> @@ -272,8 +272,8 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste <div class="tab-content"> <div class="tab-pane tab-pane-python active"> <div class="code code-tab"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> - counts = file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split(" ")</span>) \<br /> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + counts = text_file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split(" ")</span>) \<br /> .<span class="sparkop">map</span>(<span class="closure">lambda word: (word, 1)</span>) \<br /> .<span class="sparkop">reduceByKey</span>(<span class="closure">lambda a, b: a + b</span>)<br /> counts.<span class="sparkop">saveAsTextFile</span>(<span class="string">"hdfs://..."</span>) @@ -281,8 +281,8 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste </div> <div class="tab-pane tab-pane-scala"> <div class="code code-tab"> - <span class="keyword">val</span> file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> - <span class="keyword">val</span> counts = file.<span class="sparkop">flatMap</span>(<span class="closure">line => line.split(" ")</span>)<br /> + <span class="keyword">val</span> textFile = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + <span class="keyword">val</span> counts = textFile.<span class="sparkop">flatMap</span>(<span class="closure">line => line.split(" ")</span>)<br /> .<span class="sparkop">map</span>(<span class="closure">word => (word, 1)</span>)<br /> .<span class="sparkop">reduceByKey</span>(<span class="closure">_ + _</span>)<br /> counts.<span class="sparkop">saveAsTextFile</span>(<span class="string">"hdfs://..."</span>) @@ -290,8 +290,8 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste </div> <div class="tab-pane tab-pane-java"> <div class="code code-tab"> - JavaRDD<String> file = spark.textFile(<span class="string">"hdfs://..."</span>);<br /> - JavaRDD<String> words = file.<span class="sparkop">flatMap</span>(<span class="closure">new FlatMapFunction<String, String>() {<br /> + JavaRDD<String> textFile = spark.textFile(<span class="string">"hdfs://..."</span>);<br /> + JavaRDD<String> words = textFile.<span class="sparkop">flatMap</span>(<span class="closure">new FlatMapFunction<String, String>() {<br /> public Iterable<String> call(String s) { return Arrays.asList(s.split(" ")); }<br /> }</span>);<br /> JavaPairRDD<String, Integer> pairs = words.<span class="sparkop">mapToPair</span>(<span class="closure">new PairFunction<String, String, Integer>() {<br /> diff --git a/site/faq.html b/site/faq.html index 61e88decd..eac53a5f7 100644 --- a/site/faq.html +++ b/site/faq.html @@ -213,6 +213,7 @@ Spark is a fast and general processing engine compatible with Hadoop data. It ca <p class="answer">Starting in version 0.8, Spark is under the <a href="http://www.apache.org/licenses/LICENSE-2.0.html">Apache 2.0 license</a>. Previous versions used the <a href="https://github.com/mesos/spark/blob/branch-0.7/LICENSE">BSD license</a>.</p> <p class="question">How can I contribute to Spark?</p> + <p class="answer">See the <a href="https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark">Contributing to Spark wiki</a> for more information.</p> <p class="question">Where can I get more help?</p> diff --git a/site/index.html b/site/index.html index c8be9b5cd..762a2c0e0 100644 --- a/site/index.html +++ b/site/index.html @@ -212,9 +212,9 @@ <div class="col-md-5 col-sm-5 col-padded-top col-center"> <div style="text-align: left; display: inline-block;"> <div class="code"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> <br /> - file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split()</span>)<br /> + text_file.<span class="sparkop">flatMap</span>(<span class="closure">lambda line: line.split()</span>)<br /> .<span class="sparkop">map</span>(<span class="closure">lambda word: (word, 1)</span>)<br /> .<span class="sparkop">reduceByKey</span>(<span class="closure">lambda a, b: a+b</span>) </div> @@ -222,9 +222,9 @@ </div> <!-- <div class="code" style="margin-top: 20px; text-align: left; display: inline-block;"> - file = spark.textFile(<span class="string">"hdfs://..."</span>)<br/> + text_file = spark.textFile(<span class="string">"hdfs://..."</span>)<br/> <br/> - file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br/> + text_file.<span class="sparkop">filter</span>(<span class="closure">lambda line: "ERROR" in line</span>)<br/> .<span class="sparkop">count</span>() </div> --> diff --git a/site/releases/spark-release-1-3-0.html b/site/releases/spark-release-1-3-0.html index adda1fb3c..1a00a8e3a 100644 --- a/site/releases/spark-release-1-3-0.html +++ b/site/releases/spark-release-1-3-0.html @@ -195,7 +195,7 @@ <h2 id="upgrading-to-spark-13">Upgrading to Spark 1.3</h2> <p>Spark 1.3 is binary compatible with Spark 1.X releases, so no code changes are necessary. This excludes API’s marked explicitly as unstable.</p> -<p>As part of stabilizing the Spark SQL API, the <code>SchemaRDD</code> class has been extended renamed to <code>DataFrame</code>. Spark SQL’s <a href="http://spark.apache.org/docs/1.3.0/sql-programming-guide.html#migration-guide">migration guide</a> describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as “string” or “table”) be escaped using backticks.</p> +<p>As part of stabilizing the Spark SQL API, the <code>SchemaRDD</code> class has been renamed to <code>DataFrame</code>. Spark SQL’s <a href="http://spark.apache.org/docs/1.3.0/sql-programming-guide.html#migration-guide">migration guide</a> describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as “string” or “table”) be escaped using backticks.</p> <h3 id="known-issues">Known Issues</h3> <p>This release has few known issues which will be addressed in Spark 1.3.1:</p> diff --git a/sql/index.md b/sql/index.md index 4f042235d..09ce9deaa 100644 --- a/sql/index.md +++ b/sql/index.md @@ -16,7 +16,7 @@ subproject: SQL <div class="col-md-7 col-sm-7"> <h2>Integrated</h2> <p class="lead"> - Seemlessly mix SQL queries with Spark programs. + Seamlessly mix SQL queries with Spark programs. </p> <p> Spark SQL lets you query structured data as a distributed dataset (RDD) in Spark, with integrated APIs in Python, Scala and Java. |