From 090a6d74cea47ee2023e2f15aa207afe83194702 Mon Sep 17 00:00:00 2001 From: "Sean R. Owen" Date: Fri, 8 May 2015 13:59:51 +0000 Subject: Reapply my past changes, which I had only applied to .html, to .md too, and add the changes from the regenerated .html too --- community.md | 2 ++ downloads.md | 6 +++--- examples.md | 24 +++++++++++------------ faq.md | 4 ++-- index.md | 8 ++++---- releases/_posts/2015-03-13-spark-release-1-3-0.md | 2 +- site/community.html | 2 ++ site/downloads.html | 6 +++--- site/examples.html | 24 +++++++++++------------ site/faq.html | 1 + site/index.html | 8 ++++---- site/releases/spark-release-1-3-0.html | 2 +- sql/index.md | 2 +- 13 files changed, 48 insertions(+), 43 deletions(-) diff --git a/community.md b/community.md index f32c249c4..23046902d 100644 --- a/community.md +++ b/community.md @@ -28,6 +28,8 @@ navigation: +

The StackOverflow tag apache-spark is an unofficial but active forum for Spark users' questions and answers.

+

Events and Meetups

diff --git a/downloads.md b/downloads.md index 5587d5021..c96ad69f4 100644 --- a/downloads.md +++ b/downloads.md @@ -20,13 +20,13 @@ The latest release of Spark is Spark 1.3.1, released on April 17, 2015 (release notes) (git tag)
-1. Chose a Spark release: +1. Choose a Spark release:
-2. Chose a package type: +2. Choose a package type:
-3. Chose a download type: +3. Choose a download type:
4. Download Spark: diff --git a/examples.md b/examples.md index 3658d1788..18d869353 100644 --- a/examples.md +++ b/examples.md @@ -26,8 +26,8 @@ In this example, we search through the error messages in a log file:
- file = spark.textFile("hdfs://...")
- errors = file.filter(lambda line: "ERROR" in line)
+ text_file = spark.textFile("hdfs://...")
+ errors = text_file.filter(lambda line: "ERROR" in line)
# Count all the errors
errors.count()
# Count errors mentioning MySQL
@@ -38,8 +38,8 @@ In this example, we search through the error messages in a log file:
- val file = spark.textFile("hdfs://...")
- val errors = file.filter(line => line.contains("ERROR"))
+ val textFile = spark.textFile("hdfs://...")
+ val errors = textFile.filter(line => line.contains("ERROR"))
// Count all the errors
errors.count()
// Count errors mentioning MySQL
@@ -50,8 +50,8 @@ In this example, we search through the error messages in a log file:
- JavaRDD<String> file = spark.textFile("hdfs://...");
- JavaRDD<String> errors = file.filter(new Function<String, Boolean>() {
+ JavaRDD<String> textFile = spark.textFile("hdfs://...");
+ JavaRDD<String> errors = textFile.filter(new Function<String, Boolean>() {
  public Boolean call(String s) { return s.contains("ERROR"); }
});
// Count all the errors
@@ -112,8 +112,8 @@ In this example, we search through the error messages in a log file:
- file = spark.textFile("hdfs://...")
- counts = file.flatMap(lambda line: line.split(" ")) \
+ text_file = spark.textFile("hdfs://...")
+ counts = text_file.flatMap(lambda line: line.split(" ")) \
             .map(lambda word: (word, 1)) \
             .reduceByKey(lambda a, b: a + b)
counts.saveAsTextFile("hdfs://...") @@ -121,8 +121,8 @@ In this example, we search through the error messages in a log file:
- val file = spark.textFile("hdfs://...")
- val counts = file.flatMap(line => line.split(" "))
+ val textFile = spark.textFile("hdfs://...")
+ val counts = textFile.flatMap(line => line.split(" "))
                 .map(word => (word, 1))
                 .reduceByKey(_ + _)
counts.saveAsTextFile("hdfs://...") @@ -130,8 +130,8 @@ In this example, we search through the error messages in a log file:
- JavaRDD<String> file = spark.textFile("hdfs://...");
- JavaRDD<String> words = file.flatMap(new FlatMapFunction<String, String>() {
+ JavaRDD<String> textFile = spark.textFile("hdfs://...");
+ JavaRDD<String> words = textFile.flatMap(new FlatMapFunction<String, String>() {
  public Iterable<String> call(String s) { return Arrays.asList(s.split(" ")); }
});
JavaPairRDD<String, Integer> pairs = words.mapToPair(new PairFunction<String, String, Integer>() {
diff --git a/faq.md b/faq.md index 4a6ecc66a..c3e6b2da0 100644 --- a/faq.md +++ b/faq.md @@ -53,8 +53,8 @@ Spark is a fast and general processing engine compatible with Hadoop data. It ca

Starting in version 0.8, Spark is under the Apache 2.0 license. Previous versions used the BSD license.

How can I contribute to Spark?

-

Contact the mailing list or send us a pull request on GitHub (instructions here). We're glad to hear about your experience using Spark and to accept patches.

-

If you would like to report an issue, post it to the Spark issue tracker.

+ +

See the Contributing to Spark wiki for more information.

Where can I get more help?

Please post on the Spark Users mailing list. We'll be glad to help!

diff --git a/index.md b/index.md index 754bbf253..90304b64c 100644 --- a/index.md +++ b/index.md @@ -53,9 +53,9 @@ navigation:
- file = spark.textFile("hdfs://...")
+ text_file = spark.textFile("hdfs://...")
 
- file.flatMap(lambda line: line.split())
+ text_file.flatMap(lambda line: line.split())
    .map(lambda word: (word, 1))
    .reduceByKey(lambda a, b: a+b)
@@ -63,9 +63,9 @@ navigation:
diff --git a/releases/_posts/2015-03-13-spark-release-1-3-0.md b/releases/_posts/2015-03-13-spark-release-1-3-0.md index a7c2a7924..bc9c4db84 100644 --- a/releases/_posts/2015-03-13-spark-release-1-3-0.md +++ b/releases/_posts/2015-03-13-spark-release-1-3-0.md @@ -36,7 +36,7 @@ GraphX adds a handful of utility functions in this release, including conversion ## Upgrading to Spark 1.3 Spark 1.3 is binary compatible with Spark 1.X releases, so no code changes are necessary. This excludes API’s marked explicitly as unstable. -As part of stabilizing the Spark SQL API, the `SchemaRDD` class has been extended renamed to `DataFrame`. Spark SQL's [migration guide](http://spark.apache.org/docs/1.3.0/sql-programming-guide.html#migration-guide) describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as "string" or "table") be escaped using backticks. +As part of stabilizing the Spark SQL API, the `SchemaRDD` class has been renamed to `DataFrame`. Spark SQL's [migration guide](http://spark.apache.org/docs/1.3.0/sql-programming-guide.html#migration-guide) describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as "string" or "table") be escaped using backticks. ### Known Issues This release has few known issues which will be addressed in Spark 1.3.1: diff --git a/site/community.html b/site/community.html index 46fe44f2f..42701b422 100644 --- a/site/community.html +++ b/site/community.html @@ -188,6 +188,8 @@ +

The StackOverflow tag apache-spark is an unofficial but active forum for Spark users' questions and answers.

+

Events and Meetups

diff --git a/site/downloads.html b/site/downloads.html index d9d4fa9f5..e8aec835f 100644 --- a/site/downloads.html +++ b/site/downloads.html @@ -182,15 +182,15 @@ $(document).ready(function() {
  1. -

    Chose a Spark release: +

    Choose a Spark release:

  2. -

    Chose a package type: +

    Choose a package type:

  3. -

    Chose a download type: +

    Choose a download type:

  4. diff --git a/site/examples.html b/site/examples.html index fca0b08aa..2176ba99a 100644 --- a/site/examples.html +++ b/site/examples.html @@ -187,8 +187,8 @@ previous ones, and actions, which kick off a job to execute on a cluste
    - file = spark.textFile("hdfs://...")
    - errors = file.filter(lambda line: "ERROR" in line)
    + text_file = spark.textFile("hdfs://...")
    + errors = text_file.filter(lambda line: "ERROR" in line)
    # Count all the errors
    errors.count()
    # Count errors mentioning MySQL
    @@ -199,8 +199,8 @@ previous ones, and actions, which kick off a job to execute on a cluste
    - val file = spark.textFile("hdfs://...")
    - val errors = file.filter(line => line.contains("ERROR"))
    + val textFile = spark.textFile("hdfs://...")
    + val errors = textFile.filter(line => line.contains("ERROR"))
    // Count all the errors
    errors.count()
    // Count errors mentioning MySQL
    @@ -211,8 +211,8 @@ previous ones, and actions, which kick off a job to execute on a cluste
    - JavaRDD<String> file = spark.textFile("hdfs://...");
    - JavaRDD<String> errors = file.filter(new Function<String, Boolean>() {
    + JavaRDD<String> textFile = spark.textFile("hdfs://...");
    + JavaRDD<String> errors = textFile.filter(new Function<String, Boolean>() {
      public Boolean call(String s) { return s.contains("ERROR"); }
    }    );
    // Count all the errors
    @@ -272,8 +272,8 @@ previous ones, and actions, which kick off a job to execute on a cluste
    - file = spark.textFile("hdfs://...")
    - counts = file.flatMap(lambda line: line.split(" ")) \
    + text_file = spark.textFile("hdfs://...")
    + counts = text_file.flatMap(lambda line: line.split(" ")) \
                 .map(lambda word: (word, 1)) \
                 .reduceByKey(lambda a, b: a + b)
    counts.saveAsTextFile("hdfs://...") @@ -281,8 +281,8 @@ previous ones, and actions, which kick off a job to execute on a cluste
    - val file = spark.textFile("hdfs://...")
    - val counts = file.flatMap(line => line.split(" "))
    + val textFile = spark.textFile("hdfs://...")
    + val counts = textFile.flatMap(line => line.split(" "))
                     .map(word => (word, 1))
                     .reduceByKey(_ + _)
    counts.saveAsTextFile("hdfs://...") @@ -290,8 +290,8 @@ previous ones, and actions, which kick off a job to execute on a cluste
    - JavaRDD<String> file = spark.textFile("hdfs://...");
    - JavaRDD<String> words = file.flatMap(new FlatMapFunction<String, String>() {
    + JavaRDD<String> textFile = spark.textFile("hdfs://...");
    + JavaRDD<String> words = textFile.flatMap(new FlatMapFunction<String, String>() {
      public Iterable<String> call(String s) { return Arrays.asList(s.split(" ")); }
    }    );
    JavaPairRDD<String, Integer> pairs = words.mapToPair(new PairFunction<String, String, Integer>() {
    diff --git a/site/faq.html b/site/faq.html index 61e88decd..eac53a5f7 100644 --- a/site/faq.html +++ b/site/faq.html @@ -213,6 +213,7 @@ Spark is a fast and general processing engine compatible with Hadoop data. It ca

    Starting in version 0.8, Spark is under the Apache 2.0 license. Previous versions used the BSD license.

    How can I contribute to Spark?

    +

    See the Contributing to Spark wiki for more information.

    Where can I get more help?

    diff --git a/site/index.html b/site/index.html index c8be9b5cd..762a2c0e0 100644 --- a/site/index.html +++ b/site/index.html @@ -212,9 +212,9 @@
    - file = spark.textFile("hdfs://...")
    + text_file = spark.textFile("hdfs://...")
     
    - file.flatMap(lambda line: line.split())
    + text_file.flatMap(lambda line: line.split())
        .map(lambda word: (word, 1))
        .reduceByKey(lambda a, b: a+b)
    @@ -222,9 +222,9 @@
    diff --git a/site/releases/spark-release-1-3-0.html b/site/releases/spark-release-1-3-0.html index adda1fb3c..1a00a8e3a 100644 --- a/site/releases/spark-release-1-3-0.html +++ b/site/releases/spark-release-1-3-0.html @@ -195,7 +195,7 @@

    Upgrading to Spark 1.3

    Spark 1.3 is binary compatible with Spark 1.X releases, so no code changes are necessary. This excludes API’s marked explicitly as unstable.

    -

    As part of stabilizing the Spark SQL API, the SchemaRDD class has been extended renamed to DataFrame. Spark SQL’s migration guide describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as “string” or “table”) be escaped using backticks.

    +

    As part of stabilizing the Spark SQL API, the SchemaRDD class has been renamed to DataFrame. Spark SQL’s migration guide describes the upgrade process in detail. Spark SQL also now requires that column identifiers which use reserved words (such as “string” or “table”) be escaped using backticks.

    Known Issues

    This release has few known issues which will be addressed in Spark 1.3.1:

    diff --git a/sql/index.md b/sql/index.md index 4f042235d..09ce9deaa 100644 --- a/sql/index.md +++ b/sql/index.md @@ -16,7 +16,7 @@ subproject: SQL

    Integrated

    - Seemlessly mix SQL queries with Spark programs. + Seamlessly mix SQL queries with Spark programs.

    Spark SQL lets you query structured data as a distributed dataset (RDD) in Spark, with integrated APIs in Python, Scala and Java. -- cgit v1.2.3