aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorCheng Lian <lian@databricks.com>2016-06-29 22:50:53 -0700
committerXiangrui Meng <meng@databricks.com>2016-06-29 22:50:53 -0700
commitbde1d6a61593aeb62370f526542cead94919b0c0 (patch)
tree8630ee0675f1be8b45b8f4d72a9ced10f6d2eb80 /docs
parentd3af6731fa270842818ed91d6b4d14708ddae2db (diff)
downloadspark-bde1d6a61593aeb62370f526542cead94919b0c0.tar.gz
spark-bde1d6a61593aeb62370f526542cead94919b0c0.tar.bz2
spark-bde1d6a61593aeb62370f526542cead94919b0c0.zip
[SPARK-16294][SQL] Labelling support for the include_example Jekyll plugin
## What changes were proposed in this pull request? This PR adds labelling support for the `include_example` Jekyll plugin, so that we may split a single source file into multiple line blocks with different labels, and include them in multiple code snippets in the generated HTML page. ## How was this patch tested? Manually tested. <img width="923" alt="screenshot at jun 29 19-53-21" src="https://cloud.githubusercontent.com/assets/230655/16451099/66a76db2-3e33-11e6-84fb-63104c2f0688.png"> Author: Cheng Lian <lian@databricks.com> Closes #13972 from liancheng/include-example-with-labels.
Diffstat (limited to 'docs')
-rw-r--r--docs/_plugins/include_example.rb25
-rw-r--r--docs/sql-programming-guide.md41
2 files changed, 25 insertions, 41 deletions
diff --git a/docs/_plugins/include_example.rb b/docs/_plugins/include_example.rb
index f7485826a7..306888801d 100644
--- a/docs/_plugins/include_example.rb
+++ b/docs/_plugins/include_example.rb
@@ -32,8 +32,18 @@ module Jekyll
@code_dir = File.join(site.source, config_dir)
clean_markup = @markup.strip
- @file = File.join(@code_dir, clean_markup)
- @lang = clean_markup.split('.').last
+
+ parts = clean_markup.strip.split(' ')
+ if parts.length > 1 then
+ @snippet_label = ':' + parts[0]
+ snippet_file = parts[1]
+ else
+ @snippet_label = ''
+ snippet_file = parts[0]
+ end
+
+ @file = File.join(@code_dir, snippet_file)
+ @lang = snippet_file.split('.').last
code = File.open(@file).read.encode("UTF-8")
code = select_lines(code)
@@ -41,7 +51,7 @@ module Jekyll
rendered_code = Pygments.highlight(code, :lexer => @lang)
hint = "<div><small>Find full example code at " \
- "\"examples/src/main/#{clean_markup}\" in the Spark repo.</small></div>"
+ "\"examples/src/main/#{snippet_file}\" in the Spark repo.</small></div>"
rendered_code + hint
end
@@ -66,13 +76,13 @@ module Jekyll
# Select the array of start labels from code.
startIndices = lines
.each_with_index
- .select { |l, i| l.include? "$example on$" }
+ .select { |l, i| l.include? "$example on#{@snippet_label}$" }
.map { |l, i| i }
# Select the array of end labels from code.
endIndices = lines
.each_with_index
- .select { |l, i| l.include? "$example off$" }
+ .select { |l, i| l.include? "$example off#{@snippet_label}$" }
.map { |l, i| i }
raise "Start indices amount is not equal to end indices amount, see #{@file}." \
@@ -92,7 +102,10 @@ module Jekyll
if start == endline
lastIndex = endline
range = Range.new(start + 1, endline - 1)
- result += trim_codeblock(lines[range]).join
+ trimmed = trim_codeblock(lines[range])
+ # Filter out possible example tags of overlapped labels.
+ taggs_filtered = trimmed.select { |l| !l.include? '$example ' }
+ result += taggs_filtered.join
result += "\n"
end
result
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md
index 6c6bc8db6a..68419e1331 100644
--- a/docs/sql-programming-guide.md
+++ b/docs/sql-programming-guide.md
@@ -63,52 +63,23 @@ Throughout this document, we will often refer to Scala/Java Datasets of `Row`s a
<div class="codetabs">
<div data-lang="scala" markdown="1">
-The entry point into all functionality in Spark is the [`SparkSession`](api/scala/index.html#org.apache.spark.sql.SparkSession) class. To create a basic `SparkSession`, just use `SparkSession.build()`:
-
-{% highlight scala %}
-import org.apache.spark.sql.SparkSession
-
-val spark = SparkSession.build()
- .master("local")
- .appName("Word Count")
- .config("spark.some.config.option", "some-value")
- .getOrCreate()
-
-// this is used to implicitly convert an RDD to a DataFrame.
-import spark.implicits._
-{% endhighlight %}
+The entry point into all functionality in Spark is the [`SparkSession`](api/scala/index.html#org.apache.spark.sql.SparkSession) class. To create a basic `SparkSession`, just use `SparkSession.builder()`:
+{% include_example init_session scala/org/apache/spark/examples/sql/RDDRelation.scala %}
</div>
<div data-lang="java" markdown="1">
-The entry point into all functionality in Spark is the [`SparkSession`](api/java/index.html#org.apache.spark.sql.SparkSession) class. To create a basic `SparkSession`, just use `SparkSession.build()`:
+The entry point into all functionality in Spark is the [`SparkSession`](api/java/index.html#org.apache.spark.sql.SparkSession) class. To create a basic `SparkSession`, just use `SparkSession.builder()`:
-{% highlight java %}
-import org.apache.spark.sql.SparkSession
-
-SparkSession spark = SparkSession.build()
- .master("local")
- .appName("Word Count")
- .config("spark.some.config.option", "some-value")
- .getOrCreate();
-{% endhighlight %}
+{% include_example init_session java/org/apache/spark/examples/sql/JavaSparkSQL.java %}
</div>
<div data-lang="python" markdown="1">
-The entry point into all functionality in Spark is the [`SparkSession`](api/python/pyspark.sql.html#pyspark.sql.SparkSession) class. To create a basic `SparkSession`, just use `SparkSession.build`:
-
-{% highlight python %}
-from pyspark.sql import SparkSession
-
-spark = SparkSession.build \
- .master("local") \
- .appName("Word Count") \
- .config("spark.some.config.option", "some-value") \
- .getOrCreate()
-{% endhighlight %}
+The entry point into all functionality in Spark is the [`SparkSession`](api/python/pyspark.sql.html#pyspark.sql.SparkSession) class. To create a basic `SparkSession`, just use `SparkSession.builder`:
+{% include_example init_session python/sql.py %}
</div>
<div data-lang="r" markdown="1">