aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark/sql/streaming.py
diff options
context:
space:
mode:
authorNicholas Chammas <nicholas.chammas@gmail.com>2016-07-28 14:57:15 -0700
committerReynold Xin <rxin@databricks.com>2016-07-28 14:57:15 -0700
commit274f3b9ec86e4109c7678eef60f990d41dc3899f (patch)
tree2394c6f1ff3e51bd9ea6bd2b365c1e7068c61295 /python/pyspark/sql/streaming.py
parent3fd39b87bda77f3c3a4622d854f23d4234683571 (diff)
downloadspark-274f3b9ec86e4109c7678eef60f990d41dc3899f.tar.gz
spark-274f3b9ec86e4109c7678eef60f990d41dc3899f.tar.bz2
spark-274f3b9ec86e4109c7678eef60f990d41dc3899f.zip
[SPARK-16772] Correct API doc references to PySpark classes + formatting fixes
## What's Been Changed The PR corrects several broken or missing class references in the Python API docs. It also correct formatting problems. For example, you can see [here](http://spark.apache.org/docs/2.0.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.registerFunction) how Sphinx is not picking up the reference to `DataType`. That's because the reference is relative to the current module, whereas `DataType` is in a different module. You can also see [here](http://spark.apache.org/docs/2.0.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.createDataFrame) how the formatting for byte, tinyint, and so on is italic instead of monospace. That's because in ReST single backticks just make things italic, unlike in Markdown. ## Testing I tested this PR by [building the Python docs](https://github.com/apache/spark/tree/master/docs#generating-the-documentation-html) and reviewing the results locally in my browser. I confirmed that the broken or missing class references were resolved, and that the formatting was corrected. Author: Nicholas Chammas <nicholas.chammas@gmail.com> Closes #14393 from nchammas/python-docstring-fixes.
Diffstat (limited to 'python/pyspark/sql/streaming.py')
-rw-r--r--python/pyspark/sql/streaming.py8
1 files changed, 4 insertions, 4 deletions
diff --git a/python/pyspark/sql/streaming.py b/python/pyspark/sql/streaming.py
index 8bac347e13..a364555003 100644
--- a/python/pyspark/sql/streaming.py
+++ b/python/pyspark/sql/streaming.py
@@ -269,7 +269,7 @@ class DataStreamReader(OptionUtils):
.. note:: Experimental.
- :param schema: a StructType object
+ :param schema: a :class:`pyspark.sql.types.StructType` object
>>> s = spark.readStream.schema(sdf_schema)
"""
@@ -310,7 +310,7 @@ class DataStreamReader(OptionUtils):
:param path: optional string for file-system backed data sources.
:param format: optional string for format of the data source. Default to 'parquet'.
- :param schema: optional :class:`StructType` for the input schema.
+ :param schema: optional :class:`pyspark.sql.types.StructType` for the input schema.
:param options: all other string options
>>> json_sdf = spark.readStream.format("json")\
@@ -349,7 +349,7 @@ class DataStreamReader(OptionUtils):
:param path: string represents path to the JSON dataset,
or RDD of Strings storing JSON objects.
- :param schema: an optional :class:`StructType` for the input schema.
+ :param schema: an optional :class:`pyspark.sql.types.StructType` for the input schema.
:param primitivesAsString: infers all primitive values as a string type. If None is set,
it uses the default value, ``false``.
:param prefersDecimal: infers all floating-point values as a decimal type. If the values
@@ -461,7 +461,7 @@ class DataStreamReader(OptionUtils):
.. note:: Experimental.
:param path: string, or list of strings, for input path(s).
- :param schema: an optional :class:`StructType` for the input schema.
+ :param schema: an optional :class:`pyspark.sql.types.StructType` for the input schema.
:param sep: sets the single character as a separator for each field and value.
If None is set, it uses the default value, ``,``.
:param encoding: decodes the CSV files by the given encoding type. If None is set,