diff options
author | Nicholas Chammas <nicholas.chammas@gmail.com> | 2016-07-28 14:57:15 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-07-28 14:57:15 -0700 |
commit | 274f3b9ec86e4109c7678eef60f990d41dc3899f (patch) | |
tree | 2394c6f1ff3e51bd9ea6bd2b365c1e7068c61295 /python/pyspark/sql/streaming.py | |
parent | 3fd39b87bda77f3c3a4622d854f23d4234683571 (diff) | |
download | spark-274f3b9ec86e4109c7678eef60f990d41dc3899f.tar.gz spark-274f3b9ec86e4109c7678eef60f990d41dc3899f.tar.bz2 spark-274f3b9ec86e4109c7678eef60f990d41dc3899f.zip |
[SPARK-16772] Correct API doc references to PySpark classes + formatting fixes
## What's Been Changed
The PR corrects several broken or missing class references in the Python API docs. It also correct formatting problems.
For example, you can see [here](http://spark.apache.org/docs/2.0.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.registerFunction) how Sphinx is not picking up the reference to `DataType`. That's because the reference is relative to the current module, whereas `DataType` is in a different module.
You can also see [here](http://spark.apache.org/docs/2.0.0/api/python/pyspark.sql.html#pyspark.sql.SQLContext.createDataFrame) how the formatting for byte, tinyint, and so on is italic instead of monospace. That's because in ReST single backticks just make things italic, unlike in Markdown.
## Testing
I tested this PR by [building the Python docs](https://github.com/apache/spark/tree/master/docs#generating-the-documentation-html) and reviewing the results locally in my browser. I confirmed that the broken or missing class references were resolved, and that the formatting was corrected.
Author: Nicholas Chammas <nicholas.chammas@gmail.com>
Closes #14393 from nchammas/python-docstring-fixes.
Diffstat (limited to 'python/pyspark/sql/streaming.py')
-rw-r--r-- | python/pyspark/sql/streaming.py | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/python/pyspark/sql/streaming.py b/python/pyspark/sql/streaming.py index 8bac347e13..a364555003 100644 --- a/python/pyspark/sql/streaming.py +++ b/python/pyspark/sql/streaming.py @@ -269,7 +269,7 @@ class DataStreamReader(OptionUtils): .. note:: Experimental. - :param schema: a StructType object + :param schema: a :class:`pyspark.sql.types.StructType` object >>> s = spark.readStream.schema(sdf_schema) """ @@ -310,7 +310,7 @@ class DataStreamReader(OptionUtils): :param path: optional string for file-system backed data sources. :param format: optional string for format of the data source. Default to 'parquet'. - :param schema: optional :class:`StructType` for the input schema. + :param schema: optional :class:`pyspark.sql.types.StructType` for the input schema. :param options: all other string options >>> json_sdf = spark.readStream.format("json")\ @@ -349,7 +349,7 @@ class DataStreamReader(OptionUtils): :param path: string represents path to the JSON dataset, or RDD of Strings storing JSON objects. - :param schema: an optional :class:`StructType` for the input schema. + :param schema: an optional :class:`pyspark.sql.types.StructType` for the input schema. :param primitivesAsString: infers all primitive values as a string type. If None is set, it uses the default value, ``false``. :param prefersDecimal: infers all floating-point values as a decimal type. If the values @@ -461,7 +461,7 @@ class DataStreamReader(OptionUtils): .. note:: Experimental. :param path: string, or list of strings, for input path(s). - :param schema: an optional :class:`StructType` for the input schema. + :param schema: an optional :class:`pyspark.sql.types.StructType` for the input schema. :param sep: sets the single character as a separator for each field and value. If None is set, it uses the default value, ``,``. :param encoding: decodes the CSV files by the given encoding type. If None is set, |