diff options
author | Wenchen Fan <wenchen@databricks.com> | 2016-06-12 21:36:41 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-06-12 21:36:41 -0700 |
commit | e2ab79d5ea00af45c083cc9a6607d2f0905f9908 (patch) | |
tree | 750a843ae6ddba4abc3cf592a26960fb6de19189 /python | |
parent | 1f8f2b5c2a33e63367ea4881b5918f6bc0a6f52f (diff) | |
download | spark-e2ab79d5ea00af45c083cc9a6607d2f0905f9908.tar.gz spark-e2ab79d5ea00af45c083cc9a6607d2f0905f9908.tar.bz2 spark-e2ab79d5ea00af45c083cc9a6607d2f0905f9908.zip |
[SPARK-15898][SQL] DataFrameReader.text should return DataFrame
## What changes were proposed in this pull request?
We want to maintain API compatibility for DataFrameReader.text, and will introduce a new API called DataFrameReader.textFile which returns Dataset[String].
affected PRs:
https://github.com/apache/spark/pull/11731
https://github.com/apache/spark/pull/13104
https://github.com/apache/spark/pull/13184
## How was this patch tested?
N/A
Author: Wenchen Fan <wenchen@databricks.com>
Closes #13604 from cloud-fan/revert.
Diffstat (limited to 'python')
-rw-r--r-- | python/pyspark/sql/readwriter.py | 8 |
1 files changed, 4 insertions, 4 deletions
diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py index f3182b237e..0f50f672a2 100644 --- a/python/pyspark/sql/readwriter.py +++ b/python/pyspark/sql/readwriter.py @@ -291,10 +291,10 @@ class DataFrameReader(object): @ignore_unicode_prefix @since(1.6) def text(self, paths): - """Loads a text file and returns a :class:`DataFrame` with a single string column named "value". - If the directory structure of the text files contains partitioning information, - those are ignored in the resulting DataFrame. To include partitioning information as - columns, use ``read.format('text').load(...)``. + """ + Loads text files and returns a :class:`DataFrame` whose schema starts with a + string column named "value", and followed by partitioned columns if there + are any. Each line in the text file is a new row in the resulting DataFrame. |