diff options
| author | hyukjinkwon <gurwls223@gmail.com> | 2016-11-01 12:46:41 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2016-11-01 12:46:41 -0700 |
| commit | 01dd0083011741c2bbe5ae1d2a25f2c9a1302b76 (patch) | |
| tree | 7b9993165b1a4f48e64d566d93c7883a3096403d /python/pyspark/sql/readwriter.py | |
| parent | cfac17ee1cec414663b957228e469869eb7673c1 (diff) | |
| download | spark-01dd0083011741c2bbe5ae1d2a25f2c9a1302b76.tar.gz spark-01dd0083011741c2bbe5ae1d2a25f2c9a1302b76.tar.bz2 spark-01dd0083011741c2bbe5ae1d2a25f2c9a1302b76.zip | |
[SPARK-17764][SQL] Add `to_json` supporting to convert nested struct column to JSON string
## What changes were proposed in this pull request?
This PR proposes to add `to_json` function in contrast with `from_json` in Scala, Java and Python.
It'd be useful if we can convert a same column from/to json. Also, some datasources do not support nested types. If we are forced to save a dataframe into those data sources, we might be able to work around by this function.
The usage is as below:
``` scala
val df = Seq(Tuple1(Tuple1(1))).toDF("a")
df.select(to_json($"a").as("json")).show()
```
``` bash
+--------+
| json|
+--------+
|{"_1":1}|
+--------+
```
## How was this patch tested?
Unit tests in `JsonFunctionsSuite` and `JsonExpressionsSuite`.
Author: hyukjinkwon <gurwls223@gmail.com>
Closes #15354 from HyukjinKwon/SPARK-17764.
Diffstat (limited to 'python/pyspark/sql/readwriter.py')
| -rw-r--r-- | python/pyspark/sql/readwriter.py | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/python/pyspark/sql/readwriter.py b/python/pyspark/sql/readwriter.py index bc786ef95e..b0c51b1e99 100644 --- a/python/pyspark/sql/readwriter.py +++ b/python/pyspark/sql/readwriter.py @@ -161,7 +161,7 @@ class DataFrameReader(OptionUtils): mode=None, columnNameOfCorruptRecord=None, dateFormat=None, timestampFormat=None): """ Loads a JSON file (`JSON Lines text format or newline-delimited JSON - <[http://jsonlines.org/>`_) or an RDD of Strings storing JSON objects (one object per + <http://jsonlines.org/>`_) or an RDD of Strings storing JSON objects (one object per record) and returns the result as a :class`DataFrame`. If the ``schema`` parameter is not specified, this function goes |