diff options
| author | Jurriaan Pruis <email@jurriaanpruis.nl> | 2016-05-18 16:15:09 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-05-18 16:15:09 -0700 |
| commit | 32be51fba45f5e07a2a3520293c12dc7765a364d (patch) | |
| tree | 1f967ee3b668518ec4c23af817db0cc1b5bfaf21 /sql/core/src/test/resources | |
| parent | 84b23453ddb0a97e3d81306de0a5dcb64f88bdd0 (diff) | |
| download | spark-32be51fba45f5e07a2a3520293c12dc7765a364d.tar.gz spark-32be51fba45f5e07a2a3520293c12dc7765a364d.tar.bz2 spark-32be51fba45f5e07a2a3520293c12dc7765a364d.zip | |
[SPARK-15323][SPARK-14463][SQL] Fix reading of partitioned format=text datasets
https://issues.apache.org/jira/browse/SPARK-15323
I was using partitioned text datasets in Spark 1.6.1 but it broke in Spark 2.0.0.
It would be logical if you could also write those,
but not entirely sure how to solve this with the new DataSet implementation.
Also it doesn't work using `sqlContext.read.text`, since that method returns a `DataSet[String]`.
See https://issues.apache.org/jira/browse/SPARK-14463 for that issue.
Author: Jurriaan Pruis <email@jurriaanpruis.nl>
Closes #13104 from jurriaan/fix-partitioned-text-reads.
Diffstat (limited to 'sql/core/src/test/resources')
| -rw-r--r-- | sql/core/src/test/resources/text-partitioned/year=2014/data.txt | 1 | ||||
| -rw-r--r-- | sql/core/src/test/resources/text-partitioned/year=2015/data.txt | 1 |
2 files changed, 2 insertions, 0 deletions
diff --git a/sql/core/src/test/resources/text-partitioned/year=2014/data.txt b/sql/core/src/test/resources/text-partitioned/year=2014/data.txt new file mode 100644 index 0000000000..e2719428bb --- /dev/null +++ b/sql/core/src/test/resources/text-partitioned/year=2014/data.txt @@ -0,0 +1 @@ +2014-test diff --git a/sql/core/src/test/resources/text-partitioned/year=2015/data.txt b/sql/core/src/test/resources/text-partitioned/year=2015/data.txt new file mode 100644 index 0000000000..b8c03daa8c --- /dev/null +++ b/sql/core/src/test/resources/text-partitioned/year=2015/data.txt @@ -0,0 +1 @@ +2015-test |