diff options
| author | Cheng Lian <lian@databricks.com> | 2015-06-18 22:01:52 -0700 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2015-06-18 22:01:52 -0700 |
| commit | a71cbbdea581573192a59bf8472861c463c40fcb (patch) | |
| tree | 5bc75d73b723403de1c28ef31d7fae4b7dd63b03 | |
| parent | 3eaed8769c16e887edb9d54f5816b4ee6da23de5 (diff) | |
| download | spark-a71cbbdea581573192a59bf8472861c463c40fcb.tar.gz spark-a71cbbdea581573192a59bf8472861c463c40fcb.tar.bz2 spark-a71cbbdea581573192a59bf8472861c463c40fcb.zip | |
[SPARK-8458] [SQL] Don't strip scheme part of output path when writing ORC files
`Path.toUri.getPath` strips scheme part of output path (from `file:///foo` to `/foo`), which causes ORC data source only writes to the file system configured in Hadoop configuration. Should use `Path.toString` instead.
Author: Cheng Lian <lian@databricks.com>
Closes #6892 from liancheng/spark-8458 and squashes the following commits:
87f8199 [Cheng Lian] Don't strip scheme of output path when writing ORC files
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala index 77f1ca9ae0..dbce39f21d 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala @@ -111,7 +111,7 @@ private[orc] class OrcOutputWriter( new OrcOutputFormat().getRecordWriter( new Path(path, filename).getFileSystem(conf), conf.asInstanceOf[JobConf], - new Path(path, filename).toUri.getPath, + new Path(path, filename).toString, Reporter.NULL ).asInstanceOf[RecordWriter[NullWritable, Writable]] } |