diff options
| author | Aaron Davidson <aaron@databricks.com> | 2014-08-27 15:05:47 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-08-27 15:05:47 -0700 |
| commit | cc275f4b7910f6d0ad266a43bac2fdae58e9739e (patch) | |
| tree | c28f36f0320756b6378351530d42390d7e40bc4f /sql | |
| parent | 28d41d627919fcb196d9d31bad65d664770bee67 (diff) | |
| download | spark-cc275f4b7910f6d0ad266a43bac2fdae58e9739e.tar.gz spark-cc275f4b7910f6d0ad266a43bac2fdae58e9739e.tar.bz2 spark-cc275f4b7910f6d0ad266a43bac2fdae58e9739e.zip | |
[SQL] [SPARK-3236] Reading Parquet tables from Metastore mangles location
Currently we do `relation.hiveQlTable.getDataLocation.getPath`, which returns the path-part of the URI (e.g., "s3n://my-bucket/my-path" => "/my-path"). We should do `relation.hiveQlTable.getDataLocation.toString` instead, as a URI's toString returns a faithful representation of the full URI, which can later be passed into a Hadoop Path.
Author: Aaron Davidson <aaron@databricks.com>
Closes #2150 from aarondav/parquet-location and squashes the following commits:
459f72c [Aaron Davidson] [SQL] [SPARK-3236] Reading Parquet tables from Metastore mangles location
Diffstat (limited to 'sql')
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala | 2 |
1 files changed, 1 insertions, 1 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala index 10fa8314c9..47e24f0dec 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveStrategies.scala @@ -135,7 +135,7 @@ private[hive] trait HiveStrategies { .fakeOutput(projectList.map(_.toAttribute)):: Nil } else { hiveContext - .parquetFile(relation.hiveQlTable.getDataLocation.getPath) + .parquetFile(relation.hiveQlTable.getDataLocation.toString) .lowerCase .where(unresolvedOtherPredicates) .select(unresolvedProjection:_*) |