diff options
author | Wenchen Fan <wenchen@databricks.com> | 2016-04-11 22:59:42 -0700 |
---|---|---|
committer | Yin Huai <yhuai@databricks.com> | 2016-04-11 22:59:42 -0700 |
commit | 678b96e77bf77a64b8df14b19db5a3bb18febfe3 (patch) | |
tree | 1052c9b1428b03620e088b0c0524df6191b94605 /sql/hive/src/main/scala/org/apache | |
parent | 52a801124f429ab133f9a3867c1da6ebd8fa7d4e (diff) | |
download | spark-678b96e77bf77a64b8df14b19db5a3bb18febfe3.tar.gz spark-678b96e77bf77a64b8df14b19db5a3bb18febfe3.tar.bz2 spark-678b96e77bf77a64b8df14b19db5a3bb18febfe3.zip |
[SPARK-14535][SQL] Remove buildInternalScan from FileFormat
## What changes were proposed in this pull request?
Now `HadoopFsRelation` with all kinds of file formats can be handled in `FileSourceStrategy`, we can remove the branches for `HadoopFsRelation` in `FileSourceStrategy` and the `buildInternalScan` API from `FileFormat`.
## How was this patch tested?
existing tests.
Author: Wenchen Fan <wenchen@databricks.com>
Closes #12300 from cloud-fan/remove.
Diffstat (limited to 'sql/hive/src/main/scala/org/apache')
-rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala | 13 |
1 files changed, 0 insertions, 13 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala index 43f445edcb..e915f3dfe2 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/orc/OrcRelation.scala @@ -111,19 +111,6 @@ private[sql] class DefaultSource } } - override def buildInternalScan( - sqlContext: SQLContext, - dataSchema: StructType, - requiredColumns: Array[String], - filters: Array[Filter], - bucketSet: Option[BitSet], - inputFiles: Seq[FileStatus], - broadcastedConf: Broadcast[SerializableConfiguration], - options: Map[String, String]): RDD[InternalRow] = { - val output = StructType(requiredColumns.map(dataSchema(_))).toAttributes - OrcTableScan(sqlContext, output, filters, inputFiles).execute() - } - override def buildReader( sqlContext: SQLContext, dataSchema: StructType, |