diff options
| author | scwf <wangfei1@huawei.com> | 2014-10-26 16:56:03 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-10-26 16:56:11 -0700 |
| commit | f4e8c289d899fc03ee3cebd01d810f207868b448 (patch) | |
| tree | c00cee929eeb314da24fde64ee1d283251d62f25 /sql/hive | |
| parent | 3a9d66cf59ab7c9aee090e4c6067c73510e2ac26 (diff) | |
| download | spark-f4e8c289d899fc03ee3cebd01d810f207868b448.tar.gz spark-f4e8c289d899fc03ee3cebd01d810f207868b448.tar.bz2 spark-f4e8c289d899fc03ee3cebd01d810f207868b448.zip | |
[SPARK-4042][SQL] Append columns ids and names before broadcast
Append columns ids and names before broadcast ```hiveExtraConf``` in ```HadoopTableReader```.
Author: scwf <wangfei1@huawei.com>
Closes #2885 from scwf/HadoopTableReader and squashes the following commits:
a8c498c [scwf] append columns ids and names before broadcast
Diffstat (limited to 'sql/hive')
| -rw-r--r-- | sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala | 5 |
1 files changed, 3 insertions, 2 deletions
diff --git a/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala b/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala index 85965a6ea0..b7f3ade4ea 100644 --- a/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala +++ b/sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/HiveTableScan.scala @@ -68,6 +68,9 @@ case class HiveTableScan( @transient private[this] val hiveExtraConf = new HiveConf(context.hiveconf) + // append columns ids and names before broadcast + addColumnMetadataToConf(hiveExtraConf) + @transient private[this] val hadoopReader = new HadoopTableReader(attributes, relation, context, hiveExtraConf) @@ -105,8 +108,6 @@ case class HiveTableScan( hiveConf.set(serdeConstants.LIST_COLUMNS, relation.attributes.map(_.name).mkString(",")) } - addColumnMetadataToConf(hiveExtraConf) - /** * Prunes partitions not involve the query plan. * |