diff options
| author | Cheng Lian <lian.cs.zju@gmail.com> | 2014-06-23 13:24:33 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-06-23 13:24:47 -0700 |
| commit | 6d821f0c073829ed6b89bd184f6f41ee95d8fdec (patch) | |
| tree | 7d9884ab142e4f56ddbf730f8eb2a1074840429c /sql/hive | |
| parent | cf2fa4fe344102d7d8e492efa87de2cb0db7f507 (diff) | |
| download | spark-6d821f0c073829ed6b89bd184f6f41ee95d8fdec.tar.gz spark-6d821f0c073829ed6b89bd184f6f41ee95d8fdec.tar.bz2 spark-6d821f0c073829ed6b89bd184f6f41ee95d8fdec.zip | |
[SPARK-1669][SQL] Made cacheTable idempotent
JIRA issue: [SPARK-1669](https://issues.apache.org/jira/browse/SPARK-1669)
Caching the same table multiple times should end up with only 1 in-memory columnar representation of this table.
Before:
```
scala> loadTestTable("src")
...
scala> cacheTable("src")
...
scala> cacheTable("src")
...
scala> table("src")
...
== Query Plan ==
InMemoryColumnarTableScan [key#2,value#3], (InMemoryRelation [key#2,value#3], false, (InMemoryColumnarTableScan [key#2,value#3], (InMemoryRelation [key#2,value#3], false, (HiveTableScan [key#2,value#3], (MetastoreRelation default, src, None), None))))
```
After:
```
scala> loadTestTable("src")
...
scala> cacheTable("src")
...
scala> cacheTable("src")
...
scala> table("src")
...
== Query Plan ==
InMemoryColumnarTableScan [key#2,value#3], (InMemoryRelation [key#2,value#3], false, (HiveTableScan [key#2,value#3], (MetastoreRelation default, src, None), None))
```
Author: Cheng Lian <lian.cs.zju@gmail.com>
Closes #1183 from liancheng/spark-1669 and squashes the following commits:
68f8a20 [Cheng Lian] Removed an unused import
51bae90 [Cheng Lian] Made cacheTable idempotent
(cherry picked from commit a4bc442ca2c35444de8a33376b6f27c6c2a9003d)
Signed-off-by: Michael Armbrust <michael@databricks.com>
Diffstat (limited to 'sql/hive')
0 files changed, 0 insertions, 0 deletions