diff options
| author | luogankun <luogankun@gmail.com> | 2014-12-30 12:17:49 -0800 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2014-12-30 12:17:49 -0800 |
| commit | f7a41a0e79561a722e41800257dca886732ccaad (patch) | |
| tree | bc927054fe749da81467fd8c30c4e1d78745f13f /docs/sql-programming-guide.md | |
| parent | 19a8802e703e6b075a148ba73dc9dd80748d6322 (diff) | |
| download | spark-f7a41a0e79561a722e41800257dca886732ccaad.tar.gz spark-f7a41a0e79561a722e41800257dca886732ccaad.tar.bz2 spark-f7a41a0e79561a722e41800257dca886732ccaad.zip | |
[SPARK-4916][SQL][DOCS]Update SQL programming guide about cache section
`SchemeRDD.cache()` now uses in-memory columnar storage.
Author: luogankun <luogankun@gmail.com>
Closes #3759 from luogankun/SPARK-4916 and squashes the following commits:
7b39864 [luogankun] [SPARK-4916]Update SQL programming guide
6018122 [luogankun] Merge branch 'master' of https://github.com/apache/spark into SPARK-4916
0b93785 [luogankun] [SPARK-4916]Update SQL programming guide
99b2336 [luogankun] [SPARK-4916]Update SQL programming guide
Diffstat (limited to 'docs/sql-programming-guide.md')
| -rw-r--r-- | docs/sql-programming-guide.md | 5 |
1 files changed, 1 insertions, 4 deletions
diff --git a/docs/sql-programming-guide.md b/docs/sql-programming-guide.md index 2aea8a8aed..1b5fde991e 100644 --- a/docs/sql-programming-guide.md +++ b/docs/sql-programming-guide.md @@ -831,13 +831,10 @@ turning on some experimental options. ## Caching Data In Memory -Spark SQL can cache tables using an in-memory columnar format by calling `sqlContext.cacheTable("tableName")`. +Spark SQL can cache tables using an in-memory columnar format by calling `sqlContext.cacheTable("tableName")` or `schemaRDD.cache()`. Then Spark SQL will scan only required columns and will automatically tune compression to minimize memory usage and GC pressure. You can call `sqlContext.uncacheTable("tableName")` to remove the table from memory. -Note that if you call `schemaRDD.cache()` rather than `sqlContext.cacheTable(...)`, tables will _not_ be cached using -the in-memory columnar format, and therefore `sqlContext.cacheTable(...)` is strongly recommended for this use case. - Configuration of in-memory caching can be done using the `setConf` method on SQLContext or by running `SET key=value` commands using SQL. |