Skip to content

[SPARK-4916][SQL][DOCS]Update SQL programming guide about cache section #3759

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 4 commits into from
Closed

[SPARK-4916][SQL][DOCS]Update SQL programming guide about cache section #3759

Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
99b2336
[SPARK-4916]Update SQL programming guide
luogankun Dec 22, 2014
0b93785
[SPARK-4916]Update SQL programming guide
luogankun Dec 22, 2014
6018122
Merge branch 'master' of https://github.com/apache/spark into SPARK-4916
luogankun Dec 23, 2014
7b39864
[SPARK-4916]Update SQL programming guide
luogankun Dec 23, 2014
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 1 addition & 4 deletions docs/sql-programming-guide.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -831,13 +831,10 @@ turning on some experimental options.

## Caching Data In Memory

Spark SQL can cache tables using an in-memory columnar format by calling `sqlContext.cacheTable("tableName")`.
Spark SQL can cache tables using an in-memory columnar format by calling `sqlContext.cacheTable("tableName")` or `schemaRDD.cache()`.
Then Spark SQL will scan only required columns and will automatically tune compression to minimize
memory usage and GC pressure. You can call `sqlContext.uncacheTable("tableName")` to remove the table from memory.

Note that if you call `schemaRDD.cache()` rather than `sqlContext.cacheTable(...)`, tables will _not_ be cached using
the in-memory columnar format, and therefore `sqlContext.cacheTable(...)` is strongly recommended for this use case.

Configuration of in-memory caching can be done using the `setConf` method on SQLContext or by running
`SET key=value` commands using SQL.

Expand Down