diff options
author | Calvin Jia <jia.calvin@gmail.com> | 2015-05-25 16:50:43 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2015-05-25 16:50:43 -0700 |
commit | ce0051d6f7e4ed54076676644c78b52b527ba190 (patch) | |
tree | 9ba6ff8402f4346f7e770e29034a35931363e659 /docs | |
parent | 8af1bf10b70b9b67f18f618174e84365d69caa48 (diff) | |
download | spark-ce0051d6f7e4ed54076676644c78b52b527ba190.tar.gz spark-ce0051d6f7e4ed54076676644c78b52b527ba190.tar.bz2 spark-ce0051d6f7e4ed54076676644c78b52b527ba190.zip |
[SPARK-6391][DOCS] Document Tachyon compatibility.
Adds a section in the RDD persistence section of the programming-guide docs detailing Spark-Tachyon version compatibility as discussed in [[SPARK-6391]](https://issues.apache.org/jira/browse/SPARK-6391).
Author: Calvin Jia <jia.calvin@gmail.com>
Closes #6382 from calvinjia/spark-6391 and squashes the following commits:
113e863 [Calvin Jia] Move compatibility info to the offheap storage level section.
7942dc5 [Calvin Jia] Add a section in the programming-guide docs for Tachyon compatibility.
Diffstat (limited to 'docs')
-rw-r--r-- | docs/programming-guide.md | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/docs/programming-guide.md b/docs/programming-guide.md index 5d9df282ef..10f474f237 100644 --- a/docs/programming-guide.md +++ b/docs/programming-guide.md @@ -1214,9 +1214,11 @@ storage levels is: Compared to MEMORY_ONLY_SER, OFF_HEAP reduces garbage collection overhead and allows executors to be smaller and to share a pool of memory, making it attractive in environments with large heaps or multiple concurrent applications. Furthermore, as the RDDs reside in Tachyon, - the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory + the crash of an executor does not lead to losing the in-memory cache. In this mode, the memory in Tachyon is discardable. Thus, Tachyon does not attempt to reconstruct a block that it evicts - from memory. + from memory. If you plan to use Tachyon as the off heap store, Spark is compatible with Tachyon + out-of-the-box. Please refer to this <a href="http://tachyon-project.org/master/Running-Spark-on-Tachyon.html">page</a> + for the suggested version pairings. </td> </tr> </table> |