diff options
| author | Patrick Wendell <pwendell@gmail.com> | 2014-04-27 17:40:56 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-04-27 17:40:56 -0700 |
| commit | 6b3c6e5dd8e74435f71ecdb224db532550ef407b (patch) | |
| tree | 9ff06e8ce1d8dd01b2ccb5e4820adbdd02310298 /docs/api.md | |
| parent | 3d9fb09681308abd2066d0d02f2438f5a17c9dd9 (diff) | |
| download | spark-6b3c6e5dd8e74435f71ecdb224db532550ef407b.tar.gz spark-6b3c6e5dd8e74435f71ecdb224db532550ef407b.tar.bz2 spark-6b3c6e5dd8e74435f71ecdb224db532550ef407b.zip | |
SPARK-1145: Memory mapping with many small blocks can cause JVM allocation failures
This includes some minor code clean-up as well. The main change is that small files are not memory mapped. There is a nicer way to write that code block using Scala's `Try` but to make it easy to back port and as simple as possible, I opted for the more explicit but less pretty format.
Author: Patrick Wendell <pwendell@gmail.com>
Closes #43 from pwendell/block-iter-logging and squashes the following commits:
1cff512 [Patrick Wendell] Small issue from merge.
49f6c269 [Patrick Wendell] Merge remote-tracking branch 'apache/master' into block-iter-logging
4943351 [Patrick Wendell] Added a test and feedback on mateis review
a637a18 [Patrick Wendell] Review feedback and adding rewind() when reading byte buffers.
b76b95f [Patrick Wendell] Review feedback
4e1514e [Patrick Wendell] Don't memory map for small files
d238b88 [Patrick Wendell] Some logging and clean-up
Diffstat (limited to 'docs/api.md')
0 files changed, 0 insertions, 0 deletions