diff options
| author | Davies Liu <davies@databricks.com> | 2016-04-09 00:37:55 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-04-09 00:37:55 -0700 |
| commit | 90c0a04506a4972b7a2ac2b7dda0c5f8509a6e2f (patch) | |
| tree | bb0c38896d7b02e4dd612a68f22bcc982d383d08 /mllib/src | |
| parent | 520dde48d0d52dbbbbe1710a3275fdd5355dd69d (diff) | |
| download | spark-90c0a04506a4972b7a2ac2b7dda0c5f8509a6e2f.tar.gz spark-90c0a04506a4972b7a2ac2b7dda0c5f8509a6e2f.tar.bz2 spark-90c0a04506a4972b7a2ac2b7dda0c5f8509a6e2f.zip | |
[SPARK-14419] [SQL] Improve HashedRelation for key fit within Long
## What changes were proposed in this pull request?
Currently, we use java HashMap for HashedRelation if the key could fit within a Long. The java HashMap and CompactBuffer are not memory efficient, the memory used by them is also accounted accurately.
This PR introduce a LongToUnsafeRowMap (similar to BytesToBytesMap) for better memory efficiency and performance.
## How was this patch tested?
Updated existing tests.
Author: Davies Liu <davies@databricks.com>
Closes #12190 from davies/long_map2.
Diffstat (limited to 'mllib/src')
0 files changed, 0 insertions, 0 deletions