aboutsummaryrefslogtreecommitdiff
path: root/streaming/src/test
diff options
context:
space:
mode:
authorSean Owen <sowen@cloudera.com>2014-04-11 22:46:47 -0700
committerPatrick Wendell <pwendell@gmail.com>2014-04-11 22:47:17 -0700
commit4dfcb3860803d8d164ef113012c0720cbd35c2be (patch)
tree407aa7d706a64eb45f7e4d02f3f6db640b130b28 /streaming/src/test
parentdac6240cf3e5a007f27d6e765198263f993ff040 (diff)
downloadspark-4dfcb3860803d8d164ef113012c0720cbd35c2be.tar.gz
spark-4dfcb3860803d8d164ef113012c0720cbd35c2be.tar.bz2
spark-4dfcb3860803d8d164ef113012c0720cbd35c2be.zip
SPARK-1057 (alternative) Remove fastutil
(This is for discussion at this point -- I'm not suggesting this should be committed.) This is what removing fastutil looks like. Much of it is straightforward, like using `java.io` buffered stream classes, and Guava for murmurhash3. Uses of the `FastByteArrayOutputStream` were a little trickier. In only one case though do I think the change to use `java.io` actually entails an extra array copy. The rest is using `OpenHashMap` and `OpenHashSet`. These are now written in terms of more scala-like operations. `OpenHashMap` is where I made three non-trivial changes to make it work, and they need review: - It is no longer private - The key must be a `ClassTag` - Unless a lot of other code changes, the key type can't enforce being a supertype of `Null` It all works and tests pass, and I think there is reason to believe it's OK from a speed perspective. But what about those last changes? Author: Sean Owen <sowen@cloudera.com> Closes #266 from srowen/SPARK-1057-alternate and squashes the following commits: 2601129 [Sean Owen] Fix Map return type error not previously caught ec65502 [Sean Owen] Updates from matei's review 00bc81e [Sean Owen] Remove use of fastutil and replace with use of java.io, spark.util and Guava classes (cherry picked from commit 165e06a74c3d75e6b7341c120943add8b035b96a) Signed-off-by: Patrick Wendell <pwendell@gmail.com>
Diffstat (limited to 'streaming/src/test')
0 files changed, 0 insertions, 0 deletions