diff options
| author | Imran Rashid <irashid@cloudera.com> | 2015-11-06 20:06:24 +0000 |
|---|---|---|
| committer | Sean Owen <sowen@cloudera.com> | 2015-11-06 20:06:24 +0000 |
| commit | 49f1a820372d1cba41f3f00d07eb5728f2ed6705 (patch) | |
| tree | 535797cc3662bfd7d8247b2d01f6fd00b2e1b2a9 /core/src/main | |
| parent | 62bb290773c9f9fa53cbe6d4eedc6e153761a763 (diff) | |
| download | spark-49f1a820372d1cba41f3f00d07eb5728f2ed6705.tar.gz spark-49f1a820372d1cba41f3f00d07eb5728f2ed6705.tar.bz2 spark-49f1a820372d1cba41f3f00d07eb5728f2ed6705.zip | |
[SPARK-10116][CORE] XORShiftRandom.hashSeed is random in high bits
https://issues.apache.org/jira/browse/SPARK-10116
This is really trivial, just happened to notice it -- if `XORShiftRandom.hashSeed` is really supposed to have random bits throughout (as the comment implies), it needs to do something for the conversion to `long`.
mengxr mkolod
Author: Imran Rashid <irashid@cloudera.com>
Closes #8314 from squito/SPARK-10116.
Diffstat (limited to 'core/src/main')
| -rw-r--r-- | core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala b/core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala index 85fb923cd9..e8cdb6e98b 100644 --- a/core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala +++ b/core/src/main/scala/org/apache/spark/util/random/XORShiftRandom.scala @@ -60,9 +60,11 @@ private[spark] class XORShiftRandom(init: Long) extends JavaRandom(init) { private[spark] object XORShiftRandom { /** Hash seeds to have 0/1 bits throughout. */ - private def hashSeed(seed: Long): Long = { + private[random] def hashSeed(seed: Long): Long = { val bytes = ByteBuffer.allocate(java.lang.Long.SIZE).putLong(seed).array() - MurmurHash3.bytesHash(bytes) + val lowBits = MurmurHash3.bytesHash(bytes) + val highBits = MurmurHash3.bytesHash(bytes, lowBits) + (highBits.toLong << 32) | (lowBits.toLong & 0xFFFFFFFFL) } /** |