diff options
| author | Reynold Xin <rxin@databricks.com> | 2015-10-08 17:25:14 -0700 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2015-10-08 17:25:14 -0700 |
| commit | 84ea287178247c163226e835490c9c70b17d8d3b (patch) | |
| tree | 1e01cf3ce4db65842b0685d55e954e089a8ddf68 /sql/hive-thriftserver | |
| parent | 02149ff08eed3745086589a047adbce9a580389f (diff) | |
| download | spark-84ea287178247c163226e835490c9c70b17d8d3b.tar.gz spark-84ea287178247c163226e835490c9c70b17d8d3b.tar.bz2 spark-84ea287178247c163226e835490c9c70b17d8d3b.zip | |
[SPARK-10914] UnsafeRow serialization breaks when two machines have different Oops size.
UnsafeRow contains 3 pieces of information when pointing to some data in memory (an object, a base offset, and length). When the row is serialized with Java/Kryo serialization, the object layout in memory can change if two machines have different pointer width (Oops in JVM).
To reproduce, launch Spark using
MASTER=local-cluster[2,1,1024] bin/spark-shell --conf "spark.executor.extraJavaOptions=-XX:-UseCompressedOops"
And then run the following
scala> sql("select 1 xx").collect()
Author: Reynold Xin <rxin@databricks.com>
Closes #9030 from rxin/SPARK-10914.
Diffstat (limited to 'sql/hive-thriftserver')
0 files changed, 0 insertions, 0 deletions