diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-10-13 15:09:31 -0700 |
|---|---|---|
| committer | Michael Armbrust <michael@databricks.com> | 2015-10-13 15:09:31 -0700 |
| commit | ef72673b234579c161b8cbb6cafc851d9eba1bfb (patch) | |
| tree | 15ed300794eba02ef46890d190e70b9c5ba546f6 /dev/create-release | |
| parent | 0d1b73b78b600420121ea8e58ff659ae8b4feebe (diff) | |
| download | spark-ef72673b234579c161b8cbb6cafc851d9eba1bfb.tar.gz spark-ef72673b234579c161b8cbb6cafc851d9eba1bfb.tar.bz2 spark-ef72673b234579c161b8cbb6cafc851d9eba1bfb.zip | |
[SPARK-11080] [SQL] Incorporate per-JVM id into ExprId to prevent unsafe cross-JVM comparisions
In the current implementation of named expressions' `ExprIds`, we rely on a per-JVM AtomicLong to ensure that expression ids are unique within a JVM. However, these expression ids will not be _globally_ unique. This opens the potential for id collisions if new expression ids happen to be created inside of tasks rather than on the driver.
There are currently a few cases where tasks allocate expression ids, which happen to be safe because those expressions are never compared to expressions created on the driver. In order to guard against the introduction of invalid comparisons between driver-created and executor-created expression ids, this patch extends `ExprId` to incorporate a UUID to identify the JVM that created the id, which prevents collisions.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #9093 from JoshRosen/SPARK-11080.
Diffstat (limited to 'dev/create-release')
0 files changed, 0 insertions, 0 deletions