diff options
| author | Andrew Or <andrew@databricks.com> | 2015-05-08 17:15:10 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2015-05-08 17:15:10 -0700 |
| commit | bd61f07039064833108070e19b752d4c46045766 (patch) | |
| tree | 439e3f0cca8250101fa4ef74a2e719f6ecd1db19 /python/pyspark/ml | |
| parent | ffdc40ce7a799f2564f57b958d0f32f1d1636488 (diff) | |
| download | spark-bd61f07039064833108070e19b752d4c46045766.tar.gz spark-bd61f07039064833108070e19b752d4c46045766.tar.bz2 spark-bd61f07039064833108070e19b752d4c46045766.zip | |
[SPARK-7469] [SQL] DAG visualization: show SQL query operators
The DAG visualization currently displays only low-level Spark primitives (e.g. `map`, `reduceByKey`, `filter` etc.). For SQL, these aren't particularly useful. Instead, we should display higher level physical operators (e.g. `Filter`, `Exchange`, `ShuffleHashJoin`). cc marmbrus
-----------------
**Before**
<img src="https://issues.apache.org/jira/secure/attachment/12731586/before.png" width="600px"/>
-----------------
**After** (Pay attention to the words)
<img src="https://issues.apache.org/jira/secure/attachment/12731587/after.png" width="600px"/>
-----------------
Author: Andrew Or <andrew@databricks.com>
Closes #5999 from andrewor14/dag-viz-sql and squashes the following commits:
0db23a4 [Andrew Or] Merge branch 'master' of github.com:apache/spark into dag-viz-sql
1e211db [Andrew Or] Update comment
0d49fd6 [Andrew Or] Merge branch 'master' of github.com:apache/spark into dag-viz-sql
ffd237a [Andrew Or] Fix style
202dac1 [Andrew Or] Make ignoreParent false by default
e61b1ab [Andrew Or] Visualize SQL operators, not low-level Spark primitives
569034a [Andrew Or] Add a flag to ignore parent settings and scopes
Diffstat (limited to 'python/pyspark/ml')
0 files changed, 0 insertions, 0 deletions