diff options
author | Sean Zhong <seanzhong@databricks.com> | 2016-06-06 22:59:25 -0700 |
---|---|---|
committer | Cheng Lian <lian@databricks.com> | 2016-06-06 22:59:25 -0700 |
commit | 5f731d6859c4516941e5f90c99c966ef76268864 (patch) | |
tree | db6497a87bd37235bde5cd7196edc9faf6ce4505 /docs | |
parent | 0e0904a2fce3c4447c24f1752307b6d01ffbd0ad (diff) | |
download | spark-5f731d6859c4516941e5f90c99c966ef76268864.tar.gz spark-5f731d6859c4516941e5f90c99c966ef76268864.tar.bz2 spark-5f731d6859c4516941e5f90c99c966ef76268864.zip |
[SPARK-15792][SQL] Allows operator to change the verbosity in explain output
## What changes were proposed in this pull request?
This PR allows customization of verbosity in explain output. After change, `dataframe.explain()` and `dataframe.explain(true)` has different verbosity output for physical plan.
Currently, this PR only enables verbosity string for operator `HashAggregateExec` and `SortAggregateExec`. We will gradually enable verbosity string for more operators in future.
**Less verbose mode:** dataframe.explain(extended = false)
`output=[count(a)#85L]` is **NOT** displayed for HashAggregate.
```
scala> Seq((1,2,3)).toDF("a", "b", "c").createTempView("df2")
scala> spark.sql("select count(a) from df2").explain()
== Physical Plan ==
*HashAggregate(key=[], functions=[count(1)])
+- Exchange SinglePartition
+- *HashAggregate(key=[], functions=[partial_count(1)])
+- LocalTableScan
```
**Verbose mode:** dataframe.explain(extended = true)
`output=[count(a)#85L]` is displayed for HashAggregate.
```
scala> spark.sql("select count(a) from df2").explain(true) // "output=[count(a)#85L]" is added
...
== Physical Plan ==
*HashAggregate(key=[], functions=[count(1)], output=[count(a)#85L])
+- Exchange SinglePartition
+- *HashAggregate(key=[], functions=[partial_count(1)], output=[count#87L])
+- LocalTableScan
```
## How was this patch tested?
Manual test.
Author: Sean Zhong <seanzhong@databricks.com>
Closes #13535 from clockfly/verbose_breakdown_2.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions