aboutsummaryrefslogtreecommitdiff
path: root/docs
diff options
context:
space:
mode:
authorSean Zhong <seanzhong@databricks.com>2016-06-06 22:59:25 -0700
committerCheng Lian <lian@databricks.com>2016-06-06 22:59:25 -0700
commit5f731d6859c4516941e5f90c99c966ef76268864 (patch)
treedb6497a87bd37235bde5cd7196edc9faf6ce4505 /docs
parent0e0904a2fce3c4447c24f1752307b6d01ffbd0ad (diff)
downloadspark-5f731d6859c4516941e5f90c99c966ef76268864.tar.gz
spark-5f731d6859c4516941e5f90c99c966ef76268864.tar.bz2
spark-5f731d6859c4516941e5f90c99c966ef76268864.zip
[SPARK-15792][SQL] Allows operator to change the verbosity in explain output
## What changes were proposed in this pull request? This PR allows customization of verbosity in explain output. After change, `dataframe.explain()` and `dataframe.explain(true)` has different verbosity output for physical plan. Currently, this PR only enables verbosity string for operator `HashAggregateExec` and `SortAggregateExec`. We will gradually enable verbosity string for more operators in future. **Less verbose mode:** dataframe.explain(extended = false) `output=[count(a)#85L]` is **NOT** displayed for HashAggregate. ``` scala> Seq((1,2,3)).toDF("a", "b", "c").createTempView("df2") scala> spark.sql("select count(a) from df2").explain() == Physical Plan == *HashAggregate(key=[], functions=[count(1)]) +- Exchange SinglePartition +- *HashAggregate(key=[], functions=[partial_count(1)]) +- LocalTableScan ``` **Verbose mode:** dataframe.explain(extended = true) `output=[count(a)#85L]` is displayed for HashAggregate. ``` scala> spark.sql("select count(a) from df2").explain(true) // "output=[count(a)#85L]" is added ... == Physical Plan == *HashAggregate(key=[], functions=[count(1)], output=[count(a)#85L]) +- Exchange SinglePartition +- *HashAggregate(key=[], functions=[partial_count(1)], output=[count#87L]) +- LocalTableScan ``` ## How was this patch tested? Manual test. Author: Sean Zhong <seanzhong@databricks.com> Closes #13535 from clockfly/verbose_breakdown_2.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions