aboutsummaryrefslogtreecommitdiff
path: root/streaming
diff options
context:
space:
mode:
authorDongjoon Hyun <dongjoon@apache.org>2016-07-07 17:47:29 -0700
committerShivaram Venkataraman <shivaram@cs.berkeley.edu>2016-07-07 17:47:29 -0700
commit6aa7d09f4e126f42e41085dec169c813379ed354 (patch)
treecbbf96af27d5047087e6ce01b2b1b5fb0956a88c /streaming
parentf4767bcc7a9d1bdd301f054776aa45e7c9f344a7 (diff)
downloadspark-6aa7d09f4e126f42e41085dec169c813379ed354.tar.gz
spark-6aa7d09f4e126f42e41085dec169c813379ed354.tar.bz2
spark-6aa7d09f4e126f42e41085dec169c813379ed354.zip
[SPARK-16425][R] `describe()` should not fail with non-numeric columns
## What changes were proposed in this pull request? This PR prevents ERRORs when `summary(df)` is called for `SparkDataFrame` with not-numeric columns. This failure happens only in `SparkR`. **Before** ```r > df <- createDataFrame(faithful) > df <- withColumn(df, "boolean", df$waiting==79) > summary(df) 16/07/07 14:15:16 ERROR RBackendHandler: describe on 34 failed Error in invokeJava(isStatic = FALSE, objId$id, methodName, ...) : org.apache.spark.sql.AnalysisException: cannot resolve 'avg(`boolean`)' due to data type mismatch: function average requires numeric types, not BooleanType; ``` **After** ```r > df <- createDataFrame(faithful) > df <- withColumn(df, "boolean", df$waiting==79) > summary(df) SparkDataFrame[summary:string, eruptions:string, waiting:string] ``` ## How was this patch tested? Pass the Jenkins with a updated testcase. Author: Dongjoon Hyun <dongjoon@apache.org> Closes #14096 from dongjoon-hyun/SPARK-16425.
Diffstat (limited to 'streaming')
0 files changed, 0 insertions, 0 deletions