aboutsummaryrefslogtreecommitdiff
path: root/core/src/main/resources
diff options
context:
space:
mode:
authorNick Lavers <nick.lavers@videoamp.com>2017-01-17 12:14:38 +0000
committerSean Owen <sowen@cloudera.com>2017-01-17 12:14:38 +0000
commit0019005a2d0f150fd00ad926d054a8beca4bbd68 (patch)
tree5e51517dad00b0d4848206cce57c51774805933d /core/src/main/resources
parenta774bca05ec6dd0deec638048dc8672a84427f49 (diff)
downloadspark-0019005a2d0f150fd00ad926d054a8beca4bbd68.tar.gz
spark-0019005a2d0f150fd00ad926d054a8beca4bbd68.tar.bz2
spark-0019005a2d0f150fd00ad926d054a8beca4bbd68.zip
[SPARK-19219][SQL] Fix Parquet log output defaults
## What changes were proposed in this pull request? Changing the default parquet logging levels to reflect the changes made in PR [#15538](https://github.com/apache/spark/pull/15538), in order to prevent the flood of log messages by default. ## How was this patch tested? Default log output when reading from parquet 1.6 files was compared with and without this change. The change eliminates the extraneous logging and makes the output readable. Author: Nick Lavers <nick.lavers@videoamp.com> Closes #16580 from nicklavers/spark-19219-set_default_parquet_log_level.
Diffstat (limited to 'core/src/main/resources')
-rw-r--r--core/src/main/resources/org/apache/spark/log4j-defaults.properties4
1 files changed, 4 insertions, 0 deletions
diff --git a/core/src/main/resources/org/apache/spark/log4j-defaults.properties b/core/src/main/resources/org/apache/spark/log4j-defaults.properties
index 89a7963a86..2770100150 100644
--- a/core/src/main/resources/org/apache/spark/log4j-defaults.properties
+++ b/core/src/main/resources/org/apache/spark/log4j-defaults.properties
@@ -36,3 +36,7 @@ log4j.logger.org.apache.spark.repl.SparkILoop$SparkILoopInterpreter=INFO
# SPARK-9183: Settings to avoid annoying messages when looking up nonexistent UDFs in SparkSQL with Hive support
log4j.logger.org.apache.hadoop.hive.metastore.RetryingHMSHandler=FATAL
log4j.logger.org.apache.hadoop.hive.ql.exec.FunctionRegistry=ERROR
+
+# Parquet related logging
+log4j.logger.org.apache.parquet.CorruptStatistics=ERROR
+log4j.logger.parquet.CorruptStatistics=ERROR