diff options
| author | Dongjoon Hyun <dongjoon@apache.org> | 2016-06-28 13:01:18 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2016-06-28 13:01:18 +0800 |
| commit | a0da854fb3748aca0128377f0955600cb7a2b5bc (patch) | |
| tree | 6570664b9cd0c9cf53f6f4949484dabd6164c915 /docs/sql-programming-guide.md | |
| parent | 50fdd866b55cb9b51427095e56b2aafea12a7c23 (diff) | |
| download | spark-a0da854fb3748aca0128377f0955600cb7a2b5bc.tar.gz spark-a0da854fb3748aca0128377f0955600cb7a2b5bc.tar.bz2 spark-a0da854fb3748aca0128377f0955600cb7a2b5bc.zip | |
[SPARK-16221][SQL] Redirect Parquet JUL logger via SLF4J for WRITE operations
## What changes were proposed in this pull request?
[SPARK-8118](https://github.com/apache/spark/pull/8196) implements redirecting Parquet JUL logger via SLF4J, but it is currently applied only when READ operations occurs. If users use only WRITE operations, there occurs many Parquet logs.
This PR makes the redirection work on WRITE operations, too.
**Before**
```scala
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
Jun 26, 2016 9:04:38 PM INFO: org.apache.parquet.hadoop.codec.CodecConfig: Compression: SNAPPY
............ about 70 lines Parquet Log .............
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
............ about 70 lines Parquet Log .............
```
**After**
```scala
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
scala> spark.range(10).write.format("parquet").mode("overwrite").save("/tmp/p")
```
This PR also fixes some typos.
## How was this patch tested?
Manual.
Author: Dongjoon Hyun <dongjoon@apache.org>
Closes #13918 from dongjoon-hyun/SPARK-16221.
Diffstat (limited to 'docs/sql-programming-guide.md')
0 files changed, 0 insertions, 0 deletions