diff options
author | Josh Rosen <joshrosen@databricks.com> | 2015-02-13 17:45:31 -0800 |
---|---|---|
committer | Patrick Wendell <patrick@databricks.com> | 2015-02-13 17:45:31 -0800 |
commit | d06d5ee9b33505774ef1e5becc01b47492f1a2dc (patch) | |
tree | a304948c135092ef296ad89226121adbc04452cb /python/docs/pyspark.sql.rst | |
parent | 4f4c6d5a5db04a56906bacdc85d7e5589b6edada (diff) | |
download | spark-d06d5ee9b33505774ef1e5becc01b47492f1a2dc.tar.gz spark-d06d5ee9b33505774ef1e5becc01b47492f1a2dc.tar.bz2 spark-d06d5ee9b33505774ef1e5becc01b47492f1a2dc.zip |
[SPARK-5227] [SPARK-5679] Disable FileSystem cache in WholeTextFileRecordReaderSuite
This patch fixes two difficult-to-reproduce Jenkins test failures in InputOutputMetricsSuite (SPARK-5227 and SPARK-5679). The problem was that WholeTextFileRecordReaderSuite modifies the `fs.local.block.size` Hadoop configuration and this change was affecting subsequent test suites due to Hadoop's caching of FileSystem instances (see HADOOP-8490 for more details).
The fix implemented here is to disable FileSystem caching in WholeTextFileRecordReaderSuite.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #4599 from JoshRosen/inputoutputsuite-fix and squashes the following commits:
47dc447 [Josh Rosen] [SPARK-5227] [SPARK-5679] Disable FileSystem cache in WholeTextFileRecordReaderSuite
Diffstat (limited to 'python/docs/pyspark.sql.rst')
0 files changed, 0 insertions, 0 deletions