diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2015-02-13 17:45:31 -0800 |
|---|---|---|
| committer | Patrick Wendell <patrick@databricks.com> | 2015-02-13 17:45:31 -0800 |
| commit | d06d5ee9b33505774ef1e5becc01b47492f1a2dc (patch) | |
| tree | a304948c135092ef296ad89226121adbc04452cb /python/docs/pyspark.sql.rst | |
| parent | 4f4c6d5a5db04a56906bacdc85d7e5589b6edada (diff) | |
| download | spark-d06d5ee9b33505774ef1e5becc01b47492f1a2dc.tar.gz spark-d06d5ee9b33505774ef1e5becc01b47492f1a2dc.tar.bz2 spark-d06d5ee9b33505774ef1e5becc01b47492f1a2dc.zip | |
[SPARK-5227] [SPARK-5679] Disable FileSystem cache in WholeTextFileRecordReaderSuite
This patch fixes two difficult-to-reproduce Jenkins test failures in InputOutputMetricsSuite (SPARK-5227 and SPARK-5679). The problem was that WholeTextFileRecordReaderSuite modifies the `fs.local.block.size` Hadoop configuration and this change was affecting subsequent test suites due to Hadoop's caching of FileSystem instances (see HADOOP-8490 for more details).
The fix implemented here is to disable FileSystem caching in WholeTextFileRecordReaderSuite.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #4599 from JoshRosen/inputoutputsuite-fix and squashes the following commits:
47dc447 [Josh Rosen] [SPARK-5227] [SPARK-5679] Disable FileSystem cache in WholeTextFileRecordReaderSuite
Diffstat (limited to 'python/docs/pyspark.sql.rst')
0 files changed, 0 insertions, 0 deletions