diff options
| author | Imran Rashid <irashid@cloudera.com> | 2016-09-29 15:40:35 -0700 |
|---|---|---|
| committer | Marcelo Vanzin <vanzin@cloudera.com> | 2016-09-29 15:40:35 -0700 |
| commit | 3993ebca23afa4b8770695051635933a6c9d2c11 (patch) | |
| tree | 5e5918efa9e804721591f06f29b79b29f8e6c1d7 /docs | |
| parent | 29396e7d1483d027960b9a1bed47008775c4253e (diff) | |
| download | spark-3993ebca23afa4b8770695051635933a6c9d2c11.tar.gz spark-3993ebca23afa4b8770695051635933a6c9d2c11.tar.bz2 spark-3993ebca23afa4b8770695051635933a6c9d2c11.zip | |
[SPARK-17676][CORE] FsHistoryProvider should ignore hidden files
## What changes were proposed in this pull request?
FsHistoryProvider was writing a hidden file (to check the fs's clock).
Even though it deleted the file immediately, sometimes another thread
would try to scan the files on the fs in-between, and then there would
be an error msg logged which was very misleading for the end-user.
(The logged error was harmless, though.)
## How was this patch tested?
I added one unit test, but to be clear, that test was passing before. The actual change in behavior in that test is just logging (after the change, there is no more logged error), which I just manually verified.
Author: Imran Rashid <irashid@cloudera.com>
Closes #15250 from squito/SPARK-17676.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions