diff options
author | RJ Nowling <rnowling@gmail.com> | 2015-06-19 10:50:44 -0700 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-06-19 10:51:37 -0700 |
commit | a9858036bfd339b47dd6d2ed69ccbb61269c225e (patch) | |
tree | 71ffbfb3c2b9dc85087d20d53dd6c901ffd1d6d3 /streaming | |
parent | 0c32fc125c45e59f06cb55f3ba7da612d840ca86 (diff) | |
download | spark-a9858036bfd339b47dd6d2ed69ccbb61269c225e.tar.gz spark-a9858036bfd339b47dd6d2ed69ccbb61269c225e.tar.bz2 spark-a9858036bfd339b47dd6d2ed69ccbb61269c225e.zip |
Add example that reads a local file, writes to a DFS path provided by th...
...e user, reads the file back from the DFS, and compares word counts on the local and DFS versions. Useful for verifying DFS correctness.
Author: RJ Nowling <rnowling@gmail.com>
Closes #3347 from rnowling/dfs_read_write_test and squashes the following commits:
af8ccb7 [RJ Nowling] Don't use java.io.File since DFS may not be POSIX-compatible
b0ef9ea [RJ Nowling] Fix string style
07c6132 [RJ Nowling] Fix string style
7d9a8df [RJ Nowling] Fix string style
f74c160 [RJ Nowling] Fix else statement style
b9edf12 [RJ Nowling] Fix spark wc style
44415b9 [RJ Nowling] Fix local wc style
94a4691 [RJ Nowling] Fix space
df59b65 [RJ Nowling] Fix if statements
1b314f0 [RJ Nowling] Add scaladoc
a931d70 [RJ Nowling] Fix import order
0c89558 [RJ Nowling] Add example that reads a local file, writes to a DFS path provided by the user, reads the file back from the DFS, and compares word counts on the local and DFS versions. Useful for verifying DFS correctness.
Diffstat (limited to 'streaming')
0 files changed, 0 insertions, 0 deletions