diff options
| author | Tathagata Das <tathagata.das1565@gmail.com> | 2016-07-07 23:19:41 -0700 |
|---|---|---|
| committer | Tathagata Das <tathagata.das1565@gmail.com> | 2016-07-07 23:19:41 -0700 |
| commit | 5bce4580939c27876f11cd75f0dc2190fb9fa908 (patch) | |
| tree | edd49dfefae6130144fb42d96cea11c252d40725 /python | |
| parent | 6aa7d09f4e126f42e41085dec169c813379ed354 (diff) | |
| download | spark-5bce4580939c27876f11cd75f0dc2190fb9fa908.tar.gz spark-5bce4580939c27876f11cd75f0dc2190fb9fa908.tar.bz2 spark-5bce4580939c27876f11cd75f0dc2190fb9fa908.zip | |
[SPARK-16430][SQL][STREAMING] Add option maxFilesPerTrigger
## What changes were proposed in this pull request?
An option that limits the file stream source to read 1 file at a time enables rate limiting. It has the additional convenience that a static set of files can be used like a stream for testing as this will allows those files to be considered one at a time.
This PR adds option `maxFilesPerTrigger`.
## How was this patch tested?
New unit test
Author: Tathagata Das <tathagata.das1565@gmail.com>
Closes #14094 from tdas/SPARK-16430.
Diffstat (limited to 'python')
0 files changed, 0 insertions, 0 deletions