aboutsummaryrefslogtreecommitdiff
path: root/streaming
Commit message (Collapse)AuthorAgeFilesLines
* [maven-release-plugin] prepare release v1.0.2-rc1v1.0.2Tathagata Das2014-07-251-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.2-rc1"Tathagata Das2014-07-251-1/+1
| | | | This reverts commit 08f601328ad9e7334ef7deb3a9fff1343a3c4f30.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-07-251-1/+1
| | | | This reverts commit 54df1b8c31fa2de5b04ee4a5563706b2664f34f3.
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-07-251-1/+1
|
* [maven-release-plugin] prepare release v1.0.2-rc1Tathagata Das2014-07-251-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.2-rc1"Tathagata Das2014-07-251-1/+1
| | | | This reverts commit 919c87f26a2655bfd5ae03958915b6804367c1d6.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-07-251-1/+1
| | | | This reverts commit edbd02fc6873676e080101d407916efb64bdf71a.
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-07-251-1/+1
|
* [maven-release-plugin] prepare release v1.0.2-rc1Tathagata Das2014-07-251-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.1-rc3"Tathagata Das2014-07-251-1/+1
| | | | This reverts commit 70ee14f76d6c3d3f162db6bbe12797c252a0295a.
* Revert "[maven-release-plugin] prepare for next development iteration"Ubuntu2014-07-251-1/+1
| | | | This reverts commit baf92a0f2119867b1be540085ebe9f1a1c411ae8.
* [maven-release-plugin] prepare for next development iterationUbuntu2014-07-141-1/+1
|
* [maven-release-plugin] prepare release v1.0.1-rc3Ubuntu2014-07-141-1/+1
|
* [SPARK-1341] [Streaming] Throttle BlockGenerator to limit rate of data ↵Issac Buenrostro2014-07-103-1/+109
| | | | | | | | | | | | | | | | consumption. Author: Issac Buenrostro <buenrostro@ooyala.com> Closes #945 from ibuenros/SPARK-1341-throttle and squashes the following commits: 5514916 [Issac Buenrostro] Formatting changes, added documentation for streaming throttling, stricter unit tests for throttling. 62f395f [Issac Buenrostro] Add comments and license to streaming RateLimiter.scala 7066438 [Issac Buenrostro] Moved throttle code to RateLimiter class, smoother pushing when throttling active ccafe09 [Issac Buenrostro] Throttle BlockGenerator to limit rate of data consumption. (cherry picked from commit 2dd67248503306bb08946b1796821e9f9ed4d00e) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [STREAMING] SPARK-2343: Fix QueueInputDStream with oneAtATime falseManuel Laflamme2014-07-092-2/+92
| | | | | | | | | | | | | | Fix QueueInputDStream which was not removing dequeued items when used with the oneAtATime flag disabled. Author: Manuel Laflamme <manuel.laflamme@gmail.com> Closes #1285 from mlaflamm/spark-2343 and squashes the following commits: 61c9e38 [Manuel Laflamme] Unit tests for queue input stream c51d029 [Manuel Laflamme] Fix QueueInputDStream with oneAtATime false (cherry picked from commit 0eb11527d13083ced215e3fda44ed849198a57cb) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [SPARK-2362] Fix for newFilesOnly logic in file DStreamGabriele Nizzoli2014-07-081-1/+1
| | | | | | | | | | | | | The newFilesOnly logic should be inverted: the logic should be that if the flag newFilesOnly==true then only start reading files older than current time. As the code is now if newFilesOnly==true then it will start to read files that are older than 0L (that is: every file in the directory). Author: Gabriele Nizzoli <mail@nizzoli.net> Closes #1077 from gabrielenizzoli/master and squashes the following commits: 4f1d261 [Gabriele Nizzoli] Fix for newFilesOnly logic in file DStream (cherry picked from commit e6f7bfcfbf6aff7a9f8cd8e0a2166d0bf62b0912) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [maven-release-plugin] prepare for next development iterationUbuntu2014-07-041-1/+1
|
* [maven-release-plugin] prepare release v1.0.1-rc2v1.0.1Ubuntu2014-07-041-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.1-rc1"Patrick Wendell2014-06-271-1/+1
| | | | This reverts commit 7feeda3d729f9397aa15ee8750c01ef5aa601962.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-06-271-1/+1
| | | | This reverts commit ea1a455a755f83f46fc8bf242410917d93d0c52c.
* [maven-release-plugin] prepare for next development iterationUbuntu2014-06-261-1/+1
|
* [maven-release-plugin] prepare release v1.0.1-rc1Ubuntu2014-06-261-1/+1
|
* [STREAMING] SPARK-2009 Key not found exception when slow receiver startsVadim Chekan2014-06-171-1/+1
| | | | | | | | | | | | | | | | | | I got "java.util.NoSuchElementException: key not found: 1401756085000 ms" exception when using kafka stream and 1 sec batchPeriod. Investigation showed that the reason is that ReceiverLauncher.startReceivers is asynchronous (started in a thread). https://github.com/vchekan/spark/blob/master/streaming/src/main/scala/org/apache/spark/streaming/scheduler/ReceiverTracker.scala#L206 In case of slow starting receiver, such as Kafka, it easily takes more than 2sec to start. In result, no single "compute" will be called on ReceiverInputDStream before first batch job is executed and receivedBlockInfo remains empty (obviously). Batch job will cause ReceiverInputDStream.getReceivedBlockInfo call and "key not found" exception. The patch makes getReceivedBlockInfo more robust by tolerating missing values. Author: Vadim Chekan <kot.begemot@gmail.com> Closes #961 from vchekan/branch-1.0 and squashes the following commits: e86f82b [Vadim Chekan] Fixed indentation 4609563 [Vadim Chekan] Key not found exception: if receiver is slow to start, it is possible that getReceivedBlockInfo will be called before compute has been called
* [SQL] Update SparkSQL and ScalaTest in branch-1.0 to match master.Michael Armbrust2014-06-131-6/+6
| | | | | | | | | | | | | #511 and #863 got left out of branch-1.0 since we were really close to the release. Now that they have been tested a little I see no reason to leave them out. Author: Michael Armbrust <michael@databricks.com> Author: witgo <witgo@qq.com> Closes #1078 from marmbrus/branch-1.0 and squashes the following commits: 22be674 [witgo] [SPARK-1841]: update scalatest to version 2.1.5 fc8fc79 [Michael Armbrust] Include #1071 as well. c5d0adf [Michael Armbrust] Update SparkSQL in branch-1.0 to match master.
* SPARK-2113: awaitTermination() after stop() will hang in Spark StremaingLars Albertsson2014-06-112-0/+13
| | | | | | | | | | | Author: Lars Albertsson <lalle@spotify.com> Closes #1001 from lallea/contextwaiter_stopped and squashes the following commits: 93cd314 [Lars Albertsson] Mend StreamingContext stop() followed by awaitTermination(). (cherry picked from commit 4d5c12aa1c54c49377a4bafe3bcc4993d5e1a552) Signed-off-by: Patrick Wendell <pwendell@gmail.com>
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-05-261-1/+1
|
* [maven-release-plugin] prepare release v1.0.0-rc11v1.0.0Tathagata Das2014-05-261-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc11"Tathagata Das2014-05-261-1/+1
| | | | This reverts commit 2f1dc868e5714882cf40d2633fb66772baf34789.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-05-261-1/+1
| | | | This reverts commit 832dc594e7666f1d402334f8015ce29917d9c888.
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-05-251-1/+1
|
* [maven-release-plugin] prepare release v1.0.0-rc11Tathagata Das2014-05-251-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc10"Tathagata Das2014-05-251-1/+1
| | | | This reverts commit d807023479ce10aec28ef3c1ab646ddefc2e663c.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-05-251-1/+1
| | | | This reverts commit 67dd53d2556f03ce292e6889128cf441f1aa48f8.
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-05-201-1/+1
|
* [maven-release-plugin] prepare release v1.0.0-rc10Tathagata Das2014-05-201-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc9"Tathagata Das2014-05-191-1/+1
| | | | This reverts commit 920f947eb5a22a679c0c3186cf69ee75f6041c75.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-05-191-1/+1
| | | | This reverts commit f8e611955096c5c1c7db5764b9d2851b1d295f0d.
* SPARK-1878: Fix the incorrect initialization orderzsxwing2014-05-192-3/+7
| | | | | | | | | | | | | JIRA: https://issues.apache.org/jira/browse/SPARK-1878 Author: zsxwing <zsxwing@gmail.com> Closes #822 from zsxwing/SPARK-1878 and squashes the following commits: 4a47e27 [zsxwing] SPARK-1878: Fix the incorrect initialization order (cherry picked from commit 1811ba8ccb580979aa2e12019e6a82805f09ab53) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-171-1/+1
|
* [maven-release-plugin] prepare release v1.0.0-rc9Patrick Wendell2014-05-171-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc8"Patrick Wendell2014-05-161-1/+1
| | | | This reverts commit 80eea0f111c06260ffaa780d2f3f7facd09c17bc.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-161-1/+1
| | | | This reverts commit e5436b8c1a79ce108f3af402455ac5f6dc5d1eb3.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-161-1/+1
|
* [maven-release-plugin] prepare release v1.0.0-rc8Patrick Wendell2014-05-161-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc7"Patrick Wendell2014-05-161-1/+1
| | | | This reverts commit 9212b3e5bb5545ccfce242da8d89108e6fb1c464.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-161-1/+1
| | | | This reverts commit c4746aa6fe4aaf383e69e34353114d36d1eb9ba6.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-151-1/+1
|
* [maven-release-plugin] prepare release v1.0.0-rc7Patrick Wendell2014-05-151-1/+1
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc6"Patrick Wendell2014-05-141-1/+1
| | | | This reverts commit 54133abdce0246f6643a1112a5204afb2c4caa82.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-141-1/+1
| | | | This reverts commit e480bcfbd269ae1d7a6a92cfb50466cf192fe1fb.
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-12 18:31:24 +0000