aboutsummaryrefslogtreecommitdiff
path: root/streaming/src
Commit message (Collapse)AuthorAgeFilesLines
...
* | | Merge branch 'dev' into dev-mergeTathagata Das2013-01-0716-639/+37
|\ \ \
| * | | Changed locations for unit test logs.Tathagata Das2013-01-071-2/+2
| | | |
| * | | Fixed bugTathagata Das2013-01-071-1/+1
| | | |
| * | | Renamed examples and added documentation.Tathagata Das2013-01-073-21/+35
| | | |
| * | | Moved Spark Streaming examples to examples sub-project.Tathagata Das2013-01-0612-616/+0
| | | |
* | | | BufferingBlockCreator -> NetworkReceiver.BlockGeneratorPatrick Wendell2013-01-025-94/+89
| | | |
* | | | Merge branch 'dev-merge' into datahandler-fixPatrick Wendell2013-01-0246-507/+721
|\| | | | | | | | | | | | | | | | | | | Conflicts: streaming/src/main/scala/spark/streaming/dstream/DataHandler.scala
| * | | Updated Streaming Programming Guide.Tathagata Das2013-01-011-0/+1
| | | |
| * | | More classes made private[streaming] to hide from scala docs.Tathagata Das2012-12-3017-52/+135
| | | |
| * | | Refactored a whole lot to push all DStreams into the spark.streaming.dstream ↵Tathagata Das2012-12-3040-333/+401
| | | | | | | | | | | | | | | | package.
| * | | Improved jekyll and scala docs. Made many classes and method private to ↵Tathagata Das2012-12-2916-139/+201
| | | | | | | | | | | | | | | | remove them from scala docs.
* | | | Several code-quality improvements to DataHandler.Patrick Wendell2013-01-025-88/+84
|/ / / | | | | | | | | | | | | | | | | | | | | | - Changed to more accurate name: BufferingBlockCreator - Docstring now correctly reflects the abstraction offered by the class - Made internal methods private - Fixed indentation problems
* | | Modifications to make sure LocalScheduler terminate cleanly without errors ↵Tathagata Das2012-12-271-5/+8
| | | | | | | | | | | | when SparkContext is shutdown, to minimize spurious exception during master failure tests.
* | | Merge branch 'dev' of github.com:radlab/spark into dev-checkpointTathagata Das2012-12-2013-110/+683
|\| | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/ParallelCollection.scala core/src/test/scala/spark/CheckpointSuite.scala streaming/src/main/scala/spark/streaming/DStream.scala
| * | Changes in response to TD's review.Patrick Wendell2012-12-076-15/+29
| | |
| * | Adding Flume InputDStreamPatrick Wendell2012-12-078-6/+246
| |/
| * Adjust Kafka code to work with new streaming changes.Denny2012-12-052-4/+5
| |
| * Merge branch 'dev' into kafkaDenny2012-12-0522-109/+139
| |\ | | | | | | | | | | | | Conflicts: streaming/src/main/scala/spark/streaming/DStream.scala
| * | Added Kafka Wordcount producerDenny2012-11-192-25/+52
| | |
| * | Comment out code for fault-tolerance.Denny2012-11-191-17/+18
| | |
| * | Merge branch 'dev' into kafkaDenny2012-11-194-3/+173
| |\ \
| * \ \ Merge branch 'dev' into kafkaDenny2012-11-139-123/+155
| |\ \ \ | | | | | | | | | | | | | | | | | | | | Conflicts: streaming/src/main/scala/spark/streaming/DStream.scala
| * | | | change import for example functionDenny2012-11-131-1/+1
| | | | |
| * | | | Merge branch 'dev' into kafkaDenny2012-11-1226-2055/+331
| |\ \ \ \
| * | | | | Updated comment.Denny2012-11-111-1/+1
| | | | | |
| * | | | | Add comment.Denny2012-11-111-0/+2
| | | | | |
| * | | | | Kafka Stream comments.Denny2012-11-115-53/+99
| | | | | |
| * | | | | Merge branch 'dev' of github.com:radlab/spark into kafkaDenny2012-11-098-50/+263
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: streaming/src/main/scala/spark/streaming/DStream.scala
| * | | | | | Kafka Stream.Denny2012-11-099-161/+245
| | | | | | |
| * | | | | | Merge branch 'dev' of github.com:radlab/spark into kafkaDenny2012-11-0616-237/+545
| |\ \ \ \ \ \
| * | | | | | | Working on kafka.Denny2012-11-063-0/+159
| | | | | | | |
* | | | | | | | Made checkpoint data in RDDs optional to further reduce serialized size.Tathagata Das2012-12-111-3/+1
| | | | | | | |
* | | | | | | | Refactored RDD checkpointing to minimize extra fields in RDD class.Tathagata Das2012-12-041-3/+4
| |_|_|_|_|_|/ |/| | | | | |
* | | | | | | Minor modsTathagata Das2012-12-021-1/+1
| | | | | | |
* | | | | | | Made RDD checkpoint not create a new thread. Fixed bug in detecting when ↵Tathagata Das2012-12-021-4/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | spark.cleaner.delay is insufficient.
* | | | | | | Minor modifications.Tathagata Das2012-12-013-5/+22
| | | | | | |
* | | | | | | Added ssc.unionTathagata Das2012-12-012-1/+6
| | | | | | |
* | | | | | | Bug fixesTathagata Das2012-11-281-1/+1
| | | | | | |
* | | | | | | Modified bunch HashMaps in Spark to use TimeStampedHashMap and made various ↵Tathagata Das2012-11-271-4/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | | modules use CleanupTask to periodically clean up metadata.
* | | | | | | Merged branch mesos/master to branch dev.Tathagata Das2012-11-261-1/+1
| | | | | | |
* | | | | | | Modified StreamingContext API to make constructor accept the batch size ↵Tathagata Das2012-11-1916-98/+92
| | | | | | | | | | | | | | | | | | | | | | | | | | | | (since it is always needed, Patrick's suggestion). Added description to DStream and StreamingContext.
* | | | | | | Fixed bug in the number of splits in RDD after checkpointing. Modified ↵Tathagata Das2012-11-194-3/+11
| |_|_|_|_|/ |/| | | | | | | | | | | | | | | | | reduceByKeyAndWindow (naive) computation from window+reduceByKey to reduceByKey+window+reduceByKey.
* | | | | | Merge branch 'dev' of github.com:radlab/spark into devTathagata Das2012-11-172-0/+170
|\ \ \ \ \ \
| * \ \ \ \ \ Merge pull request #4 from radlab/streaming-examplePatrick Wendell2012-11-162-0/+170
| |\ \ \ \ \ \ | | |_|_|_|_|/ | |/| | | | | A "streaming page view" example.
| | * | | | | A "streaming page view" example.Patrick Wendell2012-11-162-0/+170
| | |/ / / /
* | / / / / Fixed checkpointing bug in CoGroupedRDD. CoGroupSplits kept around the RDD ↵Tathagata Das2012-11-172-3/+3
|/ / / / / | | | | | | | | | | | | | | | splits of its parent RDDs, thus checkpointing its parents did not release the references to the parent splits.
* | | | | Fixed bug in MappedValuesRDD, and set default graph checkpoint interval to ↵Tathagata Das2012-11-133-5/+22
| | | | | | | | | | | | | | | | | | | | be batch duration.
* | | | | Replaced StateRDD in StateDStream with MapPartitionsRDD.Tathagata Das2012-11-131-12/+3
| | | | |
* | | | | Optimized checkpoint writing by reusing FileSystem object. Fixed bug in ↵Tathagata Das2012-11-137-106/+128
| |_|_|/ |/| | | | | | | | | | | updating of checkpoint data in DStream where the checkpointed RDDs, upon recovery, were not recognized as checkpointed RDDs and therefore deleted from HDFS. Made InputStreamsSuite more robust to timing delays.
* | | | Speeded up CheckpointSuiteTathagata Das2012-11-121-12/+14
| | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-08-01 22:14:50 +0000