| Commit message (Collapse) | Author | Age | Files | Lines | ||
|---|---|---|---|---|---|---|
| ... | ||||||
| * | | | | | | | Fixed bug in the number of splits in RDD after checkpointing. Modified ↵ | Tathagata Das | 2012-11-19 | 4 | -3/+11 | |
| | |_|_|_|_|/ |/| | | | | | | | | | | | | | | | | | reduceByKeyAndWindow (naive) computation from window+reduceByKey to reduceByKey+window+reduceByKey. | |||||
| * | | | | | | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-11-17 | 2 | -0/+170 | |
| |\ \ \ \ \ \ | ||||||
| | * \ \ \ \ \ | Merge pull request #4 from radlab/streaming-example | Patrick Wendell | 2012-11-16 | 2 | -0/+170 | |
| | |\ \ \ \ \ \ | | |_|_|_|_|/ | |/| | | | | | A "streaming page view" example. | |||||
| | | * | | | | | A "streaming page view" example. | Patrick Wendell | 2012-11-16 | 2 | -0/+170 | |
| | | |/ / / / | ||||||
| * | / / / / | Fixed checkpointing bug in CoGroupedRDD. CoGroupSplits kept around the RDD ↵ | Tathagata Das | 2012-11-17 | 2 | -3/+3 | |
| |/ / / / / | | | | | | | | | | | | | | | | splits of its parent RDDs, thus checkpointing its parents did not release the references to the parent splits. | |||||
| * | | | | | Fixed bug in MappedValuesRDD, and set default graph checkpoint interval to ↵ | Tathagata Das | 2012-11-13 | 3 | -5/+22 | |
| | | | | | | | | | | | | | | | | | | | | | be batch duration. | |||||
| * | | | | | Replaced StateRDD in StateDStream with MapPartitionsRDD. | Tathagata Das | 2012-11-13 | 1 | -12/+3 | |
| | | | | | | ||||||
| * | | | | | Optimized checkpoint writing by reusing FileSystem object. Fixed bug in ↵ | Tathagata Das | 2012-11-13 | 7 | -106/+128 | |
| | |_|_|/ |/| | | | | | | | | | | | updating of checkpoint data in DStream where the checkpointed RDDs, upon recovery, were not recognized as checkpointed RDDs and therefore deleted from HDFS. Made InputStreamsSuite more robust to timing delays. | |||||
| * | | | | Speeded up CheckpointSuite | Tathagata Das | 2012-11-12 | 1 | -12/+14 | |
| | | | | | ||||||
| * | | | | Changed default level on calling DStream.persist() to be MEMORY_ONLY_SER. ↵ | Tathagata Das | 2012-11-12 | 2 | -2/+2 | |
| | | | | | | | | | | | | | | | | | Also changed the persist level of StateDStream to be MEMORY_ONLY_SER. | |||||
| * | | | | Fixed bugs in RawNetworkInputDStream and in its examples. Made the ↵ | Tathagata Das | 2012-11-12 | 11 | -376/+167 | |
| | | | | | | | | | | | | | | | | | ReducedWindowedDStream persist RDDs to MEMOERY_SER_ONLY by default. Removed unncessary examples. Added streaming-env.sh.template to add recommended setting for streaming. | |||||
| * | | | | Merge branch 'dev' of github.com:radlab/spark into dev | tdas | 2012-11-11 | 2 | -22/+148 | |
| |\ \ \ \ | ||||||
| | * | | | | Fixed bug in FileInputDStream that allowed it to miss new files. Added tests ↵ | Tathagata Das | 2012-11-11 | 2 | -22/+148 | |
| | | |_|/ | |/| | | | | | | | | | | in the InputStreamsSuite to test checkpointing of file and network streams. | |||||
| * / | | | Removed unnecessary files. | tdas | 2012-11-08 | 10 | -1643/+0 | |
| |/ / / | ||||||
| * | | | Fixed bug in InputStreamsSuite | tdas | 2012-11-08 | 1 | -0/+2 | |
| | | | | ||||||
| * | | | Added FailureTestsuite for testing multiple, repeated master failures. | Tathagata Das | 2012-11-06 | 7 | -49/+256 | |
| | | | | ||||||
| * | | | Added a few more comments to the checkpoint-related functions. | Tathagata Das | 2012-11-05 | 1 | -0/+8 | |
| | |/ |/| | ||||||
| * | | Made more bug fixes for checkpointing. | Tathagata Das | 2012-11-05 | 6 | -66/+97 | |
| | | | ||||||
| * | | Fixed major bugs in checkpointing. | Tathagata Das | 2012-11-05 | 8 | -85/+213 | |
| | | | ||||||
| * | | Made checkpointing of dstream graph to work with checkpointing of RDDs. For ↵ | Tathagata Das | 2012-11-04 | 15 | -188/+337 | |
| |/ | | | | streams requiring checkpointing of its RDD, the default checkpoint interval is set to 10 seconds. | |||||
| * | Fixed serialization bug in countByWindow, added countByKey and ↵ | Tathagata Das | 2012-11-02 | 5 | -48/+186 | |
| | | | | | countByKeyAndWindow, and added testcases for them. | |||||
| * | Added save operations to DStreams. | Tathagata Das | 2012-10-27 | 3 | -3/+84 | |
| | | ||||||
| * | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-10-25 | 16 | -23/+29 | |
| |\ | ||||||
| | * | Merge remote-tracking branch 'public/master' into dev | Matei Zaharia | 2012-10-24 | 16 | -23/+29 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/BlockStoreShuffleFetcher.scala core/src/main/scala/spark/KryoSerializer.scala core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/executor/Executor.scala core/src/main/scala/spark/network/Connection.scala core/src/main/scala/spark/network/ConnectionManagerTest.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/NewHadoopRDD.scala core/src/main/scala/spark/scheduler/ShuffleMapTask.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockMessage.scala core/src/main/scala/spark/storage/BlockStore.scala core/src/main/scala/spark/storage/StorageLevel.scala core/src/main/scala/spark/util/AkkaUtils.scala project/SparkBuild.scala run | |||||
| * | | Added tests for the file input stream. | Tathagata Das | 2012-10-24 | 2 | -8/+64 | |
| |/ | ||||||
| * | Minor fixes. | Tathagata Das | 2012-10-24 | 4 | -27/+27 | |
| | | ||||||
| * | Added tests for testing network input stream. | Tathagata Das | 2012-10-24 | 5 | -23/+134 | |
| | | ||||||
| * | Renamed the streaming testsuites. | Tathagata Das | 2012-10-23 | 4 | -6/+8 | |
| | | ||||||
| * | Renamed APIs to create queueStream and fileStream. | Tathagata Das | 2012-10-23 | 5 | -10/+10 | |
| | | ||||||
| * | Updated StateDStream api to use Options instead of nulls. | Tathagata Das | 2012-10-23 | 3 | -18/+18 | |
| | | ||||||
| * | Renamed the network input streams. | Tathagata Das | 2012-10-23 | 7 | -35/+40 | |
| | | ||||||
| * | Modified API of NetworkInputDStreams and got ObjectInputDStream and ↵ | Tathagata Das | 2012-10-23 | 8 | -382/+360 | |
| | | | | | RawInputDStream working. | |||||
| * | Renamed examples | Tathagata Das | 2012-10-22 | 2 | -7/+7 | |
| | | ||||||
| * | Added MapValueDStream, FlatMappedValuesDStream and CoGroupedDStream, and ↵ | Tathagata Das | 2012-10-21 | 9 | -35/+293 | |
| | | | | | therefore DStream operations mapValue, flatMapValues, cogroup, and join. Also, added tests for DStream operations filter, glom, mapPartitions, groupByKey, mapValues, flatMapValues, cogroup, and join. | |||||
| * | Fixed some bugs in tests for forgetting RDDs, and made sure that use of ↵ | Tathagata Das | 2012-10-21 | 10 | -89/+100 | |
| | | | | | manual clock leads to a zeroTime of 0 in the DStreams (more intuitive). | |||||
| * | Added functionality to forget RDDs from DStreams. | Tathagata Das | 2012-10-19 | 11 | -79/+224 | |
| | | ||||||
| * | Minor modifications. | Tathagata Das | 2012-10-15 | 3 | -13/+14 | |
| | | ||||||
| * | Refactored DStreamSuiteBase to create CheckpointSuite- testsuite for testing ↵ | Tathagata Das | 2012-10-14 | 12 | -135/+290 | |
| | | | | | checkpointing under different operations. | |||||
| * | Fixed bugs in the streaming testsuites. | Tathagata Das | 2012-10-13 | 2 | -7/+18 | |
| | | ||||||
| * | Implemented checkpointing of StreamingContext and DStream graph. | Tathagata Das | 2012-10-13 | 13 | -163/+534 | |
| | | ||||||
| * | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-17 | 5 | -31/+46 | |
| |\ | ||||||
| | * | Minor tweaks | Tathagata Das | 2012-09-14 | 2 | -22/+24 | |
| | | | ||||||
| | * | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-07 | 20 | -404/+824 | |
| | |\ | | | | | | | | | | | | | Conflicts: streaming/src/main/scala/spark/streaming/Scheduler.scala | |||||
| | | * | add warmup code for TopKWordCountRaw.scala | haoyuan | 2012-09-06 | 1 | -5/+14 | |
| | | | | ||||||
| | * | | Fixed bugs in streaming Scheduler and optimized QueueInputDStream. | Tathagata Das | 2012-09-07 | 3 | -4/+8 | |
| | | | | ||||||
| * | | | Allowed StreamingContext to be created from existing SparkContext | Tathagata Das | 2012-09-17 | 1 | -8/+5 | |
| | |/ |/| | ||||||
| * | | Merge branch 'dev' of github.com:radlab/spark into dev | haoyuan | 2012-09-07 | 9 | -313/+424 | |
| |\ \ | ||||||
| | * | | Fixed bugs and added testcases for naive reduceByKeyAndWindow. | Tathagata Das | 2012-09-06 | 5 | -87/+140 | |
| | | | | ||||||
| | * | | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-06 | 2 | -5/+5 | |
| | |\ \ | ||||||
| | * | | | Re-implemented ReducedWindowedDSteam to simplify and fix bugs. Added slice ↵ | Tathagata Das | 2012-09-06 | 7 | -276/+334 | |
| | | | | | | | | | | | | | | | | | operator to DStream. Also, refactored DStream testsuites and added tests for reduceByKeyAndWindow. | |||||
 |
index : spark | |
| Mirror of Apache Spark |
| aboutsummaryrefslogtreecommitdiff |