Commit message (Collapse) | Author | Age | Files | Lines | ||
---|---|---|---|---|---|---|
... | ||||||
* | | | | Changed default level on calling DStream.persist() to be MEMORY_ONLY_SER. ↵ | Tathagata Das | 2012-11-12 | 2 | -2/+2 | |
| | | | | | | | | | | | | | | | | Also changed the persist level of StateDStream to be MEMORY_ONLY_SER. | |||||
* | | | | Fixed bugs in RawNetworkInputDStream and in its examples. Made the ↵ | Tathagata Das | 2012-11-12 | 11 | -376/+167 | |
| | | | | | | | | | | | | | | | | ReducedWindowedDStream persist RDDs to MEMOERY_SER_ONLY by default. Removed unncessary examples. Added streaming-env.sh.template to add recommended setting for streaming. | |||||
* | | | | Merge branch 'dev' of github.com:radlab/spark into dev | tdas | 2012-11-11 | 2 | -22/+148 | |
|\ \ \ \ | ||||||
| * | | | | Fixed bug in FileInputDStream that allowed it to miss new files. Added tests ↵ | Tathagata Das | 2012-11-11 | 2 | -22/+148 | |
| | |_|/ | |/| | | | | | | | | | | in the InputStreamsSuite to test checkpointing of file and network streams. | |||||
* / | | | Removed unnecessary files. | tdas | 2012-11-08 | 10 | -1643/+0 | |
|/ / / | ||||||
* | | | Fixed bug in InputStreamsSuite | tdas | 2012-11-08 | 1 | -0/+2 | |
| | | | ||||||
* | | | Added FailureTestsuite for testing multiple, repeated master failures. | Tathagata Das | 2012-11-06 | 7 | -49/+256 | |
| | | | ||||||
* | | | Added a few more comments to the checkpoint-related functions. | Tathagata Das | 2012-11-05 | 1 | -0/+8 | |
| |/ |/| | ||||||
* | | Made more bug fixes for checkpointing. | Tathagata Das | 2012-11-05 | 6 | -66/+97 | |
| | | ||||||
* | | Fixed major bugs in checkpointing. | Tathagata Das | 2012-11-05 | 8 | -85/+213 | |
| | | ||||||
* | | Made checkpointing of dstream graph to work with checkpointing of RDDs. For ↵ | Tathagata Das | 2012-11-04 | 15 | -188/+337 | |
|/ | | | | streams requiring checkpointing of its RDD, the default checkpoint interval is set to 10 seconds. | |||||
* | Fixed serialization bug in countByWindow, added countByKey and ↵ | Tathagata Das | 2012-11-02 | 5 | -48/+186 | |
| | | | | countByKeyAndWindow, and added testcases for them. | |||||
* | Added save operations to DStreams. | Tathagata Das | 2012-10-27 | 3 | -3/+84 | |
| | ||||||
* | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-10-25 | 16 | -23/+29 | |
|\ | ||||||
| * | Merge remote-tracking branch 'public/master' into dev | Matei Zaharia | 2012-10-24 | 16 | -23/+29 | |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/BlockStoreShuffleFetcher.scala core/src/main/scala/spark/KryoSerializer.scala core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/executor/Executor.scala core/src/main/scala/spark/network/Connection.scala core/src/main/scala/spark/network/ConnectionManagerTest.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/NewHadoopRDD.scala core/src/main/scala/spark/scheduler/ShuffleMapTask.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockMessage.scala core/src/main/scala/spark/storage/BlockStore.scala core/src/main/scala/spark/storage/StorageLevel.scala core/src/main/scala/spark/util/AkkaUtils.scala project/SparkBuild.scala run | |||||
* | | Added tests for the file input stream. | Tathagata Das | 2012-10-24 | 2 | -8/+64 | |
|/ | ||||||
* | Minor fixes. | Tathagata Das | 2012-10-24 | 4 | -27/+27 | |
| | ||||||
* | Added tests for testing network input stream. | Tathagata Das | 2012-10-24 | 5 | -23/+134 | |
| | ||||||
* | Renamed the streaming testsuites. | Tathagata Das | 2012-10-23 | 4 | -6/+8 | |
| | ||||||
* | Renamed APIs to create queueStream and fileStream. | Tathagata Das | 2012-10-23 | 5 | -10/+10 | |
| | ||||||
* | Updated StateDStream api to use Options instead of nulls. | Tathagata Das | 2012-10-23 | 3 | -18/+18 | |
| | ||||||
* | Renamed the network input streams. | Tathagata Das | 2012-10-23 | 7 | -35/+40 | |
| | ||||||
* | Modified API of NetworkInputDStreams and got ObjectInputDStream and ↵ | Tathagata Das | 2012-10-23 | 8 | -382/+360 | |
| | | | | RawInputDStream working. | |||||
* | Renamed examples | Tathagata Das | 2012-10-22 | 2 | -7/+7 | |
| | ||||||
* | Added MapValueDStream, FlatMappedValuesDStream and CoGroupedDStream, and ↵ | Tathagata Das | 2012-10-21 | 9 | -35/+293 | |
| | | | | therefore DStream operations mapValue, flatMapValues, cogroup, and join. Also, added tests for DStream operations filter, glom, mapPartitions, groupByKey, mapValues, flatMapValues, cogroup, and join. | |||||
* | Fixed some bugs in tests for forgetting RDDs, and made sure that use of ↵ | Tathagata Das | 2012-10-21 | 10 | -89/+100 | |
| | | | | manual clock leads to a zeroTime of 0 in the DStreams (more intuitive). | |||||
* | Added functionality to forget RDDs from DStreams. | Tathagata Das | 2012-10-19 | 11 | -79/+224 | |
| | ||||||
* | Minor modifications. | Tathagata Das | 2012-10-15 | 3 | -13/+14 | |
| | ||||||
* | Refactored DStreamSuiteBase to create CheckpointSuite- testsuite for testing ↵ | Tathagata Das | 2012-10-14 | 12 | -135/+290 | |
| | | | | checkpointing under different operations. | |||||
* | Fixed bugs in the streaming testsuites. | Tathagata Das | 2012-10-13 | 2 | -7/+18 | |
| | ||||||
* | Implemented checkpointing of StreamingContext and DStream graph. | Tathagata Das | 2012-10-13 | 13 | -163/+534 | |
| | ||||||
* | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-17 | 5 | -31/+46 | |
|\ | ||||||
| * | Minor tweaks | Tathagata Das | 2012-09-14 | 2 | -22/+24 | |
| | | ||||||
| * | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-07 | 20 | -404/+824 | |
| |\ | | | | | | | | | | | | | Conflicts: streaming/src/main/scala/spark/streaming/Scheduler.scala | |||||
| | * | add warmup code for TopKWordCountRaw.scala | haoyuan | 2012-09-06 | 1 | -5/+14 | |
| | | | ||||||
| * | | Fixed bugs in streaming Scheduler and optimized QueueInputDStream. | Tathagata Das | 2012-09-07 | 3 | -4/+8 | |
| | | | ||||||
* | | | Allowed StreamingContext to be created from existing SparkContext | Tathagata Das | 2012-09-17 | 1 | -8/+5 | |
| |/ |/| | ||||||
* | | Merge branch 'dev' of github.com:radlab/spark into dev | haoyuan | 2012-09-07 | 9 | -313/+424 | |
|\ \ | ||||||
| * | | Fixed bugs and added testcases for naive reduceByKeyAndWindow. | Tathagata Das | 2012-09-06 | 5 | -87/+140 | |
| | | | ||||||
| * | | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-06 | 2 | -5/+5 | |
| |\ \ | ||||||
| * | | | Re-implemented ReducedWindowedDSteam to simplify and fix bugs. Added slice ↵ | Tathagata Das | 2012-09-06 | 7 | -276/+334 | |
| | | | | | | | | | | | | | | | | operator to DStream. Also, refactored DStream testsuites and added tests for reduceByKeyAndWindow. | |||||
* | | | | commit opt for grep scalibility test. | haoyuan | 2012-09-07 | 3 | -31/+45 | |
| |/ / |/| | | ||||||
* | | | Less warmup in word count | root | 2012-09-06 | 1 | -1/+1 | |
| | | | ||||||
* | | | Merge branch 'dev' of github.com:radlab/spark into dev | root | 2012-09-05 | 1 | -1/+1 | |
|\ \ \ | ||||||
| * | | | Merge branch 'dev' of github.com:radlab/spark into dev | root | 2012-09-05 | 6 | -56/+201 | |
| |\| | | ||||||
| * | | | Some changes to make important log output visible even if we set the logging ↵ | root | 2012-09-05 | 1 | -1/+1 | |
| | | | | | | | | | | | | | | | | to WARNING | |||||
* | | | | Tuning Akka for more connections | root | 2012-09-05 | 1 | -5/+5 | |
| |/ / |/| | | ||||||
* | | | Merge branch 'dev' of github.com:radlab/spark into dev | Tathagata Das | 2012-09-04 | 4 | -2/+229 | |
|\ \ \ | ||||||
| * | | | fix the compile error in TopKWordCountRaw.scala | haoyuan | 2012-09-04 | 1 | -6/+6 | |
| | | | | ||||||
| * | | | add TopKWordCountRaw.scala | haoyuan | 2012-09-04 | 1 | -0/+86 | |
| |/ / |