aboutsummaryrefslogtreecommitdiff
path: root/streaming
Commit message (Collapse)AuthorAgeFilesLines
* Minor change for PR 277.Tathagata Das2013-12-231-1/+1
|
* Minor formatting fixes.Tathagata Das2013-12-231-5/+4
|
* Added comments to BatchInfo and JobSet, based on Patrick's comment on PR 277.Tathagata Das2013-12-232-3/+26
|
* Minor updated based on comments on PR 277.Tathagata Das2013-12-202-1/+6
|
* Minor changes.Tathagata Das2013-12-189-32/+36
|
* Merge branch 'apache-master' into scheduler-updateTathagata Das2013-12-1842-406/+458
|\ | | | | | | | | | | Conflicts: streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala streaming/src/main/scala/org/apache/spark/streaming/dstream/ForEachDStream.scala
| * Use scala.binary.version in POMsMark Hamstra2013-12-151-7/+7
| |
| * Merge branch 'master' into akka-bug-fixPrashant Sharma2013-12-111-8/+1
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala pom.xml project/SparkBuild.scala streaming/pom.xml yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
| * | Style fixes and addressed review comments at #221Prashant Sharma2013-12-102-10/+10
| | |
| * | Incorporated Patrick's feedback comment on #211 and made maven ↵Prashant Sharma2013-12-071-1/+1
| | | | | | | | | | | | build/dep-resolution atleast a bit faster.
| * | Merge branch 'master' of github.com:apache/incubator-spark into scala-2.10-tempPrashant Sharma2013-11-215-11/+12
| |\ \ | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/util/collection/PrimitiveVector.scala streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala
| * \ \ Merge branch 'scala210-master' of github.com:colorant/incubator-spark into ↵Prashant Sharma2013-11-2132-295/+1500
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | scala-2.10 Conflicts: core/src/main/scala/org/apache/spark/deploy/client/Client.scala core/src/main/scala/org/apache/spark/deploy/worker/Worker.scala core/src/main/scala/org/apache/spark/executor/CoarseGrainedExecutorBackend.scala core/src/test/scala/org/apache/spark/MapOutputTrackerSuite.scala
| | * | | Various merge correctionsAaron Davidson2013-11-145-63/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | I've diff'd this patch against my own -- since they were both created independently, this means that two sets of eyes have gone over all the merge conflicts that were created, so I'm feeling significantly more confident in the resulting PR. @rxin has looked at the changes to the repl and is resoundingly confident that they are correct.
| | * | | Merge branch 'master' into scala-2.10Raymond Liu2013-11-142-7/+80
| | |\ \ \
| | * \ \ \ Merge branch 'master' into scala-2.10Raymond Liu2013-11-1331-228/+1421
| | |\ \ \ \
| * | | | | | Remove deprecated actorFor and use actorSelection everywhere.Prashant Sharma2013-11-121-1/+1
| |/ / / / /
| * | | | | Merge branch 'scala-2.10' of github.com:ScrapCodes/spark into scala-2.10Prashant Sharma2013-10-101-3/+4
| |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala project/SparkBuild.scala
| | * \ \ \ \ Merge branch 'master' into wip-merge-masterPrashant Sharma2013-10-081-2/+3
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: bagel/pom.xml core/pom.xml core/src/test/scala/org/apache/spark/ui/UISuite.scala examples/pom.xml mllib/pom.xml pom.xml project/SparkBuild.scala repl/pom.xml streaming/pom.xml tools/pom.xml In scala 2.10, a shorter representation is used for naming artifacts so changed to shorter scala version for artifacts and made it a property in pom.
| | * \ \ \ \ \ Merge branch 'master' into scala-2.10Prashant Sharma2013-10-011-1/+1
| | |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressUI.scala docs/_config.yml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * | | | | | | | fixed some warningsMartin Weindel2013-10-0515-61/+76
| |/ / / / / / /
| * | | | | | | fixed maven build for scala 2.10Prashant Sharma2013-09-261-5/+9
| | | | | | | |
| * | | | | | | Akka 2.2 migrationPrashant Sharma2013-09-226-12/+15
| | | | | | | |
| * | | | | | | Few more fixes to tests broken during mergePrashant Sharma2013-09-101-3/+3
| | | | | | | |
| * | | | | | | Merged with masterPrashant Sharma2013-09-0677-801/+1731
| |\ \ \ \ \ \ \
| * | | | | | | | code formatting, The warning related to scope exit and enter is not worth ↵Prashant Sharma2013-07-161-14/+14
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | fixing as it only affects debugging scopes and nothing else.
| * | | | | | | | Fixed warning erasure -> runtimeClassPrashant Sharma2013-07-161-1/+1
| | | | | | | | |
| * | | | | | | | Fixed warning ThrowablesPrashant Sharma2013-07-161-1/+1
| | | | | | | | |
| * | | | | | | | Fixed warning ClassManifest -> ClassTagPrashant Sharma2013-07-161-1/+1
| | | | | | | | |
| * | | | | | | | Merge branch 'master' into master-mergePrashant Sharma2013-07-126-18/+29
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: README.md core/pom.xml core/src/main/scala/spark/deploy/JsonProtocol.scala core/src/main/scala/spark/deploy/LocalSparkCluster.scala core/src/main/scala/spark/deploy/master/Master.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala pom.xml project/SparkBuild.scala streaming/src/main/scala/spark/streaming/receivers/ActorReceiver.scala
| * \ \ \ \ \ \ \ \ Merge branch 'master' into master-mergePrashant Sharma2013-07-0316-148/+292
| |\ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/RDDCheckpointData.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/Utils.scala core/src/main/scala/spark/api/python/PythonRDD.scala core/src/main/scala/spark/deploy/client/Client.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/ZippedRDD.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockManagerMaster.scala core/src/main/scala/spark/storage/BlockManagerMasterActor.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala core/src/test/scala/spark/SizeEstimatorSuite.scala pom.xml project/SparkBuild.scala repl/src/main/scala/spark/repl/SparkILoop.scala repl/src/test/scala/spark/repl/ReplSuite.scala streaming/src/main/scala/spark/streaming/StreamingContext.scala streaming/src/main/scala/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/util/MasterFailureTest.scala
| * | | | | | | | | | Updated to latest stable scala 2.10.1 and akka 2.1.2Prashant Sharma2013-05-011-12/+2
| | | | | | | | | | |
| * | | | | | | | | | Fixied other warningsPrashant Sharma2013-04-291-3/+1
| | | | | | | | | | |
| * | | | | | | | | | Fixed warning: erasure -> runtimeClassPrashant Sharma2013-04-291-4/+4
| | | | | | | | | | |
| * | | | | | | | | | Fixed Warning: ClassManifest -> ClassTagPrashant Sharma2013-04-2939-239/+285
| | | | | | | | | | |
| * | | | | | | | | | Fixed pom.xml with updated dependencies.Prashant Sharma2013-04-291-1/+11
| | | | | | | | | | |
| * | | | | | | | | | Fixed breaking tests in streaming checkpoint suite. Changed RichInt to Int ↵Prashant Sharma2013-04-251-16/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | as it is final and not serializable
| * | | | | | | | | | scala 2.10 and master mergePrashant Sharma2013-04-244-15/+16
| | | | | | | | | | |
* | | | | | | | | | | Added StatsReportListener to generate processing time statistics across ↵Tathagata Das2013-12-182-2/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | multiple batches.
* | | | | | | | | | | Refactored streaming scheduler and added listener interface.Tathagata Das2013-12-1222-195/+496
| |_|_|_|_|_|_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Refactored Scheduler + JobManager to JobGenerator + JobScheduler and added JobSet for cleaner code. Moved scheduler related code to streaming.scheduler package. - Added StreamingListener trait (similar to SparkListener) to enable gathering to streaming stats like processing times and delays. StreamingContext.addListener() to added listeners. - Deduped some code in streaming tests by modifying TestSuiteBase, and added StreamingListenerSuite.
* | | | | | | | | | Fix pom.xml for maven buildRaymond Liu2013-12-031-8/+1
| |_|_|_|_|_|_|_|/ |/| | | | | | | |
* | | | | | | | | Another set of changes to remove unnecessary semicolon (;) from Scala code.Henry Saputra2013-11-191-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Passed the sbt/sbt compile and test
* | | | | | | | | Remove the semicolons at the end of Scala code to make it more pure Scala code.Henry Saputra2013-11-195-10/+9
| |_|_|_|_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also remove unused imports as I found them along the way. Remove return statements when returning value in the Scala code. Passing compile and tests.
* | | | | | | | Made block generator thread safe to fix Kafka bug.Tathagata Das2013-11-122-7/+80
| |_|_|_|_|_|/ |/| | | | | |
* | | | | | | Merge branch 'apache-master' into transformTathagata Das2013-10-259-15/+180
|\ \ \ \ \ \ \
| * | | | | | | Exclude jopt from kafka dependency.Patrick Wendell2013-10-251-0/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Kafka uses an older version of jopt that causes bad conflicts with the version used by spark-perf. It's not easy to remove this downstream because of the way that spark-perf uses Spark (by including a spark assembly as an unmanaged jar). This fixes the problem at its source by just never including it.
| * | | | | | | Style fixesPatrick Wendell2013-10-241-9/+9
| | | | | | | |
| * | | | | | | Spacing fixPatrick Wendell2013-10-241-4/+4
| | | | | | | |
| * | | | | | | Small spacing fixPatrick Wendell2013-10-241-2/+2
| | | | | | | |
| * | | | | | | Adding Java versions and associated testsPatrick Wendell2013-10-244-0/+68
| | | | | | | |
| * | | | | | | Some clean-up of testsPatrick Wendell2013-10-243-7/+10
| | | | | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-08 15:02:49 +0000