aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Allow passing sparkHome and JARs to StreamingContext constructorMatei Zaharia2013-02-258-22/+76
| | | | | Also warns if spark.cleaner.ttl is not set in the version where you pass your own SparkContext.
* Set spark.deploy.spreadOut to true by default in 0.7 (improves locality)Matei Zaharia2013-02-251-1/+1
|
* Some tweaks to docsMatei Zaharia2013-02-253-9/+9
|
* Add a config property for Akka lifecycle event loggingMatei Zaharia2013-02-251-2/+4
|
* Fix compile errorMatei Zaharia2013-02-252-2/+2
|
* Use public method sparkContext instead of protected sc in streaming examplesMatei Zaharia2013-02-253-4/+4
|
* Change doc color scheme slightly for Spark 0.7 (to differ from 0.6)Matei Zaharia2013-02-252-16/+16
|
* Use a single setting for disabling API doc buildMatei Zaharia2013-02-252-4/+4
|
* Merge branch 'master' of github.com:mesos/sparkMatei Zaharia2013-02-251-1/+1
|\
| * Merge pull request #498 from pwendell/shutup-akkaMatei Zaharia2013-02-251-1/+1
| |\ | | | | | | Disable remote lifecycle logging from Akka.
| | * Disable remote lifecycle logging from Akka.Patrick Wendell2013-02-251-1/+1
| | | | | | | | | | | | This changes the default setting to `off` for remote lifecycle events. When this is on, it is very chatty at the INFO level. It also prints out several ERROR messages sometimes when sc.stop() is called.
* | | Change tabs to spacesMatei Zaharia2013-02-251-15/+15
|/ /
* | Get spark.default.paralellism on each call to defaultPartitioner,Matei Zaharia2013-02-251-4/+1
| | | | | | | | instead of only once, in case the user changes it across Spark uses
* | Merge pull request #459 from stephenh/bettersplitsMatei Zaharia2013-02-259-42/+94
|\ \ | |/ |/| Change defaultPartitioner to use upstream split size.
| * Use default parallelism if its set.Stephen Haberman2013-02-242-6/+19
| |
| * Merge branch 'master' into bettersplitsStephen Haberman2013-02-24117-879/+1755
| |\ | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/test/scala/spark/ShuffleSuite.scala
| * | Update more javadocs.Stephen Haberman2013-02-162-15/+17
| | |
| * | Tweak test names.Stephen Haberman2013-02-161-2/+2
| | |
| * | Remove fileServerSuite.txt.Stephen Haberman2013-02-161-1/+0
| | |
| * | Update default.parallelism docs, have StandaloneSchedulerBackend use it.Stephen Haberman2013-02-168-28/+43
| | | | | | | | | | | | | | | | | | Only brand new RDDs (e.g. parallelize and makeRDD) now use default parallelism, everything else uses their largest parent's partitioner or partition size.
| * | Change defaultPartitioner to use upstream split size.Stephen Haberman2013-02-103-6/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Previously it used the SparkContext.defaultParallelism, which occassionally ended up being a very bad guess. Looking at upstream RDDs seems to make better use of the context. Also sorted the upstream RDDs by partition size first, as if we have a hugely-partitioned RDD and tiny-partitioned RDD, it is unlikely we want the resulting RDD to be tiny-partitioned.
* | | Merge pull request #497 from ScrapCodes/dep-resolution-fixMatei Zaharia2013-02-251-1/+3
|\ \ \ | | | | | | | | Moving akka dependency resolver to shared.
| * | | Moving akka dependency resolver to shared.Prashant Sharma2013-02-251-1/+3
| | | |
* | | | Merge pull request #494 from mesos/streamingMatei Zaharia2013-02-2466-1227/+3254
|\| | | | | | | | | | | Final set of Spark Streaming changes
| * | | Merge pull request #495 from tdas/streamingTathagata Das2013-02-247-26/+14
| |\ \ \ | | | | | | | | | | Changed class paths and dependencies based on Matei's comments
| | * | | Removing duplicate doc.Tathagata Das2013-02-242-103/+2
| | | | |
| | * | | Fixed class paths and dependencies based on Matei's comments.Tathagata Das2013-02-248-24/+113
| | | | |
| * | | | Merge pull request #493 from tdas/streamingTathagata Das2013-02-2436-105/+696
| |\| | | | | | | | | | | | | Streaming bug fixes
| | * | | Added back the initial spark job before starting streaming receiversTathagata Das2013-02-241-1/+1
| | | | |
| | * | | Changed Algebird from 0.1.9 to 0.1.8Tathagata Das2013-02-242-2/+2
| | | | |
| | * | | Merge branch 'mesos-master' into streamingTathagata Das2013-02-2418-47/+361
| | |\ \ \ | | | | |/ | | | |/|
| | * | | Fixed missing dependencies in streaming/pom.xmlTathagata Das2013-02-241-4/+14
| | | | |
| | * | | Fixed bugs in examples.Tathagata Das2013-02-243-4/+3
| | | | |
| | * | | Updated streaming programming guide with Java API info, and comments from ↵Tathagata Das2013-02-233-14/+76
| | | | | | | | | | | | | | | | | | | | Patrick.
| | * | | Change spark.cleaner.delay to spark.cleaner.ttl. Updated docs.Tathagata Das2013-02-233-6/+6
| | | | |
| | * | | Fixed differences in APIs of StreamingContext and JavaStreamingContext. ↵Tathagata Das2013-02-235-26/+232
| | | | | | | | | | | | | | | | | | | | Change rawNetworkStream to rawSocketStream, and added twitter, actor, zeroMQ streams to JavaStreamingContext. Also added them to JavaAPISuite.
| | * | | Merge branch 'mesos-streaming' into streamingTathagata Das2013-02-228-8/+313
| | |\ \ \ | | |/ / / | |/| | |
| * | | | Fixed resolver for akka-zeromqTathagata Das2013-02-221-0/+3
| | | | |
| * | | | Merge pull request #480 from MLnick/streaming-eg-algebirdTathagata Das2013-02-225-5/+179
| |\ \ \ \ | | | | | | | | | | | | [Streaming] Examples using Twitter's Algebird library
| | * | | | Adding documentation for HLL and CMS examples. More efficient and clear use ↵Nick Pentreath2013-02-212-12/+36
| | | | | | | | | | | | | | | | | | | | | | | | of the monoids.
| | * | | | Bumping Algebird to 0.1.9Nick Pentreath2013-02-212-2/+2
| | | | | |
| | * | | | Merge remote-tracking branch 'upstream/streaming' into streaming-eg-algebirdNick Pentreath2013-02-21115-922/+1528
| | |\ \ \ \
| | * | | | | Streaming example using Twitter Algebird's Count Min Sketch monoidNick Pentreath2013-02-191-0/+78
| | | | | | |
| | * | | | | Dependencies and refactoring for streaming HLL example, and using ↵Nick Pentreath2013-02-193-21/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | | | context.twitterStream method
| | * | | | | Merge remote-tracking branch 'upstream/streaming' into streaming-eg-algebirdNick Pentreath2013-02-1955-1138/+2543
| | |\ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala
| | * | | | | | Adding streaming HyperLogLog example using AlgebirdNick Pentreath2013-02-193-1/+69
| | | | | | | |
| * | | | | | | Merge pull request #479 from ScrapCodes/zeromq-streamingTathagata Das2013-02-224-3/+131
| |\ \ \ \ \ \ \ | | |_|_|/ / / / | |/| | | | | | Zeromq streaming
| | * | | | | | removed document on zeromqPrashant Sharma2013-02-201-59/+0
| | | | | | | |
| | * | | | | | fixes corresponding to review feedback at pull request #479Prashant Sharma2013-02-203-6/+9
| | | | | | | |
| | * | | | | | example for demonstrating ZeroMQ streamPrashant Sharma2013-02-192-8/+77
| | | | | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-16 10:02:52 +0000