Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Stlystic changes | Denny | 2012-07-23 | 1 | -2/+2 |
| | | | | | | Conflicts: core/src/test/scala/spark/MesosSchedulerSuite.scala | ||||
* | Always destroy SparkContext in after block for the unit tests. | Denny | 2012-07-23 | 1 | -6/+11 |
| | | | | | | Conflicts: core/src/test/scala/spark/ShuffleSuite.scala | ||||
* | Merge branch 'master' into dev | Matei Zaharia | 2012-06-15 | 2 | -3/+1 |
|\ | |||||
| * | Performance improvements to shuffle operations: in particular, preserve | Matei Zaharia | 2012-06-09 | 2 | -3/+1 |
| | | | | | | | | | | RDD partitioning in more cases where it's possible, and use iterators instead of materializing collections when doing joins. | ||||
* | | Merge in engine improvements from the Spark Streaming project, developed | Matei Zaharia | 2012-06-07 | 1 | -5/+6 |
|/ | | | | | | jointly with Tathagata Das and Haoyuan Li. This commit imports the changes and ports them to Mesos 0.9, but does not yet pass unit tests due to various classes not supporting a graceful stop() yet. | ||||
* | Added an option (spark.closure.serializer) to specify the serializer for | Reynold Xin | 2012-04-09 | 1 | -0/+4 |
| | | | | closures. This enables using Kryo as the closure serializer. | ||||
* | Update Bagel unit tests to reflect API change | Ankur Dave | 2011-11-08 | 1 | -23/+21 |
| | |||||
* | Implement standalone WikipediaPageRank with custom serializer | Ankur Dave | 2011-10-09 | 1 | -0/+198 |
| | |||||
* | Update WikipediaPageRank to reflect Bagel API changes | Ankur Dave | 2011-10-09 | 2 | -100/+129 |
| | |||||
* | Remove ShortestPath for now | Ankur Dave | 2011-10-09 | 1 | -95/+0 |
| | |||||
* | Simplify and genericize type parameters in Bagel | Ankur Dave | 2011-10-09 | 1 | -85/+129 |
| | |||||
* | Fix issue #65: Change @serializable to extends Serializable in 2.9 branch | Ismael Juma | 2011-08-02 | 4 | -23/+18 |
| | | | | | | Note that we use scala.Serializable introduced in Scala 2.9 instead of java.io.Serializable. Also, case classes inherit from scala.Serializable by default. | ||||
* | Cleaned up a few issues to do with default parallelism levels. Also | Matei Zaharia | 2011-07-14 | 1 | -1/+1 |
| | | | | | renamed HadoopFileWriter to HadoopWriter (since it's not only for files) and fixed a bug for lookup(). | ||||
* | Fixed unit tests by making them clean up the SparkContext after use and | Matei Zaharia | 2011-05-13 | 1 | -0/+2 |
| | | | | | | thus clean up the various singletons (RDDCache, MapOutputTracker, etc). This isn't perfect yet (ideally we shouldn't use singleton objects at all) but we can fix that later. | ||||
* | Rename bagel to spark.bagel and Pregel to Bagel | Ankur Dave | 2011-05-09 | 4 | -18/+18 |
| | |||||
* | Move shortest path and PageRank to bagel.examples | Ankur Dave | 2011-05-03 | 2 | -2/+7 |
| | |||||
* | Refactor and add aggregator support | Ankur Dave | 2011-05-03 | 4 | -44/+88 |
| | | | | | | | | Refactored out the agg() and comp() methods from Pregel.run. Defined an implicit conversion to allow applications that don't use aggregators to avoid including a null argument for the result of the aggregator in the compute function. | ||||
* | Package combiner functions into a trait | Ankur Dave | 2011-05-03 | 4 | -66/+60 |
| | |||||
* | Add Bagel test suite | Ankur Dave | 2011-05-03 | 2 | -0/+65 |
| | | | | | | Note: This test suite currently fails for the same reason that the Spark Core test suite fails: Spark currently seems to have a bug where any test after the first one fails. | ||||
* | Clean up Bagel source and interface | Ankur Dave | 2011-05-03 | 3 | -124/+99 |
| | |||||
* | Update ShortestPath to work with controllable partitioning | Ankur Dave | 2011-05-03 | 1 | -9/+5 |
| | |||||
* | Clean up Pregel.run, add logging | Ankur Dave | 2011-05-03 | 1 | -26/+23 |
| | |||||
* | Add Bagel, an implementation of Pregel on Spark | Ankur Dave | 2011-05-03 | 3 | -0/+390 |