Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Spark standalone mode cluster scripts. | Denny | 2012-08-01 | 18 | -4/+825 |
| | | | | Heavily inspired by Hadoop cluster scripts ;-) | ||||
* | Merge pull request #158 from JoshRosen/fix/assembly-akka-config | Matei Zaharia | 2012-07-30 | 1 | -2/+55 |
|\ | | | | | Merge Akka reference.conf files in sbt assembly task | ||||
| * | Fix Akka configuration in assembly jar. | Josh Rosen | 2012-07-30 | 1 | -2/+55 |
|/ | | | | | | | | | This resolves an issue where running Spark from the assembly jar would cause a "No configuration setting found for key 'akka.version'" exception. This solution is from the Akka Team Blog: http://letitcrash.com/post/21025950392/ | ||||
* | Merge branch 'block-manager-fix' into dev | Matei Zaharia | 2012-07-30 | 9 | -88/+83 |
|\ | |||||
| * | Made BlockManagerMaster no longer be a singleton. | Matei Zaharia | 2012-07-30 | 9 | -88/+83 |
| | | | | | | | | Also cleaned up a few formatting things throughout block manager code. | ||||
* | | Merge branch 'dev' of git://github.com/tdas/spark into dev | Matei Zaharia | 2012-07-30 | 6 | -35/+130 |
|\ \ | |/ |/| | |||||
| * | Updated the new checkpoint RDD to remember partitioning of the original RDD. | Tathagata Das | 2012-07-27 | 1 | -1/+6 |
| | | |||||
| * | Merge branch 'dev' of github.com:tdas/spark into dev | Tathagata Das | 2012-07-27 | 1 | -4/+8 |
| |\ | |||||
| | * | Fixed more stuff in BoundedMemoryCache. | Tathagata Das | 2012-07-27 | 1 | -4/+8 |
| | | | |||||
| * | | Added BlockRDD and a first-cut version of checkpoint() to RDD class. | Tathagata Das | 2012-07-27 | 3 | -0/+63 |
| |/ | |||||
| * | Fixed bug in BoundedMemoryCache. | Tathagata Das | 2012-07-27 | 1 | -1/+5 |
| | | |||||
| * | Fixed bugs in block dropping code of MemoryStore and changed synchronized ↵ | Tathagata Das | 2012-07-27 | 2 | -14/+8 |
| | | | | | | | | HashMap to ConcurrentHashMap in BlockManager. | ||||
| * | Modified the block dropping code for better performance. | Tathagata Das | 2012-07-26 | 2 | -25/+51 |
| | | |||||
* | | Various reorganization and formatting fixes | Matei Zaharia | 2012-07-30 | 4 | -57/+79 |
| | | |||||
* | | Fixed AccumulatorSuite to clean up SparkContext with BeforeAndAfter | Matei Zaharia | 2012-07-28 | 1 | -10/+20 |
| | | |||||
* | | tasks cannot access value of accumulator | Imran Rashid | 2012-07-28 | 2 | -56/+21 |
| | | |||||
* | | one more minor cleanup to scaladoc | Imran Rashid | 2012-07-28 | 1 | -2/+2 |
| | | |||||
* | | fix up scaladoc, naming of type parameters | Imran Rashid | 2012-07-28 | 2 | -15/+12 |
| | | |||||
* | | rename addToAccum to addAccumulator | Imran Rashid | 2012-07-28 | 2 | -4/+4 |
| | | |||||
* | | add some functionality to Vector, delete copy in AccumulatorSuite | Imran Rashid | 2012-07-28 | 2 | -113/+33 |
| | | |||||
* | | move Vector class into core and spark.util package | Imran Rashid | 2012-07-28 | 7 | -10/+12 |
| | | |||||
* | | Accumulator now inherits from Accumulable, whcih simplifies a bunch of other ↵ | Imran Rashid | 2012-07-28 | 3 | -46/+64 |
| | | | | | | | | | | | | | | | | things (eg., no +:=) Conflicts: core/src/main/scala/spark/Accumulators.scala | ||||
* | | improve scaladoc | Imran Rashid | 2012-07-28 | 1 | -1/+1 |
| | | |||||
* | | add Accumulatable, add corresponding docs & tests for accumulators | Imran Rashid | 2012-07-28 | 3 | -0/+276 |
| | | |||||
* | | Fixed SPARK_MEM not being passed when runner is java | Matei Zaharia | 2012-07-28 | 1 | -1/+4 |
| | | |||||
* | | Add a sleep to prevent a failing test. | Matei Zaharia | 2012-07-27 | 1 | -0/+1 |
| | | | | | | | | | | | | | | | | The BlockManager's put seems to be slightly asynchronous, which can cause it to fail this test by not removing stuff from the cache before we put the next value. We should probably change the semantics of put() in this case but it's hard right now. It will also be hard for asynchronously replicated puts. | ||||
* | | Renamed test more descriptively | Matei Zaharia | 2012-07-27 | 1 | -1/+1 |
| | | |||||
* | | Added a second version of union() without varargs. | Matei Zaharia | 2012-07-27 | 3 | -4/+17 |
| | | |||||
* | | Fixed Java union methods having same erasure. | Matei Zaharia | 2012-07-27 | 3 | -25/+47 |
| | | | | | | | | | | | | | | Changed union() methods on lists to take a separate "first element" argument in order to differentiate them to the compiler, because Java 7 considered it an error to have them all take Lists parameterized with different types. | ||||
* | | Merge pull request #153 from JoshRosen/new-java-api | Matei Zaharia | 2012-07-26 | 24 | -4/+1822 |
|\ \ | | | | | | | Java API | ||||
| * | | Add persist(), splits(), glom(), and mapPartitions() to Java API. | Josh Rosen | 2012-07-26 | 5 | -3/+83 |
| | | | |||||
| * | | Remove StringOps.split() from Java WordCount. | Josh Rosen | 2012-07-25 | 1 | -5/+2 |
| | | | |||||
| * | | Minor cleanup and optimizations in Java API. | Josh Rosen | 2012-07-24 | 8 | -25/+25 |
| | | | | | | | | | | | | | | | | | | - Add override keywords. - Cache RDDs and counts in TC example. - Clean up JavaRDDLike's abstract methods. | ||||
| * | | Add type annotations to Java API methods. | Josh Rosen | 2012-07-22 | 6 | -43/+90 |
| | | | | | | | | | | | | Add missing Scala Map to java.util.Map conversions. | ||||
| * | | Use mapValues() in JavaPairRDD.cogroupResultToJava(). | Josh Rosen | 2012-07-22 | 3 | -22/+14 |
| | | | |||||
| * | | Improve Java API examples | Josh Rosen | 2012-07-22 | 5 | -198/+143 |
| | | | | | | | | | | | | | | | | | | - Replace JavaLR example with JavaHdfsLR example. - Use anonymous classes in JavaWordCount; add options. - Remove @Override annotations. | ||||
| * | | Add Java API | Josh Rosen | 2012-07-18 | 25 | -4/+1761 |
| | | | | | | | | | | | | | | | | | | Add distinct() method to RDD. Fix bug in DoubleRDDFunctions. | ||||
* | | | Stlystic changes | Denny | 2012-07-23 | 11 | -23/+23 |
| | | | | | | | | | | | | | | | | | | Conflicts: core/src/test/scala/spark/MesosSchedulerSuite.scala | ||||
* | | | Always destroy SparkContext in after block for the unit tests. | Denny | 2012-07-23 | 11 | -114/+163 |
| | | | | | | | | | | | | | | | | | | Conflicts: core/src/test/scala/spark/ShuffleSuite.scala | ||||
* | | | Fix a bug where an input path was added to a Hadoop job configuration twice | Matei Zaharia | 2012-07-23 | 1 | -4/+1 |
|/ / | |||||
* | | Allow null keys in Spark's reduce and group by | Matei Zaharia | 2012-07-12 | 1 | -5/+9 |
| | | |||||
* | | Fixes to coarse-grained Mesos scheduler in dealing with failed nodes | Matei Zaharia | 2012-07-12 | 2 | -4/+25 |
| | | |||||
* | | Formatting | Matei Zaharia | 2012-07-11 | 1 | -10/+10 |
| | | |||||
* | | Added more methods for loading/saving with new Hadoop API | Matei Zaharia | 2012-07-11 | 2 | -4/+30 |
| | | |||||
* | | More work to allow Spark to run on the standalone deploy cluster. | Matei Zaharia | 2012-07-08 | 9 | -45/+182 |
| | | |||||
* | | Added back coarse-grained Mesos scheduler based on StandaloneScheduler. | Matei Zaharia | 2012-07-08 | 11 | -416/+320 |
| | | |||||
* | | More work on standalone scheduler | Matei Zaharia | 2012-07-06 | 8 | -33/+122 |
| | | |||||
* | | Further refactoring, and start of a standalone scheduler backend | Matei Zaharia | 2012-07-06 | 13 | -39/+211 |
| | | |||||
* | | Miscellaneous bug fixes | Matei Zaharia | 2012-07-06 | 5 | -9/+15 |
| | | |||||
* | | Some refactoring to make cluster scheduler pluggable. | Matei Zaharia | 2012-07-06 | 34 | -509/+718 |
| | |