Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Rename "jobs" to "applications" in the standalone cluster | Matei Zaharia | 2013-02-17 | 31 | -286/+290 |
| | |||||
* | Renamed "splits" to "partitions" | Matei Zaharia | 2013-02-17 | 39 | -295/+305 |
| | |||||
* | Merge pull request #471 from stephenh/parallelrdd | Matei Zaharia | 2013-02-16 | 2 | -14/+9 |
|\ | | | | | Move ParallelCollection into spark.rdd package. | ||||
| * | Move ParallelCollection into spark.rdd package. | Stephen Haberman | 2013-02-16 | 2 | -14/+9 |
| | | |||||
* | | Merge pull request #470 from stephenh/morek | Matei Zaharia | 2013-02-16 | 2 | -6/+6 |
|\ \ | | | | | | | Make CoGroupedRDDs explicitly have the same key type. | ||||
| * | | Make CoGroupedRDDs explicitly have the same key type. | Stephen Haberman | 2013-02-16 | 2 | -6/+6 |
| |/ | |||||
* | | Add assertion about dependencies. | Stephen Haberman | 2013-02-16 | 1 | -1/+1 |
| | | |||||
* | | Avoid a shuffle if combineByKey is passed the same partitioner. | Stephen Haberman | 2013-02-16 | 1 | -1/+3 |
|/ | |||||
* | use appid instead of frameworkid; simplify stupid condition | Imran Rashid | 2013-02-13 | 1 | -2/+2 |
| | |||||
* | include jobid in Executor commandline args | Imran Rashid | 2013-02-13 | 2 | -3/+4 |
| | |||||
* | Merge pull request #465 from pwendell/java-sort-fix | Matei Zaharia | 2013-02-11 | 1 | -1/+1 |
|\ | | | | | SPARK-696: sortByKey should use 'ascending' parameter | ||||
| * | SPARK-696: sortByKey should use 'ascending' parameter | Patrick Wendell | 2013-02-11 | 1 | -1/+1 |
| | | |||||
* | | Fixed an exponential recursion that could happen with doCheckpoint due | Matei Zaharia | 2013-02-11 | 1 | -4/+10 |
| | | | | | | | | to lack of memoization | ||||
* | | Some bug and formatting fixes to FT | Matei Zaharia | 2013-02-10 | 5 | -16/+21 |
| | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala | ||||
* | | Detect hard crashes of workers using a heartbeat mechanism. | root | 2013-02-10 | 8 | -7/+62 |
| | | | | | | | | | | | | | | | | | | | | | | Also fixes some issues in the rest of the code with detecting workers this way. Conflicts: core/src/main/scala/spark/deploy/master/Master.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/scheduler/cluster/SparkDeploySchedulerBackend.scala core/src/main/scala/spark/scheduler/cluster/StandaloneClusterMessage.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala | ||||
* | | Tweak web UI so that people don't get confused about master URL format | Matei Zaharia | 2013-02-10 | 5 | -11/+14 |
|/ | | | | | | Conflicts: core/src/main/twirl/spark/deploy/master/index.scala.html core/src/main/twirl/spark/deploy/worker/index.scala.html | ||||
* | Merge pull request #448 from squito/fetch_maxBytesInFlight | Matei Zaharia | 2013-02-09 | 1 | -1/+1 |
|\ | | | | | add as many fetch requests as we can, subject to maxBytesInFlight | ||||
| * | add as many fetch requests as we can, subject to maxBytesInFlight | Imran Rashid | 2013-02-05 | 1 | -1/+1 |
| | | |||||
* | | Merge pull request #452 from stephenh/misc | Matei Zaharia | 2013-02-09 | 24 | -92/+83 |
|\ \ | | | | | | | Add RDD.coalesce, clean up some RDDs, other misc. | ||||
| * | | Move JavaRDDLike.coalesce into the right places. | Stephen Haberman | 2013-02-09 | 4 | -5/+15 |
| | | | |||||
| * | | Fix JavaRDDLike.coalesce return type. | Stephen Haberman | 2013-02-09 | 1 | -1/+1 |
| | | | |||||
| * | | Add back return types. | Stephen Haberman | 2013-02-09 | 19 | -31/+35 |
| | | | |||||
| * | | Remove RDD.coalesce default arguments. | Stephen Haberman | 2013-02-09 | 2 | -7/+1 |
| | | | |||||
| * | | Clean up RDDs, mainly to use getSplits. | Stephen Haberman | 2013-02-05 | 11 | -59/+30 |
| | | | | | | | | | | | | | | | | | | Also made sure clearDependencies() was calling super, to ensure the getSplits/getDependencies vars in the RDD base class get cleaned up. | ||||
| * | | Remove unneeded zipWithIndex. | Stephen Haberman | 2013-02-05 | 1 | -5/+7 |
| | | | | | | | | | | | | Also rename r->rdd and remove unneeded extra type info. | ||||
| * | | Add RDD.coalesce. | Stephen Haberman | 2013-02-05 | 2 | -0/+17 |
| | | | |||||
| * | | Add private, minor formatting. | Stephen Haberman | 2013-02-05 | 1 | -21/+14 |
| | | | |||||
* | | | Merge branch 'master' of https://github.com/mesos/spark into commutative | Mark Hamstra | 2013-02-08 | 122 | -1855/+3813 |
|\| | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/RDD.scala | ||||
| * | | Merge pull request #450 from stephenh/inlinemergepair | Matei Zaharia | 2013-02-05 | 1 | -6/+4 |
| |\ \ | | | | | | | | | Inline mergePair to look more like the narrow dep branch. | ||||
| | * | | Inline mergePair to look more like the narrow dep branch. | Stephen Haberman | 2013-02-05 | 1 | -6/+4 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | No functionality changes, I think this is just more consistent given mergePair isn't called multiple times/recursive. Also added a comment to explain the usual case of having two parent RDDs. | ||||
| * | | | Merge branch 'master' into fixdeathpactexception | Stephen Haberman | 2013-02-05 | 18 | -233/+293 |
| |\ \ \ | | | |/ | | |/| | | | | | | | | | Conflicts: core/src/main/scala/spark/deploy/worker/Worker.scala | ||||
| | * | | Merge pull request #445 from JoshRosen/pyspark_fixes | Matei Zaharia | 2013-02-03 | 1 | -2/+9 |
| | |\ \ | | | | | | | | | | | Fix exit status in PySpark unit tests; fix/optimize PySpark's RDD.take() | ||||
| | | * | | Fetch fewer objects in PySpark's take() method. | Josh Rosen | 2013-02-03 | 1 | -2/+9 |
| | | |/ | |||||
| | * | | Merge pull request #379 from stephenh/sparkmem | Matei Zaharia | 2013-02-02 | 6 | -35/+17 |
| | |\ \ | | | | | | | | | | | Add spark.executor.memory to differentiate executor memory from spark-shell | ||||
| | | * | | Fix dangling old variable names. | Stephen Haberman | 2013-02-02 | 1 | -2/+2 |
| | | | | | |||||
| | | * | | Move executorMemory up into SchedulerBackend. | Stephen Haberman | 2013-02-02 | 4 | -29/+12 |
| | | | | | |||||
| | | * | | Merge branch 'master' into sparkmem | Stephen Haberman | 2013-02-02 | 112 | -1486/+2872 |
| | | |\| | |||||
| | | * | | Fix SPARK_MEM in ExecutorRunner. | Stephen Haberman | 2013-01-22 | 1 | -1/+1 |
| | | | | | |||||
| | | * | | Restore SPARK_MEM in executorEnvs. | Stephen Haberman | 2013-01-22 | 1 | -2/+3 |
| | | | | | |||||
| | | * | | Add spark.executor.memory to differentiate executor memory from spark-shell ↵ | Stephen Haberman | 2013-01-15 | 3 | -10/+8 |
| | | | | | | | | | | | | | | | | | | | | memory. | ||||
| | * | | | Merge pull request #422 from squito/blockmanager_info | Matei Zaharia | 2013-02-02 | 6 | -38/+59 |
| | |\ \ \ | | | | | | | | | | | | | RDDInfo available from SparkContext | ||||
| | | * | | | remove unneeded (and unused) filter on block info | Imran Rashid | 2013-02-01 | 1 | -2/+0 |
| | | | | | | |||||
| | | * | | | track total partitions, in addition to cached partitions; use scala string ↵ | Imran Rashid | 2013-02-01 | 3 | -9/+13 |
| | | | | | | | | | | | | | | | | | | | | | | | | formatting | ||||
| | | * | | | fixup merge (master -> driver renaming) | Imran Rashid | 2013-02-01 | 1 | -1/+1 |
| | | | | | | |||||
| | | * | | | Merge branch 'master' into blockmanager_info | Imran Rashid | 2013-01-30 | 27 | -222/+259 |
| | | |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/storage/BlockManagerMaster.scala | ||||
| | | * | | | | rename Slaves --> Executor | Imran Rashid | 2013-01-30 | 2 | -5/+5 |
| | | | | | | | |||||
| | | * | | | | Merge branch 'master' into blockmanager_info | Imran Rashid | 2013-01-29 | 19 | -179/+183 |
| | | |\ \ \ \ | |||||
| | | * | | | | | better formatting for RDDInfo | Imran Rashid | 2013-01-28 | 1 | -3/+9 |
| | | | | | | | | |||||
| | | * | | | | | expose RDD & storage info directly via SparkContext | Imran Rashid | 2013-01-28 | 4 | -28/+41 |
| | | | | | | | | |||||
| | * | | | | | | Merge pull request #436 from stephenh/removeextraloop | Matei Zaharia | 2013-02-02 | 1 | -13/+10 |
| | |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | Once we find a split with no block, we don't have to look for more. |