aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'master' into scala-2.10Prashant Sharma2013-10-0157-136/+252
|\ | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressUI.scala docs/_config.yml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * Merge pull request #17 from rxin/optimizeReynold Xin2013-09-262-2/+1
| |\ | | | | | | | | | Remove -optimize flag
| | * Removed scala -optimize flag.Reynold Xin2013-09-262-2/+1
| | |
| * | Merge pull request #16 from pwendell/masterReynold Xin2013-09-261-1/+1
| |\ \ | | | | | | | | | | | | Bug fix in master build
| | * | Bug fix in master buildPatrick Wendell2013-09-261-1/+1
| | | |
| * | | Merge pull request #14 from kayousterhout/untangle_schedulerReynold Xin2013-09-2634-71/+62
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Improved organization of scheduling packages. This commit does not change any code -- only file organization. Please let me know if there was some masterminded strategy behind the existing organization that I failed to understand! There are two components of this change: (1) Moving files out of the cluster package, and down a level to the scheduling package. These files are all used by the local scheduler in addition to the cluster scheduler(s), so should not be in the cluster package. As a result of this change, none of the files in the local package reference files in the cluster package. (2) Moving the mesos package to within the cluster package. The mesos scheduling code is for a cluster, and represents a specific case of cluster scheduling (the Mesos-related classes often subclass cluster scheduling classes). Thus, the most logical place for it seems to be within the cluster package. The one thing about the scheduling code that seems a little funny to me is the naming of the SchedulerBackends. The StandaloneSchedulerBackend is not just for Standalone mode, but instead is used by Mesos coarse grained mode and Yarn, and the backend that *is* just for Standalone mode is instead called SparkDeploySchedulerBackend. I didn't change this because I wasn't sure if there was a reason for this naming that I'm just not aware of.
| | * | | Improved organization of scheduling packages.Kay Ousterhout2013-09-2534-71/+62
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit does not change any code -- only file organization. There are two components of this change: (1) Moving files out of the cluster package, and down a level to the scheduling package. These files are all used by the local scheduler in addition to the cluster scheduler(s), so should not be in the cluster package. As a result of this change, none of the files in the local package reference files in the cluster package. (2) Moving the mesos package to within the cluster package. The mesos scheduling code is for a cluster, and represents a specific case of cluster scheduling (the Mesos-related classes often subclass cluster scheduling classes). Thus, the most logical place for it is within the cluster package.
| * | | | Merge pull request #670 from jey/ec2-ssh-improvementsReynold Xin2013-09-261-26/+80
| |\ \ \ \ | | |_|_|/ | |/| | | EC2 SSH improvements
| | * | | Clarify error messages on SSH failureJey Kottalam2013-09-111-6/+21
| | | | |
| | * | | Generate new SSH key for the cluster, make "--identity-file" optionalJey Kottalam2013-09-061-21/+37
| | | | |
| | * | | Construct shell commands as sequences for safety and composabilityJey Kottalam2013-09-061-11/+34
| | | | |
| * | | | Merge pull request #930 from holdenk/masterReynold Xin2013-09-262-1/+11
| |\ \ \ \ | | | | | | | | | | | | Add mapPartitionsWithIndex
| | * | | | Fix formatting :)Holden Karau2013-09-231-4/+5
| | | | | |
| | * | | | Switch indent from 2 to 4 spacesHolden Karau2013-09-221-2/+2
| | | | | |
| | * | | | Fix build on ubuntuHolden Karau2013-09-141-1/+1
| | | | | |
| | * | | | Merge branch 'master' of https://github.com/mesos/sparkHolden Karau2013-09-142-5/+12
| | |\ \ \ \
| | * | | | | Make mapPartitionsWithIndex work with JavaRDD'sHolden Karau2013-09-141-2/+3
| | | | | | |
| | * | | | | Start of working on SPARK-615Holden Karau2013-09-111-0/+8
| | | | | | |
| * | | | | | Merge pull request #7 from wannabeast/memorystore-fixesReynold Xin2013-09-261-6/+8
| |\ \ \ \ \ \ | | |_|_|_|_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | some minor fixes to MemoryStore This is a repeat of #5, moved to its own branch in my repo. This makes all updates to on ; it skips on synchronizing the reads where it can get away with it.
| | * | | | | Synchronize on "entries" the remaining update to "currentMemory".Mike2013-09-191-3/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Make "currentMemory" @volatile, so that it's reads in ensureFreeSpace() are atomic and up-to-date--i.e., currentMemory can't increase while putLock is held (though it could decrease, which would only help ensureFreeSpace()).
| | * | | | | Set currentMemory to 0 in clear().Mike2013-09-111-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Remove unnecessary entries.get() call.
| | * | | | | Remove MemoryStore$Entry.dropPending, unused as of 42e0a68082.Mike2013-09-101-1/+1
| | | | | | |
| * | | | | | Merge pull request #9 from rxin/limitPatrick Wendell2013-09-262-10/+66
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Smarter take/limit implementation.
| | * | | | | | Smarter take/limit implementation.Reynold Xin2013-09-202-10/+66
| | | | | | | |
| * | | | | | | Merge remote-tracking branch 'apache-github/pr/13' into HEADPatrick Wendell2013-09-2414-15/+15
| |\ \ \ \ \ \ \ | | |_|_|_|_|_|/ | |/| | | | | |
| | * | | | | | Update build version in masterPatrick Wendell2013-09-2414-15/+15
| |/ / / / / /
| * | | | | | Merge remote-tracking branch 'pr/12'Reynold Xin2013-09-232-4/+6
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Fix spacing so java.io.tmpdir doesn't run on with SPARK_JAVA_OPTS
| | * | | | | | Fix spacing so that the java.io.tmpdir doesn't run on with SPARK_JAVA_OPTSY.CORP.YAHOO.COM\tgraves2013-09-232-4/+6
| | |/ / / / /
| * | | | | | Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/incubator-sparkReynold Xin2013-09-230-0/+0
| |\| | | | |
| | * | | | | Merge branch 'master' of github.com:mesos/sparkReynold Xin2013-09-2096-543/+2068
| | |\ \ \ \ \ | | | |/ / / / | | |/| | | |
| * | | | | | Merge branch 'master' of github.com:markhamstra/incubator-sparkReynold Xin2013-09-231-1/+0
| |\ \ \ \ \ \
| | * | | | | | Removed repetative import; fixes hidden definition compiler warning.Mark Hamstra2013-09-031-1/+0
| | |/ / / / /
| * | | | | | Merge branch 'master' of github.com:mesos/sparkReynold Xin2013-09-237-64/+123
| |\ \ \ \ \ \
| * \ \ \ \ \ \ Merge branch 'master' of github.com:mesos/sparkReynold Xin2013-09-1696-539/+2064
| |\ \ \ \ \ \ \ | | |_|/ / / / / | |/| | | | | |
| * | | | | | | Merge remote-tracking branch 'old/master'Matei Zaharia2013-09-0249-138/+950
| |\ \ \ \ \ \ \
| * \ \ \ \ \ \ \ Merge remote-tracking branch 'old/master'Matei Zaharia2013-09-01616-8290/+16182
| |\ \ \ \ \ \ \ \
| * \ \ \ \ \ \ \ \ Merge remote-tracking branch 'old/master'Matei Zaharia2013-07-16532-2195/+15893
| |\ \ \ \ \ \ \ \ \
| * | | | | | | | | | Test commit karma for Spark git.Henry Saputra2013-07-151-0/+1
| | | | | | | | | | |
* | | | | | | | | | | Fixed non termination of Executor backend, when sc.stop is not called.Prashant Sharma2013-09-302-2/+2
| | | | | | | | | | |
* | | | | | | | | | | Fixed UISuite, for case when port 4040 is already bound on machine running ↵Prashant Sharma2013-09-261-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the test.
* | | | | | | | | | | Sync with master and some build fixesPrashant Sharma2013-09-2612-78/+138
|\ \ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|/ / / / / | |/| | | | | | | | |
| * | | | | | | | | | Merge pull request #928 from jerryshao/fairscheduler-refactorReynold Xin2013-09-221-43/+56
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Refactor FairSchedulableBuilder
| | * | | | | | | | | | Change Exception to NoSuchElementException and minor style fixjerryshao2013-09-221-6/+7
| | | | | | | | | | | |
| | * | | | | | | | | | Remove infix style and othersjerryshao2013-09-221-10/+8
| | | | | | | | | | | |
| | * | | | | | | | | | Refactor FairSchedulableBuilder:jerryshao2013-09-221-39/+53
| |/ / / / / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 1. Configuration can be read from classpath if not set explicitly. 2. Add missing close handler.
| * | | | | | | | | | Merge pull request #937 from jerryshao/localProperties-fixReynold Xin2013-09-212-2/+50
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Fix PR926 local properties issues in Spark Streaming like scenarios
| | * | | | | | | | | | Add barrier for local properties unit test and fix some stylesjerryshao2013-09-222-3/+11
| | | | | | | | | | | |
| | * | | | | | | | | | Fix issue when local properties pass from parent to child threadjerryshao2013-09-182-2/+42
| | | |_|_|_|/ / / / / | | |/| | | | | | | |
| * | | | | | | | | | Merge pull request #941 from ilikerps/masterReynold Xin2013-09-211-6/+6
| |\ \ \ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | Add "org.apache." prefix to packages in spark-class
| | * | | | | | | | | | Add "org.apache." prefix to packages in spark-classAaron Davidson2013-09-201-6/+6
| | | |_|_|_|_|/ / / / | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lacking this, the if/case statements never trigger on Spark 0.8.0+.
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-09 21:00:38 +0000