aboutsummaryrefslogtreecommitdiff
path: root/yarn
Commit message (Collapse)AuthorAgeFilesLines
* Export --file for YarnClient mode to support sending extra files to worker ↵Raymond Liu2014-01-071-1/+3
| | | | on yarn cluster
* Minor typo fix for yarn clientRaymond Liu2014-01-072-2/+2
|
* merge upstream/masterliguoqiang2014-01-0322-73/+2251
|\
| * fix docs for yarnRaymond Liu2014-01-031-2/+2
| |
| * minor fix for loginfoRaymond Liu2014-01-031-1/+1
| |
| * move duplicate pom config into parent pomRaymond Liu2014-01-033-179/+84
| |
| * Using name yarn-alpha/yarn instead of yarn-2.0/yarn-2.2Raymond Liu2014-01-0314-12/+12
| |
| * Fix yarn/README.mdRaymond Liu2014-01-031-6/+4
| |
| * Clean up unused files for yarnRaymond Liu2014-01-034-311/+0
| |
| * Fix pom for build yarn/2.x with yarn/common into one jarRaymond Liu2014-01-033-34/+200
| |
| * merge yarn/scheduler yarn/common code into one directoryRaymond Liu2014-01-033-0/+0
| |
| * Need to send dummy hello message to actually estabilish akka connection.Raymond Liu2014-01-032-0/+4
| |
| * A few clean up for yarn 2.0 codeRaymond Liu2014-01-032-8/+7
| |
| * Fix yarn/README.md and update docs/running-on-yarn.mdRaymond Liu2014-01-031-2/+0
| |
| * Add README for yarn modulesRaymond Liu2014-01-031-0/+16
| |
| * some code clean up for Yarn 2.2Raymond Liu2014-01-032-3/+3
| |
| * Fix pom file for scala binary versionRaymond Liu2014-01-036-8/+8
| |
| * Fix yarn/assemble pom fileRaymond Liu2014-01-032-0/+75
| |
| * Change profile name new-yarn to hadoop2.2-yarnRaymond Liu2014-01-032-2/+2
| |
| * Fix pom for yarn code reorgnaize commitRaymond Liu2014-01-036-489/+245
| |
| * Reorganize yarn related codes into sub projects to remove duplicate files.Raymond Liu2014-01-0321-18/+2580
| |
* | Modify spark on yarn to create SparkConf processliguoqiang2014-01-034-27/+36
| |
* | Modify spark on yarn to create SparkConf processliguoqiang2014-01-037-22/+27
|/
* fix yarn-clientThomas Graves2014-01-021-4/+5
|
* Fix yarn build after sparkConf changesThomas Graves2014-01-025-56/+48
|
* Merge remote-tracking branch 'origin/master' into conf2Matei Zaharia2013-12-295-11/+19
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/SparkContext.scala core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala core/src/main/scala/org/apache/spark/scheduler/local/LocalScheduler.scala core/src/main/scala/org/apache/spark/util/MetadataCleaner.scala core/src/test/scala/org/apache/spark/scheduler/TaskResultGetterSuite.scala core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala new-yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala streaming/src/test/scala/org/apache/spark/streaming/BasicOperationsSuite.scala streaming/src/test/scala/org/apache/spark/streaming/CheckpointSuite.scala streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala streaming/src/test/scala/org/apache/spark/streaming/TestSuiteBase.scala streaming/src/test/scala/org/apache/spark/streaming/WindowOperationsSuite.scala
| * Renamed ClusterScheduler to TaskSchedulerImpl for yarn and new-yarnliguoqiang2013-12-261-2/+1
| |
| * Renamed ClusterScheduler to TaskSchedulerImpl for yarn and new-yarnliguoqiang2013-12-264-8/+11
| |
| * Merge remote branch 'upstream/master' into consolidate_schedulersKay Ousterhout2013-12-204-17/+15
| |\ | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
| * \ Merge master into 127Aaron Davidson2013-12-0811-365/+936
| |\ \
| * | | Fixed naming issues and added back ability to specify max task failures.Kay Ousterhout2013-11-131-3/+3
| | | |
| * | | Merge remote-tracking branch 'upstream/master' into consolidate_schedulersKay Ousterhout2013-11-136-172/+646
| |\ \ \ | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/scheduler/ClusterScheduler.scala
| * | | | Deduplicate Local and Cluster schedulers.Kay Ousterhout2013-10-301-3/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code in LocalScheduler/LocalTaskSetManager was nearly identical to the code in ClusterScheduler/ClusterTaskSetManager. The redundancy made making updating the schedulers unnecessarily painful and error- prone. This commit combines the two into a single TaskScheduler/ TaskSetManager.
* | | | | Various fixes to configuration codeMatei Zaharia2013-12-283-54/+54
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Got rid of global SparkContext.globalConf - Pass SparkConf to serializers and compression codecs - Made SparkConf public instead of private[spark] - Improved API of SparkContext and SparkConf - Switched executor environment vars to be passed through SparkConf - Fixed some places that were still using system properties - Fixed some tests, though others are still failing This still fails several tests in core, repl and streaming, likely due to properties not being set or cleared correctly (some of the tests run fine in isolation).
* | | | | spark-544, introducing SparkConf and related configuration overhaul.Prashant Sharma2013-12-256-17/+17
| |_|_|/ |/| | |
* | | | Merge pull request #265 from markhamstra/scala.binary.versionPatrick Wendell2013-12-151-4/+4
|\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | DRY out the POMs with scala.binary.version ...instead of hard-coding 2.10 repeatedly. As long as it's not a `<project>`-level `<artifactId>`, I think that we are okay parameterizing these.
| * | | | Use scala.binary.version in POMsMark Hamstra2013-12-151-4/+4
| | | | |
* | | | | Merge pull request #257 from tgravescs/sparkYarnFixNameReynold Xin2013-12-151-0/+1
|\ \ \ \ \ | |/ / / / |/| | | | | | | | | | | | | | | | | | | Fix the --name option for Spark on Yarn Looks like the --name option accidentally got broken in one of the merges. The Client hangs if the --name option is used right now.
| * | | | Fix the --name option for Spark on YarnThomas Graves2013-12-121-0/+1
| | |_|/ | |/| |
* | | | Merge branch 'master' into akka-bug-fixPrashant Sharma2013-12-114-309/+445
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala pom.xml project/SparkBuild.scala streaming/pom.xml yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
| * | | Merge remote-tracking branch 'origin/master' into yarn-2.2Harvey Feng2013-11-265-19/+433
| |\ \ \ | | | | | | | | | | | | | | | | | | | | Conflicts: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
| * | | | A few more style fixes in `yarn` package.Harvey Feng2013-11-233-45/+71
| | | | |
| * | | | Merge branch 'master' into yarn-cleanupHarvey Feng2013-11-216-51/+81
| |\ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala yarn/src/main/scala/org/apache/spark/deploy/yarn/WorkerRunnable.scala yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
| * | | | | Misc style changes in the 'yarn' package.Harvey Feng2013-11-174-273/+376
| | | | | |
* | | | | | Style fixes and addressed review comments at #221Prashant Sharma2013-12-101-4/+4
| | | | | |
* | | | | | fixed yarn buildPrashant Sharma2013-12-092-11/+8
| | | | | |
* | | | | | Incorporated Patrick's feedback comment on #211 and made maven ↵Prashant Sharma2013-12-071-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | build/dep-resolution atleast a bit faster.
* | | | | | A left over akka -> akka.tcp changesPrashant Sharma2013-12-061-1/+1
| | | | | |
* | | | | | Merge branch 'master' into wip-scala-2.10Prashant Sharma2013-11-275-21/+434
|\ \ \ \ \ \ | | |_|/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala core/src/main/scala/org/apache/spark/rdd/MapPartitionsRDD.scala core/src/main/scala/org/apache/spark/rdd/MapPartitionsWithContextRDD.scala core/src/main/scala/org/apache/spark/rdd/RDD.scala python/pyspark/rdd.py
| * | | | | Add YarnClientClusterScheduler and Backend.Raymond Liu2013-11-225-21/+434
| | |/ / / | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | With this scheduler, the user application is launched locally, While the executor will be launched by YARN on remote nodes. This enables spark-shell to run upon YARN.
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-10 15:55:17 +0000