aboutsummaryrefslogtreecommitdiff
path: root/yarn
Commit message (Collapse)AuthorAgeFilesLines
* Merge remote branch 'upstream/master' into consolidate_schedulersKay Ousterhout2013-12-204-17/+15
|\ | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/scheduler/cluster/ClusterTaskSetManager.scala core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala core/src/test/scala/org/apache/spark/scheduler/TaskSetManagerSuite.scala
| * Merge pull request #265 from markhamstra/scala.binary.versionPatrick Wendell2013-12-151-4/+4
| |\ | | | | | | | | | | | | | | | | | | | | | DRY out the POMs with scala.binary.version ...instead of hard-coding 2.10 repeatedly. As long as it's not a `<project>`-level `<artifactId>`, I think that we are okay parameterizing these.
| | * Use scala.binary.version in POMsMark Hamstra2013-12-151-4/+4
| | |
| * | Merge pull request #257 from tgravescs/sparkYarnFixNameReynold Xin2013-12-151-0/+1
| |\ \ | | |/ | |/| | | | | | | | | | Fix the --name option for Spark on Yarn Looks like the --name option accidentally got broken in one of the merges. The Client hangs if the --name option is used right now.
| | * Fix the --name option for Spark on YarnThomas Graves2013-12-121-0/+1
| | |
| * | Merge branch 'master' into akka-bug-fixPrashant Sharma2013-12-114-309/+445
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala pom.xml project/SparkBuild.scala streaming/pom.xml yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
| * | Style fixes and addressed review comments at #221Prashant Sharma2013-12-101-4/+4
| | |
| * | fixed yarn buildPrashant Sharma2013-12-092-11/+8
| | |
| * | Incorporated Patrick's feedback comment on #211 and made maven ↵Prashant Sharma2013-12-071-1/+1
| | | | | | | | | | | | build/dep-resolution atleast a bit faster.
| * | A left over akka -> akka.tcp changesPrashant Sharma2013-12-061-1/+1
| | |
| * | Merge branch 'master' into wip-scala-2.10Prashant Sharma2013-11-275-21/+434
| |\ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/api/python/PythonRDD.scala core/src/main/scala/org/apache/spark/rdd/MapPartitionsRDD.scala core/src/main/scala/org/apache/spark/rdd/MapPartitionsWithContextRDD.scala core/src/main/scala/org/apache/spark/rdd/RDD.scala python/pyspark/rdd.py
| * \ \ Merge branch 'master' of github.com:apache/incubator-spark into scala-2.10-tempPrashant Sharma2013-11-217-68/+90
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/util/collection/PrimitiveVector.scala streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala
| * \ \ \ Merge branch 'master' into scala-2.10Raymond Liu2013-11-146-172/+646
| |\ \ \ \
| * \ \ \ \ Merge branch 'master' into scala-2.10Raymond Liu2013-11-135-51/+278
| |\ \ \ \ \
| * \ \ \ \ \ Merge branch 'master' into wip-merge-masterPrashant Sharma2013-10-081-3/+3
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: bagel/pom.xml core/pom.xml core/src/test/scala/org/apache/spark/ui/UISuite.scala examples/pom.xml mllib/pom.xml pom.xml project/SparkBuild.scala repl/pom.xml streaming/pom.xml tools/pom.xml In scala 2.10, a shorter representation is used for naming artifacts so changed to shorter scala version for artifacts and made it a property in pom.
| * \ \ \ \ \ \ Merge branch 'master' into scala-2.10Prashant Sharma2013-10-052-1/+7
| |\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/test/scala/org/apache/spark/DistributedSuite.scala project/SparkBuild.scala
| * \ \ \ \ \ \ \ Merge branch 'master' into scala-2.10Prashant Sharma2013-10-013-5/+7
| |\ \ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/ui/jobs/JobProgressUI.scala docs/_config.yml project/SparkBuild.scala repl/src/main/scala/org/apache/spark/repl/SparkILoop.scala
| * | | | | | | | | fixed maven build for scala 2.10Prashant Sharma2013-09-261-2/+2
| | | | | | | | | |
| * | | | | | | | | Akka 2.2 migrationPrashant Sharma2013-09-221-1/+1
| | | | | | | | | |
* | | | | | | | | | Merge master into 127Aaron Davidson2013-12-0811-365/+936
|\ \ \ \ \ \ \ \ \ \ | | |_|_|_|_|_|_|_|/ | |/| | | | | | | |
| * | | | | | | | | Merge remote-tracking branch 'origin/master' into yarn-2.2Harvey Feng2013-11-265-19/+433
| |\ \ \ \ \ \ \ \ \ | | | |_|_|_|_|_|_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala
| | * | | | | | | | Add YarnClientClusterScheduler and Backend.Raymond Liu2013-11-225-21/+434
| | | |_|_|_|_|_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | With this scheduler, the user application is launched locally, While the executor will be launched by YARN on remote nodes. This enables spark-shell to run upon YARN.
| * | | | | | | | A few more style fixes in `yarn` package.Harvey Feng2013-11-233-45/+71
| | | | | | | | |
| * | | | | | | | Merge branch 'master' into yarn-cleanupHarvey Feng2013-11-216-51/+81
| |\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala yarn/src/main/scala/org/apache/spark/deploy/yarn/WorkerRunnable.scala yarn/src/main/scala/org/apache/spark/deploy/yarn/YarnAllocationHandler.scala
| | * | | | | | | Merge branch 'master' into removesemicolonscalaHenry Saputra2013-11-193-29/+58
| | |\ \ \ \ \ \ \
| | | * | | | | | | Impove Spark on Yarn Error handlingtgravescs2013-11-193-29/+58
| | | | | | | | | |
| | * | | | | | | | Another set of changes to remove unnecessary semicolon (;) from Scala code.Henry Saputra2013-11-194-5/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Passed the sbt/sbt compile and test
| | * | | | | | | | Remove the semicolons at the end of Scala code to make it more pure Scala code.Henry Saputra2013-11-195-26/+21
| | |/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also remove unused imports as I found them along the way. Remove return statements when returning value in the Scala code. Passing compile and tests.
| * / / / / / / / Misc style changes in the 'yarn' package.Harvey Feng2013-11-174-273/+376
| |/ / / / / / /
| * | | | | | / Simple cleanup on Spark's Scala code while testing core and yarn modules:Henry Saputra2013-11-154-8/+6
| | |_|_|_|_|/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | -) Remove some of unused imports as I found them -) Remove ";" in the imports statements -) Remove () at the end of method call like size that does not have size effect.
* | | | | | | Fixed naming issues and added back ability to specify max task failures.Kay Ousterhout2013-11-131-3/+3
| | | | | | |
* | | | | | | Merge remote-tracking branch 'upstream/master' into consolidate_schedulersKay Ousterhout2013-11-136-172/+646
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/scheduler/ClusterScheduler.scala
| * | | | | | Allow spark on yarn to be run from HDFS. Allows the spark.jar, app.jar, and ↵tgravescs2013-11-046-172/+646
| | |_|_|_|/ | |/| | | | | | | | | | | | | | | | log4j.properties to be put into hdfs.
* / | | | | Deduplicate Local and Cluster schedulers.Kay Ousterhout2013-10-301-3/+7
|/ / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The code in LocalScheduler/LocalTaskSetManager was nearly identical to the code in ClusterScheduler/ClusterTaskSetManager. The redundancy made making updating the schedulers unnecessarily painful and error- prone. This commit combines the two into a single TaskScheduler/ TaskSetManager.
* | | | | Fix the Worker to use CoarseGrainedExecutorBackend and modify classpath to ↵tgravescs2013-10-212-10/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | be explicit about inclusion of spark.jar and app.jar
* | | | | Fix yarn buildtgravescs2013-10-161-2/+2
| | | | |
* | | | | Merge remote-tracking branch 'tgravescs/sparkYarnDistCache'Matei Zaharia2013-10-104-45/+253
|\ \ \ \ \ | |_|_|_|/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Closes #11 Conflicts: docs/running-on-yarn.md yarn/src/main/scala/org/apache/spark/deploy/yarn/ClientArguments.scala
| * | | | Adding in the --addJars option to make SparkContext.addJar work on yarn and ↵tgravescs2013-10-033-20/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | cleanup the classpaths
| * | | | Support distributed cache files and archives on spark on yarn and attempt to ↵Y.CORP.YAHOO.COM\tgraves2013-09-234-29/+226
| |/ / / | | | | | | | | | | | | cleanup the staging directory on exit
* | | / Merging build changes in from 0.8Patrick Wendell2013-10-051-3/+3
| |_|/ |/| |
* | | Add default value to usage statementtgravescs2013-10-031-1/+1
| | |
* | | Allow users to set the application name for Spark on Yarntgravescs2013-10-022-1/+7
| |/ |/|
* | Update build version in masterPatrick Wendell2013-09-241-1/+1
| |
* | Fix spacing so that the java.io.tmpdir doesn't run on with SPARK_JAVA_OPTSY.CORP.YAHOO.COM\tgraves2013-09-232-4/+6
|/
* Use different Hadoop version for YARN artifacts.Patrick Wendell2013-09-131-0/+5
| | | | | This uses a seperate Hadoop version for YARN artifact. This means when people link against spark-yarn, things will resolve correctly.
* Fix YARN assembly generation under MavenJey Kottalam2013-09-061-33/+28
|
* Review comment changes and update to org.apache packagingY.CORP.YAHOO.COM\tgraves2013-09-037-15/+18
|
* Merge remote-tracking branch 'mesos/master' into yarnUILinkY.CORP.YAHOO.COM\tgraves2013-09-039-29/+29
|\ | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/ui/UIUtils.scala core/src/main/scala/org/apache/spark/ui/jobs/PoolTable.scala core/src/main/scala/org/apache/spark/ui/jobs/StageTable.scala docs/running-on-yarn.md
| * Add missing license headers found with RATMatei Zaharia2013-09-021-2/+2
| |
| * Fix some URLsMatei Zaharia2013-09-011-1/+1
| |
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-08 14:42:36 +0000