aboutsummaryrefslogtreecommitdiff
path: root/yarn
Commit message (Collapse)AuthorAgeFilesLines
* [maven-release-plugin] prepare for next development iterationUbuntu2014-07-142-2/+2
|
* [maven-release-plugin] prepare release v1.0.1-rc3Ubuntu2014-07-142-2/+2
|
* Updating versions for branch-1.0Patrick Wendell2014-07-121-1/+1
|
* [maven-release-plugin] prepare for next development iterationUbuntu2014-07-042-2/+2
|
* [maven-release-plugin] prepare release v1.0.1-rc2v1.0.1Ubuntu2014-07-042-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.1-rc1"Patrick Wendell2014-06-272-2/+2
| | | | This reverts commit 7feeda3d729f9397aa15ee8750c01ef5aa601962.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-06-272-2/+2
| | | | This reverts commit ea1a455a755f83f46fc8bf242410917d93d0c52c.
* [maven-release-plugin] prepare for next development iterationUbuntu2014-06-262-2/+2
|
* [maven-release-plugin] prepare release v1.0.1-rc1Ubuntu2014-06-262-2/+2
|
* Updating versions for 1.0.1 releasePatrick Wendell2014-06-261-1/+1
|
* Remove use of spark.worker.instancesKay Ousterhout2014-06-261-1/+1
| | | | | | | | | | | | | | | | | | | spark.worker.instances was added as part of this commit: https://github.com/apache/spark/commit/1617816090e7b20124a512a43860a21232ebf511 My understanding is that SPARK_WORKER_INSTANCES is supported for backwards compatibility, but spark.worker.instances is never used (SparkSubmit.scala sets spark.executor.instances) so should not have been added. @sryza @pwendell @tgravescs LMK if I'm understanding this correctly Author: Kay Ousterhout <kayousterhout@gmail.com> Closes #1214 from kayousterhout/yarn_config and squashes the following commits: 3d7c491 [Kay Ousterhout] Remove use of spark.worker.instances (cherry picked from commit 48a82a827c99526b165c78d7e88faec43568a37a) Signed-off-by: Thomas Graves <tgraves@apache.org>
* [SPARK-1930] The Container is running beyond physical memory limits, so as ↵witgo2014-06-166-18/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to be killed Author: witgo <witgo@qq.com> Closes #894 from witgo/SPARK-1930 and squashes the following commits: 564307e [witgo] Update the running-on-yarn.md 3747515 [witgo] Merge branch 'master' of https://github.com/apache/spark into SPARK-1930 172647b [witgo] add memoryOverhead docs a0ff545 [witgo] leaving only two configs a17bda2 [witgo] Merge branch 'master' of https://github.com/apache/spark into SPARK-1930 478ca15 [witgo] Merge branch 'master' into SPARK-1930 d1244a1 [witgo] Merge branch 'master' into SPARK-1930 8b967ae [witgo] Merge branch 'master' into SPARK-1930 655a820 [witgo] review commit 71859a7 [witgo] Merge branch 'master' of https://github.com/apache/spark into SPARK-1930 e3c531d [witgo] review commit e16f190 [witgo] different memoryOverhead ffa7569 [witgo] review commit 5c9581f [witgo] Merge branch 'master' into SPARK-1930 9a6bcf2 [witgo] review commit 8fae45a [witgo] fix NullPointerException e0dcc16 [witgo] Adding configuration items b6a989c [witgo] Fix container memory beyond limit, were killed (cherry picked from commit cdf2b04570871848442ca9f9e2316a37e4aaaae0) Signed-off-by: Thomas Graves <tgraves@apache.org>
* [SPARK-2080] Yarn: report HS URL in client mode, correct user in cluster mode.Thomas Graves2014-06-122-4/+6
| | | | | | | | | | | | | | | | | | | | | | | | | Yarn client mode was not setting the app's tracking URL to the History Server's URL when configured by the user. Now client mode behaves the same as cluster mode. In SparkContext.scala, the "user.name" system property had precedence over the SPARK_USER environment variable. This means that SPARK_USER was never used, since "user.name" is always set by the JVM. In Yarn cluster mode, this means the application always reported itself as being run by user "yarn" (or whatever user was running the Yarn NM). One could argue that the correct fix would be to use UGI.getCurrentUser() here, but at least for Yarn that will match what SPARK_USER is set to. Author: Marcelo Vanzin <vanzin@cloudera.com> This patch had conflicts when merged, resolved by Committer: Thomas Graves <tgraves@apache.org> Closes #1002 from vanzin/yarn-client-url and squashes the following commits: 4046e04 [Marcelo Vanzin] Set HS link in yarn-alpha also. 4c692d9 [Marcelo Vanzin] Yarn: report HS URL in client mode, correct user in cluster mode.
* [SPARK-1978] In some cases, spark-yarn does not automatically restart the ↵witgo2014-06-102-27/+34
| | | | | | | | | | | | | | | | | | | failed container Author: witgo <witgo@qq.com> Closes #921 from witgo/allocateExecutors and squashes the following commits: bc3aa66 [witgo] review commit 8800eba [witgo] Merge branch 'master' of https://github.com/apache/spark into allocateExecutors 32ac7af [witgo] review commit 056b8c7 [witgo] Merge branch 'master' of https://github.com/apache/spark into allocateExecutors 04c6f7e [witgo] Merge branch 'master' into allocateExecutors aff827c [witgo] review commit 5c376e0 [witgo] Merge branch 'master' of https://github.com/apache/spark into allocateExecutors 1faf4f4 [witgo] Merge branch 'master' into allocateExecutors 3c464bd [witgo] add time limit to allocateExecutors e00b656 [witgo] In some cases, yarn does not automatically restart the container
* Improve maven plugin configurationwitgo2014-06-011-30/+0
| | | | | | | | | | | | | Author: witgo <witgo@qq.com> Closes #786 from witgo/maven_plugin and squashes the following commits: 5de86a2 [witgo] Merge branch 'master' of https://github.com/apache/spark into maven_plugin c35ef73 [witgo] Improve maven plugin configuration Conflicts: pom.xml
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-05-262-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc11v1.0.0Tathagata Das2014-05-262-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc11"Tathagata Das2014-05-262-2/+2
| | | | This reverts commit 2f1dc868e5714882cf40d2633fb66772baf34789.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-05-262-2/+2
| | | | This reverts commit 832dc594e7666f1d402334f8015ce29917d9c888.
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-05-252-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc11Tathagata Das2014-05-252-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc10"Tathagata Das2014-05-252-2/+2
| | | | This reverts commit d807023479ce10aec28ef3c1ab646ddefc2e663c.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-05-252-2/+2
| | | | This reverts commit 67dd53d2556f03ce292e6889128cf441f1aa48f8.
* [SPARK-1870] Make spark-submit --jars work in yarn-cluster mode.Xiangrui Meng2014-05-222-53/+18
| | | | | | | | | | | | | | | | | | | | | | | | Sent secondary jars to distributed cache of all containers and add the cached jars to classpath before executors start. Tested on a YARN cluster (CDH-5.0). `spark-submit --jars` also works in standalone server and `yarn-client`. Thanks for @andrewor14 for testing! I removed "Doesn't work for drivers in standalone mode with "cluster" deploy mode." from `spark-submit`'s help message, though we haven't tested mesos yet. CC: @dbtsai @sryza Author: Xiangrui Meng <meng@databricks.com> Closes #848 from mengxr/yarn-classpath and squashes the following commits: 23e7df4 [Xiangrui Meng] rename spark.jar to __spark__.jar and app.jar to __app__.jar to avoid confliction apped $CWD/ and $CWD/* to the classpath remove unused methods a40f6ed [Xiangrui Meng] standalone -> cluster 65e04ad [Xiangrui Meng] update spark-submit help message and add a comment for yarn-client 11e5354 [Xiangrui Meng] minor changes 3e7e1c4 [Xiangrui Meng] use sparkConf instead of hadoop conf dc3c825 [Xiangrui Meng] add secondary jars to classpath in yarn (cherry picked from commit dba314029b4c9d72d7e48a2093b39edd01931f57) Signed-off-by: Tathagata Das <tathagata.das1565@gmail.com>
* [Typo] Stoped -> StoppedAndrew Or2014-05-211-1/+1
| | | | | | | | | | | Author: Andrew Or <andrewor14@gmail.com> Closes #847 from andrewor14/yarn-typo and squashes the following commits: c1906af [Andrew Or] Stoped -> Stopped (cherry picked from commit ba5d4a99425a2083fea2a9759050c5e770197e23) Signed-off-by: Reynold Xin <rxin@apache.org>
* [maven-release-plugin] prepare for next development iterationTathagata Das2014-05-202-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc10Tathagata Das2014-05-202-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc9"Tathagata Das2014-05-192-2/+2
| | | | This reverts commit 920f947eb5a22a679c0c3186cf69ee75f6041c75.
* Revert "[maven-release-plugin] prepare for next development iteration"Tathagata Das2014-05-192-2/+2
| | | | This reverts commit f8e611955096c5c1c7db5764b9d2851b1d295f0d.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-172-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc9Patrick Wendell2014-05-172-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc8"Patrick Wendell2014-05-162-2/+2
| | | | This reverts commit 80eea0f111c06260ffaa780d2f3f7facd09c17bc.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-162-2/+2
| | | | This reverts commit e5436b8c1a79ce108f3af402455ac5f6dc5d1eb3.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-162-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc8Patrick Wendell2014-05-162-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc7"Patrick Wendell2014-05-162-2/+2
| | | | This reverts commit 9212b3e5bb5545ccfce242da8d89108e6fb1c464.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-162-2/+2
| | | | This reverts commit c4746aa6fe4aaf383e69e34353114d36d1eb9ba6.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-152-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc7Patrick Wendell2014-05-152-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc6"Patrick Wendell2014-05-142-2/+2
| | | | This reverts commit 54133abdce0246f6643a1112a5204afb2c4caa82.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-142-2/+2
| | | | This reverts commit e480bcfbd269ae1d7a6a92cfb50466cf192fe1fb.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-142-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc6Patrick Wendell2014-05-142-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc5"Patrick Wendell2014-05-142-2/+2
| | | | This reverts commit 18f062303303824139998e8fc8f4158217b0dbc3.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-142-2/+2
| | | | This reverts commit d08e9604fc9958b7c768e91715c8152db2ed6fd0.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-132-2/+2
|
* [maven-release-plugin] prepare release v1.0.0-rc5Patrick Wendell2014-05-132-2/+2
|
* Revert "[maven-release-plugin] prepare release v1.0.0-rc4"Patrick Wendell2014-05-122-2/+2
| | | | This reverts commit 3d0a44833ab50360bf9feccc861cb5e8c44a4866.
* Revert "[maven-release-plugin] prepare for next development iteration"Patrick Wendell2014-05-122-2/+2
| | | | This reverts commit 9772d85c6f3893d42044f4bab0e16f8b6287613a.
* [maven-release-plugin] prepare for next development iterationPatrick Wendell2014-05-132-2/+2
|
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 18:03:59 +0000