aboutsummaryrefslogtreecommitdiff
path: root/dev/merge_spark_pr.py
Commit message (Collapse)AuthorAgeFilesLines
* [SPARK-8652] [PYSPARK] Check return value for all uses of doctest.testmod()Josh Rosen2015-06-261-1/+3
| | | | | | | | | | | | | | | | This patch addresses a critical issue in the PySpark tests: Several of our Python modules' `__main__` methods call `doctest.testmod()` in order to run doctests but forget to check and handle its return value. As a result, some PySpark test failures can go unnoticed because they will not fail the build. Fortunately, there was only one test failure which was masked by this bug: a `pyspark.profiler` doctest was failing due to changes in RDD pipelining. Author: Josh Rosen <joshrosen@databricks.com> Closes #7032 from JoshRosen/testmod-fix and squashes the following commits: 60dbdc0 [Josh Rosen] Account for int vs. long formatting change in Python 3 8b8d80a [Josh Rosen] Fix failing test. e6423f9 [Josh Rosen] Check return code for all uses of doctest.testmod().
* [SPARK-7933] Remove Patrick's username/pw from merge scriptKay Ousterhout2015-05-281-2/+2
| | | | | | | | | | Looks like this was added by accident when pwendell merged a commit back in September: fe2b1d6a209db9fe96b1c6630677955b94bd48c9 Author: Kay Ousterhout <kayousterhout@gmail.com> Closes #6485 from kayousterhout/SPARK-7933 and squashes the following commits: 7c6164a [Kay Ousterhout] [SPARK-7933] Remove Patrick's username/pw from merge script
* Fix install jira-pythonDavies Liu2015-05-231-1/+1
| | | | | | | | | | | | | | jira-pytyhon package should be installed by sudo pip install jira cc pwendell Author: Davies Liu <davies@databricks.com> Closes #6367 from davies/fix_jira_python2 and squashes the following commits: fbb3c8e [Davies Liu] Fix install jira-python
* [HOTFIX] Bug in merge scriptPatrick Wendell2015-05-131-2/+1
|
* [SPARK-7592] Always set resolution to "Fixed" in PR merge script.Patrick Wendell2015-05-121-1/+4
| | | | | | | | | | | | | | | | | The issue is that the behavior of the ASF JIRA silently changed. Now when the "Resolve Issue" transition occurs, the default resolution is "Pending Closed". We used to count on the default behavior being to set the resolution as "Fixed". The solution is to explicitly set the resolution as "Fixed" and not count on default behavior. Author: Patrick Wendell <patrick@databricks.com> Closes #6103 from pwendell/merge-script-fix and squashes the following commits: dcc16a6 [Patrick Wendell] Always set resolution to "Fixed" in PR merge script.
* [SPARK-1684] [PROJECT INFRA] Merge script should standardize SPARK-XXX prefixtexasmichelle2015-04-211-59/+140
| | | | | | | | | | | | | | | | | | | | | | | Cleans up the pull request title in the merge script to follow conventions outlined in the wiki under Contributing Code. https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark#ContributingtoSpark-ContributingCode [MODULE] SPARK-XXXX: Description Author: texasmichelle <texasmichelle@gmail.com> Closes #5149 from texasmichelle/master and squashes the following commits: 9b6b0a7 [texasmichelle] resolved variable scope issue 7d5fa20 [texasmichelle] only prompt if title has been modified 8c195bb [texasmichelle] removed erroneous line 4f1ed46 [texasmichelle] Deque removal, logic simplifications, & prompt user to pick a title (orig or modified) df73f6a [texasmichelle] reworked regex's to enforce brackets around JIRA ref 43b5aed [texasmichelle] Merge remote-tracking branch 'apache/master' 25229c6 [texasmichelle] Merge remote-tracking branch 'apache/master' aa20a6e [texasmichelle] Move code into main() and add doctest for new text parsing method 48520ba [texasmichelle] SPARK-1684: Corrected import statement 042099d [texasmichelle] SPARK-1684 Merge script should standardize SPARK-XXX prefix 8f4a7d1 [texasmichelle] SPARK-1684 Merge script should standardize SPARK-XXX prefix
* SPARK-5776 JIRA version not of form x.y.z breaks merge_spark_pr.pySean Owen2015-02-121-0/+2
| | | | | | | | | | | Consider only x.y.z verisons from JIRA. CC JoshRosen who will probably know this script well. Alternative is to call the version "2.0.0" after all in JIRA. Author: Sean Owen <sowen@cloudera.com> Closes #4570 from srowen/SPARK-5776 and squashes the following commits: fffafde [Sean Owen] Consider only x.y.z verisons from JIRA
* SPARK-4507: PR merge script should support closing multiple JIRA ticketsTakayuki Hasegawa2014-11-291-7/+11
| | | | | | | | | | | | | | This will fix SPARK-4507. For pull requests that reference multiple JIRAs in their titles, it would be helpful if the PR merge script offered to close all of them. Author: Takayuki Hasegawa <takayuki.hasegawa0311@gmail.com> Closes #3428 from hase1031/SPARK-4507 and squashes the following commits: bf6d64b [Takayuki Hasegawa] SPARK-4507: try to resolve issue when no JIRAs in title 401224c [Takayuki Hasegawa] SPARK-4507: moved codes as before ce89021 [Takayuki Hasegawa] SPARK-4507: PR merge script should support closing multiple JIRA tickets
* HOTFIX: Fix unicode error in merge script.Patrick Wendell2014-10-051-2/+1
| | | | | | | | | | | | | | | | | | The merge script builds up a big command array and sometimes this contains both unicode and ascii strings. This doesn't work if you try to join them into a single string. Longer term a solution is to go and make sure the source of all strings is unicode. This patch provides a simpler solution... just print the array rather than joining. I actually prefer printing an array here anyways since joining on spaces is lossy in the case of arguments that themselves contain spaces. Author: Patrick Wendell <pwendell@gmail.com> Closes #2645 from pwendell/merge-script and squashes the following commits: 167b792 [Patrick Wendell] HOTFIX: Fix unicode error in merge script.
* [SPARK-3425] do not set MaxPermSize for OpenJDK 1.8Matthew Farrellee2014-09-151-2/+2
| | | | | | | | | | Closes #2387 Author: Matthew Farrellee <matt@redhat.com> Closes #2301 from mattf/SPARK-3425 and squashes the following commits: 20f3c09 [Matthew Farrellee] [SPARK-3425] do not set MaxPermSize for OpenJDK 1.8
* [SPARK-2894] spark-shell doesn't accept flagsKousuke Saruta2014-08-091-0/+2
| | | | | | | | | | | | | | | | | | | | As sryza reported, spark-shell doesn't accept any flags. The root cause is wrong usage of spark-submit in spark-shell and it come to the surface by #1801 Author: Kousuke Saruta <sarutak@oss.nttdata.co.jp> Author: Cheng Lian <lian.cs.zju@gmail.com> Closes #1715, Closes #1864, and Closes #1861 Closes #1825 from sarutak/SPARK-2894 and squashes the following commits: 47f3510 [Kousuke Saruta] Merge branch 'master' of git://git.apache.org/spark into SPARK-2894 2c899ed [Kousuke Saruta] Removed useless code from java_gateway.py 98287ed [Kousuke Saruta] Removed useless code from java_gateway.py 513ad2e [Kousuke Saruta] Modified util.sh to enable to use option including white spaces 28a374e [Kousuke Saruta] Modified java_gateway.py to recognize arguments 5afc584 [Cheng Lian] Filter out spark-submit options when starting Python gateway e630d19 [Cheng Lian] Fixing pyspark and spark-shell CLI options
* Improvements to merge_spark_pr.pyJosh Rosen2014-07-311-19/+34
| | | | | | | | | | | | | | | | This commit fixes a couple of issues in the merge_spark_pr.py developer script: - Allow recovery from failed cherry-picks. - Fix detection of pull requests that have already been merged. Both of these fixes are useful when backporting changes. Author: Josh Rosen <joshrosen@apache.org> Closes #1668 from JoshRosen/pr-script-improvements and squashes the following commits: ff4f33a [Josh Rosen] Default SPARK_HOME to cwd(); detect missing JIRA credentials. ed5bc57 [Josh Rosen] Improvements for backporting using merge_spark_pr:
* Strip '@' symbols when merging pull requests.Patrick Wendell2014-06-261-1/+3
| | | | | | | | | | | | | Currently all of the commits with 'X' in them cause person X to receive e-mails every time someone makes a public fork of Spark. marmbrus who requested this. Author: Patrick Wendell <pwendell@gmail.com> Closes #1239 from pwendell/strip and squashes the following commits: 22e5a97 [Patrick Wendell] Strip '@' symbols when merging pull requests.
* HOTFIX: Support empty body in merge scriptPatrick Wendell2014-06-071-2/+3
| | | | | | | | | | Discovered in #992 Author: Patrick Wendell <pwendell@gmail.com> Closes #1007 from pwendell/hotfix and squashes the following commits: af90aa0 [Patrick Wendell] HOTFIX: Support empty body in merge script
* use env default python in merge_spark_pr.pyXiangrui Meng2014-06-031-1/+1
| | | | | | | | | | A minor change to use env default python instead of fixed `/usr/bin/python`. Author: Xiangrui Meng <meng@databricks.com> Closes #965 from mengxr/merge-pr-python and squashes the following commits: 1ae0013 [Xiangrui Meng] use env default python in merge_spark_pr.py
* Updated dev Python scripts to make them PEP8 compliant.Reynold Xin2014-05-261-194/+208
| | | | | | | | Author: Reynold Xin <rxin@apache.org> Closes #875 from rxin/pep8-dev-scripts and squashes the following commits: 04b084f [Reynold Xin] Made dev Python scripts PEP8 compliant.
* HOTFIX: Minor patch to merge script.Patrick Wendell2014-04-271-1/+1
|
* SPARK-1648 Support closing JIRA's as part of merge script.Patrick Wendell2014-04-271-9/+105
| | | | | | | | | | | | | | | | | Adds an automated hook in the merge script that can close the JIRA, set the fix versions, and leave a comment on the JIRA indicating the PR in which it was resolved. This ensures that (a) we always close JIRA's when issues are merged and (b) there is a link to the pull request in every JIRA. This requires a python library called `jira-client`. We could look at embedding this library in our project, but it seemed simple enough to just gracefully disable this feature if it is not installed. It can be installed with `pip install jira-client`. Author: Patrick Wendell <pwendell@gmail.com> Closes #570 from pwendell/jira-pr-merge and squashes the following commits: 3022b96 [Patrick Wendell] SPARK-1648 Support closing JIRA's as part of merge script.
* SPARK-1431: Allow merging conflicting pull requestsPatrick Wendell2014-04-061-4/+22
| | | | | | | | | | | Sometimes if there is a small conflict it's nice to be able to just manually fix it up rather than have another RTT with the contributor. Author: Patrick Wendell <pwendell@gmail.com> Closes #342 from pwendell/merge-conflicts and squashes the following commits: cdce61a [Patrick Wendell] SPARK-1431: Allow merging conflicting pull requests
* SPARK-1151: Update dev merge script to use spark.git instead of incubator-sparkThomas Graves2014-02-281-1/+1
| | | | | | | | Author: Thomas Graves <tgraves@apache.org> Closes #47 from tgravescs/fix_merge_script and squashes the following commits: 8209ab1 [Thomas Graves] Update dev merge script to use spark.git instead of incubator-spark
* SPARK-1073 Keep GitHub pull request title as commit summaryAndrew Ash2014-02-121-9/+10
| | | | | | | | | | | | | | | | | | | | | | | The first line of a git commit message is the line that's used with many git tools as the most concise textual description of that message. The most common use that I see is in the short log, which is a one line per commit log of recent commits. This commit moves the line Merge pull request #%s from %s. Lower into the message to reserve the first line of the resulting commit for the much more important pull request title. http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html Author: Andrew Ash <andrew@andrewash.com> Closes #574 from ash211/gh-pr-merge-title and squashes the following commits: b240823 [Andrew Ash] More merge_message improvements d2986db [Andrew Ash] Keep GitHub pull request title as commit summary
* Merge pull request #569 from pwendell/merge-fixes.Patrick Wendell2014-02-091-4/+6
| | | | | | | | | | | | | | | | | Fixes bug where merges won't close associated pull request. Previously we added "Closes #XX" in the title. Github will sometimes linbreak the title in a way that causes this to not work. This patch instead adds the line in the body. This also makes the commit format more concise for merge commits. We might consider just dropping those in the future. Author: Patrick Wendell <pwendell@gmail.com> Closes #569 and squashes the following commits: 732eba1 [Patrick Wendell] Fixes bug where merges won't close associated pull request.
* Merge pull request #565 from pwendell/dev-scripts. Closes #565.Patrick Wendell2014-02-081-0/+197
SPARK-1066: Add developer scripts to repository. These are some developer scripts I've been maintaining in a separate public repo. This patch adds them to the Spark repository so they can evolve here and are clearly accessible to all committers. I may do some small additional clean-up in this PR, but wanted to put them here in case others want to review. There are a few types of scripts here: 1. A tool to merge pull requests. 2. A script for packaging releases. 3. A script for auditing release candidates. Author: Patrick Wendell <pwendell@gmail.com> == Merge branch commits == commit 5d5d331d01f6fd59c2eb830f652955119b012173 Author: Patrick Wendell <pwendell@gmail.com> Date: Sat Feb 8 22:11:47 2014 -0800 SPARK-1066: Add developer scripts to repository.
generated by cgit v1.2.3 (git 2.39.1) at 2024-07-07 05:43:52 +0000