aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Don't use SPARK_LAUNCH_WITH_SCALA in pysparkMatei Zaharia2013-08-291-5/+0
|
* Find assembly correctly in pysparkMatei Zaharia2013-08-291-1/+3
|
* Fix finding of assembly JAR, as well as some pointers to ./runMatei Zaharia2013-08-2913-17/+18
|
* Provide more memory for testsMatei Zaharia2013-08-292-2/+2
|
* Fix PySpark for assembly run and include it in distMatei Zaharia2013-08-296-5/+41
|
* Change build and run instructions to use assembliesMatei Zaharia2013-08-2925-199/+187
| | | | | | | | | | | | | | | | This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.
* Merge pull request #871 from pwendell/expose-localPatrick Wendell2013-08-281-1/+1
|\ | | | | Expose `isLocal` in SparkContext.
| * Make local variable publicPatrick Wendell2013-08-281-1/+1
| |
* | Merge pull request #873 from pwendell/masterMatei Zaharia2013-08-281-1/+1
|\ \ | | | | | | Hot fix for command runner
| * | Adding extra argsPatrick Wendell2013-08-281-1/+1
| | |
| * | Hot fix for command runnerPatrick Wendell2013-08-281-1/+1
| |/
* | Merge pull request #865 from tgravescs/fixtmpdirMatei Zaharia2013-08-285-3/+49
|\ \ | | | | | | Spark on Yarn should use yarn approved directories for spark.local.dir and tmp
| * | Change Executor to only look at the env variable SPARK_YARN_MODEY.CORP.YAHOO.COM\tgraves2013-08-281-1/+1
| | |
| * | Updated based on review comments.Y.CORP.YAHOO.COM\tgraves2013-08-272-18/+12
| | |
| * | Allow for Executors to have different directories then the Spark Master for YarnY.CORP.YAHOO.COM\tgraves2013-08-271-0/+25
| | |
| * | Update docs and remove old reference to --user optionY.CORP.YAHOO.COM\tgraves2013-08-261-3/+1
| | |
| * | Throw exception if the yarn local dirs isn't setY.CORP.YAHOO.COM\tgraves2013-08-261-1/+5
| | |
| * | Change to use Yarn appropriate directories rather then /tmp or the user ↵Y.CORP.YAHOO.COM\tgraves2013-08-263-0/+25
| |/ | | | | | | specified spark.local.dir
* | Merge pull request #867 from tgravescs/yarnenvconfigsMatei Zaharia2013-08-273-2/+13
|\ \ | | | | | | Spark on Yarn allow users to specify environment variables
| * | fix typosY.CORP.YAHOO.COM\tgraves2013-08-262-2/+2
| | |
| * | Add ability for user to specify environment variablesY.CORP.YAHOO.COM\tgraves2013-08-263-2/+13
| |/
* | Merge pull request #859 from ianbuss/sbt_optsMatei Zaharia2013-08-261-1/+1
|\ \ | | | | | | Pass SBT_OPTS environment through to sbt_launcher
| * | Pass SBT_OPTS environment through to sbt_launcherIan Buss2013-08-231-1/+1
| | |
* | | Merge pull request #864 from rxin/json1shivaram2013-08-268-64/+77
|\ \ \ | |_|/ |/| | Revert json library change
| * | Added worker state to the cluster master JSON ui.Reynold Xin2013-08-261-1/+2
| | |
| * | Revert "Merge pull request #841 from rxin/json"Reynold Xin2013-08-268-64/+76
|/ / | | | | | | | | This reverts commit 1fb1b0992838c8cdd57eec45793e67a0490f1a52, reversing changes made to c69c48947d5102c81a9425cb380d861c3903685c.
* | Merge pull request #603 from pwendell/ec2-updatesPatrick Wendell2013-08-244-165/+146
|\ \ | | | | | | Several Improvements to EC2 Scripts
| * | Version bump for ec2 docsPatrick Wendell2013-08-241-1/+1
| | |
| * | Merge remote-tracking branch 'mesos/master' into ec2-updatesPatrick Wendell2013-08-2424-120/+762
| |\ \ | |/ / |/| |
* | | Merge pull request #860 from jey/sbt-ide-fixesMatei Zaharia2013-08-232-7/+14
|\ \ \ | |_|/ |/| | Fix IDE project generation under SBT
| * | Upgrade SBT IDE project generatorsJey Kottalam2013-08-231-2/+2
| | |
| * | Fix SBT generation of IDE project filesJey Kottalam2013-08-231-5/+12
|/ /
* | Merge pull request #701 from ScrapCodes/documentation-suggestionsMatei Zaharia2013-08-225-6/+58
|\ \ | | | | | | Documentation suggestions for spark streaming.
| * | Linking custom receiver guidePrashant Sharma2013-08-233-0/+7
| | |
| * | Corrections in documentation commentPrashant Sharma2013-08-231-3/+3
| | |
| * | Improved documentation for spark custom receiverPrashant Sharma2013-08-231-3/+48
| | |
* | | Fix code style and a nondeterministic RDD issue in ALSMatei Zaharia2013-08-221-11/+20
| | |
* | | Merge pull request #814 from holdenk/masterMatei Zaharia2013-08-221-7/+14
|\ \ \ | | | | | | | | Create less instances of the random class during ALS initialization.
| * | | FixHolden Karau2013-08-151-2/+2
| | | |
| * | | Code review feedback :)Holden Karau2013-08-121-7/+7
| | | |
| * | | Use less instances of the random class during ALS setupHolden Karau2013-08-121-7/+14
| | | |
* | | | Merge pull request #856 from jey/sbt-fix-hadoop-0.23.9Matei Zaharia2013-08-221-0/+1
|\ \ \ \ | | | | | | | | | | Re-add removed dependency to fix build under Hadoop 0.23.9
| * | | | Re-add removed dependency on 'commons-daemon'Jey Kottalam2013-08-221-0/+1
|/ / / / | | | | | | | | | | | | Fixes SBT build under Hadoop 0.23.9 and 2.0.4
* | | | Merge pull request #855 from jey/update-build-docsMatei Zaharia2013-08-227-40/+83
|\ \ \ \ | | | | | | | | | | Update build docs
| * | | | Increase ReservedCodeCacheSize to 256mJey Kottalam2013-08-211-1/+1
| | | | |
| * | | | Use Hadoop 1.2.1 in application exampleJey Kottalam2013-08-211-5/+4
| | | | |
| * | | | Revert "Allow build configuration to be set in conf/spark-env.sh"Jey Kottalam2013-08-213-23/+7
| | | | | | | | | | | | | | | | | | | | This reverts commit 66e7a38a3229eeb6d980193048ebebcda1522acb.
| * | | | Remove references to unsupported Hadoop versionsJey Kottalam2013-08-213-6/+5
| | | | |
| * | | | Example should make senseJey Kottalam2013-08-211-5/+5
| | | | |
| * | | | Update build docsJey Kottalam2013-08-213-28/+73
| | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-04 07:16:24 +0000