Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Don't use SPARK_LAUNCH_WITH_SCALA in pyspark | Matei Zaharia | 2013-08-29 | 1 | -5/+0 |
| | |||||
* | Find assembly correctly in pyspark | Matei Zaharia | 2013-08-29 | 1 | -1/+3 |
| | |||||
* | Fix finding of assembly JAR, as well as some pointers to ./run | Matei Zaharia | 2013-08-29 | 13 | -17/+18 |
| | |||||
* | Provide more memory for tests | Matei Zaharia | 2013-08-29 | 2 | -2/+2 |
| | |||||
* | Fix PySpark for assembly run and include it in dist | Matei Zaharia | 2013-08-29 | 6 | -5/+41 |
| | |||||
* | Change build and run instructions to use assemblies | Matei Zaharia | 2013-08-29 | 25 | -199/+187 |
| | | | | | | | | | | | | | | | | This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel. | ||||
* | Merge pull request #871 from pwendell/expose-local | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
|\ | | | | | Expose `isLocal` in SparkContext. | ||||
| * | Make local variable public | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
| | | |||||
* | | Merge pull request #873 from pwendell/master | Matei Zaharia | 2013-08-28 | 1 | -1/+1 |
|\ \ | | | | | | | Hot fix for command runner | ||||
| * | | Adding extra args | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
| | | | |||||
| * | | Hot fix for command runner | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
| |/ | |||||
* | | Merge pull request #865 from tgravescs/fixtmpdir | Matei Zaharia | 2013-08-28 | 5 | -3/+49 |
|\ \ | | | | | | | Spark on Yarn should use yarn approved directories for spark.local.dir and tmp | ||||
| * | | Change Executor to only look at the env variable SPARK_YARN_MODE | Y.CORP.YAHOO.COM\tgraves | 2013-08-28 | 1 | -1/+1 |
| | | | |||||
| * | | Updated based on review comments. | Y.CORP.YAHOO.COM\tgraves | 2013-08-27 | 2 | -18/+12 |
| | | | |||||
| * | | Allow for Executors to have different directories then the Spark Master for Yarn | Y.CORP.YAHOO.COM\tgraves | 2013-08-27 | 1 | -0/+25 |
| | | | |||||
| * | | Update docs and remove old reference to --user option | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 1 | -3/+1 |
| | | | |||||
| * | | Throw exception if the yarn local dirs isn't set | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 1 | -1/+5 |
| | | | |||||
| * | | Change to use Yarn appropriate directories rather then /tmp or the user ↵ | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 3 | -0/+25 |
| |/ | | | | | | | specified spark.local.dir | ||||
* | | Merge pull request #867 from tgravescs/yarnenvconfigs | Matei Zaharia | 2013-08-27 | 3 | -2/+13 |
|\ \ | | | | | | | Spark on Yarn allow users to specify environment variables | ||||
| * | | fix typos | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 2 | -2/+2 |
| | | | |||||
| * | | Add ability for user to specify environment variables | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 3 | -2/+13 |
| |/ | |||||
* | | Merge pull request #859 from ianbuss/sbt_opts | Matei Zaharia | 2013-08-26 | 1 | -1/+1 |
|\ \ | | | | | | | Pass SBT_OPTS environment through to sbt_launcher | ||||
| * | | Pass SBT_OPTS environment through to sbt_launcher | Ian Buss | 2013-08-23 | 1 | -1/+1 |
| | | | |||||
* | | | Merge pull request #864 from rxin/json1 | shivaram | 2013-08-26 | 8 | -64/+77 |
|\ \ \ | |_|/ |/| | | Revert json library change | ||||
| * | | Added worker state to the cluster master JSON ui. | Reynold Xin | 2013-08-26 | 1 | -1/+2 |
| | | | |||||
| * | | Revert "Merge pull request #841 from rxin/json" | Reynold Xin | 2013-08-26 | 8 | -64/+76 |
|/ / | | | | | | | | | This reverts commit 1fb1b0992838c8cdd57eec45793e67a0490f1a52, reversing changes made to c69c48947d5102c81a9425cb380d861c3903685c. | ||||
* | | Merge pull request #603 from pwendell/ec2-updates | Patrick Wendell | 2013-08-24 | 4 | -165/+146 |
|\ \ | | | | | | | Several Improvements to EC2 Scripts | ||||
| * | | Version bump for ec2 docs | Patrick Wendell | 2013-08-24 | 1 | -1/+1 |
| | | | |||||
| * | | Merge remote-tracking branch 'mesos/master' into ec2-updates | Patrick Wendell | 2013-08-24 | 24 | -120/+762 |
| |\ \ | |/ / |/| | | |||||
* | | | Merge pull request #860 from jey/sbt-ide-fixes | Matei Zaharia | 2013-08-23 | 2 | -7/+14 |
|\ \ \ | |_|/ |/| | | Fix IDE project generation under SBT | ||||
| * | | Upgrade SBT IDE project generators | Jey Kottalam | 2013-08-23 | 1 | -2/+2 |
| | | | |||||
| * | | Fix SBT generation of IDE project files | Jey Kottalam | 2013-08-23 | 1 | -5/+12 |
|/ / | |||||
* | | Merge pull request #701 from ScrapCodes/documentation-suggestions | Matei Zaharia | 2013-08-22 | 5 | -6/+58 |
|\ \ | | | | | | | Documentation suggestions for spark streaming. | ||||
| * | | Linking custom receiver guide | Prashant Sharma | 2013-08-23 | 3 | -0/+7 |
| | | | |||||
| * | | Corrections in documentation comment | Prashant Sharma | 2013-08-23 | 1 | -3/+3 |
| | | | |||||
| * | | Improved documentation for spark custom receiver | Prashant Sharma | 2013-08-23 | 1 | -3/+48 |
| | | | |||||
* | | | Fix code style and a nondeterministic RDD issue in ALS | Matei Zaharia | 2013-08-22 | 1 | -11/+20 |
| | | | |||||
* | | | Merge pull request #814 from holdenk/master | Matei Zaharia | 2013-08-22 | 1 | -7/+14 |
|\ \ \ | | | | | | | | | Create less instances of the random class during ALS initialization. | ||||
| * | | | Fix | Holden Karau | 2013-08-15 | 1 | -2/+2 |
| | | | | |||||
| * | | | Code review feedback :) | Holden Karau | 2013-08-12 | 1 | -7/+7 |
| | | | | |||||
| * | | | Use less instances of the random class during ALS setup | Holden Karau | 2013-08-12 | 1 | -7/+14 |
| | | | | |||||
* | | | | Merge pull request #856 from jey/sbt-fix-hadoop-0.23.9 | Matei Zaharia | 2013-08-22 | 1 | -0/+1 |
|\ \ \ \ | | | | | | | | | | | Re-add removed dependency to fix build under Hadoop 0.23.9 | ||||
| * | | | | Re-add removed dependency on 'commons-daemon' | Jey Kottalam | 2013-08-22 | 1 | -0/+1 |
|/ / / / | | | | | | | | | | | | | Fixes SBT build under Hadoop 0.23.9 and 2.0.4 | ||||
* | | | | Merge pull request #855 from jey/update-build-docs | Matei Zaharia | 2013-08-22 | 7 | -40/+83 |
|\ \ \ \ | | | | | | | | | | | Update build docs | ||||
| * | | | | Increase ReservedCodeCacheSize to 256m | Jey Kottalam | 2013-08-21 | 1 | -1/+1 |
| | | | | | |||||
| * | | | | Use Hadoop 1.2.1 in application example | Jey Kottalam | 2013-08-21 | 1 | -5/+4 |
| | | | | | |||||
| * | | | | Revert "Allow build configuration to be set in conf/spark-env.sh" | Jey Kottalam | 2013-08-21 | 3 | -23/+7 |
| | | | | | | | | | | | | | | | | | | | | This reverts commit 66e7a38a3229eeb6d980193048ebebcda1522acb. | ||||
| * | | | | Remove references to unsupported Hadoop versions | Jey Kottalam | 2013-08-21 | 3 | -6/+5 |
| | | | | | |||||
| * | | | | Example should make sense | Jey Kottalam | 2013-08-21 | 1 | -5/+5 |
| | | | | | |||||
| * | | | | Update build docs | Jey Kottalam | 2013-08-21 | 3 | -28/+73 |
| | | | | |