Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Fixes suggested by Patrick | Matei Zaharia | 2013-08-31 | 4 | -14/+8 |
| | |||||
* | More updates, describing changes to recommended use of environment vars | Matei Zaharia | 2013-08-31 | 8 | -93/+84 |
| | | | | and new Python stuff | ||||
* | Update docs about HDFS versions | Matei Zaharia | 2013-08-30 | 3 | -20/+41 |
| | |||||
* | More doc improvements + better warnings when you haven't built Spark | Matei Zaharia | 2013-08-30 | 7 | -57/+58 |
| | |||||
* | New hardware provisioning doc, and updates to menus | Matei Zaharia | 2013-08-30 | 2 | -9/+83 |
| | |||||
* | Change docs color theme for 0.8 | Matei Zaharia | 2013-08-30 | 1 | -1/+1 |
| | |||||
* | Merge pull request #875 from shivaram/build-fix | Reynold Xin | 2013-08-30 | 2 | -11/+8 |
|\ | | | | | Fix broken build by removing addIntercept | ||||
| * | Fix broken build by removing addIntercept | Shivaram Venkataraman | 2013-08-30 | 2 | -11/+8 |
|/ | |||||
* | Merge pull request #863 from shivaram/etrain-ridge | Evan Sparks | 2013-08-29 | 12 | -297/+755 |
|\ | | | | | Adding linear regression and refactoring Ridge regression to use SGD | ||||
| * | Center & scale variables in Ridge, Lasso. | Shivaram Venkataraman | 2013-08-25 | 10 | -228/+347 |
| | | | | | | | | | | Also add a unit test that checks if ridge regression lowers cross-validation error. | ||||
| * | Fixing typos in Java tests, and addressing alignment issues. | Evan Sparks | 2013-08-18 | 4 | -16/+16 |
| | | |||||
| * | Centralizing linear data generator and mllib regression tests to use it. | Evan Sparks | 2013-08-18 | 9 | -282/+84 |
| | | |||||
| * | Adding Linear Regression, and refactoring Ridge Regression. | Evan Sparks | 2013-08-18 | 8 | -176/+713 |
| | | |||||
* | | Merge pull request #819 from shivaram/sgd-cleanup | Evan Sparks | 2013-08-29 | 8 | -48/+160 |
|\ \ | | | | | | | Change SVM to use {0,1} labels | ||||
| * | | Add an option to turn off data validation, test it. | Shivaram Venkataraman | 2013-08-25 | 5 | -18/+28 |
| | | | | | | | | | | | | | | | Also moves addIntercept to have default true to make it similar to validateData option | ||||
| * | | Specify label format in LogisticRegression. | Shivaram Venkataraman | 2013-08-13 | 1 | -0/+6 |
| | | | |||||
| * | | Fix SVM model and unit test to work with {0,1}. | Shivaram Venkataraman | 2013-08-13 | 5 | -12/+18 |
| | | | | | | | | | | | | Also rename validateFuncs to validators. | ||||
| * | | Change SVM to use {0,1} labels. | Shivaram Venkataraman | 2013-08-13 | 7 | -26/+116 |
| | | | | | | | | | | | | | | | Also add a data validation check to make sure classification labels are always 0 or 1 and add an appropriate test case. | ||||
* | | | Merge pull request #857 from mateiz/assembly | Matei Zaharia | 2013-08-29 | 49 | -310/+504 |
|\ \ \ | | | | | | | | | Change build and run instructions to use assemblies | ||||
| * | | | Update Maven docs | Matei Zaharia | 2013-08-29 | 1 | -29/+26 |
| | | | | |||||
| * | | | Fix path to assembly in make-distribution.sh | Matei Zaharia | 2013-08-29 | 1 | -1/+1 |
| | | | | |||||
| * | | | Update some build instructions because only sbt assembly and mvn package | Matei Zaharia | 2013-08-29 | 5 | -15/+15 |
| | | | | | | | | | | | | | | | | are now needed | ||||
| * | | | Update Maven build to create assemblies expected by new scripts | Matei Zaharia | 2013-08-29 | 11 | -47/+222 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This includes the following changes: - The "assembly" package now builds in Maven by default, and creates an assembly containing both hadoop-client and Spark, unlike the old BigTop distribution assembly that skipped hadoop-client - There is now a bigtop-dist package to build the old BigTop assembly - The repl-bin package is no longer built by default since the scripts don't reply on it; instead it can be enabled with -Prepl-bin - Py4J is now included in the assembly/lib folder as a local Maven repo, so that the Maven package can link to it - run-example now adds the original Spark classpath as well because the Maven examples assembly lists spark-core and such as provided - The various Maven projects add a spark-yarn dependency correctly | ||||
| * | | | Don't use SPARK_LAUNCH_WITH_SCALA in pyspark | Matei Zaharia | 2013-08-29 | 1 | -5/+0 |
| | | | | |||||
| * | | | Find assembly correctly in pyspark | Matei Zaharia | 2013-08-29 | 1 | -1/+3 |
| | | | | |||||
| * | | | Fix finding of assembly JAR, as well as some pointers to ./run | Matei Zaharia | 2013-08-29 | 13 | -17/+18 |
| | | | | |||||
| * | | | Provide more memory for tests | Matei Zaharia | 2013-08-29 | 2 | -2/+2 |
| | | | | |||||
| * | | | Fix PySpark for assembly run and include it in dist | Matei Zaharia | 2013-08-29 | 6 | -5/+41 |
| | | | | |||||
| * | | | Change build and run instructions to use assemblies | Matei Zaharia | 2013-08-29 | 25 | -199/+187 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel. | ||||
* | | | | Merge pull request #874 from jerryshao/fix-report-bug | Reynold Xin | 2013-08-29 | 1 | -2/+2 |
|\ \ \ \ | |/ / / |/| | | | Fix removed block zero size log reporting | ||||
| * | | | Fix removed block zero size log reporting | jerryshao | 2013-08-30 | 1 | -2/+2 |
|/ / / | |||||
* | | | Merge pull request #871 from pwendell/expose-local | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
|\ \ \ | | | | | | | | | Expose `isLocal` in SparkContext. | ||||
| * | | | Make local variable public | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
| | | | | |||||
* | | | | Merge pull request #873 from pwendell/master | Matei Zaharia | 2013-08-28 | 1 | -1/+1 |
|\ \ \ \ | | | | | | | | | | | Hot fix for command runner | ||||
| * | | | | Adding extra args | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
| | | | | | |||||
| * | | | | Hot fix for command runner | Patrick Wendell | 2013-08-28 | 1 | -1/+1 |
| |/ / / | |||||
* | | | | Merge pull request #865 from tgravescs/fixtmpdir | Matei Zaharia | 2013-08-28 | 5 | -3/+49 |
|\ \ \ \ | | | | | | | | | | | Spark on Yarn should use yarn approved directories for spark.local.dir and tmp | ||||
| * | | | | Change Executor to only look at the env variable SPARK_YARN_MODE | Y.CORP.YAHOO.COM\tgraves | 2013-08-28 | 1 | -1/+1 |
| | | | | | |||||
| * | | | | Updated based on review comments. | Y.CORP.YAHOO.COM\tgraves | 2013-08-27 | 2 | -18/+12 |
| | | | | | |||||
| * | | | | Allow for Executors to have different directories then the Spark Master for Yarn | Y.CORP.YAHOO.COM\tgraves | 2013-08-27 | 1 | -0/+25 |
| | | | | | |||||
| * | | | | Update docs and remove old reference to --user option | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 1 | -3/+1 |
| | | | | | |||||
| * | | | | Throw exception if the yarn local dirs isn't set | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 1 | -1/+5 |
| | | | | | |||||
| * | | | | Change to use Yarn appropriate directories rather then /tmp or the user ↵ | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 3 | -0/+25 |
| |/ / / | | | | | | | | | | | | | specified spark.local.dir | ||||
* | | | | Merge pull request #867 from tgravescs/yarnenvconfigs | Matei Zaharia | 2013-08-27 | 3 | -2/+13 |
|\ \ \ \ | | | | | | | | | | | Spark on Yarn allow users to specify environment variables | ||||
| * | | | | fix typos | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 2 | -2/+2 |
| | | | | | |||||
| * | | | | Add ability for user to specify environment variables | Y.CORP.YAHOO.COM\tgraves | 2013-08-26 | 3 | -2/+13 |
| |/ / / | |||||
* | | | | Merge pull request #859 from ianbuss/sbt_opts | Matei Zaharia | 2013-08-26 | 1 | -1/+1 |
|\ \ \ \ | | | | | | | | | | | Pass SBT_OPTS environment through to sbt_launcher | ||||
| * | | | | Pass SBT_OPTS environment through to sbt_launcher | Ian Buss | 2013-08-23 | 1 | -1/+1 |
| | | | | | |||||
* | | | | | Merge pull request #864 from rxin/json1 | shivaram | 2013-08-26 | 8 | -64/+77 |
|\ \ \ \ \ | |_|/ / / |/| | | | | Revert json library change | ||||
| * | | | | Added worker state to the cluster master JSON ui. | Reynold Xin | 2013-08-26 | 1 | -1/+2 |
| | | | | |