aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Update docs about HDFS versionsMatei Zaharia2013-08-303-20/+41
|
* More doc improvements + better warnings when you haven't built SparkMatei Zaharia2013-08-307-57/+58
|
* New hardware provisioning doc, and updates to menusMatei Zaharia2013-08-302-9/+83
|
* Change docs color theme for 0.8Matei Zaharia2013-08-301-1/+1
|
* Merge pull request #875 from shivaram/build-fixReynold Xin2013-08-302-11/+8
|\ | | | | Fix broken build by removing addIntercept
| * Fix broken build by removing addInterceptShivaram Venkataraman2013-08-302-11/+8
|/
* Merge pull request #863 from shivaram/etrain-ridgeEvan Sparks2013-08-2912-297/+755
|\ | | | | Adding linear regression and refactoring Ridge regression to use SGD
| * Center & scale variables in Ridge, Lasso.Shivaram Venkataraman2013-08-2510-228/+347
| | | | | | | | | | Also add a unit test that checks if ridge regression lowers cross-validation error.
| * Fixing typos in Java tests, and addressing alignment issues.Evan Sparks2013-08-184-16/+16
| |
| * Centralizing linear data generator and mllib regression tests to use it.Evan Sparks2013-08-189-282/+84
| |
| * Adding Linear Regression, and refactoring Ridge Regression.Evan Sparks2013-08-188-176/+713
| |
* | Merge pull request #819 from shivaram/sgd-cleanupEvan Sparks2013-08-298-48/+160
|\ \ | | | | | | Change SVM to use {0,1} labels
| * | Add an option to turn off data validation, test it.Shivaram Venkataraman2013-08-255-18/+28
| | | | | | | | | | | | | | | Also moves addIntercept to have default true to make it similar to validateData option
| * | Specify label format in LogisticRegression.Shivaram Venkataraman2013-08-131-0/+6
| | |
| * | Fix SVM model and unit test to work with {0,1}.Shivaram Venkataraman2013-08-135-12/+18
| | | | | | | | | | | | Also rename validateFuncs to validators.
| * | Change SVM to use {0,1} labels.Shivaram Venkataraman2013-08-137-26/+116
| | | | | | | | | | | | | | | Also add a data validation check to make sure classification labels are always 0 or 1 and add an appropriate test case.
* | | Merge pull request #857 from mateiz/assemblyMatei Zaharia2013-08-2949-310/+504
|\ \ \ | | | | | | | | Change build and run instructions to use assemblies
| * | | Update Maven docsMatei Zaharia2013-08-291-29/+26
| | | |
| * | | Fix path to assembly in make-distribution.shMatei Zaharia2013-08-291-1/+1
| | | |
| * | | Update some build instructions because only sbt assembly and mvn packageMatei Zaharia2013-08-295-15/+15
| | | | | | | | | | | | | | | | are now needed
| * | | Update Maven build to create assemblies expected by new scriptsMatei Zaharia2013-08-2911-47/+222
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This includes the following changes: - The "assembly" package now builds in Maven by default, and creates an assembly containing both hadoop-client and Spark, unlike the old BigTop distribution assembly that skipped hadoop-client - There is now a bigtop-dist package to build the old BigTop assembly - The repl-bin package is no longer built by default since the scripts don't reply on it; instead it can be enabled with -Prepl-bin - Py4J is now included in the assembly/lib folder as a local Maven repo, so that the Maven package can link to it - run-example now adds the original Spark classpath as well because the Maven examples assembly lists spark-core and such as provided - The various Maven projects add a spark-yarn dependency correctly
| * | | Don't use SPARK_LAUNCH_WITH_SCALA in pysparkMatei Zaharia2013-08-291-5/+0
| | | |
| * | | Find assembly correctly in pysparkMatei Zaharia2013-08-291-1/+3
| | | |
| * | | Fix finding of assembly JAR, as well as some pointers to ./runMatei Zaharia2013-08-2913-17/+18
| | | |
| * | | Provide more memory for testsMatei Zaharia2013-08-292-2/+2
| | | |
| * | | Fix PySpark for assembly run and include it in distMatei Zaharia2013-08-296-5/+41
| | | |
| * | | Change build and run instructions to use assembliesMatei Zaharia2013-08-2925-199/+187
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.
* | | | Merge pull request #874 from jerryshao/fix-report-bugReynold Xin2013-08-291-2/+2
|\ \ \ \ | |/ / / |/| | | Fix removed block zero size log reporting
| * | | Fix removed block zero size log reportingjerryshao2013-08-301-2/+2
|/ / /
* | | Merge pull request #871 from pwendell/expose-localPatrick Wendell2013-08-281-1/+1
|\ \ \ | | | | | | | | Expose `isLocal` in SparkContext.
| * | | Make local variable publicPatrick Wendell2013-08-281-1/+1
| | | |
* | | | Merge pull request #873 from pwendell/masterMatei Zaharia2013-08-281-1/+1
|\ \ \ \ | | | | | | | | | | Hot fix for command runner
| * | | | Adding extra argsPatrick Wendell2013-08-281-1/+1
| | | | |
| * | | | Hot fix for command runnerPatrick Wendell2013-08-281-1/+1
| |/ / /
* | | | Merge pull request #865 from tgravescs/fixtmpdirMatei Zaharia2013-08-285-3/+49
|\ \ \ \ | | | | | | | | | | Spark on Yarn should use yarn approved directories for spark.local.dir and tmp
| * | | | Change Executor to only look at the env variable SPARK_YARN_MODEY.CORP.YAHOO.COM\tgraves2013-08-281-1/+1
| | | | |
| * | | | Updated based on review comments.Y.CORP.YAHOO.COM\tgraves2013-08-272-18/+12
| | | | |
| * | | | Allow for Executors to have different directories then the Spark Master for YarnY.CORP.YAHOO.COM\tgraves2013-08-271-0/+25
| | | | |
| * | | | Update docs and remove old reference to --user optionY.CORP.YAHOO.COM\tgraves2013-08-261-3/+1
| | | | |
| * | | | Throw exception if the yarn local dirs isn't setY.CORP.YAHOO.COM\tgraves2013-08-261-1/+5
| | | | |
| * | | | Change to use Yarn appropriate directories rather then /tmp or the user ↵Y.CORP.YAHOO.COM\tgraves2013-08-263-0/+25
| |/ / / | | | | | | | | | | | | specified spark.local.dir
* | | | Merge pull request #867 from tgravescs/yarnenvconfigsMatei Zaharia2013-08-273-2/+13
|\ \ \ \ | | | | | | | | | | Spark on Yarn allow users to specify environment variables
| * | | | fix typosY.CORP.YAHOO.COM\tgraves2013-08-262-2/+2
| | | | |
| * | | | Add ability for user to specify environment variablesY.CORP.YAHOO.COM\tgraves2013-08-263-2/+13
| |/ / /
* | | | Merge pull request #859 from ianbuss/sbt_optsMatei Zaharia2013-08-261-1/+1
|\ \ \ \ | | | | | | | | | | Pass SBT_OPTS environment through to sbt_launcher
| * | | | Pass SBT_OPTS environment through to sbt_launcherIan Buss2013-08-231-1/+1
| | | | |
* | | | | Merge pull request #864 from rxin/json1shivaram2013-08-268-64/+77
|\ \ \ \ \ | |_|/ / / |/| | | | Revert json library change
| * | | | Added worker state to the cluster master JSON ui.Reynold Xin2013-08-261-1/+2
| | | | |
| * | | | Revert "Merge pull request #841 from rxin/json"Reynold Xin2013-08-268-64/+76
|/ / / / | | | | | | | | | | | | | | | | This reverts commit 1fb1b0992838c8cdd57eec45793e67a0490f1a52, reversing changes made to c69c48947d5102c81a9425cb380d861c3903685c.
* | | | Merge pull request #603 from pwendell/ec2-updatesPatrick Wendell2013-08-244-165/+146
|\ \ \ \ | | | | | | | | | | Several Improvements to EC2 Scripts
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-19 01:48:17 +0000