aboutsummaryrefslogtreecommitdiff
path: root/examples
Commit message (Collapse)AuthorAgeFilesLines
* Minor YARN build cleanupsJey Kottalam2013-09-061-14/+0
|
* Add missing license headers found with RATMatei Zaharia2013-09-021-0/+17
|
* Move some classes to more appropriate packages:Matei Zaharia2013-09-015-13/+11
| | | | | | * RDD, *RDDFunctions -> org.apache.spark.rdd * Utils, ClosureCleaner, SizeEstimator -> org.apache.spark.util * JavaSerializer, KryoSerializer -> org.apache.spark.serializer
* Fix some URLsMatei Zaharia2013-09-011-1/+1
|
* Initial work to rename package to org.apache.sparkMatei Zaharia2013-09-0153-220/+220
|
* Update Maven build to create assemblies expected by new scriptsMatei Zaharia2013-08-291-6/+56
| | | | | | | | | | | | | | | This includes the following changes: - The "assembly" package now builds in Maven by default, and creates an assembly containing both hadoop-client and Spark, unlike the old BigTop distribution assembly that skipped hadoop-client - There is now a bigtop-dist package to build the old BigTop assembly - The repl-bin package is no longer built by default since the scripts don't reply on it; instead it can be enabled with -Prepl-bin - Py4J is now included in the assembly/lib folder as a local Maven repo, so that the Maven package can link to it - run-example now adds the original Spark classpath as well because the Maven examples assembly lists spark-core and such as provided - The various Maven projects add a spark-yarn dependency correctly
* Fix finding of assembly JAR, as well as some pointers to ./runMatei Zaharia2013-08-298-13/+13
|
* Change build and run instructions to use assembliesMatei Zaharia2013-08-294-0/+452
| | | | | | | | | | | | | | | | This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.
* Remove redundant dependencies from POMsJey Kottalam2013-08-181-4/+0
|
* Updates to repl and example POMs to match SBT buildJey Kottalam2013-08-161-0/+10
|
* Maven build now also works with YARNJey Kottalam2013-08-161-57/+0
|
* Don't mark hadoop-client as 'provided'Jey Kottalam2013-08-161-1/+0
|
* Maven build now works with CDH hadoop-2.0.0-mr1Jey Kottalam2013-08-161-44/+0
|
* Initial changes to make Maven build agnostic of hadoop versionJey Kottalam2013-08-161-84/+60
|
* make SparkHadoopUtil a member of SparkEnvJey Kottalam2013-08-151-2/+1
|
* Merge pull request #762 from shivaram/sgd-cleanupEvan Sparks2013-08-111-0/+85
|\ | | | | Refactor SGD options into a new class.
| * Add setters for optimizer, gradient in SGD.Shivaram Venkataraman2013-08-081-1/+1
| | | | | | | | Also remove java-specific constructor for LabeledPoint.
| * Merge branch 'master' of git://github.com/mesos/spark into sgd-cleanupShivaram Venkataraman2013-08-061-0/+116
| |\ | | | | | | | | | | | | Conflicts: mllib/src/main/scala/spark/mllib/util/MLUtils.scala
| * | Refactor GLM algorithms and add Java testsShivaram Venkataraman2013-08-061-0/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds Java examples and unit tests for all GLM algorithms to make sure the MLLib interface works from Java. Changes include - Introduce LabeledPoint and avoid using Doubles in train arguments - Rename train to run in class methods - Make the optimizer a member variable of GLM to make sure the builder pattern works
* | | Fixed path to JavaALS.java and JavaKMeans.java, fixed hadoop2-yarn profileAlexander Pivovarov2013-08-103-0/+6
| | |
* | | Optimize Scala PageRank to use reduceByKeyMatei Zaharia2013-08-101-8/+4
| | |
* | | Merge pull request #789 from MLnick/masterMatei Zaharia2013-08-101-0/+50
|\ \ \ | | | | | | | | Adding Scala version of PageRank example
| * | | Style changes as per Matei's commentsNick Pentreath2013-08-081-9/+8
| | | |
| * | | Adding Scala version of PageRank exampleNick Pentreath2013-08-071-0/+51
| | |/ | |/|
* | | Merge pull request #786 from shivaram/mllib-javaMatei Zaharia2013-08-093-0/+180
|\ \ \ | | | | | | | | Java fixes, tests and examples for ALS, KMeans
| * | | Remove Java-specific constructor for Rating.Shivaram Venkataraman2013-08-081-3/+3
| | | | | | | | | | | | | | | | | | | | The scala constructor works for native type java types. Modify examples to match this.
| * | | Java examples, tests for KMeans and ALSShivaram Venkataraman2013-08-063-0/+180
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init
* / / Optimize JavaPageRank to use reduceByKey instead of groupByKeyMatei Zaharia2013-08-081-9/+8
|/ /
* | Got rid of unnecessary map functionstayhf2013-08-061-6/+2
| |
* | changes as reviewer requestedstayhf2013-08-061-10/+1
| |
* | Updated code with reviewer's suggestionsstayhf2013-08-051-47/+47
| |
* | Simple PageRank algorithm implementation in Java for SPARK-760stayhf2013-08-031-0/+129
|/
* Add Apache license headers and LICENSE and NOTICE filesMatei Zaharia2013-07-1645-1/+766
|
* pom cleanupMark Hamstra2013-07-081-1/+0
|
* Explicit dependencies for scala-library and scalap to prevent 2.9.2 vs. ↵Mark Hamstra2013-07-081-0/+5
| | | | 2.9.3 problems
* Fixing missed hbase dependency in examples hadoop2-yarn profileKonstantin Boudnik2013-07-011-0/+5
|
* Merge pull request #577 from skumargithub/masterMatei Zaharia2013-06-291-0/+50
|\ | | | | Example of cumulative counting using updateStateByKey
| * Removed unused code, clarified intent of the program, batch size to 1 secondunknown2013-05-061-5/+3
| |
| * Modified as per TD's suggestionsunknown2013-04-301-17/+6
| |
| * Examaple of cumulative counting using updateStateByKeyunknown2013-04-221-0/+63
| |
* | Merge remote-tracking branch 'mrpotes/master'Matei Zaharia2013-06-293-15/+12
|\ \
| * | Fix usage and parameter extractionJames Phillpotts2013-06-253-12/+9
| | |
| * | Include a default OAuth implementation, and update examples and ↵James Phillpotts2013-06-253-3/+3
| | | | | | | | | | | | JavaStreamingContext
* | | Merge branch 'master' into streamingTathagata Das2013-06-2441-167/+825
|\| | | | | | | | | | | | | | Conflicts: .gitignore
| * | Merge remote-tracking branch 'milliondreams/casdemo'Matei Zaharia2013-06-182-0/+231
| |\ \ | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala
| | * | Fixing the style as per feedbackRohit Rai2013-06-131-35/+37
| | | |
| | * | Example to write the output to cassandraRohit Rai2013-06-031-5/+43
| | | |
| | * | A better way to read column value if you are sure the column exists in every ↵Rohit Rai2013-06-031-2/+4
| | | | | | | | | | | | | | | | row.
| | * | Adding deps to examples/pom.xmlRohit Rai2013-06-021-0/+35
| | | | | | | | | | | | | | | | Fixing exclusion in examples deps in SparkBuild.scala
| | * | Removing infix callRohit Rai2013-06-021-3/+3
| | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-07 17:16:20 +0000