aboutsummaryrefslogtreecommitdiff
path: root/examples
Commit message (Collapse)AuthorAgeFilesLines
* Merge branch 'master' of git://github.com/mesos/spark into scala-2.10Prashant Sharma2013-09-151-14/+0
|\ | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/SparkContext.scala project/SparkBuild.scala
| * Minor YARN build cleanupsJey Kottalam2013-09-061-14/+0
| |
* | Merged with masterPrashant Sharma2013-09-0659-494/+2086
|\|
| * Add missing license headers found with RATMatei Zaharia2013-09-021-0/+17
| |
| * Move some classes to more appropriate packages:Matei Zaharia2013-09-015-13/+11
| | | | | | | | | | | | * RDD, *RDDFunctions -> org.apache.spark.rdd * Utils, ClosureCleaner, SizeEstimator -> org.apache.spark.util * JavaSerializer, KryoSerializer -> org.apache.spark.serializer
| * Fix some URLsMatei Zaharia2013-09-011-1/+1
| |
| * Initial work to rename package to org.apache.sparkMatei Zaharia2013-09-0153-220/+220
| |
| * Update Maven build to create assemblies expected by new scriptsMatei Zaharia2013-08-291-6/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This includes the following changes: - The "assembly" package now builds in Maven by default, and creates an assembly containing both hadoop-client and Spark, unlike the old BigTop distribution assembly that skipped hadoop-client - There is now a bigtop-dist package to build the old BigTop assembly - The repl-bin package is no longer built by default since the scripts don't reply on it; instead it can be enabled with -Prepl-bin - Py4J is now included in the assembly/lib folder as a local Maven repo, so that the Maven package can link to it - run-example now adds the original Spark classpath as well because the Maven examples assembly lists spark-core and such as provided - The various Maven projects add a spark-yarn dependency correctly
| * Fix finding of assembly JAR, as well as some pointers to ./runMatei Zaharia2013-08-298-13/+13
| |
| * Change build and run instructions to use assembliesMatei Zaharia2013-08-294-0/+452
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit makes Spark invocation saner by using an assembly JAR to find all of Spark's dependencies instead of adding all the JARs in lib_managed. It also packages the examples into an assembly and uses that as SPARK_EXAMPLES_JAR. Finally, it replaces the old "run" script with two better-named scripts: "run-examples" for examples, and "spark-class" for Spark internal classes (e.g. REPL, master, etc). This is also designed to minimize the confusion people have in trying to use "run" to run their own classes; it's not meant to do that, but now at least if they look at it, they can modify run-examples to do a decent job for them. As part of this, Bagel's examples are also now properly moved to the examples package instead of bagel.
| * Remove redundant dependencies from POMsJey Kottalam2013-08-181-4/+0
| |
| * Updates to repl and example POMs to match SBT buildJey Kottalam2013-08-161-0/+10
| |
| * Maven build now also works with YARNJey Kottalam2013-08-161-57/+0
| |
| * Don't mark hadoop-client as 'provided'Jey Kottalam2013-08-161-1/+0
| |
| * Maven build now works with CDH hadoop-2.0.0-mr1Jey Kottalam2013-08-161-44/+0
| |
| * Initial changes to make Maven build agnostic of hadoop versionJey Kottalam2013-08-161-84/+60
| |
| * make SparkHadoopUtil a member of SparkEnvJey Kottalam2013-08-151-2/+1
| |
| * Merge pull request #762 from shivaram/sgd-cleanupEvan Sparks2013-08-111-0/+85
| |\ | | | | | | Refactor SGD options into a new class.
| | * Add setters for optimizer, gradient in SGD.Shivaram Venkataraman2013-08-081-1/+1
| | | | | | | | | | | | Also remove java-specific constructor for LabeledPoint.
| | * Merge branch 'master' of git://github.com/mesos/spark into sgd-cleanupShivaram Venkataraman2013-08-061-0/+116
| | |\ | | | | | | | | | | | | | | | | Conflicts: mllib/src/main/scala/spark/mllib/util/MLUtils.scala
| | * | Refactor GLM algorithms and add Java testsShivaram Venkataraman2013-08-061-0/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds Java examples and unit tests for all GLM algorithms to make sure the MLLib interface works from Java. Changes include - Introduce LabeledPoint and avoid using Doubles in train arguments - Rename train to run in class methods - Make the optimizer a member variable of GLM to make sure the builder pattern works
| * | | Fixed path to JavaALS.java and JavaKMeans.java, fixed hadoop2-yarn profileAlexander Pivovarov2013-08-103-0/+6
| | | |
| * | | Optimize Scala PageRank to use reduceByKeyMatei Zaharia2013-08-101-8/+4
| | | |
| * | | Merge pull request #789 from MLnick/masterMatei Zaharia2013-08-101-0/+50
| |\ \ \ | | | | | | | | | | Adding Scala version of PageRank example
| | * | | Style changes as per Matei's commentsNick Pentreath2013-08-081-9/+8
| | | | |
| | * | | Adding Scala version of PageRank exampleNick Pentreath2013-08-071-0/+51
| | | |/ | | |/|
| * | | Merge pull request #786 from shivaram/mllib-javaMatei Zaharia2013-08-093-0/+180
| |\ \ \ | | | | | | | | | | Java fixes, tests and examples for ALS, KMeans
| | * | | Remove Java-specific constructor for Rating.Shivaram Venkataraman2013-08-081-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | The scala constructor works for native type java types. Modify examples to match this.
| | * | | Java examples, tests for KMeans and ALSShivaram Venkataraman2013-08-063-0/+180
| | |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init
| * / / Optimize JavaPageRank to use reduceByKey instead of groupByKeyMatei Zaharia2013-08-081-9/+8
| |/ /
| * | Got rid of unnecessary map functionstayhf2013-08-061-6/+2
| | |
| * | changes as reviewer requestedstayhf2013-08-061-10/+1
| | |
| * | Updated code with reviewer's suggestionsstayhf2013-08-051-47/+47
| | |
| * | Simple PageRank algorithm implementation in Java for SPARK-760stayhf2013-08-031-0/+129
| |/
| * Add Apache license headers and LICENSE and NOTICE filesMatei Zaharia2013-07-1645-1/+766
| |
* | Merge branch 'master' into master-mergePrashant Sharma2013-07-121-0/+4
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: README.md core/pom.xml core/src/main/scala/spark/deploy/JsonProtocol.scala core/src/main/scala/spark/deploy/LocalSparkCluster.scala core/src/main/scala/spark/deploy/master/Master.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala pom.xml project/SparkBuild.scala streaming/src/main/scala/spark/streaming/receivers/ActorReceiver.scala
| * pom cleanupMark Hamstra2013-07-081-1/+0
| |
| * Explicit dependencies for scala-library and scalap to prevent 2.9.2 vs. ↵Mark Hamstra2013-07-081-0/+5
| | | | | | | | 2.9.3 problems
* | Merge branch 'master' into master-mergePrashant Sharma2013-07-039-18/+395
|\| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/pom.xml core/src/main/scala/spark/MapOutputTracker.scala core/src/main/scala/spark/RDD.scala core/src/main/scala/spark/RDDCheckpointData.scala core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/Utils.scala core/src/main/scala/spark/api/python/PythonRDD.scala core/src/main/scala/spark/deploy/client/Client.scala core/src/main/scala/spark/deploy/master/MasterWebUI.scala core/src/main/scala/spark/deploy/worker/Worker.scala core/src/main/scala/spark/deploy/worker/WorkerWebUI.scala core/src/main/scala/spark/rdd/BlockRDD.scala core/src/main/scala/spark/rdd/ZippedRDD.scala core/src/main/scala/spark/scheduler/cluster/StandaloneSchedulerBackend.scala core/src/main/scala/spark/storage/BlockManager.scala core/src/main/scala/spark/storage/BlockManagerMaster.scala core/src/main/scala/spark/storage/BlockManagerMasterActor.scala core/src/main/scala/spark/storage/BlockManagerUI.scala core/src/main/scala/spark/util/AkkaUtils.scala core/src/test/scala/spark/SizeEstimatorSuite.scala pom.xml project/SparkBuild.scala repl/src/main/scala/spark/repl/SparkILoop.scala repl/src/test/scala/spark/repl/ReplSuite.scala streaming/src/main/scala/spark/streaming/StreamingContext.scala streaming/src/main/scala/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/spark/streaming/dstream/KafkaInputDStream.scala streaming/src/main/scala/spark/streaming/util/MasterFailureTest.scala
| * Fixing missed hbase dependency in examples hadoop2-yarn profileKonstantin Boudnik2013-07-011-0/+5
| |
| * Merge pull request #577 from skumargithub/masterMatei Zaharia2013-06-291-0/+50
| |\ | | | | | | Example of cumulative counting using updateStateByKey
| | * Removed unused code, clarified intent of the program, batch size to 1 secondunknown2013-05-061-5/+3
| | |
| | * Modified as per TD's suggestionsunknown2013-04-301-17/+6
| | |
| | * Examaple of cumulative counting using updateStateByKeyunknown2013-04-221-0/+63
| | |
| * | Merge remote-tracking branch 'mrpotes/master'Matei Zaharia2013-06-293-15/+12
| |\ \
| | * | Fix usage and parameter extractionJames Phillpotts2013-06-253-12/+9
| | | |
| | * | Include a default OAuth implementation, and update examples and ↵James Phillpotts2013-06-253-3/+3
| | | | | | | | | | | | | | | | JavaStreamingContext
| * | | Merge branch 'master' into streamingTathagata Das2013-06-2441-167/+825
| |\| | | | | | | | | | | | | | | | | | Conflicts: .gitignore
| | * | Merge remote-tracking branch 'milliondreams/casdemo'Matei Zaharia2013-06-182-0/+231
| | |\ \ | | | | | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala
| | | * | Fixing the style as per feedbackRohit Rai2013-06-131-35/+37
| | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-15 12:16:39 +0000