aboutsummaryrefslogtreecommitdiff
path: root/examples/src/main/java
Commit message (Collapse)AuthorAgeFilesLines
* fixed job name and usage information for the JavaSparkPi exampleKevin Mader2014-01-221-2/+2
|
* Added StreamingContext.awaitTermination to streaming examples.Tathagata Das2014-01-204-0/+4
|
* Updated java API docs for streaming, along with very minor changes in the ↵Tathagata Das2014-01-161-2/+1
| | | | code examples.
* Merge remote-tracking branch 'apache/master' into driver-testTathagata Das2014-01-104-1/+9
|\ | | | | | | | | Conflicts: streaming/src/main/scala/org/apache/spark/streaming/DStreamGraph.scala
| * Minor clean-upPatrick Wendell2014-01-091-1/+1
| |
| * Set default logging to WARN for Spark streaming examples.Patrick Wendell2014-01-094-0/+8
| | | | | | | | | | | | This programatically sets the log level to WARN by default for streaming tests. If the user has already specified a log4j.properties file, the user's file will take precedence over this default.
* | Merge branch 'standalone-driver' into driver-testTathagata Das2014-01-0914-107/+162
|\| | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/org/apache/spark/SparkContext.scala core/src/main/scala/org/apache/spark/deploy/worker/DriverRunner.scala examples/src/main/java/org/apache/spark/streaming/examples/JavaNetworkWordCount.java streaming/src/main/scala/org/apache/spark/streaming/Checkpoint.scala streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/main/scala/org/apache/spark/streaming/scheduler/JobGenerator.scala
| * Merge pull request #313 from tdas/project-refactorPatrick Wendell2014-01-072-8/+9
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Refactored the streaming project to separate external libraries like Twitter, Kafka, Flume, etc. At a high level, these are the following changes. 1. All the external code was put in `SPARK_HOME/external/` as separate SBT projects and Maven modules. Their artifact names are `spark-streaming-twitter`, `spark-streaming-kafka`, etc. Both SparkBuild.scala and pom.xml files have been updated. References to external libraries and repositories have been removed from the settings of root and streaming projects/modules. 2. To avail the external functionality (say, creating a Twitter stream), the developer has to `import org.apache.spark.streaming.twitter._` . For Scala API, the developer has to call `TwitterUtils.createStream(streamingContext, ...)`. For the Java API, the developer has to call `TwitterUtils.createStream(javaStreamingContext, ...)`. 3. Each external project has its own scala and java unit tests. Note the unit tests of each external library use classes of the streaming unit tests (`TestSuiteBase`, `LocalJavaStreamingContext`, etc.). To enable this code sharing among test classes, `dependsOn(streaming % "compile->compile,test->test")` was used in the SparkBuild.scala . In the streaming/pom.xml, an additional `maven-jar-plugin` was necessary to capture this dependency (see comment inside the pom.xml for more information). 4. Jars of the external projects have been added to examples project but not to the assembly project. 5. In some files, imports have been rearrange to conform to the Spark coding guidelines.
| | * Removed XYZFunctions and added XYZUtils as a common Scala and Java interface ↵Tathagata Das2014-01-072-8/+6
| | | | | | | | | | | | for creating XYZ streams.
| | * Merge remote-tracking branch 'apache/master' into project-refactorTathagata Das2014-01-0614-15/+19
| | |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: examples/src/main/java/org/apache/spark/streaming/examples/JavaFlumeEventCount.java streaming/src/main/scala/org/apache/spark/streaming/StreamingContext.scala streaming/src/main/scala/org/apache/spark/streaming/api/java/JavaStreamingContext.scala streaming/src/test/java/org/apache/spark/streaming/JavaAPISuite.java streaming/src/test/scala/org/apache/spark/streaming/InputStreamsSuite.scala streaming/src/test/scala/org/apache/spark/streaming/TestSuiteBase.scala
| | * | Changed JavaStreamingContextWith*** to ***Function in streaming.api.java.*** ↵Tathagata Das2014-01-062-6/+6
| | | | | | | | | | | | | | | | package. Also fixed packages of Flume and MQTT tests.
| | * | Refactored kafka, flume, zeromq, mqtt as separate external projects, with ↵Tathagata Das2013-12-302-6/+9
| | | | | | | | | | | | | | | | their own self-contained scala API, java API, scala unit tests and java unit tests. Updated examples to use the external projects.
| * | | Issue #318 : minor style updates per review from Reynold XinSean Owen2014-01-0710-33/+2
| | | |
| * | | Merge remote-tracking branch 'upstream/master'Sean Owen2014-01-0614-15/+19
| |\ \ \ | | | |/ | | |/|
| | * | Removing SPARK_EXAMPLES_JAR in the codePatrick Wendell2014-01-0514-14/+18
| | | |
| | * | run-example -> bin/run-examplePrashant Sharma2014-01-021-1/+1
| | |/
| * / Suggested small changes to Java code for slightly more standard style, ↵Sean Owen2014-01-0214-83/+164
| |/ | | | | | | encapsulation and in some cases performance
| * Fixed job name in the java streaming example.azuryyu2013-12-241-1/+1
| |
* | Changed the way StreamingContext finds and reads checkpoint files, and added ↵Tathagata Das2014-01-091-4/+3
|/ | | | JavaStreamingContext.getOrCreate.
* Merge branch 'master' into scala-2.10Raymond Liu2013-11-131-0/+98
|\
| * Upgrade Kafka 0.7.2 to Kafka 0.8.0-beta1 for Spark Streamingjerryshao2013-10-121-0/+98
| |
* | fixed some warningsMartin Weindel2013-10-054-5/+3
|/
* Initial work to rename package to org.apache.sparkMatei Zaharia2013-09-0113-84/+84
|
* Merge pull request #762 from shivaram/sgd-cleanupEvan Sparks2013-08-111-0/+85
|\ | | | | Refactor SGD options into a new class.
| * Add setters for optimizer, gradient in SGD.Shivaram Venkataraman2013-08-081-1/+1
| | | | | | | | Also remove java-specific constructor for LabeledPoint.
| * Merge branch 'master' of git://github.com/mesos/spark into sgd-cleanupShivaram Venkataraman2013-08-061-0/+116
| |\ | | | | | | | | | | | | Conflicts: mllib/src/main/scala/spark/mllib/util/MLUtils.scala
| * | Refactor GLM algorithms and add Java testsShivaram Venkataraman2013-08-061-0/+85
| | | | | | | | | | | | | | | | | | | | | | | | | | | This change adds Java examples and unit tests for all GLM algorithms to make sure the MLLib interface works from Java. Changes include - Introduce LabeledPoint and avoid using Doubles in train arguments - Rename train to run in class methods - Make the optimizer a member variable of GLM to make sure the builder pattern works
* | | Fixed path to JavaALS.java and JavaKMeans.java, fixed hadoop2-yarn profileAlexander Pivovarov2013-08-102-0/+0
| | |
* | | Merge pull request #786 from shivaram/mllib-javaMatei Zaharia2013-08-092-0/+168
|\ \ \ | | | | | | | | Java fixes, tests and examples for ALS, KMeans
| * | | Remove Java-specific constructor for Rating.Shivaram Venkataraman2013-08-081-3/+3
| | | | | | | | | | | | | | | | | | | | The scala constructor works for native type java types. Modify examples to match this.
| * | | Java examples, tests for KMeans and ALSShivaram Venkataraman2013-08-062-0/+168
| | |/ | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init
* / | Optimize JavaPageRank to use reduceByKey instead of groupByKeyMatei Zaharia2013-08-081-9/+8
|/ /
* | Got rid of unnecessary map functionstayhf2013-08-061-6/+2
| |
* | changes as reviewer requestedstayhf2013-08-061-10/+1
| |
* | Updated code with reviewer's suggestionsstayhf2013-08-051-47/+47
| |
* | Simple PageRank algorithm implementation in Java for SPARK-760stayhf2013-08-031-0/+129
|/
* Add Apache license headers and LICENSE and NOTICE filesMatei Zaharia2013-07-169-0/+153
|
* Java indentation 4 --> 2 spacesNick Pentreath2013-03-203-200/+200
|
* A few cosmetic changes for JavaKMeansNick Pentreath2013-03-191-1/+4
|
* Adding Java K-Means exampleNick Pentreath2013-03-191-0/+111
|
* Changes to more closely match line length limit styleNick Pentreath2013-03-171-3/+5
|
* Adding Java versions of Pi and LogQueryNick Pentreath2013-03-152-0/+160
|
* Pass a code JAR to SparkContext in our examples. Fixes SPARK-594.Matei Zaharia2013-02-256-9/+17
|
* Fixed bugs in examples.Tathagata Das2013-02-241-1/+1
|
* Changed networkStream to socketStream and pluggableNetworkStream to become ↵Tathagata Das2013-02-181-1/+1
| | | | networkStream as a way to create streams from arbitrary network receiver.
* Moved Java streaming examples to examples/src/main/java/spark/streaming/... ↵Tathagata Das2013-02-143-0/+174
| | | | and fixed logging in NetworkInputTracker to highlight errors when receiver deregisters/shuts down.
* Some doc and usability improvements:Matei Zaharia2012-10-122-2/+2
| | | | | | | - Added a StorageLevels class for easy access to StorageLevel constants in Java - Added doc comments on Function classes in Java - Updated Accumulator and HadoopWriter docs slightly
* Renamed apply() to call() in Java API and allowed it to throw ExceptionsMatei Zaharia2012-08-123-21/+22
|
* Remove StringOps.split() from Java WordCount.Josh Rosen2012-07-251-5/+2
|
* Minor cleanup and optimizations in Java API.Josh Rosen2012-07-241-6/+7
| | | | | | - Add override keywords. - Cache RDDs and counts in TC example. - Clean up JavaRDDLike's abstract methods.
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-08 17:01:50 +0000