Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Add Maven build file for streaming, and fix some issues in SBT file | Matei Zaharia | 2013-01-20 | 1 | -0/+17 |
| | | | | | | As part of this, changed our Scala 2.9.2 Kafka library to be available as a local Maven repository, following the example in (http://blog.dub.podval.org/2010/01/maven-in-project-repository.html) | ||||
* | Merge branch 'master' into streaming | Matei Zaharia | 2013-01-20 | 1 | -39/+20 |
|\ | | | | | | | | | Conflicts: core/src/main/scala/spark/api/python/PythonRDD.scala | ||||
| * | Minor formatting fixes | Matei Zaharia | 2013-01-20 | 1 | -2/+2 |
| | | |||||
| * | Use only one update function and pass in transpose of ratings matrix where ↵ | Nick Pentreath | 2013-01-17 | 1 | -29/+3 |
| | | | | | | | | appropriate | ||||
| * | Fixed index error missing first argument | Nick Pentreath | 2013-01-17 | 1 | -1/+1 |
| | | |||||
| * | Adding default command line args to SparkALS | Nick Pentreath | 2013-01-17 | 1 | -10/+17 |
| | | |||||
* | | Merge branch 'mesos-streaming' into streaming | Tathagata Das | 2013-01-20 | 4 | -1/+175 |
|\ \ | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/api/java/JavaRDDLike.scala core/src/main/scala/spark/api/java/JavaSparkContext.scala core/src/test/scala/spark/JavaAPISuite.java | ||||
| * | | NetworkWordCount example | Patrick Wendell | 2013-01-17 | 2 | -1/+63 |
| | | | |||||
| * | | Adding queueStream and some slight refactoring | Patrick Wendell | 2013-01-17 | 1 | -0/+62 |
| | | | |||||
| * | | Small doc fix | Patrick Wendell | 2013-01-17 | 1 | -1/+1 |
| | | | |||||
| * | | Two changes: | Patrick Wendell | 2013-01-14 | 1 | -2/+2 |
| | | | | | | | | | | | | | | | - Updating countByX() types based on bug fix - Porting new documentation to Java | ||||
| * | | Flume example and bug fix | Patrick Wendell | 2013-01-14 | 1 | -0/+50 |
| | | | |||||
* | | | Merge branch 'master' into streaming | Tathagata Das | 2013-01-15 | 2 | -1/+12 |
|\ \ \ | |/ / |/| / | |/ | | | | | | | | | | | Conflicts: core/src/main/scala/spark/rdd/CoGroupedRDD.scala core/src/main/scala/spark/rdd/FilteredRDD.scala docs/_layouts/global.html docs/index.md run | ||||
| * | Update examples/src/main/scala/spark/examples/LocalLR.scala | Eric Zhang | 2013-01-13 | 1 | -1/+1 |
| | | | | | | fix spelling mistake | ||||
| * | Rename environment variable for hadoop profiles to hadoopVersion | Shivaram Venkataraman | 2013-01-12 | 1 | -2/+2 |
| | | |||||
| * | Activate hadoop2 profile in pom.xml with -Dhadoop=2 | Shivaram Venkataraman | 2013-01-10 | 1 | -0/+6 |
| | | |||||
| * | Activate hadoop1 if property hadoop is missing. hadoop2 can be activated now | Shivaram Venkataraman | 2013-01-08 | 1 | -1/+3 |
| | | | | | | | | by using -Dhadoop -Phadoop2. | ||||
| * | Activate hadoop1 profile by default for maven builds | Shivaram Venkataraman | 2013-01-07 | 1 | -0/+3 |
| | | |||||
* | | Removed stream id from the constructor of NetworkReceiver to make it easier ↵ | Tathagata Das | 2013-01-13 | 1 | -7/+8 |
| | | | | | | | | for PluggableNetworkInputDStream. | ||||
* | | Making the Twitter example distributed. | Patrick Wendell | 2013-01-07 | 2 | -37/+62 |
| | | | | | | | | | | | | This adds a distributed (receiver-based) implementation of the Twitter dstream. It also changes the example to perform a distributed sort rather than collecting the dataset at one node. | ||||
* | | Moved Twitter example to the where the other examples are. | Tathagata Das | 2013-01-07 | 2 | -0/+105 |
| | | |||||
* | | Renamed examples and added documentation. | Tathagata Das | 2013-01-07 | 10 | -274/+97 |
| | | |||||
* | | Moved Spark Streaming examples to examples sub-project. | Tathagata Das | 2013-01-06 | 12 | -0/+615 |
|/ | |||||
* | Mark hadoop dependencies provided in all library artifacts | Thomas Dudziak | 2012-12-10 | 1 | -0/+3 |
| | |||||
* | Use the same output directories that SBT had in subprojects | Matei Zaharia | 2012-12-10 | 1 | -1/+3 |
| | | | | This will make it easier to make the "run" script work with a Maven build | ||||
* | Updated versions in the pom.xml files to match current master | Thomas Dudziak | 2012-11-27 | 1 | -1/+1 |
| | |||||
* | Addressed code review comments | Thomas Dudziak | 2012-11-27 | 1 | -0/+1 |
| | |||||
* | Added maven and debian build files | Thomas Dudziak | 2012-11-20 | 1 | -0/+100 |
| | |||||
* | Fix K-means example a little | root | 2012-11-10 | 1 | -16/+11 |
| | |||||
* | Some doc and usability improvements: | Matei Zaharia | 2012-10-12 | 2 | -2/+2 |
| | | | | | | | - Added a StorageLevels class for easy access to StorageLevel constants in Java - Added doc comments on Function classes in Java - Updated Accumulator and HadoopWriter docs slightly | ||||
* | Conflict fixed | Mosharaf Chowdhury | 2012-10-02 | 10 | -10/+10 |
|\ | |||||
| * | More updates to documentation | Matei Zaharia | 2012-09-25 | 10 | -10/+10 |
| | | |||||
* | | Bug fix. Fixed log messages. Updated BroadcastTest example to have iterations. | Mosharaf Chowdhury | 2012-08-30 | 1 | -3/+7 |
|/ | |||||
* | Cache points in SparkLR example. | Josh Rosen | 2012-08-26 | 1 | -2/+2 |
| | |||||
* | Renamed apply() to call() in Java API and allowed it to throw Exceptions | Matei Zaharia | 2012-08-12 | 4 | -21/+25 |
| | |||||
* | move Vector class into core and spark.util package | Imran Rashid | 2012-07-28 | 7 | -88/+6 |
| | |||||
* | Remove StringOps.split() from Java WordCount. | Josh Rosen | 2012-07-25 | 1 | -5/+2 |
| | |||||
* | Minor cleanup and optimizations in Java API. | Josh Rosen | 2012-07-24 | 2 | -10/+13 |
| | | | | | | - Add override keywords. - Cache RDDs and counts in TC example. - Clean up JavaRDDLike's abstract methods. | ||||
* | Improve Java API examples | Josh Rosen | 2012-07-22 | 5 | -198/+143 |
| | | | | | | - Replace JavaLR example with JavaHdfsLR example. - Use anonymous classes in JavaWordCount; add options. - Remove @Override annotations. | ||||
* | Add Java API | Josh Rosen | 2012-07-18 | 5 | -0/+355 |
| | | | | | | Add distinct() method to RDD. Fix bug in DoubleRDDFunctions. | ||||
* | Add System.exit(0) at the end of all the example programs. | Matei Zaharia | 2012-06-05 | 12 | -0/+19 |
| | |||||
* | Format the code as coding style agreed by Matei/TD/Haoyuan | haoyuan | 2012-02-09 | 1 | -1/+1 |
| | |||||
* | Some fixes to the examples (mostly to use functional API) | Matei Zaharia | 2012-01-31 | 4 | -76/+72 |
| | |||||
* | Merge pull request #103 from edisontung/master | Matei Zaharia | 2012-01-13 | 2 | -56/+142 |
|\ | | | | | Made improvements to takeSample. Also changed SparkLocalKMeans to SparkKMeans | ||||
| * | Revert de01b6deaaee1b43321e0aac330f4a98c0ea61c6^..HEAD | Edison Tung | 2011-12-01 | 1 | -73/+0 |
| | | |||||
| * | Renamed SparkLocalKMeans to SparkKMeans | Edison Tung | 2011-12-01 | 1 | -56/+62 |
| | | |||||
| * | Added KMeans examples | Edison Tung | 2011-11-21 | 2 | -0/+153 |
| | | | | | | | | | | LocalKMeans runs locally with a randomly generated dataset. SparkLocalKMeans takes an input file and runs KMeans on it. | ||||
* | | Merge commit 'ad4ebff42c1b738746b2b9ecfbb041b6d06e3e16' | Matei Zaharia | 2011-12-14 | 1 | -0/+18 |
|\ \ | |||||
| * | | Report errors in tasks to the driver via a Mesos status update | Ankur Dave | 2011-11-14 | 1 | -0/+18 |
| |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When a task throws an exception, the Spark executor previously just logged it to a local file on the slave and exited. This commit causes Spark to also report the exception back to the driver using a Mesos status update, so the user doesn't have to look through a log file on the slave. Here's what the reporting currently looks like: # ./run spark.examples.ExceptionHandlingTest master@203.0.113.1:5050 [...] 11/10/26 21:04:13 INFO spark.SimpleJob: Lost TID 1 (task 0:1) 11/10/26 21:04:13 INFO spark.SimpleJob: Loss was due to java.lang.Exception: Testing exception handling [...] 11/10/26 21:04:16 INFO spark.SparkContext: Job finished in 5.988547328 s | ||||
* / | Fixed LocalFileLR to deal with a change in Scala IO sources | Matei Zaharia | 2011-12-01 | 1 | -1/+1 |
|/ | | | | (you can no longer iterate over a Source multiple times). |