aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Optimize Scala PageRank to use reduceByKeyMatei Zaharia2013-08-101-8/+4
|
* Merge pull request #789 from MLnick/masterMatei Zaharia2013-08-101-0/+50
|\ | | | | Adding Scala version of PageRank example
| * Style changes as per Matei's commentsNick Pentreath2013-08-081-9/+8
| |
| * Adding Scala version of PageRank exampleNick Pentreath2013-08-071-0/+51
| |
* | Merge pull request #795 from mridulm/masterMatei Zaharia2013-08-102-7/+42
|\ \ | | | | | | Fix bug reported in PR 791 : a race condition in ConnectionManager and Connection
| * | Change line sizeMridul Muralidharan2013-08-081-5/+9
| | |
| * | Attempt to fix bug reported in PR 791 : a race condition in ↵Mridul Muralidharan2013-08-082-7/+38
| | | | | | | | | | | | ConnectionManager and Connection
* | | Merge remote-tracking branch 'origin/pr/792'Matei Zaharia2013-08-1010-259/+279
|\ \ \ | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/ui/jobs/IndexPage.scala core/src/main/scala/spark/ui/jobs/StagePage.scala
| * | | Shortened names, as per Matei's suggestionKay Ousterhout2013-08-102-14/+13
| | | |
| * | | Only print event queue full error message onceKay Ousterhout2013-08-091-1/+3
| | | |
| * | | Style fix: removing unnecessary return typeKay Ousterhout2013-08-091-6/+6
| | | |
| * | | Style fixes based on code reviewKay Ousterhout2013-08-093-132/+110
| | | |
| * | | Refactored SparkListener to process all events asynchronously.Kay Ousterhout2013-08-099-165/+196
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit fixes issues where SparkListeners that take a while to process events slow the DAGScheduler. This commit also fixes a bug in the UI where if a user goes to a web page of a stage that does not exist, they can create a memory leak (granted, this is not an issue at small scale -- probably only an issue if someone actively tried to DOS the UI).
| * | | Fixed 3 bugs that caused UI to crash (including SPARK-810).Kay Ousterhout2013-08-072-169/+191
| |/ / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | One bug caused the UI to crash if you try to look at a job's status before any of the tasks have finished. The second bug was a concurrency issue where two different threads (the scheduling thread and a UI thread) could be reading/updating the data structures in JobProgressListener concurrently. The third bug mis-used an Option, also causing the UI to crash under certain conditions.
* | | Merge pull request #785 from anfeng/masterPatrick Wendell2013-08-101-0/+25
|\ \ \ | | | | | | | | expose HDFS file system stats via Executor metrics
| * | | reformat registerFileSystemStat()anfeng2013-08-061-1/+3
| | | |
| * | | expose HDFS file system stats via Executor metricsanfeng2013-08-061-0/+23
| | | |
* | | | Merge pull request #800 from dlyubimov/HBASE_VERSIONMatei Zaharia2013-08-091-1/+4
|\ \ \ \ | | | | | | | | | | Pull HBASE_VERSION in the head of sbt build
| * | | | fewer wordsDmitriy Lyubimov2013-08-091-1/+1
| | | | |
| * | | | Pull HBASE_VERSION in the head of sbt buildDmitriy Lyubimov2013-08-091-1/+4
| | | | |
* | | | | Merge pull request #786 from shivaram/mllib-javaMatei Zaharia2013-08-0911-31/+471
|\ \ \ \ \ | | | | | | | | | | | | Java fixes, tests and examples for ALS, KMeans
| * | | | | Remove Java-specific constructor for Rating.Shivaram Venkataraman2013-08-084-15/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The scala constructor works for native type java types. Modify examples to match this.
| * | | | | Add a test case for random initialization.Shivaram Venkataraman2013-08-062-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | Also workaround a bug where double[][] class cast fails
| * | | | | Java examples, tests for KMeans and ALSShivaram Venkataraman2013-08-0611-28/+466
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init
* | | | | | Merge pull request #799 from woggle/sync-fixMatei Zaharia2013-08-091-9/+7
|\ \ \ \ \ \ | | | | | | | | | | | | | | Remove extra synchronization in ResultTask
| * | | | | | Remove extra synchronization in ResultTaskCharles Reiss2013-08-091-9/+7
| | |/ / / / | |/| | | |
* | | | | | Merge pull request #775 from pwendell/print-launch-commandPatrick Wendell2013-08-093-5/+26
|\ \ \ \ \ \ | |/ / / / / |/| | | | | Log the launch command for Spark daemons
| * | | | | Using quotes when printing out commandPatrick Wendell2013-08-091-1/+2
| | | | | |
| * | | | | Make output formatting consistent between bash/scalaPatrick Wendell2013-08-032-3/+2
| | | | | |
| * | | | | Log the launch command for Spark executorsPatrick Wendell2013-08-031-3/+13
| | | | | |
| * | | | | Log the launch command for Spark daemonsPatrick Wendell2013-08-022-2/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For debugging and analysis purposes, it's nice to have the exact command used to launch Spark contained within the logs. This adds the necessary hooks to make that possible.
* | | | | | Merge pull request #788 from shane-huang/sparkjavaoptsMatei Zaharia2013-08-092-1/+7
|\ \ \ \ \ \ | | | | | | | | | | | | | | For standalone mode, add worker local env setting of SPARK_JAVA_OPTS as ...
| * | | | | | add comment in spark-env.sh.template for SPARK_JAVA_OPTSshane-huang2013-08-091-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Signed-off-by: shane-huang <shengsheng.huang@intel.com>
| * | | | | | For standalone mode, add worker local env setting of SPARK_JAVA_OPTS as ↵shane-huang2013-08-071-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | default and let application env override default options if applicable Signed-off-by: shane-huang <shengsheng.huang@intel.com>
* | | | | | | Optimize JavaPageRank to use reduceByKey instead of groupByKeyMatei Zaharia2013-08-081-9/+8
| | | | | | |
* | | | | | | Fixed a typo in mllib inline documentation.Reynold Xin2013-08-081-1/+1
| | | | | | |
* | | | | | | Merge pull request #797 from mateiz/chill-0.3.1Matei Zaharia2013-08-084-7/+19
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | Update to Chill 0.3.1
| * | | | | | | Add test for Kryo with WrappedArray (which was failing in Chill 0.3.0)Matei Zaharia2013-08-081-1/+13
| | | | | | | |
| * | | | | | | Update to Chill 0.3.1Matei Zaharia2013-08-083-6/+6
|/ / / / / / /
* | | | | | | Merge pull request #796 from pwendell/bootstrap-designPatrick Wendell2013-08-0811-37/+45
|\ \ \ \ \ \ \ | |_|_|_|_|/ / |/| | | | | | Bootstrap re-design
| * | | | | | Reverting boostrap changePatrick Wendell2013-08-074-1863/+852
| | | | | | |
| * | | | | | Change I forgot to merge inPatrick Wendell2013-08-071-1/+1
| | | | | | |
| * | | | | | Bumping font size to 14px and fixing sytle issue in progress barsPatrick Wendell2013-08-072-2/+7
| | | | | | |
| * | | | | | Merge branch 'master' into bootstrap-designPatrick Wendell2013-08-0748-413/+1151
| |\ \ \ \ \ \ | |/ / / / / / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/ui/UIUtils.scala core/src/main/scala/spark/ui/jobs/IndexPage.scala core/src/main/scala/spark/ui/storage/RDDPage.scala
* | | | | | | Merge pull request #790 from kayousterhout/fix_throughputMatei Zaharia2013-08-073-17/+6
|\ \ \ \ \ \ \ | |_|_|_|_|_|/ |/| | | | | | Fixed issue in UI that decreased scheduler throughput by 5x or more
| * | | | | | Fixed issue in UI that limited scheduler throughput.Kay Ousterhout2013-08-073-17/+6
|/ / / / / / | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Removal of items from ArrayBuffers in the UI code was slow and significantly impacted scheduler throughput. This commit improves scheduler throughput by 5x.
* | | | | | Merge pull request #747 from mateiz/improved-lrMatei Zaharia2013-08-061-27/+26
|\ \ \ \ \ \ | | | | | | | | | | | | | | Update the Python logistic regression example
| * | | | | | Fix string parsing and style in LRMatei Zaharia2013-07-311-1/+1
| | | | | | |
| * | | | | | Update the Python logistic regression example to read from a file andMatei Zaharia2013-07-291-27/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | batch input records for more efficient NumPy computations
* | | | | | | Merge pull request #724 from dlyubimov/SPARK-826Matei Zaharia2013-08-067-22/+162
|\ \ \ \ \ \ \ | |_|_|/ / / / |/| | | | | | SPARK-826: fold(), reduce(), collect() always attempt to use java serialization
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-19 19:15:09 +0000