Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | Java examples, tests for KMeans and ALS | Shivaram Venkataraman | 2013-08-06 | 1 | -0/+5 |
| | | | | | | | | | | | | | | | - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init | ||||
* | Merge pull request #782 from WANdisco/master | Reynold Xin | 2013-08-05 | 2 | -0/+2 |
|\ | | | | | SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD | ||||
| * | SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD | Alexander Pivovarov | 2013-08-05 | 2 | -0/+2 |
| | | |||||
* | | Merge pull request #769 from markhamstra/NegativeCores | Matei Zaharia | 2013-08-05 | 3 | -9/+27 |
|\ \ | | | | | | | SPARK-847 + SPARK-845: Zombie workers and negative cores | ||||
| * | | Moved handling of timed out workers within the Master actor | Mark Hamstra | 2013-08-05 | 2 | -1/+11 |
| | | | |||||
| * | | milliseconds -> seconds in timeOutDeadWorkers logging | Mark Hamstra | 2013-08-05 | 1 | -1/+1 |
| | | | |||||
| * | | Timeout zombie workers | Mark Hamstra | 2013-08-05 | 1 | -6/+12 |
| | | | |||||
| * | | Only reduce the number of cores once when removing an executor | Mikhail Bautin | 2013-08-05 | 1 | -2/+4 |
| | | | |||||
* | | | Merge pull request #774 from pwendell/job-description | Matei Zaharia | 2013-08-05 | 7 | -17/+39 |
|\ \ \ | |/ / |/| | | Show user-defined job name in UI | ||||
| * | | Show user-defined job name in UI | Patrick Wendell | 2013-08-02 | 7 | -17/+39 |
| |/ | |||||
* | | SPARK-850: give better console message | Bill Zhao | 2013-08-05 | 1 | -1/+2 |
| | | |||||
* | | Merge pull request #772 from karenfeng/ui-843 | Matei Zaharia | 2013-08-02 | 2 | -1/+7 |
|\ \ | |/ |/| | Show app duration | ||||
| * | Shows time the app has been running | Karen Feng | 2013-08-02 | 2 | -1/+7 |
| | | |||||
* | | Merge pull request #743 from pwendell/app-metrics | Patrick Wendell | 2013-08-01 | 4 | -6/+50 |
|\ \ | | | | | | | Add application metrics to standalone master | ||||
| * | | under_scores --> camelCase for config options | Patrick Wendell | 2013-08-01 | 1 | -1/+1 |
| | | | |||||
| * | | Small style fix | Patrick Wendell | 2013-08-01 | 1 | -2/+1 |
| | | | |||||
| * | | Adding application-level metrics. | Patrick Wendell | 2013-08-01 | 4 | -6/+51 |
| |/ | | | | | | | This adds metrics for applications in the deploy Master. | ||||
* | | Minor refactoring and code cleanup | Patrick Wendell | 2013-08-01 | 5 | -121/+28 |
| | | |||||
* | | Slight refactoring to SparkContext functions | Patrick Wendell | 2013-08-01 | 2 | -7/+10 |
| | | |||||
* | | Increasing inter job arrival | Patrick Wendell | 2013-08-01 | 1 | -3/+1 |
| | | |||||
* | | Minor style fixes | Patrick Wendell | 2013-08-01 | 2 | -2/+1 |
|/ | |||||
* | Merge pull request #695 from xiajunluan/pool_ui | Patrick Wendell | 2013-08-01 | 22 | -262/+607 |
|\ | | | | | Enhance job ui in spark ui system with adding pool information | ||||
| * | fix bug of spark "SubmitStage" listener as unit test error | Andrew xia | 2013-08-01 | 1 | -3/+7 |
| | | |||||
| * | change function name "setName" to "setProperties" as "setName" is also ↵ | Andrew xia | 2013-08-01 | 1 | -4/+7 |
| | | | | | | | | member of Thread class | ||||
| * | Merge branch 'master' into Pool_UI | Andrew xia | 2013-07-31 | 322 | -963/+7453 |
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/scheduler/DAGScheduler.scala core/src/main/scala/spark/scheduler/SparkListener.scala core/src/main/scala/spark/scheduler/cluster/ClusterTaskSetManager.scala core/src/main/scala/spark/scheduler/cluster/TaskSetManager.scala core/src/main/scala/spark/scheduler/local/LocalTaskSetManager.scala core/src/main/scala/spark/ui/jobs/IndexPage.scala core/src/main/scala/spark/ui/jobs/JobProgressUI.scala | ||||
| * | | refactor codes less than 100 character per line | Andrew xia | 2013-07-30 | 9 | -24/+39 |
| | | | |||||
| * | | refactor job ui with pool information | Andrew xia | 2013-07-30 | 17 | -136/+116 |
| | | | |||||
| * | | Enhance job ui in spark ui system with adding pool information | Andrew xia | 2013-07-12 | 20 | -199/+530 |
| | | | |||||
* | | | Merge pull request #760 from karenfeng/heading-update | Matei Zaharia | 2013-08-01 | 1 | -3/+2 |
|\ \ \ | | | | | | | | | Clean up web UI page headers | ||||
| * | | | Reduced size of heading | Karen Feng | 2013-07-31 | 1 | -2/+2 |
| | | | | |||||
| * | | | Removed master URL from job UI, reduced heading size of basic spark pages | Karen Feng | 2013-07-31 | 1 | -3/+2 |
| | |/ | |/| | |||||
* | | | Removing extra imports | Patrick Wendell | 2013-08-01 | 1 | -4/+0 |
| | | | |||||
* | | | Merge branch 'master' into master-json | Patrick Wendell | 2013-08-01 | 53 | -576/+865 |
|\ \ \ | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/deploy/master/ui/IndexPage.scala | ||||
| * \ \ | Merge remote-tracking branch 'dlyubimov/SPARK-827' | Matei Zaharia | 2013-07-31 | 1 | -1/+1 |
| |\ \ \ | | | | | | | | | | | | | | | | | | | | | Conflicts: docs/configuration.md | ||||
| | * | | | changes per comments. | Dmitriy Lyubimov | 2013-07-27 | 1 | -1/+1 |
| | | | | | |||||
| | * | | | style | Dmitriy Lyubimov | 2013-07-26 | 1 | -1/+1 |
| | | | | | |||||
| | * | | | Maximum task failures configurable | Dmitriy Lyubimov | 2013-07-26 | 1 | -1/+1 |
| | | | | | |||||
| * | | | | Merge pull request #757 from BlackNiuza/result_task_generation | Matei Zaharia | 2013-07-31 | 1 | -0/+2 |
| |\ \ \ \ | | | | | | | | | | | | | Bug fix: SPARK-837 | ||||
| | * | | | | write and read generation in ResultTask | BlackNiuza | 2013-08-01 | 1 | -0/+2 |
| | | | | | | |||||
| * | | | | | Merge pull request #749 from benh/spark-executor-uri | Matei Zaharia | 2013-07-31 | 2 | -12/+32 |
| |\ \ \ \ \ | | | | | | | | | | | | | | | Added property 'spark.executor.uri' for launching on Mesos. | ||||
| | * | | | | | Used 'uri.split('/').last' instead of 'new File(uri).getName()'. | Benjamin Hindman | 2013-07-31 | 2 | -2/+2 |
| | | | | | | | |||||
| | * | | | | | Added property 'spark.executor.uri' for launching on Mesos without | Benjamin Hindman | 2013-07-29 | 2 | -12/+32 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | requiring Spark to be installed. Using 'make_distribution.sh' a user can put a Spark distribution at a URI supported by Mesos (e.g., 'hdfs://...') and then set that when launching their job. Also added SPARK_EXECUTOR_URI for the REPL. | ||||
| * | | | | | | Merge pull request #754 from rxin/compression | Matei Zaharia | 2013-07-31 | 7 | -38/+180 |
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | | Compression codec change | ||||
| | * | | | | | | Changed other LZF uses to use the compression codec interface. | Reynold Xin | 2013-07-31 | 2 | -29/+22 |
| | | | | | | | | |||||
| | * | | | | | | Renamed compressionOutputStream and compressionInputStream to ↵ | Reynold Xin | 2013-07-30 | 3 | -10/+10 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | compressedOutputStream and compressedInputStream. | ||||
| | * | | | | | | Updated the configuration option for Snappy block size to be consistent with ↵ | Reynold Xin | 2013-07-30 | 1 | -1/+1 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the documentation. | ||||
| | * | | | | | | Added Snappy dependency to Maven build files. | Reynold Xin | 2013-07-30 | 1 | -0/+4 |
| | | | | | | | | |||||
| | * | | | | | | Added unit test for compression codecs. | Reynold Xin | 2013-07-30 | 1 | -0/+62 |
| | | | | | | | | |||||
| | * | | | | | | CompressionCodec cleanup. Moved it to spark.io package. | Reynold Xin | 2013-07-30 | 6 | -70/+91 |
| | | | | | | | | |||||
| | * | | | | | | Merge branch 'lazy_file_open' of github.com:lyogavin/spark into compression | Reynold Xin | 2013-07-30 | 5 | -3/+65 |
| | |\ \ \ \ \ \ | | | |_|_|_|/ / | | |/| | | | | | | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala |