aboutsummaryrefslogtreecommitdiff
path: root/core
Commit message (Collapse)AuthorAgeFilesLines
* Java examples, tests for KMeans and ALSShivaram Venkataraman2013-08-061-0/+5
| | | | | | | | | | | | | | | - Changes ALS to accept RDD[Rating] instead of (Int, Int, Double) making it easier to call from Java - Renames class methods from `train` to `run` to enable static methods to be called from Java. - Add unit tests which check if both static / class methods can be called. - Also add examples which port the main() function in ALS, KMeans to the examples project. Couple of minor changes to existing code: - Add a toJavaRDD method in RDD to convert scala RDD to java RDD easily - Workaround a bug where using double[] from Java leads to class cast exception in KMeans init
* Merge pull request #782 from WANdisco/masterReynold Xin2013-08-052-0/+2
|\ | | | | SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDD
| * SHARK-94 Log the files computed by HadoopRDD and NewHadoopRDDAlexander Pivovarov2013-08-052-0/+2
| |
* | Merge pull request #769 from markhamstra/NegativeCoresMatei Zaharia2013-08-053-9/+27
|\ \ | | | | | | SPARK-847 + SPARK-845: Zombie workers and negative cores
| * | Moved handling of timed out workers within the Master actorMark Hamstra2013-08-052-1/+11
| | |
| * | milliseconds -> seconds in timeOutDeadWorkers loggingMark Hamstra2013-08-051-1/+1
| | |
| * | Timeout zombie workersMark Hamstra2013-08-051-6/+12
| | |
| * | Only reduce the number of cores once when removing an executorMikhail Bautin2013-08-051-2/+4
| | |
* | | Merge pull request #774 from pwendell/job-descriptionMatei Zaharia2013-08-057-17/+39
|\ \ \ | |/ / |/| | Show user-defined job name in UI
| * | Show user-defined job name in UIPatrick Wendell2013-08-027-17/+39
| |/
* | SPARK-850: give better console messageBill Zhao2013-08-051-1/+2
| |
* | Merge pull request #772 from karenfeng/ui-843Matei Zaharia2013-08-022-1/+7
|\ \ | |/ |/| Show app duration
| * Shows time the app has been runningKaren Feng2013-08-022-1/+7
| |
* | Merge pull request #743 from pwendell/app-metricsPatrick Wendell2013-08-014-6/+50
|\ \ | | | | | | Add application metrics to standalone master
| * | under_scores --> camelCase for config optionsPatrick Wendell2013-08-011-1/+1
| | |
| * | Small style fixPatrick Wendell2013-08-011-2/+1
| | |
| * | Adding application-level metrics.Patrick Wendell2013-08-014-6/+51
| |/ | | | | | | This adds metrics for applications in the deploy Master.
* | Minor refactoring and code cleanupPatrick Wendell2013-08-015-121/+28
| |
* | Slight refactoring to SparkContext functionsPatrick Wendell2013-08-012-7/+10
| |
* | Increasing inter job arrivalPatrick Wendell2013-08-011-3/+1
| |
* | Minor style fixesPatrick Wendell2013-08-012-2/+1
|/
* Merge pull request #695 from xiajunluan/pool_uiPatrick Wendell2013-08-0122-262/+607
|\ | | | | Enhance job ui in spark ui system with adding pool information
| * fix bug of spark "SubmitStage" listener as unit test errorAndrew xia2013-08-011-3/+7
| |
| * change function name "setName" to "setProperties" as "setName" is also ↵Andrew xia2013-08-011-4/+7
| | | | | | | | member of Thread class
| * Merge branch 'master' into Pool_UIAndrew xia2013-07-31322-963/+7453
| |\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/SparkContext.scala core/src/main/scala/spark/scheduler/DAGScheduler.scala core/src/main/scala/spark/scheduler/SparkListener.scala core/src/main/scala/spark/scheduler/cluster/ClusterTaskSetManager.scala core/src/main/scala/spark/scheduler/cluster/TaskSetManager.scala core/src/main/scala/spark/scheduler/local/LocalTaskSetManager.scala core/src/main/scala/spark/ui/jobs/IndexPage.scala core/src/main/scala/spark/ui/jobs/JobProgressUI.scala
| * | refactor codes less than 100 character per lineAndrew xia2013-07-309-24/+39
| | |
| * | refactor job ui with pool informationAndrew xia2013-07-3017-136/+116
| | |
| * | Enhance job ui in spark ui system with adding pool informationAndrew xia2013-07-1220-199/+530
| | |
* | | Merge pull request #760 from karenfeng/heading-updateMatei Zaharia2013-08-011-3/+2
|\ \ \ | | | | | | | | Clean up web UI page headers
| * | | Reduced size of headingKaren Feng2013-07-311-2/+2
| | | |
| * | | Removed master URL from job UI, reduced heading size of basic spark pagesKaren Feng2013-07-311-3/+2
| | |/ | |/|
* | | Removing extra importsPatrick Wendell2013-08-011-4/+0
| | |
* | | Merge branch 'master' into master-jsonPatrick Wendell2013-08-0153-576/+865
|\ \ \ | | | | | | | | | | | | | | | | Conflicts: core/src/main/scala/spark/deploy/master/ui/IndexPage.scala
| * \ \ Merge remote-tracking branch 'dlyubimov/SPARK-827'Matei Zaharia2013-07-311-1/+1
| |\ \ \ | | | | | | | | | | | | | | | | | | | | Conflicts: docs/configuration.md
| | * | | changes per comments.Dmitriy Lyubimov2013-07-271-1/+1
| | | | |
| | * | | styleDmitriy Lyubimov2013-07-261-1/+1
| | | | |
| | * | | Maximum task failures configurableDmitriy Lyubimov2013-07-261-1/+1
| | | | |
| * | | | Merge pull request #757 from BlackNiuza/result_task_generationMatei Zaharia2013-07-311-0/+2
| |\ \ \ \ | | | | | | | | | | | | Bug fix: SPARK-837
| | * | | | write and read generation in ResultTaskBlackNiuza2013-08-011-0/+2
| | | | | |
| * | | | | Merge pull request #749 from benh/spark-executor-uriMatei Zaharia2013-07-312-12/+32
| |\ \ \ \ \ | | | | | | | | | | | | | | Added property 'spark.executor.uri' for launching on Mesos.
| | * | | | | Used 'uri.split('/').last' instead of 'new File(uri).getName()'.Benjamin Hindman2013-07-312-2/+2
| | | | | | |
| | * | | | | Added property 'spark.executor.uri' for launching on Mesos withoutBenjamin Hindman2013-07-292-12/+32
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | requiring Spark to be installed. Using 'make_distribution.sh' a user can put a Spark distribution at a URI supported by Mesos (e.g., 'hdfs://...') and then set that when launching their job. Also added SPARK_EXECUTOR_URI for the REPL.
| * | | | | | Merge pull request #754 from rxin/compressionMatei Zaharia2013-07-317-38/+180
| |\ \ \ \ \ \ | | | | | | | | | | | | | | | | Compression codec change
| | * | | | | | Changed other LZF uses to use the compression codec interface.Reynold Xin2013-07-312-29/+22
| | | | | | | |
| | * | | | | | Renamed compressionOutputStream and compressionInputStream to ↵Reynold Xin2013-07-303-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | compressedOutputStream and compressedInputStream.
| | * | | | | | Updated the configuration option for Snappy block size to be consistent with ↵Reynold Xin2013-07-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | the documentation.
| | * | | | | | Added Snappy dependency to Maven build files.Reynold Xin2013-07-301-0/+4
| | | | | | | |
| | * | | | | | Added unit test for compression codecs.Reynold Xin2013-07-301-0/+62
| | | | | | | |
| | * | | | | | CompressionCodec cleanup. Moved it to spark.io package.Reynold Xin2013-07-306-70/+91
| | | | | | | |
| | * | | | | | Merge branch 'lazy_file_open' of github.com:lyogavin/spark into compressionReynold Xin2013-07-305-3/+65
| | |\ \ \ \ \ \ | | | |_|_|_|/ / | | |/| | | | | | | | | | | | | | | | | | | | | Conflicts: project/SparkBuild.scala
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-12 18:36:04 +0000