aboutsummaryrefslogtreecommitdiff
Commit message (Collapse)AuthorAgeFilesLines
* Merge pull request #917 from pwendell/mastershivaram2013-09-091-0/+7
|\ | | | | Document libgfortran dependency for MLBase
| * Document fortran dependency for MLBasePatrick Wendell2013-09-091-0/+7
|/
* Merge pull request #915 from ooyala/masterMatei Zaharia2013-09-091-1/+9
|\ | | | | Get rid of / improve ugly NPE when Utils.deleteRecursively() fails
| * Style fix: put body of if within curly bracesEvan Chan2013-09-091-1/+3
| |
| * Print out more friendly error if listFiles() failsEvan Chan2013-09-091-1/+7
| | | | | | | | listFiles() could return null if the I/O fails, and this currently results in an ugly NPE which is hard to diagnose.
* | Merge pull request #916 from mateiz/mkdist-fixMatei Zaharia2013-09-091-1/+1
|\ \ | | | | | | Fix copy issue in https://github.com/mesos/spark/pull/899
| * | Fix copy issue in https://github.com/mesos/spark/pull/899Matei Zaharia2013-09-091-1/+1
|/ /
* | Merge pull request #907 from stephenh/document_coalesce_shuffleMatei Zaharia2013-09-092-4/+27
|\ \ | | | | | | Add better docs for coalesce.
| * | Use a set since shuffle could change order.Stephen Haberman2013-09-091-1/+1
| | |
| * | Reword 'evenly distributed' to 'distributed with a hash partitioner.Stephen Haberman2013-09-091-2/+2
| | |
| * | Add better docs for coalesce.Stephen Haberman2013-09-082-4/+27
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Include the useful tip that if shuffle=true, coalesce can actually increase the number of partitions. This makes coalesce more like a generic `RDD.repartition` operation. (Ideally this `RDD.repartition` could automatically choose either a coalesce or a shuffle if numPartitions was either less than or greater than, respectively, the current number of partitions.)
* | | Merge pull request #912 from tgravescs/ganglia-pomMatei Zaharia2013-09-091-0/+4
|\ \ \ | | | | | | | | Add metrics-ganglia to core pom file
| * | | Add metrics-ganglia to core pom fileY.CORP.YAHOO.COM\tgraves2013-09-091-0/+4
|/ / /
* | | Merge pull request #911 from pwendell/ganglia-sinkMatei Zaharia2013-09-091-0/+5
|\ \ \ | | | | | | | | Adding Manen dependency for Ganglia
| * | | Adding Manen dependencyPatrick Wendell2013-09-091-0/+5
| | | |
* | | | Merge pull request #890 from mridulm/masterMatei Zaharia2013-09-083-2/+17
|\ \ \ \ | | | | | | | | | | Fix hash bug
| * | | | Address review comments - rename toHash to nonNegativeHashMridul Muralidharan2013-09-043-3/+3
| | | | |
| * | | | Fix hash bug - caused failure after 35k stages, sighMridul Muralidharan2013-09-043-2/+17
| | | | |
* | | | | Merge pull request #909 from mateiz/exec-id-fixReynold Xin2013-09-082-7/+7
|\ \ \ \ \ | | | | | | | | | | | | Fix an instance where full standalone mode executor IDs were passed to
| * | | | | Fix an instance where full standalone mode executor IDs were passed toMatei Zaharia2013-09-082-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | StandaloneSchedulerBackend instead of the smaller IDs used within Spark (that lack the application name). This was reported by ClearStory in https://github.com/clearstorydata/spark/pull/9. Also fixed some messages that said slave instead of executor.
* | | | | | Merge pull request #910 from mateiz/ml-doc-tweaksMatei Zaharia2013-09-083-18/+17
|\ \ \ \ \ \ | | | | | | | | | | | | | | Small tweaks to MLlib docs
| * | | | | | Small tweaks to MLlib docsMatei Zaharia2013-09-083-18/+17
|/ / / / / /
* | | | | | Merge pull request #905 from mateiz/docs2Matei Zaharia2013-09-0826-166/+499
|\ \ \ \ \ \ | | | | | | | | | | | | | | Job scheduling and cluster mode docs
| * | | | | | Fix some review commentsMatei Zaharia2013-09-082-2/+2
| | | | | | |
| * | | | | | Fix unit test failure due to changed defaultMatei Zaharia2013-09-081-1/+1
| | | | | | |
| * | | | | | Updated cluster diagram to show cachesMatei Zaharia2013-09-082-0/+0
| | | | | | |
| * | | | | | Review commentsMatei Zaharia2013-09-082-1/+48
| | | | | | |
| * | | | | | Some tweaks to CDH/HDP docMatei Zaharia2013-09-081-10/+52
| | | | | | |
| * | | | | | Added cluster overview doc, made logo higher-resolution, and added moreMatei Zaharia2013-09-087-15/+88
| | | | | | | | | | | | | | | | | | | | | | | | | | | | details on monitoring
| * | | | | | More fair scheduler docs and property names.Matei Zaharia2013-09-0814-98/+185
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also changed uses of "job" terminology to "application" when they referred to an entire Spark program, to avoid confusion.
| * | | | | | Work in progress:Matei Zaharia2013-09-0810-49/+133
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Add job scheduling docs - Rename some fair scheduler properties - Organize intro page better - Link to Apache wiki for "contributing to Spark"
* | | | | | | Merge pull request #896 from atalwalkar/masterMatei Zaharia2013-09-082-1/+508
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | updated content
| * | | | | | | respose to PR commentsAmeet Talwalkar2013-09-082-25/+352
| | | | | | | |
| * | | | | | | Merge remote-tracking branch 'upstream/master'Ameet Talwalkar2013-09-0838-261/+786
| |\ \ \ \ \ \ \ | |/ / / / / / / |/| | | | | | |
* | | | | | | | Merge pull request #906 from pwendell/ganglia-sinkPatrick Wendell2013-09-0812-36/+166
|\ \ \ \ \ \ \ \ | |_|_|/ / / / / |/| | | | / / / | | |_|_|/ / / | |/| | | | | Clean-up of Metrics Code/Docs and Add Ganglia Sink
| * | | | | | Adding sc name in metrics sourcePatrick Wendell2013-09-085-9/+14
| | | | | | |
| * | | | | | Fixing package name in template confPatrick Wendell2013-09-081-7/+7
| | | | | | |
| * | | | | | Adding more docs and some code cleanupPatrick Wendell2013-09-085-20/+62
| | | | | | |
| * | | | | | Ganglia sinkPatrick Wendell2013-09-082-0/+83
| | | | | | |
* | | | | | | Merge pull request #908 from pwendell/masterMatei Zaharia2013-09-081-1/+7
|\ \ \ \ \ \ \ | |_|_|_|_|/ / |/| | | | | | Fix target JVM version in scala build
| * | | | | | Fix target JVM version in scala buildPatrick Wendell2013-09-081-1/+7
| |/ / / / /
* | | | | | Merge pull request #898 from ilikerps/660Matei Zaharia2013-09-086-4/+81
|\ \ \ \ \ \ | |_|_|/ / / |/| | | | | SPARK-660: Add StorageLevel support in Python
| * | | | | Whoopsy daisyAaron Davidson2013-09-081-1/+0
| | | | | |
| * | | | | Export StorageLevel and refactorAaron Davidson2013-09-076-29/+65
| | | | | |
| * | | | | Remove reflection, hard-code StorageLevelsAaron Davidson2013-09-073-35/+26
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sc.StorageLevel -> StorageLevel pathway is a bit janky, but otherwise the shell would have to call a private method of SparkContext. Having StorageLevel available in sc also doesn't seem like the end of the world. There may be a better solution, though. As for creating the StorageLevel object itself, this seems to be the best way in Python 2 for creating singleton, enum-like objects: http://stackoverflow.com/questions/36932/how-can-i-represent-an-enum-in-python
| * | | | | Memoize StorageLevels read from JVMAaron Davidson2013-09-062-3/+10
| | | | | |
| * | | | | SPARK-660: Add StorageLevel support in PythonAaron Davidson2013-09-054-1/+45
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It uses reflection... I am not proud of that fact, but it at least ensures compatibility (sans refactoring of the StorageLevel stuff).
* | | | | | Merge pull request #900 from pwendell/cdh-docsMatei Zaharia2013-09-082-0/+77
|\ \ \ \ \ \ | | | | | | | | | | | | | | Provide docs to describe running on CDH/HDP cluster.
| * | | | | | File renamePatrick Wendell2013-09-072-4/+2
| | | | | | |
| * | | | | | Changes based on feedbackPatrick Wendell2013-09-071-12/+24
| | | | | | |
generated by cgit v1.2.3 (git 2.39.1) at 2024-09-16 14:56:29 +0000