Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | SPARK-894 - Not all WebUI fields delivered VIA JSON | David McCauley | 2013-09-11 | 1 | -1/+3 |
| | |||||
* | Merge pull request #917 from pwendell/master | shivaram | 2013-09-09 | 1 | -0/+7 |
|\ | | | | | Document libgfortran dependency for MLBase | ||||
| * | Document fortran dependency for MLBase | Patrick Wendell | 2013-09-09 | 1 | -0/+7 |
|/ | |||||
* | Merge pull request #915 from ooyala/master | Matei Zaharia | 2013-09-09 | 1 | -1/+9 |
|\ | | | | | Get rid of / improve ugly NPE when Utils.deleteRecursively() fails | ||||
| * | Style fix: put body of if within curly braces | Evan Chan | 2013-09-09 | 1 | -1/+3 |
| | | |||||
| * | Print out more friendly error if listFiles() fails | Evan Chan | 2013-09-09 | 1 | -1/+7 |
| | | | | | | | | listFiles() could return null if the I/O fails, and this currently results in an ugly NPE which is hard to diagnose. | ||||
* | | Merge pull request #916 from mateiz/mkdist-fix | Matei Zaharia | 2013-09-09 | 1 | -1/+1 |
|\ \ | | | | | | | Fix copy issue in https://github.com/mesos/spark/pull/899 | ||||
| * | | Fix copy issue in https://github.com/mesos/spark/pull/899 | Matei Zaharia | 2013-09-09 | 1 | -1/+1 |
|/ / | |||||
* | | Merge pull request #907 from stephenh/document_coalesce_shuffle | Matei Zaharia | 2013-09-09 | 2 | -4/+27 |
|\ \ | | | | | | | Add better docs for coalesce. | ||||
| * | | Use a set since shuffle could change order. | Stephen Haberman | 2013-09-09 | 1 | -1/+1 |
| | | | |||||
| * | | Reword 'evenly distributed' to 'distributed with a hash partitioner. | Stephen Haberman | 2013-09-09 | 1 | -2/+2 |
| | | | |||||
| * | | Add better docs for coalesce. | Stephen Haberman | 2013-09-08 | 2 | -4/+27 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Include the useful tip that if shuffle=true, coalesce can actually increase the number of partitions. This makes coalesce more like a generic `RDD.repartition` operation. (Ideally this `RDD.repartition` could automatically choose either a coalesce or a shuffle if numPartitions was either less than or greater than, respectively, the current number of partitions.) | ||||
* | | | Merge pull request #912 from tgravescs/ganglia-pom | Matei Zaharia | 2013-09-09 | 1 | -0/+4 |
|\ \ \ | | | | | | | | | Add metrics-ganglia to core pom file | ||||
| * | | | Add metrics-ganglia to core pom file | Y.CORP.YAHOO.COM\tgraves | 2013-09-09 | 1 | -0/+4 |
|/ / / | |||||
* | | | Merge pull request #911 from pwendell/ganglia-sink | Matei Zaharia | 2013-09-09 | 1 | -0/+5 |
|\ \ \ | | | | | | | | | Adding Manen dependency for Ganglia | ||||
| * | | | Adding Manen dependency | Patrick Wendell | 2013-09-09 | 1 | -0/+5 |
| | | | | |||||
* | | | | Merge pull request #890 from mridulm/master | Matei Zaharia | 2013-09-08 | 3 | -2/+17 |
|\ \ \ \ | | | | | | | | | | | Fix hash bug | ||||
| * | | | | Address review comments - rename toHash to nonNegativeHash | Mridul Muralidharan | 2013-09-04 | 3 | -3/+3 |
| | | | | | |||||
| * | | | | Fix hash bug - caused failure after 35k stages, sigh | Mridul Muralidharan | 2013-09-04 | 3 | -2/+17 |
| | | | | | |||||
* | | | | | Merge pull request #909 from mateiz/exec-id-fix | Reynold Xin | 2013-09-08 | 2 | -7/+7 |
|\ \ \ \ \ | | | | | | | | | | | | | Fix an instance where full standalone mode executor IDs were passed to | ||||
| * | | | | | Fix an instance where full standalone mode executor IDs were passed to | Matei Zaharia | 2013-09-08 | 2 | -7/+7 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | StandaloneSchedulerBackend instead of the smaller IDs used within Spark (that lack the application name). This was reported by ClearStory in https://github.com/clearstorydata/spark/pull/9. Also fixed some messages that said slave instead of executor. | ||||
* | | | | | | Merge pull request #910 from mateiz/ml-doc-tweaks | Matei Zaharia | 2013-09-08 | 3 | -18/+17 |
|\ \ \ \ \ \ | | | | | | | | | | | | | | | Small tweaks to MLlib docs | ||||
| * | | | | | | Small tweaks to MLlib docs | Matei Zaharia | 2013-09-08 | 3 | -18/+17 |
|/ / / / / / | |||||
* | | | | | | Merge pull request #905 from mateiz/docs2 | Matei Zaharia | 2013-09-08 | 26 | -166/+499 |
|\ \ \ \ \ \ | | | | | | | | | | | | | | | Job scheduling and cluster mode docs | ||||
| * | | | | | | Fix some review comments | Matei Zaharia | 2013-09-08 | 2 | -2/+2 |
| | | | | | | | |||||
| * | | | | | | Fix unit test failure due to changed default | Matei Zaharia | 2013-09-08 | 1 | -1/+1 |
| | | | | | | | |||||
| * | | | | | | Updated cluster diagram to show caches | Matei Zaharia | 2013-09-08 | 2 | -0/+0 |
| | | | | | | | |||||
| * | | | | | | Review comments | Matei Zaharia | 2013-09-08 | 2 | -1/+48 |
| | | | | | | | |||||
| * | | | | | | Some tweaks to CDH/HDP doc | Matei Zaharia | 2013-09-08 | 1 | -10/+52 |
| | | | | | | | |||||
| * | | | | | | Added cluster overview doc, made logo higher-resolution, and added more | Matei Zaharia | 2013-09-08 | 7 | -15/+88 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | details on monitoring | ||||
| * | | | | | | More fair scheduler docs and property names. | Matei Zaharia | 2013-09-08 | 14 | -98/+185 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also changed uses of "job" terminology to "application" when they referred to an entire Spark program, to avoid confusion. | ||||
| * | | | | | | Work in progress: | Matei Zaharia | 2013-09-08 | 10 | -49/+133 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | - Add job scheduling docs - Rename some fair scheduler properties - Organize intro page better - Link to Apache wiki for "contributing to Spark" | ||||
* | | | | | | | Merge pull request #896 from atalwalkar/master | Matei Zaharia | 2013-09-08 | 2 | -1/+508 |
|\ \ \ \ \ \ \ | | | | | | | | | | | | | | | | | updated content | ||||
| * | | | | | | | respose to PR comments | Ameet Talwalkar | 2013-09-08 | 2 | -25/+352 |
| | | | | | | | | |||||
| * | | | | | | | Merge remote-tracking branch 'upstream/master' | Ameet Talwalkar | 2013-09-08 | 38 | -261/+786 |
| |\ \ \ \ \ \ \ | |/ / / / / / / |/| | | | | | | | |||||
* | | | | | | | | Merge pull request #906 from pwendell/ganglia-sink | Patrick Wendell | 2013-09-08 | 12 | -36/+166 |
|\ \ \ \ \ \ \ \ | |_|_|/ / / / / |/| | | | / / / | | |_|_|/ / / | |/| | | | | | Clean-up of Metrics Code/Docs and Add Ganglia Sink | ||||
| * | | | | | | Adding sc name in metrics source | Patrick Wendell | 2013-09-08 | 5 | -9/+14 |
| | | | | | | | |||||
| * | | | | | | Fixing package name in template conf | Patrick Wendell | 2013-09-08 | 1 | -7/+7 |
| | | | | | | | |||||
| * | | | | | | Adding more docs and some code cleanup | Patrick Wendell | 2013-09-08 | 5 | -20/+62 |
| | | | | | | | |||||
| * | | | | | | Ganglia sink | Patrick Wendell | 2013-09-08 | 2 | -0/+83 |
| | | | | | | | |||||
* | | | | | | | Merge pull request #908 from pwendell/master | Matei Zaharia | 2013-09-08 | 1 | -1/+7 |
|\ \ \ \ \ \ \ | |_|_|_|_|/ / |/| | | | | | | Fix target JVM version in scala build | ||||
| * | | | | | | Fix target JVM version in scala build | Patrick Wendell | 2013-09-08 | 1 | -1/+7 |
| |/ / / / / | |||||
* | | | | | | Merge pull request #898 from ilikerps/660 | Matei Zaharia | 2013-09-08 | 6 | -4/+81 |
|\ \ \ \ \ \ | |_|_|/ / / |/| | | | | | SPARK-660: Add StorageLevel support in Python | ||||
| * | | | | | Whoopsy daisy | Aaron Davidson | 2013-09-08 | 1 | -1/+0 |
| | | | | | | |||||
| * | | | | | Export StorageLevel and refactor | Aaron Davidson | 2013-09-07 | 6 | -29/+65 |
| | | | | | | |||||
| * | | | | | Remove reflection, hard-code StorageLevels | Aaron Davidson | 2013-09-07 | 3 | -35/+26 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | The sc.StorageLevel -> StorageLevel pathway is a bit janky, but otherwise the shell would have to call a private method of SparkContext. Having StorageLevel available in sc also doesn't seem like the end of the world. There may be a better solution, though. As for creating the StorageLevel object itself, this seems to be the best way in Python 2 for creating singleton, enum-like objects: http://stackoverflow.com/questions/36932/how-can-i-represent-an-enum-in-python | ||||
| * | | | | | Memoize StorageLevels read from JVM | Aaron Davidson | 2013-09-06 | 2 | -3/+10 |
| | | | | | | |||||
| * | | | | | SPARK-660: Add StorageLevel support in Python | Aaron Davidson | 2013-09-05 | 4 | -1/+45 |
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | It uses reflection... I am not proud of that fact, but it at least ensures compatibility (sans refactoring of the StorageLevel stuff). | ||||
* | | | | | | Merge pull request #900 from pwendell/cdh-docs | Matei Zaharia | 2013-09-08 | 2 | -0/+77 |
|\ \ \ \ \ \ | | | | | | | | | | | | | | | Provide docs to describe running on CDH/HDP cluster. | ||||
| * | | | | | | File rename | Patrick Wendell | 2013-09-07 | 2 | -4/+2 |
| | | | | | | |