diff options
author | Patrick Wendell <pwendell@apache.org> | 2014-02-15 01:21:24 +0000 |
---|---|---|
committer | Patrick Wendell <pwendell@apache.org> | 2014-02-15 01:21:24 +0000 |
commit | bf1756db29d9af0c635533da5e4c96ea6e5c6639 (patch) | |
tree | 5e7f10a4a3a3bf78f93953b50c784e5df778f87d /site/releases/spark-release-0-8-1.html | |
parent | 7d65463580219ba0f2847a2ef4bd7896eb7daacd (diff) | |
download | spark-website-bf1756db29d9af0c635533da5e4c96ea6e5c6639.tar.gz spark-website-bf1756db29d9af0c635533da5e4c96ea6e5c6639.tar.bz2 spark-website-bf1756db29d9af0c635533da5e4c96ea6e5c6639.zip |
Adjustment to credits for Prabeesh K
Diffstat (limited to 'site/releases/spark-release-0-8-1.html')
-rw-r--r-- | site/releases/spark-release-0-8-1.html | 86 |
1 files changed, 43 insertions, 43 deletions
diff --git a/site/releases/spark-release-0-8-1.html b/site/releases/spark-release-0-8-1.html index eba6631ed..4f42ceacd 100644 --- a/site/releases/spark-release-0-8-1.html +++ b/site/releases/spark-release-0-8-1.html @@ -163,7 +163,7 @@ <p>Apache Spark 0.8.1 is a maintenance and performance release for the Scala 2.9 version of Spark. It also adds several new features, such as standalone mode high availability, that will appear in Spark 0.9 but developers wanted to have in Scala 2.9. Contributions to 0.8.1 came from 41 developers.</p> <h3 id="yarn-22-support">YARN 2.2 Support</h3> -<p>Support has been added for running Spark on YARN 2.2 and newer. Due to a change in the YARN API between previous versions and 2.2+, this was not supported in Spark 0.8.0. See the <a href="/docs/0.8.1/running-on-yarn.html">YARN documentation</a> for specific instructions on how to build Spark for YARN 2.2+. We’ve also included a pre-compiled binary for YARN 2.2.</p> +<p>Support has been added for running Spark on YARN 2.2 and newer. Due to a change in the YARN API between previous versions and 2.2+, this was not supported in Spark 0.8.0. See the <a href="/docs/0.8.1/running-on-yarn.html">YARN documentation</a> for specific instructions on how to build Spark for YARN 2.2+. We’ve also included a pre-compiled binary for YARN 2.2.</p> <h3 id="high-availability-mode-for-standalone-cluster-manager">High Availability Mode for Standalone Cluster Manager</h3> <p>The standalone cluster manager now has a high availability (H/A) mode which can tolerate master failures. This is particularly useful for long-running applications such as streaming jobs and the shark server, where the scheduler master previously represented a single point of failure. Instructions for deploying H/A mode are included <a href="/docs/0.8.1/spark-standalone.html#high-availability">in the documentation</a>. The current implementation uses Zookeeper for coordination.</p> @@ -174,7 +174,7 @@ <ul> <li>Optimized hashtables for shuffle data - reduces memory and CPU consumption</li> <li>Efficient encoding for JobConfs - improves latency for stages reading large numbers of blocks from HDFS, S3, and HBase</li> - <li>Shuffle file consolidation (off by default) - reduces the number of files created in large shuffles for better filesystem performance. This change works best on filesystems newer than ext3 (we recommend ext4 or XFS), and it will be the default in Spark 0.9, but we’ve left it off by default for compatibility. We recommend users turn this on unless they are using ext3 by setting <code>spark.shuffle.consolidateFiles</code> to “true”.</li> + <li>Shuffle file consolidation (off by default) - reduces the number of files created in large shuffles for better filesystem performance. This change works best on filesystems newer than ext3 (we recommend ext4 or XFS), and it will be the default in Spark 0.9, but we’ve left it off by default for compatibility. We recommend users turn this on unless they are using ext3 by setting <code>spark.shuffle.consolidateFiles</code> to “true”.</li> <li>Torrent broadcast (off by default) - a faster broadcast implementation for large objects.</li> <li>Support for fetching large result sets - allows tasks to return large results without tuning Akka buffer sizes.</li> </ul> @@ -211,47 +211,47 @@ <h3 id="credits">Credits</h3> <ul> - <li>Michael Armbrust – build fix</li> - <li>Pierre Borckmans – typo fix in documentation</li> - <li>Evan Chan – <code>local://</code> scheme for dependency jars</li> - <li>Ewen Cheslack-Postava – <code>add</code> method for python accumulators, support for setting config properties in python</li> - <li>Mosharaf Chowdhury – optimized broadcast implementation</li> - <li>Frank Dai – documentation fix</li> - <li>Aaron Davidson – shuffle file consolidation, H/A mode for standalone scheduler, cleaned up representation of block IDs, several improvements and bug fixes</li> - <li>Tathagata Das – new streaming operators, fix for kafka concurrency bug</li> - <li>Ankur Dave – support for pausing spot clusters on EC2</li> - <li>Harvey Feng – optimization to JobConf broadcasts, bug fixes, YARN 2.2 build</li> - <li>Ali Ghodsi – YARN 2.2 build</li> - <li>Thomas Graves – Spark YARN integration including secure HDFS access over YARN</li> - <li>Li Guoqiang – fix for Maven build</li> - <li>Stephen Haberman – bug fix</li> - <li>Haidar Hadi – documentation fix</li> - <li>Nathan Howell – bug fix relating to YARN</li> - <li>Holden Karau – Java version of <code>mapPartitionsWithIndex</code></li> - <li>Du Li – bug fix in make-distrubion.sh</li> - <li>Raymond Liu – work on YARN 2.2 build</li> - <li>Xi Liu – bug fix and code clean-up</li> - <li>David McCauley – bug fix in standalone mode JSON output</li> - <li>Michael (wannabeast) – bug fix in memory store</li> - <li>Fabrizio Milo – typos in documentation, clean-up in DAGScheduler, typo in scaladoc</li> - <li>Mridul Muralidharan – fixes to metadata cleaner and speculative execution</li> - <li>Sundeep Narravula – build fix, bug fixes in scheduler and tests, code clean-up</li> - <li>Kay Ousterhout – optimized result fetching, new information in UI, scheduler clean-up and bug fixes</li> - <li>Nick Pentreath – implicit feedback variant of ALS algorithm</li> - <li>Imran Rashid – improvement to executor launch</li> - <li>Ahir Reddy – spark support for SIMR</li> - <li>Josh Rosen – memory use optimization, clean up of BlockManager code, Java and Python clean-up/fixes</li> - <li>Henry Saputra – build fix</li> - <li>Jerry Shao – refactoring of fair scheduler, support for running Spark as a specific user, bug fix</li> - <li>Mingfei Shi – documentation for JobLogger</li> - <li>Andre Schumacher – sortByKey in PySpark and associated changes</li> - <li>Karthik Tunga – bug fix in launch script</li> - <li>Patrick Wendell – <code>repartition</code> operator, shuffle write metrics, various fixes and release management</li> - <li>Neal Wiggins – import clean-up, documentation fixes</li> - <li>Andrew Xia – bug fix in UI</li> - <li>Reynold Xin – task killing, support for setting job properties in Spark shell, logging improvements, Kryo improvements, several bug fixes</li> - <li>Matei Zaharia – optimized hashmap for shuffle data, PySpark documentation, optimizations to Kryo serializer</li> - <li>Wu Zeming – bug fix in executors UI</li> + <li>Michael Armbrust – build fix</li> + <li>Pierre Borckmans – typo fix in documentation</li> + <li>Evan Chan – <code>local://</code> scheme for dependency jars</li> + <li>Ewen Cheslack-Postava – <code>add</code> method for python accumulators, support for setting config properties in python</li> + <li>Mosharaf Chowdhury – optimized broadcast implementation</li> + <li>Frank Dai – documentation fix</li> + <li>Aaron Davidson – shuffle file consolidation, H/A mode for standalone scheduler, cleaned up representation of block IDs, several improvements and bug fixes</li> + <li>Tathagata Das – new streaming operators, fix for kafka concurrency bug</li> + <li>Ankur Dave – support for pausing spot clusters on EC2</li> + <li>Harvey Feng – optimization to JobConf broadcasts, bug fixes, YARN 2.2 build</li> + <li>Ali Ghodsi – YARN 2.2 build</li> + <li>Thomas Graves – Spark YARN integration including secure HDFS access over YARN</li> + <li>Li Guoqiang – fix for Maven build</li> + <li>Stephen Haberman – bug fix</li> + <li>Haidar Hadi – documentation fix</li> + <li>Nathan Howell – bug fix relating to YARN</li> + <li>Holden Karau – Java version of <code>mapPartitionsWithIndex</code></li> + <li>Du Li – bug fix in make-distrubion.sh</li> + <li>Raymond Liu – work on YARN 2.2 build</li> + <li>Xi Liu – bug fix and code clean-up</li> + <li>David McCauley – bug fix in standalone mode JSON output</li> + <li>Michael (wannabeast) – bug fix in memory store</li> + <li>Fabrizio Milo – typos in documentation, clean-up in DAGScheduler, typo in scaladoc</li> + <li>Mridul Muralidharan – fixes to metadata cleaner and speculative execution</li> + <li>Sundeep Narravula – build fix, bug fixes in scheduler and tests, code clean-up</li> + <li>Kay Ousterhout – optimized result fetching, new information in UI, scheduler clean-up and bug fixes</li> + <li>Nick Pentreath – implicit feedback variant of ALS algorithm</li> + <li>Imran Rashid – improvement to executor launch</li> + <li>Ahir Reddy – spark support for SIMR</li> + <li>Josh Rosen – memory use optimization, clean up of BlockManager code, Java and Python clean-up/fixes</li> + <li>Henry Saputra – build fix</li> + <li>Jerry Shao – refactoring of fair scheduler, support for running Spark as a specific user, bug fix</li> + <li>Mingfei Shi – documentation for JobLogger</li> + <li>Andre Schumacher – sortByKey in PySpark and associated changes</li> + <li>Karthik Tunga – bug fix in launch script</li> + <li>Patrick Wendell – <code>repartition</code> operator, shuffle write metrics, various fixes and release management</li> + <li>Neal Wiggins – import clean-up, documentation fixes</li> + <li>Andrew Xia – bug fix in UI</li> + <li>Reynold Xin – task killing, support for setting job properties in Spark shell, logging improvements, Kryo improvements, several bug fixes</li> + <li>Matei Zaharia – optimized hashmap for shuffle data, PySpark documentation, optimizations to Kryo serializer</li> + <li>Wu Zeming – bug fix in executors UI</li> </ul> <p>Thanks to everyone who contributed!</p> |