diff options
75 files changed, 1191 insertions, 849 deletions
diff --git a/_layouts/global.html b/_layouts/global.html index e8b3b2899..2820ae0f5 100644 --- a/_layouts/global.html +++ b/_layouts/global.html @@ -52,6 +52,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="{{site.url}}js/lang-tabs.js"></script> +<script src="{{site.url}}js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="{{site.url}}js/lang-tabs.js"></script> - </body> </html> diff --git a/downloads.md b/downloads.md index a56f3e4ed..2df8ec5d3 100644 --- a/downloads.md +++ b/downloads.md @@ -7,30 +7,31 @@ navigation: show: true --- +<script type="text/javascript"> +$(document).ready(function() { + initDownloads(); + initReleaseNotes(); +}); +</script> + ## Download Spark -The latest release is Spark 1.0.2, released August 5, 2014 [(release notes)]({{site.url}}releases/spark-release-1-0-2.html) [(git tag)](https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=8fb6f00e195fb258f3f70f04756e07c259a2351f) -Pre-built packages: +Select below to download a release of Spark. +The latest release of Spark is Spark 1.0.2 +<a href="{{site.url}}releases/spark-release-1-0-2.html">(release notes)</a>: + +1. Chose a Spark release: + <select id="sparkVersionSelect" onChange="javascript:onVersionSelect();"></select><br> + +2. Chose a package type: + <select id="sparkPackageSelect" onChange="javascript:onPackageSelect();"></select><br> -* For Hadoop 1 (HDP1, CDH3): -<a href="http://www.apache.org/dyn/closer.cgi/spark/spark-1.0.2/spark-1.0.2-bin-hadoop1.tgz" onClick="trackOutboundLink(this, 'Release Download Links', 'apache_spark-1.0.1-bin-hadoop1.tgz'); return false;"> find an Apache mirror </a> -or -<a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop1.tgz" onClick="trackOutboundLink(this, 'Release Download Links', 'cloudfront_spark-1.0.2-bin-hadoop1.tgz'); return false;"> direct file download </a> -* For CDH4: -<a href="http://www.apache.org/dyn/closer.cgi/spark/spark-1.0.2/spark-1.0.2-bin-cdh4.tgz" onClick="trackOutboundLink(this, 'Release Download Links', 'apache_spark-1.0.1-bin-cdh4.tgz'); return false;"> find an Apache mirror </a> -or -<a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-cdh4.tgz" onClick="trackOutboundLink(this, 'Release Download Links', 'cloudfront_spark-1.0.2-bin-cdh4.tgz'); return false;"> direct file download </a> -* For Hadoop 2 (HDP2, CDH5): -<a href="http://www.apache.org/dyn/closer.cgi/spark/spark-1.0.2/spark-1.0.2-bin-hadoop2.tgz" onClick="trackOutboundLink(this, 'Release Download Links', 'apache_spark-1.0.2-bin-hadoop2.tgz'); return false;"> find an Apache mirror </a> -or -<a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop2.tgz" onClick="trackOutboundLink(this, 'Release Download Links', 'cloudfront_spark-1.0.2-bin-hadoop2.tgz'); return false;"> direct file download </a> +3. Chose a download type: + <select id="sparkDownloadSelect" onChange="javascript:onDownloadSelect()"></select><br> -Pre-built packages, third-party (NOTE: may include non ASF-compatible licenses): - -* For MapRv3: <a href="http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr3.tgz" onClick="trackOutboundLink(this, 'Release Downlaod Links', 'mapr-v3_spark-1.0.2.tgz'); return false;">direct file download (external)</a> -* For MapRv4: <a href="http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr4.tgz" onClick="trackOutboundLink(this, 'Release Downlaod Links', 'mapr-v4_spark-1.0.2.tgz'); return false;">direct file download (external)</a> +4. Download Spark: <span id="spanDownloadLink"></span> -Verify your download: [signatures and checksums](http://www.apache.org/dist/spark/spark-1.0.2/) +5. This release can be verfied using the <span id="sparkDownloadVerify"></span>. ### Link with Spark Spark artifacts are [hosted in Maven Central](http://search.maven.org/#browse%7C1686516968). You can add a Maven dependency with the following coordinates: @@ -39,31 +40,16 @@ Spark artifacts are [hosted in Maven Central](http://search.maven.org/#browse%7C artifactId: spark-core_2.10 version: 1.0.2 -### Development Version -If you are interested in working with the newest under-development code or contributing to Spark development, you can also check out the master branch from Git: <tt>git clone git://github.com/apache/spark.git</tt>. +### Development and Maintenance Branches +If you are interested in working with the newest under-development code or contributing to Spark development, you can also check out the master branch from Git: + + # Master development branch + git clone git://github.com/apache/spark.git + + # 1.1 maintenance branch with stability fixes on top of Spark 1.1.0 + git clone git://github.com/apache/spark.git -b branch-1.1 Once you've downloaded Spark, you can find instructions for installing and building it on the <a href="{{site.url}}documentation.html">documentation page</a>. <h3 id="all-releases">All Releases</h3> -* [Spark 1.0.2](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2.tgz) (August 5, 2014) [(release notes)]({{site.url}}releases/spark-release-1-0-2.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop2.tgz), third-party: [MapRv3](http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr3.tgz), [MapRv4](http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr4.tgz)) -* [Spark 1.0.1](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1.tgz) (July 11, 2014) [(release notes)]({{site.url}}releases/spark-release-1-0-1.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1-bin-hadoop2.tgz), third-party: [MapRv3](http://package.mapr.com/tools/apache-spark/1.0.1/spark-1.0.1-bin-mapr3.tgz), [MapRv4](http://package.mapr.com/tools/apache-spark/1.0.1/spark-1.0.1-bin-mapr3.tgz)) -* [Spark 1.0.0](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0.tgz) (May 30, 2014) [(release notes)]({{site.url}}releases/spark-release-1-0-0.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0-bin-hadoop2.tgz)) -* [Spark 0.9.2](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2.tgz) (Jul 23, 2014) [(release notes)]({{site.url}}releases/spark-release-0-9-2.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2-bin-hadoop2.tgz)) -* [Spark 0.9.1](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1.tgz) (Apr 9, 2014) [(release notes)]({{site.url}}releases/spark-release-0-9-1.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1-bin-hadoop2.tgz)) -* [Spark 0.9.0](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating.tgz) (Feb 2, 2014) [(release notes)]({{site.url}}releases/spark-release-0-9-0.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating-bin-hadoop2.tgz)) -* [Spark 0.8.1](http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating.tgz) (Dec 19, 2013) [(release notes)]({{site.url}}releases/spark-release-0-8-1.html) (prebuilt: [Hadoop1 [HDP1, CDH3]](http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating-bin-hadoop1.tgz), [CDH4](http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating-bin-cdh4.tgz), [Hadoop2 [HDP2, CDH5]](http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating-bin-hadoop2.tgz)) -* [Spark 0.8.0](http://d3kbcqa49mib13.cloudfront.net/spark-0.8.0-incubating.tgz) (Sept 25, 2013) [(release notes)]({{site.url}}releases/spark-release-0-8-0.html) (prebuilt: [Hadoop 1 / CDH3](http://spark-project.org/download/spark-0.8.0-incubating-bin-hadoop1.tgz), [CDH4](http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz)) -* [Spark 0.7.3](http://spark-project.org/download/spark-0.7.3-sources.tgz) (July 16, 2013) [(release notes)]({{site.url}}releases/spark-release-0-7-3.html) (prebuilt: -[Hadoop1 / CDH3](http://spark-project.org/download/spark-0.7.3-prebuilt-hadoop1.tgz), [CDH4](http://spark-project.org/download/spark-0.7.3-prebuilt-cdh4.tgz)) -* [Spark 0.7.2](http://spark-project.org/download/spark-0.7.2-sources.tgz) (June 2, 2013) [(release notes)]({{site.url}}releases/spark-release-0-7-2.html) (prebuilt: -[Hadoop1 / CDH3](http://spark-project.org/download/spark-0.7.2-prebuilt-hadoop1.tgz), [CDH4](http://spark-project.org/download/spark-0.7.2-prebuilt-cdh4.tgz)) -* [Spark 0.7.0](http://spark-project.org/download/spark-0.7.0-sources.tgz) (February 27, 2013) [(release notes)]({{site.url}}releases/spark-release-0-7-0.html) -* [Spark 0.6.2](http://spark-project.org/download/spark-0.6.2-sources.tgz) (February 7, 2013) [(release notes)]({{site.url}}releases/spark-release-0-6-2.html) -* [Spark 0.6.1](https://github.com/downloads/mesos/spark/spark-0.6.1-sources.tgz) (November 22, 2012) [(release notes)]({{site.url}}releases/spark-release-0-6-1.html) -* [Spark 0.5.2](https://github.com/downloads/mesos/spark/spark-0.5.2-sources.tgz) (November 22, 2012) [(release notes)]({{site.url}}releases/spark-release-0-5-2.html) -* [Spark 0.6.0](http://github.com/downloads/mesos/spark/spark-0.6.0-sources.tar.gz) (October 15, 2012) [(release notes)]({{site.url}}releases/spark-release-0-6-0.html) -* [Spark 0.5.1](http://github.com/downloads/mesos/spark/spark-0.5.1.tgz) (October 11, 2012) [(release notes)]({{site.url}}releases/spark-release-0-5-1.html) -* [Spark 0.5.0](https://github.com/mesos/spark/tarball/v0.5.0) (June 12, 2012) [(release notes)]({{site.url}}releases/spark-release-0-5-0.html) -* [Spark 0.3 for Scala 2.9](https://github.com/mesos/spark/tarball/0.3-scala-2.9) (July 14, 2011) [(release notes)]({{site.url}}releases/spark-release-0-3.html) -* [Spark 0.3 for Scala 2.8](https://github.com/mesos/spark/tarball/0.3-scala-2.8) (July 14, 2011) [(release notes)]({{site.url}}releases/spark-release-0-3.html) -* [Spark 0.2](https://github.com/mesos/spark/tarball/alpha-0.2) (May 24, 2011) +<ul id="sparkReleaseNotes"> diff --git a/js/downloads.js b/js/downloads.js new file mode 100644 index 000000000..4431da591 --- /dev/null +++ b/js/downloads.js @@ -0,0 +1,148 @@ +// Script for generating Spark download links +// No dependencies; pure javascript. + +releases = {}; + +function addRelease(version, releaseDate, packages, downloadable) { + releases[version] = {released: releaseDate, packages: packages, downloadable: downloadable}; +} + +var sources = ["sources"]; +var packagesV1 = sources.concat(["hadoop1", "cdh4"]); +var packagesV2 = packagesV1.concat(["hadoop2"]); +var packagesV3 = packagesV2.concat(["mapr3", "mapr4"]); + +addRelease("1.0.2", new Date(2014, 8, 5), packagesV3, true); +addRelease("1.0.1", new Date(2014, 7, 11), packagesV3); +addRelease("1.0.0", new Date(2014, 5, 30), packagesV2); +addRelease("0.9.2", new Date(2014, 7, 23), packagesV2, true); +addRelease("0.9.1", new Date(2014, 4, 9), packagesV2); +addRelease("0.9.0-incubating", new Date(2014, 2, 2), packagesV2); +addRelease("0.8.1-incubating", new Date(2013, 12, 19), packagesV2, true); +addRelease("0.8.0-incubating", new Date(2013, 9, 25), packagesV1, true); +addRelease("0.7.3", new Date(2013, 7, 16), packagesV1, true); +addRelease("0.7.2", new Date(2013, 6, 2), packagesV1); +addRelease("0.7.0", new Date(2013, 2, 27), sources); + +function append(el, contents) { + el.innerHTML = el.innerHTML + contents; +}; + +function empty(el) { + el.innerHTML = ""; +} + +function getSelectedValue(selectEl) { + return selectEl.options[selectEl.selectedIndex].value; +} + +function versionShort(version) { return version.replace(/-incubating/, ""); } + +function initDownloads() { + var versionSelect = document.getElementById("sparkVersionSelect"); + + // Populate versions + for (var version in releases) { + var releaseDate = releases[version].released; + var downloadable = releases[version].downloadable; + + if (!downloadable) { continue; } + + // Don't display incubation status here + var title = versionShort(version) + " (" + releaseDate.toDateString().slice(4) + ")"; + append(versionSelect, "<option value=\"" + version + "\">" + title + "</option>"); + } + + // Populate packages and (transitively) releases + onVersionSelect(); +} + +function initReleaseNotes() { + var releaseNotes = document.getElementById("sparkReleaseNotes"); + for (var version in releases) { + var releaseDate = releases[version].released; + var verShort = versionShort(version); + var contents = "<li><a href='releases/spark-release-$verUrl.html'>Spark $ver</a> ($date)</li>" + .replace(/\$verUrl/, verShort.replace(/\./g, "-")) + .replace(/\$ver/, verShort) + .replace(/\$date/, releaseDate.toDateString().slice(4)) + append(releaseNotes, contents); + } +} + +function onVersionSelect() { + var versionSelect = document.getElementById("sparkVersionSelect"); + var packageSelect = document.getElementById("sparkPackageSelect"); + var verifyLink = document.getElementById("sparkDownloadVerify"); + + empty(packageSelect); + empty(verifyLink); + + var version = getSelectedValue(versionSelect); + var packages = releases[version]["packages"]; + for (var idx in packages) { + append(packageSelect, "<option>" + packages[idx] + "</option>") + } + + var href = "http://www.apache.org/dist/spark/spark-" + version + "/"; + var link = "<a href=\"" + href + "\">" + versionShort(version) + " signatures and checksums</a>" + append(verifyLink, link); + + // Populate releases + onPackageSelect(); + updateDownloadLink(); +} + +function onPackageSelect() { + var downloadSelect = document.getElementById("sparkDownloadSelect"); + var packageSelect = document.getElementById("sparkPackageSelect"); + + empty(downloadSelect); + + var pkg = getSelectedValue(packageSelect); + + if (pkg.toLowerCase().indexOf("mapr") > -1) { + var external = "External Download (MAY CONTAIN INCOMPATIBLE LICENSES)"; + append(downloadSelect, "<option value='external'>" + external + "</option>"); + } else { + append(downloadSelect, "<option value='direct'>Direct Download</option>") + append(downloadSelect, "<option value='apache'>Select Apache Mirror</option>") + } + updateDownloadLink(); +} + +function onDownloadSelect() { + updateDownloadLink(); +} + +function updateDownloadLink() { + var versionSelect = document.getElementById("sparkVersionSelect"); + var packageSelect = document.getElementById("sparkPackageSelect"); + var downloadSelect = document.getElementById("sparkDownloadSelect"); + var downloadLink = document.getElementById("spanDownloadLink"); + + empty(downloadLink); + + var version = getSelectedValue(versionSelect); + var pkg = getSelectedValue(packageSelect); + var download = getSelectedValue(downloadSelect); + + var link = "http://d3kbcqa49mib13.cloudfront.net/spark-$ver-bin-$pkg.tgz"; + if (version <= "0.7.3") { + link = "http://spark-project.org/download/spark-$ver-bin-$pkg.tgz"; + } + if (pkg.toLowerCase().indexOf("mapr") > -1) { + link = "http://package.mapr.com/tools/apache-spark/$ver/spark-$ver-bin-$pkg.tgz" + } else if (download == "apache") { + link = "http://www.apache.org/dyn/closer.cgi/spark/spark-$ver/spark-$ver-bin-$pkg.tgz"; + } + link = link.replace(/\$ver/g, version).replace(/\$pkg/g, pkg); + + // Sources are handled as a special case here, since they don't follow the bin-[package] syntax + link = link.replace(/-bin-sources/, ""); + + var text = link.split("/").reverse()[0]; + + var contents = "<a href=\"" + link + "\">" + text + "</a>"; + append(downloadLink, contents); +} diff --git a/site/community.html b/site/community.html index 6997eebf3..10ac8c0b6 100644 --- a/site/community.html +++ b/site/community.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -272,9 +277,5 @@ A wide range of contributors now develop the project (over 250 developers from 5 </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/documentation.html b/site/documentation.html index 7e04c6edc..6bd84f6d6 100644 --- a/site/documentation.html +++ b/site/documentation.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -341,9 +346,5 @@ The following papers have been published about Spark and related projects. </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/downloads.html b/site/downloads.html index 2e4db0086..2377386ba 100644 --- a/site/downloads.html +++ b/site/downloads.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -157,34 +162,39 @@ </div> <div class="col-md-9 col-md-pull-3"> - <h2 id="download-spark">Download Spark</h2> -<p>The latest release is Spark 1.0.2, released August 5, 2014 <a href="/releases/spark-release-1-0-2.html">(release notes)</a> <a href="https://git-wip-us.apache.org/repos/asf?p=spark.git;a=commit;h=8fb6f00e195fb258f3f70f04756e07c259a2351f">(git tag)</a></p> - -<p>Pre-built packages:</p> - -<ul> - <li>For Hadoop 1 (HDP1, CDH3): -<a href="http://www.apache.org/dyn/closer.cgi/spark/spark-1.0.2/spark-1.0.2-bin-hadoop1.tgz" onclick="trackOutboundLink(this, 'Release Download Links', 'apache_spark-1.0.1-bin-hadoop1.tgz'); return false;"> find an Apache mirror </a> -or -<a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop1.tgz" onclick="trackOutboundLink(this, 'Release Download Links', 'cloudfront_spark-1.0.2-bin-hadoop1.tgz'); return false;"> direct file download </a></li> - <li>For CDH4: -<a href="http://www.apache.org/dyn/closer.cgi/spark/spark-1.0.2/spark-1.0.2-bin-cdh4.tgz" onclick="trackOutboundLink(this, 'Release Download Links', 'apache_spark-1.0.1-bin-cdh4.tgz'); return false;"> find an Apache mirror </a> -or -<a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-cdh4.tgz" onclick="trackOutboundLink(this, 'Release Download Links', 'cloudfront_spark-1.0.2-bin-cdh4.tgz'); return false;"> direct file download </a></li> - <li>For Hadoop 2 (HDP2, CDH5): -<a href="http://www.apache.org/dyn/closer.cgi/spark/spark-1.0.2/spark-1.0.2-bin-hadoop2.tgz" onclick="trackOutboundLink(this, 'Release Download Links', 'apache_spark-1.0.2-bin-hadoop2.tgz'); return false;"> find an Apache mirror </a> -or -<a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop2.tgz" onclick="trackOutboundLink(this, 'Release Download Links', 'cloudfront_spark-1.0.2-bin-hadoop2.tgz'); return false;"> direct file download </a></li> -</ul> - -<p>Pre-built packages, third-party (NOTE: may include non ASF-compatible licenses):</p> - -<ul> - <li>For MapRv3: <a href="http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr3.tgz" onclick="trackOutboundLink(this, 'Release Downlaod Links', 'mapr-v3_spark-1.0.2.tgz'); return false;">direct file download (external)</a></li> - <li>For MapRv4: <a href="http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr4.tgz" onclick="trackOutboundLink(this, 'Release Downlaod Links', 'mapr-v4_spark-1.0.2.tgz'); return false;">direct file download (external)</a></li> -</ul> - -<p>Verify your download: <a href="http://www.apache.org/dist/spark/spark-1.0.2/">signatures and checksums</a></p> + <script type="text/javascript"> +$(document).ready(function() { + initDownloads(); + initReleaseNotes(); +}); +</script> + +<h2 id="download-spark">Download Spark</h2> + +<p>Select below to download a release of Spark. +The latest release of Spark is Spark 1.0.2 +<a href="/releases/spark-release-1-0-2.html">(release notes)</a>:</p> + +<ol> + <li> + <p>Chose a Spark release: + <select id="sparkVersionSelect" onchange="javascript:onVersionSelect();"></select><br /></p> + </li> + <li> + <p>Chose a package type: + <select id="sparkPackageSelect" onchange="javascript:onPackageSelect();"></select><br /></p> + </li> + <li> + <p>Chose a download type: + <select id="sparkDownloadSelect" onchange="javascript:onDownloadSelect()"></select><br /></p> + </li> + <li> + <p>Download Spark: <span id="spanDownloadLink"></span></p> + </li> + <li> + <p>This release can be verfied using the <span id="sparkDownloadVerify"></span>.</p> + </li> +</ol> <h3 id="link-with-spark">Link with Spark</h3> <p>Spark artifacts are <a href="http://search.maven.org/#browse%7C1686516968">hosted in Maven Central</a>. You can add a Maven dependency with the following coordinates:</p> @@ -194,35 +204,20 @@ artifactId: spark-core_2.10 version: 1.0.2 </code></pre> -<h3 id="development-version">Development Version</h3> -<p>If you are interested in working with the newest under-development code or contributing to Spark development, you can also check out the master branch from Git: <tt>git clone git://github.com/apache/spark.git</tt>.</p> +<h3 id="development-and-maintenance-branches">Development and Maintenance Branches</h3> +<p>If you are interested in working with the newest under-development code or contributing to Spark development, you can also check out the master branch from Git:</p> + +<pre><code># Master development branch +git clone git://github.com/apache/spark.git + +# 1.1 maintenance branch with stability fixes on top of Spark 1.1.0 +git clone git://github.com/apache/spark.git -b branch-1.1 +</code></pre> -<p>Once you’ve downloaded Spark, you can find instructions for installing and building it on the <a href="/documentation.html">documentation page</a>.</p> +<p>Once you’ve downloaded Spark, you can find instructions for installing and building it on the <a href="/documentation.html">documentation page</a>.</p> <h3 id="all-releases">All Releases</h3> -<ul> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2.tgz">Spark 1.0.2</a> (August 5, 2014) <a href="/releases/spark-release-1-0-2.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.2-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>, third-party: <a href="http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr3.tgz">MapRv3</a>, <a href="http://package.mapr.com/tools/apache-spark/1.0.2/spark-1.0.2-bin-mapr4.tgz">MapRv4</a>)</li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1.tgz">Spark 1.0.1</a> (July 11, 2014) <a href="/releases/spark-release-1-0-1.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.1-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>, third-party: <a href="http://package.mapr.com/tools/apache-spark/1.0.1/spark-1.0.1-bin-mapr3.tgz">MapRv3</a>, <a href="http://package.mapr.com/tools/apache-spark/1.0.1/spark-1.0.1-bin-mapr3.tgz">MapRv4</a>)</li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0.tgz">Spark 1.0.0</a> (May 30, 2014) <a href="/releases/spark-release-1-0-0.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-1.0.0-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>)</li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2.tgz">Spark 0.9.2</a> (Jul 23, 2014) <a href="/releases/spark-release-0-9-2.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.2-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>) </li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1.tgz">Spark 0.9.1</a> (Apr 9, 2014) <a href="/releases/spark-release-0-9-1.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.1-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>)</li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating.tgz">Spark 0.9.0</a> (Feb 2, 2014) <a href="/releases/spark-release-0-9-0.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.9.0-incubating-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>)</li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating.tgz">Spark 0.8.1</a> (Dec 19, 2013) <a href="/releases/spark-release-0-8-1.html">(release notes)</a> (prebuilt: <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating-bin-hadoop1.tgz">Hadoop1 [HDP1, CDH3]</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating-bin-cdh4.tgz">CDH4</a>, <a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.8.1-incubating-bin-hadoop2.tgz">Hadoop2 [HDP2, CDH5]</a>)</li> - <li><a href="http://d3kbcqa49mib13.cloudfront.net/spark-0.8.0-incubating.tgz">Spark 0.8.0</a> (Sept 25, 2013) <a href="/releases/spark-release-0-8-0.html">(release notes)</a> (prebuilt: <a href="http://spark-project.org/download/spark-0.8.0-incubating-bin-hadoop1.tgz">Hadoop 1 / CDH3</a>, <a href="http://spark-project.org/download/spark-0.8.0-incubating-bin-cdh4.tgz">CDH4</a>)</li> - <li><a href="http://spark-project.org/download/spark-0.7.3-sources.tgz">Spark 0.7.3</a> (July 16, 2013) <a href="/releases/spark-release-0-7-3.html">(release notes)</a> (prebuilt: -<a href="http://spark-project.org/download/spark-0.7.3-prebuilt-hadoop1.tgz">Hadoop1 / CDH3</a>, <a href="http://spark-project.org/download/spark-0.7.3-prebuilt-cdh4.tgz">CDH4</a>)</li> - <li><a href="http://spark-project.org/download/spark-0.7.2-sources.tgz">Spark 0.7.2</a> (June 2, 2013) <a href="/releases/spark-release-0-7-2.html">(release notes)</a> (prebuilt: -<a href="http://spark-project.org/download/spark-0.7.2-prebuilt-hadoop1.tgz">Hadoop1 / CDH3</a>, <a href="http://spark-project.org/download/spark-0.7.2-prebuilt-cdh4.tgz">CDH4</a>)</li> - <li><a href="http://spark-project.org/download/spark-0.7.0-sources.tgz">Spark 0.7.0</a> (February 27, 2013) <a href="/releases/spark-release-0-7-0.html">(release notes)</a></li> - <li><a href="http://spark-project.org/download/spark-0.6.2-sources.tgz">Spark 0.6.2</a> (February 7, 2013) <a href="/releases/spark-release-0-6-2.html">(release notes)</a></li> - <li><a href="https://github.com/downloads/mesos/spark/spark-0.6.1-sources.tgz">Spark 0.6.1</a> (November 22, 2012) <a href="/releases/spark-release-0-6-1.html">(release notes)</a></li> - <li><a href="https://github.com/downloads/mesos/spark/spark-0.5.2-sources.tgz">Spark 0.5.2</a> (November 22, 2012) <a href="/releases/spark-release-0-5-2.html">(release notes)</a></li> - <li><a href="http://github.com/downloads/mesos/spark/spark-0.6.0-sources.tar.gz">Spark 0.6.0</a> (October 15, 2012) <a href="/releases/spark-release-0-6-0.html">(release notes)</a></li> - <li><a href="http://github.com/downloads/mesos/spark/spark-0.5.1.tgz">Spark 0.5.1</a> (October 11, 2012) <a href="/releases/spark-release-0-5-1.html">(release notes)</a></li> - <li><a href="https://github.com/mesos/spark/tarball/v0.5.0">Spark 0.5.0</a> (June 12, 2012) <a href="/releases/spark-release-0-5-0.html">(release notes)</a></li> - <li><a href="https://github.com/mesos/spark/tarball/0.3-scala-2.9">Spark 0.3 for Scala 2.9</a> (July 14, 2011) <a href="/releases/spark-release-0-3.html">(release notes)</a></li> - <li><a href="https://github.com/mesos/spark/tarball/0.3-scala-2.8">Spark 0.3 for Scala 2.8</a> (July 14, 2011) <a href="/releases/spark-release-0-3.html">(release notes)</a></li> - <li><a href="https://github.com/mesos/spark/tarball/alpha-0.2">Spark 0.2</a> (May 24, 2011)</li> +<ul id="sparkReleaseNotes"> </ul> </div> @@ -238,9 +233,5 @@ version: 1.0.2 </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/examples.html b/site/examples.html index 21aed0bdc..6a1ec0468 100644 --- a/site/examples.html +++ b/site/examples.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -427,9 +432,5 @@ previous ones, and <em>actions</em>, which kick off a job to execute on a cluste </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/faq.html b/site/faq.html index 8ecab79ac..d21f0a7cf 100644 --- a/site/faq.html +++ b/site/faq.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -222,9 +227,5 @@ streaming, interactive queries, and machine learning. </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/graphx/index.html b/site/graphx/index.html index cb4561f59..b1d0a27ba 100644 --- a/site/graphx/index.html +++ b/site/graphx/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -285,9 +290,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/index.html b/site/index.html index 5d3ccddd9..93eb48c0f 100644 --- a/site/index.html +++ b/site/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -351,9 +356,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/js/downloads.js b/site/js/downloads.js new file mode 100644 index 000000000..4431da591 --- /dev/null +++ b/site/js/downloads.js @@ -0,0 +1,148 @@ +// Script for generating Spark download links +// No dependencies; pure javascript. + +releases = {}; + +function addRelease(version, releaseDate, packages, downloadable) { + releases[version] = {released: releaseDate, packages: packages, downloadable: downloadable}; +} + +var sources = ["sources"]; +var packagesV1 = sources.concat(["hadoop1", "cdh4"]); +var packagesV2 = packagesV1.concat(["hadoop2"]); +var packagesV3 = packagesV2.concat(["mapr3", "mapr4"]); + +addRelease("1.0.2", new Date(2014, 8, 5), packagesV3, true); +addRelease("1.0.1", new Date(2014, 7, 11), packagesV3); +addRelease("1.0.0", new Date(2014, 5, 30), packagesV2); +addRelease("0.9.2", new Date(2014, 7, 23), packagesV2, true); +addRelease("0.9.1", new Date(2014, 4, 9), packagesV2); +addRelease("0.9.0-incubating", new Date(2014, 2, 2), packagesV2); +addRelease("0.8.1-incubating", new Date(2013, 12, 19), packagesV2, true); +addRelease("0.8.0-incubating", new Date(2013, 9, 25), packagesV1, true); +addRelease("0.7.3", new Date(2013, 7, 16), packagesV1, true); +addRelease("0.7.2", new Date(2013, 6, 2), packagesV1); +addRelease("0.7.0", new Date(2013, 2, 27), sources); + +function append(el, contents) { + el.innerHTML = el.innerHTML + contents; +}; + +function empty(el) { + el.innerHTML = ""; +} + +function getSelectedValue(selectEl) { + return selectEl.options[selectEl.selectedIndex].value; +} + +function versionShort(version) { return version.replace(/-incubating/, ""); } + +function initDownloads() { + var versionSelect = document.getElementById("sparkVersionSelect"); + + // Populate versions + for (var version in releases) { + var releaseDate = releases[version].released; + var downloadable = releases[version].downloadable; + + if (!downloadable) { continue; } + + // Don't display incubation status here + var title = versionShort(version) + " (" + releaseDate.toDateString().slice(4) + ")"; + append(versionSelect, "<option value=\"" + version + "\">" + title + "</option>"); + } + + // Populate packages and (transitively) releases + onVersionSelect(); +} + +function initReleaseNotes() { + var releaseNotes = document.getElementById("sparkReleaseNotes"); + for (var version in releases) { + var releaseDate = releases[version].released; + var verShort = versionShort(version); + var contents = "<li><a href='releases/spark-release-$verUrl.html'>Spark $ver</a> ($date)</li>" + .replace(/\$verUrl/, verShort.replace(/\./g, "-")) + .replace(/\$ver/, verShort) + .replace(/\$date/, releaseDate.toDateString().slice(4)) + append(releaseNotes, contents); + } +} + +function onVersionSelect() { + var versionSelect = document.getElementById("sparkVersionSelect"); + var packageSelect = document.getElementById("sparkPackageSelect"); + var verifyLink = document.getElementById("sparkDownloadVerify"); + + empty(packageSelect); + empty(verifyLink); + + var version = getSelectedValue(versionSelect); + var packages = releases[version]["packages"]; + for (var idx in packages) { + append(packageSelect, "<option>" + packages[idx] + "</option>") + } + + var href = "http://www.apache.org/dist/spark/spark-" + version + "/"; + var link = "<a href=\"" + href + "\">" + versionShort(version) + " signatures and checksums</a>" + append(verifyLink, link); + + // Populate releases + onPackageSelect(); + updateDownloadLink(); +} + +function onPackageSelect() { + var downloadSelect = document.getElementById("sparkDownloadSelect"); + var packageSelect = document.getElementById("sparkPackageSelect"); + + empty(downloadSelect); + + var pkg = getSelectedValue(packageSelect); + + if (pkg.toLowerCase().indexOf("mapr") > -1) { + var external = "External Download (MAY CONTAIN INCOMPATIBLE LICENSES)"; + append(downloadSelect, "<option value='external'>" + external + "</option>"); + } else { + append(downloadSelect, "<option value='direct'>Direct Download</option>") + append(downloadSelect, "<option value='apache'>Select Apache Mirror</option>") + } + updateDownloadLink(); +} + +function onDownloadSelect() { + updateDownloadLink(); +} + +function updateDownloadLink() { + var versionSelect = document.getElementById("sparkVersionSelect"); + var packageSelect = document.getElementById("sparkPackageSelect"); + var downloadSelect = document.getElementById("sparkDownloadSelect"); + var downloadLink = document.getElementById("spanDownloadLink"); + + empty(downloadLink); + + var version = getSelectedValue(versionSelect); + var pkg = getSelectedValue(packageSelect); + var download = getSelectedValue(downloadSelect); + + var link = "http://d3kbcqa49mib13.cloudfront.net/spark-$ver-bin-$pkg.tgz"; + if (version <= "0.7.3") { + link = "http://spark-project.org/download/spark-$ver-bin-$pkg.tgz"; + } + if (pkg.toLowerCase().indexOf("mapr") > -1) { + link = "http://package.mapr.com/tools/apache-spark/$ver/spark-$ver-bin-$pkg.tgz" + } else if (download == "apache") { + link = "http://www.apache.org/dyn/closer.cgi/spark/spark-$ver/spark-$ver-bin-$pkg.tgz"; + } + link = link.replace(/\$ver/g, version).replace(/\$pkg/g, pkg); + + // Sources are handled as a special case here, since they don't follow the bin-[package] syntax + link = link.replace(/-bin-sources/, ""); + + var text = link.split("/").reverse()[0]; + + var contents = "<a href=\"" + link + "\">" + text + "</a>"; + append(downloadLink, contents); +} diff --git a/site/mailing-lists.html b/site/mailing-lists.html index c46d08a46..dfb791a24 100644 --- a/site/mailing-lists.html +++ b/site/mailing-lists.html @@ -51,6 +51,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -175,9 +180,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/mllib/index.html b/site/mllib/index.html index 6da0205d6..4ac0dde7a 100644 --- a/site/mllib/index.html +++ b/site/mllib/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -186,9 +191,9 @@ points = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> .<span class="sparkop">map</span>(<span class="closure">parsePoint</span>)<br /> <br /> - model = KMeans.<span class="sparkop">train</span>(points, k=10) + model = KMeans.<span class="sparkop">train</span>(points) </div> - <div class="caption">Calling MLlib in Python</div> + <div class="caption">Calling MLlib in Scala</div> </div> </div> </div> @@ -242,18 +247,16 @@ <div class="col-md-4 col-padded"> <h3>Algorithms</h3> <p> - MLlib 1.1 contains the following algorithms: + MLlib 0.9 contains the following algorithms: </p> <ul class="list-narrow"> - <li>linear SVM and logistic regression</li> - <li>classification and regression tree</li> - <li>k-means clustering</li> - <li>recommendation via alternating least squares</li> - <li>singular value decomposition</li> - <li>linear regression with L<sub>1</sub>- and L<sub>2</sub>-regularization</li> - <li>multinomial naive Bayes</li> - <li>basic statistics</li> - <li>feature transformations</li> + <li>K-means clustering with <a href="http://theory.stanford.edu/~sergei/papers/vldb12-kmpar.pdf">K-means|| initialization</a>.</li> + <li>L<sub>1</sub>- and L<sub>2</sub>-regularized <a href="http://en.wikipedia.org/wiki/Linear_regression">linear regression</a>.</li> + <li>L<sub>1</sub>- and L<sub>2</sub>-regularized <a href="http://en.wikipedia.org/wiki/Logistic_regression">logistic regression</a>.</li> + <li><a href="http://www.hpl.hp.com/personal/Robert_Schreiber/papers/2008%20AAIM%20Netflix/netflix_aaim08(submitted).pdf">Alternating least squares</a> collaborative filtering, with explicit + ratings or <a href="http://www2.research.att.com/~yifanhu/PUB/cf.pdf">implicit feedback</a>.</li> + <li><a href="http://en.wikipedia.org/wiki/Naive_Bayes_classifier">Naive Bayes</a> multinomial classification.</li> + <li>Stochastic gradient descent.</li> </ul> <p>Refer to the <a href="/docs/latest/mllib-guide.html">MLlib guide</a> for usage examples.</p> </div> @@ -311,9 +314,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/amp-camp-2013-registration-ope.html b/site/news/amp-camp-2013-registration-ope.html index e4b584365..0ac5bf0e0 100644 --- a/site/news/amp-camp-2013-registration-ope.html +++ b/site/news/amp-camp-2013-registration-ope.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Registration open for AMP Camp training camp in Berkeley</h2> -<p>Want to learn how to use Spark, Shark, GraphX, and related technologies in person? The AMP Lab is hosting a two-day training workshop for them on August 29th and 30th in Berkeley. The workshop will include tutorials, talks from users, and over four hours of hands-on exercises. <a href="http://ampcamp.berkeley.edu/amp-camp-three-berkeley-2013/">Registration is now open on the AMP Camp website</a>, for a price of $250 per person. We recommend signing up early because last year’s workshop was sold out.</p> +<p>Want to learn how to use Spark, Shark, GraphX, and related technologies in person? The AMP Lab is hosting a two-day training workshop for them on August 29th and 30th in Berkeley. The workshop will include tutorials, talks from users, and over four hours of hands-on exercises. <a href="http://ampcamp.berkeley.edu/amp-camp-three-berkeley-2013/">Registration is now open on the AMP Camp website</a>, for a price of $250 per person. We recommend signing up early because last year’s workshop was sold out.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/announcing-the-first-spark-summit.html b/site/news/announcing-the-first-spark-summit.html index 8a9112749..e63239dd1 100644 --- a/site/news/announcing-the-first-spark-summit.html +++ b/site/news/announcing-the-first-spark-summit.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -185,9 +190,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/fourth-spark-screencast-published.html b/site/news/fourth-spark-screencast-published.html index 3b85019f3..8f2c8d35d 100644 --- a/site/news/fourth-spark-screencast-published.html +++ b/site/news/fourth-spark-screencast-published.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -185,9 +190,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/index.html b/site/news/index.html index 5fe8d8515..7e5492361 100644 --- a/site/news/index.html +++ b/site/news/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -196,7 +201,7 @@ Contributions to this release came from 28 developers. </p> <h3 class="entry-title"><a href="/news/spark-1-0-1-released.html">Spark 1.0.1 released</a></h3> <div class="entry-date">July 11, 2014</div> </header> - <div class="entry-content"><p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-1.html" title="Spark Release 1.0.1">Spark 1.0.1</a>! This release includes contributions from 70 developers. Spark 1.0.0 includes fixes across several areas of Spark, including the core API, PySpark, and MLlib. It also includes new features in Spark’s (alpha) SQL library, including support for JSON data and performance and stability fixes.</p> + <div class="entry-content"><p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-1.html" title="Spark Release 1.0.1">Spark 1.0.1</a>! This release includes contributions from 70 developers. Spark 1.0.0 includes fixes across several areas of Spark, including the core API, PySpark, and MLlib. It also includes new features in Spark’s (alpha) SQL library, including support for JSON data and performance and stability fixes.</p> </div> </article> @@ -219,8 +224,8 @@ organizations using Spark, focused on use cases and ongoing development.</p> <h3 class="entry-title"><a href="/news/spark-1-0-0-released.html">Spark 1.0.0 released</a></h3> <div class="entry-date">May 30, 2014</div> </header> - <div class="entry-content"><p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-0.html" title="Spark Release 1.0.0">Spark 1.0.0</a>! Spark 1.0.0 is the first in the 1.X line of releases, providing API stability for Spark’s core interfaces. It is Spark’s largest release ever, with contributions from 117 developers. -This release expands Spark’s standard libraries, introducing a new SQL package (Spark SQL) that lets users integrate SQL queries into existing Spark workflows. MLlib, Spark’s machine learning library, is expanded with sparse vector support and several new algorithms. The GraphX and Streaming libraries also introduce new features and optimizations. Spark’s core engine adds support for secured YARN clusters, a unified tool for submitting Spark applications, and several performance and stability improvements.</p> + <div class="entry-content"><p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-0.html" title="Spark Release 1.0.0">Spark 1.0.0</a>! Spark 1.0.0 is the first in the 1.X line of releases, providing API stability for Spark’s core interfaces. It is Spark’s largest release ever, with contributions from 117 developers. +This release expands Spark’s standard libraries, introducing a new SQL package (Spark SQL) that lets users integrate SQL queries into existing Spark workflows. MLlib, Spark’s machine learning library, is expanded with sparse vector support and several new algorithms. The GraphX and Streaming libraries also introduce new features and optimizations. Spark’s core engine adds support for secured YARN clusters, a unified tool for submitting Spark applications, and several performance and stability improvements.</p> </div> </article> @@ -257,7 +262,7 @@ Contributions to this release came from 37 developers. </p> <h3 class="entry-title"><a href="/news/submit-talks-to-spark-summit-2014.html">Submissions and registration open for Spark Summit 2014</a></h3> <div class="entry-date">March 20, 2014</div> </header> - <div class="entry-content"><p>After last year’s successful <a href="http://spark-summit.org/2013">first Spark Summit</a>, registrations + <div class="entry-content"><p>After last year’s successful <a href="http://spark-summit.org/2013">first Spark Summit</a>, registrations and talk submissions are now open for <a href="http://spark-summit.org/2014">Spark Summit 2014</a>. This will be a 3-day event in San Francisco organized by multiple companies in the Spark community. The event will run <strong>June 30th to July 2nd</strong> in San Francisco, CA.</p> @@ -270,7 +275,7 @@ The event will run <strong>June 30th to July 2nd</strong> in San Francisco, CA.< <h3 class="entry-title"><a href="/news/spark-becomes-tlp.html">Spark becomes top-level Apache project</a></h3> <div class="entry-date">February 27, 2014</div> </header> - <div class="entry-content"><p>The Apache Software Foundation <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">announced</a> today that Spark has graduated from the Apache Incubator to become a top-level Apache project, signifying that the project’s community and products have been well-governed under the ASF’s meritocratic process and principles. This is a major step for the community and we are very proud to share this news with users as we complete Spark’s move to Apache. Read more about Spark’s growth during the past year and from contributors and users in the ASF’s <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">press release</a>.</p> + <div class="entry-content"><p>The Apache Software Foundation <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">announced</a> today that Spark has graduated from the Apache Incubator to become a top-level Apache project, signifying that the project’s community and products have been well-governed under the ASF’s meritocratic process and principles. This is a major step for the community and we are very proud to share this news with users as we complete Spark’s move to Apache. Read more about Spark’s growth during the past year and from contributors and users in the ASF’s <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">press release</a>.</p> </div> </article> @@ -281,8 +286,8 @@ The event will run <strong>June 30th to July 2nd</strong> in San Francisco, CA.< <div class="entry-date">February 2, 2014</div> </header> <div class="entry-content"><p>We are happy to announce the availability of <a href="/releases/spark-release-0-9-0.html" title="Spark Release 0.9.0"> -Spark 0.9.0</a>! Spark 0.9.0 is a major release and Spark’s largest release ever, with contributions from 83 developers. -This release expands Spark’s standard libraries, introducing a new graph computation package (GraphX) and adding several new features to the machine learning and stream-processing packages. It also makes major improvements to the core engine, +Spark 0.9.0</a>! Spark 0.9.0 is a major release and Spark’s largest release ever, with contributions from 83 developers. +This release expands Spark’s standard libraries, introducing a new graph computation package (GraphX) and adding several new features to the machine learning and stream-processing packages. It also makes major improvements to the core engine, including external aggregations, a simplified H/A mode for long lived applications, and hardened YARN support.</p> @@ -294,7 +299,7 @@ hardened YARN support.</p> <h3 class="entry-title"><a href="/news/spark-0-8-1-released.html">Spark 0.8.1 released</a></h3> <div class="entry-date">December 19, 2013</div> </header> - <div class="entry-content"><p>We’ve just posted <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">Spark Release 0.8.1</a>, a maintenance and performance release for the Scala 2.9 version of Spark. 0.8.1 includes support for YARN 2.2, a high availability mode for the standalone scheduler, optimizations to the shuffle, and many other improvements. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> + <div class="entry-content"><p>We’ve just posted <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">Spark Release 0.8.1</a>, a maintenance and performance release for the Scala 2.9 version of Spark. 0.8.1 includes support for YARN 2.2, a high availability mode for the standalone scheduler, optimizations to the shuffle, and many other improvements. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> </div> </article> @@ -325,7 +330,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-0-8-0-released.html">Spark 0.8.0 released</a></h3> <div class="entry-date">September 25, 2013</div> </header> - <div class="entry-content"><p>We’re proud to announce the release of <a href="/releases/spark-release-0-8-0.html" title="Spark Release 0.8.0">Apache Spark 0.8.0</a>. Spark 0.8.0 is a major release that includes many new capabilities and usability improvements. It’s also our first release under the Apache incubator. It is the largest Spark release yet, with contributions from 67 developers and 24 companies. Major new features include an expanded monitoring framework and UI, a machine learning library, and support for running Spark inside of YARN.</p> + <div class="entry-content"><p>We’re proud to announce the release of <a href="/releases/spark-release-0-8-0.html" title="Spark Release 0.8.0">Apache Spark 0.8.0</a>. Spark 0.8.0 is a major release that includes many new capabilities and usability improvements. It’s also our first release under the Apache incubator. It is the largest Spark release yet, with contributions from 67 developers and 24 companies. Major new features include an expanded monitoring framework and UI, a machine learning library, and support for running Spark inside of YARN.</p> </div> </article> @@ -335,7 +340,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-user-survey-and-powered-by-page.html">Spark user survey and "Powered By" page</a></h3> <div class="entry-date">September 5, 2013</div> </header> - <div class="entry-content"><p>As we continue developing Spark, we would love to get feedback from users and hear what you’d like us to work on next. We’ve decided that a good way to do that is a survey – we hope to run this at regular intervals. If you have a few minutes to participate, <a href="https://docs.google.com/forms/d/1eMXp4GjcIXglxJe5vYYBzXKVm-6AiYt1KThJwhCjJiY/viewform">fill in the survey here</a>. Your time is greatly appreciated.</p> + <div class="entry-content"><p>As we continue developing Spark, we would love to get feedback from users and hear what you’d like us to work on next. We’ve decided that a good way to do that is a survey – we hope to run this at regular intervals. If you have a few minutes to participate, <a href="https://docs.google.com/forms/d/1eMXp4GjcIXglxJe5vYYBzXKVm-6AiYt1KThJwhCjJiY/viewform">fill in the survey here</a>. Your time is greatly appreciated.</p> </div> </article> @@ -355,7 +360,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/amp-camp-2013-registration-ope.html">Registration open for AMP Camp training camp in Berkeley</a></h3> <div class="entry-date">July 23, 2013</div> </header> - <div class="entry-content"><p>Want to learn how to use Spark, Shark, GraphX, and related technologies in person? The AMP Lab is hosting a two-day training workshop for them on August 29th and 30th in Berkeley. The workshop will include tutorials, talks from users, and over four hours of hands-on exercises. <a href="http://ampcamp.berkeley.edu/amp-camp-three-berkeley-2013/">Registration is now open on the AMP Camp website</a>, for a price of $250 per person. We recommend signing up early because last year’s workshop was sold out.</p> + <div class="entry-content"><p>Want to learn how to use Spark, Shark, GraphX, and related technologies in person? The AMP Lab is hosting a two-day training workshop for them on August 29th and 30th in Berkeley. The workshop will include tutorials, talks from users, and over four hours of hands-on exercises. <a href="http://ampcamp.berkeley.edu/amp-camp-three-berkeley-2013/">Registration is now open on the AMP Camp website</a>, for a price of $250 per person. We recommend signing up early because last year’s workshop was sold out.</p> </div> </article> @@ -386,7 +391,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-0-7-3-released.html">Spark 0.7.3 released</a></h3> <div class="entry-date">July 16, 2013</div> </header> - <div class="entry-content"><p>We’ve just posted <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">Spark Release 0.7.3</a>, a maintenance release that contains several fixes, including streaming API updates and new functionality for adding JARs to a <code>spark-shell</code> session. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> + <div class="entry-content"><p>We’ve just posted <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">Spark Release 0.7.3</a>, a maintenance release that contains several fixes, including streaming API updates and new functionality for adding JARs to a <code>spark-shell</code> session. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> </div> </article> @@ -416,7 +421,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-0-7-2-released.html">Spark 0.7.2 released</a></h3> <div class="entry-date">June 2, 2013</div> </header> - <div class="entry-content"><p>We’re happy to announce the release of <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">Spark 0.7.2</a>, a new maintenance release that includes several bug fixes and improvements, as well as new code examples and API features. We recommend that all users update to this release. Head over to the <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> + <div class="entry-content"><p>We’re happy to announce the release of <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">Spark 0.7.2</a>, a new maintenance release that includes several bug fixes and improvements, as well as new code examples and API features. We recommend that all users update to this release. Head over to the <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> </div> </article> @@ -442,7 +447,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/strata-exercises-now-available-online.html">Strata exercises now available online</a></h3> <div class="entry-date">March 17, 2013</div> </header> - <div class="entry-content"><p>At this year’s <a href="http://strataconf.com/strata2013">Strata</a> conference, the AMP Lab hosted a full day of tutorials on Spark, Shark, and Spark Streaming, including online exercises on Amazon EC2. Those exercises are now <a href="http://ampcamp.berkeley.edu/big-data-mini-course/">available online</a>, letting you learn Spark and Shark at your own pace on an EC2 cluster with real data. They are a great resource for learning the systems. You can also find <a href="http://ampcamp.berkeley.edu/amp-camp-two-strata-2013/">slides</a> from the Strata tutorials online, as well as <a href="http://ampcamp.berkeley.edu/amp-camp-one-berkeley-2012/">videos</a> from the AMP Camp workshop we held at Berkeley in August.</p> + <div class="entry-content"><p>At this year’s <a href="http://strataconf.com/strata2013">Strata</a> conference, the AMP Lab hosted a full day of tutorials on Spark, Shark, and Spark Streaming, including online exercises on Amazon EC2. Those exercises are now <a href="http://ampcamp.berkeley.edu/big-data-mini-course/">available online</a>, letting you learn Spark and Shark at your own pace on an EC2 cluster with real data. They are a great resource for learning the systems. You can also find <a href="http://ampcamp.berkeley.edu/amp-camp-two-strata-2013/">slides</a> from the Strata tutorials online, as well as <a href="http://ampcamp.berkeley.edu/amp-camp-one-berkeley-2012/">videos</a> from the AMP Camp workshop we held at Berkeley in August.</p> </div> </article> @@ -452,7 +457,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-0-7-0-released.html">Spark 0.7.0 released</a></h3> <div class="entry-date">February 27, 2013</div> </header> - <div class="entry-content"><p>We’re proud to announce the release of <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">Spark 0.7.0</a>, a new major version of Spark that adds several key features, including a <a href="/docs/latest/python-programming-guide.html">Python API</a> for Spark and an <a href="/docs/latest/streaming-programming-guide.html">alpha of Spark Streaming</a>. This release is the result of the largest group of contributors yet behind a Spark release – 31 contributors from inside and outside Berkeley. Head over to the <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">release notes</a> to read more about the new features, or <a href="/downloads.html">download</a> the release today.</p> + <div class="entry-content"><p>We’re proud to announce the release of <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">Spark 0.7.0</a>, a new major version of Spark that adds several key features, including a <a href="/docs/latest/python-programming-guide.html">Python API</a> for Spark and an <a href="/docs/latest/streaming-programming-guide.html">alpha of Spark Streaming</a>. This release is the result of the largest group of contributors yet behind a Spark release – 31 contributors from inside and outside Berkeley. Head over to the <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">release notes</a> to read more about the new features, or <a href="/downloads.html">download</a> the release today.</p> </div> </article> @@ -462,7 +467,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/run-spark-and-shark-on-amazon-emr.html">Spark/Shark Tutorial for Amazon EMR</a></h3> <div class="entry-date">February 24, 2013</div> </header> - <div class="entry-content"><p>This weekend, Amazon posted an <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">article</a> and code that make it easy to launch Spark and Shark on Elastic MapReduce. The article includes examples of how to run both interactive Scala commands and SQL queries from Shark on data in S3. Head over to the <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">Amazon article</a> for details. We’re very excited because, to our knowledge, this makes Spark the first non-Hadoop engine that you can launch with EMR.</p> + <div class="entry-content"><p>This weekend, Amazon posted an <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">article</a> and code that make it easy to launch Spark and Shark on Elastic MapReduce. The article includes examples of how to run both interactive Scala commands and SQL queries from Shark on data in S3. Head over to the <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">Amazon article</a> for details. We’re very excited because, to our knowledge, this makes Spark the first non-Hadoop engine that you can launch with EMR.</p> </div> </article> @@ -497,7 +502,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/video-from-first-spark-development-meetup.html">Video up from first Spark development meetup</a></h3> <div class="entry-date">December 21, 2012</div> </header> - <div class="entry-content"><p>On December 18th, we held the first of a series of Spark development meetups, for people interested in learning the Spark codebase and contributing to the project. There was quite a bit more demand than we anticipated, with over 80 people signing up and 64 attending. The first meetup was an <a href="http://www.meetup.com/spark-users/events/94101942/">introduction to Spark internals</a>. Thanks to one of the attendees, there’s now a <a href="http://www.youtube.com/watch?v=49Hr5xZyTEA">video of the meetup</a> on YouTube. We’ve also posted the <a href="http://files.meetup.com/3138542/dev-meetup-dec-2012.pptx">slides</a>. Look to see more development meetups on Spark and Shark in the future.</p> + <div class="entry-content"><p>On December 18th, we held the first of a series of Spark development meetups, for people interested in learning the Spark codebase and contributing to the project. There was quite a bit more demand than we anticipated, with over 80 people signing up and 64 attending. The first meetup was an <a href="http://www.meetup.com/spark-users/events/94101942/">introduction to Spark internals</a>. Thanks to one of the attendees, there’s now a <a href="http://www.youtube.com/watch?v=49Hr5xZyTEA">video of the meetup</a> on YouTube. We’ve also posted the <a href="http://files.meetup.com/3138542/dev-meetup-dec-2012.pptx">slides</a>. Look to see more development meetups on Spark and Shark in the future.</p> </div> </article> @@ -507,7 +512,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-and-shark-in-the-news.html">Spark in the news</a></h3> <div class="entry-date">December 21, 2012</div> </header> - <div class="entry-content"><p>Recently, we’ve seen quite a bit of coverage of Spark in the news. I wanted to list some of the more recent articles, for readers interested in learning more.</p> + <div class="entry-content"><p>Recently, we’ve seen quite a bit of coverage of Spark in the news. I wanted to list some of the more recent articles, for readers interested in learning more.</p> <ul> <li>Curt Monash, editor of the popular DBMS2 blog, wrote a great <a href="http://www.dbms2.com/2012/12/13/introduction-to-spark-shark-bdas-and-amplab/">introduction to Spark and Shark</a>, as well as a more detailed <a href="http://www.dbms2.com/2012/12/13/spark-shark-and-rdds-technology-notes/">technical overview</a>.</li> @@ -517,7 +522,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <li><a href="http://data-informed.com/spark-an-open-source-engine-for-iterative-data-mining/">DataInformed</a> interviewed two Spark users and wrote about their applications in anomaly detection, predictive analytics and data mining.</li> </ul> -<p>In other news, there will be a full day of tutorials on Spark and Shark at the <a href="http://strataconf.com/strata2013">O’Reilly Strata conference</a> in February. They include a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27438">introduction to Spark, Shark and BDAS</a> Tuesday morning, and a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27440">hands-on exercise session</a>. </p> +<p>In other news, there will be a full day of tutorials on Spark and Shark at the <a href="http://strataconf.com/strata2013">O’Reilly Strata conference</a> in February. They include a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27438">introduction to Spark, Shark and BDAS</a> Tuesday morning, and a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27440">hands-on exercise session</a>. </p> </div> </article> @@ -527,7 +532,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-0-6-1-and-0-5-2-released.html">Spark 0.6.1 and 0.5.2 out</a></h3> <div class="entry-date">November 22, 2012</div> </header> - <div class="entry-content"><p>Today we’ve made available two maintenance releases for Spark: <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a> and <a href="/releases/spark-release-0-5-2.html" title="Spark Release 0.5.2">0.5.2</a>. They both contain important bug fixes as well as some new features, such as the ability to build against Hadoop 2 distributions. We recommend that users update to the latest version for their branch; for new users, we recommend <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a>.</p> + <div class="entry-content"><p>Today we’ve made available two maintenance releases for Spark: <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a> and <a href="/releases/spark-release-0-5-2.html" title="Spark Release 0.5.2">0.5.2</a>. They both contain important bug fixes as well as some new features, such as the ability to build against Hadoop 2 distributions. We recommend that users update to the latest version for their branch; for new users, we recommend <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a>.</p> </div> </article> @@ -557,7 +562,7 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co <h3 class="entry-title"><a href="/news/spark-meetups.html">We've started hosting a Bay Area Spark User Meetup</a></h3> <div class="entry-date">January 10, 2012</div> </header> - <div class="entry-content"><p>We’ve started hosting a regular <a href="http://www.meetup.com/spark-users/">Bay Area Spark User Meetup</a>. Sign up on the meetup.com page to be notified about events and meet other Spark developers and users.</p> + <div class="entry-content"><p>We’ve started hosting a regular <a href="http://www.meetup.com/spark-users/">Bay Area Spark User Meetup</a>. Sign up on the meetup.com page to be notified about events and meet other Spark developers and users.</p> </div> </article> @@ -576,9 +581,5 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/nsdi-paper.html b/site/news/nsdi-paper.html index 3e693b4da..f4826281d 100644 --- a/site/news/nsdi-paper.html +++ b/site/news/nsdi-paper.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/run-spark-and-shark-on-amazon-emr.html b/site/news/run-spark-and-shark-on-amazon-emr.html index 2911934a0..5709b4b08 100644 --- a/site/news/run-spark-and-shark-on-amazon-emr.html +++ b/site/news/run-spark-and-shark-on-amazon-emr.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark/Shark Tutorial for Amazon EMR</h2> -<p>This weekend, Amazon posted an <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">article</a> and code that make it easy to launch Spark and Shark on Elastic MapReduce. The article includes examples of how to run both interactive Scala commands and SQL queries from Shark on data in S3. Head over to the <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">Amazon article</a> for details. We’re very excited because, to our knowledge, this makes Spark the first non-Hadoop engine that you can launch with EMR.</p> +<p>This weekend, Amazon posted an <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">article</a> and code that make it easy to launch Spark and Shark on Elastic MapReduce. The article includes examples of how to run both interactive Scala commands and SQL queries from Shark on data in S3. Head over to the <a href="http://aws.amazon.com/articles/Elastic-MapReduce/4926593393724923">Amazon article</a> for details. We’re very excited because, to our knowledge, this makes Spark the first non-Hadoop engine that you can launch with EMR.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-6-1-and-0-5-2-released.html b/site/news/spark-0-6-1-and-0-5-2-released.html index f8ecbef6d..b0063c7ee 100644 --- a/site/news/spark-0-6-1-and-0-5-2-released.html +++ b/site/news/spark-0-6-1-and-0-5-2-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 0.6.1 and 0.5.2 out</h2> -<p>Today we’ve made available two maintenance releases for Spark: <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a> and <a href="/releases/spark-release-0-5-2.html" title="Spark Release 0.5.2">0.5.2</a>. They both contain important bug fixes as well as some new features, such as the ability to build against Hadoop 2 distributions. We recommend that users update to the latest version for their branch; for new users, we recommend <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a>.</p> +<p>Today we’ve made available two maintenance releases for Spark: <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a> and <a href="/releases/spark-release-0-5-2.html" title="Spark Release 0.5.2">0.5.2</a>. They both contain important bug fixes as well as some new features, such as the ability to build against Hadoop 2 distributions. We recommend that users update to the latest version for their branch; for new users, we recommend <a href="/releases/spark-release-0-6-1.html" title="Spark Release 0.6.1">0.6.1</a>.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-6-2-released.html b/site/news/spark-0-6-2-released.html index 78eccbade..2e17df352 100644 --- a/site/news/spark-0-6-2-released.html +++ b/site/news/spark-0-6-2-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-7-0-released.html b/site/news/spark-0-7-0-released.html index b0e6dde60..61b1d9861 100644 --- a/site/news/spark-0-7-0-released.html +++ b/site/news/spark-0-7-0-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 0.7.0 released</h2> -<p>We’re proud to announce the release of <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">Spark 0.7.0</a>, a new major version of Spark that adds several key features, including a <a href="/docs/latest/python-programming-guide.html">Python API</a> for Spark and an <a href="/docs/latest/streaming-programming-guide.html">alpha of Spark Streaming</a>. This release is the result of the largest group of contributors yet behind a Spark release – 31 contributors from inside and outside Berkeley. Head over to the <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">release notes</a> to read more about the new features, or <a href="/downloads.html">download</a> the release today.</p> +<p>We’re proud to announce the release of <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">Spark 0.7.0</a>, a new major version of Spark that adds several key features, including a <a href="/docs/latest/python-programming-guide.html">Python API</a> for Spark and an <a href="/docs/latest/streaming-programming-guide.html">alpha of Spark Streaming</a>. This release is the result of the largest group of contributors yet behind a Spark release – 31 contributors from inside and outside Berkeley. Head over to the <a href="/releases/spark-release-0-7-0.html" title="Spark Release 0.7.0">release notes</a> to read more about the new features, or <a href="/downloads.html">download</a> the release today.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-7-2-released.html b/site/news/spark-0-7-2-released.html index 89490adb1..5d4a77e27 100644 --- a/site/news/spark-0-7-2-released.html +++ b/site/news/spark-0-7-2-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 0.7.2 released</h2> -<p>We’re happy to announce the release of <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">Spark 0.7.2</a>, a new maintenance release that includes several bug fixes and improvements, as well as new code examples and API features. We recommend that all users update to this release. Head over to the <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> +<p>We’re happy to announce the release of <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">Spark 0.7.2</a>, a new maintenance release that includes several bug fixes and improvements, as well as new code examples and API features. We recommend that all users update to this release. Head over to the <a href="/releases/spark-release-0-7-2.html" title="Spark Release 0.7.2">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-7-3-released.html b/site/news/spark-0-7-3-released.html index da5b58ccf..cf68260f8 100644 --- a/site/news/spark-0-7-3-released.html +++ b/site/news/spark-0-7-3-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 0.7.3 released</h2> -<p>We’ve just posted <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">Spark Release 0.7.3</a>, a maintenance release that contains several fixes, including streaming API updates and new functionality for adding JARs to a <code>spark-shell</code> session. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> +<p>We’ve just posted <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">Spark Release 0.7.3</a>, a maintenance release that contains several fixes, including streaming API updates and new functionality for adding JARs to a <code>spark-shell</code> session. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-7-3.html" title="Spark Release 0.7.3">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-8-0-released.html b/site/news/spark-0-8-0-released.html index 8d13ba741..3ed34f433 100644 --- a/site/news/spark-0-8-0-released.html +++ b/site/news/spark-0-8-0-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 0.8.0 released</h2> -<p>We’re proud to announce the release of <a href="/releases/spark-release-0-8-0.html" title="Spark Release 0.8.0">Apache Spark 0.8.0</a>. Spark 0.8.0 is a major release that includes many new capabilities and usability improvements. It’s also our first release under the Apache incubator. It is the largest Spark release yet, with contributions from 67 developers and 24 companies. Major new features include an expanded monitoring framework and UI, a machine learning library, and support for running Spark inside of YARN.</p> +<p>We’re proud to announce the release of <a href="/releases/spark-release-0-8-0.html" title="Spark Release 0.8.0">Apache Spark 0.8.0</a>. Spark 0.8.0 is a major release that includes many new capabilities and usability improvements. It’s also our first release under the Apache incubator. It is the largest Spark release yet, with contributions from 67 developers and 24 companies. Major new features include an expanded monitoring framework and UI, a machine learning library, and support for running Spark inside of YARN.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-8-1-released.html b/site/news/spark-0-8-1-released.html index c9406bcd3..d9144d891 100644 --- a/site/news/spark-0-8-1-released.html +++ b/site/news/spark-0-8-1-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 0.8.1 released</h2> -<p>We’ve just posted <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">Spark Release 0.8.1</a>, a maintenance and performance release for the Scala 2.9 version of Spark. 0.8.1 includes support for YARN 2.2, a high availability mode for the standalone scheduler, optimizations to the shuffle, and many other improvements. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> +<p>We’ve just posted <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">Spark Release 0.8.1</a>, a maintenance and performance release for the Scala 2.9 version of Spark. 0.8.1 includes support for YARN 2.2, a high availability mode for the standalone scheduler, optimizations to the shuffle, and many other improvements. We recommend that all users update to this release. Visit the <a href="/releases/spark-release-0-8-1.html" title="Spark Release 0.8.1">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-9-0-released.html b/site/news/spark-0-9-0-released.html index 86d4ce563..3c72c9974 100644 --- a/site/news/spark-0-9-0-released.html +++ b/site/news/spark-0-9-0-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -161,8 +166,8 @@ <p>We are happy to announce the availability of <a href="/releases/spark-release-0-9-0.html" title="Spark Release 0.9.0"> -Spark 0.9.0</a>! Spark 0.9.0 is a major release and Spark’s largest release ever, with contributions from 83 developers. -This release expands Spark’s standard libraries, introducing a new graph computation package (GraphX) and adding several new features to the machine learning and stream-processing packages. It also makes major improvements to the core engine, +Spark 0.9.0</a>! Spark 0.9.0 is a major release and Spark’s largest release ever, with contributions from 83 developers. +This release expands Spark’s standard libraries, introducing a new graph computation package (GraphX) and adding several new features to the machine learning and stream-processing packages. It also makes major improvements to the core engine, including external aggregations, a simplified H/A mode for long lived applications, and hardened YARN support.</p> @@ -188,9 +193,5 @@ to read about the new features, or <a href="/downloads.html">download</a> the re </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-9-1-released.html b/site/news/spark-0-9-1-released.html index e6831d203..e551e0c73 100644 --- a/site/news/spark-0-9-1-released.html +++ b/site/news/spark-0-9-1-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -187,9 +192,5 @@ to read about the new features, or <a href="/downloads.html">download</a> the re </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-0-9-2-released.html b/site/news/spark-0-9-2-released.html index 321a6c686..74faa858c 100644 --- a/site/news/spark-0-9-2-released.html +++ b/site/news/spark-0-9-2-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -186,9 +191,5 @@ to read about the new features, or <a href="/downloads.html">download</a> the re </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-1-0-0-released.html b/site/news/spark-1-0-0-released.html index 579c096e9..c0f9548a7 100644 --- a/site/news/spark-1-0-0-released.html +++ b/site/news/spark-1-0-0-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,8 +165,8 @@ <h2>Spark 1.0.0 released</h2> -<p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-0.html" title="Spark Release 1.0.0">Spark 1.0.0</a>! Spark 1.0.0 is the first in the 1.X line of releases, providing API stability for Spark’s core interfaces. It is Spark’s largest release ever, with contributions from 117 developers. -This release expands Spark’s standard libraries, introducing a new SQL package (Spark SQL) that lets users integrate SQL queries into existing Spark workflows. MLlib, Spark’s machine learning library, is expanded with sparse vector support and several new algorithms. The GraphX and Streaming libraries also introduce new features and optimizations. Spark’s core engine adds support for secured YARN clusters, a unified tool for submitting Spark applications, and several performance and stability improvements.</p> +<p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-0.html" title="Spark Release 1.0.0">Spark 1.0.0</a>! Spark 1.0.0 is the first in the 1.X line of releases, providing API stability for Spark’s core interfaces. It is Spark’s largest release ever, with contributions from 117 developers. +This release expands Spark’s standard libraries, introducing a new SQL package (Spark SQL) that lets users integrate SQL queries into existing Spark workflows. MLlib, Spark’s machine learning library, is expanded with sparse vector support and several new algorithms. The GraphX and Streaming libraries also introduce new features and optimizations. Spark’s core engine adds support for secured YARN clusters, a unified tool for submitting Spark applications, and several performance and stability improvements.</p> <p>Visit the <a href="/releases/spark-release-1-0-0.html" title="Spark Release 1.0.0">release notes</a> to read about the new features, or <a href="/downloads.html">download</a> the release today.</p> @@ -184,9 +189,5 @@ This release expands Spark’s standard libraries, introducing a new SQL package </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-1-0-1-released.html b/site/news/spark-1-0-1-released.html index f93285f2e..1384ca187 100644 --- a/site/news/spark-1-0-1-released.html +++ b/site/news/spark-1-0-1-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark 1.0.1 released</h2> -<p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-1.html" title="Spark Release 1.0.1">Spark 1.0.1</a>! This release includes contributions from 70 developers. Spark 1.0.0 includes fixes across several areas of Spark, including the core API, PySpark, and MLlib. It also includes new features in Spark’s (alpha) SQL library, including support for JSON data and performance and stability fixes.</p> +<p>We are happy to announce the availability of <a href="/releases/spark-release-1-0-1.html" title="Spark Release 1.0.1">Spark 1.0.1</a>! This release includes contributions from 70 developers. Spark 1.0.0 includes fixes across several areas of Spark, including the core API, PySpark, and MLlib. It also includes new features in Spark’s (alpha) SQL library, including support for JSON data and performance and stability fixes.</p> <p>Visit the <a href="/releases/spark-release-1-0-1.html" title="Spark Release 1.0.1">release notes</a> to read about this release or <a href="/downloads.html">download</a> the release today.</p> @@ -183,9 +188,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-1-0-2-released.html b/site/news/spark-1-0-2-released.html index e164592d1..fd63eee44 100644 --- a/site/news/spark-1-0-2-released.html +++ b/site/news/spark-1-0-2-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -183,9 +188,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-accepted-into-apache-incubator.html b/site/news/spark-accepted-into-apache-incubator.html index 25558cceb..c8e0621e8 100644 --- a/site/news/spark-accepted-into-apache-incubator.html +++ b/site/news/spark-accepted-into-apache-incubator.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-and-shark-in-the-news.html b/site/news/spark-and-shark-in-the-news.html index 307ede41a..ad06821b7 100644 --- a/site/news/spark-and-shark-in-the-news.html +++ b/site/news/spark-and-shark-in-the-news.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Spark in the news</h2> -<p>Recently, we’ve seen quite a bit of coverage of Spark in the news. I wanted to list some of the more recent articles, for readers interested in learning more.</p> +<p>Recently, we’ve seen quite a bit of coverage of Spark in the news. I wanted to list some of the more recent articles, for readers interested in learning more.</p> <ul> <li>Curt Monash, editor of the popular DBMS2 blog, wrote a great <a href="http://www.dbms2.com/2012/12/13/introduction-to-spark-shark-bdas-and-amplab/">introduction to Spark and Shark</a>, as well as a more detailed <a href="http://www.dbms2.com/2012/12/13/spark-shark-and-rdds-technology-notes/">technical overview</a>.</li> @@ -170,7 +175,7 @@ <li><a href="http://data-informed.com/spark-an-open-source-engine-for-iterative-data-mining/">DataInformed</a> interviewed two Spark users and wrote about their applications in anomaly detection, predictive analytics and data mining.</li> </ul> -<p>In other news, there will be a full day of tutorials on Spark and Shark at the <a href="http://strataconf.com/strata2013">O’Reilly Strata conference</a> in February. They include a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27438">introduction to Spark, Shark and BDAS</a> Tuesday morning, and a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27440">hands-on exercise session</a>. </p> +<p>In other news, there will be a full day of tutorials on Spark and Shark at the <a href="http://strataconf.com/strata2013">O’Reilly Strata conference</a> in February. They include a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27438">introduction to Spark, Shark and BDAS</a> Tuesday morning, and a three-hour <a href="http://strataconf.com/strata2013/public/schedule/detail/27440">hands-on exercise session</a>. </p> <p> @@ -191,9 +196,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-becomes-tlp.html b/site/news/spark-becomes-tlp.html index f5879005e..9887fa98d 100644 --- a/site/news/spark-becomes-tlp.html +++ b/site/news/spark-becomes-tlp.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,9 +165,9 @@ <h2>Spark becomes top-level Apache project</h2> -<p>The Apache Software Foundation <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">announced</a> today that Spark has graduated from the Apache Incubator to become a top-level Apache project, signifying that the project’s community and products have been well-governed under the ASF’s meritocratic process and principles. This is a major step for the community and we are very proud to share this news with users as we complete Spark’s move to Apache. Read more about Spark’s growth during the past year and from contributors and users in the ASF’s <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">press release</a>.</p> +<p>The Apache Software Foundation <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">announced</a> today that Spark has graduated from the Apache Incubator to become a top-level Apache project, signifying that the project’s community and products have been well-governed under the ASF’s meritocratic process and principles. This is a major step for the community and we are very proud to share this news with users as we complete Spark’s move to Apache. Read more about Spark’s growth during the past year and from contributors and users in the ASF’s <a href="https://blogs.apache.org/foundation/entry/the_apache_software_foundation_announces50">press release</a>.</p> -<p>As part of this change, note that Spark’s <a href="/community.html">mailing lists</a> have moved to <tt>@spark.apache.org</tt> addresses, although the old <tt>@spark.incubator.apache.org</tt> addresses also still work.</p> +<p>As part of this change, note that Spark’s <a href="/community.html">mailing lists</a> have moved to <tt>@spark.apache.org</tt> addresses, although the old <tt>@spark.incubator.apache.org</tt> addresses also still work.</p> <p> @@ -183,9 +188,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-featured-in-wired.html b/site/news/spark-featured-in-wired.html index cee2e63de..6af970ae1 100644 --- a/site/news/spark-featured-in-wired.html +++ b/site/news/spark-featured-in-wired.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-mailing-lists-moving-to-apache.html b/site/news/spark-mailing-lists-moving-to-apache.html index 8fede6399..97c0368f2 100644 --- a/site/news/spark-mailing-lists-moving-to-apache.html +++ b/site/news/spark-mailing-lists-moving-to-apache.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -192,9 +197,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-meetups.html b/site/news/spark-meetups.html index 565946604..df43671a2 100644 --- a/site/news/spark-meetups.html +++ b/site/news/spark-meetups.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>We've started hosting a Bay Area Spark User Meetup</h2> -<p>We’ve started hosting a regular <a href="http://www.meetup.com/spark-users/">Bay Area Spark User Meetup</a>. Sign up on the meetup.com page to be notified about events and meet other Spark developers and users.</p> +<p>We’ve started hosting a regular <a href="http://www.meetup.com/spark-users/">Bay Area Spark User Meetup</a>. Sign up on the meetup.com page to be notified about events and meet other Spark developers and users.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-screencasts-published.html b/site/news/spark-screencasts-published.html index 00f729e08..c411f6c3c 100644 --- a/site/news/spark-screencasts-published.html +++ b/site/news/spark-screencasts-published.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -187,9 +192,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-summit-2013-is-a-wrap.html b/site/news/spark-summit-2013-is-a-wrap.html index 1ed22f298..525adcb47 100644 --- a/site/news/spark-summit-2013-is-a-wrap.html +++ b/site/news/spark-summit-2013-is-a-wrap.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -187,9 +192,5 @@ Over 450 Spark developers and enthusiasts from 13 countries and more than 180 co </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-summit-2014-videos-posted.html b/site/news/spark-summit-2014-videos-posted.html index 816bc1da5..3e42a50ac 100644 --- a/site/news/spark-summit-2014-videos-posted.html +++ b/site/news/spark-summit-2014-videos-posted.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -183,9 +188,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-summit-agenda-posted.html b/site/news/spark-summit-agenda-posted.html index b7688a304..abd36f745 100644 --- a/site/news/spark-summit-agenda-posted.html +++ b/site/news/spark-summit-agenda-posted.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -189,9 +194,5 @@ about the latest happenings in Spark.</p> </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-tips-from-quantifind.html b/site/news/spark-tips-from-quantifind.html index 2666c0a2b..5f7915850 100644 --- a/site/news/spark-tips-from-quantifind.html +++ b/site/news/spark-tips-from-quantifind.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -186,9 +191,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-user-survey-and-powered-by-page.html b/site/news/spark-user-survey-and-powered-by-page.html index 57c65d1b7..53cd41961 100644 --- a/site/news/spark-user-survey-and-powered-by-page.html +++ b/site/news/spark-user-survey-and-powered-by-page.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,9 +165,9 @@ <h2>Spark user survey and "Powered By" page</h2> -<p>As we continue developing Spark, we would love to get feedback from users and hear what you’d like us to work on next. We’ve decided that a good way to do that is a survey – we hope to run this at regular intervals. If you have a few minutes to participate, <a href="https://docs.google.com/forms/d/1eMXp4GjcIXglxJe5vYYBzXKVm-6AiYt1KThJwhCjJiY/viewform">fill in the survey here</a>. Your time is greatly appreciated.</p> +<p>As we continue developing Spark, we would love to get feedback from users and hear what you’d like us to work on next. We’ve decided that a good way to do that is a survey – we hope to run this at regular intervals. If you have a few minutes to participate, <a href="https://docs.google.com/forms/d/1eMXp4GjcIXglxJe5vYYBzXKVm-6AiYt1KThJwhCjJiY/viewform">fill in the survey here</a>. Your time is greatly appreciated.</p> -<p>In parallel, we are starting a <a href="https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark">“powered by” page</a> on the Apache Spark wiki for organizations that are using, or contributing to, Spark. Sign up if you’d like to support the project! This is a great way to let the world know you’re using Spark, and can also be helpful to generate leads for recruiting. You can also add yourself when you fill the survey.</p> +<p>In parallel, we are starting a <a href="https://cwiki.apache.org/confluence/display/SPARK/Powered+By+Spark">“powered by” page</a> on the Apache Spark wiki for organizations that are using, or contributing to, Spark. Sign up if you’d like to support the project! This is a great way to let the world know you’re using Spark, and can also be helpful to generate leads for recruiting. You can also add yourself when you fill the survey.</p> <p>Thanks for taking the time to give feedback.</p> @@ -185,9 +190,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/spark-version-0-6-0-released.html b/site/news/spark-version-0-6-0-released.html index 13f1f5161..be089e329 100644 --- a/site/news/spark-version-0-6-0-released.html +++ b/site/news/spark-version-0-6-0-released.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/strata-exercises-now-available-online.html b/site/news/strata-exercises-now-available-online.html index da95ff170..af5fd9917 100644 --- a/site/news/strata-exercises-now-available-online.html +++ b/site/news/strata-exercises-now-available-online.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Strata exercises now available online</h2> -<p>At this year’s <a href="http://strataconf.com/strata2013">Strata</a> conference, the AMP Lab hosted a full day of tutorials on Spark, Shark, and Spark Streaming, including online exercises on Amazon EC2. Those exercises are now <a href="http://ampcamp.berkeley.edu/big-data-mini-course/">available online</a>, letting you learn Spark and Shark at your own pace on an EC2 cluster with real data. They are a great resource for learning the systems. You can also find <a href="http://ampcamp.berkeley.edu/amp-camp-two-strata-2013/">slides</a> from the Strata tutorials online, as well as <a href="http://ampcamp.berkeley.edu/amp-camp-one-berkeley-2012/">videos</a> from the AMP Camp workshop we held at Berkeley in August.</p> +<p>At this year’s <a href="http://strataconf.com/strata2013">Strata</a> conference, the AMP Lab hosted a full day of tutorials on Spark, Shark, and Spark Streaming, including online exercises on Amazon EC2. Those exercises are now <a href="http://ampcamp.berkeley.edu/big-data-mini-course/">available online</a>, letting you learn Spark and Shark at your own pace on an EC2 cluster with real data. They are a great resource for learning the systems. You can also find <a href="http://ampcamp.berkeley.edu/amp-camp-two-strata-2013/">slides</a> from the Strata tutorials online, as well as <a href="http://ampcamp.berkeley.edu/amp-camp-one-berkeley-2012/">videos</a> from the AMP Camp workshop we held at Berkeley in August.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/submit-talks-to-spark-summit-2014.html b/site/news/submit-talks-to-spark-summit-2014.html index d18b2bc57..f32ba2188 100644 --- a/site/news/submit-talks-to-spark-summit-2014.html +++ b/site/news/submit-talks-to-spark-summit-2014.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,12 +165,12 @@ <h2>Submissions and registration open for Spark Summit 2014</h2> -<p>After last year’s successful <a href="http://spark-summit.org/2013">first Spark Summit</a>, registrations +<p>After last year’s successful <a href="http://spark-summit.org/2013">first Spark Summit</a>, registrations and talk submissions are now open for <a href="http://spark-summit.org/2014">Spark Summit 2014</a>. This will be a 3-day event in San Francisco organized by multiple companies in the Spark community. The event will run <strong>June 30th to July 2nd</strong> in San Francisco, CA.</p> -<p>If you’d like to present at the Summit, <a href="http://spark-summit.org/submit">submit a talk</a> +<p>If you’d like to present at the Summit, <a href="http://spark-summit.org/submit">submit a talk</a> before April 11th, 2014. We welcome talks on use cases, open source development, and applications built on Spark.</p> @@ -190,9 +195,5 @@ on Spark.</p> </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/two-weeks-to-spark-summit-2014.html b/site/news/two-weeks-to-spark-summit-2014.html index 97f9bac3a..1203e9d43 100644 --- a/site/news/two-weeks-to-spark-summit-2014.html +++ b/site/news/two-weeks-to-spark-summit-2014.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -165,7 +170,7 @@ will be held in San Francisco on June 30th to July 2nd. The Summit will contain <a href="http://spark-summit.org/2014/agenda">presentations</a> from over 50 organizations using Spark, focused on use cases and ongoing development.</p> -<p>If you’d like to come to the Summit, you can still +<p>If you’d like to come to the Summit, you can still <a href="http://www.eventbrite.com/e/2014-spark-summit-registration-registration-10381067051">register</a> online to attend in person. Otherwise, the Summit will offer a free live video stream; <a href="https://docs.google.com/forms/d/1Jv_l-m1PQ1CPwKVIpOrWIDVBaWsJlJv14ZYkbezVjTs/viewform">register for the stream</a> in advance to view it.</p> @@ -191,9 +196,5 @@ online to attend in person. Otherwise, the Summit will offer a free live video s </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/news/video-from-first-spark-development-meetup.html b/site/news/video-from-first-spark-development-meetup.html index 62972080b..10b59df2d 100644 --- a/site/news/video-from-first-spark-development-meetup.html +++ b/site/news/video-from-first-spark-development-meetup.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -160,7 +165,7 @@ <h2>Video up from first Spark development meetup</h2> -<p>On December 18th, we held the first of a series of Spark development meetups, for people interested in learning the Spark codebase and contributing to the project. There was quite a bit more demand than we anticipated, with over 80 people signing up and 64 attending. The first meetup was an <a href="http://www.meetup.com/spark-users/events/94101942/">introduction to Spark internals</a>. Thanks to one of the attendees, there’s now a <a href="http://www.youtube.com/watch?v=49Hr5xZyTEA">video of the meetup</a> on YouTube. We’ve also posted the <a href="http://files.meetup.com/3138542/dev-meetup-dec-2012.pptx">slides</a>. Look to see more development meetups on Spark and Shark in the future.</p> +<p>On December 18th, we held the first of a series of Spark development meetups, for people interested in learning the Spark codebase and contributing to the project. There was quite a bit more demand than we anticipated, with over 80 people signing up and 64 attending. The first meetup was an <a href="http://www.meetup.com/spark-users/events/94101942/">introduction to Spark internals</a>. Thanks to one of the attendees, there’s now a <a href="http://www.youtube.com/watch?v=49Hr5xZyTEA">video of the meetup</a> on YouTube. We’ve also posted the <a href="http://files.meetup.com/3138542/dev-meetup-dec-2012.pptx">slides</a>. Look to see more development meetups on Spark and Shark in the future.</p> <p> @@ -181,9 +186,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-3.html b/site/releases/spark-release-0-3.html index c7a4881e3..8d5c7b65f 100644 --- a/site/releases/spark-release-0-3.html +++ b/site/releases/spark-release-0-3.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -176,7 +181,7 @@ <h3>Native Types for SequenceFiles</h3> -<p>In working with SequenceFiles, which store objects that implement Hadoop’s Writable interface, Spark will now let you use native types for certain common Writable types, like IntWritable and Text. For example:</p> +<p>In working with SequenceFiles, which store objects that implement Hadoop’s Writable interface, Spark will now let you use native types for certain common Writable types, like IntWritable and Text. For example:</p> <div class="code"> <span class="comment">// Will read a SequenceFile of (IntWritable, Text)</span><br /> @@ -232,9 +237,5 @@ squares.saveAsSequenceFile(<span class="string">"hdfs://..."</span>) </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-5-0.html b/site/releases/spark-release-0-5-0.html index faa3460e1..de40ae1f5 100644 --- a/site/releases/spark-release-0-5-0.html +++ b/site/releases/spark-release-0-5-0.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -164,10 +169,10 @@ <h3>Mesos 0.9 Support</h3> -<p>This release runs on <a href="http://www.mesosproject.org/">Apache Mesos 0.9</a>, the first Apache Incubator release of Mesos, which contains significant usability and stability improvements. Most notable are better memory accounting for applications with long-term memory use, easier access of old jobs’ traces and logs (by keeping a history of executed tasks on the web UI), and simpler installation.</p> +<p>This release runs on <a href="http://www.mesosproject.org/">Apache Mesos 0.9</a>, the first Apache Incubator release of Mesos, which contains significant usability and stability improvements. Most notable are better memory accounting for applications with long-term memory use, easier access of old jobs’ traces and logs (by keeping a history of executed tasks on the web UI), and simpler installation.</p> <h3>Performance Improvements</h3> -<p>Spark’s scheduling is more communication-efficient when sending out operations on RDDs with large lineage graphs. In addition, the cache replacement policy has been improved to more smartly replace data when an RDD does not fit in the cache, shuffles are more efficient, and the serializer used for shipping closures is now configurable, making it possible to use faster libraries than Java serialization there.</p> +<p>Spark’s scheduling is more communication-efficient when sending out operations on RDDs with large lineage graphs. In addition, the cache replacement policy has been improved to more smartly replace data when an RDD does not fit in the cache, shuffles are more efficient, and the serializer used for shipping closures is now configurable, making it possible to use faster libraries than Java serialization there.</p> <h3>Debug Improvements</h3> @@ -179,11 +184,11 @@ <h3>EC2 Launch Script Improvements</h3> -<p>Spark’s EC2 launch scripts are now included in the main package, and have the ability to discover and use the latest Spark AMI automatically instead of launching a hardcoded machine image ID.</p> +<p>Spark’s EC2 launch scripts are now included in the main package, and have the ability to discover and use the latest Spark AMI automatically instead of launching a hardcoded machine image ID.</p> <h3>New Hadoop API Support</h3> -<p>You can now use Spark to read and write data to storage formats in the new <tt>org.apache.mapreduce</tt> packages (the “new Hadoop” API). In addition, this release fixes an issue caused by a HDFS initialization bug in some recent versions of HDFS.</p> +<p>You can now use Spark to read and write data to storage formats in the new <tt>org.apache.mapreduce</tt> packages (the “new Hadoop” API). In addition, this release fixes an issue caused by a HDFS initialization bug in some recent versions of HDFS.</p> <p> @@ -204,9 +209,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-5-1.html b/site/releases/spark-release-0-5-1.html index 3776a1ff3..d58546711 100644 --- a/site/releases/spark-release-0-5-1.html +++ b/site/releases/spark-release-0-5-1.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -193,7 +198,7 @@ <h3>EC2 Improvements</h3> -<p>Spark’s EC2 launch script now configures Spark’s memory limit automatically based on the machine’s available RAM.</p> +<p>Spark’s EC2 launch script now configures Spark’s memory limit automatically based on the machine’s available RAM.</p> <p> @@ -214,9 +219,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-5-2.html b/site/releases/spark-release-0-5-2.html index b67ae72f0..516756f4f 100644 --- a/site/releases/spark-release-0-5-2.html +++ b/site/releases/spark-release-0-5-2.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -183,9 +188,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-6-0.html b/site/releases/spark-release-0-6-0.html index 5e2ee0b4d..734a97525 100644 --- a/site/releases/spark-release-0-6-0.html +++ b/site/releases/spark-release-0-6-0.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -172,11 +177,11 @@ <h3>Java API</h3> -<p>Java programmers can now use Spark through a new <a href="/docs/0.6.0/java-programming-guide.html">Java API layer</a>. This layer makes available all of Spark’s features, including parallel transformations, distributed datasets, broadcast variables, and accumulators, in a Java-friendly manner.</p> +<p>Java programmers can now use Spark through a new <a href="/docs/0.6.0/java-programming-guide.html">Java API layer</a>. This layer makes available all of Spark’s features, including parallel transformations, distributed datasets, broadcast variables, and accumulators, in a Java-friendly manner.</p> <h3>Expanded Documentation</h3> -<p>Spark’s <a href="/docs/0.6.0/">documentation</a> has been expanded with a new <a href="/docs/0.6.0/quick-start.html">quick start guide</a>, additional deployment instructions, configuration guide, tuning guide, and improved <a href="/docs/0.6.0/api/core">Scaladoc</a> API documentation.</p> +<p>Spark’s <a href="/docs/0.6.0/">documentation</a> has been expanded with a new <a href="/docs/0.6.0/quick-start.html">quick start guide</a>, additional deployment instructions, configuration guide, tuning guide, and improved <a href="/docs/0.6.0/api/core">Scaladoc</a> API documentation.</p> <h3>Engine Changes</h3> @@ -199,7 +204,7 @@ <h3>Enhanced Debugging</h3> -<p>Spark’s log now prints which operation in your program each RDD and job described in your logs belongs to, making it easier to tie back to which parts of your code experience problems.</p> +<p>Spark’s log now prints which operation in your program each RDD and job described in your logs belongs to, making it easier to tie back to which parts of your code experience problems.</p> <h3>Maven Artifacts</h3> @@ -258,9 +263,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-6-1.html b/site/releases/spark-release-0-6-1.html index 4802c4d3a..dbf89f97a 100644 --- a/site/releases/spark-release-0-6-1.html +++ b/site/releases/spark-release-0-6-1.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -198,9 +203,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-6-2.html b/site/releases/spark-release-0-6-2.html index ffc6b2e69..1f320004c 100644 --- a/site/releases/spark-release-0-6-2.html +++ b/site/releases/spark-release-0-6-2.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -212,9 +217,5 @@ In total, eleven people contributed to this release: </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-7-0.html b/site/releases/spark-release-0-7-0.html index 16467dc45..16a91fd0a 100644 --- a/site/releases/spark-release-0-7-0.html +++ b/site/releases/spark-release-0-7-0.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -186,7 +191,7 @@ <h3>New Operations</h3> -<p>This release adds several RDD transformations, including <tt>keys</tt>, <tt>values</tt>, <tt>keyBy</tt>, <tt>subtract</tt>, <tt>coalesce</tt>, <tt>zip</tt>. It also adds <tt>SparkContext.hadoopConfiguration</tt> to allow programs to configure Hadoop input/output settings globally across operations. Finally, it adds the <tt>RDD.toDebugString()</tt> method, which can be used to print an RDD’s lineage graph for troubleshooting.</p> +<p>This release adds several RDD transformations, including <tt>keys</tt>, <tt>values</tt>, <tt>keyBy</tt>, <tt>subtract</tt>, <tt>coalesce</tt>, <tt>zip</tt>. It also adds <tt>SparkContext.hadoopConfiguration</tt> to allow programs to configure Hadoop input/output settings globally across operations. Finally, it adds the <tt>RDD.toDebugString()</tt> method, which can be used to print an RDD’s lineage graph for troubleshooting.</p> <h3>EC2 Improvements</h3> @@ -223,7 +228,7 @@ <h3>Credits</h3> -<p>Spark 0.7 was the work of many contributors from Berkeley and outside—in total, 31 different contributors, of which 20 were from outside Berkeley. Here are the people who contributed, along with areas they worked on:</p> +<p>Spark 0.7 was the work of many contributors from Berkeley and outside—in total, 31 different contributors, of which 20 were from outside Berkeley. Here are the people who contributed, along with areas they worked on:</p> <ul> <li>Mikhail Bautin -- Maven build</li> @@ -280,9 +285,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-7-2.html b/site/releases/spark-release-0-7-2.html index 7bcef288b..3621048fa 100644 --- a/site/releases/spark-release-0-7-2.html +++ b/site/releases/spark-release-0-7-2.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -222,9 +227,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-7-3.html b/site/releases/spark-release-0-7-3.html index 45c474fa2..2b8ed466d 100644 --- a/site/releases/spark-release-0-7-3.html +++ b/site/releases/spark-release-0-7-3.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -216,9 +221,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-8-0.html b/site/releases/spark-release-0-8-0.html index 1d981a2b3..1005fe589 100644 --- a/site/releases/spark-release-0-8-0.html +++ b/site/releases/spark-release-0-8-0.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -204,13 +209,13 @@ <li>The examples build has been isolated from the core build, substantially reducing the potential for dependency conflicts.</li> <li>The Spark Streaming Twitter API has been updated to use OAuth authentication instead of the deprecated username/password authentication in Spark 0.7.0.</li> <li>Several new example jobs have been added, including PageRank implementations in Java, Scala and Python, examples for accessing HBase and Cassandra, and MLlib examples.</li> - <li>Support for running on Mesos has been improved – now you can deploy a Spark assembly JAR as part of the Mesos job, instead of having Spark pre-installed on each machine. The default Mesos version has also been updated to 0.13.</li> + <li>Support for running on Mesos has been improved – now you can deploy a Spark assembly JAR as part of the Mesos job, instead of having Spark pre-installed on each machine. The default Mesos version has also been updated to 0.13.</li> <li>This release includes various optimizations to PySpark and to the job scheduler.</li> </ul> <h3 id="compatibility">Compatibility</h3> <ul> - <li><strong>This release changes Spark’s package name to ‘org.apache.spark’</strong>, so those upgrading from Spark 0.7 will need to adjust their imports accordingly. In addition, we’ve moved the <code>RDD</code> class to the org.apache.spark.rdd package (it was previously in the top-level package). The Spark artifacts published through Maven have also changed to the new package name.</li> + <li><strong>This release changes Spark’s package name to ‘org.apache.spark’</strong>, so those upgrading from Spark 0.7 will need to adjust their imports accordingly. In addition, we’ve moved the <code>RDD</code> class to the org.apache.spark.rdd package (it was previously in the top-level package). The Spark artifacts published through Maven have also changed to the new package name.</li> <li>In the Java API, use of Scala’s <code>Option</code> class has been replaced with <code>Optional</code> from the Guava library.</li> <li>Linking against Spark for arbitrary Hadoop versions is now possible by specifying a dependency on <code>hadoop-client</code>, instead of rebuilding <code>spark-core</code> against your version of Hadoop. See the documentation <a href="http://spark.incubator.apache.org/docs/0.8.0/scala-programming-guide.html#linking-with-spark">here</a> for details.</li> <li>If you are building Spark, you’ll now need to run <code>sbt/sbt assembly</code> instead of <code>package</code>.</li> @@ -220,73 +225,73 @@ <p>Spark 0.8.0 was the result of the largest team of contributors yet. The following developers contributed to this release:</p> <ul> - <li>Andrew Ash – documentation, code cleanup and logging improvements</li> - <li>Mikhail Bautin – bug fix</li> - <li>Konstantin Boudnik – Maven build, bug fixes, and documentation</li> - <li>Ian Buss – sbt configuration improvement</li> - <li>Evan Chan – API improvement, bug fix, and documentation</li> - <li>Lian Cheng – bug fix</li> - <li>Tathagata Das – performance improvement in streaming receiver and streaming bug fix</li> - <li>Aaron Davidson – Python improvements, bug fix, and unit tests</li> - <li>Giovanni Delussu – coalesced RDD feature</li> - <li>Joseph E. Gonzalez – improvement to zipPartitions</li> - <li>Karen Feng – several improvements to web UI</li> - <li>Andy Feng – HDFS metrics</li> - <li>Ali Ghodsi – configuration improvements and locality-aware coalesce</li> - <li>Christoph Grothaus – bug fix</li> - <li>Thomas Graves – support for secure YARN cluster and various YARN-related improvements</li> - <li>Stephen Haberman – bug fix, documentation, and code cleanup</li> - <li>Mark Hamstra – bug fixes and Maven build</li> - <li>Benjamin Hindman – Mesos compatibility and documentation</li> - <li>Liang-Chi Hsieh – bug fixes in build and in YARN mode</li> - <li>Shane Huang – shuffle improvements, bug fix</li> - <li>Ethan Jewett – Spark/HBase example</li> - <li>Holden Karau – bug fix and EC2 improvement</li> - <li>Kody Koeniger – JDBV RDD implementation</li> - <li>Andy Konwinski – documentation</li> - <li>Jey Kottalam – PySpark optimizations, Hadoop agnostic build (lead), and bug fixes</li> - <li>Andrey Kouznetsov – Bug fix</li> - <li>S. Kumar – Spark Streaming example</li> - <li>Ryan LeCompte – topK method optimization and serialization improvements</li> - <li>Gavin Li – compression codecs and pipe support</li> - <li>Harold Lim – fair scheduler</li> - <li>Dmitriy Lyubimov – bug fix</li> - <li>Chris Mattmann – Apache mentor</li> - <li>David McCauley – JSON API improvement</li> - <li>Sean McNamara – added <code>takeOrdered</code> function, bug fixes, and a build fix</li> - <li>Mridul Muralidharan – YARN integration (lead) and scheduler improvements</li> - <li>Marc Mercer – improvements to UI json output</li> - <li>Christopher Nguyen – bug fixes</li> - <li>Erik van Oosten – example fix</li> - <li>Kay Ousterhout – fix for scheduler regression and bug fixes</li> - <li>Xinghao Pan – MLLib contributions</li> - <li>Hiral Patel – bug fix</li> - <li>James Phillpotts – updated Twitter API for Spark streaming</li> - <li>Nick Pentreath – scala pageRank example, bagel improvement, and several Java examples</li> - <li>Alexander Pivovarov – logging improvement and Maven build</li> - <li>Mike Potts – configuration improvement</li> - <li>Rohit Rai – Spark/Cassandra example</li> - <li>Imran Rashid – bug fixes and UI improvement</li> - <li>Charles Reiss – bug fixes, code cleanup, performance improvements</li> - <li>Josh Rosen – Python API improvements, Java API improvements, EC2 scripts and bug fixes</li> - <li>Henry Saputra – Apache mentor</li> - <li>Jerry Shao – bug fixes, metrics system</li> - <li>Prashant Sharma – documentation</li> - <li>Mingfei Shi – joblogger and bug fix</li> - <li>Andre Schumacher – several PySpark features</li> - <li>Ginger Smith – MLLib contribution</li> - <li>Evan Sparks – contributions to MLLib</li> - <li>Ram Sriharsha – bug fix and RDD removal feature</li> - <li>Ameet Talwalkar – MLlib contributions</li> - <li>Roman Tkalenko – code refactoring and cleanup</li> - <li>Chu Tong – Java PageRank algorithm and bug fix in bash scripts</li> - <li>Shivaram Venkataraman – bug fixes, contributions to MLLib, netty shuffle fixes, and Java API additions</li> - <li>Patrick Wendell – release manager, bug fixes, documentation, metrics system, and web UI</li> - <li>Andrew Xia – fair scheduler (lead), metrics system, and ui improvements</li> - <li>Reynold Xin – shuffle improvements, bug fixes, code refactoring, usability improvements, MLLib contributions</li> - <li>Matei Zaharia – MLLib contributions, documentation, examples, UI improvements, PySpark improvements, and bug fixes</li> - <li>Wu Zeming – bug fix in scheduler</li> - <li>Bill Zhao – log message improvement</li> + <li>Andrew Ash – documentation, code cleanup and logging improvements</li> + <li>Mikhail Bautin – bug fix</li> + <li>Konstantin Boudnik – Maven build, bug fixes, and documentation</li> + <li>Ian Buss – sbt configuration improvement</li> + <li>Evan Chan – API improvement, bug fix, and documentation</li> + <li>Lian Cheng – bug fix</li> + <li>Tathagata Das – performance improvement in streaming receiver and streaming bug fix</li> + <li>Aaron Davidson – Python improvements, bug fix, and unit tests</li> + <li>Giovanni Delussu – coalesced RDD feature</li> + <li>Joseph E. Gonzalez – improvement to zipPartitions</li> + <li>Karen Feng – several improvements to web UI</li> + <li>Andy Feng – HDFS metrics</li> + <li>Ali Ghodsi – configuration improvements and locality-aware coalesce</li> + <li>Christoph Grothaus – bug fix</li> + <li>Thomas Graves – support for secure YARN cluster and various YARN-related improvements</li> + <li>Stephen Haberman – bug fix, documentation, and code cleanup</li> + <li>Mark Hamstra – bug fixes and Maven build</li> + <li>Benjamin Hindman – Mesos compatibility and documentation</li> + <li>Liang-Chi Hsieh – bug fixes in build and in YARN mode</li> + <li>Shane Huang – shuffle improvements, bug fix</li> + <li>Ethan Jewett – Spark/HBase example</li> + <li>Holden Karau – bug fix and EC2 improvement</li> + <li>Kody Koeniger – JDBV RDD implementation</li> + <li>Andy Konwinski – documentation</li> + <li>Jey Kottalam – PySpark optimizations, Hadoop agnostic build (lead), and bug fixes</li> + <li>Andrey Kouznetsov – Bug fix</li> + <li>S. Kumar – Spark Streaming example</li> + <li>Ryan LeCompte – topK method optimization and serialization improvements</li> + <li>Gavin Li – compression codecs and pipe support</li> + <li>Harold Lim – fair scheduler</li> + <li>Dmitriy Lyubimov – bug fix</li> + <li>Chris Mattmann – Apache mentor</li> + <li>David McCauley – JSON API improvement</li> + <li>Sean McNamara – added <code>takeOrdered</code> function, bug fixes, and a build fix</li> + <li>Mridul Muralidharan – YARN integration (lead) and scheduler improvements</li> + <li>Marc Mercer – improvements to UI json output</li> + <li>Christopher Nguyen – bug fixes</li> + <li>Erik van Oosten – example fix</li> + <li>Kay Ousterhout – fix for scheduler regression and bug fixes</li> + <li>Xinghao Pan – MLLib contributions</li> + <li>Hiral Patel – bug fix</li> + <li>James Phillpotts – updated Twitter API for Spark streaming</li> + <li>Nick Pentreath – scala pageRank example, bagel improvement, and several Java examples</li> + <li>Alexander Pivovarov – logging improvement and Maven build</li> + <li>Mike Potts – configuration improvement</li> + <li>Rohit Rai – Spark/Cassandra example</li> + <li>Imran Rashid – bug fixes and UI improvement</li> + <li>Charles Reiss – bug fixes, code cleanup, performance improvements</li> + <li>Josh Rosen – Python API improvements, Java API improvements, EC2 scripts and bug fixes</li> + <li>Henry Saputra – Apache mentor</li> + <li>Jerry Shao – bug fixes, metrics system</li> + <li>Prashant Sharma – documentation</li> + <li>Mingfei Shi – joblogger and bug fix</li> + <li>Andre Schumacher – several PySpark features</li> + <li>Ginger Smith – MLLib contribution</li> + <li>Evan Sparks – contributions to MLLib</li> + <li>Ram Sriharsha – bug fix and RDD removal feature</li> + <li>Ameet Talwalkar – MLlib contributions</li> + <li>Roman Tkalenko – code refactoring and cleanup</li> + <li>Chu Tong – Java PageRank algorithm and bug fix in bash scripts</li> + <li>Shivaram Venkataraman – bug fixes, contributions to MLLib, netty shuffle fixes, and Java API additions</li> + <li>Patrick Wendell – release manager, bug fixes, documentation, metrics system, and web UI</li> + <li>Andrew Xia – fair scheduler (lead), metrics system, and ui improvements</li> + <li>Reynold Xin – shuffle improvements, bug fixes, code refactoring, usability improvements, MLLib contributions</li> + <li>Matei Zaharia – MLLib contributions, documentation, examples, UI improvements, PySpark improvements, and bug fixes</li> + <li>Wu Zeming – bug fix in scheduler</li> + <li>Bill Zhao – log message improvement</li> </ul> <p>Thanks to everyone who contributed! @@ -311,9 +316,5 @@ We’d especially like to thank Patrick Wendell for acting as the release manage </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-8-1.html b/site/releases/spark-release-0-8-1.html index c00b8ceb2..d04c4e85d 100644 --- a/site/releases/spark-release-0-8-1.html +++ b/site/releases/spark-release-0-8-1.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -163,7 +168,7 @@ <p>Apache Spark 0.8.1 is a maintenance and performance release for the Scala 2.9 version of Spark. It also adds several new features, such as standalone mode high availability, that will appear in Spark 0.9 but developers wanted to have in Scala 2.9. Contributions to 0.8.1 came from 41 developers.</p> <h3 id="yarn-22-support">YARN 2.2 Support</h3> -<p>Support has been added for running Spark on YARN 2.2 and newer. Due to a change in the YARN API between previous versions and 2.2+, this was not supported in Spark 0.8.0. See the <a href="/docs/0.8.1/running-on-yarn.html">YARN documentation</a> for specific instructions on how to build Spark for YARN 2.2+. We’ve also included a pre-compiled binary for YARN 2.2.</p> +<p>Support has been added for running Spark on YARN 2.2 and newer. Due to a change in the YARN API between previous versions and 2.2+, this was not supported in Spark 0.8.0. See the <a href="/docs/0.8.1/running-on-yarn.html">YARN documentation</a> for specific instructions on how to build Spark for YARN 2.2+. We’ve also included a pre-compiled binary for YARN 2.2.</p> <h3 id="high-availability-mode-for-standalone-cluster-manager">High Availability Mode for Standalone Cluster Manager</h3> <p>The standalone cluster manager now has a high availability (H/A) mode which can tolerate master failures. This is particularly useful for long-running applications such as streaming jobs and the shark server, where the scheduler master previously represented a single point of failure. Instructions for deploying H/A mode are included <a href="/docs/0.8.1/spark-standalone.html#high-availability">in the documentation</a>. The current implementation uses Zookeeper for coordination.</p> @@ -174,7 +179,7 @@ <ul> <li>Optimized hashtables for shuffle data - reduces memory and CPU consumption</li> <li>Efficient encoding for JobConfs - improves latency for stages reading large numbers of blocks from HDFS, S3, and HBase</li> - <li>Shuffle file consolidation (off by default) - reduces the number of files created in large shuffles for better filesystem performance. This change works best on filesystems newer than ext3 (we recommend ext4 or XFS), and it will be the default in Spark 0.9, but we’ve left it off by default for compatibility. We recommend users turn this on unless they are using ext3 by setting <code>spark.shuffle.consolidateFiles</code> to “true”.</li> + <li>Shuffle file consolidation (off by default) - reduces the number of files created in large shuffles for better filesystem performance. This change works best on filesystems newer than ext3 (we recommend ext4 or XFS), and it will be the default in Spark 0.9, but we’ve left it off by default for compatibility. We recommend users turn this on unless they are using ext3 by setting <code>spark.shuffle.consolidateFiles</code> to “true”.</li> <li>Torrent broadcast (off by default) - a faster broadcast implementation for large objects.</li> <li>Support for fetching large result sets - allows tasks to return large results without tuning Akka buffer sizes.</li> </ul> @@ -211,47 +216,47 @@ <h3 id="credits">Credits</h3> <ul> - <li>Michael Armbrust – build fix</li> - <li>Pierre Borckmans – typo fix in documentation</li> - <li>Evan Chan – <code>local://</code> scheme for dependency jars</li> - <li>Ewen Cheslack-Postava – <code>add</code> method for python accumulators, support for setting config properties in python</li> - <li>Mosharaf Chowdhury – optimized broadcast implementation</li> - <li>Frank Dai – documentation fix</li> - <li>Aaron Davidson – shuffle file consolidation, H/A mode for standalone scheduler, cleaned up representation of block IDs, several improvements and bug fixes</li> - <li>Tathagata Das – new streaming operators, fix for kafka concurrency bug</li> - <li>Ankur Dave – support for pausing spot clusters on EC2</li> - <li>Harvey Feng – optimization to JobConf broadcasts, bug fixes, YARN 2.2 build</li> - <li>Ali Ghodsi – YARN 2.2 build</li> - <li>Thomas Graves – Spark YARN integration including secure HDFS access over YARN</li> - <li>Li Guoqiang – fix for Maven build</li> - <li>Stephen Haberman – bug fix</li> - <li>Haidar Hadi – documentation fix</li> - <li>Nathan Howell – bug fix relating to YARN</li> - <li>Holden Karau – Java version of <code>mapPartitionsWithIndex</code></li> - <li>Du Li – bug fix in make-distrubion.sh</li> - <li>Raymond Liu – work on YARN 2.2 build</li> - <li>Xi Liu – bug fix and code clean-up</li> - <li>David McCauley – bug fix in standalone mode JSON output</li> - <li>Michael (wannabeast) – bug fix in memory store</li> - <li>Fabrizio Milo – typos in documentation, clean-up in DAGScheduler, typo in scaladoc</li> - <li>Mridul Muralidharan – fixes to metadata cleaner and speculative execution</li> - <li>Sundeep Narravula – build fix, bug fixes in scheduler and tests, code clean-up</li> - <li>Kay Ousterhout – optimized result fetching, new information in UI, scheduler clean-up and bug fixes</li> - <li>Nick Pentreath – implicit feedback variant of ALS algorithm</li> - <li>Imran Rashid – improvement to executor launch</li> - <li>Ahir Reddy – spark support for SIMR</li> - <li>Josh Rosen – memory use optimization, clean up of BlockManager code, Java and Python clean-up/fixes</li> - <li>Henry Saputra – build fix</li> - <li>Jerry Shao – refactoring of fair scheduler, support for running Spark as a specific user, bug fix</li> - <li>Mingfei Shi – documentation for JobLogger</li> - <li>Andre Schumacher – sortByKey in PySpark and associated changes</li> - <li>Karthik Tunga – bug fix in launch script</li> - <li>Patrick Wendell – <code>repartition</code> operator, shuffle write metrics, various fixes and release management</li> - <li>Neal Wiggins – import clean-up, documentation fixes</li> - <li>Andrew Xia – bug fix in UI</li> - <li>Reynold Xin – task killing, support for setting job properties in Spark shell, logging improvements, Kryo improvements, several bug fixes</li> - <li>Matei Zaharia – optimized hashmap for shuffle data, PySpark documentation, optimizations to Kryo serializer</li> - <li>Wu Zeming – bug fix in executors UI</li> + <li>Michael Armbrust – build fix</li> + <li>Pierre Borckmans – typo fix in documentation</li> + <li>Evan Chan – <code>local://</code> scheme for dependency jars</li> + <li>Ewen Cheslack-Postava – <code>add</code> method for python accumulators, support for setting config properties in python</li> + <li>Mosharaf Chowdhury – optimized broadcast implementation</li> + <li>Frank Dai – documentation fix</li> + <li>Aaron Davidson – shuffle file consolidation, H/A mode for standalone scheduler, cleaned up representation of block IDs, several improvements and bug fixes</li> + <li>Tathagata Das – new streaming operators, fix for kafka concurrency bug</li> + <li>Ankur Dave – support for pausing spot clusters on EC2</li> + <li>Harvey Feng – optimization to JobConf broadcasts, bug fixes, YARN 2.2 build</li> + <li>Ali Ghodsi – YARN 2.2 build</li> + <li>Thomas Graves – Spark YARN integration including secure HDFS access over YARN</li> + <li>Li Guoqiang – fix for Maven build</li> + <li>Stephen Haberman – bug fix</li> + <li>Haidar Hadi – documentation fix</li> + <li>Nathan Howell – bug fix relating to YARN</li> + <li>Holden Karau – Java version of <code>mapPartitionsWithIndex</code></li> + <li>Du Li – bug fix in make-distrubion.sh</li> + <li>Raymond Liu – work on YARN 2.2 build</li> + <li>Xi Liu – bug fix and code clean-up</li> + <li>David McCauley – bug fix in standalone mode JSON output</li> + <li>Michael (wannabeast) – bug fix in memory store</li> + <li>Fabrizio Milo – typos in documentation, clean-up in DAGScheduler, typo in scaladoc</li> + <li>Mridul Muralidharan – fixes to metadata cleaner and speculative execution</li> + <li>Sundeep Narravula – build fix, bug fixes in scheduler and tests, code clean-up</li> + <li>Kay Ousterhout – optimized result fetching, new information in UI, scheduler clean-up and bug fixes</li> + <li>Nick Pentreath – implicit feedback variant of ALS algorithm</li> + <li>Imran Rashid – improvement to executor launch</li> + <li>Ahir Reddy – spark support for SIMR</li> + <li>Josh Rosen – memory use optimization, clean up of BlockManager code, Java and Python clean-up/fixes</li> + <li>Henry Saputra – build fix</li> + <li>Jerry Shao – refactoring of fair scheduler, support for running Spark as a specific user, bug fix</li> + <li>Mingfei Shi – documentation for JobLogger</li> + <li>Andre Schumacher – sortByKey in PySpark and associated changes</li> + <li>Karthik Tunga – bug fix in launch script</li> + <li>Patrick Wendell – <code>repartition</code> operator, shuffle write metrics, various fixes and release management</li> + <li>Neal Wiggins – import clean-up, documentation fixes</li> + <li>Andrew Xia – bug fix in UI</li> + <li>Reynold Xin – task killing, support for setting job properties in Spark shell, logging improvements, Kryo improvements, several bug fixes</li> + <li>Matei Zaharia – optimized hashmap for shuffle data, PySpark documentation, optimizations to Kryo serializer</li> + <li>Wu Zeming – bug fix in executors UI</li> </ul> <p>Thanks to everyone who contributed!</p> @@ -275,9 +280,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-9-0.html b/site/releases/spark-release-0-9-0.html index 341787473..9a84d4dce 100644 --- a/site/releases/spark-release-0-9-0.html +++ b/site/releases/spark-release-0-9-0.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -265,87 +270,87 @@ <p>The following developers contributed to this release:</p> <ul> - <li>Andrew Ash – documentation improvements</li> - <li>Pierre Borckmans – documentation fix</li> - <li>Russell Cardullo – graphite sink for metrics</li> - <li>Evan Chan – local:// URI feature</li> - <li>Vadim Chekan – bug fix</li> - <li>Lian Cheng – refactoring and code clean-up in several locations, bug fixes</li> - <li>Ewen Cheslack-Postava – Spark EC2 and PySpark improvements</li> - <li>Mosharaf Chowdhury – optimized broadcast</li> - <li>Dan Crankshaw – GraphX contributions</li> - <li>Haider Haidi – documentation fix</li> - <li>Frank Dai – Naive Bayes classifier in MLlib, documentation improvements</li> - <li>Tathagata Das – new operators, fixes, and improvements to Spark Streaming (lead)</li> - <li>Ankur Dave – GraphX contributions</li> - <li>Henry Davidge – warning for large tasks</li> - <li>Aaron Davidson – shuffle file consolidation, H/A mode for standalone scheduler, various improvements and fixes</li> - <li>Kyle Ellrott – GraphX contributions</li> - <li>Hossein Falaki – new statistical operators, Scala and Python examples in MLlib</li> - <li>Harvey Feng – hadoop file optimizations and YARN integration</li> - <li>Ali Ghodsi – support for SIMR</li> - <li>Joseph E. Gonzalez – GraphX contributions</li> - <li>Thomas Graves – fixes and improvements for YARN support (lead)</li> - <li>Rong Gu – documentation fix</li> - <li>Stephen Haberman – bug fixes</li> - <li>Walker Hamilton – bug fix</li> - <li>Mark Hamstra – scheduler improvements and fixes, build fixes</li> - <li>Damien Hardy – Debian build fix</li> - <li>Nathan Howell – sbt upgrade</li> - <li>Grace Huang – improvements to metrics code</li> - <li>Shane Huang – separation of admin and user scripts:</li> - <li>Prabeesh K – MQTT integration for Spark Streaming and code fix</li> - <li>Holden Karau – sbt build improvements and Java API extensions</li> - <li>KarthikTunga – bug fix</li> - <li>Grega Kespret – bug fix</li> - <li>Marek Kolodziej – optimized random number generator</li> - <li>Jey Kottalam – EC2 script improvements</li> - <li>Du Li – bug fixes</li> - <li>Haoyuan Li – tachyon support in EC2</li> - <li>LiGuoqiang – fixes to build and YARN integration</li> - <li>Raymond Liu – build improvement and various fixes for YARN support</li> - <li>George Loentiev – Maven build fixes</li> - <li>Akihiro Matsukawa – GraphX contributions</li> - <li>David McCauley – improvements to json endpoint</li> - <li>Mike – bug fixes</li> - <li>Fabrizio (Misto) Milo – bug fix</li> - <li>Mridul Muralidharan – speculation improvements, several bug fixes</li> - <li>Tor Myklebust – Python mllib bindings, instrumentation for task serailization</li> - <li>Sundeep Narravula – bug fix</li> - <li>Binh Nguyen – Java API improvements and version upgrades</li> - <li>Adam Novak – bug fix</li> - <li>Andrew Or – external sorting</li> - <li>Kay Ousterhout – several bug fixes and improvements to Spark scheduler</li> - <li>Sean Owen – style fixes</li> - <li>Nick Pentreath – ALS implicit feedback algorithm</li> - <li>Pillis – <code>Vector.random()</code> method</li> - <li>Imran Rashid – bug fix</li> - <li>Ahir Reddy – support for SIMR</li> - <li>Luca Rosellini – script loading for Scala shell</li> - <li>Josh Rosen – fixes, clean-up, and extensions to scala and Java API’s</li> - <li>Henry Saputra – style improvements and clean-up</li> - <li>Andre Schumacher – Python improvements and bug fixes</li> - <li>Jerry Shao – multi-user support, various fixes and improvements</li> - <li>Prashant Sharma – Scala 2.10 support, configuration system, several smaller fixes</li> - <li>Shiyun – style fix</li> - <li>Wangda Tan – UI improvement and bug fixes</li> - <li>Matthew Taylor – bug fix</li> - <li>Jyun-Fan Tsai – documentation fix</li> - <li>Takuya Ueshin – bug fix</li> - <li>Shivaram Venkataraman – sbt build optimization, EC2 improvements, Java and Python API</li> - <li>Jianping J Wang – GraphX contributions</li> - <li>Martin Weindel – build fix</li> - <li>Patrick Wendell – standalone driver submission, various fixes, release manager</li> - <li>Neal Wiggins – bug fix</li> - <li>Andrew Xia – bug fixes and code cleanup</li> - <li>Reynold Xin – GraphX contributions, task killing, various fixes, improvements and optimizations</li> - <li>Dong Yan – bug fix</li> - <li>Haitao Yao – bug fix</li> - <li>Xusen Yin – bug fix</li> - <li>Fengdong Yu – documentation fixes</li> - <li>Matei Zaharia – new configuration system, Python MLlib bindings, scheduler improvements, various fixes and optimizations</li> - <li>Wu Zeming – bug fix</li> - <li>Nan Zhu – documentation improvements</li> + <li>Andrew Ash – documentation improvements</li> + <li>Pierre Borckmans – documentation fix</li> + <li>Russell Cardullo – graphite sink for metrics</li> + <li>Evan Chan – local:// URI feature</li> + <li>Vadim Chekan – bug fix</li> + <li>Lian Cheng – refactoring and code clean-up in several locations, bug fixes</li> + <li>Ewen Cheslack-Postava – Spark EC2 and PySpark improvements</li> + <li>Mosharaf Chowdhury – optimized broadcast</li> + <li>Dan Crankshaw – GraphX contributions</li> + <li>Haider Haidi – documentation fix</li> + <li>Frank Dai – Naive Bayes classifier in MLlib, documentation improvements</li> + <li>Tathagata Das – new operators, fixes, and improvements to Spark Streaming (lead)</li> + <li>Ankur Dave – GraphX contributions</li> + <li>Henry Davidge – warning for large tasks</li> + <li>Aaron Davidson – shuffle file consolidation, H/A mode for standalone scheduler, various improvements and fixes</li> + <li>Kyle Ellrott – GraphX contributions</li> + <li>Hossein Falaki – new statistical operators, Scala and Python examples in MLlib</li> + <li>Harvey Feng – hadoop file optimizations and YARN integration</li> + <li>Ali Ghodsi – support for SIMR</li> + <li>Joseph E. Gonzalez – GraphX contributions</li> + <li>Thomas Graves – fixes and improvements for YARN support (lead)</li> + <li>Rong Gu – documentation fix</li> + <li>Stephen Haberman – bug fixes</li> + <li>Walker Hamilton – bug fix</li> + <li>Mark Hamstra – scheduler improvements and fixes, build fixes</li> + <li>Damien Hardy – Debian build fix</li> + <li>Nathan Howell – sbt upgrade</li> + <li>Grace Huang – improvements to metrics code</li> + <li>Shane Huang – separation of admin and user scripts:</li> + <li>Prabeesh K – MQTT integration for Spark Streaming and code fix</li> + <li>Holden Karau – sbt build improvements and Java API extensions</li> + <li>KarthikTunga – bug fix</li> + <li>Grega Kespret – bug fix</li> + <li>Marek Kolodziej – optimized random number generator</li> + <li>Jey Kottalam – EC2 script improvements</li> + <li>Du Li – bug fixes</li> + <li>Haoyuan Li – tachyon support in EC2</li> + <li>LiGuoqiang – fixes to build and YARN integration</li> + <li>Raymond Liu – build improvement and various fixes for YARN support</li> + <li>George Loentiev – Maven build fixes</li> + <li>Akihiro Matsukawa – GraphX contributions</li> + <li>David McCauley – improvements to json endpoint</li> + <li>Mike – bug fixes</li> + <li>Fabrizio (Misto) Milo – bug fix</li> + <li>Mridul Muralidharan – speculation improvements, several bug fixes</li> + <li>Tor Myklebust – Python mllib bindings, instrumentation for task serailization</li> + <li>Sundeep Narravula – bug fix</li> + <li>Binh Nguyen – Java API improvements and version upgrades</li> + <li>Adam Novak – bug fix</li> + <li>Andrew Or – external sorting</li> + <li>Kay Ousterhout – several bug fixes and improvements to Spark scheduler</li> + <li>Sean Owen – style fixes</li> + <li>Nick Pentreath – ALS implicit feedback algorithm</li> + <li>Pillis – <code>Vector.random()</code> method</li> + <li>Imran Rashid – bug fix</li> + <li>Ahir Reddy – support for SIMR</li> + <li>Luca Rosellini – script loading for Scala shell</li> + <li>Josh Rosen – fixes, clean-up, and extensions to scala and Java API’s</li> + <li>Henry Saputra – style improvements and clean-up</li> + <li>Andre Schumacher – Python improvements and bug fixes</li> + <li>Jerry Shao – multi-user support, various fixes and improvements</li> + <li>Prashant Sharma – Scala 2.10 support, configuration system, several smaller fixes</li> + <li>Shiyun – style fix</li> + <li>Wangda Tan – UI improvement and bug fixes</li> + <li>Matthew Taylor – bug fix</li> + <li>Jyun-Fan Tsai – documentation fix</li> + <li>Takuya Ueshin – bug fix</li> + <li>Shivaram Venkataraman – sbt build optimization, EC2 improvements, Java and Python API</li> + <li>Jianping J Wang – GraphX contributions</li> + <li>Martin Weindel – build fix</li> + <li>Patrick Wendell – standalone driver submission, various fixes, release manager</li> + <li>Neal Wiggins – bug fix</li> + <li>Andrew Xia – bug fixes and code cleanup</li> + <li>Reynold Xin – GraphX contributions, task killing, various fixes, improvements and optimizations</li> + <li>Dong Yan – bug fix</li> + <li>Haitao Yao – bug fix</li> + <li>Xusen Yin – bug fix</li> + <li>Fengdong Yu – documentation fixes</li> + <li>Matei Zaharia – new configuration system, Python MLlib bindings, scheduler improvements, various fixes and optimizations</li> + <li>Wu Zeming – bug fix</li> + <li>Nan Zhu – documentation improvements</li> </ul> <p><em>Thanks to everyone who contributed!</em></p> @@ -369,9 +374,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-9-1.html b/site/releases/spark-release-0-9-1.html index d75c6f9a2..637b07b32 100644 --- a/site/releases/spark-release-0-9-1.html +++ b/site/releases/spark-release-0-9-1.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -235,7 +240,7 @@ <li>Andrew Tulloch - Minor updates to MLLib</li> <li>Bijay Bisht - Fix for hadoop-client for Hadoop < 1.0.1 and for bug in Spark on Mesos + CDH4.5.0</li> <li>Bouke van der Bijl - Bug fix in Python depickling</li> - <li>Bryn Keller - Support for HBase’s TableOutputFormat</li> + <li>Bryn Keller - Support for HBase’s TableOutputFormat</li> <li>Chen Chao - Bug fix in spark-shell script, and improvements to streaming programming guide</li> <li>Christian Lundgren - Support for C3 EC2 instance type</li> <li>Diana Carroll - Improvements to PySpark programming guide</li> @@ -245,7 +250,7 @@ <li>jianghan - Bug fixes in Java examples</li> <li>Josh Rosen - Bug fix in PySpark string serialization and exception handling</li> <li>Jyotiska NK - Improvements to PySpark doc and examples</li> - <li>Kay Ousterhout - Multiple bug fixes in scheduler’s handling of task failures</li> + <li>Kay Ousterhout - Multiple bug fixes in scheduler’s handling of task failures</li> <li>Kousuke Saruta - Use of https to access github</li> <li>Mark Grover - Bug fix in distribution tar.gz</li> <li>Matei Zaharia - Bug fixes in handling of task failures due to NPE, and cleaning up of scheduler data structures </li> @@ -258,10 +263,10 @@ <li>Raymond Liu - Changed working directory in ZookeeperPersistenceEngine</li> <li>Reynold Xin - Improvements to docs and test infrastructure</li> <li>Sandy Ryza - Multiple important Yarn bug fixes and improvements</li> - <li>Sean Owen - Bug fixes and improvements for MLLib’s ALS</li> + <li>Sean Owen - Bug fixes and improvements for MLLib’s ALS</li> <li>Shixiong Zhu - Fixed thread-unsafe use of SimpleDateFormat</li> <li>shiyun.wxm - UI bug fix</li> - <li>Stevo Slavić - Bug fix in window’s run-example script</li> + <li>Stevo Slavić - Bug fix in window’s run-example script</li> <li>Tathagata Das - Improvements to streaming docs</li> <li>Tom Graves - Bug fixes in YARN deployment modes</li> <li>Xiangrui Meng - Improvements to ALS and GLM, and MLLib programming guide</li> @@ -287,9 +292,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-0-9-2.html b/site/releases/spark-release-0-9-2.html index 65c47e222..109abe291 100644 --- a/site/releases/spark-release-0-9-2.html +++ b/site/releases/spark-release-0-9-2.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -175,7 +180,7 @@ <h4 id="spark-core">Spark Core</h4> <ul> - <li>ExternalAppendOnlyMap doesn’t always find matching keys. (<a href="https://issues.apache.org/jira/browse/SPARK-2043">SPARK-2043</a>)</li> + <li>ExternalAppendOnlyMap doesn’t always find matching keys. (<a href="https://issues.apache.org/jira/browse/SPARK-2043">SPARK-2043</a>)</li> <li>Jobs hang due to akka frame size settings. (<a href="https://issues.apache.org/jira/browse/SPARK-1112">SPARK-1112</a>, <a href="https://issues.apache.org/jira/browse/SPARK-2156">SPARK-2156</a>)</li> <li>HDFS FileSystems continually pile up in the FS cache. (<a href="https://issues.apache.org/jira/browse/SPARK-1676">SPARK-1676</a>)</li> <li>Unneeded lock in ShuffleMapTask.deserializeInfo. (<a href="https://issues.apache.org/jira/browse/SPARK-1775">SPARK-1775</a>)</li> @@ -184,15 +189,15 @@ <h4 id="pyspark">PySpark</h4> <ul> - <li>IPython won’t run standalone Python script. (<a href="https://issues.apache.org/jira/browse/SPARK-1134">SPARK-1134</a>)</li> - <li>The hash method used by partitionBy doesn’t deal with None correctly. (<a href="https://issues.apache.org/jira/browse/SPARK-1468">SPARK-1468</a>)</li> + <li>IPython won’t run standalone Python script. (<a href="https://issues.apache.org/jira/browse/SPARK-1134">SPARK-1134</a>)</li> + <li>The hash method used by partitionBy doesn’t deal with None correctly. (<a href="https://issues.apache.org/jira/browse/SPARK-1468">SPARK-1468</a>)</li> <li>PySpark crashes if too many tasks complete quickly. (<a href="https://issues.apache.org/jira/browse/SPARK-2282">SPARK-2282</a>)</li> </ul> <h4 id="mllib">MLlib</h4> <ul> <li>Make MLlib work on Python 2.6. (<a href="https://issues.apache.org/jira/browse/SPARK-1421">SPARK-1421</a>)</li> - <li>Fix PySpark’s Naive Bayes implementation. (<a href="https://issues.apache.org/jira/browse/SPARK-2433">SPARK-2433</a>)</li> + <li>Fix PySpark’s Naive Bayes implementation. (<a href="https://issues.apache.org/jira/browse/SPARK-2433">SPARK-2433</a>)</li> </ul> <h4 id="streaming">Streaming</h4> @@ -260,9 +265,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-1-0-0.html b/site/releases/spark-release-1-0-0.html index 76649de87..a3dc54131 100644 --- a/site/releases/spark-release-1-0-0.html +++ b/site/releases/spark-release-1-0-0.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -192,11 +197,11 @@ <p>Spark 1.0 adds support for Java 8 <a href="http://docs.oracle.com/javase/tutorial/java/javaOO/lambdaexpressions.html">new lambda syntax</a> in its Java bindings. Java 8 supports a concise syntax for writing anonymous functions, similar to the closure syntax in Scala and Python. This change requires small changes for users of the current Java API, which are noted in the documentation. Spark’s Python API has been extended to support several new functions. We’ve also included several stability improvements in the Python API, particularly for large datasets. PySpark now supports running on YARN as well.</p> <h3 id="documentation">Documentation</h3> -<p>Spark’s <a href="/docs/latest/programming-guide.html">programming guide</a> has been significantly expanded to centrally cover all supported languages and discuss more operators and aspects of the development life cycle. The <a href="/docs/latest/mllib-guide.html">MLlib guide</a> has also been expanded with significantly more detail and examples for each algorithm, while documents on configuration, YARN and Mesos have also been revamped.</p> +<p>Spark’s <a href="/docs/latest/programming-guide.html">programming guide</a> has been significantly expanded to centrally cover all supported languages and discuss more operators and aspects of the development life cycle. The <a href="/docs/latest/mllib-guide.html">MLlib guide</a> has also been expanded with significantly more detail and examples for each algorithm, while documents on configuration, YARN and Mesos have also been revamped.</p> <h3 id="smaller-changes">Smaller Changes</h3> <ul> - <li>PySpark now works with more Python versions than before – Python 2.6+ instead of 2.7+, and NumPy 1.4+ instead of 1.7+.</li> + <li>PySpark now works with more Python versions than before – Python 2.6+ instead of 2.7+, and NumPy 1.4+ instead of 1.7+.</li> <li>Spark has upgraded to Avro 1.7.6, adding support for Avro specific types.</li> <li>Internal instrumentation has been added to allow applications to monitor and instrument Spark jobs.</li> <li>Support for off-heap storage in Tachyon has been added via a special build target.</li> @@ -213,123 +218,123 @@ <p>The following developers contributed to this release:</p> <ul> - <li>Aaron Davidson – packaging and deployment improvements, several bug fixes, local[*] mode</li> - <li>Aaron Kimball – documentation improvements</li> - <li>Abhishek Kumar – Python configuration fixes</li> - <li>Ahir Reddy – PySpark build, fixes, and cancellation support</li> - <li>Allan Douglas R. de Oliveira – Improvements to spark-ec2 scripts</li> - <li>Andre Schumacher – Parquet support and optimizations</li> - <li>Andrew Ash – Mesos documentation and other doc improvements, bug fixes</li> - <li>Andrew Or – history server (lead), garbage collection (lead), spark-submit, PySpark and YARN improvements</li> - <li>Andrew Tulloch – MLlib contributions and code clean-up</li> - <li>Andy Konwinski – documentation fix</li> - <li>Anita Tailor – Cassandra example</li> - <li>Ankur Dave – GraphX (lead) optimizations, documentation, and usability</li> - <li>Archer Shao – bug fixes</li> - <li>Arun Ramakrishnan – improved random sampling</li> - <li>Baishuo – test improvements</li> - <li>Bernardo Gomez Palacio – spark-shell improvements and Mesos updates</li> - <li>Bharath Bhushan – bug fix</li> - <li>Bijay Bisht – bug fixes</li> - <li>Binh Nguyen – dependency fix</li> - <li>Bouke van der Bijl – fixes for PySpark on Mesos and other Mesos fixes</li> - <li>Bryn Keller – improvement to HBase support and unit tests</li> - <li>Chen Chao – documentation, bug fix, and code clean-up</li> - <li>Cheng Hao – performance and feature improvements in Spark SQL</li> - <li>Cheng Lian – column storage and other improvements in Spark SQL</li> - <li>Christian Lundgren – improvement to spark-ec2 scripts</li> - <li>DB Tsai – L-BGFS optimizer in MLlib, MLlib documentation and fixes</li> - <li>Dan McClary – Improvement to stats counter</li> - <li>Daniel Darabos – GraphX performance improvement</li> - <li>Davis Shepherd – bug fix</li> - <li>Diana Carroll – documentation and bug fix</li> - <li>Egor Pakhomov – local iterator for RDD’s</li> - <li>Emtiaz Ahmed – bug fix</li> - <li>Erik Selin – bug fix</li> - <li>Ethan Jewett – documentation improvement</li> - <li>Evan Chan – automatic clean-up of application data</li> - <li>Evan Sparks – MLlib optimizations and doc improvement</li> - <li>Frank Dai – code clean-up in MLlib</li> - <li>Guoquiang Li – build improvements and several bug fixes</li> - <li>Ghidireac – bug fix</li> - <li>Haoyuan Li – Tachyon storage level for RDD’s</li> - <li>Harvey Feng – spark-ec2 update</li> - <li>Henry Saputra – code clean-up</li> - <li>Henry Cook – Spark SQL improvements</li> - <li>Holden Karau – cross validation in MLlib, Python and core engine improvements</li> - <li>Ivan Wick – Mesos bug fix</li> - <li>Jey Kottalam – sbt build improvement</li> - <li>Jerry Shao – Spark metrics and Spark SQL improvements</li> - <li>Jiacheng Guo – bug fix</li> - <li>Jianghan – bug fix</li> - <li>Jianping J Wang – JBLAS support in MLlib</li> - <li>Joseph E. Gonzalez – GraphX improvements, fixes, and documentation</li> - <li>Josh Rosen – PySpark improvements and bug fixes</li> - <li>Jyotiska NK – documentation, test improvements, and bug fix</li> - <li>Kan Zhang – bug fixes in Spark core, SQL, and PySpark</li> - <li>Kay Ousterhout – bug fixes and code refactoring in scheduler</li> - <li>Kelvin Chu – automatic clean-up of application data</li> - <li>Kevin Mader – example fix</li> - <li>Koert Kuipers – code visibility fix</li> - <li>Kousuke Saruta – documentation and build fixes</li> - <li>Kyle Ellrott – improved memory usage for DISK_ONLY persistence</li> - <li>Larva Boy – approximate counts in Spark SQL</li> - <li>Madhu Siddalingaiah – ec2 fixes</li> - <li>Manish Amde – decision trees in MLlib</li> - <li>Marcelo Vanzin – improvements and fixes to YARN support, dependency clean-up</li> - <li>Mark Grover – build fixes</li> - <li>Mark Hamstra – build and dependency improvements, scheduler bug fixes</li> - <li>Margin Jaggi – MLlib documentation improvements</li> - <li>Matei Zaharia – Python versions of several MLlib algorithms, spark-submit improvements, bug fixes, and documentation improvements</li> - <li>Michael Armbrust – Spark SQL (lead), including schema support for RDD’s, catalyst optimizer, and Hive support</li> - <li>Mridul Muralidharan – code visibility changes and bug fixes</li> - <li>Nan Zhu – bug and stability fixes, code clean-up, documentation, and new features</li> - <li>Neville Li – bug fix</li> - <li>Nick Lanham – Tachyon bundling in distribution script</li> - <li>Nirmal Reddy – code clean-up</li> - <li>OuYang Jin – local mode and json improvements</li> - <li>Patrick Wendell – release manager, build improvements, bug fixes, and code clean-up</li> - <li>Petko Nikolov – new utility functions</li> - <li>Prabeesh K – typo fix</li> - <li>Prabin Banka – new PySpark API’s</li> - <li>Prashant Sharma – PySpark improvements, Java 8 lambda support, and build improvements</li> - <li>Punya Biswal – Java API improvements</li> - <li>Qiuzhuang Lian – bug fixes</li> - <li>Rahul Singhal – build improvements, bug fixes</li> - <li>Raymond Liu – YARN build fixes and UI improvements</li> - <li>Reynold Xin – bug fixes, internal changes, Spark SQL improvements, build fixes, and style improvements</li> - <li>Reza Zadeh – SVD implementation in MLlib and other MLlib contributions</li> - <li>Roman Pastukhov – clean-up of broadcast files</li> - <li>Rong Gu – Tachyon storage level for RDD’s</li> - <li>Sandeep Sing – several bug fixes, MLLib improvements and fixes to Spark examples</li> - <li>Sandy Ryza – spark-submit script and several YARN improvements</li> - <li>Saurabh Rawat – Java API improvements</li> - <li>Sean Owen – several build improvements, code clean-up, and MLlib fixes</li> - <li>Semih Salihoglu – GraphX improvements</li> - <li>Shaocun Tian – bug fix in MLlib</li> - <li>Shivaram Venkataraman – bug fixes</li> - <li>Shixiong Zhu – code style and correctness fixes</li> - <li>Shiyun Wxm – typo fix</li> - <li>Stevo Slavic – bug fix</li> - <li>Sumedh Mungee – documentation fix</li> - <li>Sundeep Narravula – “cancel” button in Spark UI</li> - <li>Takayu Ueshin – bug fixes and improvements to Spark SQL</li> - <li>Tathagata Das – web UI and other improvements to Spark Streaming (lead), bug fixes, state clean-up, and release manager</li> - <li>Timothy Chen – Spark SQL improvements</li> - <li>Ted Malaska – improved Flume support</li> - <li>Tom Graves – Hadoop security integration (lead) and YARN support</li> - <li>Tianshuo Deng – Bug fix</li> - <li>Tor Myklebust – improvements to ALS</li> - <li>Wangfei – Spark SQL docs</li> - <li>Wang Tao – code clean-up</li> - <li>William Bendon – JSON support changes and bug fixes</li> - <li>Xiangrui Meng – several improvements to MLlib (lead)</li> - <li>Xuan Nguyen – build fix</li> - <li>Xusen Yin – MLlib contributions and bug fix</li> - <li>Ye Xianjin – test fixes</li> - <li>Yinan Li – addFile improvement</li> - <li>Yin Hua – Spark SQL improvements</li> - <li>Zheng Peng – bug fixes</li> + <li>Aaron Davidson – packaging and deployment improvements, several bug fixes, local[*] mode</li> + <li>Aaron Kimball – documentation improvements</li> + <li>Abhishek Kumar – Python configuration fixes</li> + <li>Ahir Reddy – PySpark build, fixes, and cancellation support</li> + <li>Allan Douglas R. de Oliveira – Improvements to spark-ec2 scripts</li> + <li>Andre Schumacher – Parquet support and optimizations</li> + <li>Andrew Ash – Mesos documentation and other doc improvements, bug fixes</li> + <li>Andrew Or – history server (lead), garbage collection (lead), spark-submit, PySpark and YARN improvements</li> + <li>Andrew Tulloch – MLlib contributions and code clean-up</li> + <li>Andy Konwinski – documentation fix</li> + <li>Anita Tailor – Cassandra example</li> + <li>Ankur Dave – GraphX (lead) optimizations, documentation, and usability</li> + <li>Archer Shao – bug fixes</li> + <li>Arun Ramakrishnan – improved random sampling</li> + <li>Baishuo – test improvements</li> + <li>Bernardo Gomez Palacio – spark-shell improvements and Mesos updates</li> + <li>Bharath Bhushan – bug fix</li> + <li>Bijay Bisht – bug fixes</li> + <li>Binh Nguyen – dependency fix</li> + <li>Bouke van der Bijl – fixes for PySpark on Mesos and other Mesos fixes</li> + <li>Bryn Keller – improvement to HBase support and unit tests</li> + <li>Chen Chao – documentation, bug fix, and code clean-up</li> + <li>Cheng Hao – performance and feature improvements in Spark SQL</li> + <li>Cheng Lian – column storage and other improvements in Spark SQL</li> + <li>Christian Lundgren – improvement to spark-ec2 scripts</li> + <li>DB Tsai – L-BGFS optimizer in MLlib, MLlib documentation and fixes</li> + <li>Dan McClary – Improvement to stats counter</li> + <li>Daniel Darabos – GraphX performance improvement</li> + <li>Davis Shepherd – bug fix</li> + <li>Diana Carroll – documentation and bug fix</li> + <li>Egor Pakhomov – local iterator for RDD’s</li> + <li>Emtiaz Ahmed – bug fix</li> + <li>Erik Selin – bug fix</li> + <li>Ethan Jewett – documentation improvement</li> + <li>Evan Chan – automatic clean-up of application data</li> + <li>Evan Sparks – MLlib optimizations and doc improvement</li> + <li>Frank Dai – code clean-up in MLlib</li> + <li>Guoquiang Li – build improvements and several bug fixes</li> + <li>Ghidireac – bug fix</li> + <li>Haoyuan Li – Tachyon storage level for RDD’s</li> + <li>Harvey Feng – spark-ec2 update</li> + <li>Henry Saputra – code clean-up</li> + <li>Henry Cook – Spark SQL improvements</li> + <li>Holden Karau – cross validation in MLlib, Python and core engine improvements</li> + <li>Ivan Wick – Mesos bug fix</li> + <li>Jey Kottalam – sbt build improvement</li> + <li>Jerry Shao – Spark metrics and Spark SQL improvements</li> + <li>Jiacheng Guo – bug fix</li> + <li>Jianghan – bug fix</li> + <li>Jianping J Wang – JBLAS support in MLlib</li> + <li>Joseph E. Gonzalez – GraphX improvements, fixes, and documentation</li> + <li>Josh Rosen – PySpark improvements and bug fixes</li> + <li>Jyotiska NK – documentation, test improvements, and bug fix</li> + <li>Kan Zhang – bug fixes in Spark core, SQL, and PySpark</li> + <li>Kay Ousterhout – bug fixes and code refactoring in scheduler</li> + <li>Kelvin Chu – automatic clean-up of application data</li> + <li>Kevin Mader – example fix</li> + <li>Koert Kuipers – code visibility fix</li> + <li>Kousuke Saruta – documentation and build fixes</li> + <li>Kyle Ellrott – improved memory usage for DISK_ONLY persistence</li> + <li>Larva Boy – approximate counts in Spark SQL</li> + <li>Madhu Siddalingaiah – ec2 fixes</li> + <li>Manish Amde – decision trees in MLlib</li> + <li>Marcelo Vanzin – improvements and fixes to YARN support, dependency clean-up</li> + <li>Mark Grover – build fixes</li> + <li>Mark Hamstra – build and dependency improvements, scheduler bug fixes</li> + <li>Margin Jaggi – MLlib documentation improvements</li> + <li>Matei Zaharia – Python versions of several MLlib algorithms, spark-submit improvements, bug fixes, and documentation improvements</li> + <li>Michael Armbrust – Spark SQL (lead), including schema support for RDD’s, catalyst optimizer, and Hive support</li> + <li>Mridul Muralidharan – code visibility changes and bug fixes</li> + <li>Nan Zhu – bug and stability fixes, code clean-up, documentation, and new features</li> + <li>Neville Li – bug fix</li> + <li>Nick Lanham – Tachyon bundling in distribution script</li> + <li>Nirmal Reddy – code clean-up</li> + <li>OuYang Jin – local mode and json improvements</li> + <li>Patrick Wendell – release manager, build improvements, bug fixes, and code clean-up</li> + <li>Petko Nikolov – new utility functions</li> + <li>Prabeesh K – typo fix</li> + <li>Prabin Banka – new PySpark API’s</li> + <li>Prashant Sharma – PySpark improvements, Java 8 lambda support, and build improvements</li> + <li>Punya Biswal – Java API improvements</li> + <li>Qiuzhuang Lian – bug fixes</li> + <li>Rahul Singhal – build improvements, bug fixes</li> + <li>Raymond Liu – YARN build fixes and UI improvements</li> + <li>Reynold Xin – bug fixes, internal changes, Spark SQL improvements, build fixes, and style improvements</li> + <li>Reza Zadeh – SVD implementation in MLlib and other MLlib contributions</li> + <li>Roman Pastukhov – clean-up of broadcast files</li> + <li>Rong Gu – Tachyon storage level for RDD’s</li> + <li>Sandeep Sing – several bug fixes, MLLib improvements and fixes to Spark examples</li> + <li>Sandy Ryza – spark-submit script and several YARN improvements</li> + <li>Saurabh Rawat – Java API improvements</li> + <li>Sean Owen – several build improvements, code clean-up, and MLlib fixes</li> + <li>Semih Salihoglu – GraphX improvements</li> + <li>Shaocun Tian – bug fix in MLlib</li> + <li>Shivaram Venkataraman – bug fixes</li> + <li>Shixiong Zhu – code style and correctness fixes</li> + <li>Shiyun Wxm – typo fix</li> + <li>Stevo Slavic – bug fix</li> + <li>Sumedh Mungee – documentation fix</li> + <li>Sundeep Narravula – “cancel” button in Spark UI</li> + <li>Takayu Ueshin – bug fixes and improvements to Spark SQL</li> + <li>Tathagata Das – web UI and other improvements to Spark Streaming (lead), bug fixes, state clean-up, and release manager</li> + <li>Timothy Chen – Spark SQL improvements</li> + <li>Ted Malaska – improved Flume support</li> + <li>Tom Graves – Hadoop security integration (lead) and YARN support</li> + <li>Tianshuo Deng – Bug fix</li> + <li>Tor Myklebust – improvements to ALS</li> + <li>Wangfei – Spark SQL docs</li> + <li>Wang Tao – code clean-up</li> + <li>William Bendon – JSON support changes and bug fixes</li> + <li>Xiangrui Meng – several improvements to MLlib (lead)</li> + <li>Xuan Nguyen – build fix</li> + <li>Xusen Yin – MLlib contributions and bug fix</li> + <li>Ye Xianjin – test fixes</li> + <li>Yinan Li – addFile improvement</li> + <li>Yin Hua – Spark SQL improvements</li> + <li>Zheng Peng – bug fixes</li> </ul> <p><em>Thanks to everyone who contributed!</em></p> @@ -353,9 +358,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-1-0-1.html b/site/releases/spark-release-1-0-1.html index 27f0ff5df..b7ab1f8a9 100644 --- a/site/releases/spark-release-1-0-1.html +++ b/site/releases/spark-release-1-0-1.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -218,76 +223,76 @@ <p>The following developers contributed to this release:</p> <ul> - <li>Aaron Davidson – bug fixes in PySpark and Spark core</li> - <li>Ali Ghodsi – documentation update</li> - <li>Anant – compatibility fix for spark-ec2 script</li> - <li>Anatoli Fomenko – MLlib doc fix</li> - <li>Andre Schumacher – nested Parquet data</li> - <li>Andrew Ash – documentation</li> - <li>Andrew Or – bug fixes and documentation</li> - <li>Ankur Dave – bug fixes</li> - <li>Arkadiusz Komarzewski – doc fix</li> - <li>Baishuo – sql fix</li> - <li>Chen Chao – comment fix and bug fix</li> - <li>Cheng Hao – SQL features</li> - <li>Cheng Lian – SQL features</li> - <li>Christian Tzolov – build improvmenet</li> - <li>Clément MATHIEU – doc updates </li> - <li>CodingCat – doc updates and bug fix </li> - <li>Colin McCabe – bug fix</li> - <li>Daoyuan – SQL joins</li> - <li>David Lemieux – bug fix</li> - <li>Derek Ma – bug fix</li> - <li>Doris Xin – bug fix</li> - <li>Erik Selin – PySpark fix</li> - <li>Gang Bai – bug fix</li> - <li>Guoqiang Li – bug fixes</li> - <li>Henry Saputra – documentation</li> - <li>Jiang – doc fix</li> - <li>Joy Yoj – bug fix</li> - <li>Jyotiska NK – test improvement</li> - <li>Kan Zhang – PySpark SQL features</li> - <li>Kay Ousterhout – documentation fix</li> - <li>LY Lai – bug fix</li> - <li>Lars Albertsson – bug fix </li> - <li>Lei Zhang – SQL fix and feature</li> - <li>Mark Hamstra – bug fix</li> - <li>Matei Zaharia – doc updates and bug fix</li> - <li>Matthew Farrellee – bug fixes</li> - <li>Michael Armbrust – sql features and fixes</li> - <li>Neville Li – buf fix</li> - <li>Nick Chammas – doc fix</li> - <li>Ori Kremer – bug fix</li> - <li>Patrick Wendell – documentation and release manager</li> - <li>Prashant Sharma – bug and doc fixes</li> - <li>Qiuzhuang.Lian – bug fix</li> - <li>Raymond Liu – bug fix</li> - <li>Ravikanth Nawada – bug fixes</li> - <li>Reynold Xin – bug and doc fixes</li> - <li>Sameer Agarwal – optimization</li> - <li>Sandy Ryza – doc fix</li> - <li>Sean Owen – bug fix</li> - <li>Sebastien Rainville – bug fix</li> - <li>Shixiong Zhu – code clean-up</li> - <li>Szul, Piotr – bug fix</li> - <li>Takuya UESHIN – bug fixes and SQL features</li> - <li>Thomas Graves – bug fix </li> - <li>Uri Laserson – bug fix</li> - <li>Vadim Chekan – bug fix</li> - <li>Varakhedi Sujeet – ec2 r3 support</li> - <li>Vlad – doc fix</li> - <li>Wang Lianhui – bug fix</li> - <li>Wenchen Fan – optimization</li> - <li>William Benton – SQL feature</li> - <li>Xi Liu – SQL feature</li> - <li>Xiangrui Meng – bug fix</li> - <li>Ximo Guanter Gonzalbez – SQL feature</li> - <li>Yadid Ayzenberg – doc fix</li> - <li>Yijie Shen – buf fix</li> - <li>Yin Huai – JSON support and bug fixes</li> - <li>Zhen Peng – bug fix</li> - <li>Zichuan Ye – ec2 fixes</li> - <li>Zongheng Yang – sql fixes</li> + <li>Aaron Davidson – bug fixes in PySpark and Spark core</li> + <li>Ali Ghodsi – documentation update</li> + <li>Anant – compatibility fix for spark-ec2 script</li> + <li>Anatoli Fomenko – MLlib doc fix</li> + <li>Andre Schumacher – nested Parquet data</li> + <li>Andrew Ash – documentation</li> + <li>Andrew Or – bug fixes and documentation</li> + <li>Ankur Dave – bug fixes</li> + <li>Arkadiusz Komarzewski – doc fix</li> + <li>Baishuo – sql fix</li> + <li>Chen Chao – comment fix and bug fix</li> + <li>Cheng Hao – SQL features</li> + <li>Cheng Lian – SQL features</li> + <li>Christian Tzolov – build improvmenet</li> + <li>Clément MATHIEU – doc updates </li> + <li>CodingCat – doc updates and bug fix </li> + <li>Colin McCabe – bug fix</li> + <li>Daoyuan – SQL joins</li> + <li>David Lemieux – bug fix</li> + <li>Derek Ma – bug fix</li> + <li>Doris Xin – bug fix</li> + <li>Erik Selin – PySpark fix</li> + <li>Gang Bai – bug fix</li> + <li>Guoqiang Li – bug fixes</li> + <li>Henry Saputra – documentation</li> + <li>Jiang – doc fix</li> + <li>Joy Yoj – bug fix</li> + <li>Jyotiska NK – test improvement</li> + <li>Kan Zhang – PySpark SQL features</li> + <li>Kay Ousterhout – documentation fix</li> + <li>LY Lai – bug fix</li> + <li>Lars Albertsson – bug fix </li> + <li>Lei Zhang – SQL fix and feature</li> + <li>Mark Hamstra – bug fix</li> + <li>Matei Zaharia – doc updates and bug fix</li> + <li>Matthew Farrellee – bug fixes</li> + <li>Michael Armbrust – sql features and fixes</li> + <li>Neville Li – buf fix</li> + <li>Nick Chammas – doc fix</li> + <li>Ori Kremer – bug fix</li> + <li>Patrick Wendell – documentation and release manager</li> + <li>Prashant Sharma – bug and doc fixes</li> + <li>Qiuzhuang.Lian – bug fix</li> + <li>Raymond Liu – bug fix</li> + <li>Ravikanth Nawada – bug fixes</li> + <li>Reynold Xin – bug and doc fixes</li> + <li>Sameer Agarwal – optimization</li> + <li>Sandy Ryza – doc fix</li> + <li>Sean Owen – bug fix</li> + <li>Sebastien Rainville – bug fix</li> + <li>Shixiong Zhu – code clean-up</li> + <li>Szul, Piotr – bug fix</li> + <li>Takuya UESHIN – bug fixes and SQL features</li> + <li>Thomas Graves – bug fix </li> + <li>Uri Laserson – bug fix</li> + <li>Vadim Chekan – bug fix</li> + <li>Varakhedi Sujeet – ec2 r3 support</li> + <li>Vlad – doc fix</li> + <li>Wang Lianhui – bug fix</li> + <li>Wenchen Fan – optimization</li> + <li>William Benton – SQL feature</li> + <li>Xi Liu – SQL feature</li> + <li>Xiangrui Meng – bug fix</li> + <li>Ximo Guanter Gonzalbez – SQL feature</li> + <li>Yadid Ayzenberg – doc fix</li> + <li>Yijie Shen – buf fix</li> + <li>Yin Huai – JSON support and bug fixes</li> + <li>Zhen Peng – bug fix</li> + <li>Zichuan Ye – ec2 fixes</li> + <li>Zongheng Yang – sql fixes</li> </ul> <p><em>Thanks to everyone who contributed!</em></p> @@ -311,9 +316,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/releases/spark-release-1-0-2.html b/site/releases/spark-release-1-0-2.html index ac126d23f..ebebc85ab 100644 --- a/site/releases/spark-release-1-0-2.html +++ b/site/releases/spark-release-1-0-2.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -176,7 +181,7 @@ <h4 id="spark-core">Spark Core</h4> <ul> <li>Avoid pulling in the entire RDD or PairRDDFunctions in various operators (<a href="https://issues.apache.org/jira/browse/SPARK-2534">SPARK-2534</a>)</li> - <li>RangePartitioner’s binary search does not use the given Ordering (<a href="https://issues.apache.org/jira/browse/SPARK-2598">SPARK-2598</a>)</li> + <li>RangePartitioner’s binary search does not use the given Ordering (<a href="https://issues.apache.org/jira/browse/SPARK-2598">SPARK-2598</a>)</li> <li>Exception in accumulator update should not crash DAGScheduler and SparkContext (<a href="https://issues.apache.org/jira/browse/SPARK-2323">SPARK-2323</a>)</li> </ul> @@ -268,9 +273,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/research.html b/site/research.html index 78ce22d4a..77c80794e 100644 --- a/site/research.html +++ b/site/research.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -217,9 +222,5 @@ Spark offers an abstraction called <a href="http://www.cs.berkeley.edu/~matei/pa </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/screencasts/1-first-steps-with-spark.html b/site/screencasts/1-first-steps-with-spark.html index 8b650b5d0..db3565cf9 100644 --- a/site/screencasts/1-first-steps-with-spark.html +++ b/site/screencasts/1-first-steps-with-spark.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -166,7 +171,7 @@ <li>Introduce the API using the Spark interactive shell to explore a file.</li> </ol> -<div class="video-container video-square shadow"><iframe width="755" height="705" src="//www.youtube.com/embed/bWorBGOFBWY?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> +<div class="video-container video-square shadow"><iframe width="755" height="705" src="http://www.youtube.com/embed/bWorBGOFBWY?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> <p>Check out the next spark screencast in the series, <a href="/screencasts/2-spark-documentation-overview.html">Spark Screencast #2 - Overview of Spark Documentation</a>.</p> @@ -191,9 +196,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/screencasts/2-spark-documentation-overview.html b/site/screencasts/2-spark-documentation-overview.html index a6843d92c..633a8046f 100644 --- a/site/screencasts/2-spark-documentation-overview.html +++ b/site/screencasts/2-spark-documentation-overview.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -162,7 +167,7 @@ <p>This is our 2nd Spark screencast. In it, we take a tour of the documentation available for Spark users online.</p> -<div class="video-container video-square shadow"><iframe width="755" height="705" src="//www.youtube.com/embed/Dbqe_rv-NJQ?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> +<div class="video-container video-square shadow"><iframe width="755" height="705" src="http://www.youtube.com/embed/Dbqe_rv-NJQ?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> <p>Check out the next spark screencast in the series, <a href="/screencasts/3-transformations-and-caching.html">Spark Screencast #3 - Transformations and Caching</a>.</p> @@ -191,9 +196,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/screencasts/3-transformations-and-caching.html b/site/screencasts/3-transformations-and-caching.html index 62eada53d..f9283b605 100644 --- a/site/screencasts/3-transformations-and-caching.html +++ b/site/screencasts/3-transformations-and-caching.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -162,7 +167,7 @@ <p>In this third Spark screencast, we demonstrate more advanced use of RDD actions and transformations, as well as caching RDDs in memory.</p> -<div class="video-container video-square shadow"><iframe width="755" height="705" src="//www.youtube.com/embed/TtvxKzO9jXE?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> +<div class="video-container video-square shadow"><iframe width="755" height="705" src="http://www.youtube.com/embed/TtvxKzO9jXE?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> <p>Check out the next spark screencast in the series, <a href="/screencasts/4-a-standalone-job-in-spark.html">Spark Screencast #4 - A Standalone Job in Scala</a>.</p> @@ -187,9 +192,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/screencasts/4-a-standalone-job-in-spark.html b/site/screencasts/4-a-standalone-job-in-spark.html index 25f6a5c72..af2c9b17a 100644 --- a/site/screencasts/4-a-standalone-job-in-spark.html +++ b/site/screencasts/4-a-standalone-job-in-spark.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -162,7 +167,7 @@ <p>In this Spark screencast, we create a standalone Apache Spark job in Scala. In the job, we create a spark context and read a file into an RDD of strings; then apply transformations and actions to the RDD and print out the results.</p> -<div class="video-container video-16x9 shadow"><iframe width="755" height="425" src="//www.youtube.com/embed/GaBn-YjlR8Q?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> +<div class="video-container video-16x9 shadow"><iframe width="755" height="425" src="http://www.youtube.com/embed/GaBn-YjlR8Q?autohide=0&showinfo=0&list=PL-x35fyliRwhKT-NpTKprPW1bkbdDcTTW" frameborder="0" allowfullscreen=""></iframe></div> <p>For more information and links to other Spark screencasts, check out the <a href="/documentation.html">Spark documentation page</a>.</p> @@ -185,9 +190,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/screencasts/index.html b/site/screencasts/index.html index 45d7f10dd..7c7f1ae85 100644 --- a/site/screencasts/index.html +++ b/site/screencasts/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -217,9 +222,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/sql/index.html b/site/sql/index.html index 7a6dcfe5f..08934e0bb 100644 --- a/site/sql/index.html +++ b/site/sql/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -338,9 +343,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> diff --git a/site/streaming/index.html b/site/streaming/index.html index 4aa38d7ac..e6251386d 100644 --- a/site/streaming/index.html +++ b/site/streaming/index.html @@ -48,6 +48,11 @@ <body> +<script src="https://code.jquery.com/jquery.js"></script> +<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> +<script src="/js/lang-tabs.js"></script> +<script src="/js/downloads.js"></script> + <div class="container" style="max-width: 1200px;"> <div class="masthead"> @@ -313,9 +318,5 @@ </div> -<script src="https://code.jquery.com/jquery.js"></script> -<script src="//netdna.bootstrapcdn.com/bootstrap/3.0.3/js/bootstrap.min.js"></script> -<script src="/js/lang-tabs.js"></script> - </body> </html> |