diff options
author | Andy Konwinski <andrew@apache.org> | 2013-08-23 17:17:53 +0000 |
---|---|---|
committer | Andy Konwinski <andrew@apache.org> | 2013-08-23 17:17:53 +0000 |
commit | 81d6089b47ec4d3e7fe17074f3b5fadec8070071 (patch) | |
tree | 1401e9f4bc6e1b9f4596ebecc5b7332d9ed96f3a /site/index.html | |
parent | 71bac61ea11df8144a9a3d2be75ef996517b136d (diff) | |
download | spark-website-81d6089b47ec4d3e7fe17074f3b5fadec8070071.tar.gz spark-website-81d6089b47ec4d3e7fe17074f3b5fadec8070071.tar.bz2 spark-website-81d6089b47ec4d3e7fe17074f3b5fadec8070071.zip |
Initial port of Spark website from spark-project.org wordpress to Jekyll.
Diffstat (limited to 'site/index.html')
-rw-r--r-- | site/index.html | 182 |
1 files changed, 181 insertions, 1 deletions
diff --git a/site/index.html b/site/index.html index 470c467a5..cfd0f4da7 100644 --- a/site/index.html +++ b/site/index.html @@ -1 +1,181 @@ -<html><head><title>Apache Spark</title></head><body><h3>Future home of Apache Spark (Incubating).</h3><p>Test</p></body></html> +<!DOCTYPE html> +<!--[if IE 6]> +<html id="ie6" dir="ltr" lang="en-US"> +<![endif]--> +<!--[if IE 7]> +<html id="ie7" dir="ltr" lang="en-US"> +<![endif]--> +<!--[if IE 8]> +<html id="ie8" dir="ltr" lang="en-US"> +<![endif]--> +<!--[if !(IE 6) | !(IE 7) | !(IE 8) ]><!--> +<html dir="ltr" lang="en-US"> +<!--<![endif]--> +<head> + <link rel="shortcut icon" href="favicon.ico" /> + <meta charset="UTF-8" /> + <meta name="viewport" content="width=device-width" /> + <title>Home | Spark</title> + + <link rel="stylesheet" type="text/css" media="all" href="/css/style.css" /> + <link rel="stylesheet" href="/css/pygments-default.css"> + + <script type="text/javascript">//<![CDATA[ + // Google Analytics for WordPress by Yoast v4.2.8 | http://yoast.com/wordpress/google-analytics/ + var _gaq = _gaq || []; + _gaq.push(['_setAccount', 'UA-32518208-1']); + _gaq.push(['_trackPageview']); + (function () { + var ga = document.createElement('script'); + ga.type = 'text/javascript'; + ga.async = true; + ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js'; + var s = document.getElementsByTagName('script')[0]; + s.parentNode.insertBefore(ga, s); + })(); + //]]></script> + + <link rel='canonical' href='/index.html' /> + + <style type="text/css"> + #site-title, + #site-description { + position: absolute !important; + clip: rect(1px 1px 1px 1px); /* IE6, IE7 */ + clip: rect(1px, 1px, 1px, 1px); + } + </style> + <style type="text/css" id="custom-background-css"> + body.custom-background { background-color: #f1f1f1; } + </style> +</head> + +<!--body class="page two-column right-sidebar"--> +<body class="page"> +<div id="page" class="hfeed"> + + <header id="branding" role="banner"> + <hgroup> + <h1 id="site-title"><span><a href="/" title="Spark" rel="home">Spark</a></span></h1> + <h2 id="site-description">Lightning-Fast Cluster Computing</h2> + </hgroup> + + <a href="/"> + <img src="/images/spark-project-header1.png" width="1000" height="220" alt="Spark: Lightning-Fast Cluster Computing" title="Spark: Lightning-Fast Cluster Computing" /> + </a> + + <nav id="access" role="navigation"> + <h3 class="assistive-text">Main menu</h3> + <div class="menu-main-menu-container"> + <ul id="menu-main-menu" class="menu"> + + <li class="menu-item menu-item-type-post_type menu-item-object-page current-menu-item"> + <a href="/index.html">Home</a> + </li> + + <li class="menu-item menu-item-type-post_type menu-item-object-page "> + <a href="/downloads.html">Downloads</a> + </li> + + <li class="menu-item menu-item-type-post_type menu-item-object-page "> + <a href="/documentation.html">Documentation</a> + </li> + + <li class="menu-item menu-item-type-post_type menu-item-object-page "> + <a href="/examples.html">Examples</a> + </li> + + <li class="menu-item menu-item-type-post_type menu-item-object-page "> + <a href="/mailing-lists.html">Mailing Lists</a> + </li> + + <li class="menu-item menu-item-type-post_type menu-item-object-page "> + <a href="/research.html">Research</a> + </li> + + <li class="menu-item menu-item-type-post_type menu-item-object-page "> + <a href="/faq.html">FAQ</a> + </li> + + </ul></div> + </nav><!-- #access --> +</header><!-- #branding --> + + + + <div id="main"> + <div id="primary"> + <div id="content" role="main"> + + <article class="page type-page status-publish hentry"> + <h2 id="what-is-spark">What is Spark?</h2> +<p>Spark is an open source cluster computing system that aims to make data analytics <em>fast</em> — both fast to run and fast to write. +To run programs faster, Spark provides primitives for in-memory cluster computing: your job can load data into memory and query it repeatedly much more quickly than with disk-based systems like Hadoop MapReduce. +To make programming faster, Spark provides clean, concise APIs in <a href="http://www.scala-lang.org" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://www.scala-lang.org']);">Scala</a>, <a href="/docs/latest/quick-start.html#a-standalone-job-in-java">Java</a> and <a href="/docs/latest/quick-start.html#a-standalone-job-in-python">Python</a>. You can also use Spark interactively from the Scala and Python shells to rapidly query big datasets.</p> + +<h2 id="what-can-it-do">What can it do?</h2> +<p>Spark was initially developed for two applications where keeping data in memory helps: <em>iterative</em> algorithms, which are common in machine learning, and <em>interactive</em> data mining. In both cases, Spark can run up to <b>100x</b> faster than Hadoop MapReduce. However, you can use Spark for general data processing too. Check out our <a href="/examples.html">example jobs</a>. +Spark is also the engine behind <a href="http://shark.cs.berkeley.edu" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://shark.cs.berkeley.edu']);">Shark</a>, a fully <a href="http://hive.apache.org" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://hive.apache.org']);">Apache Hive</a>-compatible data warehousing system that can run 100x faster than Hive. +While Spark is a new engine, it can access any data source supported by Hadoop, making it easy to run over existing data.</p> + +<h2 id="who-uses-it">Who uses it?</h2> +<p>Spark was developed in the <a href="https://amplab.cs.berkeley.edu" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://amplab.cs.berkeley.edu']);">UC Berkeley AMPLab</a>. It’s used by several groups of researchers at Berkeley to run large-scale applications such as spam filtering and traffic prediction. It’s also used to accelerate data analytics at <a href="http://www.yahoo.com" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://www.yahoo.com']);">Yahoo!</a>, <a href="http://www.conviva.com" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://www.conviva.com']);">Conviva</a>, <a href="http://www.quantifind.com" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://www.quantifind.com']);">Quantifind</a>, and other companies — in total, 17 companies have contributed to Spark! Spark is <a href="https://github.com/mesos/spark" onclick="javascript:_gaq.push(['_trackEvent','outbound-article','http://github.com']);">open source</a> under a BSD license, so <a href="/downloads.html">download</a> it to check it out.</p> + +<h2 id="apache-incubator-notice">Apache Incubator notice</h2> +<p>Apache Spark is an effort undergoing incubation at The Apache Software Foundation (ASF), sponsored by the Apache Incubator. Incubation is required of all newly accepted projects until a further review indicates that the infrastructure, communications, and decision making process have stabilized in a manner consistent with other successful ASF projects. While incubation status is not necessarily a reflection of the completeness or stability of the code, it does indicate that the project has yet to be fully endorsed by the ASF.</p> + + + </article><!-- #post --> + + </div><!-- #content --> + + <div id="secondary" class="widget-area" role="complementary"> + +<h3 class="widget-title">Latest News</h3> +<div class="latestnewswidget"> + + <div><a href="/news/amp-camp-2013-registration-ope.html">Registration open for AMP Camp training camp in Berkeley</a> <span class="post-info">(July 23, 2013)</span></div> + + <div><a href="/news/spark-mailing-lists-moving-to-apache.html">Spark mailing lists moving to Apache</a> <span class="post-info">(July 22, 2013)</span></div> + + <div><a href="/news/spark-0-7-3-released.html">Spark 0.7.3 released</a> <span class="post-info">(July 16, 2013)</span></div> + + <div><a href="/news/spark-featured-in-wired.html">Spark featured in Wired</a> <span class="post-info">(June 21, 2013)</span></div> + + </div> + +<div style="text-align:right"><a href="/news/index.html">News Archive</a></div> + +<p><!-- Not porting the following to Pygments since it becomes a lot less colorful --></p> + +<div class="code" style="margin-top: 20px;"> + file = spark.textFile(<span class="string">"hdfs://..."</span>)<br /> + <br /> + file.<span class="sparkop">flatMap</span>(<span class="closure">line => line.split(" ")</span>)<br /> + .<span class="sparkop">map</span>(<span class="closure">word => (word, 1)</span>)<br /> + .<span class="sparkop">reduceByKey</span>(<span class="closure">_ + _</span>) + </div> +<div class="caption">Word Count implemented in Spark</div> + +<div align="center" style="margin-top: 20px;"> + <img src="/images/spark-lr.png" alt="Logistic regression performance in Spark vs Hadoop" /> + </div> +<div class="caption">Logistic regression in Spark vs Hadoop</div> +<h2 style="text-align:center"><a href="/downloads"><img src="/images/download.png" alt="Download" style="vertical-align: middle" /> Download Spark</a></h2> + + </div> + + <footer id="colophon" role="contentinfo"> + <div id="site-generator"> + <p>Spark is an open source project developed at the UC Berkeley <a href="https://amplab.cs.berkeley.edu">AMPLab</a>.</p> + <a class="amp-logo" style="background:url(/images/amplab-small.png)" href="https://amplab.cs.berkeley.edu/" title="Brought to you by the UC Berkeley AMPLab." rel="generator"><!--Brought to you by the UC Berkeley AMPLab--> </a> + </div> +</footer><!-- #colophon --> + + </div><!-- #primary --> + </div><!-- #main --> +</div><!-- #page --> + + +</body> +</html> |