diff options
| author | Patrick Wendell <pwendell@apache.org> | 2014-03-03 00:37:16 +0000 |
|---|---|---|
| committer | Patrick Wendell <pwendell@apache.org> | 2014-03-03 00:37:16 +0000 |
| commit | f3f94677a2cede1fa6577fb60729d4271e6a1293 (patch) | |
| tree | b43dc7a9ccc2cf68a5eb54f075ea70e930197145 /site/docs/0.9.0/spark-debugger.html | |
| parent | 89007cd107a1d851e7a5ed553d8a0c5b2ece386d (diff) | |
| download | spark-website-f3f94677a2cede1fa6577fb60729d4271e6a1293.tar.gz spark-website-f3f94677a2cede1fa6577fb60729d4271e6a1293.tar.bz2 spark-website-f3f94677a2cede1fa6577fb60729d4271e6a1293.zip | |
Various doc updates that haven't been pushed to the live 0.9 site.
Diffstat (limited to 'site/docs/0.9.0/spark-debugger.html')
| -rw-r--r-- | site/docs/0.9.0/spark-debugger.html | 14 |
1 files changed, 2 insertions, 12 deletions
diff --git a/site/docs/0.9.0/spark-debugger.html b/site/docs/0.9.0/spark-debugger.html index f783faba4..09f65a8a6 100644 --- a/site/docs/0.9.0/spark-debugger.html +++ b/site/docs/0.9.0/spark-debugger.html @@ -130,7 +130,7 @@ <div class="container" id="content"> <h1 class="title">The Spark Debugger</h1> - <p><strong>Summary:</strong> The Spark debugger provides replay debugging for deterministic (logic) errors in Spark programs. It’s currently in development, but you can try it out in the <a href="https://github.com/apache/incubator-spark/tree/arthur">arthur branch</a>.</p> + <p><strong>Summary:</strong> The Spark debugger provides replay debugging for deterministic (logic) errors in Spark programs. It’s currently in development, but you can try it out in the <a href="https://github.com/apache/spark/tree/arthur">arthur branch</a>.</p> <h2 id="introduction">Introduction</h2> @@ -149,7 +149,7 @@ <h2 id="approach">Approach</h2> -<p>As your Spark program runs, the slaves report key events back to the master – for example, RDD creations, RDD contents, and uncaught exceptions. (A full list of event types is in <a href="https://github.com/apache/incubator-spark/blob/arthur/core/src/main/scala/spark/EventLogging.scala">EventLogging.scala</a>.) The master logs those events, and you can load the event log into the debugger after your program is done running.</p> +<p>As your Spark program runs, the slaves report key events back to the master – for example, RDD creations, RDD contents, and uncaught exceptions. (A full list of event types is in <a href="https://github.com/apache/spark/blob/arthur/core/src/main/scala/spark/EventLogging.scala">EventLogging.scala</a>.) The master logs those events, and you can load the event log into the debugger after your program is done running.</p> <p><em>A note on nondeterminism:</em> For fault recovery, Spark requires RDD transformations (for example, the function passed to <code>RDD.map</code>) to be deterministic. The Spark debugger also relies on this property, and it can also warn you if your transformation is nondeterministic. This works by checksumming the contents of each RDD and comparing the checksums from the original execution to the checksums after recomputing the RDD in the debugger.</p> @@ -292,16 +292,6 @@ <hr>--> - <footer> - <hr> - <p style="text-align: center; veritcal-align: middle; color: #999;"> - Apache Spark is an effort undergoing incubation at the Apache Software Foundation. - <a href="http://incubator.apache.org"> - <img style="margin-left: 20px;" src="img/incubator-logo.png" /> - </a> - </p> - </footer> - </div> <!-- /container --> <script src="js/vendor/jquery-1.8.0.min.js"></script> |