diff options
author | Mark Hamstra <markhamstra@gmail.com> | 2013-03-13 13:16:14 -0700 |
---|---|---|
committer | Mark Hamstra <markhamstra@gmail.com> | 2013-03-13 13:16:14 -0700 |
commit | cd5b947cf64ce0c8abb4b4bf5f37550522eac8e1 (patch) | |
tree | debaac35a063960f9122ea1a8f71e1cc047a3e5c | |
parent | a6de80218ac9c917f4b33a33950c71ae88223926 (diff) | |
parent | 2d477fd36f218f127552a0f5b711df80ec6ba303 (diff) | |
download | spark-cd5b947cf64ce0c8abb4b4bf5f37550522eac8e1.tar.gz spark-cd5b947cf64ce0c8abb4b4bf5f37550522eac8e1.tar.bz2 spark-cd5b947cf64ce0c8abb4b4bf5f37550522eac8e1.zip |
Merge branch 'master' of https://github.com/mesos/spark into WithThing
-rw-r--r-- | core/src/main/scala/spark/storage/BlockManagerMessages.scala | 8 | ||||
-rw-r--r-- | docs/quick-start.md | 4 |
2 files changed, 6 insertions, 6 deletions
diff --git a/core/src/main/scala/spark/storage/BlockManagerMessages.scala b/core/src/main/scala/spark/storage/BlockManagerMessages.scala index 1494f90103..cff48d9909 100644 --- a/core/src/main/scala/spark/storage/BlockManagerMessages.scala +++ b/core/src/main/scala/spark/storage/BlockManagerMessages.scala @@ -49,16 +49,16 @@ class UpdateBlockInfo( blockManagerId.writeExternal(out) out.writeUTF(blockId) storageLevel.writeExternal(out) - out.writeInt(memSize.toInt) - out.writeInt(diskSize.toInt) + out.writeLong(memSize) + out.writeLong(diskSize) } override def readExternal(in: ObjectInput) { blockManagerId = BlockManagerId(in) blockId = in.readUTF() storageLevel = StorageLevel(in) - memSize = in.readInt() - diskSize = in.readInt() + memSize = in.readLong() + diskSize = in.readLong() } } diff --git a/docs/quick-start.md b/docs/quick-start.md index 36d024f13a..216f7c9cc5 100644 --- a/docs/quick-start.md +++ b/docs/quick-start.md @@ -189,7 +189,7 @@ public class SimpleJob { } {% endhighlight %} -This job simply counts the number of lines containing 'a' and the number containing 'b' in a system log file. Note that like in the Scala example, we initialize a SparkContext, though we use the special `JavaSparkContext` class to get a Java-friendly one. We also create RDDs (represented by `JavaRDD`) and run transformations on them. Finally, we pass functions to Spark by creating classes that extend `spark.api.java.function.Function`. The [Java programming guide]("java-programming-guide") describes these differences in more detail. +This job simply counts the number of lines containing 'a' and the number containing 'b' in a system log file. Note that like in the Scala example, we initialize a SparkContext, though we use the special `JavaSparkContext` class to get a Java-friendly one. We also create RDDs (represented by `JavaRDD`) and run transformations on them. Finally, we pass functions to Spark by creating classes that extend `spark.api.java.function.Function`. The [Java programming guide](java-programming-guide.html) describes these differences in more detail. To build the job, we also write a Maven `pom.xml` file that lists Spark as a dependency. Note that Spark artifacts are tagged with a Scala version. @@ -265,7 +265,7 @@ print "Lines with a: %i, lines with b: %i" % (numAs, numBs) This job simply counts the number of lines containing 'a' and the number containing 'b' in a system log file. Like in the Scala and Java examples, we use a SparkContext to create RDDs. We can pass Python functions to Spark, which are automatically serialized along with any variables that they reference. -For jobs that use custom classes or third-party libraries, we can add those code dependencies to SparkContext to ensure that they will be available on remote machines; this is described in more detail in the [Python programming guide](python-programming-guide). +For jobs that use custom classes or third-party libraries, we can add those code dependencies to SparkContext to ensure that they will be available on remote machines; this is described in more detail in the [Python programming guide](python-programming-guide.html). `SimpleJob` is simple enough that we do not need to specify any code dependencies. We can run this job using the `pyspark` script: |