aboutsummaryrefslogtreecommitdiff
path: root/docs/tuning.md
diff options
context:
space:
mode:
authorSandy Ryza <sandy@cloudera.com>2014-10-21 21:53:09 -0700
committerPatrick Wendell <pwendell@gmail.com>2014-10-21 21:53:09 -0700
commit6bb56faea8d238ea22c2de33db93b1b39f492b3a (patch)
tree4f172ee623cd64827b6c78d8459554cd395b1660 /docs/tuning.md
parent856b081729057f9da31a86e4bfa0dc0013492042 (diff)
downloadspark-6bb56faea8d238ea22c2de33db93b1b39f492b3a.tar.gz
spark-6bb56faea8d238ea22c2de33db93b1b39f492b3a.tar.bz2
spark-6bb56faea8d238ea22c2de33db93b1b39f492b3a.zip
SPARK-1813. Add a utility to SparkConf that makes using Kryo really easy
Author: Sandy Ryza <sandy@cloudera.com> Closes #789 from sryza/sandy-spark-1813 and squashes the following commits: 48b05e9 [Sandy Ryza] Simplify b824932 [Sandy Ryza] Allow both spark.kryo.classesToRegister and spark.kryo.registrator at the same time 6a15bb7 [Sandy Ryza] Small fix a2278c0 [Sandy Ryza] Respond to review comments 6ef592e [Sandy Ryza] SPARK-1813. Add a utility to SparkConf that makes using Kryo really easy
Diffstat (limited to 'docs/tuning.md')
-rw-r--r--docs/tuning.md17
1 files changed, 2 insertions, 15 deletions
diff --git a/docs/tuning.md b/docs/tuning.md
index 8fb2a0433b..9b5c9adac6 100644
--- a/docs/tuning.md
+++ b/docs/tuning.md
@@ -47,24 +47,11 @@ registration requirement, but we recommend trying it in any network-intensive ap
Spark automatically includes Kryo serializers for the many commonly-used core Scala classes covered
in the AllScalaRegistrar from the [Twitter chill](https://github.com/twitter/chill) library.
-To register your own custom classes with Kryo, create a public class that extends
-[`org.apache.spark.serializer.KryoRegistrator`](api/scala/index.html#org.apache.spark.serializer.KryoRegistrator) and set the
-`spark.kryo.registrator` config property to point to it, as follows:
+To register your own custom classes with Kryo, use the `registerKryoClasses` method.
{% highlight scala %}
-import com.esotericsoftware.kryo.Kryo
-import org.apache.spark.serializer.KryoRegistrator
-
-class MyRegistrator extends KryoRegistrator {
- override def registerClasses(kryo: Kryo) {
- kryo.register(classOf[MyClass1])
- kryo.register(classOf[MyClass2])
- }
-}
-
val conf = new SparkConf().setMaster(...).setAppName(...)
-conf.set("spark.serializer", "org.apache.spark.serializer.KryoSerializer")
-conf.set("spark.kryo.registrator", "mypackage.MyRegistrator")
+conf.registerKryoClasses(Seq(classOf[MyClass1], classOf[MyClass2]))
val sc = new SparkContext(conf)
{% endhighlight %}