Home | Trees | Indices | Help |
|
---|
|
>>> from pyspark.context import SparkContext >>> sc = SparkContext('local', 'test') >>> b = sc.broadcast([1, 2, 3, 4, 5]) >>> b.value [1, 2, 3, 4, 5]
>>> from pyspark.broadcast import _broadcastRegistry >>> _broadcastRegistry[b.bid] = b >>> from cPickle import dumps, loads >>> loads(dumps(b)).value [1, 2, 3, 4, 5]
>>> sc.parallelize([0, 0]).flatMap(lambda x: b.value).collect() [1, 2, 3, 4, 5, 1, 2, 3, 4, 5]
>>> large_broadcast = sc.broadcast(list(range(10000)))
Classes | |
Broadcast A broadcast variable created with SparkContext.broadcast(). |
Home | Trees | Indices | Help |
|
---|
Generated by Epydoc 3.0.1 on Sun Feb 2 22:20:28 2014 | http://epydoc.sourceforge.net |