aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark/conf.py
diff options
context:
space:
mode:
authorGregory Owen <greowen@gmail.com>2014-07-21 18:55:01 -0700
committerPatrick Wendell <pwendell@gmail.com>2014-07-21 18:55:01 -0700
commitc3462c65684885299cf037d56c88bd53c08c6348 (patch)
treef975d0237f8a6d4edb88e47c3dfa25ccc46ac133 /python/pyspark/conf.py
parent511a7314037219c23e824ea5363bf7f1df55bab3 (diff)
downloadspark-c3462c65684885299cf037d56c88bd53c08c6348.tar.gz
spark-c3462c65684885299cf037d56c88bd53c08c6348.tar.bz2
spark-c3462c65684885299cf037d56c88bd53c08c6348.zip
[SPARK-2086] Improve output of toDebugString to make shuffle boundaries more clear
Changes RDD.toDebugString() to show hierarchy and shuffle transformations more clearly New output: ``` (3) FlatMappedValuesRDD[325] at apply at Transformer.scala:22 | MappedValuesRDD[324] at apply at Transformer.scala:22 | CoGroupedRDD[323] at apply at Transformer.scala:22 +-(5) MappedRDD[320] at apply at Transformer.scala:22 | | MappedRDD[319] at apply at Transformer.scala:22 | | MappedValuesRDD[318] at apply at Transformer.scala:22 | | MapPartitionsRDD[317] at apply at Transformer.scala:22 | | ShuffledRDD[316] at apply at Transformer.scala:22 | +-(10) MappedRDD[315] at apply at Transformer.scala:22 | | ParallelCollectionRDD[314] at apply at Transformer.scala:22 +-(100) MappedRDD[322] at apply at Transformer.scala:22 | ParallelCollectionRDD[321] at apply at Transformer.scala:22 ``` Author: Gregory Owen <greowen@gmail.com> Closes #1364 from GregOwen/to-debug-string and squashes the following commits: 08f5c78 [Gregory Owen] toDebugString: prettier debug printing to show shuffles and joins more clearly 1603f7b [Gregory Owen] toDebugString: prettier debug printing to show shuffles and joins more clearly
Diffstat (limited to 'python/pyspark/conf.py')
0 files changed, 0 insertions, 0 deletions