diff options
| author | Gregory Owen <greowen@gmail.com> | 2014-07-21 18:55:01 -0700 |
|---|---|---|
| committer | Patrick Wendell <pwendell@gmail.com> | 2014-07-21 18:55:01 -0700 |
| commit | c3462c65684885299cf037d56c88bd53c08c6348 (patch) | |
| tree | f975d0237f8a6d4edb88e47c3dfa25ccc46ac133 /project/MimaExcludes.scala | |
| parent | 511a7314037219c23e824ea5363bf7f1df55bab3 (diff) | |
| download | spark-c3462c65684885299cf037d56c88bd53c08c6348.tar.gz spark-c3462c65684885299cf037d56c88bd53c08c6348.tar.bz2 spark-c3462c65684885299cf037d56c88bd53c08c6348.zip | |
[SPARK-2086] Improve output of toDebugString to make shuffle boundaries more clear
Changes RDD.toDebugString() to show hierarchy and shuffle transformations more clearly
New output:
```
(3) FlatMappedValuesRDD[325] at apply at Transformer.scala:22
| MappedValuesRDD[324] at apply at Transformer.scala:22
| CoGroupedRDD[323] at apply at Transformer.scala:22
+-(5) MappedRDD[320] at apply at Transformer.scala:22
| | MappedRDD[319] at apply at Transformer.scala:22
| | MappedValuesRDD[318] at apply at Transformer.scala:22
| | MapPartitionsRDD[317] at apply at Transformer.scala:22
| | ShuffledRDD[316] at apply at Transformer.scala:22
| +-(10) MappedRDD[315] at apply at Transformer.scala:22
| | ParallelCollectionRDD[314] at apply at Transformer.scala:22
+-(100) MappedRDD[322] at apply at Transformer.scala:22
| ParallelCollectionRDD[321] at apply at Transformer.scala:22
```
Author: Gregory Owen <greowen@gmail.com>
Closes #1364 from GregOwen/to-debug-string and squashes the following commits:
08f5c78 [Gregory Owen] toDebugString: prettier debug printing to show shuffles and joins more clearly
1603f7b [Gregory Owen] toDebugString: prettier debug printing to show shuffles and joins more clearly
Diffstat (limited to 'project/MimaExcludes.scala')
| -rw-r--r-- | project/MimaExcludes.scala | 8 |
1 files changed, 8 insertions, 0 deletions
diff --git a/project/MimaExcludes.scala b/project/MimaExcludes.scala index e0f433b26f..4d86e1a0d8 100644 --- a/project/MimaExcludes.scala +++ b/project/MimaExcludes.scala @@ -62,6 +62,14 @@ object MimaExcludes { ProblemFilters.exclude[MissingMethodProblem]( "org.apache.spark.storage.MemoryStore.Entry"), ProblemFilters.exclude[MissingMethodProblem]( + "org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$debugChildren$1"), + ProblemFilters.exclude[MissingMethodProblem]( + "org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$firstDebugString$1"), + ProblemFilters.exclude[MissingMethodProblem]( + "org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$shuffleDebugString$1"), + ProblemFilters.exclude[MissingMethodProblem]( + "org.apache.spark.rdd.RDD.org$apache$spark$rdd$RDD$$debugString$1"), + ProblemFilters.exclude[MissingMethodProblem]( "org.apache.spark.rdd.PairRDDFunctions.org$apache$spark$rdd$PairRDDFunctions$$" + "createZero$1") ) ++ |