diff options
| author | Cheng Lian <lian@databricks.com> | 2016-02-27 00:28:30 +0800 |
|---|---|---|
| committer | Cheng Lian <lian@databricks.com> | 2016-02-27 00:28:30 +0800 |
| commit | 99dfcedbfd4c83c7b6a343456f03e8c6e29968c5 (patch) | |
| tree | eb08a31c82b94e1582a0b6aa265038cb5fd44403 /docs | |
| parent | 5c3912e5c90ce659146c3056430d100604378b71 (diff) | |
| download | spark-99dfcedbfd4c83c7b6a343456f03e8c6e29968c5.tar.gz spark-99dfcedbfd4c83c7b6a343456f03e8c6e29968c5.tar.bz2 spark-99dfcedbfd4c83c7b6a343456f03e8c6e29968c5.zip | |
[SPARK-13457][SQL] Removes DataFrame RDD operations
## What changes were proposed in this pull request?
This is another try of PR #11323.
This PR removes DataFrame RDD operations except for `foreach` and `foreachPartitions` (they are actions rather than transformations). Original calls are now replaced by calls to methods of `DataFrame.rdd`.
PR #11323 was reverted because it introduced a regression: both `DataFrame.foreach` and `DataFrame.foreachPartitions` wrap underlying RDD operations with `withNewExecutionId` to track Spark jobs. But they are removed in #11323.
## How was the this patch tested?
No extra tests are added. Existing tests should do the work.
Author: Cheng Lian <lian@databricks.com>
Closes #11388 from liancheng/remove-df-rdd-ops.
Diffstat (limited to 'docs')
0 files changed, 0 insertions, 0 deletions