diff options
| author | Sun Rui <rui.sun@intel.com> | 2016-04-29 16:41:07 -0700 |
|---|---|---|
| committer | Shivaram Venkataraman <shivaram@cs.berkeley.edu> | 2016-04-29 16:41:07 -0700 |
| commit | 4ae9fe091c2cb8388c581093d62d3deaef40993e (patch) | |
| tree | fd84ce605c0ea8bd9d0b2e307119bd5d8651c9f5 /R/pkg/NAMESPACE | |
| parent | d78fbcc3cc9c379b4a548ebc816c6f71cc71a16e (diff) | |
| download | spark-4ae9fe091c2cb8388c581093d62d3deaef40993e.tar.gz spark-4ae9fe091c2cb8388c581093d62d3deaef40993e.tar.bz2 spark-4ae9fe091c2cb8388c581093d62d3deaef40993e.zip | |
[SPARK-12919][SPARKR] Implement dapply() on DataFrame in SparkR.
## What changes were proposed in this pull request?
dapply() applies an R function on each partition of a DataFrame and returns a new DataFrame.
The function signature is:
dapply(df, function(localDF) {}, schema = NULL)
R function input: local data.frame from the partition on local node
R function output: local data.frame
Schema specifies the Row format of the resulting DataFrame. It must match the R function's output.
If schema is not specified, each partition of the result DataFrame will be serialized in R into a single byte array. Such resulting DataFrame can be processed by successive calls to dapply().
## How was this patch tested?
SparkR unit tests.
Author: Sun Rui <rui.sun@intel.com>
Author: Sun Rui <sunrui2016@gmail.com>
Closes #12493 from sun-rui/SPARK-12919.
Diffstat (limited to 'R/pkg/NAMESPACE')
| -rw-r--r-- | R/pkg/NAMESPACE | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/R/pkg/NAMESPACE b/R/pkg/NAMESPACE index 002e469efb..647db22747 100644 --- a/R/pkg/NAMESPACE +++ b/R/pkg/NAMESPACE @@ -45,6 +45,7 @@ exportMethods("arrange", "covar_samp", "covar_pop", "crosstab", + "dapply", "describe", "dim", "distinct", |