diff options
| author | zero323 <zero323@users.noreply.github.com> | 2017-02-24 08:22:30 -0800 |
|---|---|---|
| committer | Holden Karau <holden@us.ibm.com> | 2017-02-24 08:22:30 -0800 |
| commit | 4a5e38f5747148022988631cae0248ae1affadd3 (patch) | |
| tree | 6294b22089734ebeda4782a2262e333e337c013f /examples | |
| parent | 8f33731e796750e6f60dc9e2fc33a94d29d198b4 (diff) | |
| download | spark-4a5e38f5747148022988631cae0248ae1affadd3.tar.gz spark-4a5e38f5747148022988631cae0248ae1affadd3.tar.bz2 spark-4a5e38f5747148022988631cae0248ae1affadd3.zip | |
[SPARK-19161][PYTHON][SQL] Improving UDF Docstrings
## What changes were proposed in this pull request?
Replaces `UserDefinedFunction` object returned from `udf` with a function wrapper providing docstring and arguments information as proposed in [SPARK-19161](https://issues.apache.org/jira/browse/SPARK-19161).
### Backward incompatible changes:
- `pyspark.sql.functions.udf` will return a `function` instead of `UserDefinedFunction`. To ensure backward compatible public API we use function attributes to mimic `UserDefinedFunction` API (`func` and `returnType` attributes). This should have a minimal impact on the user code.
An alternative implementation could use dynamical sub-classing. This would ensure full backward compatibility but is more fragile in practice.
### Limitations:
Full functionality (retained docstring and argument list) is achieved only in the recent Python version. Legacy Python version will preserve only docstrings, but not argument list. This should be an acceptable trade-off between achieved improvements and overall complexity.
### Possible impact on other tickets:
This can affect [SPARK-18777](https://issues.apache.org/jira/browse/SPARK-18777).
## How was this patch tested?
Existing unit tests to ensure backward compatibility, additional tests targeting proposed changes.
Author: zero323 <zero323@users.noreply.github.com>
Closes #16534 from zero323/SPARK-19161.
Diffstat (limited to 'examples')
0 files changed, 0 insertions, 0 deletions