diff options
| author | Wenchen Fan <wenchen@databricks.com> | 2016-03-02 15:26:34 -0800 |
|---|---|---|
| committer | Reynold Xin <rxin@databricks.com> | 2016-03-02 15:26:34 -0800 |
| commit | 4dd24811d9035c52c5965fca2fc6431aac6963fc (patch) | |
| tree | 49771acf8775633de5fe38cf9b38c929b70c99c6 /python/pyspark/sql/functions.py | |
| parent | e2780ce8252ded93a695125c0a745d8b93193cca (diff) | |
| download | spark-4dd24811d9035c52c5965fca2fc6431aac6963fc.tar.gz spark-4dd24811d9035c52c5965fca2fc6431aac6963fc.tar.bz2 spark-4dd24811d9035c52c5965fca2fc6431aac6963fc.zip | |
[SPARK-13594][SQL] remove typed operations(e.g. map, flatMap) from python DataFrame
## What changes were proposed in this pull request?
Remove `map`, `flatMap`, `mapPartitions` from python DataFrame, to prepare for Dataset API in the future.
## How was this patch tested?
existing tests
Author: Wenchen Fan <wenchen@databricks.com>
Closes #11445 from cloud-fan/python-clean.
Diffstat (limited to 'python/pyspark/sql/functions.py')
| -rw-r--r-- | python/pyspark/sql/functions.py | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/python/pyspark/sql/functions.py b/python/pyspark/sql/functions.py index b30cc6799e..92e724fef4 100644 --- a/python/pyspark/sql/functions.py +++ b/python/pyspark/sql/functions.py @@ -616,10 +616,10 @@ def log(arg1, arg2=None): If there is only one argument, then this takes the natural logarithm of the argument. - >>> df.select(log(10.0, df.age).alias('ten')).map(lambda l: str(l.ten)[:7]).collect() + >>> df.select(log(10.0, df.age).alias('ten')).rdd.map(lambda l: str(l.ten)[:7]).collect() ['0.30102', '0.69897'] - >>> df.select(log(df.age).alias('e')).map(lambda l: str(l.e)[:7]).collect() + >>> df.select(log(df.age).alias('e')).rdd.map(lambda l: str(l.e)[:7]).collect() ['0.69314', '1.60943'] """ sc = SparkContext._active_spark_context |