aboutsummaryrefslogtreecommitdiff
path: root/python/pyspark/ml/util.py
diff options
context:
space:
mode:
authorBurak Yavuz <brkyvz@gmail.com>2016-04-20 10:32:01 -0700
committerMichael Armbrust <michael@databricks.com>2016-04-20 10:32:01 -0700
commit80bf48f437939ddc3bb82c8c7530c8ae419f8427 (patch)
treee9be7bd9acac75d677eaad8ba69c84890d4913d3 /python/pyspark/ml/util.py
parent834277884fcdaab4758604272881ffb2369e25f0 (diff)
downloadspark-80bf48f437939ddc3bb82c8c7530c8ae419f8427.tar.gz
spark-80bf48f437939ddc3bb82c8c7530c8ae419f8427.tar.bz2
spark-80bf48f437939ddc3bb82c8c7530c8ae419f8427.zip
[SPARK-14555] First cut of Python API for Structured Streaming
## What changes were proposed in this pull request? This patch provides a first cut of python APIs for structured streaming. This PR provides the new classes: - ContinuousQuery - Trigger - ProcessingTime in pyspark under `pyspark.sql.streaming`. In addition, it contains the new methods added under: - `DataFrameWriter` a) `startStream` b) `trigger` c) `queryName` - `DataFrameReader` a) `stream` - `DataFrame` a) `isStreaming` This PR doesn't contain all methods exposed for `ContinuousQuery`, for example: - `exception` - `sourceStatuses` - `sinkStatus` They may be added in a follow up. This PR also contains some very minor doc fixes in the Scala side. ## How was this patch tested? Python doc tests TODO: - [ ] verify Python docs look good Author: Burak Yavuz <brkyvz@gmail.com> Author: Burak Yavuz <burak@databricks.com> Closes #12320 from brkyvz/stream-python.
Diffstat (limited to 'python/pyspark/ml/util.py')
-rw-r--r--python/pyspark/ml/util.py15
1 files changed, 0 insertions, 15 deletions
diff --git a/python/pyspark/ml/util.py b/python/pyspark/ml/util.py
index 841bfb47e1..7003e587ad 100644
--- a/python/pyspark/ml/util.py
+++ b/python/pyspark/ml/util.py
@@ -17,7 +17,6 @@
import sys
import uuid
-from functools import wraps
if sys.version > '3':
basestring = str
@@ -39,20 +38,6 @@ def _jvm():
raise AttributeError("Cannot load _jvm from SparkContext. Is SparkContext initialized?")
-def keyword_only(func):
- """
- A decorator that forces keyword arguments in the wrapped method
- and saves actual input keyword arguments in `_input_kwargs`.
- """
- @wraps(func)
- def wrapper(*args, **kwargs):
- if len(args) > 1:
- raise TypeError("Method %s forces keyword arguments." % func.__name__)
- wrapper._input_kwargs = kwargs
- return func(*args, **kwargs)
- return wrapper
-
-
class Identifiable(object):
"""
Object with a unique ID.