diff options
| author | Josh Rosen <joshrosen@databricks.com> | 2017-03-27 10:23:28 -0700 |
|---|---|---|
| committer | Josh Rosen <joshrosen@databricks.com> | 2017-03-27 10:23:28 -0700 |
| commit | 314cf51ded52834cfbaacf58d3d05a220965ca2a (patch) | |
| tree | ccdce4568c45ef87cfe0de91b8fbc12fb3f83f73 /python/setup.py | |
| parent | 0588dc7c0a9f3180dddae0dc202a6d41eb43464f (diff) | |
| download | spark-314cf51ded52834cfbaacf58d3d05a220965ca2a.tar.gz spark-314cf51ded52834cfbaacf58d3d05a220965ca2a.tar.bz2 spark-314cf51ded52834cfbaacf58d3d05a220965ca2a.zip | |
[SPARK-20102] Fix nightly packaging and RC packaging scripts w/ two minor build fixes
## What changes were proposed in this pull request?
The master snapshot publisher builds are currently broken due to two minor build issues:
1. For unknown reasons, the LFTP `mkdir -p` command began throwing errors when the remote directory already exists. This change of behavior might have been caused by configuration changes in the ASF's SFTP server, but I'm not entirely sure of that. To work around this problem, this patch updates the script to ignore errors from the `lftp mkdir -p` commands.
2. The PySpark `setup.py` file references a non-existent `pyspark.ml.stat` module, causing Python packaging to fail by complaining about a missing directory. The fix is to simply drop that line from the setup script.
## How was this patch tested?
The LFTP fix was tested by manually running the failing commands on AMPLab Jenkins against the ASF SFTP server. The PySpark fix was tested locally.
Author: Josh Rosen <joshrosen@databricks.com>
Closes #17437 from JoshRosen/spark-20102.
Diffstat (limited to 'python/setup.py')
| -rw-r--r-- | python/setup.py | 1 |
1 files changed, 0 insertions, 1 deletions
diff --git a/python/setup.py b/python/setup.py index 47eab98e0f..f50035435e 100644 --- a/python/setup.py +++ b/python/setup.py @@ -167,7 +167,6 @@ try: 'pyspark.ml', 'pyspark.ml.linalg', 'pyspark.ml.param', - 'pyspark.ml.stat', 'pyspark.sql', 'pyspark.streaming', 'pyspark.bin', |