diff options
| author | Michael Nazario <mnazario@palantir.com> | 2015-05-29 14:13:44 -0700 |
|---|---|---|
| committer | Davies Liu <davies@databricks.com> | 2015-05-29 14:13:44 -0700 |
| commit | 1c5b19827a091b5aba69a967600e7ca35ed3bcfd (patch) | |
| tree | d350243fffdced567492b8029773005b512a2abc /bin | |
| parent | 5f48e5c33bafa376be5741e260a037c66103fdcd (diff) | |
| download | spark-1c5b19827a091b5aba69a967600e7ca35ed3bcfd.tar.gz spark-1c5b19827a091b5aba69a967600e7ca35ed3bcfd.tar.bz2 spark-1c5b19827a091b5aba69a967600e7ca35ed3bcfd.zip | |
[SPARK-7899] [PYSPARK] Fix Python 3 pyspark/sql/types module conflict
This PR makes the types module in `pyspark/sql/types` work with pylint static analysis by removing the dynamic naming of the `pyspark/sql/_types` module to `pyspark/sql/types`.
Tests are now loaded using `$PYSPARK_DRIVER_PYTHON -m module` rather than `$PYSPARK_DRIVER_PYTHON module.py`. The old method adds the location of `module.py` to `sys.path`, so this change prevents accidental use of relative paths in Python.
Author: Michael Nazario <mnazario@palantir.com>
Closes #6439 from mnazario/feature/SPARK-7899 and squashes the following commits:
366ef30 [Michael Nazario] Remove hack on random.py
bb8b04d [Michael Nazario] Make doctests consistent with other tests
6ee4f75 [Michael Nazario] Change test scripts to use "-m"
673528f [Michael Nazario] Move _types back to types
Diffstat (limited to 'bin')
| -rwxr-xr-x | bin/pyspark | 6 |
1 files changed, 1 insertions, 5 deletions
diff --git a/bin/pyspark b/bin/pyspark index 8acad61137..7cb19c51b4 100755 --- a/bin/pyspark +++ b/bin/pyspark @@ -90,11 +90,7 @@ if [[ -n "$SPARK_TESTING" ]]; then unset YARN_CONF_DIR unset HADOOP_CONF_DIR export PYTHONHASHSEED=0 - if [[ -n "$PYSPARK_DOC_TEST" ]]; then - exec "$PYSPARK_DRIVER_PYTHON" -m doctest $1 - else - exec "$PYSPARK_DRIVER_PYTHON" $1 - fi + exec "$PYSPARK_DRIVER_PYTHON" -m $1 exit fi |