diff options
| author | Jeff Zhang <zjffdu@apache.org> | 2016-09-02 10:08:14 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-09-02 10:08:14 -0700 |
| commit | ea662286561aa9fe321cb0a0e10cdeaf60440b90 (patch) | |
| tree | 27d425342a202577df1a1c287e3c89b33513c2e2 /python | |
| parent | 6bcbf9b74351b5ac5221e3c309cb98e6f9cc7c5a (diff) | |
| download | spark-ea662286561aa9fe321cb0a0e10cdeaf60440b90.tar.gz spark-ea662286561aa9fe321cb0a0e10cdeaf60440b90.tar.bz2 spark-ea662286561aa9fe321cb0a0e10cdeaf60440b90.zip | |
[SPARK-17261] [PYSPARK] Using HiveContext after re-creating SparkContext in Spark 2.0 throws "Java.lang.illegalStateException: Cannot call methods on a stopped sparkContext"
## What changes were proposed in this pull request?
Set SparkSession._instantiatedContext as None so that we can recreate SparkSession again.
## How was this patch tested?
Tested manually using the following command in pyspark shell
```
spark.stop()
spark = SparkSession.builder.enableHiveSupport().getOrCreate()
spark.sql("show databases").show()
```
Author: Jeff Zhang <zjffdu@apache.org>
Closes #14857 from zjffdu/SPARK-17261.
Diffstat (limited to 'python')
| -rw-r--r-- | python/pyspark/sql/session.py | 1 |
1 files changed, 1 insertions, 0 deletions
diff --git a/python/pyspark/sql/session.py b/python/pyspark/sql/session.py index 61fa107497..8418abf99c 100644 --- a/python/pyspark/sql/session.py +++ b/python/pyspark/sql/session.py @@ -595,6 +595,7 @@ class SparkSession(object): """Stop the underlying :class:`SparkContext`. """ self._sc.stop() + SparkSession._instantiatedContext = None @since(2.0) def __enter__(self): |