diff options
| author | Bryan Cutler <cutlerb@gmail.com> | 2016-05-20 16:41:57 -0700 |
|---|---|---|
| committer | Andrew Or <andrew@databricks.com> | 2016-05-20 16:41:57 -0700 |
| commit | 021c19702c720b4466b016498917d47f99000e13 (patch) | |
| tree | 8625f6f51a36795a0256f0eafe8f01c81f90a9df /python | |
| parent | 127bf1bb07967e2e4f99ad7abaa7f6fab3b3f407 (diff) | |
| download | spark-021c19702c720b4466b016498917d47f99000e13.tar.gz spark-021c19702c720b4466b016498917d47f99000e13.tar.bz2 spark-021c19702c720b4466b016498917d47f99000e13.zip | |
[SPARK-15456][PYSPARK] Fixed PySpark shell context initialization when HiveConf not present
## What changes were proposed in this pull request?
When PySpark shell cannot find HiveConf, it will fallback to create a SparkSession from a SparkContext. This fixes a bug caused by using a variable to SparkContext before it was initialized.
## How was this patch tested?
Manually starting PySpark shell and using the SparkContext
Author: Bryan Cutler <cutlerb@gmail.com>
Closes #13237 from BryanCutler/pyspark-shell-session-context-SPARK-15456.
Diffstat (limited to 'python')
| -rw-r--r-- | python/pyspark/shell.py | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/python/pyspark/shell.py b/python/pyspark/shell.py index ef46d3065e..ac5ce87a3f 100644 --- a/python/pyspark/shell.py +++ b/python/pyspark/shell.py @@ -44,9 +44,9 @@ try: .enableHiveSupport()\ .getOrCreate() except py4j.protocol.Py4JError: - spark = SparkSession(sc) + spark = SparkSession.builder.getOrCreate() except TypeError: - spark = SparkSession(sc) + spark = SparkSession.builder.getOrCreate() sc = spark.sparkContext atexit.register(lambda: sc.stop()) |