diff options
| author | Yin Huai <yhuai@databricks.com> | 2016-06-28 07:54:44 -0700 |
|---|---|---|
| committer | Davies Liu <davies.liu@gmail.com> | 2016-06-28 07:54:44 -0700 |
| commit | 0923c4f5676691e28e70ecb05890e123540b91f0 (patch) | |
| tree | fc4b3fff3d5ab0f07080ec90cabeae9786bae147 /docs/mllib-migration-guides.md | |
| parent | e158478a9fff5e63ae0336a54b3f360d0cd38921 (diff) | |
| download | spark-0923c4f5676691e28e70ecb05890e123540b91f0.tar.gz spark-0923c4f5676691e28e70ecb05890e123540b91f0.tar.bz2 spark-0923c4f5676691e28e70ecb05890e123540b91f0.zip | |
[SPARK-16224] [SQL] [PYSPARK] SparkSession builder's configs need to be set to the existing Scala SparkContext's SparkConf
## What changes were proposed in this pull request?
When we create a SparkSession at the Python side, it is possible that a SparkContext has been created. For this case, we need to set configs of the SparkSession builder to the Scala SparkContext's SparkConf (we need to do so because conf changes on a active Python SparkContext will not be propagated to the JVM side). Otherwise, we may create a wrong SparkSession (e.g. Hive support is not enabled even if enableHiveSupport is called).
## How was this patch tested?
New tests and manual tests.
Author: Yin Huai <yhuai@databricks.com>
Closes #13931 from yhuai/SPARK-16224.
Diffstat (limited to 'docs/mllib-migration-guides.md')
0 files changed, 0 insertions, 0 deletions