diff options
author | Ilya Ganelin <ilya.ganelin@capitalone.com> | 2015-04-17 18:28:42 -0700 |
---|---|---|
committer | Andrew Or <andrew@databricks.com> | 2015-04-17 18:28:42 -0700 |
commit | c5ed510135aee3a1a0402057b3b5229892aa6f3a (patch) | |
tree | b1daa6fe6fb42c924059c9442e10b8c4595d148b /docs/quick-start.md | |
parent | a452c59210cf2c8ff8601cdb11401eea6dc14973 (diff) | |
download | spark-c5ed510135aee3a1a0402057b3b5229892aa6f3a.tar.gz spark-c5ed510135aee3a1a0402057b3b5229892aa6f3a.tar.bz2 spark-c5ed510135aee3a1a0402057b3b5229892aa6f3a.zip |
[SPARK-6703][Core] Provide a way to discover existing SparkContext's
I've added a static getOrCreate method to the static SparkContext object that allows one to either retrieve a previously created SparkContext or to instantiate a new one with the provided config. The method accepts an optional SparkConf to make usage intuitive.
Still working on a test for this, basically want to create a new context from scratch, then ensure that subsequent calls don't overwrite that.
Author: Ilya Ganelin <ilya.ganelin@capitalone.com>
Closes #5501 from ilganeli/SPARK-6703 and squashes the following commits:
db9a963 [Ilya Ganelin] Closing second spark context
1dc0444 [Ilya Ganelin] Added ref equality check
8c884fa [Ilya Ganelin] Made getOrCreate synchronized
cb0c6b7 [Ilya Ganelin] Doc updates and code cleanup
270cfe3 [Ilya Ganelin] [SPARK-6703] Documentation fixes
15e8dea [Ilya Ganelin] Updated comments and added MiMa Exclude
0e1567c [Ilya Ganelin] Got rid of unecessary option for AtomicReference
dfec4da [Ilya Ganelin] Changed activeContext to AtomicReference
733ec9f [Ilya Ganelin] Fixed some bugs in test code
8be2f83 [Ilya Ganelin] Replaced match with if
e92caf7 [Ilya Ganelin] [SPARK-6703] Added test to ensure that getOrCreate both allows creation, retrieval, and a second context if desired
a99032f [Ilya Ganelin] Spacing fix
d7a06b8 [Ilya Ganelin] Updated SparkConf class to add getOrCreate method. Started test suite implementation
Diffstat (limited to 'docs/quick-start.md')
0 files changed, 0 insertions, 0 deletions