[SPARK-11463] [PYSPARK] only install signal in main thread

Only install signal in main thread, or it will fail to create context in not-main thread.

Author: Davies Liu <davies@databricks.com>

Closes #9574 from davies/python_signal.

1 files changed, 4 insertions, 1 deletions

diff --git a/python/pyspark/context.py b/python/pyspark/context.py
index afd74d937a..77710a1339 100644
--- a/python/pyspark/context.py
+++ b/python/pyspark/context.py
@@ -21,6 +21,7 @@ import os
 import shutil
 import signal
 import sys
+import threading
 from threading import RLock
 from tempfile import NamedTemporaryFile
 
@@ -222,7 +223,9 @@ class SparkContext(object):
         def signal_handler(signal, frame):
             self.cancelAllJobs()
 
-        signal.signal(signal.SIGINT, signal_handler)
+        # see http://stackoverflow.com/questions/23206787/
+        if isinstance(threading.current_thread(), threading._MainThread):
+            signal.signal(signal.SIGINT, signal_handler)
 
     def _initialize_context(self, jconf):
         """