diff options
author | Devaraj K <devaraj@apache.org> | 2016-03-08 10:48:31 +0000 |
---|---|---|
committer | Sean Owen <sowen@cloudera.com> | 2016-03-08 10:48:31 +0000 |
commit | 9bf76ddde53cf87c9cc3a17b4d357372372cc7be (patch) | |
tree | 723e5da8af2963a38717ac4ca3e2204b0b855c33 /core | |
parent | 7771c7314ff01b666b36c97982f62c5ef26aca08 (diff) | |
download | spark-9bf76ddde53cf87c9cc3a17b4d357372372cc7be.tar.gz spark-9bf76ddde53cf87c9cc3a17b4d357372372cc7be.tar.bz2 spark-9bf76ddde53cf87c9cc3a17b4d357372372cc7be.zip |
[SPARK-13117][WEB UI] WebUI should use the local ip not 0.0.0.0
## What changes were proposed in this pull request?
In WebUI, now Jetty Server starts with SPARK_LOCAL_IP config value if it
is configured otherwise it starts with default value as '0.0.0.0'.
It is continuation as per the closed PR https://github.com/apache/spark/pull/11133 for the JIRA SPARK-13117 and discussion in SPARK-13117.
## How was this patch tested?
This has been verified using the command 'netstat -tnlp | grep <PID>' to check on which IP/hostname is binding with the below steps.
In the below results, mentioned PID in the command is the corresponding process id.
#### Without the patch changes,
Web UI(Jetty Server) is not taking the value configured for SPARK_LOCAL_IP and it is listening to all the interfaces.
###### Master
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 3930
tcp6 0 0 :::8080 :::* LISTEN 3930/java
```
###### Worker
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 4090
tcp6 0 0 :::8081 :::* LISTEN 4090/java
```
###### History Server Process,
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 2471
tcp6 0 0 :::18080 :::* LISTEN 2471/java
```
###### Driver
```
[devarajstobdtserver2 spark-master]$ netstat -tnlp | grep 6556
tcp6 0 0 :::4040 :::* LISTEN 6556/java
```
#### With the patch changes
##### i. With SPARK_LOCAL_IP configured
If the SPARK_LOCAL_IP is configured then all the processes Web UI(Jetty Server) is getting bind to the configured value.
###### Master
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 1561
tcp6 0 0 x.x.x.x:8080 :::* LISTEN 1561/java
```
###### Worker
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 2229
tcp6 0 0 x.x.x.x:8081 :::* LISTEN 2229/java
```
###### History Server
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 3747
tcp6 0 0 x.x.x.x:18080 :::* LISTEN 3747/java
```
###### Driver
```
[devarajstobdtserver2 spark-master]$ netstat -tnlp | grep 6013
tcp6 0 0 x.x.x.x:4040 :::* LISTEN 6013/java
```
##### ii. Without SPARK_LOCAL_IP configured
If the SPARK_LOCAL_IP is not configured then all the processes Web UI(Jetty Server) will start with the '0.0.0.0' as default value.
###### Master
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 4573
tcp6 0 0 :::8080 :::* LISTEN 4573/java
```
###### Worker
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 4703
tcp6 0 0 :::8081 :::* LISTEN 4703/java
```
###### History Server
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 4846
tcp6 0 0 :::18080 :::* LISTEN 4846/java
```
###### Driver
```
[devarajstobdtserver2 sbin]$ netstat -tnlp | grep 5437
tcp6 0 0 :::4040 :::* LISTEN 5437/java
```
Author: Devaraj K <devaraj@apache.org>
Closes #11490 from devaraj-kavali/SPARK-13117-v1.
Diffstat (limited to 'core')
-rw-r--r-- | core/src/main/scala/org/apache/spark/ui/WebUI.scala | 6 |
1 files changed, 4 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/ui/WebUI.scala b/core/src/main/scala/org/apache/spark/ui/WebUI.scala index fe4949b9f6..23d312525f 100644 --- a/core/src/main/scala/org/apache/spark/ui/WebUI.scala +++ b/core/src/main/scala/org/apache/spark/ui/WebUI.scala @@ -134,8 +134,10 @@ private[spark] abstract class WebUI( def bind() { assert(!serverInfo.isDefined, "Attempted to bind %s more than once!".format(className)) try { - serverInfo = Some(startJettyServer("0.0.0.0", port, sslOptions, handlers, conf, name)) - logInfo("Started %s at http://%s:%d".format(className, publicHostName, boundPort)) + var host = Option(conf.getenv("SPARK_LOCAL_IP")).getOrElse("0.0.0.0") + serverInfo = Some(startJettyServer(host, port, sslOptions, handlers, conf, name)) + logInfo("Bound %s to %s, and started at http://%s:%d".format(className, host, + publicHostName, boundPort)) } catch { case e: Exception => logError("Failed to bind %s".format(className), e) |