diff options
author | Kazuaki Ishizaki <ishizaki@jp.ibm.com> | 2016-04-19 23:30:34 +0200 |
---|---|---|
committer | Herman van Hovell <hvanhovell@questtec.nl> | 2016-04-19 23:30:34 +0200 |
commit | 0b8369d8548c0204b9c24d826c731063b72360b8 (patch) | |
tree | 3d7106cbb3e20a8cac0199e93f167ebe17f79d53 | |
parent | 5cb2e3360985bc9e67aee038befa93c258f2016a (diff) | |
download | spark-0b8369d8548c0204b9c24d826c731063b72360b8.tar.gz spark-0b8369d8548c0204b9c24d826c731063b72360b8.tar.bz2 spark-0b8369d8548c0204b9c24d826c731063b72360b8.zip |
[SPARK-14656][CORE] Fix Benchmark.getPorcessorName() always return "Unknown processor" on Linux
## What changes were proposed in this pull request?
This PR returns correct processor name in ```/proc/cpuinfo``` on Linux from ```Benchmark.getPorcessorName()```. Now, this return ```Unknown processor```.
Since ```Utils.executeAndGetOutput(Seq("which", "grep"))``` return ```/bin/grep\n```, it is failed to execute ```/bin/grep\n```. This PR strips ```\n``` at the end of the line of a result of ```Utils.executeAndGetOutput()```
Before applying this PR
````
Java HotSpot(TM) 64-Bit Server VM 1.8.0_66-b17 on Linux 2.6.32-504.el6.x86_64
Unknown processor
back-to-back filter: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
Dataset 472 / 503 21.2 47.2 1.0X
DataFrame 51 / 58 198.0 5.1 9.3X
RDD 189 / 211 52.8 18.9 2.5X
````
After applying this PR
```
Java HotSpot(TM) 64-Bit Server VM 1.8.0_66-b17 on Linux 2.6.32-504.el6.x86_64
Intel(R) Xeon(R) CPU E5-2667 v2 3.30GHz
back-to-back filter: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
Dataset 490 / 502 20.4 49.0 1.0X
DataFrame 55 / 61 183.4 5.5 9.0X
RDD 210 / 237 47.7 21.0 2.3X
```
## How was this patch tested?
Run Benchmark programs on Linux by hand
Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
Closes #12411 from kiszk/SPARK-14656.
-rw-r--r-- | core/src/main/scala/org/apache/spark/util/Benchmark.scala | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/core/src/main/scala/org/apache/spark/util/Benchmark.scala b/core/src/main/scala/org/apache/spark/util/Benchmark.scala index 9e40bafd52..3718542810 100644 --- a/core/src/main/scala/org/apache/spark/util/Benchmark.scala +++ b/core/src/main/scala/org/apache/spark/util/Benchmark.scala @@ -96,9 +96,9 @@ private[spark] object Benchmark { Utils.executeAndGetOutput(Seq("/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string")) } else if (SystemUtils.IS_OS_LINUX) { Try { - val grepPath = Utils.executeAndGetOutput(Seq("which", "grep")) + val grepPath = Utils.executeAndGetOutput(Seq("which", "grep")).stripLineEnd Utils.executeAndGetOutput(Seq(grepPath, "-m", "1", "model name", "/proc/cpuinfo")) - .replaceFirst("model name[\\s*]:[\\s*]", "") + .stripLineEnd.replaceFirst("model name[\\s*]:[\\s*]", "") }.getOrElse("Unknown processor") } else { System.getenv("PROCESSOR_IDENTIFIER") |