diff options
author | Kazuaki Ishizaki <ishizaki@jp.ibm.com> | 2016-03-22 21:01:52 -0700 |
---|---|---|
committer | Reynold Xin <rxin@databricks.com> | 2016-03-22 21:01:52 -0700 |
commit | 0d51b60443ae78fa46988a6aed2397db9c35f96d (patch) | |
tree | 583ec8a2633cac2144fe68a6687a0261ef564106 /core/src/main/scala/org/apache | |
parent | 4700adb98e4a37c2b0ef7123eca8a9a03bbdbe78 (diff) | |
download | spark-0d51b60443ae78fa46988a6aed2397db9c35f96d.tar.gz spark-0d51b60443ae78fa46988a6aed2397db9c35f96d.tar.bz2 spark-0d51b60443ae78fa46988a6aed2397db9c35f96d.zip |
[SPARK-14072][CORE] Show JVM/OS version information when we run a benchmark program
## What changes were proposed in this pull request?
This PR allows us to identify what JVM is used when someone ran a benchmark program. In some cases, a JVM version may affect performance result. Thus, it would be good to show processor information and JVM version information.
```
model name : Intel(R) Xeon(R) CPU E5-2697 v2 2.70GHz
JVM information : OpenJDK 64-Bit Server VM, 1.7.0_65-mockbuild_2014_07_14_06_19-b00
Int and String Scan: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized 981 / 994 10.7 93.5 1.0X
SQL Parquet MR 2518 / 2542 4.2 240.1 0.4X
```
```
model name : Intel(R) Xeon(R) CPU E5-2697 v2 2.70GHz
JVM information : IBM J9 VM, pxa6480sr2-20151023_01 (SR2)
String Dictionary: Best/Avg Time(ms) Rate(M/s) Per Row(ns) Relative
-------------------------------------------------------------------------------------------
SQL Parquet Vectorized 693 / 740 15.1 66.1 1.0X
SQL Parquet MR 2501 / 2562 4.2 238.5 0.3X
```
## How was this patch tested?
Tested by using existing benchmark programs
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)
Author: Kazuaki Ishizaki <ishizaki@jp.ibm.com>
Closes #11893 from kiszk/SPARK-14072.
Diffstat (limited to 'core/src/main/scala/org/apache')
-rw-r--r-- | core/src/main/scala/org/apache/spark/util/Benchmark.scala | 18 |
1 files changed, 17 insertions, 1 deletions
diff --git a/core/src/main/scala/org/apache/spark/util/Benchmark.scala b/core/src/main/scala/org/apache/spark/util/Benchmark.scala index b562b58f1b..9e40bafd52 100644 --- a/core/src/main/scala/org/apache/spark/util/Benchmark.scala +++ b/core/src/main/scala/org/apache/spark/util/Benchmark.scala @@ -64,6 +64,7 @@ private[spark] class Benchmark( val firstBest = results.head.bestMs // The results are going to be processor specific so it is useful to include that. + println(Benchmark.getJVMOSInfo()) println(Benchmark.getProcessorName()) printf("%-35s %16s %12s %13s %10s\n", name + ":", "Best/Avg Time(ms)", "Rate(M/s)", "Per Row(ns)", "Relative") @@ -91,16 +92,31 @@ private[spark] object Benchmark { * This should return something like "Intel(R) Core(TM) i7-4870HQ CPU @ 2.50GHz" */ def getProcessorName(): String = { - if (SystemUtils.IS_OS_MAC_OSX) { + val cpu = if (SystemUtils.IS_OS_MAC_OSX) { Utils.executeAndGetOutput(Seq("/usr/sbin/sysctl", "-n", "machdep.cpu.brand_string")) } else if (SystemUtils.IS_OS_LINUX) { Try { val grepPath = Utils.executeAndGetOutput(Seq("which", "grep")) Utils.executeAndGetOutput(Seq(grepPath, "-m", "1", "model name", "/proc/cpuinfo")) + .replaceFirst("model name[\\s*]:[\\s*]", "") }.getOrElse("Unknown processor") } else { System.getenv("PROCESSOR_IDENTIFIER") } + cpu + } + + /** + * This should return a user helpful JVM & OS information. + * This should return something like + * "OpenJDK 64-Bit Server VM 1.8.0_65-b17 on Linux 4.1.13-100.fc21.x86_64" + */ + def getJVMOSInfo(): String = { + val vmName = System.getProperty("java.vm.name") + val runtimeVersion = System.getProperty("java.runtime.version") + val osName = System.getProperty("os.name") + val osVersion = System.getProperty("os.version") + s"${vmName} ${runtimeVersion} on ${osName} ${osVersion}" } /** |