1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
|
# Scala library benchmarks
This directory is a standalone SBT project
that makes use of the [SBT plugin for JMH](https://github.com/ktoso/sbt-jmh),
with the usual directory structure:
source code for the benchmarks, which utilize [JMH](http://openjdk.java.net/projects/code-tools/jmh/),
should be placed in `src/main/scala`.
The benchmarks require first building Scala into `../../build/pack`.
They can then be (built and) run from `sbt` with "`jmh:run`".
"`jmh:run -h`" displays the usual JMH options available.
## some useful HotSpot options
Adding these to the `jmh:run` command line may help if you're using the HotSpot (Oracle, OpenJDK) compiler.
They require prefixing with `-jvmArgs`.
See [the Java documentation](http://docs.oracle.com/javase/8/docs/technotes/tools/unix/java.html) for more options.
### viewing JIT compilation events
Adding `-XX:+PrintCompilation` shows when Java methods are being compiled or deoptimized.
At the most basic level,
these messages will tell you whether the code that you're measuring is still being tuned,
so that you know whether you're running enough warm-up iterations.
See [Kris Mok's notes](https://gist.github.com/rednaxelafx/1165804#file-notes-md) to interpret the output in detail.
### consider GC events
If you're not explicitly performing `System.gc()` calls outside of your benchmarking code,
you should add the JVM option `-verbose:gc` to understand the effect that GCs may be having on your tests.
### "diagnostic" options
These require the `-XX:+UnlockDiagnosticVMOptions` JVM option.
#### viewing inlining events
Add `-XX:+PrintInlining`.
#### viewing the disassembled code
To show the assembly code corresponding to the code generated by the JIT compiler for specific methods,
add `-XX:CompileCommand=print,scala.collection.mutable.OpenHashMap::*`,
for example, to show all of the methods in the `scala.collection.mutable.OpenHashMap` class.
If you're running OpenJDK, you may need to install the disassembler library (`hsdis-amd64.so` for the `amd64` architecture).
In Debian, this is available in <a href="https://packages.debian.org/search?keywords=libhsdis0-fcml">the `libhsdis0-fcml` package</a>.
To show it for _all_ methods, add `-XX:+PrintAssembly`.
(This is usually excessive.)
## useful reading
* [OpenJDK advice on microbenchmarks](https://wiki.openjdk.java.net/display/HotSpot/MicroBenchmarks)
* Brian Goetz's "Java theory and practice" articles:
* "[Dynamic compilation and performance measurement](http://www.ibm.com/developerworks/java/library/j-jtp12214/)"
* "[Anatomy of a flawed benchmark](http://www.ibm.com/developerworks/java/library/j-jtp02225/)"
* [Doug Lea's JSR 166 benchmarks](http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/src/test/loops/)
* "[Measuring performance](http://docs.scala-lang.org/overviews/parallel-collections/performance.html)" of Scala parallel collections
## legacy frameworks
An older version of the benchmarking framework is still present in this directory, in the following locations:
<dl>
<dt><code>bench</code></dt>
<dd>A script to run the old benchmarks.</dd>
<dt><code>source.list</code></dt>
<dd>A temporary file used by <code>bench</code>.</dd>
<dt><code>src/scala/</code></dt>
<dd>The older benchmarks, including the previous framework.</dd>
</dl>
Another, older set of benchmarks is present in `../benchmarking/`.
|
