lampepfl/compiler-benchmark

Name: compiler-benchmark

Owner: Programming Methods Laboratory EPFL

Description: Benchmarks for scalac

Forked from: scala/compiler-benchmark

Created: 2017-05-11 14:33:36.0

Updated: 2017-06-03 21:38:58.0

Pushed: 2017-07-12 09:07:06.0

Homepage:

Size: 255

Language: Scala

GitHub Committers

User	Most Recent Commit	# Commits

Other Committers

User	Email	Most Recent Commit	# Commits

README

JMH benchmarks for the Scala Compiler

Goal: define a set of JMH benchmarks for the compiler to help drive performance tuning and catch performance regressions.

Based on:

• OpenJDK JMH, the definitive Java benchmarking tool.
• via sbt-jmh.

Structure

| Project | Benchmark of| | ————- | ————- | | compilation | The equivalent of scalac ... | | micro | Finer grained parts of the compiler | | jvm | Pure Java benchmarks to demonstrate JVM quirks | | infrastructure | Code to persist benchmark results and metadata |

Recipes

Learning about JMH options

 compilation/jmh:run -help

Benchmarking compiler performance

• (optional) add new Scala sources (say aardvark) to a new directory in src/main/corpus

ilation/jmh:run (Cold|Warm|Hot)CompilationBenchmark 
p source=(<subdir of corpus>|/path/to/src/dir|@/path/to/argsfile)
p extraArgs=-nowarn

Using aliases

Avoid the tedium of typing all that out with:

 hot -psource=scalap 
 cold -psource=better-files 

Changing Scala Version

 set scalaVersion in ThisBuild := "2.12.0-ab61fed-SNAPSHOT"
 set scalaHome in ThisBuild := Some(file("/code/scala/build/pack"))
 set scalaHome in compilation := "2.11.1" // if micro project isn't compatible with "2.11.1"

Persisting results

• Provide INFLUX_PASSWORD as an environment variable
• Replace jmh/run with jmh:runMain scala.bench.UploadingRunner

Results will be uploading into an InfluxDB instance at https://scala-ci.typesafe.com/influx/

These results will be plotted in our Grafana dashboard

Note that the https://github.com/scala/compiler-benchq project, which will trigger benchmarks for merges and sets of commits that we're backtesting.

Collecting profiling data

 .../jmh:run Benchmark -prof jmh.extras.JFR // Java Flight Recorder

Using Graal

Install Graal VM and JDK8 with JVMCI.

ilation/jmh:run CompileSourcesBenchmark 
-jvm      /path/to/labsjdk1.8.0_92-jvmci-0.20/bin/java
-jvmArgs -XX:+UnlockExperimentalVMOptions
-jvmArgs -XX:+EnableJVMCI
-jvmArgs -Djvmci.class.path.append=/path/to/graalvm-0.15-re/lib/jvmci/graal.jar
-jvmArgs -Xbootclasspath/a:/path/to/graalvm-0.15-re/lib/truffle-api.jar
-jvmArgs -Djvmci.Compiler=graal 
-jvmArgs -XX:+UseJVMCICompiler 

Exporting an args file from SBT

rl https://gist.githubusercontent.com/retronym/78d016a3f10c62da2fd47cacac867f25/raw/65d9a1e8458d5984784ecf411d6c4d257bfdf0c1/ArgsFile.scala >  ~/.sbt/0.13/plugins/ArgsFile.scala
 /code/someProject
t core/compile:argsFilePrint
o] Set current project to root (in build file:/Users/jason/code/better-files/)
recation
oding
8
ture
guage:implicitConversions
guage:reflectiveCalls
hecked
tal-warnings
nt
line-warnings
-adapted-args
rn-dead-code
ture
rs/jason/code/better-files/core/src/main/scala/better/files/Cmds.scala
rs/jason/code/better-files/core/src/main/scala/better/files/File.scala
rs/jason/code/better-files/core/src/main/scala/better/files/Implicits.scala
rs/jason/code/better-files/core/src/main/scala/better/files/package.scala
rs/jason/code/better-files/core/src/main/scala/better/files/Scanner.scala
rs/jason/code/better-files/core/src/main/scala/better/files/ThreadBackedFileMonitor.scala
cess] Total time: 0 s, completed 02/09/2016 11:51:58 AM

Place this output into a file. The full path to that file can be passed to the -source= option to run the benchmark on that project: -psource=@/path/to/args/file