Fetching and Building Mission Control 8+

As described in a previous post, Mission Control is now on GitHub. Since this alters how to fetch and build OpenJDK Mission Control, this is an updated version of my old post on how to fetch and build JMC from version 8 and up.

Getting Git

First step is to get Git, the SCM used for OpenJDK Mission Control. Installing Git is different for different platforms, but here is a link to how to get started:

https://git-scm.com/book/en/v2/Getting-Started-Installing-Git

Installing the Skara Tooling (Optional)

This is an optional step, making it easier if you want to contribute to Mission Control:

http://hirt.se/blog/?p=1186

Cloning the Source

Once Git is installed properly, getting the source is as easy as cloning the jmc repo. First change into the directory where you want to check out jmc. Then run:

git clone https://github.com/openjdk/jmc.git

Getting Maven

Since you probably have some Java experience, you probably already have Maven installed on your system. If you do not, you now need to install it. Simply follow the instructions here:

https://maven.apache.org/install.html

Building Mission Control

First we need to ensure that Java 8 is on our path. Some of the build components still use JDK 8, so this is important.

java –version

This will show the Java version in use. If this is not a Java 8 JDK, change your path. Once done, we are now ready to build Mission Control. Open up two terminals. Yep, two!

In the first one, go to where your cloned JMC resides and type in and execute the following commands (for Windows, replace the dash (/) with a backslash (\)):

cd releng/third-party
mvn p2:site
mvn jetty:run

Now, leave that terminal open with the jetty running. Do not touch.

In the second terminal, go to your cloned jmc directory. First we will need to build and install the core libraries:

cd core
mvn install

Next run maven in the jmc root:

mvn clean package

JMC should now be building. The first time you build Maven will download all of the third party dependencies. This will take some time. Subsequent builds will be less painful. On my system, the first build took 6:01 min. The subsequent clean package build took 2:38.

Running Mission Control

To start your recently built Mission Control, run:

Windows

target\products\org.openjdk.jmc\win32\win32\x86_64\jmc.exe -vm %JAVA_HOME%\bin

Mac OS X

target/products/org.openjdk.jmc/macosx/cocoa/x86_64/JDK\ Mission\ Control.app/Contents/MacOS/jmc -vm $JAVA_HOME/bin

Contributing to JDK Mission Control

To contribute to JDK Mission Control, you need to have signed an Oracle Contributor Agreement. More information can be found here:

http://openjdk.java.net/contribute/

Don’t forget to join the dev list:

http://mail.openjdk.java.net/mailman/listinfo/jmc-dev

We also have a Slack (for contributors), which you can join here:

https://join.slack.com/t/jdkmissioncontrol/signup

More Info

For more information on how to run tests, use APIs etc, there is a README.md file in the root of the repo. Let me know in the comments section if there is something you think I should add to this blog post and/or the README!

Mission Control is Now Officially on GitHub!

Since this morning, the JDK Mission Control (JMC) project has gone full Skara! mc_512x512This means that the next version (JMC 8.0) will be developed over at GitHub.

To contribute to JDK Mission Control, you (or the company you work for) need to have signed an OCA, like for any other OpenJDK-project. If you already have an OpenJDK username, you can associate your GitHub account with it.

Just after we open sourced JMC, I created a temporary mirror on GitHub to experiment with working with JMC at GitHub. That mirror is now closed for business. Please use the official OpenJDK one from now on:

https://github.com/openjdk/jmc

If you forked or stared the old repo, please feel free to fork and/or star the new one!

Compressing Flight Recordings

Flight recordings are nifty binary recordings of what is going on in the runtime and the application running on it. A flight recording contains a wide variety of information, such as various kinds of profiling information, threat stall information and a whole host of other information. All adhering to a common event model and with the ability to dynamically add new event types.

In the versions of JFR since JDK 9, some care was taken to reduce the memory footprint by LEB 128 encoding integers, noting that many things, like constant pool indices, usually occupy relatively low numbers. The memory footprint was cut in about half, compared to previous versions of JFR.

Now, sometimes you may want to compress the JFR data even further. The question then is – how much can you save if you compress the recordings further, and what algorithms would be best suited for doing the compression? What if you want the compression activity to use as little CPU as possible?

My friend and colleague at Datadog, Jaroslav Bachorik, set out to answer that question for some typical recording shapes that we see at Datadog, using a set of compression algorithms from Apache Commons Compress (bzip2, LZMA, LZ4), the built in GZip, a dedicated LZ4 library, XZ, and Snappy.

Below is a table of his findings for “small” (~1.5 MiB) and “large” (~5 MiB) recordings from one of our services. The benchmark was run on a MacBook Pro 2019. Now, you’d have to test on your own recordings to truly know, but I suspect that these results will hold up pretty well with other kinds of loads as well.

Algorithm Recording Size Throughput Compression Ratio Utility
Gzip small 24.299 3.28 79.647
Gzip large 5.762 3.54 20.436
BZip2 small 6.616 3.51 23.198
BZip2 large 1.518 3.84 5.826
LZ4 small 133.115 2.40 319.394
LZ4 large 38.901 2.57 100.009
LZ4 (Apache) small 0.055 2.74 0.152
LZ4 (Apache) large 0.013 3.00 0.039
LZMA small 1.828 4.31 7.882
LZMA large 0.351 4.37 1.533
Snappy small 134.598 2.27 305.494
Snappy large 35.986 2.49 89.692
XZ small 1.847 4.31 7.964
XZ large 0.349 4.37 1.523

Throughput is recordings/s. Utility is throughput * compression ratio, and meant to capture the combination of compression strength and performance. Note that the numbers are not normalized – only compare numbers in the same size category.

Summary / TL;DR

  • The built-in GZip is doing a fairly good/balanced job of compressing flight recordings
  • You can get the best utility out of LZ4, closely followed by Snappy, but you sacrifice some compression
  • If you’re prepared to pay for it, LZMA and XZ give a good compression ratio
  • All credz to Jaroslav for his JMH-benchmark and all the data

JFR is Coming to OpenJDK 8!

I recently realized that this isn’t common knowledge, so I thought I’d take the opportunity to talk about the JDK Flight Recorder coming to OpenJDK 8! The backport is a collaboration between Red Hat, Alibaba, Azul and Datadog. These are exciting times for production time profiling nerds like me. Smile

The repository for the backport is available here:

http://hg.openjdk.java.net/jdk8u/jdk8u-jfr-incubator/

The proposed CSR is available here:

https://bugs.openjdk.java.net/browse/JDK-8230764

The backport is keeping the same interfaces and pretty much the same implementation as is available in OpenJDK 11, and is fully compatible. There were a few security fixes, due to there not being any module system to rely upon for isolation of the internals, also, some events will not be available (e.g. the Module related events) but other than that the API and tools work exactly the same.

JDK Mission Control will, of course, be updated to work flawlessly with the OpenJDK 8 version of JFR as well. The changes will be minute and are only necessary since Mission Control has some built-in assumptions that no longer hold true.

You can already build and try out OpenJDK 8 with JFR simply by building the JDK available in the repository mentioned above. Also, Aleksey Shipilev provide binaries – see here for details.

Have fun! Smile

Flight Recorder & Mission Control at Code One 2019

Code One is rapidly approaching (September 16-19). For fans of JDK Flight Recorder and JDK Mission Control, there will be a lot of relevant activities at Code One. This is an attempt to list them. If I missed something, please let me know!

Sessions

Here are the regular sessions:

Session Name

Presenters Day Time

Location

JDK Mission Control: Where We Are, Where We Are Going [DEV4284]

David Buck Monday 9:00 Moscone South
Room 301

Introduction to JDK Mission Control and JDK Flight Recorder [DEV2316]

Marcus Hirt
Klara Ward
Monday 16:00 Moscone South
Room 202
Improving Observability in Your Application with JFR and JMC [DEV3460] Marcus Hirt
Mario Torre
Tuesday 11:30 Moscone South
Room 201
Java Flight Recorder: Black Box of Java Applications[DEV3957] Poonam Parhar Wednesday 12:30

Moscone South
Room 203

Robotics on JDK 11? With Modules? Are You… [DEV2329] Marcus Hirt
Miro Wengner
Robert Savage
Wednesday 16:00

Moscone South
Room 313

Four Productive Ways to Use Open Source JFR and JMC Revisited [DEV3118] Sven Reimers
Martin Klähn
Thursday 11:15 Moscone South
Room 304
Enhanced Java Flight Recorder at Alibaba [DEV3667] Sanhong Li
Fangxi Yin
Guangyu Zhu
Thursday 12:15 Moscone South
Room 203

Performance Monitoring with Java Flight Recorder on OpenJDK [DEV2406]

Hirofumi Iwasaki
Hiroaki Nakada
Thursday 13:15 Moscone South
Room 201

Again, if I’ve missed one, please let me know!

Other Activities

  • There is going to be a hackergarten session around JMC and JFR, Wednesday at 14:30-16:00, inside of the Groundbreakers booth in the Exhibition Area.
  • On Friday a few JMC project members are planning to meet up for some coding between 10:00 and 12:00, and then have lunch together at 12:00. Ping me (Marcus) for an invite.
  • On Wednesday at 18:00 a few JMC project members are planning to go for dinner. Ping me (Marcus) for an invite.

Summary

  • Lots to do at Code One 2019 for fans of JFR and JMC.
  • Helpful links above. Winking smile

Deep Distributed Tracing with OpenTracing and the JDK Flight Recorder

Recently I had a talk at Code One about using OpenTracing together with the JDK Flight Recorder to do deep tracing. Since the session wasn’t recorded, I though I’d do a blog about it instead. Here we go…

Distributed tracing has been of interest for a very long time. Multiple companies have sprung up around the idea over the years, and most APM (Application Performance Management) solutions are built around the idea. Google released a paper around their large scale distributed systems tracing infrastructure in 2010 – Dapper, and there are now several open source alternatives for distributed tracing available inspired by the paper, such as Jaeger and Zipkin.

In Java land, pretty much all of the APMs are doing pretty much the same thing: they use BCI (byte code instrumentation) for getting the data, and then they present that data to the end-user in various ways, oftentimes using some kind of analysis to recognize common problems and suggesting solutions to the end users of the APM. The real differentiation is knowing what data to get, and what to do with the data once captured.

Since there was no standard, one problem was for vendors to inject helpful, vendor specific, information into the distributed traces. The vendor of a software component may have a quite good idea about what information would be helpful to solve problems. Some vendors support APM specific APIs for contributing the data, but more often than not the instrumentation is done using BCI by scores of developers working for the various APM companies. The same is true for maintainers of open source components – either skip the problem entirely and let the APM vendors come up with good instrumentations points (if your component is popular enough), or pick a popular APM and integrate with it. That is, until OpenTracing came along…

Introduction to OpenTracing

OpenTracing is an open source, vendor neutral, distributed tracing API. In other words, library developers can interact with one API to support multiple APM/Tracer vendors. Also, customers can add contextual information to distributed traces without worrying about vendor lock-in. Contributors to OpenTracing include LightStep, Jaeger, Skywalking and Datadog, and the specification is available on GitHub:

https://github.com/opentracing

The core API concepts in OpenTracing are (from the slides of my talk, DEV5435):


Trace


– A distributed operation, potentially spanning multiple processes


– Implicitly defined by the individual Spans in the trace (more soon)


– Can be thought of as a directed acyclic graph (DAG) of Spans


– The span in the root of the DAG is called the root Span


– The edges between the Spans are called References


Span


– Has an operation name


– Has a start timestamp


– Has a finish timestamp


– Has a SpanContext


• Has Baggage Items (key/value pairs which cross process boundaries)


• Implementation specific state used to identify the span across process boundaries)


– Zero or more key/value Span Tags


– Zero or more Span Logs (key/value + timestamp)


Reference


– Defines a direct casual relationship between two spans


– ChildOf


• Parent depends on the child in some way


• Note that it is legal for a finish timestamp of a child to be after that of any parent


– FollowFrom


• Parent does not depend on the result of the child in any way


• Note that it is legal for a FollowsFrom child to be started after the end of any ancestor

Also worth noting is that a Scope is a thread local activation of a span.

The Example

As an example, we’ll be using a simple application consisting of three microservices. It is part of the back-end of a fictional robot store. Robots can be ordered at the Orders service, and they will be produced in a Factory. There is also a Customers service keeping track of the customers. Finally there is a load generator that can be used to exercise the services.

image 

The code is available under https://github.com/thegreystone/problematic-microservices. (Yes, as the name indicates, the services come pre-packaged with built-in problems. :))

The services, as well as the load generator, have built-in tracing support, so for a full systems run with the load generator, you would get a trace (a DAG of spans), looking something along the lines of:

image

Or, in Jaeger, where you have time on the X axis:

tracing

In this case I have scrolled down a bit to focus on the factory. As can be seen, there is great variability in the time it takes to create a chassis and/or paint a robot. We have multiple production lanes, and we’d expect times across the factory lanes to be more even, not to mention much faster. So what gives?

Well, we can expand the operation to see if there was some additional information:

tracing_details

Now, sometimes the tags may include crucial pieces information that may help you solve the problem without needing any additional information. In this particular case, though, knowing that we were building a pink BB-8 isn’t really doing the trick.

What would be the next step? All too often the next step would be to look at the code around the instrumentation point, trying to figure out what was going on at the time simply from analyzing the code. Sometimes that may be quite hard. The problem may be in third party code not expected to behave badly. There may even be some other piece of code not directly in the code path causing the problems, perhaps an agent misbehaving and causing long lasting safe points in the JVM.

So, we’re screwed then? Nah. What if you had a magic tool that could record what was going on in the JVM and the application at the time of the incident? Something providing not only method profiling information, but a deeper view, including information about vm operations, memory allocation profiling, events for the usual application caused thread halts and much, much more. Something that could be always on, with very low overhead. And let’s say you ran with a tracer that added some contextual information, such as information that could be used to identify traces, spans and thread local span activations in the recorded data, and which allowed you to use your favourite tracer too? Then things would get interesting indeed…

Running with the JFR Tracer

For Code One I wrote a little delegating JFR tracer, which allows you to record contextual information into the flight recorder. It was meant as an example on how to do deep distributed tracing. Deep enough to solve entire classes of problems that are hard to solve without more detailed knowledge.

The tracer works with Oracle JDK 7+ and OpenJDK 11+ (it is a multi-release jar, a.k.a. mrjar), and the source is available on GitHub here:

https://github.com/thegreystone/jfr-tracer

The bundle is available from Maven Central, and here is the dependency you need to add:

<dependency>
<groupId>se.hirt.jmc</groupId>
<artifactId>jfr-tracer</artifactId>
<version>0.0.3</version>
</dependency>

Next you need to initiate your tracer and pass it to the constructor of the DelegatingJfrTracer, like so:

GlobalTracer.register(new DelegatingJfrTracer(yourFavTracer));

That’s it. When the tracer is running you will get contextual information recorded into the flight recorder.

Looking at the Recording

Dumping the flight recorder for the factory, and looking at the dump in the Threads view, might look something like this:

image

We can see that we have these long lasting monitor enter (Java Blocking) events, and looking at the stack traces directly by selecting individual events, or at the Lock Instances page, it is fairly obvious where the contention is:

image

We can, of course, create a custom OpenTracing view to make it easier to directly finding and homing in on long lasting traces (I’ll create a repo for a ready made one with some more flair at some point). Simply go the the Event Browser, and right click on the Open Tracing folder. Select “Create a new page using the selected event types”. You will now have a new page in the Outline. You can right click on the title on the page to rename it and switch icon.

Next select an arbitrary event, and right click on it. Select Group-By->Trace Id. In the new Group By table that appeared, select Visible Columns to enable (at least) the attribute showing the longest duration (the total duration of (wall clock) time the trace spent in the process that the recording came from). Next sort on the Longest Duration column.

In this case I’ve ran a few more (press enter in the single step load generator a few times, or let it just continuously add load):

image

You can, of course add additional tables with groupings that can be useful, for example, per thread. To quickly home in the entire user interface on a trace id of interest, just select a trace and choose “Store and Set as Focused Selection”:

SNAGHTML38aee12

Now you can go back to, for example, the Threads view, and click the Time Range: Set button in the upper right corner. Voila, you are in exactly the right place. You may also want to view concurrently occurring events in the same threads (see check boxes on top), and enable additional thread lanes:

SNAGHTML38f17fd

Summary

  • Distributed tracing is great, especially in today’s world of (very µ and plenty) µ-services.
  • For the Java platform, injecting trace/span-identifying information as contextual information into the JDK Flight Recorder is dynamite.
  • A simple example on how to do this automagically is available on my GitHub as a delegating Tracer, in an mrjar, supporting Oracle JDK 7+ and OpenJDK 11+:
        https://github.com/thegreystone/jfr-tracer
  • The slides for my Code One presentations can be found here:
        https://oracle.rainfocus.com/widget/oracle/oow18/catalogcodeone18?search=hirt
    (The relevant session for this blog is DEV5435.)
  • The JDK Flight Recorder r0xx0rz.
  • JDK Mission Control r0xx0rz.

Note that since the article was written, I have donated the tracer to OpenTracing.
See https://github.com/opentracing-contrib/java-jfr-tracer.

JMC Open Sourced!

This is going to be a short blog post, because it is past bedtime and I’m frankly pretty beat. Just wanted to say thank you to everyone who helped open source Java Mission Control in the relatively short period of time it was done in.

So a huge thank you to everyone on the Stockholm JMC team: Klara Ward, Ola Westin, Henrik Dafgård, Erik Greijus and Per Kroon. And to Dalibor Topic, who had the thankless job to go through all the source and built artifacts checking on, among other things, licenses and copyrights. Also thank you to Donald Smith for handling the internal license approval process, and Mark Reinhold for advice. Also, a big thank you to Guru, who is prepping to start building Oracle binary builds of the OpenJDK project. And to Iris who published the repos and set up the project on OpenJDK. And to everyone who I’ve forgotten to mention, since my brain is no longer really functioning at 2 a.m. You guys rock!

The repositories are available here:

      http://hg.openjdk.java.net/jmc

  • The jmc repo contains the source, which is buildable using Maven.
    Everything you need to know about building JMC is in the README.txt
    Please read it first, thoroughly, before asking questions.
  • The jmc-graphics repo contains artwork and graphic files, like splash screens and icons.

We also have a slack at https://jdkmissioncontrol.slack.com/.

If you have any questions, don’t hesitate to ask!

Kind regards,
Marcus

JMC & JFR in JDK 9 Labs

So JavaOne 2017 has now come and gone. This year it was a pretty great one! The Java keynote was IMHO the best one in years, with plenty of great announcements and interesting content. The sessions I managed to attend were also very high quality. Not to mention the dinners and the parties. Winking smile

One of the sessions I did this year was a hands-on-lab on Java Mission Control 6 and Java Flight Recorder in JDK 9.

IMG_6670

Since it was full, and since people who could not attend have asked for it, I promised to put the HoL on my blog. Well, here it is!

There are three files:

  1. The lab instructions.
  2. The full lab content for Windows x64 (easiest, but largest download, 856MiB).
  3. The project files only (all other platforms, 39MiB).

Hope this helps!

Please let me know if you find any errors in the labs, or if you get stuck!

/M

JMC 6 Automated Analysis Headless

This post will be a little bit wider than the title implies; it will be about using the JMC core JFR APIs in general. That said, one of the things we will be using it for, towards the end, will be to run the JMC 6 automated analysis headless.

This article will also cheat by talking about JMC 6.1, so that anyone reading this article will have relevant information for JMC 6.1 as well. Winking smile 

The JAR-files Needed

To run these examples you will need to have the following Jar-files from the JDK_HOME/lib/missioncontrol/plugins folder on your classpath:

Prefix

Version needed

Comment

com.oracle.jmc.common

>= 6.0.0

Classes common to JMC, such as stacktrace definitions, the content type system (quantities/ units of measurement) etc.
com.oracle.jmc.flightrecorder

>= 6.0.0

The Java Flight Recorder parser, classes for extracting information from a recording.
com.oracle.jmc.flightrecorder.rules

>= 6.0.0

The core definitions and classes for automated analysis of recordings.
com.oracle.jmc.flightrecorder.rules.jdk

>= 6.0.0

Contains the rules for the JDK (such as rules for synchronization trouble, long lasting VM operations and much more).
com.oracle.jmc.flightrecorder.ext.wls.parser

>= 6.0.0

(Optional, only relevant for users wanting WebLogic Server extensions)
com.oracle.jmc.flightrecorder.ext.wls.rules

>= 6.0.0

(Optional, only relevant for users wanting WebLogic Server extensions)

Loading a Recording

Loading a recording is done through a call to the JfrLoaderToolkit#loadEvents(). It takes a file argument, and returns an IItemCollection. The IItemCollection can be thought of as a collection of events.

The IItemCollection supports operations like filter(), and getAggregate(). Used correctly, you should rarely need to rely on external iteration.

Here is an example which loads a recording, and calculates the standard deviation for the java monitor enter events in a recording:

import java.io.File;

import com.oracle.jmc.common.IDisplayable;
import com.oracle.jmc.common.item.Aggregators;
import com.oracle.jmc.common.item.IItemCollection;
import com.oracle.jmc.common.item.ItemFilters;
import com.oracle.jmc.common.unit.IQuantity;
import com.oracle.jmc.flightrecorder.JfrAttributes;
import com.oracle.jmc.flightrecorder.JfrLoaderToolkit;
import com.oracle.jmc.flightrecorder.jdk.JdkTypeIDs;

/**
 * Finds out the standard deviation for the java monitor enter events.
 */
public class LoadRecording {
	public static void main(String[] args) throws Exception {
		DemoToolkit.verifyFirstFileArgument(RunRulesOnFile.class, args);
		
		IItemCollection events = JfrLoaderToolkit.loadEvents(new File(args[0]));
		IQuantity aggregate = events.apply(ItemFilters.type(JdkTypeIDs.MONITOR_ENTER))
				.getAggregate(Aggregators.stddev(JfrAttributes.DURATION));
		
		System.out.println("The standard deviation for the Java monitor enter events was "
				+ aggregate.displayUsing(IDisplayable.AUTO));
	}
}

Note the stream-like syntax. The JMC libraries work well together with streams, but are compiled on JDK 7, and can run on a JDK 7 compliant runtime. Also note that some class names start with Jfr whilst others class names start with Jdk. The difference being that some concepts, like DURATION, are intrinsic to JFR, whilst other are defined in terms of the Java JDK classes or runtime, like MONITOR_ENTER.

The JMC core libraries provide common statistical aggregators, and accessors for common attributes. Should you feel something is missing, it is easy to add to the built-in operations.

Also note the IQuantity returned by the aggregator. The built-in system for handling quantities and units of measurement makes it easy to calculate with, and display, quantities in different units of measurement.

Performing an Automated Analysis Headless

It is also easy to run the JMC rules headless using these core libraries. Simply evaluate the rules against the IItemCollection. Here is a simple example iterating through the rules and evaluating them one by one:

import java.io.File;
import java.util.concurrent.RunnableFuture;

import com.oracle.example.jmc6jfr.rules.util.DemoToolkit;
import com.oracle.jmc.common.item.IItemCollection;
import com.oracle.jmc.common.util.IPreferenceValueProvider;
import com.oracle.jmc.flightrecorder.JfrLoaderToolkit;
import com.oracle.jmc.flightrecorder.rules.IRule;
import com.oracle.jmc.flightrecorder.rules.Result;
import com.oracle.jmc.flightrecorder.rules.RuleRegistry;

public class RunRulesOnFileSimple {
	public static void main(String[] args) throws Exception {
		File recording = DemoToolkit.verifyRecordingArgument(RunRulesOnFileSimple.class, args);
		IItemCollection events = JfrLoaderToolkit.loadEvents(recording);
		
		for (IRule rule : RuleRegistry.getRules()) {
			RunnableFuture<Result> future = rule.evaluate(events, IPreferenceValueProvider.DEFAULT_VALUES);
			future.run();
			Result result = future.get();
			if (result.getScore() > 50) {
				System.out.println(String.format("[Score: %3.0f] Rule ID: %s, Rule name: %s, Short description: %s",
						result.getScore(), result.getRule().getId(), result.getRule().getName(),
						result.getShortDescription()));
			}
		}
	}
}

That said, if you are not constrained to have to run on JDK 7, you can always run the rules in parallel, for example by employing parallel streams:

import java.io.File;
import java.util.List;
import java.util.concurrent.ExecutionException;
import java.util.concurrent.Executor;
import java.util.concurrent.Executors;
import java.util.concurrent.RunnableFuture;
import java.util.stream.Collectors;
import java.util.stream.Stream;

import com.oracle.jmc.common.item.IItemCollection;
import com.oracle.jmc.common.util.IPreferenceValueProvider;
import com.oracle.jmc.flightrecorder.JfrLoaderToolkit;
import com.oracle.jmc.flightrecorder.rules.IRule;
import com.oracle.jmc.flightrecorder.rules.Result;
import com.oracle.jmc.flightrecorder.rules.RuleRegistry;

/**
 * Runs the rules on the events in the specified file in parallel, then prints
 * them in order of descending score.
 */
public class RunRulesOnFile {
	private final static Executor EXECUTOR = Executors
			.newFixedThreadPool(Runtime.getRuntime().availableProcessors() - 1);
	private static int limit;

	public static void main(String[] args) throws Exception {
		if (args.length == 0) {
			System.out.println(
					"Usage: RunRulesOnFile <recording file> [<limit>]\n\tThe recording file must be a flight recording from JDK 7 or above. The limit, if set, will only report rules triggered with a score higher or equal than the limit.");
			System.exit(2);
		}
		IItemCollection events = JfrLoaderToolkit.loadEvents(new File(args[0]));
		if (args.length > 1) {
			limit = Integer.parseInt(args[1]);
		}
		Stream<RunnableFuture<Result>> resultFutures = RuleRegistry.getRules().stream()
				.map((IRule r) -> evaluate(r, events));
		List<Result> results = resultFutures.parallel().map((RunnableFuture<Result> runnable) -> get(runnable))
				.collect(Collectors.toList());
		results.sort((Result r1, Result r2) -> Double.compare(r2.getScore(), r1.getScore()));
		results.stream().forEach(RunRulesOnFile::printResult);
	}

	public static RunnableFuture<Result> evaluate(IRule rule, IItemCollection events) {
		RunnableFuture<Result> evaluation = rule.evaluate(events, IPreferenceValueProvider.DEFAULT_VALUES);
		EXECUTOR.execute(evaluation);
		return evaluation;
	}

	public static Result get(RunnableFuture<Result> resultFuture) {
		try {
			return resultFuture.get();
		} catch (InterruptedException | ExecutionException e) {
			e.printStackTrace();
		}
		return null;
	}

	private static void printResult(Result result) {
		if (result.getScore() >= limit) {
			System.out.printf("(%.0f) [%s]: %s\nDetails:\n%s\n============<End of Result>============\n",
					result.getScore(), result.getRule().getId(), result.getShortDescription(),
					result.getLongDescription() == null ? "<no description>" : result.getLongDescription());
		}
	}
}

If you do not need that kind of control, there is a class available for performing automated analysis included with the JMC core library. To run an automatic analysis, simply run the class:

com.oracle.jmc.flightrecorder.rules.report.JfrRulesReport <files> –format <format> –min <severity>

…where files is one or more recordings, and <format> is and <min> is minimum severity to report [ok | info | warning]. In JMC 6.1.0 there is one additional class for generating the file, which produces an HTML in a format and style very similar to the looks of the JMC 6.0.0 Automated Analysis page:

com.oracle.jmc.flightrecorder.rules.report.html.JfrHtmlRulesReport <file> [<outputfile>]

…where <file> is the (single) recording to analyze, and where the optional <outputfile> is where the resulting HTML should be written. If no outputfile is specified, the result of the analysis will be written on stdout.

Adding Custom Rules

It’s easy to add you own heuristics. Do you find yourself with thousands of recordings, and would like to add a Bayesiean network to do some machine learning? The rules are all pure Java, so you can pretty much do whatever you want in a rule.

The easiest way to get started writing your own custom rules is to get yourself an Eclipse Oxygen or later, and install JMC and the experimental PDE plug-in into your Eclipse.

  1. Install Eclipse Oxygen or later.
  2. Got to the Mission Control homepage, and find the Eclipse Update site.
  3. Follow the instructions to install the Mission Control plug-ins.
    image
  4. Got to the Mission Control homepage, and find the Experimental Update site.
  5. Install the PDE plug-in.
    image

Here is a cheat sheet for the eclipse update sites for the JMC 6.0.0 release:

http://download.oracle.com/technology/products/missioncontrol/updatesites/base/6.0.0/eclipse/

http://download.oracle.com/technology/products/missioncontrol/updatesites/supported/6.0.0/eclipse/

http://download.oracle.com/technology/products/missioncontrol/updatesites/experimental/6.0.0/eclipse/

(JMC 6.1.0 will be released according to the same URL pattern.)

To start building your rule, press ctrl-n (or click the File | New | Other… menu) to bring up the New wizard.

image

Select Plug-in Project and hit Next. Name your rule project something exciting.

pic

Unclick that this plug-in will make contributions to the UI and hit Next.

pic2

Next select the Simple JFR Rule Wizard and click Finish (or Next, if you really wish to do some further customizations).

pic3

You will now have a new project in your workspace, containing an example rule. If you have compilation errors, you need to set JMC to be your target platform (see the next section). You can either just export your rule, put it on the class path with the other core libraries, or you can try out your rule by running JMC from within Eclipse with your new rule.

Starting JMC from within Eclipse with Workspace Plug-ins

Running JMC from within Eclipse with any plug-ins you are currently developing is a simple matter of setting the plug-in development target platform to your JMC installation, and launching that platform with your workspace plug-ins.

First set JMC to be your target platform.

  1. Go to Preferences in the main menu (Window | Preferences on Windows).
  2. Find Target Platform by typing Tar in the filter box:
    image
  3. Click Add… and type JMC 6 as name.
    image
  4. Press Add… and select Installation. Hit Next.
    image
  5. Browse to the JMC 6 installation directory (JDK_9_HOME/lib/missioncontrol), and hit Ok.
    SNAGHTML1538a63d
  6. Hit Finish, and Finish. You should now see your new platform.
    image
  7. Select the new JMC 6 platform, and hit Apply and Close to activate it.
    image

Everything should now compile cleanly. Next step is to run JMC with your new rule.

  1. Context click on your project and select Run as Eclipse Application
    SNAGHTML153e5cb4
  2. This should normally be it. If you for some reason run into trouble, go to Run Configurations, and make sure that your launcher is using the com.oracle.jmc.rcp.application.product.
    SNAGHTML1540e14b

You should now be able to see your rule in action. If you have not changed the rule code, try setting the environment variable in the launcher to various values between 0 and 100 and see what happens when you run the rule in JMC:

image

image

Exporting a Plug-in

You can export your rule by context-clicking on the project and selecting Export. In the Export wizard, select Deployable plug-ins and Fragments.

image 

Click Next, select a folder to export the plug-in to and Finish. The resulting jar can either be put on the class path to be included in the headless analysis, or put in the dropins folder of any Java Mission Control installation where you would want the rule to be available.

Summary

This blog described how to:

  • Get started using the JMC core libraries to read Java Flight Recordings (JDK 7, 8 and 9)
  • Get started doing headless analysis of Java Flight Recordings (JDK 7, 8 and 9)
  • Get started creating custom rules for analyzing Java Flight Recordings

Hope it helps someone!