Allocation Profiling in Java Mission Control

With the Hotspot JDK 7u40 there is a nifty new tool called Java Mission Control. Users of the nifty old tool JRockit Mission Control will recognize a lot of the features. This particular blog entry focuses on how to do allocation profiling with the Java Flight Recorder.

The JVM is a wonderful little piece of virtualization technology that gives you the illusion of an infinite heap. However that comes at a cost. Sometimes the JVM will need to find out exactly what on your heap is in use, and throw the rest away – a so called garbage collection. This is because the physical memory available to the JVM indeed is limited, and memory will have to be reclaimed and reused when it is no longer needed. In timing sensitive applications, such as trading systems and telco applications, such pauses can be quite costly. There are various tuning that can be done for the GC to make it less likely to occur. But I digress. One way to garbage collect less, is of course to allocate less.

Sometimes you may want to find out where the allocation pressure is caused by your application. There are various reasons as to why you may have reached this conclusion. The most common one is probably that the JVM is having to garbage collect more often and/or longer than you think is reasonable.

The JFR Allocation Event

In the Java Flight Recorder implementation in HotSpot 7u40 there are two allocation events that can assist in finding out where the allocations are taking place in your application: the Allocation in new TLAB and the Allocation outside TLAB events.

Just like for most events provided with the HotSpot JDK, there is of course a custom user interface for analyzing this in Mission Control, but before going there I thought we’d take a moment to discuss the actual events.

First you need to make sure they are recorded. If you bring up the default (continuous) template, you will notice that allocation profiling is off by default. Either turn it on, or use the profiling template. The reason it is off by default is that it may produce quite a lot of events, and it is a bit hard to estimate the overhead since it will vary a lot with the particular allocation behaviour of your application.

The Log tab in the Events tab group is an excellent place to look at individual events. Let’s check out what these actually contain:

They contain the allocation size of whatever was allocated, the stack trace for what caused the allocation, the class of what was allocated, and time information. In the case of the inside TLAB allocation events, they also contain the size of the TLAB.

Note that we, in the case of the (inside) TLAB allocation events, are not emitting an event for each and every location – that would be way too expensive. We are instead creating an event for the first allocation in a new TLAB. This means that we get a sampling of sorts of the thread local allocations taking place.

Also, note that in JRockit we used to have TLA events that perfectly corresponded to the allocations done in the nursery, and large object allocation events that corresponded to allocation done directly in old space. This is a little bit more complicated in HotSpot. The outside TLAB allocation events can both correspond to allocations done due to an allocation causing a young collection, and the subsequent allocation of the object in the Eden as well as the direct allocation of a humongous object directly in old space. In other words, don’t worry too much if you see the allocation of a few small objects among the outside of TLAB events. Normally these would be very few though, so in practice the distinction is probably not that important.

Using the Allocations Tab

The allocations tab actually consists of three sub tabs:

1. General
2. Allocation in new TLAB
3. Allocation outside TLAB

The General tab provides an overview and some statistics of the available events, such as total memory allocated for the two different event types. Depending on what your overarching goal is (e.g. tuning TLAB size, reducing overall allocation) you may use this information differently. In the example below it would seem prudent to focus on the inside TLAB allocations, as they contribute way more to the total allocation pressure.

The Allocation in new TLAB tab provides three more types of visualization specific to the in new TLAB events: Allocation by Class, Allocation by Thread and Allocation Profile. The Allocation Profile is simply the aggregated stack trace tree for all of the events. In the example below it can be seen that Integer object allocations correspond to almost all the allocations in the system. Selecting the Integer class in the histogram table shows the aggregated stack traces for all the Integer allocations, and in this example all of them originate from the same place, as seen below (click in pictures to show them in full size).

The Allocation outside TLAB tab works exactly the same way as the Allocation in new TLAB tab, only this time it is, of course, for the Allocation outside TLAB events:

Limitations

Since the Allocation in new TLAB events only represents a sample of the total thread local allocations, having only one event will say very little about the actual distribution. The more events, the more accurate the picture. Also, if the allocation behaviour vary wildly during the recording, it may be hard to establish a representative picture of the thread local allocations.

Thanks to Nikita Salnikov for requesting this blog entry!

Further reading and useful links

Related Blogs:
Java Mission Control Finally Released
Creating Flight Recordings
My JavaOne 2013 Sessions
Low Overhead Method Profiling with Mission Control

The Mission Control home page:
http://oracle.com/missioncontrol

Mission Control Base update site for Eclipse:
http://download.oracle.com/technology/products/missioncontrol/updatesites/base/5.2.0/eclipse/

Mission Control Experimental update site (Mission Control plug-ins):
http://download.oracle.com/technology/products/missioncontrol/updatesites/experimental/5.2.0/eclipse/

The Mission Control Facebook Community Page (not kidding):
http://www.facebook.com/pages/Java-Mission-Control/275169442493206

Mission Control on Twitter:
@javamissionctrl

Me on Twitter:
@hirt