Using GraalVM, JavaFX and a Clanker to Build a Cross Platform Desktop App

So, this particular rabbit hole started when I, to my dismay, was running out of diskspace on one of my disks, filling it up with local AI models. I wanted to use a pretty and nice tool to start reclaiming space, something using a sunburst visualization. After some quick research, I couldn’t find anything that I really liked. Nothing was pretty, except DaisyDisk on Mac. But that was only available on Mac. And there was nothing out there that allowed you to interact with it whilst data was being collected. Also the stuff that I found was committed to just one of many useful visualizations. And I wanted to dynamically be able to switch theme. And coloring. And <insert impossibly long list of wants>…

Since there was a bunch of other more technical things I’d wanted to try for a while anyways, I decided to go ahead and use this project as an excuse to try them:

  1. Is it possible to reasonably use GraalVM native image to make binary desktop apps with Java?
  2. Is it possible to use GraalVM with JavaFX?
  3. If I need to do native bindings, how does that work in GraalVM?

I had already used GraalVM native image for various other projects where it makes sense, such as in my email MCP server, but it had always mostly been for smaller Java projects where I needed quick startup and low memory utilization. To make this one work well, I was pretty sure I’d need to do platform specific magic to make it as fast as can be.

DiskSpace showing the default sunburst visualization.

My Friend the AI Model

This project was a proper collaboration between me and my AI models, in an area where they usually are very strong — a new greenfield project. Well defined outcomes. Very strong design cues and up-front technology choices. I worked on the design document first, together with the AI. I laid out incremental goals, like first trying to get the thing to compile with a simple scene, trying to get a simple native binding to work, etc. Before implementing anything, that design/architectural document was discussed, debated, edited and finalized. Next, I was providing guidance and help every step of the way, sometimes coding actively when I felt things were hitting a dead end and I could just do the right thing quicker myself.

Design language is pretty hard to communicate, and abstract ideas like how much space something needs to take to “look just right”, or the particular shade of grey that is required to make something “look good” are so subjective and very hard to get to a good place without a lot of back and forth.

At the same time, very technical work was sometimes (not always) executed flawlessly, saving tons of time. Did this project go quicker than if I had done it just on my own? Undoubtedly so. All the hunting down weird linkage problems, setting up the project to build properly on all platforms. All that tedious, no fun work, went massively faster. I spent most of the time doing back and forth with the AI model and getting it to do the right thing, and providing it with all the ideas and context to be able to solve most of its problems with as little additional context as possible. Which brings me to…

DiskSpace, now in DarkMode, showing the squarified tree map visualization.

Performance

You’d think that performance in the rendering would not ever become a problem in a project like this, but lo and behold, it did. Every once in a while when scanning my biggest and most diversely populated disks (C: on my Windows box), it would freeze. I first suspected GC. Since I was going to do the analysis with JFR anyways, I also introduced custom JFR events for context, to make it easier to attribute the JFR data to actual real world events.

In the end, I added four of them:

  1. Scan
    One event per scan run (carrying strategy, root path, file count, total bytes, using the @DataAmount annotation to ensure everything is rendered properly in JMC).
  2. Render
    A render event per UI repaint (mode, trigger, node count, canvas dimensions).
  3. VisualizationActive
    An event spanning the time a specific visualization was active, with a repaint counter.
  4. UserAction
    An event per keypress or click.

The events were all stitched together via a per-scan correlation ID, so that it’s easy to contrast different scans to each other and pivot between scans and all the renders, visualizations and user actions changing them. For the render event, I disabled stack trace capturing. It wouldn’t be that exciting given the other events I have, not to mention the additional cost of walking the stacks over and over again.

@Name("se.hirt.diskspace.Render")
@Label("Visualization Render") 
@Category({"DiskSpace", "UI"})
@Description("One repaint of the visualization on the JavaFX UI thread.")
@StackTrace(false) // 60+ Hz: per-event stack capture unnecessary work here
public static class RenderEvent extends Event {
  @Label("Mode")
  @Description("SUNBURST or HEATMAP.")
  String mode;
  @Label("Trigger")
  @Description("scan-update / mode-change / resize / ...") 
  String trigger; 
  @Label("Node Count")
  @Description("Live file count at render time.")
  int nodeCount;
  @Label("Width")
  int widthPx;
  @Label("Height")
  int heightPx;
  @Label("Scan ID")
  @Description("Correlation ID matching the Scan event.")
  long scanId;
}

JFR isn’t properly supported in the GraalVM JDK version (21) used by GluonFX, so I ran the project in plain Java when measuring. I then ran the recordings through my JMC AI view (coming in JMC 10, I’ll write more about it later), which was able to quickly root cause the algorithmic complexity in the squarified tree map renderer as the culprit, and suggest improvements.

Properly chosen contextual events is as usual King, and the self describing nature of JFR makes adding them, with the proper metadata, a breeze for the right AI tools to use. No parser updates, or any additional changes required to have them use it other than just adding it.

A Few More Fun Bits

This project has more under-the-hood stuff than fits one post, but a few bits are worth calling out:

The Windows MFT Scanner

Walking a directory tree the obvious way (FindFirstFile / FindNextFile) issues a
syscall per file just to read its size. On a C: drive with millions of files, that can add up to minutes. NTFS has a much better idea: the Master File Table is already a giant table of every file’s metadata, and Windows exposes the whole thing via FSCTL_ENUM_USN_DATA. Read it once, get every file on the volume in a few seconds.

The catch is that it needs SeBackupPrivilege, which means UAC elevation. That’s the reason DiskSpace on Windows has an auto-elevation preference (Ask once / Always / Never). Once elevated, DiskSpace will drop all privileges except for the backup one, and scans drop from minutes to seconds on the biggest drives I can find to throw at it.

The macOS Bulk Scanner

The macOS equivalent of the MFT trick is getattrlistbulk(2). Instead of one stat syscall per file, it returns metadata for roughly 500–800 entries per call. Combined with a per-directory ForkJoinPool and parallelism tuned to the storage profile, APFS scans finish very quickly. 

The storage-profile detection (the thing that decides “parallel-16 for NVMe, sequential for spinning disks”) walks the IORegistry via DiskArbitration + IOKit to find the underlying media type. Same plumbing diskutil info uses internally, just routed through GraalVM’s @CFunction bindings rather than the command line.

Interact While It’s Still Scanning

The thing that bugged me about every other disk-usage tool I tried: you click “scan”, wait until it finishes,
then you can explore. With DiskSpace the sunburst materialises as data streams in. You can drill into a folder before its size is even known, switch tabs, switch visualization modes (sunburst ↔ treemap), or kick off a second scan on a different drive — all without waiting for anything to complete.

The volume picker has the same property. If you have a dodgy SD card that mounts but doesn’t respond, the old picker would block startup for thirty seconds or more. The new one, thanks to one virtual thread per volume root, shows up instantly and fills in each row as it resolves, with the slow ones spinning quietly until they decide to answer.

So Animate, Much LERPs, Wow

The drill-in / drill-out animation on the sunburst (the one where you click a sector and the whole visualization smoothly reflows so that sector becomes the new centre) is built out of nothing but linear interpolation and a single ease-out curve. The layout system computes the before and after sector positions (depth, start angle, sweep angle) and the animation just lerps each value from one to the other over a few hundred milliseconds.

The fun bits aren’t in the lerp itself, they’re in what you give it at the endpoints. New sectors that
weren’t visible before the drill, deeper nodes that only show up at the new zoom level, start their animation
with sweep = 0 at the midpoint of their final position. So instead of popping into existence, they blossom outward from a single angular point until they reach full width. Departing sectors do the inverse: they collapse to their own midpoint and fade to transparent. The outgoing inner ring stays in place and fades out while the new one fades in over it.

Native Image + JavaFX + Native Bindings

GraalVM native-image + Gluon Substrate + JavaFX gives you a single-binary desktop app with native application startup times. The platform-specific APIs (the MFT scanner, getattrlistbulk,
DiskArbitration, IOKit, the Win32 storage queries) are all bound through GraalVM’s @CFunction annotations — a handful of lines of Java per native function, no JNI shim, no separate C side, no -Djava.library.path dance at runtime. The native-image build resolves the symbols at link time against the system libraries, and at runtime the call overhead is comparable to a regular C function call. Clean and fast.

Sub-pixel Arc Culling

A small rendering trick on the sunburst side: when an arc would project to less than one screen pixel, it’s literally invisible — so don’t issue the draw call at all. On deep trees with millions of leaves that drops the per-frame node walk to a fraction of what it would otherwise be. Same principle a frustum-culling game engine uses, just in polar coordinates.

TL;DR

  • The AI was immensely helpful getting to the bottom of various linkage problems and ensuring that the project would build on the various platforms.
  • AI ended up doing much of the typing for this greenfield project.
  • Even the latest models need a ton of supervision and course corrections to yield good results.
  • With today’s AI models, building well still takes the same skill as building it yourself.
  • Yes, GraalVM native image works well for building native desktop apps with Java. GluonFX bridges it well to JavaFX, and @CFunction makes native bindings painless.

I just released 0.3.1 of DiskSpace here:
https://github.com/thegreystone/diskspace/releases

This release marks the first community contribution! Many thanks to Scott Gerring (@scottgerring,
https://github.com/scottgerring) for adding APFS clone deduplication to the macOS bulk scanner, so that scan totals on Mac now match Disk Utility and df properly.

Please give it a spin, and if you like it, consider giving it a star on GitHub. This will help me getting free signing of the DiskSpace artifacts from the SignPath foundation in the future.

JDK Mission Control 9.0.0 Released!

The 9.0.0 GA release of JDK Mission Control was just tagged in the JMC repo at GitHub! Since this is the source release, it may still take a bit of time until the downstream vendors release binary builds of JDK Mission Control 9.0.0. I will try to remember to tweet or say something on the JMC Facebook page once the binaries start showing up.


Mission Control 9.0 – New and Noteworthy


General


JMC 9 – New Release!
This is the latest (2024) major release of JDK Mission Control. JMC 9 requires JDK 17+ to run and introduces several new features, enhancements, and bug fixes. This version continues to support connecting to, and parsing JFR recordings from, OpenJDK 8u272+ and Oracle JDK 7u40+, and can open and visualize flight recordings from JDK 7 and 8. JDK Mission Control is available for Windows (x86_64), Mac OS X (ARM and x86_64), and Linux (ARM and x86_64).

jmc


Eclipse 4.30 support
The Mission Control client is now built to run optimally on Eclipse 2023-12 and later. To install JDK Mission Control into Eclipse, go to the update site (Help | Install New Software…). The URL to the update site will be vendor specific, and some vendors will instead provide an archive with the update site.

eclipse


Support for Linux/aarch64
JMC 9 is now built for Linux aarch64.

linuxaarch64


Support for dark mode
JMC 9 now supports dark mode. Go to Preferences, General | Appearance, and select the Dark theme to enable.

darkmode


Minor bugfixes and improvements
There are 118 fixes and improvements in this release. Check out the JMC 9.0 Result Dashboard for more information.

issues


Add user configuration for local JVM refresh interval
Previously the JVM Browser checked every 5000 ms for new JVMs. This can now be configured.

refreshinterval


Core


Better JFR parser performance
Multiple efforts have been made to reduce allocations in the JMC parser, including: reduced allocation of Doubles, reduced allocation rate in ParserStats. Also, when duration events aren’t ordered by their end time (e.g. events which stack so that the last event finishes first, or file reads with overlaps) `DisjointBuilder.add` can be slow because of the linear search for the lane, and then a linear time reordering. This has been improved with a binary search.

jfrperformance


Support checkpoint event sizes beyond u4 limit
The JMC JFR parser now support checkpoint event sizes beyond the u4 limit.

checkpointsize


Move non-Eclipse dependent classes from org.openjdk.jmc.ui.common to org.openjdk.jmc.common
There were a number of classes previously in jmc.ui.common that would be a great asset to the core distribution (and the third-party applications that consume jmc-core), and these classes now live in jmc.common. Please see JMC-7308 for further information.

reorganize


Move rjmx bundle from application to core
The rjmx classes and related services (FlightRecorderService) are now exposed for third-party application usage. Please see JMC-7069 for further information.

reorganize


Move org.openjdk.jmc.flightrecorder.configuration bundle from application to core
The org.openjdk.jmc.flightrecorder.configuration bundle contains many classes useful for working with jfr, and are now available in core. Please see JMC-7307 for further information.

reorganize


Java Flight Recorder (JFR)


The Event Browser now supports searching and showing event type ids
Searching in the search bar now also searches event type IDs, and there is also a (by default hidden) column that makes it easy to show the event type IDs for the shown events.

eventtypeid


Add support for enabling jfr on native images
Previously JMC was unable to start flightrecorder on a graalvm native image, even if there is built-in jfr support. This has now been fixed.

native-image-alt


Java based flamegraph visualization
The previous flamegraph visualization takes place in an embedded browser component (provided by the Eclipse platform), unfortunately this approach has some drawbacks, the first being a bit slow. This view is now using a Java (Swing) based flamegraph library. Also, the flame graph model creation performance have been improved.

flamegraph


Visualization and Rule for FileChannel.force()
The File I/O page has been updated to show force related information. There are two new columns added – Force Count and Update Metadata. Both are hidden by default and can be enabled by right clicking the table. The chart will also include a File Force row. There is a preference setting for the associated file force rule, where the peak duration warning limit can be set. See JMC PR#533 for more information.

fileforce


Rule that checks on G1 pause time target compliance
New rule that looks at the pause time target and compares it to the actual pauses.

rule


Rule that looks at finalization statistics
JDK 18 comes with a FinalizationStatistics event that helps users find where in their application finalizers are run. This is important as finalization has been deprecated for removal in a future release. For more information about finalization and its flaws, see https://openjdk.java.net/jeps/421. Even if an application doesn’t implement any finalize() methods, it may rely on third-party libraries that does. Static analysis of third-party libraries using “jdeprscan –for-removal” can be used to list those classes, but it will not tell if they are being used. For example, an application may be missing a call to a close() method, so the resource is cleaned up by the finalizer, which is sub-optimal.

rule


Rule that detects GC Inverted Parallelism
Rule inspired by the “Inverted Parallelism” analysis in Garbagecat. See JMC-8144 for more information.

rule


Support for the new JPLIS agent events
There is now a new page and rule for loaded JPLIS agents. See JMC-8054 for more information.

agent


Twitter plug-in removed
Due to changes in APIs and cost of maintenance, the Twitter plug-in has been removed.

twitterplugin


Bug Fixes


Area: Agent
Issue: 8045
Synopsis: retransformClasses() doesn’t re-transform all needed classes

The retransformClasses() methods in Agent and AgentController use Class.forName() to try to get the class objects of classes needed to re-transform. This obviously doesn’t work for classes loaded by classloaders different from the one which loads the agent. Those classes would be instrumented if they were loaded after their event probes were defined the AgentController. But when loaded earlier they would not be instrumented. This has been fixed.

Area: Agent
Issue: 8048
Synopsis: Agent throws exceptions on missing or empty descriptions

When the description of an event or value is empty or missing, the agent fails with exceptions. This has now been fixed.

Area: Console
Issue: 8154
Synopsis: Some JMX attributes are missing unit specifications in the Console

The missing unit specifications have now been added.

Area: Core
Issue: 8063
Synopsis: IMCFrame Type cache not synchronized

The type cache used in the IMCFrame Type inner class wasn’t synchronized and could cause a concurrent modification exception during e.g. JFR parsing. This has been fixed.

Area: Core
Issue: 8156
Synopsis: JfrRulesReport.printReport does not respect verbosity for text and json

The verbosity flag for text and json reports didn’t work. This has been fixed.

Area: Core
Issue: 8041
Synopsis: JfrRulesReport json reports produce incomplete results

While generating JFR Rules Reports in json format, the results were incomplete. The components “message” and “detailedMessage” were not populated. This has been fixed.

Area: JFR
Issue: 7885
Synopsis: Graphical rendering of dependency view fails due to heap memory drain

Also JMC-7496. The dependency view drains the heap memory and causes out-of-memory exceptions and performance delays. This has been improved.


Known Issues


Area: JFR
Issue: 7071
Synopsis: JMC can’t attach to jlinked JVMs

This one is still under investigation, but it seems JMC can’t attach to certain jlinked images.

Area: JFR
Issue: 7003
Synopsis: The graph view, heatmap view and dependency view does not work on Windows

This is due to a problem with the Windows based browser component in SWT. We’re hoping for a fix in the component for a future version of the Eclipse platform.

JDK Mission Control 8.3.0 Released!

The latest release of JDK Mission Control was recently released! Since I am a bit late with this blog, there are already some binary releases available, for example:

Eclipse Mission Control:
https://adoptium.net/jmc/

Zulu Mission Control:
https://www.azul.com/products/components/azul-mission-control/

 

Mission Control 8.3 – New and Noteworthy


General


JMC 8.3 – New Release!
This is a new minor release of JDK Mission Control. The JMC application requires JDK 11+ to run, but can still be used to connect to, and parse JFR recordings from, OpenJDK 8u272+ and Oracle JDK 7u40+. It can also still open and visualize flight recordings from JDK 7 and 8. JDK Mission Control is built for Windows (x86_64), Mac OS X (ARM and x86_64), as well as Linux (x86_64).

jmc[1]


Eclipse 4.24 support
The Mission Control client is now built to run optimally on Eclipse 2022-06 and later. To install JDK Mission Control into Eclipse, go to the update site (Help | Install New Software…). The URL to the update site will be vendor specific, and some vendors will instead provide an archive with the update site.

eclipse[1]


Minor bugfixes and improvements
There are 51 fixes and improvements in this release. Check out the JMC 8.3 Result Dashboard (https://bugs.openjdk.org/secure/Dashboard.jspa?selectPageId=21205) for more information.

issues


Core


Parser improvements
The performance of the FastAccessNumberMap has been improved for sparse values.

noimage[1]


Java Flight Recorder (JFR)


Dependency View
There is a new view for visualizing call dependencies. There are two modes of operation in the view, chord diagram and edge bundling. In the edge bundling visualization, hover over packages to see dependencies highlighted in colors: green means that methods in the linked package is called by methods in the package being hovered over, yellow means that methods in the linked package are calling mathods in the package being hovered over. Red means that methods in the packages are calling each other. To show the Dependency View, go to Window | Show View | Other… and select the Dependency View under the Mission Control folder.

dependency


Graph Pruning
The graph view in JMC can now be pruned to focus on the most impactful nodes. Select the target number of nodes, and the visualization will show at most that many number of nodes.

graphpruning


Selectable Attribute
It is now possible to select which attribute to use for the weights in the trace view and the flame graph view.

attribute


Parser improvement
The parser now supports parsing events with char fields.


Bug Fixes


Area: General
Issue: 7813
Synopsis: Unable to open Help page in macOS M1 when JMC started with JDK11

The help page was inaccessible, throwing an error on macOS M1 when JMC is run on JDK 11.0.16. This is now fixed.

Area: General
Issue: 7321
Synopsis: Unable to view JMC Help Contents (HTTP ERROR 500 ) when booted with JDK 17 or higher

The help page was inaccessible, throwing an error, when running on JDK 17+. This is now fixed.

Area: JFR
Issue: 7812
Synopsis: Unable to open links from Automated Result analysis page

Links in the results of the automated analysis results would not open properly on Linux and Mac OS X. Now they do open in situ.


Known Issues


Area: General
Issue: 4270
Synopsis: Hibernation and time

After the bugfix of https://bugs.openjdk.java.net/browse/JDK-6523160 in JDK 8, the RuntimeMXBean#getUptime() attribute was re-implemented to mean “Elapsed time of JVM process”, whilst it previously was implemented as time since start of the JVM process. The uptime attribute is used by JMC, together with RuntimeMXBean#getStartTime(), to estimate the actual server time. This means that time stamps, as well as remaining time for a flight recording, can be wrong for processes on machines that have been hibernated.

Area: General
Issue: 7953
Synopsis: Unable to install JMC Plugins on Eclipse 4.25

Because of updates to the naming of certain platform dependencies in Eclipse 4.25 (e.g. JUnit 5 bundles now have name of the form junit-* instead of Orbit variants org.junit.*), it will no longer be possible to install the plug-in version of JMC into Eclipse 4.25+. This will be resolved in a later version of JMC. A workaround for now is to build JMC from the mainline (9.0 EA), and installing the plug-in version from the resulting update site archive.

Area: JFR
Issue: 7947
Synopsis: JMC crashes while performing flight recording on MacOS 13.0_x64

JMC can crash when completing a recording on MacOS 13.0 on x64. It seems to be related to running JavaScript in the Browser component. Eclipse is investigating the issue here.

Area: JFR
Issue: 7071
Synopsis: JMC can’t attach to jlinked JVMs

This one is still under investigation, but it seems JMC can’t attach to certain jlinked images.

Area: JFR
Issue: 7003
Synopsis: The graph, flame graph view, heatmap view and dependency view does not work on Windows

This is due to a problem with the Windows based browser component in SWT. We’re hoping for a fix in the component for a future version of the Eclipse platform.


TL;DR

There is a new version of JMC out. Have fun! Smile

JDK Mission Control 8.2.0 Released!

The latest release of JDK Mission Control was just released! Since this is the source release, it may still take a bit of time until the downstream vendors release binary builds of JDK Mission Control 8.2.0. I will try to remember to tweet or say something on the JMC Facebook page once the binaries start showing up.

Here’s what’s new:

Mission Control 8.2 – New and Noteworthy


General


JMC 8.2 – New Release!
This is a new minor release of JDK Mission Control. The JMC application requires JDK 11+ to run, but can still be used to connect to, and parse JFR recordings from, OpenJDK 8u272+ and Oracle JDK 7u40+. It can also still open and visualize flight recordings from JDK 7 and 8.

jmc


Eclipse 4.22 support
The Mission Control client is now built to run optimally on Eclipse 2021-06 and later. To install JDK Mission Control into Eclipse, go to the update site (Help | Install New Software…). The URL to the update site will be vendor specific, and some vendors will instead provide an archive with the update site.

eclipse


Minor bugfixes and improvements
There are 83 fixes and improvements in this release. Check out the JMC 8.2 Result Dashboard (https://bugs.openjdk.java.net/secure/Dashboard.jspa?selectPageId=20804) for more information.

issues


Binary build for Apple ARM
JDK Mission Control is now built for Apple ARM, allowing JMC to be run natively (without Rosetta x86 emulation) on Apple M1.


Core


Parser support for async profiler
Parser support has been added for frame types generated by async profiler, such as Native, C++ and Kernel.


System.gc() rule
There is now a new rule for explicit invocations of System.gc().

dukegc


Java Flight Recorder (JFR)


Heat map view
A new heat map view has been added, which is handy for seeing when events are taking place. Use Window | Show View | Other…, and select the Heatmap View under Mission Control and click Open to open the view.

heatmap


Websocket for selections
There is a new websocket API available that pushes stack trace data from selections in the JFR UI as JSON on a user defined port. This allows for programmatic control of the visualization directly in the browser. Tools like observablehq.com can be used to invent new visualizations, or to alter the visualization. To get started, simply go to the Flight Recorder preferences in JMC, and select the Websocket port to use (0 to disable). A set of example visualizations are available here: https://observablehq.com/collection/@cimi/java-mission-control.

websocket


Bug Fixes


Area: JFR
Issue: 7403
Synopsis: JFR parser struct types hashcode problem

Some JFR parser struct types were using lazily initialized attributes which happen to be a part of hashCode/equals computations.

Area: JFR
Issue: 7532
Synopsis: Delays in rendering of JMX console graphs

Sometimes the updates of the JMX console graphs would be severely delayed on MacOS. This is now fixed.

Area: JFR
Issue: 7068
Synopsis: JfrRecordingTest (uitest) hangs on the automated analysis page

Trying to run uitests on Fedora hangs on JfrRecordingTest. This was fixed after the Eclipse platform update.


Known Issues


Area: General
Issue: 4270
Synopsis: Hibernation and time

After the bugfix of https://bugs.openjdk.java.net/browse/JDK-6523160 in JDK 8, the RuntimeMXBean#getUptime() attribute was re-implemented to mean “Elapsed time of JVM process”, whilst it previously was implemented as time since start of the JVM process. The uptime attribute is used by JMC, together with RuntimeMXBean#getStartTime(), to estimate the actual server time. This means that time stamps, as well as remaining time for a flight recording, can be wrong for processes on machines that have been hibernated.

Area: JFR
Issue: 7071
Synopsis: JMC can’t attach to jlinked JVMs

This one is still under investigation, but it seems JMC can’t attach to certain jlinked images.

Area: JFR
Issue: 7003
Synopsis: The graph and flame graph view does not work on Windows

This is due to a problem with the Windows based browser component in SWT. We’re hoping for a fix in the component for a future version of the Eclipse platform.

JMC Core Now on Maven Central!

Good news! The JDK Mission Control core library bundles are now available on Maven Central, making it easier than ever to use things like the the JMC JDK Flight Recorder parser to transparently parse and extract information from flight recordings ranging from Oracle JDK 7 and up to the very latest versions of OpenJDK.

I have updated my JShell example so that you can see an example of them being used.

For just using the parser, you will typically need the common and flightrecorder bundles. If you also need the rules engine, you add flightrecorder.rules, and if you want the base set of heuristics for the jdk, also add flightrecorder.rules.jdk.

For example:

	<properties>
		<jmc.version>8.0.1</jmc.version>
	</properties>
	<dependencies>
		<dependency>
			<groupid>org.openjdk.jmc</groupid>
			<artifactid>common</artifactid>
			<version>${jmc.version}</version>
		</dependency>
		<dependency>
			<groupid>org.openjdk.jmc</groupid>
			<artifactid>flightrecorder</artifactid>
			<version>${jmc.version}</version>
		</dependency>
		<dependency>
			<groupid>org.openjdk.jmc</groupid>
			<artifactid>flightrecorder.rules</artifactid>
			<version>${jmc.version}</version>
		</dependency>
		<dependency>
			<groupid>org.openjdk.jmc</groupid>
			<artifactid>flightrecorder.rules.jdk</artifactid>
			<version>${jmc.version}</version>
		</dependency>
	</dependencies>

Finally! 🙂

JMC 8 Binary Releases!

Two binary releases of JMC 8 are now available:

Since I could not find the project provided release notes at any of the vendor sites, I’ve taken the liberty of reproducing them below.

Mission Control 8.0 – New and Noteworthy


General


JMC 8.0 – Major New Release!
This is a major new release of Java Mission Control. All binary build of the JMC application using the Eclipse 2020-09 platform will now require JDK 11+ to run, but can be used with OpenJDK 8u272+ and Oracle JDK 7u40+. It can also open and visualize flight recordings from JDK 7 and 8.

jmc[1]


Eclipse 4.16 support
The Mission Control client is now built to run optimally on Eclipse 2020-06 and later. To install Java Mission Control into Eclipse, go to the update site (Help | Install New Software…). The URL to the update site will be vendor specific, and some vendors will instead provide an archive with the update site.

eclipse[1]


JOverflow in the Base Distribution
JOverflow is now part of the base distribution of Mission Control, and has been converted to SWT. It also has a brand new TreeMap visualization of the heap occupied by type. To open the new view, go to Window | Show view | Other… and start typing JOverflow, and select JOverflow TreeMap.

joverflow


Minor bugfixes and improvements
There are over two hundred fixes and improvements in this release. Check out the JMC 8 Result Dashboard (https://bugs.openjdk.java.net/secure/Dashboard.jspa?selectPageId=19536″) for more information.

issues


Java Flight Recorder (JFR)


Flame Graph View Improvements
The Flame Graph View is now available in the standard perspective. It can now be configured to be rendered either as an icicle graph or a flame graph, and the tool tips have been improved. Click on anything representing a selection of events to get the aggregate of stack traces represented by that selection rendered in the Flame Graph View.

flameview


New Graph View
An early access version of a Graph View is now available. The Graph View will show a directed graph where the individual methods are nodes in the graphs, and the edges will represent calls from one method to another. The size of the node will represent the “self” contribution, and the thickness of an edge will correlate with the number of calls from one method to another in the events. To open up the graph view, go to Window | Show view | Other…, then select Mission Control / Graph View. Once the Graph View has opened, it can be docked where you want it to be, or dragged out into a new top level window which can be handled by your window manager. Currently there is no graph pruning, and large graph will take a very long time to layout and render, during which the UI will freeze. Therefore we’ve imposed a user configurable limit on the number of nodes a graph can have for now.

graphview


Predecessors and Successors
The predecessors and successors tabs have been brought back to the Method Profiling page. This means that it is now easy to select a method and easily see wherever that method was called from, and to see what was then called from that method. This is a temporary solution until we have a separate Butterfly/Sandwich view where any selection of a method in a set of events can be rendered in a separately configured view.

predsuc


JMC Agent


Agent 1.0.0
This is the first version where we build and release the JMC agent. The JMC agent is a byte code instrumentation agent, providing an easy way to declaratively generate JFR events from most methods, even methods for which the source code is not available. Either configure the agent with an XML configuration file, or use the MBean to dynamically change the instrumentation whilst the application is running.

agent


Converter Functions
Now any public static method can be used to convert objects of a specific type to one of the types supported by flight recorder. The converter to use must be explicitly declared per recorded data.

converter


Bug Fixes


Area: JFR
Issue: 5734
Synopsis: Overflow in stacktrace tooltips fixed

On Mac, the tooltips shown for the stacktraces could overflow. This is now fixed.


Known Issues


Area: General
Issue: 4270
Synopsis: Hibernation and time

After the bugfix of https://bugs.openjdk.java.net/browse/JDK-6523160 in JDK 8, the RuntimeMXBean#getUptime() attribute was re-implemented to mean “Elapsed time of JVM process”, whilst it previously was implemented as time since start of the JVM process. The uptime attribute is used by JMC, together with RuntimeMXBean#getStartTime(), to estimate the actual server time. This means that time stamps, as well as remaining time for a flight recording, can be wrong for processes on machines that have been hibernated.

Area: JFR
Issue: 7122
Synopsis: Rules evaluation never complete

Sometimes the rules evaluation may never complete for the GC rules.

Area: JFR
Issue: 7071
Synopsis: JMC can’t attach to jlinked JVMs

This one is still under investigation, but it seems JMC can’t attach to certain jlinked images.

Area: JFR
Issue: 7068
Synopsis: JfrRecordingTest (uitest) hangs on the automated analysis page

Trying to run uitests on Fedora hangs on JfrRecordingTest.

Area: JFR
Issue: 7007
Synopsis: Unable to edit run configurations for eclipse project after installing JMC plugin

Avoid installing the experimental JMC launcher plug-in into Eclipse until this problem has been resolved.

Area: JFR
Issue: 7003
Synopsis: The graph view does not work on Windows

This is due to a bug in the chromium browser plug-in used by JMC on Windows. See the issue for more information.

Area: JFR
Issue: 6265
Synopsis: JMC crashes with Webkit2+GTK 4

See the issue for more information.

Area: JFR
Issue: 6265
Synopsis: JMC crashes with Webkit2+GTK 4

See the issue for more information.

Area: JFR
Issue: 5412
Synopsis: Dragging and dropping a JFR file into an open analysis page does not work

The expected behaviour would be to open the recording whenever a file is dropped in the editor area, but the behaviour will be defined by the embedded browser component, and not very useful.

Fetching and Building Mission Control 8+

As described in a previous post, Mission Control is now on GitHub. Since this alters how to fetch and build OpenJDK Mission Control, this is an updated version of my old post on how to fetch and build JMC from version 8 and up.

Getting Git

First step is to get Git, the SCM used for OpenJDK Mission Control. Installing Git is different for different platforms, but here is a link to how to get started:

https://git-scm.com/book/en/v2/Getting-Started-Installing-Git

Installing the Skara Tooling (Optional)

This is an optional step, making it easier if you want to contribute to Mission Control:

https://hirt.se/blog/?p=1186

Cloning the Source

Once Git is installed properly, getting the source is as easy as cloning the jmc repo. First change into the directory where you want to check out jmc. Then run:

git clone https://github.com/openjdk/jmc.git

Getting Maven

Since you probably have some Java experience, you probably already have Maven installed on your system. If you do not, you now need to install it. Simply follow the instructions here:

https://maven.apache.org/install.html

Building Mission Control

First we need to ensure that Java 8 is on our path. Some of the build components still use JDK 8, so this is important.

java –version

This will show the Java version in use. If this is not a Java 8 JDK, change your path. Once done, we are now ready to build Mission Control. Open up two terminals. Yep, two!

In the first one, go to where your cloned JMC resides and type in and execute the following commands (for Windows, replace the dash (/) with a backslash (\)):

cd releng/third-party
mvn p2:site
mvn jetty:run

Now, leave that terminal open with the jetty running. Do not touch.

In the second terminal, go to your cloned jmc directory. First we will need to build and install the core libraries:

cd core
mvn install

Next run maven in the jmc root:

mvn clean package

JMC should now be building. The first time you build Maven will download all of the third party dependencies. This will take some time. Subsequent builds will be less painful. On my system, the first build took 6:01 min. The subsequent clean package build took 2:38.

Running Mission Control

To start your recently built Mission Control, run:

Windows

target\products\org.openjdk.jmc\win32\win32\x86_64\jmc.exe -vm %JAVA_HOME%\bin

Mac OS X

target/products/org.openjdk.jmc/macosx/cocoa/x86_64/JDK\ Mission\ Control.app/Contents/MacOS/jmc -vm $JAVA_HOME/bin

Contributing to JDK Mission Control

To contribute to JDK Mission Control, you need to have signed an Oracle Contributor Agreement. More information can be found here:

http://openjdk.java.net/contribute/

Don’t forget to join the dev list:

http://mail.openjdk.java.net/mailman/listinfo/jmc-dev

We also have a Slack (for contributors), which you can join here:

https://join.slack.com/t/jdkmissioncontrol/signup

More Info

For more information on how to run tests, use APIs etc, there is a README.md file in the root of the repo. Let me know in the comments section if there is something you think I should add to this blog post and/or the README!

Using the Skara Tooling

I’m writing this for myself as much as I’m writing this to share. After only a day of using JMC with Skara, I’ve fallen in love with it. I spend less time painstakingly putting together review e-mails, copying and pasting code to comment on certain lines of code, cloning separate repos to do parallel work efficiently, setting up new workspaces for the these repos etc. Props to the Skara team for saving me time by cutting out a big chunk of the stuff not related to coding and a whole lot of ceremony.

Note that the Skara tooling can be used outside of the scope of OpenJDK – git sync alone is a good reason for why everyone who wants to reduce ceremony can benefit from the Skara tooling.

So, here are a few tips on how to get started:

  1. Clone Skara:
    git clone https://github.com/openjdk/skara
  2. Build it:
    gradlew (win) or sh gradlew (mac/linux)
  3. Install it:
    git config --global include.path "%CD%/skara.gitconfig" (win) or git config --global include.path "$PWD/skara.gitconfig" (mac/linux)
  4. Set where to sync your forks from:
    git config --global sync.from upstream

For folks on Red Hat distros, 2 and 3 can be replaced by make install. For more information on the installation, see the Skara wiki.

Some Examples

To sync your fork with upstream and pull the changes:
git sync --pull

Note: if the sync fails with the error message “No remote provided to fetch from, please set the –from flag”, remember to set the remote for your repo, e.g.
git remote add upstream https://github.com/openjdk/jmc

To list the open PRs:
git pr list

To create a PR:
git pr create

To push your committed changes in your branch to your fork, creating the remote branch:
git publish

JMC Workflow

Below is the typical work-flow for JMC.

First ensure that you have a fork of JMC. Either fork it on github.com, or on the command line:
git fork https://github.com/openjdk/jmc jmc

You typically just create that one fork and stick with it.

  1. (Optional) Sync up your fork with upstream:
    git sync --pull
  2. Create a branch to work on, with a name you pick, typically related to the work you plan on doing:
    git checkout –b <branchname>
  3. Make your changes / fix your bug / add amazing stuff
  4. (Optional) Run jcheck locally:
    git jcheck local
  5. Push your changes to the new branch on your fork:
    git publish (which is pretty much git push --set-upstream origin <branchname>)
  6. Create the PR, either on GitHub, or from the command line:
    git pr create

Summary / TL;DR

  • I ❤️ Skara

Mission Control is Now Officially on GitHub!

Since this morning, the JDK Mission Control (JMC) project has gone full Skara! mc_512x512This means that the next version (JMC 8.0) will be developed over at GitHub.

To contribute to JDK Mission Control, you (or the company you work for) need to have signed an OCA, like for any other OpenJDK-project. If you already have an OpenJDK username, you can associate your GitHub account with it.

Just after we open sourced JMC, I created a temporary mirror on GitHub to experiment with working with JMC at GitHub. That mirror is now closed for business. Please use the official OpenJDK one from now on:

https://github.com/openjdk/jmc

If you forked or stared the old repo, please feel free to fork and/or star the new one!

JFR is Coming to OpenJDK 8!

I recently realized that this isn’t common knowledge, so I thought I’d take the opportunity to talk about the JDK Flight Recorder coming to OpenJDK 8! The backport is a collaboration between Red Hat, Alibaba, Azul and Datadog. These are exciting times for production time profiling nerds like me. Smile

The repository for the backport is available here:

http://hg.openjdk.java.net/jdk8u/jdk8u-jfr-incubator/

The proposed CSR is available here:

https://bugs.openjdk.java.net/browse/JDK-8230764

The backport is keeping the same interfaces and pretty much the same implementation as is available in OpenJDK 11, and is fully compatible. There were a few security fixes, due to there not being any module system to rely upon for isolation of the internals, also, some events will not be available (e.g. the Module related events) but other than that the API and tools work exactly the same.

JDK Mission Control will, of course, be updated to work flawlessly with the OpenJDK 8 version of JFR as well. The changes will be minute and are only necessary since Mission Control has some built-in assumptions that no longer hold true.

You can already build and try out OpenJDK 8 with JFR simply by building the JDK available in the repository mentioned above. Also, Aleksey Shipilev provide binaries – see here for details.

Have fun! Smile