AI Assistant for OpenJDK Contributors

Subtitle: ”Soon You’ll Know What I Did This Summer”

This summer my family and I decided to stay in Küssnacht during my summer vacation. The summers here are lovely, and for a family of six going for a vacation abroad it typically means a lot of time is spent preparing, packing and unpacking, for both directions of the journey. A staycation seemed like a good choice. Most days were spent in the water with the family, and on one occasion, we were joined by a team member and his family. It can get rather hot here, and the lake is the perfect place to cool down.

Me on a SUP overlooking part of my family on another SUP on Lake Lucerne

That said, I did end up in front of my computer a bit in the evenings after everyone else had fallen asleep. There were a few things I really wanted to get done this vacation. One was upgrading my server. Another one was to put some left over Raspberry Pi 4s and a few new Raspberry Pi 5s to good use in my 19” rack, creating my very own a little Kubernetes cluster.

Recently I was also playing with Custom GPTs, which is OpenAI’s way of letting you build little RAG (Retrieval Augmented Generation) systems without writing code. I first played a bit with Custom GPTs on a train ride back from the Datadog office in Paris a little while ago, and it was a quite lot of fun, but for some of the things I wanted it to do, it performed rather badly, and I couldn’t find any fitting APIs available to help my Custom GPT do better.

Since I was setting up a Kubernetes cluster anyway, I thought I might as well hack something together to make my OpenJDK Project Assistant a little bit better.

Setting Up the Cluster

Ah, the pain and the anguish. I ended up using my Apache server as a proxy (since I still want https://hirt.se to serve my old homepage and blog) for an Nginx that is acting as reverse proxy and load balancer to my Kubernetes cluster.

If anyone wants to know how I set up the cluster, that is an entirely different blog that probably already exists in a thousand versions, written by people with more patience to write about configuration than me, so I will not dwell on that. Suffice to say that I consulted many blogs and Claude 3.5 Sonnet to get it all up and running. Here are the steps in short:

  • Format and setup the Raspberries with Raspberry Pi OS (64-bit) (I used the Raspberry Pi Imager).
  • I added my Pis to the hosts file of my server and to my dhcpd config by MAC address, so that I can refer to the Pis by name instead of IP (I’m using dnsmasq).
  • I installed kubernetes on them, which involved properly enabling cgroups in /boot/firmware/cmdline.txt, setting up containerd properly, getting the debian package for k8s, installing kubelet, kubeadm, kubectl, turning off swap, and finally using kubeadm init to create the cluster and then applying flannel. Then using the join command you get from performing the init to have the other Pis join the cluster.

There was a bunch of trial and error, and there is probably a much easier way. Anyway, I now have a cluster. I can easily deploy. I can easily scale. What more could you ever wish for?

The API exposed to the CustomGPT is available to the world at https://api.hirt.se and described here, should anyone be interested. The implementation is available on GitHub – feel free to contribute if the custom GPT isn’t doing what you think it should be doing or if you simply want to make it more capable.

What Can the OpenJDK Project Assistant Do?

It can do lots of things. It can get information about open PRs and repos without overwhelming GPT with information that will invariably make it fail (this is what typically happened before the api.hirt.se API). For example, here’s getting the oldest open PR for the JDK project:

image (3)

It can summarize the information in the PRs:

image (4)

 

You can also ask for information about the related bug:

image

It can answer questions around people involved in OpenJDK:

image

For people like me, who have a really hard time remembering names, it can help answer questions about people involved in OpenJDK:

image

And about projects and groups:
image

Please let me know if you find it useful!

Summary

  • I recently had a stay-at-home vacation in Küssnacht.
  • Aside from spending a lot of the time in Lake Lucerne with my family, I did have some fun building a Kubernetes cluster out of my Raspberry Pis.
  • I put the cluster to good use to support my OpenJDK Project Assistant Custom GPT.
  • If you’re one of the few OpenJDK committers that also have access to Custom GPTs, or you simply like the idea, feel free to put a star on my API repo, or even better – contribute things you want it to be able to do, or do better!
  • Silly bonus: I also noticed that the silly old slogan image generator I was using on my homepage isn’t available under https (after my various upgrades, I now have permanent redirects to always use https). I couldn’t find any replacement that worked as a drop-in replacement, so I made my own variant that is now also running in my little cluster. Smile

Picture looking out over lake Lucerne after vacation dinner with my wife.

JDK Mission Control 9.0.0 Released!

The 9.0.0 GA release of JDK Mission Control was just tagged in the JMC repo at GitHub! Since this is the source release, it may still take a bit of time until the downstream vendors release binary builds of JDK Mission Control 9.0.0. I will try to remember to tweet or say something on the JMC Facebook page once the binaries start showing up.


Mission Control 9.0 – New and Noteworthy


General


JMC 9 – New Release!
This is the latest (2024) major release of JDK Mission Control. JMC 9 requires JDK 17+ to run and introduces several new features, enhancements, and bug fixes. This version continues to support connecting to, and parsing JFR recordings from, OpenJDK 8u272+ and Oracle JDK 7u40+, and can open and visualize flight recordings from JDK 7 and 8. JDK Mission Control is available for Windows (x86_64), Mac OS X (ARM and x86_64), and Linux (ARM and x86_64).

jmc


Eclipse 4.30 support
The Mission Control client is now built to run optimally on Eclipse 2023-12 and later. To install JDK Mission Control into Eclipse, go to the update site (Help | Install New Software…). The URL to the update site will be vendor specific, and some vendors will instead provide an archive with the update site.

eclipse


Support for Linux/aarch64
JMC 9 is now built for Linux aarch64.

linuxaarch64


Support for dark mode
JMC 9 now supports dark mode. Go to Preferences, General | Appearance, and select the Dark theme to enable.

darkmode


Minor bugfixes and improvements
There are 118 fixes and improvements in this release. Check out the JMC 9.0 Result Dashboard for more information.

issues


Add user configuration for local JVM refresh interval
Previously the JVM Browser checked every 5000 ms for new JVMs. This can now be configured.

refreshinterval


Core


Better JFR parser performance
Multiple efforts have been made to reduce allocations in the JMC parser, including: reduced allocation of Doubles, reduced allocation rate in ParserStats. Also, when duration events aren’t ordered by their end time (e.g. events which stack so that the last event finishes first, or file reads with overlaps) `DisjointBuilder.add` can be slow because of the linear search for the lane, and then a linear time reordering. This has been improved with a binary search.

jfrperformance


Support checkpoint event sizes beyond u4 limit
The JMC JFR parser now support checkpoint event sizes beyond the u4 limit.

checkpointsize


Move non-Eclipse dependent classes from org.openjdk.jmc.ui.common to org.openjdk.jmc.common
There were a number of classes previously in jmc.ui.common that would be a great asset to the core distribution (and the third-party applications that consume jmc-core), and these classes now live in jmc.common. Please see JMC-7308 for further information.

reorganize


Move rjmx bundle from application to core
The rjmx classes and related services (FlightRecorderService) are now exposed for third-party application usage. Please see JMC-7069 for further information.

reorganize


Move org.openjdk.jmc.flightrecorder.configuration bundle from application to core
The org.openjdk.jmc.flightrecorder.configuration bundle contains many classes useful for working with jfr, and are now available in core. Please see JMC-7307 for further information.

reorganize


Java Flight Recorder (JFR)


The Event Browser now supports searching and showing event type ids
Searching in the search bar now also searches event type IDs, and there is also a (by default hidden) column that makes it easy to show the event type IDs for the shown events.

eventtypeid


Add support for enabling jfr on native images
Previously JMC was unable to start flightrecorder on a graalvm native image, even if there is built-in jfr support. This has now been fixed.

native-image-alt


Java based flamegraph visualization
The previous flamegraph visualization takes place in an embedded browser component (provided by the Eclipse platform), unfortunately this approach has some drawbacks, the first being a bit slow. This view is now using a Java (Swing) based flamegraph library. Also, the flame graph model creation performance have been improved.

flamegraph


Visualization and Rule for FileChannel.force()
The File I/O page has been updated to show force related information. There are two new columns added – Force Count and Update Metadata. Both are hidden by default and can be enabled by right clicking the table. The chart will also include a File Force row. There is a preference setting for the associated file force rule, where the peak duration warning limit can be set. See JMC PR#533 for more information.

fileforce


Rule that checks on G1 pause time target compliance
New rule that looks at the pause time target and compares it to the actual pauses.

rule


Rule that looks at finalization statistics
JDK 18 comes with a FinalizationStatistics event that helps users find where in their application finalizers are run. This is important as finalization has been deprecated for removal in a future release. For more information about finalization and its flaws, see https://openjdk.java.net/jeps/421. Even if an application doesn’t implement any finalize() methods, it may rely on third-party libraries that does. Static analysis of third-party libraries using “jdeprscan –for-removal” can be used to list those classes, but it will not tell if they are being used. For example, an application may be missing a call to a close() method, so the resource is cleaned up by the finalizer, which is sub-optimal.

rule


Rule that detects GC Inverted Parallelism
Rule inspired by the “Inverted Parallelism” analysis in Garbagecat. See JMC-8144 for more information.

rule


Support for the new JPLIS agent events
There is now a new page and rule for loaded JPLIS agents. See JMC-8054 for more information.

agent


Twitter plug-in removed
Due to changes in APIs and cost of maintenance, the Twitter plug-in has been removed.

twitterplugin


Bug Fixes


Area: Agent
Issue: 8045
Synopsis: retransformClasses() doesn’t re-transform all needed classes

The retransformClasses() methods in Agent and AgentController use Class.forName() to try to get the class objects of classes needed to re-transform. This obviously doesn’t work for classes loaded by classloaders different from the one which loads the agent. Those classes would be instrumented if they were loaded after their event probes were defined the AgentController. But when loaded earlier they would not be instrumented. This has been fixed.

Area: Agent
Issue: 8048
Synopsis: Agent throws exceptions on missing or empty descriptions

When the description of an event or value is empty or missing, the agent fails with exceptions. This has now been fixed.

Area: Console
Issue: 8154
Synopsis: Some JMX attributes are missing unit specifications in the Console

The missing unit specifications have now been added.

Area: Core
Issue: 8063
Synopsis: IMCFrame Type cache not synchronized

The type cache used in the IMCFrame Type inner class wasn’t synchronized and could cause a concurrent modification exception during e.g. JFR parsing. This has been fixed.

Area: Core
Issue: 8156
Synopsis: JfrRulesReport.printReport does not respect verbosity for text and json

The verbosity flag for text and json reports didn’t work. This has been fixed.

Area: Core
Issue: 8041
Synopsis: JfrRulesReport json reports produce incomplete results

While generating JFR Rules Reports in json format, the results were incomplete. The components “message” and “detailedMessage” were not populated. This has been fixed.

Area: JFR
Issue: 7885
Synopsis: Graphical rendering of dependency view fails due to heap memory drain

Also JMC-7496. The dependency view drains the heap memory and causes out-of-memory exceptions and performance delays. This has been improved.


Known Issues


Area: JFR
Issue: 7071
Synopsis: JMC can’t attach to jlinked JVMs

This one is still under investigation, but it seems JMC can’t attach to certain jlinked images.

Area: JFR
Issue: 7003
Synopsis: The graph view, heatmap view and dependency view does not work on Windows

This is due to a problem with the Windows based browser component in SWT. We’re hoping for a fix in the component for a future version of the Eclipse platform.

JDK Mission Control 8.2.0 Released!

The latest release of JDK Mission Control was just released! Since this is the source release, it may still take a bit of time until the downstream vendors release binary builds of JDK Mission Control 8.2.0. I will try to remember to tweet or say something on the JMC Facebook page once the binaries start showing up.

Here’s what’s new:

Mission Control 8.2 – New and Noteworthy


General


JMC 8.2 – New Release!
This is a new minor release of JDK Mission Control. The JMC application requires JDK 11+ to run, but can still be used to connect to, and parse JFR recordings from, OpenJDK 8u272+ and Oracle JDK 7u40+. It can also still open and visualize flight recordings from JDK 7 and 8.

jmc


Eclipse 4.22 support
The Mission Control client is now built to run optimally on Eclipse 2021-06 and later. To install JDK Mission Control into Eclipse, go to the update site (Help | Install New Software…). The URL to the update site will be vendor specific, and some vendors will instead provide an archive with the update site.

eclipse


Minor bugfixes and improvements
There are 83 fixes and improvements in this release. Check out the JMC 8.2 Result Dashboard (https://bugs.openjdk.java.net/secure/Dashboard.jspa?selectPageId=20804) for more information.

issues


Binary build for Apple ARM
JDK Mission Control is now built for Apple ARM, allowing JMC to be run natively (without Rosetta x86 emulation) on Apple M1.


Core


Parser support for async profiler
Parser support has been added for frame types generated by async profiler, such as Native, C++ and Kernel.


System.gc() rule
There is now a new rule for explicit invocations of System.gc().

dukegc


Java Flight Recorder (JFR)


Heat map view
A new heat map view has been added, which is handy for seeing when events are taking place. Use Window | Show View | Other…, and select the Heatmap View under Mission Control and click Open to open the view.

heatmap


Websocket for selections
There is a new websocket API available that pushes stack trace data from selections in the JFR UI as JSON on a user defined port. This allows for programmatic control of the visualization directly in the browser. Tools like observablehq.com can be used to invent new visualizations, or to alter the visualization. To get started, simply go to the Flight Recorder preferences in JMC, and select the Websocket port to use (0 to disable). A set of example visualizations are available here: https://observablehq.com/collection/@cimi/java-mission-control.

websocket


Bug Fixes


Area: JFR
Issue: 7403
Synopsis: JFR parser struct types hashcode problem

Some JFR parser struct types were using lazily initialized attributes which happen to be a part of hashCode/equals computations.

Area: JFR
Issue: 7532
Synopsis: Delays in rendering of JMX console graphs

Sometimes the updates of the JMX console graphs would be severely delayed on MacOS. This is now fixed.

Area: JFR
Issue: 7068
Synopsis: JfrRecordingTest (uitest) hangs on the automated analysis page

Trying to run uitests on Fedora hangs on JfrRecordingTest. This was fixed after the Eclipse platform update.


Known Issues


Area: General
Issue: 4270
Synopsis: Hibernation and time

After the bugfix of https://bugs.openjdk.java.net/browse/JDK-6523160 in JDK 8, the RuntimeMXBean#getUptime() attribute was re-implemented to mean “Elapsed time of JVM process”, whilst it previously was implemented as time since start of the JVM process. The uptime attribute is used by JMC, together with RuntimeMXBean#getStartTime(), to estimate the actual server time. This means that time stamps, as well as remaining time for a flight recording, can be wrong for processes on machines that have been hibernated.

Area: JFR
Issue: 7071
Synopsis: JMC can’t attach to jlinked JVMs

This one is still under investigation, but it seems JMC can’t attach to certain jlinked images.

Area: JFR
Issue: 7003
Synopsis: The graph and flame graph view does not work on Windows

This is due to a problem with the Windows based browser component in SWT. We’re hoping for a fix in the component for a future version of the Eclipse platform.

Contributing to OpenJDK Mission Control

Hacktoberfest Banner

Since this month is Hacktoberfest, I thought it would be a good idea to talk a bit about how to contribute to the OpenJDK Mission Control project. Some of the content of this blog post will be applicable to any of the OpenJDK projects, especially the Skara (OpenJDK on Git) bits.

The OpenJDK Mission Control Project

The OpenJDK Mission Control project is the observability tools suite for OpenJDK. It contains a JMX Console, a JFR visualizer and analyzer, a heap waste analysis tool, and many other little useful tools and utilities. Since it is all open source, pretty much anyone can contribute to the project.

The project is on GitHub:
https://github.com/openjdk/jmc

The first step to contribute to JDK Mission Control is to simply fork the repository on GitHub. This establishes a copy of the repository where you can freely make changes as you please. Whilst it is technically possible to make the changes in the master branch, it will save time and effort if you later want to contribute the effort to make the changes in a branch:
git checkout -b my-jmc-test

Building JMC

First of all, ensure that you have jdk11 active in your shell, and verify that this is the case using:
java -version

There are multiple ways to build JMC. The easiest way is to simply use the build script (don’t do this just yet):
./build.sh –packageJmc

There is also a way to build JMC using Docker (don’t do this just yet either):
docker-compose -f docker/docker-compose.yml run jmc

These are however not the best ways when you’re developing JMC using an IDE. The third party dependencies for JMC need to be available through a p2 repository, and you want to install a build of the JMC core libraries into your maven cache.

So, to set things properly up for development, it is better to first install the core libraries:

cd $JMC_ROOT/core
mvn install

Next, build the p2 site and start jetty to expose it on a well known port:

cd $JMC_ROOT/releng/third-party
mvn p2:site
mvn jetty:run

Then leave jetty running for as long as you are developing JMC. You will need it up and running so that it can be found both when building from the command line, as well as when compiling JMC from within the Eclipse development environment.

To build the JMC application, next do the following in a separate shell (since you have jetty with the p2 site for the third-party dependencies up and running in the previous one):

cd $JMC_ROOT
mvn package

After this, you can use the build script to run the built JMC product:
./build.sh –run

For alternative ways of launching JMC, see the platform specific documentation in the README.md.

Developing JMC

Many that I’ve talked to, especially when JMC was shipped with the Oracle JDK, believed that JMC is a native application. If you’ve browsed the repo, you’ve already seen that it is a Java application, more specifically an Eclipse RCP application. Since it is an Eclipse RCP application, it’s easiest to develop JMC using Eclipse.

First set up your development environment, following the Developer Guide. It is slightly involved, but luckily does not need to happen very often.

Next, in your branch in your fork, commit the changes you want to contribute, and create a pull request, just like you would for any other open source project on GitHub.

Now, if this is your first OpenJDK PR, the OpenJDK bot will likely complain about a few different things, for example:

  • You need to have your GitHub account associated with a company that has a signed Oracle Contributor Agreement (OCA), or you must have signed an OCA yourself.
  • The PR needs to have an associated issue in the Java Bug System.
  • There is some problem with the testing or formatting of your code.

Let’s take a quick look at these three problems.

The Oracle Contributor Agreement

Like all open source projects, there needs to be a Contributor Agreement in place. This is to protect everyone backing the project, as well as the customers depending on the project. For example, the contributor agreement ensures that the source code you’re contributing isn’t violating any patent rights, and that the source code you’re contributing is yours to contribute.

Many larger companies already have an OCA signed, so the first step might be to check with your company if one is already signed. In my case, I both have a personal OCA signed (since I was contributing before Datadog signed an OCA), and one signed by my employer, Datadog.

You will know that the OCA status is not properly set up for your GitHub account when the OCA label is set in the PR, and the following text can be found in the PR:
⚠️ OCA signatory status must be verified

The OpenJDK bot will write helpful messages in the PR to help guide you through getting your OCA status verified.

The Java Bug System

Once you have a few commits under your belt, and become an OpenJDK author, you have access to the Java Bug System (JBS): https://bugs.openjdk.java.net/. So, what do you do before then? If the PR passes a first cursory check by the reviewers, a reviewer will simply create an Issue in JBS for you.

Fixing Issues

If you end up having an issue, the details of the test run in the PR will hopefully be enough to sort it out. If not, you can run mvn verify locally and look at the test logs. If it is formatting, then check if the formatting problem was in core or not, and either run mvn spotless:apply in core or in the root of the project.

Skara – the OpenJDK Git Tooling

Skara is the project name for the tooling around developing OpenJDK on Git(Hub). It actually insulates a lot of the GitHub specifics, making it possible, should the need ever arise, to move the development and development process somewhere else. The project also contains the aforementioned bot that helps, for example, to verify that there is a related JBS issue, and that there is a signed OCA. Skara also contains some useful git extensions which make working with OpenJDK on GitHub smoother.

To set things up, do the following:

Clone Skara:
git clone https://github.com/openjdk/skara

Build it:
gradlew (win) or sh gradlew (mac/linux)

Install it:
git config –global include.path “%CD%/skara.gitconfig” (win), or
git config –global include.path “$PWD/skara.gitconfig” (mac/linux)

Set where to sync your forks from:
git config –global sync.from upstream

Here are some examples:

To sync your fork with upstream and pull the changes:
git sync –pull

Note: if the sync fails with the error message “No remote provided to fetch from, please set the –from flag” or “error: upstream is not a known git remote, nor a proper git URI”, remember to set the remote for your repo, e.g.

git remote add upstream https://github.com/openjdk/jmc

To list the open PRs:
git pr list

To create a PR:
git pr create

To push your committed changes in your branch to your fork, creating the remote branch:
git publish

So, the normal workflow when working with OpenJDK JMC using the Skara tooling becomes:

Note: First ensure that you have a fork of JMC, and that your current directory is the root of that fork. You typically just create that one fork and stick with it.

  1. (Optional) Sync up your fork with upstream:
    git sync –pull
  2. Create a branch to work on, with a name you pick, typically related to the work you plan on doing:
    git checkout –b <branchname>
  3. Make your changes / fix your bug / add amazing stuff
  4. (Optional) Run jcheck locally:
    git jcheck local
  5. Push your changes to the new branch on your fork:
    git publish (which is pretty much git push –set-upstream origin <branchname>)
  6. Create the PR, either on GitHub, or from the command line:
    git pr create

Once the PR is created, the bot will check that everything is okay, and the PR will be reviewed.

Interacting with the Skara Bot

Getting the PR merged is handled a bit differently in OpenJDK compared to normal GitHub projects. First of all, all the prerequisites must first be fulfilled, like the OCA status of the contributor being verified, the change being properly reviewed, jcheck passing, the tests passing, the PR having a matching issue in JBS etc. Once that is all taken care of, the bot will helpfully ask, in a message in the PR, for the author of the PR to integrate the changes. This is simply done by typing /integrate in message in the PR. The bot will automatically rebase on the latest changes in the target branch (normally master) and squash your commits. In other words, it is perfectly fine to have multiple fixes and other commits happening in the PR after the initial commit for the PR. It is actually much preferred to force-updating the PR, as it’s easier to follow along with the review.

If the PR author is not a committer on the project, the bot will inform that the PR is ready to be sponsored by a committer, which is normally done by the reviewer of the PR. This is done by writing /sponsor in a separate message in the PR.

When the PR is merged, the corresponding JBS issue is automatically closed.

Other Related Repos

There are a few additional repos that are related to the OpenJDK JMC project, but that aren’t currently OpenJDK projects. Two examples are the jmc-jshell and the jmc-tutorial repositories. The jmc-tutorial is a good resource for learning about JDK Mission Control. Even though it is not officially an OpenJDK repository, it can still be a good place to start contributing to the OpenJDK JMC community.

Summary

  • Contributing to OpenJDK is easier than ever before now that it’s on GitHub.
  • Skara makes it even easier.
  • It’s Hacktoberfest – commits to the JMC project (and related repos) count!
  • JBS is a good source for JMC starter bugs.
  • If you need any help, the JDK Mission Control slack is a good place for asking questions! Ping me or any of the JMC folks for an invite. 🙂
  • Finally, here’s a practical guide to OpenJDK projects and the roles:
    OpenJDK Projects (java.net)