Sometime it can be very useful to have programmatic access to the Diagnostic Commands available in the Oracle JRE. This is of course both highly dependent on the Oracle JDK and very unsupported, so, as usual, please use responsibly.

Here comes the usual disclaimer:

The following blog entry will describe UNSUPPORTED functionality. This means that relying on the described APIs or functionality may BREAK your code/plugin with any given update of the JDK and/or Mission Control.

The Attach API

There is an API that allows a Java process to attach to another Java process. This is the API used by several different Java tools to both enumerate the locally running Java processes, as well as do things like start up the management agent. The API is usually just called the Attach API, and one of the entry points is the com.sun.tools.attach.VirtualMachine class.

You can use the VirtualMachine class to enumerate all the locally running Java processes (with the same effective user as the process doing the listing) on the machine like this:

List<VirtualMachineDescriptor> vmList = VirtualMachine.list();
for (VirtualMachineDescriptor descriptor : vmList) {
    System.out.println(String.format(“%s %s”, descriptor.id(), descriptor.displayName()));
}

This is how you attach to a running Java process:

VirtualMachine.attach(<PID>); 

<PID> is the process ID as a String. The reason it is a String is probably to not assume that all Operating Systems use integer values for their process identifiers. I guess having the identifier as a String could also come in handy when addressing Isolates/MVM and other more complicated scenarios where what constitutes the “Java Process” becomes a bit more blurry. Anyways, once you have your VirtualMachine there is all kinds of fun you can do, like loading Java agents and agent libraries etc. The most important thing, however, is to detach. Never forget to detach, or you will start leaking process handles.

Invoking Diagnostic Commands

To invoke diagnostic commands, we will need to cast our VirtualMachine to a HotSpotVirtualMachine. Here is how to execute the Thread.print DiagnosticCommand:

VirtualMachine vm = VirtualMachine.attach(<PID>);
HotSpotVirtualMachine hsvm = (HotSpotVirtualMachine) vm;
hsvm.executeJCmd(“Thread.print”);

Or, here as a compilable example:

import java.io.BufferedInputStream;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.InputStream;

import sun.tools.attach.HotSpotVirtualMachine;

import com.sun.tools.attach.AttachNotSupportedException;
import com.sun.tools.attach.VirtualMachine;

public class DiagnosticCommandExample {

    public static void main(String[] args) throws AttachNotSupportedException,
            IOException {
        if (args.length == 0) {
            System.out.println(“Usage: InvokeDiagnosticCommand <PID>”);
            System.exit(2);
        }
        VirtualMachine vm = VirtualMachine.attach(args[0]);
        if (!(vm instanceof HotSpotVirtualMachine)) {
            System.out.println(“Only works on HotSpot!”);
            System.exit(3);
        }
        HotSpotVirtualMachine hsvm = (HotSpotVirtualMachine) vm;
        System.out.println(readInputStreamAsString(hsvm.executeJCmd(“Thread.print”)));
    }

    public static String readInputStreamAsString(InputStream in)
            throws IOException {
        BufferedInputStream bis = new BufferedInputStream(in);
        ByteArrayOutputStream buf = new ByteArrayOutputStream();
        int result = bis.read();
        while (result != -1) {
            byte b = (byte) result;
            buf.write(b);
            result = bis.read();
        }
        return buf.toString();
    }
}

Summary

Using the attach API is not supported and the APIs may change at any time. That said, it can still be quite useful for putting together small utilities.

Clarification: Alan Bateman pointed out to me that whilst linking to the sun.tools.attach.HotSpotVirtualMachine class is indeed unsupported, the Attach API in itself is supported and documented.

Last week I ran into Gerrit Grünwald (@hansolo_) whilst demoing JMC in the Oracle booth at JFokus. I’d built a little RPi sensor board to have in my summer house, and since there is a lot of interest in the Internet of Things, I thought I’d contribute by showing how easy it is to hook up the JMX Console to one of those things.

We also had a very neat demo sporting a plug-in for the JMC console for controlling our resident LEGO robot (caught in an intimate moment with another robot in the picture below).

Anyways, Gerrit wanted to know how to extend the JMC JMX Console with custom JavaFX visualization. Since the Java FX part will be much easier once JDK 8 and JMC 5.3.0 is out, I’ll leave that for a (not too distant) future blog post. Here is the first step though – how to build your own JMC console plug-in!

Preparations

The easiest way to get going is by installing the JMC PDE plug-in into Eclipse.

1. Get an Eclipse classic
   For JMC 5.2, get Eclipse 3.8.2 up to 4.2.x (Juno)
   For JMC 5.3 (out soon), get Eclipse 3.8.2 and any 4.x (Kepler will work fine)
2. Install JMC from the update site
   The update site link can be found here: http://www.oracle.com/missioncontrol
   Follow the installation instructions.
   Note: Later releases of Eclipse are pickier about certificate validation – if you run into problems because download.oracle.com maps to akamai hosts, try using http instead of https in the update site URL.
3. Next install the PDE plug-in from the experimental update site
   The URL for the experimental update site can be found on the Mission Control page too: http://www.oracle.com/missioncontrol
   The plug-in to install is the Development/Java Mission Control PDE Integration one.

Next you need to set the target platform to your JMC installation.

1. Go to Preferences
2. Type targ in the filter box and select the “Target Platform” node in the preferences tree
3. Click Add, to open the wizard for adding a new target platform
4. Click next, since you want to start with an empty target definition
5. Name the target JMC and click Add
6. Select Installation and hit Next
7. Select your JAVA_HOME/lib/missioncontrol folder
8. Hit Finish

You are now set to start developing your first Mission Control plug-in.

JMC JMX Plug-in Hello World

First we need to create a new plug-in project. Select File | New | Other. Select Plug-in Project and hit next. Name your project and hit next. Name your plug-in and hit next. Now you should be able to select a custom wizard for creating a JMC console plug-in:

Select Simple Mission Control Console Tab Wizard and hit Next. Change the class name and tab name to your liking and finish the wizard. You should now have a new project. With the current 5.2.0 PDE plug-in, you may have a weird compilation error due to an unnecessary import as well. If you do, simply right click on the offending class and choose organize imports to get rid of the import (remember to save the class after fixing the problem):

You can now run Mission Control with your new plug-in by selecting the project and choosing Run As | Eclipse Application from the context menu. If you connect with the management console, you should see something like this:

Building the Plug-in

This is how you build your plug-in:

1. Select the plug-in project and choose Export… from the context menu
2. In the following wizard, select Deployable plug-ins and fragments
3. Select a Directory for where to put the plug-in and hit Finish.

That’s it. You should now have a plug-in jar file.

Installing the Plug-in

How easy it is to install your plug-in depends on the version of JMC you are using.

For 5.3.0 you simply copy the plug-in into the JMC_HOME (i.e. JDK_HOME/lib/missioncontrol) dropins folder. Done!

For 5.2.0 this is quite a lot more elaborate. You first need to generate some p2 metadata. This can be done using Eclipse like this:

eclipse -application org.eclipse.equinox.p2.publisher.FeaturesAndBundlesPublisher -metadataRepository <metadatadir> -artifactRepository <metadatadir> -bundles <pluginsfolder> -compress –publishArtifacts

Where <metadatadir> is where you want the metadata published, and <pluginsfolder> is the plug-ins folder where you exported your plug-in.

For example:

macbookpro:~marcus$ /Users/marcus/Applications/eclipse_4.2.2/Eclipse.app/Contents/MacOS/eclipse -consoleLog -application org.eclipse.equinox.p2.publisher.FeaturesAndBundlesPublisher -metadataRepository file:/Users/marcus/Builds/repo -artifactRepository file:/Users/marcus/Builds/repo -bundles /Users/marcus/Builds/plugins -compress -publishArtifacts

The result in <metadatadir> should be two jars and a plugins folder with your plug-in in it.

Next you need to tell p2 to install the plug-in. JMC includes the p2 director, so installing can be done like this (using sudo for administrative rights – on windows you might want to open a shell with admin rights):

sudo jmc -nosplash -application org.eclipse.equinox.p2.director –repository <metadatadir> –installIU <pluginname>/<full version> -destination <jmchome> -profile JMC

For example:

macbookpro:~marcus$ sudo /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/missioncontrol/Java\ Mission\ Control.app/Contents/MacOS/jmc -nosplash -application org.eclipse.equinox.p2.director -repository file:/Users/marcus/Builds/repo -installIU se.hirt.blogs.jmxconsole/1.0.0.201402132124 -destination /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/missioncontrol/ -profile JMC

Uninstalling

If you wish to uninstall the plug-in, you just remove it from the dropins folder in 5.3.0. For 5.2.0 you use the uninstallIU command:

sudo jmc -nosplash -application org.eclipse.equinox.p2.director –uninstallIU <pluginname>/<full version> -destination <jmchome> -profile JMC

For example:

macbookpro:~marcus$ sudo /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/missioncontrol/Java\ Mission\ Control.app/Contents/MacOS/jmc -nosplash -application org.eclipse.equinox.p2.director -uninstallIU se.hirt.blogs.jmxconsole/1.0.0.201402132124 -destination /Library/Java/JavaVirtualMachines/jdk1.7.0_45.jdk/Contents/Home/lib/missioncontrol/ -profile JMC

Summary

This blog entry described how to create a custom plug-in for the Java Mission Control JMX Console, and how to install the newly created plug-in. The installation process of custom plug-ins is simpler in the upcoming 5.3.0 version of Mission Control.

So, in JRockit there was this neat little dynamic MBean from which you could access all the JVM performance counters as attributes. Tonight I ended up in an e-mail thread leading me to think about how to retrieve the HotSpot ones. This is of course all very unsupported, and counter names/content and even the very API is subject to change at any given release. I am providing this information mostly as a reminder to myself. Who knows, someone might find this useful.

I will now go ahead and show how something similar to the PerformanceCounters MBean available in JRockit can be built. I recommend against doing this in any kind of production scenario, as the PerformanceCounter API is totally unsupported. Please don’t ask me what a certain counter means – you probably shouldn’t be using these in the first place. This is all for lolz. 🙂

Here comes the usual disclaimer:

The following blog entry will describe UNSUPPORTED functionality. This means that relying on the described APIs or functionality may BREAK your code/plugin with any given update of the JDK and/or Mission Control.

Performance Counter Hello World

If you ever used JRockit, you may fondly remember the old jrcmd utility, which now exists in a HotSpot incarnation as jcmd. Well, jrcmd had an option, –l, which would list all the the performance counters in the target JVM. The following code will let you implement something similar to the jrcmd –l command. Note that the PerformanceCounter API resides in a package most IDEs will restrict access to. You will need to change your project settings accordingly for the following to compile.

Anyways, here is a hack to list all the performance counters for a particular PID:

import java.io.IOException;
import java.nio.ByteBuffer;

import sun.management.counter.Counter;
import sun.management.counter.perf.PerfInstrumentation;
import sun.misc.Perf;

public class PerfCounterTest {

    public static void main(String[] args) throws IOException {
        if (args.length != 1) {
            System.out.println(“Usage: PerfCounterTest <PID>”);
            System.exit(2);
        }
        Perf p = Perf.getPerf();
        ByteBuffer buffer = p.attach(Integer.parseInt(args[0]), “r”);
        PerfInstrumentation perfInstrumentation = new PerfInstrumentation(buffer);
        for (Counter counter : perfInstrumentation.getAllCounters()) {
            System.out.println(String.format(
                    “%s = %s [Variability: %s, Units: %s]”, counter.getName(),
                    String.valueOf(counter.getValue()),
                    counter.getVariability(), counter.getUnits()));
        }

    }
}

Note that you will need the “sun.misc.Perf.getPerf” permission to be able to access the Perf instance when running with a security manager.

From here to a fully functional Dynamic MBean, the step is pretty short.

PerformanceCounterMBean á la JRockit for HotSpot

Here is an example of how code exposing the Performance Counters as a Dynamic MBean can look like:

import java.io.IOException;
import java.nio.ByteBuffer;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

import javax.management.Attribute;
import javax.management.AttributeList;
import javax.management.AttributeNotFoundException;
import javax.management.DynamicMBean;
import javax.management.InvalidAttributeValueException;
import javax.management.MBeanAttributeInfo;
import javax.management.MBeanException;
import javax.management.MBeanInfo;
import javax.management.MalformedObjectNameException;
import javax.management.ObjectName;
import javax.management.ReflectionException;
import javax.management.RuntimeOperationsException;

import sun.misc.Perf;
import sun.management.counter.Counter;
import sun.management.counter.perf.PerfInstrumentation;

public class PerfCounters implements DynamicMBean {
    public final static ObjectName MBEAN_OBJECT_NAME;
    public final Map<String, Counter> counterMap;
    private final MBeanInfo info;

    static {
        MBEAN_OBJECT_NAME = createObjectName(“se.hirt.management:type=PerfCounters”);
    }

    public PerfCounters() {
        counterMap = setUpCounters();
        info = createMBeanInfo();
    }

    @Override
    public Object getAttribute(String attribute)
            throws AttributeNotFoundException, MBeanException,
            ReflectionException {
        if (attribute == null) {
            throw new RuntimeOperationsException(new IllegalArgumentException(
                    “The attribute name cannot be null.”),
                    “Cannot invoke getAttribute on ” + MBEAN_OBJECT_NAME
                            + ” with null as attribute name.”);
        }
        Counter c = counterMap.get(attribute);
        if (c == null) {
            throw new AttributeNotFoundException(
                    “Could not find the attribute ” + attribute);
        }
        return c.getValue();
    }

    @Override
    public void setAttribute(Attribute attribute)
            throws AttributeNotFoundException, InvalidAttributeValueException,
            MBeanException, ReflectionException {
        if (attribute == null) {
            throw new RuntimeOperationsException(new IllegalArgumentException(
                    “The attribute name cannot be null.”),
                    “Cannot invoke setAttribute on ” + MBEAN_OBJECT_NAME
                            + ” with null as attribute name.”);
        }
        Counter c = counterMap.get(attribute);
        if (c == null) {
            throw new AttributeNotFoundException(
                    “Could not find the attribute ” + attribute + “.”);
        }
        throw new RuntimeOperationsException(
                new UnsupportedOperationException(),
                “All attributes on the PerfCounters MBean are read only.”);
    }

    @Override
    public AttributeList getAttributes(String[] attributes) {
        AttributeList attributeList = new AttributeList();
        for (String attribute : attributes) {
            try {
                attributeList.add(new Attribute(attribute,
                        getAttribute(attribute)));
            } catch (AttributeNotFoundException | MBeanException
                    | ReflectionException e) {
                // Seems this one is not supposed to throw exceptions. Try to
                // get as many as possible.
            }
        }
        return attributeList;
    }

    @Override
    public AttributeList setAttributes(AttributeList attributes) {
        // Seems this one is not supposed to throw exceptions.
        // Just ignore.
        return null;
    }

    @Override
    public Object invoke(String actionName, Object[] params, String[] signature)
            throws MBeanException, ReflectionException {
        throw new MBeanException(new UnsupportedOperationException(
                MBEAN_OBJECT_NAME + ” does not have any operations.”));
    }

    @Override
    public MBeanInfo getMBeanInfo() {
        return info;
    }

    private static ObjectName createObjectName(String name) {
        try {
            return new ObjectName(name);
        } catch (MalformedObjectNameException e) {
            // This will not happen – known to be wellformed.
            e.printStackTrace();
        }
        return null;
    }

    private Map<String, Counter> setUpCounters() {
        Map<String, Counter> counters = new HashMap<>();
        Perf p = Perf.getPerf();
        try {
            ByteBuffer buffer = p.attach(0, “r”);
            PerfInstrumentation perfInstrumentation = new PerfInstrumentation(
                    buffer);
            for (Counter counter : perfInstrumentation.getAllCounters()) {
                counters.put(counter.getName(), counter);
            }
        } catch (IllegalArgumentException | IOException e) {
            System.err.println(“Failed to access performance counters. No counters will be available!”);
            e.printStackTrace();
        }
        return counters;
    }

    private MBeanInfo createMBeanInfo() {
        Collection<Counter> counters = counterMap.values();
        List<MBeanAttributeInfo> attributes = new ArrayList<>(counters.size());
        for (Counter c : counters) {
            if (!c.isVector()) {
                String typeName = “java.lang.String”;
                synchronized (c) {
                    Object value = c.getValue();
                    if (value != null) {
                        typeName = value.getClass().getName();
                    }
                }
                attributes.add(new MBeanAttributeInfo(c.getName(), typeName,
                        String.format(“%s [%s,%s]”, c.getName(), c.getUnits(),
                                c.getVariability()), true, false, false));
            }
        }
        MBeanAttributeInfo[] attributesArray = attributes.toArray(new MBeanAttributeInfo[attributes.size()]);
        return new MBeanInfo(
                this.getClass().getName(),
                “An MBean exposing the available JVM Performance Counters as attributes.”,
                attributesArray, null, null, null);
    }
}

And here is a little test application that registers the MBean with the platform MBean server and then waits around, giving you an opportunity to hook up with Mission Control and try it out:

import java.io.IOException;
import java.lang.management.ManagementFactory;

import javax.management.InstanceAlreadyExistsException;
import javax.management.MBeanRegistrationException;
import javax.management.NotCompliantMBeanException;

public class PerfCounterMBeanRunner {

    public static void main(String[] args) throws InstanceAlreadyExistsException, MBeanRegistrationException, NotCompliantMBeanException, IOException {
        ManagementFactory.getPlatformMBeanServer().registerMBean(new PerfCounters(), PerfCounters.MBEAN_OBJECT_NAME);
        System.out.println(“Press enter to quit!”);
        System.in.read();
    }

}

If you use the JMX Console to connect to a JVM where the PerfCounters MBean is registered, you should be able to find the PerfCounters MBean using the MBean Browser. There should be quite a few performance counters available:

Conclusion

Sample code for using the proprietary JVM performance counter API was provided, as well as example code on how to implement a version of the old jrcmd –l command, as well as the old PerformanceCounter MBean of JRockit. Again, these examples are using proprietary APIs that may break at any given release. Use responsibly!

If you would like to use a stronger PBE cipher than triple DES & SHA-1 for storing passwords in JMC, this is how to go about it:

1. Download a crypto provider containing the cipher you want. For example the latest provider jar from Bouncy Castle (at the time of writing, this was bcprov-jdk15on-150.jar)
   
2. Copy the jar to JAVA_HOME/jre/lib/ext
   
3. Edit your JAVA_HOME/jre/lib/security/java.security file to include the Bouncy Castle provider. Look for the security.provider entries and add a new entry:

   security.provider.N=org.bouncycastle.jce.provider.BouncyCastleProvider
   , where N is the number of the last entry plus one, for example:

   security.provider.11=org.bouncycastle.jce.provider.BouncyCastleProvider

4. For access to the strongest ciphers, download the Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for your JDK version (this one for example), and unpack the two jar files into JAVA_HOME/jre/lib/security/.

Finally you need to select which cipher to use in the preferences (Window | Preferences, click Java Mission Control). Before adding bouncy castle, you should have had something like this:

After adding Bouncy you should see something like this:

Good luck!

More Raspberry Pi fun with Java! For an Adafruit LCD Shield library, check out my previous blog.

So, instead of just playing around for fun, I decided to actually do something useful. In this case I wanted a little remote sensor station, which I can plug-in wherever I want to, and communicate with over http over GSM.

Building It

First of all you need to bunker up on as many DS18B20s and as many AM2302/DHT11/DHT22’s you think you’ll need. There are plenty of articles out there on how to configure them. I decided to do it a tiny bit differently to most examples by hooking up the power pin to the 5V rail (both since the specs indicate that you can run longer cable lengths if you do so and since more current can be sourced from the 5V rail). The data is pulled up to the 3.3V rail, just like in all the examples. Note that a bunch of DS18B20s can be hooked up in parallel and communicated with over a single GPIO port. Also note that this is NOT possible with the DHT sensors, which will need one GPIO port per sensor.

Ultimately I decided to have two (optional) DS18B20s and one AM2302 on my board, and I used an Adafruit Perma Proto to hook it all up. Mostly because they are pretty. 😉 To connect the AM2302, I used a Molex jumper assembly, and for the DS18B20s I decided to use RJ-11s. I can highly recommend the RJ-11s! The connectors take a bit of space on the card, and they do require you to mutilate the poor Perma Proto a bit (drill larger holes), but attaching/removing sensors is a joy. Here is a picture of the thingy with all the sensors connected:

In my build I will also use an LCD shield for local control and readouts without necessarily having access to a monitor. I also added a few buttons and LEDs, mostly to satisfy my youngest son.

I guess those could come in handy for diagnostics later. 🙂

Now, the 1-wire protocol used by the DS18B20 (Dallas 1-wire) and the 1-wire protocol used by the DHT sensors (AM2302, DHT11, DHT22) are quite different. Both of these require a bit of time sensitive bit-banging. Fortunately, that work has already been done for us.

For the DS18B20 there are two modules available which will expose all the DS18B20s as files under /sys/bus/w1/devices. Simply add w1-gpio and w1-therm to /etc/modules. Note that you must have the DS18B20s on GPIO port 4 for the module to work.

For the DHT sensors there is no module, but I’ve adapted the Adafruit native library described here, to create a native library for Java. Read the README.txt provided in the zip for more information.

Java Access

Once you have added the modules and put native libraries on the library path, there is only a tiny bit of configuration left to do. The library should automatically discover all DS18B20s you have. If you only happen to have one AM2302 on GPIO-pin 22, you are good to go. If not, you need to edit the property file dhtsensors.properties in the w1.jar to correspond to your setup:

# Specify devices to use with the dht library in this file.
# If you have no such devices, just comment everything out.
#
# Type can be one of the following:
# 2302 (for AM2302)
# 11 (for DHT-11)
# 22 (for DHT-22)
sensor0.pin=22
sensor0.type=2302

If you have no DHT sensors, simply comment out the lines. If you have more than one, simply add two more lines for every sensor, indicating which type and what pin they are on. For example:

sensor0.pin=22
sensor0.type=2302
sensor1.pin=23
sensor1.type=2302
sensor2.pin=24
sensor2.type=22

To test that everything is set up properly, simply run:

sudo java -Djava.library.path=<library path> –jar w1.jar

Note that Pi4J isn’t needed on the classpath. We’re relying on the native library and the w1-gpio mod to provide the necessary interfaces.

The output should look something along the lines of this:

Found 4 sensors!

Temperature(28-000004f7960a):23,38°C
Temperature(28-000004f78caf):23,56°C
Temperature(2302@22-temperature):23,00°C
Humidity(2302@22-hygrometer):36,50%

Temperature(28-000004f7960a):23,38°C
Temperature(28-000004f78caf):23,56°C
Temperature(2302@22-temperature):23,00°C
Humidity(2302@22-hygrometer):36,50%

Temperature(28-000004f7960a):23,44°C
Temperature(28-000004f78caf):23,56°C
Temperature(2302@22-temperature):23,10°C
Humidity(2302@22-hygrometer):36,50%

Library Usage

The code is very simple. To get all the sensors, do:

Set<Sensor> sensors = Sensors.getSensors();

To print the sensors like in the example, do:

System.out.println(String.format(“Found %d sensors!”, sensors.size()));
while (true) {
    for (Sensor sensor : sensors) {
        System.out.println(String.format(“%s(%s):%3.2f%s”, sensor.getPhysicalQuantity(),
                                     sensor.getID(), sensor.getValue(), sensor.getUnitString()));
        Thread.sleep(1000);
    }
    System.out.println(“”);
    System.out.flush();
}       

That’s really all there is to it.

Conclusion

A simple Java library to read a few different kinds of temperature and hygrometer sensors was introduced.

Downloads:

• w1.zip (new, read the README.txt)

Since release 7u40, Java can run on LEGO Mindstorms EV3. In this post the JMC Team will explore how to use Java Mission Control to monitor Java applications running on Mindstorms. This post is a guest post by the JMC Team.

What We Have Done

Our primary goal was to show that we can monitor a Java application running on a Java SE embedded platform over JMX.
We started off by unpacking our LEGO Mindstorms kit and built the demo robot which was a good way to get to know the components. A simple demo program was present in the default firmware so it was quite easy to get it to run.

When we had familiarized ourselves with the different sensors and motors we moved on to actually running Java on the system. The leJOS project provides a Linux distribution that can run on Mindstorms and a set of libraries to control just about anything you can connect to the system.

After installing leJOS we verified that we could run Java programs and monitor them using Java Mission Control. Finally we wrote a program that allowed us to control the robot using the MBean management functionality in Java Mission Control.

Running Java programs on Mindstorms

What you need:

• LEGO Mindstorms EV3
• A MicroSD card
• An SD card reader
• A USB WiFi adapter that is supported by leJOS
  We used a Netgear WNA1100 adapter which is identical to the official LEGO WiFi adapter. A list of alterative adapters can be found at http://www.rjmcnamara.com/lego-mindstorms-ev3/mindstorms-ev3-compatible-wi-fi-dongles/ but we have not tested any of them. It is worth pointing out that the adapter we used sticks out quite a bit so it has to be considered when choosing what to build.
• A WiFi network with a DHCP server
  Just about any WiFi router should work.
• A Linux computer to use for writing the leJOS system to the SD card
  If you don’t have a Linux computer you can follow the guide at http://sourceforge.net/p/lejos/wiki/Running%20Ubuntu%20with%20VirtualBox/ to get a virtual Linux system.
• A computer for developing and compiling Java programs.

Installing leJOS

To run leJOS you need a bootable SD card with leJOS on it. When you turn on the Mindstorms system with this card inserted it will boot the leJOS distribution. If the SD card slot is empty it will boot the default LEGO system. This is very nice since you don’t have to make any permanent changes to the firmware. You only have to remove the card in order to make it behave as a normal Mindstorms system again.

A handy tip is to attach a small piece of tape to the SD card. This makes it much easier to remove the card from the Mindstorms system if you need to update it later or if you want to keep several cards with different contents.

To install leJOS on an SD card, follow the guide at http://sourceforge.net/p/lejos/wiki/Creating%20a%20bootable%20SD%20card/.
Note that when you insert the SD card into the Linux system for formatting and writing, it is not always clear which device the SD card is using. If the card was mounted automatically you can use the command mount to see which devices are mounted. You can also use the command dmesg to see messages that come when hardware changes are detected. In any event, take care not to select the device for your hard disk.

When the SD card is complete you can start using it:

• Insert the WiFi USB stick into the USB port on the Mindstorms system
• Insert the SD card into the SD card slot on the Mindstorms brick and turn it on

If everything worked correctly you will after a while see the leJOS logo and one or two IP addresses on the screen. One is for USB and Bluetooth networking and the other is for WiFi.

Connect to the WiFi IP address using ssh and login as root with no password. This will give you a pretty normal Linux prompt from where you can run your programs.

You can try running a demo program:

cd /home/root/lejos/samples
jrun EV3HelloWorld

This should result in some beeping and blinking.

Development environment

If you follow the instructions at http://sourceforge.net/p/lejos/wiki/Developing%20with%20leJOS/ then you will get an Eclipse development environment where you can do everything from within Eclipse.

Minimum requirements

If you can’t or don’t want to do everything from within Eclipse then you can set up your own environment. For a minimal setup all you need to do is to get hold of the ev3 library.

The first option is to get the latest source files from git. You can do this either by running git clone git://git.code.sf.net/p/lejos/ev3 lejos-ev3 or, if you don’t have git installed, downloading a snapshot of the code from http://sourceforge.net/p/lejos/ev3/ci/master/tree/. The subdirectories contains various libraries and demo projects complete with Eclipse project files. The ev3classes subdirectory contains the source code for the leJOS API so import at least that into your favorite development environment.

You should preferably build the DbusJava and ev3classes projects with ant. This will give you an ev3classes.jar file which should be placed in /home/root/lejos/lib on your Mindstorms system so that you have the same version of the library when you run your programs.

The second option is to take the ev3classes.jar file from the leJOS installation and add it to your classpath. You can find this jar file on your Mindstorms system in /home/root/lejos/lib. Note however that the ev3classes.jar file that is distributed with the leJOS installer is not necessarily the latest so we recommend getting the source code from git and building it yourself if you can.

Development process

In short, the development process consists of these stages:

• Write a program and compile it on your development system
• Upload the class or jar files to the Mindstorms system
• Run the program on the Mindstorms system

If you followed the guide on the lejos wiki then you can follow their instructions. Otherwise, you can use your favorite scp application to transfer files and your favorite ssh terminal to run your programs.
Whichever way you choose, start off with a simple program like this one:

import lejos.hardware.Button;
import lejos.hardware.LCD;

public class HelloWorld {
    public static void main(String[] args) throws Exception {
        LCD.clear();
        LCD.drawString(“Hello World”, 0, 5);
        Button.waitForAnyPress();
        LCD.clear();
        LCD.refresh();
    }
}

Compile the program and create a jar file with the class:

javac -cp ev3classes.jar HelloWorld.java
jar cvf HelloWorld.jar HelloWorld.class

Copy the jar file to the Mindstorms system using scp and start it from an ssh terminal:

jrun -cp HelloWorld.jar HelloWorld

Connecting Java Mission Control to a remote Java process

Java Mission Control is a tool for monitoring and analyzing Java processes. It has been bundled together with JDK since version 7u40. Java Mission Control uses JMX to communicate with remote Java processes. To enable the management agent on a Java process you can add the following parameters when starting it:

-Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

You can select just about any free port number instead of 7091. If you are running multiple Java processes on the same system then you need to use different ports for each process. Depending on how the leJOS system detects which IP address is the primary one, you may also have to add the following parameter:

-Djava.rmi.server.hostname=The IP address of the Mindstorms WiFi adapter

Once the Java process is up and running you can connect to it using Java Mission Control. Add a connection to the Mindstorms IP address and the port number that you chose. Expand your new connection in the JVM browser view and open a JMX Console on the MBean Server.

 
If everything worked as it should you will see a console window displaying the status of the Java process.

This procedure is the same for any remote Java process that you want to monitor. Java Mission Control can auto-detect Java processes that can be monitored through a protocol called JDP. You can read more about JDP at https://hirt.se/blog/?p=388.

Monitoring and controlling Mindstorms using MBeans

Now you have a Java process running on the Mindstorms system and you can communicate with it using JMX. This allows you to see a lot of information about what the Java process is doing. The next step is to monitor and control the sensors and motors connected to the Mindstorms system. To do this we implement a custom MBean that expose the leJOS libraries.

You can read more about MBeans and MXBeans at http://docs.oracle.com/javase/tutorial/jmx/mbeans/index.html.

Creating an MBean

An MBean is basically a Java class that implements an interface. Start off by creating the interface:

public interface MotorMXBean {

    public void moveForward();

    public void stopMotor();

    public int getSpeed();

    public void setSpeed(int speed);
}

Then create a class implementing the interface:

import lejos.hardware.motor.NXTRegulatedMotor;

public class Motor implements MotorMXBean {
    final NXTRegulatedMotor m_motor;

    public Motor(NXTRegulatedMotor motor) {
        m_motor = motor;
    }

    @Override
    public void moveForward() {
        m_motor.forward();
    }

    @Override
    public void stopMotor() {
        m_motor.stop();
    }

    @Override
    public int getSpeed() {
        return m_motor.getSpeed();
    }

    @Override
    public void setSpeed(int speed) {
        m_motor.setSpeed(speed);
    }
}

This exposes a small subset of everything that can be done but it will give you an idea of how things work. You don’t have to expose the leJOS library methods directly, often it can be more useful to have more complex operations like ”find a ball” or monitor values like ”number of balls found”.

With the MBeans implemented, it is time to create an instance and register it with the platform MBean server.

import java.lang.management.ManagementFactory;
import javax.management.MBeanServer;
import javax.management.ObjectName;

public class MBeanRunner {
    public static void main(String[] args) throws Exception {
        MBeanServer mbs = ManagementFactory.getPlatformMBeanServer();
        Motor m = new Motor(lejos.hardware.motor.Motor.B);
        ObjectName name = new ObjectName(“lejostest:name=MotorB”);
        mbs.registerMBean(m, name);
        System.out.println(“Server is running. Press enter to exit”);
        System.in.read();
    }
}

This is a very simple program that doesn’t do anything on its own. It registers a motor controller with the MBean server and waits for commands. Compile the classes and place them on the Mindstorms system. Connect a motor to port B on the Mindstorms system and start the program:

jrun -cp Test.jar -Dcom.sun.management.jmxremote=true -Djava.rmi.server.hostname=192.168.2.3 -Dcom.sun.management.jmxremote.port=7091 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false MBeanRunner

Start Java Mission Control and connect to the Mindstorms system. Go to the MBean Browser and locate the lejostest/MotorB MBean. Go to the Operations tab, select moveForward and click on Execute:

The motor should start running. Go to the Attributes tab and you will see a list of values for all attributes that are exposed with getter methods. Attributes that have setter methods can be set so try clicking on the value for the speed attribute, enter a new value and finish with enter:

The motor should now change speed. To stop the motor, go back to the operations tab and execute the stopMotor operation.

Conclusion

If you have followed this guide you will have accomplished three things:

1. Running Java programs on a Mindstorms system
2. Monitoring a remote Java process with Java Mission Control
3. Exposing application specific data and operations in an MBean

All of these things can be done independently. You can run Java programs on Mindstorms without monitoring them, any program running on a supported JVM can be monitored with Java Mission Control, and there are other methods of monitoring MBeans.
Enjoy.

More information about Java Mission Control can be found at http://www.oracle.com/missioncontrol.
More information about leJOS can be found at http://www.lejos.org/ and at http://sourceforge.net/projects/lejos/.

We’re interrupting the ordinary programming for a series of Raspberry PI related articles. Yep, I finally bought one and started having some fun.

I know I am sooo late to the party. My excuse is that having really young kids translates into a constant time shortage. That said, I bought myself a Raspberry Pi just in time for some vacation. One of the first things I did was to get an LCD shield for the Raspberry PI. I settled on this particular kit:
http://www.adafruit.com/products/1115

It’s a really nice little LCD kit which also features a few buttons. The shield uses a port extender (MCP23017) to get another 16 GPIO ports over I²C. This is quite nice, since you probably want to use your GPIO ports for other things. Also, you can hook up several I²C devices on the same bus, so using the I²C pins for the LCD shield doesn’t mean that you’ve used them up.

The LCD shield can be run in various different modes. My little example library uses 16×2 characters by default.

Building the Kit

The instructions for building the kit are excellent, and there are just a few things I’d like to note:

1. Get a stacking header
   I fortunately came to the conclusion I would need a stacking header before soldering everything together, but unfortunately after I had ordered the LCD shield. I am oh so happy I decided to wait for the header to arrive before finally putting the kit together.
2. If you want to attach a standard flat 26 pin cable, trim out some space in the LCD PCB for the little orientation peg on the cable header 
   This I did not do – I ended up trimming down the cable header with my Dremel instead, since it’s too hard to access that area after soldering on the LCD.
3. (Optional) Unused GPIO pins for free!
   E.g. GPA5 appears to not be in use. Feel free to use it for your own purposes. The library initializes it as yet another input pin by default, but that is easily changed.
4. (Optional) Use interrupts for the buttons
   This will cost you a GPIO pin (and you cannot use one of the unused ones on the LCD shield’s MCP23017), but will allow you to poll the button pins a lot less often without risking missing a button press. You may want to do some additional soldering on pin 20 of the MCP23017 – INTA is not hooked up by default. This also requires you to hack my Java code a little bit to initialize GPINTEN, and to read from INTF and INTCAP,

Testing the Kit

Once you’ve built the kit, follow the instructions here to try out the LCD kit using the demo python code. Once you have your LCD shield up and running you can move on to getting everything to run from Java.

Using the LCD Shield from Java

Once everything is working from python, you are ready to run things from Java. I mostly did a straight port of the Python library.

1. First download and install Pi4J
   This will be required for most future posts I might put on the blog.
2. Next download lcd.jar
   The source is available with lcd_src.zip.

Now you can run the included demos by just invoking the jar like this:
sudo java –Xbootclasspath/a:/opt/pi4j/lib/pi4j-core.jar –jar lcd.jar

When running the demos, use the up/down buttons to select a demo, then press select to run a demo. When the demo is done, use up/down to select a new demo.

API Examples

Here is how the API can be used to show some text:

LCD lcd = new LCD();
lcd.setText("Hello World!\n2nd Hello World!");

Here is an example with some buttons:

final LCD lcd = new LCD();
lcd.setText("LCD Test!\nPress a button...");
ButtonPressedObserver observer = new ButtonPressedObserver(lcd);
observer.addButtonListener(new ButtonListener() {
    @Override public void onButtonPressed(Button button) {
        lcd.clear();
        lcd.setText(button.toString());
    }
});

For more examples, check out the demos in the se.hirt.pi.adafruitlcd.demo package. I currently have most of the implementation in my python port LCD class. Anything that isn’t purely reading/setting registers, I’ve handled outside of that class (such as the polling of buttons). If I ever find some time I might clean up the API and move away from the current python port.

Tips and Tricks

• If you want to teach the LCD to do additional tricks, the LCD controller used is the HD44780
  Note that it might be valuable to read the spec if for no other reason than to understand the limits (e.g. quite small DDRAM buffer).
• MCP23017 is an awesome little thing
  Ever wanted more GPIO ports? Using the MCP23017 is one way to get plenty more. Just remember to provide a unique address by configuring A0-A2 (see page 8 of the spec). Connecting them all to ground will yield 0x20 (which is what the LCD shield uses). Hooking them all up to 5v will yield 0x27.

• Be careful with how much current you source/sink, both using the MCP23017 (25mA) and the RPi (16mA GPIO, 50mA 3.3v rail, 5V rail pretty much depends on your power supply). If you need to source/sink more than the IO pins can handle, you can use a Darlington Array, for example ULN2803.

This blog post is the third in a series of posts on using unsupported functionality in the Oracle JDK and/or Java Mission Control.

I will use the JMC flight recorder parser in this example, and since the JMC parser is unsupported I’ll just go ahead and use my usual disclaimer:

The following blog entry will describe UNSUPPORTED functionality. This means that relying on the described APIs or functionality may BREAK your code/plugin with any given update of the JDK and/or Mission Control.

I strongly recommend against using these APIs in production code. There will be supported APIs for reading reading recordings eventually and they may be quite different to what I show here. Also note that older versions of the parsers are not guaranteed to be forward compatible with newer versions of the binary format. In short, this is NOT supported and WILL CHANGE!

You may want to first read the introduction to the JFR parsers and about the relational key before continuing.

Ready? Here we go…

A Flight Recording

First off we need a recording to parse. If you have your own favourite recording, you’re all set. If you don’t you can have mine. The recording I linked to is a recording of WLS running the MedRec example app with some load. The benefit of using that recording is that it, aside from the standard events, also contains events provided by the Java API for creating third party events (by the WebLogic Diagnostics Framework). Quite useful events, all bound together using the relational key in various ways. Don’t forget to unzip the recording before use.

The recording is pretty new and really requires a JMC 5.3.0 with the WLS plug-in for the best result. JMC 5.3.0 has not been released yet, but don’t worry; this blog entry will not even require you to start JMC.

That said, since all blog posts require a pretty picture, here is one of the servlet invocations in the recording:

Looking at Metadata

Let’s first write a little program that lists the available event types and their metadata. Use the knowledge obtained in the JFR parser blog entry to obtain a copy of the JMC Flight Recorder Parser.

Next, use the example below to iterate through the metadata and print out the event type name, all the attributes and the relational key for the the individual attributes:

import java.io.File;
import java.util.Collection;

import com.jrockit.mc.flightrecorder.FlightRecording;
import com.jrockit.mc.flightrecorder.FlightRecordingLoader;
import com.jrockit.mc.flightrecorder.spi.IEventType;
import com.jrockit.mc.flightrecorder.spi.IField;

public class FlightRecorderMetaData {
    public static void main(String[] args) throws ClassNotFoundException {
        FlightRecording recording = FlightRecordingLoader.loadFile(new File("C:\\demo\\wldf.jfr"));

        for (IEventType type : recording.getEventTypes()) {
            System.out.println(type.getName());
            printAttributes(type.getFields());
        }
    }

    private static void printAttributes(Collection<IField> fields) {
        for (IField field : fields) {
            System.out.println(String.format("   %s (relkey: %s)", field.getName(), field.getRelationalKey()));
        }
    }
}

Here is a slightly more convoluted example, printing everything out in a nice tree, also showing how many events were found per event type plus some additional stats. It takes a flight recording as its only argument.

Using the Relational Key

Looking at the metadata for the WLDF recording, it can be seen that a lot of the event attributes provided with the WLDF specific events are actually using the relational key for various different things. There is a relational key for servlet URIs, there is another one for the ECID (Enterprise Context ID) and so on. Let’s say we wanted to find all the events with a relational key for ECID, and then group them on ECID (effectively grouping the events per transaction). That could look something like this:

import java.io.File;
import java.util.ArrayList;
import java.util.Collection;
import java.util.HashMap;
import java.util.LinkedList;
import java.util.List;
import java.util.Map;
import java.util.Map.Entry;

import com.jrockit.mc.flightrecorder.FlightRecording;
import com.jrockit.mc.flightrecorder.FlightRecordingLoader;
import com.jrockit.mc.flightrecorder.spi.IEvent;
import com.jrockit.mc.flightrecorder.spi.IEventType;
import com.jrockit.mc.flightrecorder.spi.IField;
import com.jrockit.mc.flightrecorder.spi.IView;

public class FlightRecorderECID {
    private static final String KEY_ECID = "http://www.oracle.com/fmw/ECID";

    public static void main(String[] args) throws ClassNotFoundException {
        FlightRecording recording = FlightRecordingLoader.loadFile(new File("C:\\demo\\wldf.jfr"));    

        Collection<IEventType> ecidTypes = new LinkedList<>();
        for (IEventType type : recording.getEventTypes()) {
            if (type.getPath().startsWith("wls/")) { // just optimization, can remove
                if (hasECID(type)) {
                    ecidTypes.add(type);                   
                }
            }
        }
       
        IView ecidView = recording.createView();
        ecidView.setEventTypes(ecidTypes);
        Map<String, List<IEvent>> eventMap = buildEventMap(ecidView);
       
        System.out.println("ECIDS:");
        for (Entry<String, List<IEvent>> ecid : eventMap.entrySet()) {
            System.out.println(String.format("%s [%d events]", ecid.getKey(), ecid.getValue().size()));
        }
    }

    private static Map<String, List<IEvent>> buildEventMap(IView ecidView) {
        Map<String, List<IEvent>> map = new HashMap<>();
        for (IEvent event : ecidView) {
            String ecid = getEcid(event);
            if (ecid != null) {
                addToMap(map, ecid, event);
            }
        }
        return map;
    }

    private static void addToMap(Map<String, List<IEvent>> map, String ecid, IEvent event) {
        List<IEvent> eventList = map.get(ecid);
        if (eventList == null) {
            eventList = new ArrayList<>();
            map.put(ecid, eventList);
        }
        eventList.add(event);
    }

    private static String getEcid(IEvent event) {
        IField ecidField = getEcidField(event.getEventType());
        if (ecidField == null) {
            return null;
        }
        return String.valueOf(ecidField.getValue(event));
    }

    private static boolean hasECID(IEventType type) {
        return getEcidField(type) != null;
    }

    private static IField getEcidField(IEventType type) {
        for (IField field : type.getFields()) {
            if (field.isRelational() && KEY_ECID.equals(field.getRelationalKey())) {
                return field;
            }
        }
        return null;
    }
}

The output should look something like this:

8ec006a7-30e9-4fac-be7f-716f42d3cbc8-00000ed0 [1 events]
8ec006a7-30e9-4fac-be7f-716f42d3cbc8-00000237 [45 events]
8ec006a7-30e9-4fac-be7f-716f42d3cbc8-00000236 [2 events]
8ec006a7-30e9-4fac-be7f-716f42d3cbc8-00000d58 [2 events]
8ec006a7-30e9-4fac-be7f-716f42d3cbc8-0000022a [2 events]
8ec006a7-30e9-4fac-be7f-716f42d3cbc8-0000022d [2 events]
8ec006a7-30e9-4fac-be7f-716f42d3cbc8-0000022c [12 events]

Running the same example but using the relational key for URI (http://www.oracle.com/wls/Servlet/servletName) will look something like this:

/console/index.jsp [10 events]
/_async/AsyncResponseService [195 events]
/medrec/registerPatient.action [90 events]
/physician-web/physician/addPrescription.action [448 events]
/console/images/button_bg_mo.png [7 events]
/physician-web/physician/viewPatients.action [432 events]
/medrec/admin/home.action [48 events]
/console/jsp/changemgmt/ChangeManager.jsp [9 events]
/physician-web/physician/createRecord.action [720 events]
/medrec-jaxws-services/PatientFacadeService [275 events]
/medrec-jaxrpc-services/JaxRpcRecordCreationFacadeBroker [235 events]

Conclusion

This blog provided some examples on how to read out metadata from a flight recording and a simple example showing the use of the relational key.

This blog post explains the relational key – a little piece of event attribute metadata that is used to associate attributes in different event types with each other. This is a quick break in my series of posts about unsupported functionality, the reason for the break being that you will need to be armed with this knowledge for my next instalment in that series.

The Relational Key

The relational key is a little piece of event type attribute metadata on the form of an URI. It is normally used as a marker to hint to JMC that it often makes sense to find all events (of different event types) with the same attribute value for attributes with the same relational key. For example, the Java Flight Recorder has quite a lot of different GC events. Almost all of them are related to a certain GC ID. It often makes sense to find all events related to the same GC ID. The picture below lists some common GC events, and their attribute metadata.

The relational key can be seen in action in how Mission Control presents the alternatives for the Operative Set. If you go to the GC view, pick a garbage collection and open the context menu, you will find that there is an option to add all the events with the same GC ID to the operative set.

Moving over to the events tab, and selecting to only show the operative set, a range of different events, all associated with that particular garbage collection, can be seen.

Another very useful example is the ECID attribute in the events produced by the WebLogic Diagnostics Framework (see the “Putting it All Together” section of my post about the Operative Set).

Conclusion

The relational key is a handy little piece of attribute metadata that is used to give hints to the JMC user interface that different event types are related to each other. It is used by Mission Control to, for example, control what attributes are listed in the Operative Set context menu.

This blog post is the second in a series of posts on using unsupported functionality in the Oracle JDK and/or Java Mission Control.

Just as there is no supported way to add custom events to the Java Flight Recorder (yet), there is even less in terms of programmatic support to read recordings. There are however two different unsupported parsers available. Both are POJO APIs, and except for supportability / future API breakage issues, there is in practice nothing stopping you from using either one to read your recordings.

Disclaimer:

The following blog entry will describe UNSUPPORTED functionality. This means that relying on the described APIs or functionality may BREAK your code/plugin with any given update of the JDK and/or Mission Control.

I strongly recommend against using these APIs in production code. There will be supported APIs for reading reading recordings eventually and they may be quite different. Also note that older versions of the parsers are not guaranteed to be forward compatible with newer versions of the binary format. In short, this is NOT supported and WILL CHANGE!

That said, it can sometimes be quite useful to have programmatic access to flight recorder data. One could, for example, do automatic filtering and/or reporting on recordings, and then use JMC to do detailed analysis on the ones that are flagged as interesting.

The Reference Parser

The reference parser shipped with the Oracle JDK can be likened to a SAX XML parser. It allows you to iterate through the recording, uses little memory, is somewhat limited, and provides a very simple Java model. If your needs are simple, this may be the perfect parser to use. It is available by default in JDK 7u4 and later. The parser API is located in the JDK_HOME/jre/lib/jfr.jar file.

Here is an example snippet that iterates through a recording and prints out the total event count:

I will not spend too much time on this parser.

Strengths:

• Uses little memory
• Included in the Hotspot JDK (>=7u4)

Weaknesses:

• Slow
• Simple model/API

The JMC Parser

If the reference parser is the SAX XML parser equivalent, then the JMC parser is the DOM equivalent. The jar for the parser is easily nicked from the JMC distribution, either from the update site or from the JDK installation. In the JDK, you’ll find the jar containing the JMC flight recorder parser at JDK_HOME/lib/missioncontrol/plugins/com.jrockit.mc.flightrecorder_<version>.jar.

You will also need to add the JDK_HOME/lib/missioncontrol/plugins/com.jrockit.mc.common_<version>.jar to your path.

Here is the equivalent event counting example using the JMC parser:

Strengths:

• Fast
• The jar can be used with any JDK (>=6)
• Rich API
• Good for complex analysis scenarios

Weaknesses:

• Builds a Java model of the recording – uses more memory

Summary

This blog post lists two different UNSUPPORTED parsers that can be used to read java flight recordings programmatically from Java. A very simple example showing how to iterate through the events in a recording was shown for both of the parsers.