Embarrassing Bug

Writing the JRockit book has made me take trips down memory lane more than once. The last read through made me think of one of the more embarrassing bugs we’ve had in JRockit Mission Control. Since it’s too late to add content to the book – most of the chapters are already finalized, or in the process of being finalized -  I thought I’d just put it here on the blog. 🙂

So, one of the more embarrassing bugs in Mission Control would be when an attibute was misspelt in the Management Console. An attribute ending with the word count lacked the letter ‘o‘. The bug report stated that the "customer was not amused".

I couldn’t help being slightly amused though.

JRockit R28/JRockit Mission Control 4.0 is out!

The next major release of JRockit is finally out! Here are some highlights:

  • Includes the all new JRockit Flight Recorder – supersedes the old JRockit Runtime Analyser. The new flight recorder is inspired by the “black box” in airplanes. It uses a highly efficient recording engine and thread local buffers to capture data about the runtime and the application running in the JVM. It can be configured to always be on, so that whenever anything “interesting” happens, data can be dumped for some time back. Think of it as your own personal profiling time machine.
  • Automatic shortest path calculation in Memleak – no longer any need for running around in circles when trying to find your way back to a thread root from an instance.
  • Memleak can now show class loader related information and split graphs on a per class loader basis.
  • More easily configured JMX agent – default port for both RMI Registry and RMI Server can be configured, and is by default the same, allowing easier configuration of firewalls.
  • Up to 64 GB (was 4GB) compressed references.
  • Per thread allocation profiling in the Management Console.
  • Native Memory Tracking – it is now possible to track native memory allocations with very high resolution. The information can either be accessed using JRCMD, or the dedicated Native Memory Tracking experimental plug-in for the Management Console (alas only available for the upcoming 4.0.1 release).
  • JRockit can now produce heap dumps in HPROF format.
  • Cooperative suspension – JRockit is no longer using system signals for stopping threads, which could lead to hangs if signals were lost or blocked (for example bad NFS shares). Now threads check periodically to see if they are suspended.
  • VPAT/Section 508 compliant JRMC – greatly improved keyboard navigation and screen reader support.

See New and Noteworthy for more information.

JRockit Mission Control 4.0.0 can be downloaded from here:
http://www.oracle.com/technology/software/products/jrockit/index.html

There is even a book to go with JRMC 4.0.0/JRockit R28!  🙂http://www.packtpub.com/oracle-jrockit-the-definitive-guide/book/

Laziness is the Mother of All Inventions

I recently got tired of monitoring the weather for various locations to find a nice time and spot to fly my planes. Enter WindBot. WindBot sends me an e-mail whenever new opportunities are found:

—–

The WindBot@hirt.se has found changes to the available flying opportunities for you in Trollbäcken:

New opportunities:
==================
Friday 15:00 (2010-04-23) [wind 2.0m/s, precipitation 0.0mm, cloud cover 43.0%]
Friday 22:00 (2010-04-23) [wind 2.0m/s, precipitation 0.0mm, cloud cover 100.0%]

Lost opportunities:
===================

Triggering condition was:
MicroPlaneFlyCondition [wind speed <= 2.0, precipitation <= 1.0, 5 <= time <= 22]

/WindBot v0.0.3

—–

Lovely! 🙂

JRockit Mission Control @ OOW 2009

There are a bunch of JRockit related sessions at OOW 2009! Joy!

Here are the ones I am involved in:

Session ID Session Title Date/Time Room

S309689

The Next Generation of Profiling and Diagnostics Tools

Sunday
10/11/2009
10:30 – 11:30

Hilton Hotel
Golden Gate 4/5

S309676

Advanced Java Diagnostics and Profiling with Oracle JRockit Mission Control

Monday
10/12/2009
11:30 – 12:30

Hilton Hotel
Continental Ballroom 4

Other exciting sessions:

Session ID Session Title Date/Time Room
S309305 Oracle JRockit: What’s New and What’s Coming Monday
10/12/2009
16:00 – 17:00
Marriott Hotel
Salon 7
S309680 Latency Is a Bug: Tuning Your Java Virtual Machine for Optimal Performance Monday
10/12/2009
10:15 – 11:15
Hilton Hotel
Continental Ballroom 4
S309331 A Next-Generation Platform for Virtualized Java Wednesday
10/14/2009
13:00 – 13:30
Moscone South
Room 309

What do I do for a living?

I often get asked what I work with, and I’ve always had a hard time answering that. The problem usually starts when I introduce the concept of a Java Virtual Machine. So I thought I’d write a few lines on the evolution on computing to sort of explain how what I work with came about.

At the dawn of electrical computing programming was done by manually closing and opening electrical circuits. This evolved into punching holes into paper cards, representing the closed and open circuits. Eventually better methods of providing the instructions to the processing units came along, but at the hardware level the processing units still need to get its instructions as sequences of ones and zeroes. At first assemblers appeared that had simple mnemonics, or textual symbols, that were more or less a handy but direct representation of the sequences of ones and zeroes. Higher level programming languages, such as C, appeared that provided means of expressing the meaning of the program, the semantics, using better and easier to read syntax. Other programs, so called compilers, were responsible for analyzing the meaning of the program and translating it into machine code. As the processors became more advanced, so did the compilers, and today it is very complicated for a programmer to, even by hand, produce better and more efficient programs by hand in assembly, than the machine code produced by an optimizing compiler. Even with intimate knowledge of the processor architecture, it can be very hard to, for instance, decide upon the best way to schedule the instructions to best utilize parallel instruction pipes.

Java is a modern programming language, and represents another leap in abstraction. Its syntax is reminiscent to that of C, one of the most popular and successful imperative programming languages to date. Using a C-like syntax made the transition to Java easier for programmers already knowledgeable in C, and helped drive adoption.

Quite unlike C, however, Java programs are not compiled into machine code that the hardware which will execute the program can understand. Java programs are instead translated to platform independent byte code, which another piece of software, the Java Virtual Machine, or JVM, translates into machine code. You can view the Java Virtual Machine as a hardware abstraction layer, with its own machine code. The JVM does the final translation from the Java byte code to the machine code of the underlying hardware. Since the JVM is translating the Java byte code while it is running, the JVM has more information about the runtime characteristics of the program running in the JVM than a static compiler could ever have. This means that there is more information available to do advanced optimizations.

At the beginning, Java got a reputation for being very slow. The early JVMs were interpreting and translating the Java byte code line by line. This was rather inefficient. Today, high performance JVMs like JRockit are analyzing the semantics of the Java program while it is executing. It translates the program into machine code using a JIT (Just In Time) compiling strategy; translating method by method into machine code as it is required. It uses an internal profiler to find out how the application is executing, and dynamically does optimizations to the running program; in effect recompiling methods while the program is running. A modern JVM has a high performance, adaptive memory management system that efficiently handles the allocation and reclaiming of memory for the developer. Today, a Java program running in a modern JVM can outperform a well written C program.

So, to finally answer the question, I work with JRockit – one of the fastest and most advanced JVMs in the world.

Getting JRockit Stack Dumps

Sometimes it may seem hard to get a simple thread stack dump from your JRockit. You may for instance have started it using the -Xnohup (-Xrs) option, or as a service. I’ll list three different ways of looking at your threads in such situations.

The first way of getting your thread dump is to use jrcmd. My previous blog shows some basic jrcmd usage, so I’ll just mention the command name: print_threads. Basic usage is jrcmd <PID> print_threads, for example jrcmd 780 print_threads.

The second way is to simply start the management console on the JRockit you want to monitor. After connecting the console to the JVM and switching to the threads view, you should be looking at something similar to the screen shot below.

thread_dumps

The third way is to use the MBean Browser’s capability to invoke arbitrary operations on MBeans (also described in the last blog). Go to the MBean Browser in the JRockit Management Console. Select the DiagnosticCommand MBean, and switch to the operations tab. Select the execute operation that takes a String argument and returns a String. Click the String button, and fill in the argument print_threads.

invoking_dc

When you execute the operation you should see something like this:

after_dc

There are also ways of doing this programmatically, for example by using the JRockit JMAPI, or by invoking the execute operation programmatically as described in another old blog.

Simple Exception Profiling with JRockit

Exception profiling is the business of finding out what exceptions are being thrown within your application and from where. In JRockit Mission Control you can find out how many exceptions have been thrown using JRA, and you can count the exceptions using the JRockit Management Console. Sadly there is no way of doing powerful exception profiling (i.e. looking at the stack traces for the exceptions, aggregating them and visualizing them directly in the JRockit Mission Control Client) just yet. This will be incorporated in a future version of Mission Control.

There is fortunately a way to do exception profiling with JRockit today. There are exception related verbose flags available in JRockit. These are examples on how to use them:

java -Xverbose:exceptions=debug

java -Xverbose:exceptions=trace

For a list of valid verbose options, please see http://e-docs.bea.com/jrockit/jrdocs/refman/optionX.html#wp999543.

Sometimes you really just want to enable this profiling for a little while, for example to avoid cluttering your log files. Then there is a nifty tool for JRockit called jrcmd you can use to turn on/off the exception profiling. It can be found under JROCKIT_HOME\bin\.

First use jrcmd to list all running java processes. Here is an example of what it may look like:

>jrcmd

10984 jrockit.tools.jrcmd.JrCmd

9396 C:\Java\eclipse3.3.1.1\plugins\org.eclipse.equinox.launcher_1.0.1.R33x_v200
70828.jar -data D:\Data\workspaces\workspace_3.3_facade -os win32 -ws win32 -arc
h x86 -showsplash -launcher C:\Java\eclipse3.3.1.1\eclipse.exe -name Eclipse –l
auncher.library C:\Java\eclipse3.3.1.1\plugins\org.eclipse.equinox.launcher.win3
2.win32.x86_1.0.2.R331_v20071019\eclipse_1021.dll -startup C:\Java\eclipse3.3.1.
1\plugins\org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar -exitdata 1908_7
c -vm D:\jrockits\R27.4.0_R27.4.0-90_1.5.0\bin\javaw.exe -vmargs -Xms256m -Xmx51
2m -XgcPrio:deterministic -XpauseTarget:40 -Dosgi.bundlefile.limit=100 -jar C:\J
ava\eclipse3.3.1.1\plugins\org.eclipse.equinox.launcher_1.0.1.R33x_v20070828.jar

10976 Gegga

The numbers are the PIDs of the processes, and are used as the first argument to jrcmd to specify on what JRockit to address. You can use jrcmd to check what commands are available. Here is an example:

>jrcmd 10976 help
10976:
The following commands are available:
        kill_rmp_server
        start_rmp_server
        kill_management_server
        start_management_server
        checkjrarecording
        stopjrarecording
        startjrarecording
        print_object_summary
        memleakserver
        print_codegenlist
        print_class_summary
        run_optfile
        dump_codelayout
        dump_codelist
        dump_codemap
        print_utf8pool
        print_properties
        print_threads
        datadump_request
        runsystemgc
        runfinalization
        heap_diagnostics
        oom_diagnostics
        print_exceptions
        version
        timestamp
        command_line
        memprof
        sanity
        verbosity
        set_filename
        help
        print_memusage
For more information about a specific command use ‘help <command>’.
Parameters to commands are optional unless otherwise stated.

Let’s get more information about the print_exceptions command:

>jrcmd 10976 help print_exceptions

10976:
Enable printing of Java exceptions thrown in the VM.
To turn exception printing off completely you need to set exceptions = false
even if it was turned on by stacktraces = true. NOTE: This handler is
deprecated. The preferred way of displaying exceptions is to use the
‘exceptions’ logging module: -Xverbose:exceptions.
Values for the parameters can be "true|all|false"
true  – print all exception except java/util/EmptyStackException,
java/lang/ClassNotFoundException and
java/security/PrivilegedActionException
all   – print all exception
false – don’t print exceptions
        exceptions  – print exceptions (string, false)
        stacktraces – print stacktrace (string, false)

Never mind that it is using an old handler in R27.6. The functionality will be the same. Let’s enable the exception profiling:

>jrcmd 10976 print_exceptions stacktraces=true

If your application is throwing exceptions, you should now be seeing traces being logged to the console of that application. To turn off the exception logging, we do as the help suggests:

>jrcmd 10976 print_exceptions exceptions=false

Now, an even easier way to access the jrockit diagnostic commands is to use the JRockit Management Console. Simply connect the JRockit Management Console to the JRockit you wish to enable exception profiling on (from JRockit Mission Control). (Click the images to look at them in full size.)

start_console

Once the console is started, open the MBean Browser tab and move to the DiagnosticCommand MBean under bea.jrockit.management:

diagnostic_command

Double clicking on the ArrayList value of AllCommands will open up a list of all the available commands:

list_commands 
Now, select the operations tab and the executeDefault method that takes a String argument and press invoke. This will provide you with a dialog with push buttons for the arguments. In this case a single button with the label String. Press the String button and fill out the command. For example, print_exceptions stacktraces=true.

print_traces

Press Ok on both of the dialogs and exception profiling should now be enabled. To turn it off again, just invoke the operation again and enter print_exceptions exceptions=false, analogously with how you would use jrcmd.

An even easier way for Oracle employees is to pick up a recent internal build of JRockit and use the Diagnostic Command tab in the JRockit Management Console. 🙂 Soon coming to a JRockit near you!