How To Measure

Timers

There are currently two methods for execution time measurement available in Java:
  • System.currentTimeMillis() enables the measurement of time with millisecond precision. The accuracy of this timer can be problematic because the resolution heavily depends on the underlying operating system and its configuration.
  • System.nanoTime() provides a much better resolution and accuracy than System.currentTimeMillis(). However, due to hardware timer limitations and Linux kernel restrictions the values provided by this method have nanosecond precision but actually only microsecond accuracy. It is considered generally more reliable than System.currentTimeMillis() though.

For our measurements the System.nanoTime() method was used under the premise that a variance of a few microseconds is acceptable.

Measurement method

It is important to not impair performance in order to gain measurements as accurate as possible. Several methods were considered, none yielded the desired results. Most existing Java profiling mechanisms have a noticable effect on runtime speed. Therefore basically two approaches were used:
  • inserting System.nanoTime() directly in the code and
  • using AOP for creating an aspect and weaving it into the code

The resulting execution times were aggregated using a buffer in a separate thread and periodically flushed over the network to a separate loghost using Log4J Socket Server in order to not impair IO performance on the ingesting host.

JIT

The JIT compiler does lots of optimizations after the application is started. Classes aren't loaded by the JVM until the first time they are referenced, and code is compiled from bytecode to machine code only after it has been executed a number of times. We observed that it takes the JVM approximately the first 10,000 ingests to "warm up". Only then the measured times can be considered representative.

Add new attachment

In order to upload a new attachment to this page, please use the following box to find the file, then click on “Upload”.
« This page (revision-2) was last changed on 09-Jul-2008 10:03 by KST [RSS]