14.2 CPU

Java provides a virtual machine runtime system that is just that: an abstraction of a CPU that runs in software. These virtual machines run on a real CPU, and in this section I discuss the performance characteristics of those real CPUs.

14.2.1 CPU Load

The CPU and many other parts of the system can be monitored using system-level utilities. On Windows, the task manager and performance monitor can be used for monitoring. On Unix, a performance monitor (such as perfmeter) is usually available, as well as utilities such as vmstat. Two aspects of the CPU are worth watching as primary performance points. These are the CPU utilization (usually expressed in percentage terms) and the runnable queue of processes and threads (often called the load or the task queue). The first indicator is simply the percentage of the CPU (or CPUs) being used by all the various threads. If this is up to 100% for significant periods of time, you may have a problem. On the other hand, if it isn't, the CPU is underutilized, but that is usually preferable. Low CPU usage can indicate that your application may be blocked for significant periods on disk or network I/O. High CPU usage can indicate thrashing (lack of RAM) or CPU contention (indicating that you need to tune the code and reduce the number of instructions being processed to reduce the impact on the CPU).

A reasonable target is 75% CPU utilization. This means that the system is being worked toward its optimum, but that you have left some slack for spikes due to other system or application requirements. However, note that if more than 50% of the CPU is used by system processes (i.e., administrative and operating-system processes), your CPU is probably underpowered. This can be identified by looking at the load of the system over some period when you are not running any applications.

The second performance indicator, the runnable queue, indicates the average number of processes or threads waiting to be scheduled for the CPU by the operating system. They are runnable processes, but the CPU has no time to run them and is keeping them waiting for some significant amount of time. As soon as the run queue goes above zero, the system may display contention for resources, but there is usually some value above zero that still gives acceptable performance for any particular system. You need to determine what that value is in order to use this statistic as a useful warning indicator. A simplistic way to do this is to create a short program that repeatedly does some simple activity. You can then time each run of that activity. You can run copies of this process one after the other so that more and more copies are simultaneously running. Keep increasing the number of copies being run until the run queue starts increasing. By watching the times recorded for the activity, you can graph that time against the run queue. This should give you some indication of when the runnable queue becomes too large for useful responses on your system, and you can then set system threshold monitors to watch for that level and alert the administrator if the threshold is exceeded. (One guideline from Adrian Cockcroft is that performance starts to degrade if the run queue grows bigger than four times the number of CPUs.)

If you can upgrade the CPU of the target environment, doubling the CPU speed is usually better than doubling the number of CPUs. And remember that parallelism in an application doesn't necessarily need multiple CPUs. If I/O is significant, the CPU will have plenty of time for many threads.

14.2.2 Process Priorities

The operating system also has the ability to prioritize the processes in terms of providing CPU time by allocating process priority levels. CPU priorities provide a way to throttle high-demand CPU processes, thus giving other processes a greater share of the CPU. If there are other processes that need to run on the same machine but it doesn't matter if they were run more slowly, you can give your application processes a (much) higher priority than those other processes, thus allowing your application the lion's share of CPU time on a congested system. This is worth keeping in mind. If your application consists of multiple processes, you should also consider the possibility of giving your various processes different levels of priority.

Being tempted to adjust the priority levels of processes, however, is often a sign that the CPU is underpowered for the tasks you have given it.