The following demonstrates running the cputimes program on an idle system.
We use an interval of 1 second and a count of 3,
# ./cputimes 1 3
2005 Apr 27 23:37:58,
THREADS TIME (ns)
KERNEL 10795499
PROCESS 20941091
IDLE 970707443
2005 Apr 27 23:37:59,
THREADS TIME (ns)
KERNEL 8919418
PROCESS 77446789
IDLE 910555040
2005 Apr 27 23:38:00,
THREADS TIME (ns)
KERNEL 8615123
PROCESS 78314246
IDLE 810100417
In the above output, we can see a breakdown of CPU time into the catagories
KERNEL, PROCESS and IDLE. The time is measured in nanoseconds. Most of the
time is in the IDLE category, as the system is idle. Very little time
was spent serving the kernel.
In the following example, several programs are run to hog the CPUs,
# ./cputimes 1 3
2005 Apr 27 23:40:58,
THREADS TIME (ns)
KERNEL 11398807
PROCESS 992254664
2005 Apr 27 23:40:59,
THREADS TIME (ns)
KERNEL 9205260
PROCESS 987561182
2005 Apr 27 23:41:00,
THREADS TIME (ns)
KERNEL 9196669
PROCESS 877850474
Now there is no IDLE category, as the system is 100% utilised.
The programs were the following,
while :; do :; done &
which keeps the CPU busy.
In the following example a different style of program is run to hog the CPUs,
while :; do date; done
This causes many processes to be created and destroyed in a hurry, and can
be difficult to troubleshoot (tools like prstat cannot sample quick enough
to easily identify what is going on). The following is the cputimes output,
# ./cputimes 1 3
2005 Apr 27 23:45:30,
THREADS TIME (ns)
KERNEL 192647392
PROCESS 835397568
2005 Apr 27 23:45:31,
THREADS TIME (ns)
KERNEL 168773713
PROCESS 810825730
2005 Apr 27 23:45:32,
THREADS TIME (ns)
KERNEL 151676122
PROCESS 728477272
Now the kernel is doing a substantial amount of work to create and destroy
these processes.
In the following example, a large amount of network activity occurs while
cputimes is running,
# ./cputimes 1 6
2005 Apr 27 23:49:29,
THREADS TIME (ns)
KERNEL 10596399
PROCESS 21793920
IDLE 974395713
2005 Apr 27 23:49:30,
THREADS TIME (ns)
KERNEL 251465759
IDLE 357436576
PROCESS 508986422
2005 Apr 27 23:49:31,
THREADS TIME (ns)
IDLE 9758227
KERNEL 367645318
PROCESS 385427847
2005 Apr 27 23:49:32,
THREADS TIME (ns)
IDLE 28351679
KERNEL 436022725
PROCESS 451304688
2005 Apr 27 23:49:33,
THREADS TIME (ns)
KERNEL 262586158
PROCESS 325238896
IDLE 358243503
2005 Apr 27 23:49:34,
THREADS TIME (ns)
KERNEL 10075578
PROCESS 238170506
IDLE 647956998
Initially the system is idle. A command is run to cause heavy network
activity, which peaks during the fourth sample - during which the kernel
is using around 40% of the CPU. The Solaris 10 command "intrstat" can
help to analyse this activity further.
Longer samples are possible. The following is a 60 second sample,
# ./cputimes 60 1
2005 Apr 27 23:53:02,
THREADS TIME (ns)
KERNEL 689808449
PROCESS 8529562214
IDLE 50406951876
#
cputimes has a "-a" option to print all processes. The following is a
single 1 second sample with -a,
# ./cputimes -a 1 1
2005 Apr 28 00:00:32,
THREADS TIME (ns)
svc.startd 51042
nautilus 130645
in.routed 131823
fmd 152822
nscd 307042
dsdm 415799
mixer_applet2 551066
gnome-smproxy 587234
xscreensaver 672270
fsflush 1060196
java_vm 1552988
wnck-applet 2060870
dtrace 2398658
acroread 2614687
soffice.bin 2825117
mozilla-bin 5497488
KERNEL 13541120
metacity 28924204
gnome-terminal 74304348
Xorg 289631407
IDLE 465054209
The times are in nanoseconds, and multiple processes with the same name
have their times aggregated. The above output is at an amazing resolution -
svc.startd ran for 51 microseconds, and soffice.bin ran for 28 milliseconds.
The following is a 10 second sample on an idle desktop,
# ./cputimes -a 10 1
2005 Apr 28 00:03:57,
THREADS TIME (ns)
snmpd 127859
fmd 171897
inetd 177134
svc.configd 185006
mapping-daemon 197674
miniserv.pl 305603
gconfd-2 330511
xscreensaver 443207
sendmail 473434
nautilus 506799
gnome-vfs-daemon 549037
gnome-panel 770631
nscd 885353
svc.startd 1181286
gnome-netstatus- 4329671
mixer_applet2 4833519
dtrace 6244366
in.routed 6556075
fsflush 9553155
soffice.bin 13954327
java_vm 16285243
acroread 32126193
gnome-terminal 34891991
Xorg 35553412
mozilla-bin 67855629
KERNEL 94834997
IDLE 9540941846
Wow, maybe not as idle as I thought!