[Gllug] what is causing disk activity ?
Nix
nix at esperi.org.uk
Sat Nov 22 15:52:54 UTC 2008
On 20 Nov 2008, Alain Williams outgrape:
> dm-4 61.40 0.00 491.20 0 2456
This is the interesting device.
Ensure that CONFIG_BLK_DEV_IO_TRACE is turned on in your kernel config,
compile in debugfs and mount it on /sys/kernel/debug, get a newish
blktrace from somewhere (they don't do official releases), look in
/dev/mapper to find the device with minor number 4, and try
btrace {that device}
e.g., here, in the middle of a massive run with branch prediction on,
spraying huge amounts of branch prediction stats to the disk, I see
^Cloki:/etc/shai-hulud/install/gcc/scripts# btrace /dev/mapper/raid-packages | head -50
253,4 0 1 0.000000000 339 C W 2740740 + 4 [0]
253,4 0 2 0.000083848 339 C W 2740748 + 4 [0]
253,4 0 3 0.000087340 339 C W 2740744 + 4 [0]
253,4 0 4 0.000093652 339 C W 2740756 + 4 [0]
253,4 0 5 0.000100018 339 C W 2740636 + 4 [0]
253,4 0 6 0.000164725 339 C W 2740764 + 4 [0]
253,4 0 7 0.000174556 339 C W 2740760 + 4 [0]
253,4 0 8 0.000195906 339 C W 5330652 + 4 [0]
253,4 0 9 0.000226264 339 C W 5330648 + 4 [0]
253,4 0 10 0.000290183 339 C W 5330660 + 4 [0]
253,4 0 11 0.000296921 339 C W 5330656 + 4 [0]
253,4 0 12 0.000310085 339 C W 5331340 + 4 [0]
253,4 0 13 0.000317870 339 C W 5331336 + 4 [0]
253,4 0 14 0.000515274 339 C W 5331972 + 4 [0]
253,4 0 15 0.000525973 339 C W 5331968 + 4 [0]
253,4 0 16 0.000549746 339 C W 5331980 + 4 [0]
253,4 0 17 0.000558702 339 C W 5331976 + 4 [0]
253,4 0 18 0.000654444 997 Q W 5324676 + 4 [kjournald]
253,4 0 19 0.000702772 997 Q W 5324680 + 4 [kjournald]
253,4 0 20 0.000719463 997 Q W 5324684 + 4 [kjournald]
253,4 0 21 0.000729997 997 Q W 27892636 + 4 [kjournald]
253,4 0 22 0.000739956 997 Q W 27892640 + 4 [kjournald]
253,4 0 23 0.000755496 997 Q W 5324692 + 4 [kjournald]
253,4 0 24 0.000770960 997 Q W 5324696 + 4 [kjournald]
253,4 0 25 0.000788813 997 Q W 5324700 + 4 [kjournald]
253,4 0 26 0.000796106 997 Q W 5324704 + 4 [kjournald]
253,4 0 27 0.000813098 997 Q W 5324716 + 4 [kjournald]
253,4 0 28 0.000827977 997 Q W 5324740 + 4 [kjournald]
253,4 0 29 0.000842602 997 Q W 5324744 + 4 [kjournald]
253,4 0 30 0.000855606 997 Q W 5324748 + 4 [kjournald]
253,4 0 31 0.000863674 997 Q W 5324752 + 4 [kjournald]
253,4 0 32 0.000883474 997 Q W 5324756 + 4 [kjournald]
253,4 0 33 0.000891380 997 Q W 5324760 + 4 [kjournald]
253,4 0 34 0.000911216 997 Q W 5324764 + 4 [kjournald]
253,4 0 35 0.000920004 997 Q W 5324768 + 4 [kjournald]
253,4 0 36 0.000936322 997 Q W 5324772 + 4 [kjournald]
253,4 0 37 0.000948411 997 Q W 27892644 + 4 [kjournald]
253,4 0 38 0.000959165 997 Q W 27892648 + 4 [kjournald]
253,4 0 39 0.000976039 997 Q W 27892652 + 4 [kjournald]
253,4 0 40 0.000987396 997 Q W 27892656 + 4 [kjournald]
253,4 0 41 0.001005590 997 Q W 27892660 + 4 [kjournald]
253,4 0 42 0.001017835 997 Q W 27892664 + 4 [kjournald]
253,4 0 43 0.001034828 997 Q W 27892668 + 4 [kjournald]
253,4 0 44 0.001046691 997 Q W 27892672 + 4 [kjournald]
253,4 0 45 0.001066429 997 Q W 27892676 + 4 [kjournald]
253,4 0 46 0.001078557 997 Q W 27892680 + 4 [kjournald]
253,4 0 47 0.001099321 997 Q W 27892684 + 4 [kjournald]
253,4 0 48 0.001108637 997 Q W 27892688 + 4 [kjournald]
253,4 0 49 0.001126514 997 Q W 27892692 + 4 [kjournald]
253,4 0 50 0.001138416 997 Q W 5325008 + 4 [kjournald]
[et seq ad nauseam]
which tells me that my journal is too small for hte write load: it's
spending all its time doing frantic journal commits to free up space,
and not much time actually doing process-initiated reads or writes.
(I'd expect this: the drive is constantly being slammed, mostly over
NFS. blktrace can't really track NFS-initiated writes well at all: how
could it tell what the process at the other end was? The right way to
keep an eye on *those* is wireshark.)
--
`Not even vi uses vi key bindings for its command line.' --- PdS
--
Gllug mailing list - Gllug at gllug.org.uk
http://lists.gllug.org.uk/mailman/listinfo/gllug
More information about the GLLUG
mailing list