cause of IO wait

[Bryan O'Neal]

I too would like some answers on how to track down the source of Io
wait but I can ask some other questions. Did you check the raid
controllers health? BBU in good shape? Still have all your cache? did
you end up in write through? Did you tweak things before and lose your
tweaks becuse they were not in the appropriate confs? by tweaks I mean
things like you fs levelers or disabling atime etc.

On 6/27/11, der.hans <PLUGd at lufthans.com> wrote:
> moin moin,
>
> I've got a machine experiencing a lot of IO wait.
>
> We had power at a datacenter go down last week. Since then IO wait has
> been over 35%. At first we thought it was due to 3ware RAID verify taking
> place due to the crash. That took a few days, then the weekly verify
> started. We stopped that and IO wait stayed high. 8 disks in a RAID 10.
>
> Load avg is also very high, presumably due to the IO wait.
>
> smartctl short tests didn't turn up any issues.
>
> We're not swapping at all.
>
> Disk read and write are fairly low.
>
> Network traffic is down as is the total number of process and the number
> of running processes. No evidence of network errors on the box or at the
> switch.
>
> Not much going on in the logs. We've stopped several reporting processes
> in order to reduce disk access.
>
> On the positive side, entropy has been staying high :).
>
> IO wait is not explicitly disk? It could be network, serial, USB, etc.?
>
> How do I determine what resource is causing the IO wait? Is there a way to
> track to a specific process?
>
> vmstat, iostat, top and lots of other tools have been great at showing
> that there's overall IO wait ( I've been able to show that almost all
> processors have high wait, one was only at 5% ), but I haven't yet
> determined what and how.
>
> The server is running CentOS in case that matters.
>
> ciao,
>
> der.hans
> --
> #  http://www.LuftHans.com/        http://www.LuftHans.com/Classes/
> #  Hope has two beautiful daughters: Anger and Courage. Anger at the way
> #  things are, and Courage to struggle to create things as they should be.
> #  -- St. Augustine
> ---------------------------------------------------
> PLUG-discuss mailing list - PLUG-discuss at lists.plug.phoenix.az.us
> To subscribe, unsubscribe, or to change your mail settings:
> http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
>

-- 
Sent from my mobile device