Mailing List Archive: Block IO Questions

Block IO Questions

May 1, 2012, 9:59 PM

Post #1 of 6 (1725 views)

Hey Xen Users,

I have a question about Block IO Performance in Xen, and was wondering if
anyone can help explain it to me.

I am running Windows 7 x64 HVM on an LVM Partition.

The main partition is on a SSD Drive, I also have another set of LVM
partitions on storage drives that I mount from time to time.

I found the IO speeds to be a bit... unusual.

For example, I tried to copy data off of a USB Drive to my main drive (SSD
LVM Partition) and the speed started at 300Mbps, then slowly over roughly a
minute dropped down to 15Mbps. Similarly I tried doing the same to a
storage LVM and it started at 176Mbps then dropped to the same 15~ Mbps
speed.

Given that it starts at a higher amount am I to assume this is a mistake in
the early calculations? Or is this a problem with disk configuration?

I should note that WEI rates the disk at a 6.8 in this HVM, so it
"shouldn't" be slow I would imagine.

If anyone has an insights I would greatly appreciate it, as a last resort I
may consider PV Drivers, I am open to everyones opinion about that.

Thanks,

~Casey

Re: Block IO Questions [ In reply to ]

simar at gmx

May 2, 2012, 12:40 PM

Post #2 of 6 (1648 views)

Permalink

Hi!
I have to say I've got similar strange behavior since I upgraded my home
grown storage server (Intel DQ35JO, disks attached to Q35, Core2 E6550,
40+ Mbps write) with brand new hardware (ASUS P8H77M-Pro, disks attached
to H77, Pentium G840, ok not that brand new). I didn't get much more
than your 10 Mbps.
System is a 3.2(.15) based Debian squeeze+backports for Dom0 and 3.2
based Debian testing+sid for DomU.

I improved the situation somwhat by setting xen-blkback's reqs parameter
to 512 (default is 64 but I don't fully understand what this actually
sets. There is a tempting comment in xen-blkback.c ;-) ).

I still don't get more then 40-50 Mbps in a linux PV vm (bonnie++, xfs)
the same Filesystem on md RAID5+lvm does 150 Mbps write 300 Mbps read
mounted in Dom0.

By the way I can't find any means by which a Dom0 can inform a vm of a
physical block size other than 512 (like 4k for example), but perhaps
that's irrelevant here.

Best regards
Omar Siam

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Re: Block IO Questions [ In reply to ]

G.Bakalarski at icm

May 2, 2012, 3:18 PM

Post #3 of 6 (1649 views)

Permalink

Hi

Did you try other (i.e. non-Debian kernel).

I'd suggest e.g. SLES 11 SP2 or even alpine linux ... And see if this
helps ...

We had problems with Block IO (unstable troughput) ) Xen'ed Debian
kernels (Squeeze or Wheezy, 2.6.32 or 3.2) ... Just krenels ...

GB
> I have to say I've got similar strange behavior since I upgraded my home
> grown storage server (Intel DQ35JO, disks attached to Q35, Core2 E6550,
> 40+ Mbps write) with brand new hardware (ASUS P8H77M-Pro, disks attached
> to H77, Pentium G840, ok not that brand new). I didn't get much more
> than your 10 Mbps.
> System is a 3.2(.15) based Debian squeeze+backports for Dom0 and 3.2
> based Debian testing+sid for DomU.
>
> I improved the situation somwhat by setting xen-blkback's reqs parameter
> to 512 (default is 64 but I don't fully understand what this actually
> sets. There is a tempting comment in xen-blkback.c ;-) ).
>
> I still don't get more then 40-50 Mbps in a linux PV vm (bonnie++, xfs)
> the same Filesystem on md RAID5+lvm does 150 Mbps write 300 Mbps read
> mounted in Dom0.
>
> By the way I can't find any means by which a Dom0 can inform a vm of a
> physical block size other than 512 (like 4k for example), but perhaps
> that's irrelevant here.
>
> Best regards
> Omar Siam
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Re: Block IO Questions [ In reply to ]

simar at gmx

May 4, 2012, 2:59 AM

Post #4 of 6 (1653 views)

Permalink

Thanks for the tip, perhaps I give it a try at the weekend.

I tried a vanilla 3.2.16+ from git with the debian kernel .config. That
didn't change anything.
By the way Xen is 4.1.2 debian sid recompiled for debian squeeze on
debian squeeze (there were just some minor python issues).

Am 03.05.2012 00:18, schrieb G.Bakalarski@icm.edu.pl:
> Hi
>
>
> Did you try other (i.e. non-Debian kernel).
>
> I'd suggest e.g. SLES 11 SP2 or even alpine linux ... And see if this
> helps ...
>
> We had problems with Block IO (unstable troughput) ) Xen'ed Debian
> kernels (Squeeze or Wheezy, 2.6.32 or 3.2) ... Just krenels ...
>
> GB

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Re: Block IO Questions [ In reply to ]

G.Bakalarski at icm

May 4, 2012, 3:10 AM

Post #5 of 6 (1643 views)

Permalink

Hi,
> Thanks for the tip, perhaps I give it a try at the weekend.
>
> I tried a vanilla 3.2.16+ from git with the debian kernel .config. That
> didn't change anything.

But the .config settings were the clue in our case.

When we recompiled debian's wheezy kernel source with .config from
SLES 11 SP2 or Alpine Linux (recent xen'ed kernel) the block
IO issue disappeared. We investigated further and it seems that
NUMA options OFF do the trick (Xen + NUMA problem???) but we did not
debug in details ...

Good luck

GB

> By the way Xen is 4.1.2 debian sid recompiled for debian squeeze on
> debian squeeze (there were just some minor python issues).
>
> Am 03.05.2012 00:18, schrieb G.Bakalarski@icm.edu.pl:
>> Hi
>>
>>
>> Did you try other (i.e. non-Debian kernel).
>>
>> I'd suggest e.g. SLES 11 SP2 or even alpine linux ... And see if this
>> helps ...
>>
>> We had problems with Block IO (unstable troughput) ) Xen'ed Debian
>> kernels (Squeeze or Wheezy, 2.6.32 or 3.2) ... Just krenels ...
>>
>> GB
>
>
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

Re: Block IO Questions [ In reply to ]

cdelorme at gmail

May 4, 2012, 7:27 AM

Post #6 of 6 (1659 views)

Permalink

Thanks for looking into that for us, I may have to give that a try too.

On Fri, May 4, 2012 at 6:10 AM, <G.Bakalarski@icm.edu.pl> wrote:

> Hi,
> > Thanks for the tip, perhaps I give it a try at the weekend.
> >
> > I tried a vanilla 3.2.16+ from git with the debian kernel .config. That
> > didn't change anything.
>
> But the .config settings were the clue in our case.
>
> When we recompiled debian's wheezy kernel source with .config from
> SLES 11 SP2 or Alpine Linux (recent xen'ed kernel) the block
> IO issue disappeared. We investigated further and it seems that
> NUMA options OFF do the trick (Xen + NUMA problem???) but we did not
> debug in details ...
>
> Good luck
>
> GB
>
> > By the way Xen is 4.1.2 debian sid recompiled for debian squeeze on
> > debian squeeze (there were just some minor python issues).
> >
> > Am 03.05.2012 00:18, schrieb G.Bakalarski@icm.edu.pl:
> >> Hi
> >>
> >>
> >> Did you try other (i.e. non-Debian kernel).
> >>
> >> I'd suggest e.g. SLES 11 SP2 or even alpine linux ... And see if this
> >> helps ...
> >>
> >> We had problems with Block IO (unstable troughput) ) Xen'ed Debian
> >> kernels (Squeeze or Wheezy, 2.6.32 or 3.2) ... Just krenels ...
> >>
> >> GB
> >
> >
> >
>
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>