Mailing List Archive

very slow io with drbd and nfs
Hi
here is the structure

NFSCLIENT------------(NFSSERVER) drbd node1 active
|
(NFSSERVER ) drbd node2 passive

case 1. if i copy 63M file (which is on nfs server) from nfs client it tooks
approximately 55 seconds which is 1.2 MB in second very slow
(root@nfsclient:/# time cp /nfsmount/file_name /nfsmount/tmp_file)

case 2. disconnecting drbd connection with "drbdadm disconnect
resource_name" on node 2 after that

if i copy 63M file (which is on nfs server) from nfs client it tooks
approximately 5 seconds which is 12 MB in second which is normal and means nfs
client, server and network connection has no problem
(root@nfsclient:/# time cp /nfsmount/file_name /nfsmount/tmp_file2)


case 3. connecting drbd source again with drbd connect resource_name on node
2 and waiting for both nodes became uptodate
after that

if i copy file from drbd block device to drbd block device on nfs server,
completed with 25 MB / second which is normal and means drbd has no problem
(root@nfsserver:/# time cp /drbdmountpint/file_name /drbdmountpoint/tmp_file)

as you can see i am very confused and have no idea where is the problem

any suggestions

both nfs server node1 and node2 has centos 5.4 32 bit drbd version 8.0.16
which is in default yum repository

all networks are gbit. Nfs network and drbd NETWORKS ARE SEPARATED between
each other with separated gbit nics. there is no router or any other active
network device, they are connected with crossover cable
and last i used protocol B and C also.
-------------------------------------------
dbrbd configuration file is below

global { usage-count no; }
resource repdata {
protocol A;
startup {
wfc-timeout 0; degr-wfc-timeout 120;
}
disk {
on-io-error detach;
}
net {
}
syncer {
rate 80M;
}
on node1.kkoc.com.tr {
device /dev/drbd0;
disk /dev/sda1;
address 172.16.0.1:7788;
meta-disk internal;
}
on node2.kkoc.com.tr {
device /dev/drbd0;
disk /dev/md0;
address 172.16.0.2:7788;
meta-disk internal;
}
}

-------------------------------------------

thanks for your replies
***************************************************************************

Bu elektronik posta ve onunla iletilen bütün dosyalar sadece göndericisi
tarafýndan almasý amaçlanan yetkili gerçek ya da tüzel kiþinin kullanýmý
içindir. Eðer söz konusu yetkili alýcý deðilseniz bu elektronik postanýn
içeriðini açýklamanýz, kopyalamanýz, yönlendirmeniz ve kullanmanýz
kesinlikle yasaktýr ve bu elektronik postayý derhal silmeniz gerekmektedir.
KAMÝLKOÇ OTOBÜSLERÝ A.Þ.bu mesajýn içerdiði bilgilerin doðruluðu veya
eksiksiz olduðu konusunda herhangi bir garanti vermemektedir. Bu nedenle bu
bilgilerin ne þekilde olursa olsun içeriðinden, iletilmesinden, alýnmasýndan
ve saklanmasýndan sorumlu deðildir. Bu mesajdaki görüþler yalnýzca gönderen
kiþiye aittir ve KAMÝLKOÇ OTOBÜSLERÝ A.Þ.nin görüþlerini yansýtmayabilir.
Bu e-posta bilinen bütün bilgisayar virüslerine karþý taranmýþtýr.

***************************************************************************

This e-mail and any files transmitted with it are confidential and intended
solely for the use of the individual or entity to whom they are addressed.
If you are not the intended recipient you are hereby notified that any
dissemination, forwarding, copying or use of any of theinformation is
strictly prohibited, and the e-mail should immediately be deleted.KAMILKOC
OTOBUSLERI AS. makes no warranty as to the accuracy or completeness of any
information contained in this message and hereby excludes any liability of
any kind for the information contained therein or for the information
transmission, reception, storage or use of such in any way whatsoever. The
opinions expressed in this message belong to sender alone and may not
necessarily reflect the opinions of KAMILKOC OTOBUSLERI A.S.
This e-mail has been scanned for all known computer viruses.

***************************************************************************
Re: very slow io with drbd and nfs [ In reply to ]
From: Fatih <fatih@kamilkoc.com.tr>
> NFSCLIENT------------(NFSSERVER) drbd node1 active
> (NFSSERVER) drbd node2 passive
> If I copy a 63M file (which is on nfs server) from an nfs client, it takes
> approximately 55 seconds which is 1.2 MB per second, very slow.

> "drbdadm disconnect resource_name" on node 2. After that, if I copy a 63M
> file (which is on nfs server) from nfs client, it takes approximately 5
> seconds which is 12 MB per second which is normal.

> As you can see I am very confused and have no idea where the problem is.
> Both nfs server node1 and node2 have centos 5.4 32 bit drbd version 8.0.16
> which is in default yum repository.

> All networks are gbit. Nfs network and drbd NETWORKS ARE SEPARATED between

> each other with separated gbit nics. There is no router or any other active

> network device, they are connected with a crossover cable.

> resource repdata {
> protocol A;
> startup {
> wfc-timeout 0; degr-wfc-timeout 120;
> }
> disk {
> on-io-error detach;
> }

When you're using DRBD 8, if your disk controllers have battery backup, you
can get performance gains by putting the options

no-disk-flushes;
no-md-flushes;

...in your disk{} section. Then run "drbdadm adjust repdata" on both
nodes. If you disk controllers don't have battery backup, and you do this,
you could have problems if the power goes out. But a 1.2M/sec read speed
is pretty problematic in and of itself. Make sure you Read The Fine
Manual about what those options do before putting them in place.

What's the FS that the NFS server is using? Has it been mounted with
noatime? What mount options is the NFS client using? Those things may
be useful to know. HTH anyway,

--
Matt G / Dances With Crows
The Crow202 Blog: http://crow202.org/wordpress/
There is no Darkness in Eternity/But only Light too dim for us to see


_______________________________________________
drbd-user mailing list
drbd-user@lists.linbit.com
http://lists.linbit.com/mailman/listinfo/drbd-user